Search
Search LeCompute
GPUs, memory, runtimes and costs: find the analysis you need.
47 articles
Results with a spelling close to your search.
- Silicon
PsiQuantum turns to light to reconnect GPUs
PsiQuantum has unveiled an optical circuit switch derived from its quantum platform. Its value will depend on scaling the hardware and choosing useful connections in software.
- Costs
Google runs AI in orbit. Can it run a data center there?
A Google satellite has returned a Gemma 4 response. Beyond the haiku, Suncatcher tests the power, cooling and communications needed to keep computing in space.
- Costs
GPU cloud in France and Europe: choosing an H100, H200 or B300 instance
Compare exact instance sizes, GPU counts, regions and operating costs before renting a European GPU server from Scaleway, OVHcloud or another provider.
- Costs
LLM inference cost: when can a GPU beat an API?
Calculate API and self-hosted inference costs with a fixed quality target, input/output mix, full operating cost and a server capacity check.
- Runtimes
LLM KV cache: calculate memory before increasing context or concurrency
A worked KV cache calculation in GiB, with the limits of PagedAttention, FP8 quantization and prefix caching when sizing an inference server.
- Runtimes
vLLM vs llama.cpp vs TensorRT-LLM: choose a runtime and test it fairly
Compare model support, memory, continuous batching and deployment paths before measuring useful throughput and latency on your own workload.
- Runtimes
DeepSeek opens DeepInfer: model weights are only the beginning
DeepSeek has released its inference engine. The code shows how it speeds up models, and how much infrastructure still separates that engine from a production service.
- Kernel & Perf
NVIDIA Minerva: finding what slows down an AI cluster
An AI job can slow down without a GPU failing outright. Minerva connects infrastructure and workload signals to help operators investigate the dependencies behind that slowdown.
- Edge AI
LFM3 vs Qwen: what matters on phones and embedded devices
Compare LFM3 with small Qwen models on phones: iPhone measurements, real memory use, task quality and integration into iOS, Android and embedded applications.
- Kernel & Perf
Flapping Airplanes: ThunderKittens, NVIDIA and the Megakernel Bet
A GPU can be busy without using all its resources. Follow an inference request to understand ThunderKittens, megakernels and the research behind Flapping Airplanes.
- Kernel & Perf
Scion: why Anthropic replaced Git for its agents
An agent finishes a patch, then waits for test machines to fetch the code. Scion provides a starting point for understanding that hidden cost and the infrastructure choices behind it.
- Costs
Alibaba wants to supply the chip, model and cloud. What choices remain?
Alibaba is bringing chips, models and agent services together. Open weights preserve some choice; adaptations and application state determine what can move elsewhere.
- Kernel & Perf
FlashAttention-4 explains why faster matrix multiplication is not enough
FlashAttention-4 keeps evolving on Blackwell. Between matrix products, softmax and data movement can determine how fast the kernel progresses.
- Silicon
AMD challenges NVIDIA on CPUs as AI agents change the workload
AMD challenges NVIDIA Vera with EPYC Venice. Agent tool execution puts CPUs in focus, but per-core, server and rack estimates measure different things.
- Costs
Google and NVIDIA want more data centres connected by promising flexible demand
Google, NVIDIA and Emerald AI want power flexibility to unlock faster grid connections. Who guarantees the promised reductions?
- Costs
NVIDIA wants to buy Hugging Face: who will control open AI distribution?
NVIDIA promises to keep Hugging Face open. Who will shape the easiest paths to choosing and deploying a model?
- Silicon
Huawei challenges NVIDIA with the whole machine
Huawei has announced Atlas 960E with up to 4,096 accelerators. Can competition move from the chip to the whole system?
- Runtimes
Jev: why generate text when your application needs a decision?
Pick a category, route a request, score a document. Jev returns answers your code can use directly. Here is what that specialization changes and how to evaluate it.
- Silicon
800 VDC: why AI data centers are rewiring their power
A single GPU rack already draws more than 2,000 amps. At 1 MW, today's 54 V distribution would need close to 20,000: here is why NVIDIA and the hyperscalers are moving to 800 V DC, and what remains unsolved.
- Runtimes
Speculative decoding: when a small model makes your LLM faster
Your model fits in memory, but the answer arrives slowly. A smaller model can draft the next tokens. Here is when that extra work pays off.
- Silicon
Threadripper Halo Station vs DGX Station: what the memory specs mean
Your model has outgrown one GPU. AMD and NVIDIA offer different ways to make room, but total memory alone will not tell you which system fits your work.
- Runtimes
NVIDIA PAIR: when more machines make local AI faster
Your GPU is busy while another computer sits idle. NVIDIA PAIR can send it requests, but whether you finish sooner depends on how the work is shared.
- Silicon
Ryzen AI Halo vs DGX Spark: What 128GB Doesn't Tell You
Ryzen AI Halo and DGX Spark both offer 128 GB, but performance changes with model and protocol. Vendor claims and independent LLM tests, audited.
- Silicon
BlueField-4 vs Pensando Salina: the DPU enters the inference data path
BlueField-4 and Pensando Salina compute no tokens. They move networking, storage, security and selected KV-cache transfers away from the host CPU. Here is the data path that can free the GPU, or merely move the bottleneck.
- Costs
MLPerf Endpoints: can inference providers finally be compared?
MLPerf Endpoints measures an inference service through its API at several load levels. Version 0.7 finally traces the curve between throughput, TTFT, TPOT and quality, but it does not normalize cost yet.
- Runtimes
MLPerf RAG finally measures the full pipeline, but not yet the service
MLPerf E2E-RAG times ingestion, retrieval, reranking and four LLM roles inside a multi-hop loop. Its throughput is comparable, but the Offline scenario and 35% reference accuracy bound the verdict.
- Silicon
UALink 2.0 vs NVLink 6: when the network computes all-reduce
UALink 2.0 standardizes compute inside the network; NVLink 6 integrates it into its switches. Here is what the mechanism changes, and why Helios and Vera Rubin still cannot be ranked.
- Runtimes
DWDP for MoE inference: move weights, not tokens
Expert parallelism moves tokens to weights. DWDP reverses the path, prefetching remote expert weights and removing cross-rank synchronization. The bet works inside a narrow regime.
- Silicon
M3 Ultra Mac Studio 96 GB for local LLMs: what still fits
Assess a 96 GB M3 Ultra for local LLMs: 819 GB/s bandwidth, model weights, context and MLX. Includes the distinction from the current M5 Ultra catalog.
- Silicon
RTX PRO 6000 or three RTX 5090s: 96 GB is not one memory pool
An RTX PRO 6000 and three RTX 5090s each add up to 96 GB of GDDR7. For an LLM, that is where the equality ends: one pool, one sharded model and three independent replicas pay different costs.
- Runtimes
vLLM 0.27 turns the KV cache into peer-to-peer memory
vLLM 0.27 lets every engine serve CPU-resident KV blocks to another peer over NIXL, filters tiers per request and emits routable cache events. The mechanism is shipped; the production speedup is not proven.
- Silicon
RTX Spark on Windows: CUDA is native, the LLM stack is not yet
RTX Spark brings up to 128 GB of unified memory and native CUDA to Windows on Arm. As of August 12, CUDA remains a developer preview, TensorRT-RTX is ready, and the LLM serving stack still needs qualification.
- Runtimes
Kimi K3 opens 2.8T parameters, not a local model
Kimi K3 activates 104 billion parameters out of 2.8 trillion, mixes KDA with MLA across a one-million-token context, and needs an eight-B300 or eight-MI355X node in vLLM's launch recipes.
- Runtimes
Latency against latency: what your scheduler actually decides
An inference engine's scheduler does not trade throughput against latency. It trades one latency against another: time to first token versus time between the tokens that follow. Three major engines, three opposite defaults, and nothing tells the operator.
- Runtimes
MoE serving: at what batch size does the network take over?
A mixture-of-experts does not remove the cost, it moves it twice: from compute to memory, then from memory to the network. The second shift has a threshold, it can be computed, and it decides your topology.
- Costs
What stops a driving model from crossing a border
A driving model is a function of its data distribution. But the cost of moving it is not where intuition puts it: perception recalibrates for almost nothing, behaviour does not, and validation dominates everything. The entry price for a market is a simulation bill.
- Silicon
AMD MI455X vs NVIDIA Rubin: can more memory beat NVLink?
The MI455X carries 432 GB of HBM4, versus 288 GB on Rubin. AMD still has to prove that UALoE can turn that capacity advantage into rack-scale performance.
- Runtimes
GGUF vs GPTQ vs AWQ: a format, two algorithms, one problem
GGUF is a file format, while GPTQ and AWQ are quantization algorithms. At 4 bits, each attacks the outlier problem differently. Anatomy of three constructions, from GPTQ's Hessian to GGUF's k-quants, and of what the choice of format decides for you.
- Costs
Stargate, Spud and subscriptions: the compute lever behind OpenAI's comeback
10 gigawatts 'secured years ahead of schedule', 0.3 gigawatt plugged in at Abilene. Behind OpenAI's comeback in AI coding, a compute bet that must be read in three states: the secured, the built, the plugged-in. And a subscription economics that agents cracked open.
- Runtimes
Nobody knows how to measure coding agents anymore
On July 8, 2026, OpenAI disavowed SWE-Bench Pro: ~30% broken tasks. It is the third reference instrument declared dead in three generations. Anatomy of a metrology crisis: noise ceiling, contamination, and a harness effect worth a whole model generation.
- Costs
GPT-5.6 and the Codex merger: anatomy of OpenAI's comeback
On July 8, 2026, OpenAI disavowed the reference benchmark for agentic coding. On the 9th, it merged Codex into the ChatGPT app and launched GPT-5.6. Behind the sequence, a comeback that turns less on intelligence than on prices, subscriptions and the harness.
- Silicon
DGX Spark vs RTX 5090: capacity or speed, the choice that decides your local LLM
The DGX Spark and the RTX 5090 answer two opposite questions: hosting a very large model, or serving a mid-size one fast. The duel in numbers, measurements and mechanisms included.
- Runtimes
Prefill/decode disaggregation reaches production: state of play, May 2026
NVIDIA Dynamo 1.0, SGLang 0.5.12, TensorRT-LLM 1.3, Tenstorrent Blackhole: prefill/decode disaggregation reaches production. Implementations, benchmarks, and counter-arguments.
- Runtimes
The KV cache is no longer a side effect: it is the center of LLM serving in 2026
The KV cache is no longer a VRAM byproduct you just absorb: it has its own quantization (TurboQuant, ~3 bits), its transfer protocol, its tiered storage (KVBM G1→G4) and its own routing. Anatomy of that shift.
- Silicon
How much VRAM do you need to run an LLM locally?
How much VRAM does a local LLM need? The ~2 GB per billion parameters rule, the weight of the KV cache, what quantization changes, and what actually fits on your card.
- Silicon
RTX Spark vs DGX Spark: confirmed specs and the memory question
RTX Spark PCs are shipping: compare official laptop and desktop configurations with DGX Spark, while keeping bandwidth assumptions and measured results separate.
- Silicon
TorchTPU, XLA, JAX: how Google attacks NVIDIA's software lock-in
TorchTPU, PyTorch/XLA, JAX, XLA and vLLM form a stack whose keystone is a compiler. How Google attacks the one asset that keeps developers on NVIDIA: the cost of leaving CUDA.