Silicon
Ryzen AI Halo vs DGX Spark: What 128GB Doesn't Tell You
Ryzen AI Halo and DGX Spark both offer 128 GB, but performance changes with model and protocol. Vendor claims and independent LLM tests, audited.
Deep dives
All deep dives
Every LeCompute article available in English: silicon, inference runtimes, edge AI, costs and system observability.
37 articles in English , page 2 of 4
All articles (page 2)
Silicon
BlueField-4 vs Pensando Salina: the DPU enters the inference data path
BlueField-4 and Pensando Salina compute no tokens. They move networking, storage, security and selected KV-cache transfers away from the host CPU. Here is the data path that can free the GPU, or merely move the bottleneck.
Costs
MLPerf Endpoints: can inference providers finally be compared?
MLPerf Endpoints measures an inference service through its API at several load levels. Version 0.7 finally traces the curve between throughput, TTFT, TPOT and quality, but it does not normalize cost yet.
Runtimes
MLPerf RAG finally measures the full pipeline, but not yet the service
MLPerf E2E-RAG times ingestion, retrieval, reranking and four LLM roles inside a multi-hop loop. Its throughput is comparable, but the Offline scenario and 35% reference accuracy bound the verdict.
Silicon
UALink 2.0 vs NVLink 6: when the network computes all-reduce
UALink 2.0 standardizes compute inside the network; NVLink 6 integrates it into its switches. Here is what the mechanism changes, and why Helios and Vera Rubin still cannot be ranked.
Runtimes
DWDP for MoE inference: move weights, not tokens
Expert parallelism moves tokens to weights. DWDP reverses the path, prefetching remote expert weights and removing cross-rank synchronization. The bet works inside a narrow regime.
Silicon
The M3 Ultra Mac Studio is down to 96 GB: what local LLM users lose
Apple launched the M3 Ultra Mac Studio with 256 GB and 512 GB options. Only 96 GB remains. The chip did not get slower, but an entire class of local models has left the store.
Silicon
RTX PRO 6000 or three RTX 5090s: 96 GB is not one memory pool
An RTX PRO 6000 and three RTX 5090s each add up to 96 GB of GDDR7. For an LLM, that is where the equality ends: one pool, one sharded model and three independent replicas pay different costs.
Runtimes
vLLM 0.27 turns the KV cache into peer-to-peer memory
vLLM 0.27 lets every engine serve CPU-resident KV blocks to another peer over NIXL, filters tiers per request and emits routable cache events. The mechanism is shipped; the production speedup is not proven.
Silicon
RTX Spark on Windows: CUDA is native, the LLM stack is not yet
RTX Spark brings up to 128 GB of unified memory and native CUDA to Windows on Arm. As of August 12, CUDA remains a developer preview, TensorRT-RTX is ready, and the LLM serving stack still needs qualification.
Runtimes
Kimi K3 opens 2.8T parameters, not a local model
Kimi K3 activates 104 billion parameters out of 2.8 trillion, mixes KDA with MLA across a one-million-token context, and needs an eight-B300 or eight-MI355X node in vLLM's launch recipes.
Runtimes
Latency against latency: what your scheduler actually decides
An inference engine's scheduler does not trade throughput against latency. It trades one latency against another: time to first token versus time between the tokens that follow. Three major engines, three opposite defaults, and nothing tells the operator.