Featured
MLPerf E2E-RAG times ingestion, retrieval, reranking and four LLM roles inside a multi-hop loop. Its throughput is comparable, but the Offline scenario and 35% reference accuracy bound the verdict.
LeCompute is a French publication that dissects AI infrastructure: GPUs and silicon, HBM memory and interconnects, inference runtimes like vLLM and llama.cpp, quantization and edge AI. Technical analysis and reproducible benchmarks, from the silicon to the code, no hype.
Lire nos articles en françaisSilicon
BlueField-4 and Pensando Salina compute no tokens. They move networking, storage, security and selected KV-cache transfers away from the host CPU. Here is the data path that can free the GPU, or merely move the bottleneck.
Costs
MLPerf Endpoints measures an inference service through its API at several load levels. Version 0.7 finally traces the curve between throughput, TTFT, TPOT and quality, but it does not normalize cost yet.
Silicon
UALink 2.0 standardizes compute inside the network; NVLink 6 integrates it into its switches. Here is what the mechanism changes, and why Helios and Vera Rubin still cannot be ranked.
Runtimes
Expert parallelism moves tokens to weights. DWDP reverses the path, prefetching remote expert weights and removing cross-rank synchronization. The bet works inside a narrow regime.
Silicon
Apple launched the M3 Ultra Mac Studio with 256 GB and 512 GB options. Only 96 GB remains. The chip did not get slower, but an entire class of local models has left the store.
Silicon
An RTX PRO 6000 and three RTX 5090s each add up to 96 GB of GDDR7. For an LLM, that is where the equality ends: one pool, one sharded model and three independent replicas pay different costs.
