Silicon
How much VRAM do you need to run an LLM locally?
How much VRAM does a local LLM need? The ~2 GB per billion parameters rule, the weight of the KV cache, what quantization changes, and what actually fits on your card.
GPUs & Silicon
Silicon: GPUs, memory and interconnects
GPUs, HBM memory, NVLink interconnects and microarchitecture: what really sets the performance and cost of AI compute, from the die to the datacenter.
15 articles , page 2 of 2
Articles: Silicon (page 2)
Silicon
RTX Spark vs DGX Spark: confirmed specs and the memory question
Windows on Arm or Linux, up to 128 GB of memory: what NVIDIA confirms, what DGX Spark measurements tell us, and what remains unknown about RTX Spark.
Silicon
TorchTPU, XLA, JAX: how Google attacks NVIDIA's software lock-in
TorchTPU, PyTorch/XLA, JAX, XLA and vLLM form a stack whose keystone is a compiler. How Google attacks the one asset that keeps developers on NVIDIA: the cost of leaving CUDA.