One parameter count cannot describe both compute and memory

A dense model pays for every parameter twice. The weights occupy memory, then every token passes through all of them. A MoE Mixture-of-Experts. An architecture where the network is split into many experts, of which a router activates only a small subset per token. Per-token compute follows the number of active parameters; memory follows the total count, since every expert must stay resident in VRAM, ready to be called. breaks that symmetry. Kimi K3 contains 2.8 trillion parameters, yet its router selects only 16 of 896 routed experts for a token, alongside two shared experts. Moonshot reports 104 billion active parameters on that path.

The active count estimates arithmetic, not deployable size. The next token may select another expert set. Serving cannot wait for SSD reads every time routing changes, so the complete expert pool must remain resident across accelerators or in a tier fast enough to avoid stalling every layer.

PropertyPublished valueSystems consequence
Total parameters2.8 trillionStorage capacity and expert placement
Active parameters104 billion per tokenCompute scale, not memory footprint
Experts896 routed, top 16, plus 2 sharedAll-to-all traffic between ranks
Layers93: 69 KDA + 24 Gated MLATwo state types for the runtime
Context1,048,576 tokensMaximum capacity, not constant throughput
Weights / activationsMXFP4 / MXFP8 after QATLow-precision kernels are mandatory
Official checkpoint96 shards, 1,560,936,091,448 bytes1.56 TB before caches and workspace
Table 1: Moonshot's published architecture. Checkpoint size is our sum of the 96 official safetensors shards on August 12, 2026.

Pure four-bit storage for 2.8 trillion scalar weights would require 1.40 TB before scales, non-quantized tensors and layout overhead. The actual safetensors shards total 1.56 TB. That number excludes tokenizer and documentation files, yet it still precedes the KV cache The stored key and value vectors an LLM has already computed for every token it has processed. It avoids recomputing attention over the whole history, at the cost of a memory footprint that grows with the context length. , activations, CUDA graphs, communication buffers and DSpark draft model.

This is why the phrase “104B active” is easy to misuse. Kimi K3 has the per-token compute shape of a very large sparse model and the capacity requirements of a multi-trillion-parameter model. It is not a 104B checkpoint with optional experts.

KDA compresses history; MLA keeps exact reach

Full attention compares each new token with prior tokens. Its stored state grows with sequence length, and a one-million-token window becomes expensive when repeated across every layer. Kimi K3 assigns three quarters of its attention layers to Kimi Delta Attention instead.

KDA processes chunks in parallel, then carries a recurrent matrix into the next chunk. During decode The autoregressive generation phase of an LLM: one token is produced at a time while rereading weights and the relevant context state. At low batch sizes, that traffic can make decode memory-bandwidth-bound. Its share of total cost still depends on input and output lengths, batching and cache reuse. , that state does not append one full key and value entry for every past token. Moonshot also lower-bounds the recurrence decay, keeping its numerical range finite enough to turn diagonal tiles into dense matrix operations.

Recurrent state is compact because it does not retain direct access to every earlier token. Kimi K3 therefore inserts one Gated MLA layer after every three KDA layers and ends the backbone with another global MLA layer. These 24 layers cache a compressed latent representation of keys and values. The 69 KDA layers control growth; the MLA layers restore global retrieval capacity. Long context is not free, but fewer layers carry a sequence-length-dependent cache.

Token incoming state sequence
KDA ×3 compact recurrence fixed state
Gated MLA global latent attention growing cache
LatentMoE 16 routed experts sparse compute
AttnRes cross-depth mixture residual
Figure 1: the 3:1 pattern repeats through the backbone, followed by a final global Gated MLA layer.

Attention Residuals addresses depth rather than sequence length. A conventional residual stream accumulates transformations in order. AttnRes learns weighted access to representations from several previous blocks. Moonshot attributes an approximately 2.5x scaling-efficiency improvement over Kimi K2 to KDA, AttnRes, Stable LatentMoE and the revised training recipe together.

That 2.5x number is not inference throughput, memory savings or agent score. It comes from the team’s scaling-law analysis and bundles several changes. The paper does not expose an ablation that assigns the ratio to KDA alone.

Native MXFP4 starts before the checkpoint is written

Kimi K3 ships with MXFP4 Microscaling FP4. A block format in which 32 FP4 values share one scale factor. That shared scale is what makes 4-bit storage usable without collapsing accuracy. NVIDIA's NVFP4 variant uses finer 16-value blocks. weights and MXFP8 activations. Moonshot applies QAT Quantization-Aware Training. Rather than compressing an already-trained model (and taking the accuracy hit), you briefly retrain it while simulating quantization, so it learns to live in low precision (often int4). It sharply reduces the degradation compared with naive post-training quantization. from supervised fine-tuning onward. The model adapts to four-bit weight constraints during post-training instead of taking one final rounding pass after its behavior has been learned.

This design brings the official files down to 1.56 TB. A BF16 representation of 2.8 trillion weights would approach 5.6 TB before overhead. The smaller format does not run efficiently on arbitrary hardware just because four-bit values fit in memory. MXFP4 groups values behind shared scales, and the runtime needs kernels that consume that representation without expanding entire experts into a high-precision copy.

vLLM 0.27 and SGLang 0.5.17 landed the relevant model, kernel and compressed-checkpoint paths. SGLang says its launch support is verified on GB300 and AMD MI35x. vLLM supports Hopper, Blackwell and MI355X, but its Kimi guide currently relies on Docker images because several dependencies, including FlashInfer, are pre-release. Weight availability and serving maturity are separate milestones.

Eight B300 GPUs are the entry point

vLLM’s quick start presents eight B300 or eight MI355X GPUs as the easiest paths. Its FAQ sets the NVIDIA minimum at one eight-B300 or GB300 NVL72 node and also supports sixteen B200 GPUs. Production deployments commonly add expert and data parallelism across nodes connected through NVLink or RDMA.

The expert router creates communication that a dense parameter count does not show. Tokens must travel to the ranks holding their selected experts, then return for the rest of the layer. Sparse routing avoids evaluating all 896 experts, but it exchanges some compute for all-to-all traffic. Placement, load balance and interconnect behavior become part of model performance.

Hybrid attention also changes prefix reuse. An MLA layer restores KV blocks for a shared prefix. A KDA layer must restore the exact recurrent state at that prefix boundary. vLLM implements both in its hybrid prefix cache, yet disables prefix caching by default for Kimi K3 while the design evolves. Operators have to pass --enable-prefix-caching explicitly.

The 370 token/s claim is not a controlled 3.14x comparison

vLLM reports 111 token/s in TP8 and 118 token/s in TP16 on GB300 NVL72 GPUs at batch size one. DSpark speculative decoding raises the published figures to 331 and 370 token/s. The draft model proposes multiple tokens from Kimi K3 intermediate states; Kimi K3 verifies them in parallel.

The hardware and concurrency are clear. The public commands reveal a workload mismatch. The non-speculative run uses random prompts around 8,192 input and 1,024 output tokens. The DSpark run uses the low_entropy category of SPEED Bench, up to 10,240 input and 1,536 output tokens. Low-entropy text is exactly where speculative drafts tend to achieve longer acceptance runs.

The 370 token/s result is a documented target on sixteen GB300 GPUs. Pairing it with 118 token/s to claim a universal 3.14x does not isolate DSpark because prompt distribution and lengths also change. A valid ablation would replay identical tokenized inputs, sampling parameters and output limits with the draft model enabled and disabled.

The same launch post mentions more than 2,000 tokens per GPU second at the throughput end of its Pareto frontier. That fleet-efficiency unit is not comparable with 370 tokens per second for one user. Without the exact concurrency and latency target for the plotted point, it should not become a cost estimate.

Agent benchmarks compare systems, not bare weights

Moonshot reports 67.5 on DeepSWE, 88.3 on Terminal-Bench 2.1, 42.0 on SWE-Marathon and 91.2 on BrowseComp. Its paper says Kimi K3 trails Claude Fable 5 and GPT-5.6 Sol overall while exceeding the other models included in its suite.

The footnotes prevent a clean bare-model ranking. Kimi K3 uses the Kimi Code harness Everything wrapped around a model to turn it into an agent: the exposed tool set and their schemas, the system prompt, project-context injections, the execution loop with permissions and sandbox, the subagents, hooks and MCP servers. At identical model, switching harness moves scores by tens of points on agentic evaluations. on DeepSWE and Terminal-Bench. GPT-5.6 Sol uses Codex; Claude entries can use Terminus 2 or Claude Code. SWE-Marathon ran on an H20-calibrated branch from before the final v1.1 tasks, and Claude Fable 5 hit fallbacks on 35% of those tasks. PostTrainBench mixes H20 with the benchmark’s H100 setting and uses different agent harnesses.

BrowseComp adds context management. Kimi K3 scores 91.2 with compaction triggered at 300,000 tokens and 90.4 with the full 1M window and no compaction. The better number belongs to the model plus a memory strategy.

The evidence supports a narrower verdict. Kimi K3 is competitive with frontier systems across several published evaluations. It does not prove superiority under identical model budget, harness, hardware and evaluation procedure. Internal testing should hold the harness constant and record task completion, tool calls, consumed tokens, latency and infrastructure cost.

Open weights, with two commercial thresholds

The Kimi K3 License permits copying, modification, distribution, deployment and derivative works. It also adds conditions that a standard permissive software license does not carry. A model-as-a-service business whose corporate group exceeds $20 million in aggregate revenue over any consecutive twelve months must reach a separate agreement with Moonshot before commercial use. A product above 100 million monthly active users or $20 million in monthly revenue must display “Kimi K3” prominently.

The clauses exclude internal use as defined by the license and access through Moonshot products or certified inference partners. They barely affect a research lab inspecting the checkpoint, but they matter to a platform turning the model into a commercial API. “Open-weight” describes the release without implying unconditional OSI-style rights.

Openness moves scarcity into the serving stack

Kimi K3 makes a previously API-bound scale of model inspectable. Researchers can examine routing across 896 experts, hybrid state built for million-token sessions and quantization integrated into post-training. The release is substantial precisely because it exposes the awkward systems problems along with the weights.

Scarcity now sits in 1.56 TB of fast-access parameters, all-to-all communication, fresh MXFP4 kernels, correct hybrid caching and evaluation that separates a model from its agent wrapper. The useful next milestone is not a larger total parameter count. It is a deployment whose cost, latency and reproducibility outsiders can verify.

Sources and method