MoE inference: when all-to-all overtakes weight loading

A mixture-of-experts does not remove the cost, it moves it twice: from compute to memory, then from memory to the network. The second shift has a threshold, it can be computed, and it decides your topology.

The number that carries the argument

A is sold on a single number, and that number is a ratio. DeepSeek-V4-Pro carries 1.6 trillion parameters and activates only 49 billion per token. GLM-5.2 holds 744 billion and wakes about forty. Qwen3.5-397B-A17B pushes further still: 512 experts per layer, ten selected per token. The commercial argument fits on one line, and it is seductive. You pay for the compute of 49 billion parameters and you get the capacity of 1.6 trillion.

Model	Total	Active	Routed experts	Top-k	Size of one expert
DeepSeek-V4-Pro	1.6 T	49 B	384	6	66.1 M
Kimi-K2.6	~1 T	32 B	384	8	44.0 M
GLM-5.2	744 B	~40 B	256	8	37.7 M
Qwen3.5-397B-A17B	397 B	17 B	512	10	12.6 M
DeepSeek-V3.2	671 B	37 B	256	8	44.0 M

Values read directly from the config.json files on July 28, 2026. Expert size is computed from those files (3 x hidden size x intermediate size, SwiGLU), not taken from a vendor announcement.

That ratio has climbed steadily. Mixtral-8x7B opened the way in late 2023 with a factor of 3.6 between total and active. DeepSeek-V3 took it to 18 a year later. DeepSeek-V4-Pro reaches 32.7 today, with Kimi K2.6 close behind at 31.2. Put differently: in thirty months, the share of the model that works on any given token has fallen from more than a quarter to three percent.

It is worth saying where this architecture comes from, because it heads off a common misreading. The MoE was not invented to make serving cheaper. It was invented for training, and the scaling laws for routed models say so without ambiguity: at a fixed training compute budget, a routed model learns better than a dense one, and the gap widens as you scale. DeepSeekMoE sums it up with one figure: using 39.6% of the compute, their 16-billion-parameter model beats LLaMA2 7B on most benchmarks. Serving simply inherits the decision. It copes.

But that ratio of 32.7 describes the arithmetic of a forward pass. It says nothing about what crosses your memory bus, and nothing at all about what crosses your network. The moment the model no longer fits on one GPU, those two journeys are what set your throughput, not the multiplication count.

Anyone who has served a large MoE in production knows the communication cost that comes with it. scatters the experts across GPUs, so at every layer each token has to be shipped to the machines holding the experts it selected, and the results brought back. That traffic is real, it is measurable, and it gets cited almost everywhere as the MoE problem.

Except that it is not the first cost you hit, and it is not always the dominant one. There is a regime where it is negligible and another where it rules everything. The boundary between them can be computed. That is what this article does.

First shift: from compute to memory

Start with the vocabulary trap, because it contaminates nearly every cost comparison written on the subject. Active parameters are not loaded parameters.

DeepSeek-V4-Pro’s 49 billion active parameters describe the compute for one token. They do not describe what your server must host. The router can name any of the 384 experts in a layer, and it changes its mind at every token. All 1.6 trillion parameters must therefore be resident, permanently, somewhere the GPU can reach them. In that is roughly 1,600 GB, in 4-bit about 800. You are not serving a well-mannered 49-billion-parameter model. You are serving a 1.6-trillion-parameter model of which you read a fraction at a time.

That fraction depends on the batch, and this is where the mechanics get interesting. At batch 1, a token passes through its six experts and you read only those: memory traffic per token is roughly the 49 billion active parameters. Now serve a thousand requests in parallel. Routing has no reason to send them to the same experts, and it does not. Across 384 experts per layer with six draws per token, a batch that size ends up touching everyone. You reread nearly the whole model at every step, but you reread it once for the entire batch. The memory cost per token collapses: it becomes the total weight divided by batch size.

You can verify this without instrumenting anything, by looking at the other end of the hardware spectrum. When experts are offloaded to CPU memory to serve a large MoE on a reasonable machine, KTransformers publishes its decode throughput.

Model	Active parameters	Hardware	Decode (tokens/s)
Qwen3.5-35B-A3B	3 B	4x RTX 5090	97.5
Qwen3.5-397B-A17B	17 B	4x RTX 5090	34.0
Kimi-K2.6	32 B	4x RTX 5090	29.1
DeepSeek-V4-Pro	49 B	2x RTX 5090	5.6

Readings from the public KTransformers dashboard, consulted on July 28, 2026. The two Qwen3.5 models share a precision and a GPU count, which makes them the only genuinely controlled comparison in the table.

Take the two models from the same family, same precision, same GPU count. Their totals differ by a factor of 11. Their active parameters by 5.7. Their throughput by 2.9. Throughput tracks neither exactly, and the fixed cost of each step explains why it falls short of the active-parameter ratio. But it clearly follows that scale rather than the scale of the totals, which is the point here. Keep going down the same ladder and DeepSeek-V4-Pro, with its 49 billion active, drops to 5.6 tokens per second.

Nothing travels over a network here: it all happens between DDR and the processor. Throughput follows what must be read per token, and what must be read per token follows the active parameters. The regime is memory-bandwidth-bound, and you can see it with the naked eye.

What holds on a workstation holds in the machine room too. NVIDIA’s work on latent MoE architectures puts it plainly: in low-latency serving, MoE inference is dominated by the memory-bandwidth cost of loading model weights. Not by communication.

So the first shift in cost sits right there, and it has nothing to do with the network. The MoE saves you compute and rebills you in memory: in capacity first, since 1,600 GB has to live somewhere, and in bandwidth second, since those weights must be reread at every step. That is why DeepSeek-V4-Pro fits on a node of eight B200s but overflows an NVL4 tray on GB200, where it occupies two. And why it does not fit at all on eight 96 GB RTX PRO 6000s.

Second shift: traffic that never amortizes

Now look at what crosses the network, and above all how it behaves as the load grows.

An expert-parallel MoE layer runs two collectives, and only two. Dispatch sends each token’s hidden state to the GPUs holding the experts its router picked. Combine brings those experts’ outputs back to the originating GPU, where they are weighted and recombined. Nothing else moves.

The volume follows directly. On DeepSeek-V3 a hidden state is 7,168 values, the router selects eight experts, dispatch goes out in FP8 (one byte per value) and combine comes back in BF16 (two bytes). A token therefore costs 8 x 7,168 x 3 bytes at each layer, about 168 KiB. The model has 58 MoE layers out of 61, which brings the bill to roughly 10 MB per token for the full traversal.

Ten megabytes is not much. Set against the same model’s 671 GB of weights it is derisory, and that is precisely what makes the subject treacherous. But the two numbers do not behave the same way as load rises.

Weight traffic is shared. Those 671 GB are read once per step and serve every token in the batch, so their per-token cost melts as the batch grows. traffic belongs to each token outright: the hidden state of token i helps token j not at all, it must make the trip on its own behalf. Double the batch and you double the volume exchanged exactly. The 10 MB per token do not budge by a single byte.

One cost that amortizes, one that does not. Two curves with no reason to stay parallel.

A second effect complicates the picture, that of the parallelism degree. The more widely you scatter experts across GPUs, the more destinations each token must address, and the larger the share of traffic that leaves the fast domain for the external network.

EP degree	Dispatch	Combine
EP8	77 us	114 us
EP32	155 us	273 us
EP64	173 us	314 us
EP128	192 us	369 us
EP256	194 us	360 us

DeepEP version 1 low-latency kernels, measured on H800 and ConnectX-7 400 Gb/s, DeepSeek-V3/R1 production configuration: 128 tokens per batch, hidden size 7,168, top-8, FP8 dispatch and BF16 combine.

Combine at EP128 is about 3.2 times slower than at EP8, on the same model and the same load. These figures come from version 1 of the library, and that is a caveat worth keeping: version 2 changed transport, scales to EP2048 and cuts the number of streaming multiprocessors involved by four, but it removed its zero-SM low-latency mode and never republished its latency tables. Version 1 remains the only measured reference available.

So we have, on one side, a memory cost that falls as the inverse of the batch. On the other, a network cost that is constant per token and climbs with the number of GPUs you spread your experts across. Those two curves cross somewhere. The question is where.

Where the two curves cross

The calculation takes four steps, and each rests on a number we have already put down or that can be read off a datasheet. Take DeepSeek-V3 on an H800 node, the configuration with the most complete public measurements.

First term, the network. We established it: about 10 MB per token to cross the 58 MoE layers, FP8 dispatch and BF16 combine. This volume does not depend on the batch.

Second term, memory. The model weighs 671 GB in FP8. At a large batch, routing sweeps all of it at every step, and that read benefits the whole batch. Per token, the volume is therefore 671 GB divided by batch size.

Third step, the volume ratio. 671 billion bytes on one side, ten million on the other. At batch 1, memory moves about 67,000 times more bytes than the network; at batch 100, still 670 times more. Call this the volume ratio: it equals 67,000 divided by batch size. It would take a batch of 67,000 for the two volumes to meet, which happens in no real deployment. If we stopped at volumes, the matter would be settled and the network would never count.

Fourth step, and it is the one that overturns the result: the two journeys do not happen at the same speed. The 671 GB are read from , which on an H800 delivers 3.35 TB/s. The 10 MB cross the interconnect, and DeepEP measures on that same hardware a useful all-to-all throughput of about 50 GB/s once traffic goes over between nodes. Note that these are achieved rates, not advertised link speeds: that is the only honest comparison. This speed ratio is about 67 in memory’s favour.

The two times equalize when the volume ratio falls to the speed ratio. The first is 67,000 divided by the batch, the second is 67: equality lands at a batch of roughly one thousand. The coincidence between those two numbers carries no physical meaning, it merely makes the arithmetic pleasant to write down.

Run the same calculation on traffic that never leaves the fast domain and you get another answer. DeepEP measures about 160 GB/s intranode on H800, which brings the speed ratio down to 21 and pushes the crossover out to a batch of three thousand.

There is the answer to the title, and here is what it means. Below a thousand concurrent requests, reading weights dominates so completely that all-to-all traffic vanishes into the noise: your MoE keeps its promise, it charges you 49 billion parameters instead of 1.6 trillion and the network costs you almost nothing. Above it, the hierarchy inverts, and every additional GPU you spread your experts across makes the problem worse instead of diluting it.

Does the order of magnitude hold up against measurements? It does, provided each measurement is read with its regime, because published figures run from 20% to 79% and they do not describe the same thing. Work on semantic parallelism measures up to 59.2% of expert-layer latency on eight GPUs. COMET, at ByteDance, reports 47% of full model execution time. All those points fall in the upper zone, where batch and spread are large enough for the network to have taken over.

One measurement deserves a pause, because it qualifies the nature of the cost. UBEP, on DeepSeek-R1 and V3.2 decoding across 128 ranks, finds that communication occupies about half the wall-clock time to produce a token. But the network hardware itself is only actually busy transferring for a fifth of it. The difference is waiting: GPUs sit idle for a dependency to clear rather than transmitting. The distinction matters for anyone deciding what to buy, because it says that on this configuration, more bandwidth would only fix part of the problem.

And if you want the proof by absurdity, it exists in production: some providers serve DeepSeek V3 and R1 on a single H200 or B200 node. All traffic then stays on internal links, InfiniBand sees nothing pass, and the problem everyone describes as intrinsic to MoE simply evaporates. If a MoE were by nature a network problem, that deployment would be impossible. It runs.

Why topology decides for you

The figure of a thousand tokens is not a constant of nature. It depends on how fast your GPUs talk to each other, and that speed does not vary continuously: it falls off a cliff at a precise point.

Inside an NVLink domain, every GPU is joined by a switched fabric. A Hopper has 900 GB/s of link there, a Blackwell 1.8 TB/s across its eighteen 100 GB/s links. Cross the domain boundary and traffic falls back on the network cards: a ConnectX-7 offers 400 Gb/s per port, of which DeepEP extracts about 50 GB/s useful in all-to-all.

Comparing those two families of numbers calls for care, because the first are advertised link speeds and the last is a rate actually achieved. On achieved rates, the only rigorous comparison, the gap DeepEP measures between the inside of an H800 node and the external network is about 160 GB/s against 50, a factor of three. Measured against the advertised link speeds of recent generations the two diverge by more than thirty, but at that point you are no longer comparing the same thing. Keep the factor of three, it carries the argument comfortably.

Where does that boundary sit? The answer is brutally concrete. On a classic HGX baseboard the domain stops at eight GPUs. A GB200 NVL72 takes it to 72, joined by nine switch trays for 130 TB/s aggregate, and the architecture allows up to 576 GPUs in a single domain. The Rubin generation announces 3.6 TB/s per GPU and 260 TB/s per NVL72 for the second half of 2026, a vendor figure no public deployment can verify yet.

What this changes for you states simply: your expert parallelism degree is not chosen in the abstract, it is chosen relative to the size of your domain. DeepSeek serves its decode stage at EP144 spread over eighteen nodes, so every all-to-all necessarily crosses the external network. The same EP144 across two adjoining NVL72s would keep half that traffic inside. And an EP72 fitting inside a single NVL72 would never leave it.

Redo the previous section’s calculation with a wider domain’s rates in place of the 50 GB/s and the crossover batch moves up. That is the entire economic case for the NVL72, and it is what gives the fast-link fight its stakes: 200G, ratified in April 2025, targets 1,024 accelerators per pod with load/store memory semantics, for hardware expected in late 2026. Whoever wins that fight will not gain benchmark points. They will move the threshold past which serving a MoE becomes a network problem. We took that duel apart in our MI455X versus Rubin comparison.

Imbalance, and why it hits prefill

One cost remains that we have not discussed, and it appears on no datasheet. The router does not distribute tokens evenly.

Here is what happens in practice. At each layer, every GPU computes the experts it hosts, then everyone meets at combine to reassemble the outputs. That meeting is a barrier: the layer only advances once the last GPU has finished. If a popular expert receives three times the average number of tokens, the GPU hosting it takes three times as long, and all the others wait. Layer latency is not set by average load, it is set by the busiest rank.

UltraEP’s measurements give the scale of it. At EP64, the ratio between the busiest rank’s load and the average runs from 1.30 to 4.01 depending on the model and the traffic type, and their system brings it back to 1.01. A factor of 4 means you are paying for four GPUs to get the work of one. Drop to the individual expert level rather than the rank, and on Qwen3-235B the max-over-mean ratio spreads between 10 and 20 depending on the layer and the request domain.

This imbalance has an architectural cause we crossed earlier without naming. The current trend is not only “more total parameters”, it is “many small experts”: DeepSeek-V4-Pro lines up 384 experts of 66 M against Mixtral’s 8 experts of 176 M. The finer and more numerous the experts, the more routing specializes, and the further load distribution drifts from uniform.

You might think profiling settles it: measure which experts are popular, replicate them across several GPUs, and balance returns. That is exactly what does, DeepSeek’s balancer, and it works: at scale it delivers 1.49x on and 2.54x on . The detail is clever, since adding 32 redundant experts to the original 256 also frees parallelism degrees from powers of two and allows configurations of 12 or 72.

Except that profiling assumes the past predicts the present, and it does not. ReLibra measured the intersection rate between the busiest experts of two consecutive micro-batches, across three models from 30 to 235 billion parameters. That rate stays below 0.5 in most configurations tested: from one micro-batch to the next, more than half the hot expert set has changed. You are balancing for a distribution that no longer exists.

Imbalance costs more than time. Without balancing, peak MoE activation memory rises to twice what it would be under perfectly even load in training, and eleven times in serving. That is no longer a latency question, it is a question of whether your batch fits in memory. You can lose throughput not because your GPUs are waiting, but because you had to shrink the batch to avoid running out of memory.

One last point, and it reconnects to the article’s axis. Prefill is the main source of this imbalance. In decode, where the regime is memory-bound, compute-side load skew dilutes into access latency: the GPUs were already waiting on HBM, and a little extra work changes little. If your traffic is dominated by small-batch decode, imbalance costs you considerably less than the literature suggests.

A word of honesty to close: everything we have just cited concerns DeepSeek-V3, Qwen3-235B, GLM-4.5 and Mixtral. Nobody has published imbalance measurements for DeepSeek-V4, GLM-5.2, Kimi K2.6 or Qwen3.5-397B, that is to say precisely the 384- and 512-expert models where the phenomenon should be sharpest. We are describing a trend on the previous generation, and extrapolating.

What makes disaggregation mandatory

Everything above converges on an architectural conclusion, and as it happens a very concrete software constraint imposes it anyway.

DeepEP’s two kernel families do not serve the same regime. The normal kernels target throughput and accept occupying streaming multiprocessors; the low-latency kernels go pure RDMA for decode’s small batches. Those two modes cannot coexist in the same communication group. You therefore cannot serve prefill and decode in one instance without sacrificing one of them. Disaggregation is not an optimization you add when you have time: it is the direct consequence of the two phases living in opposite regimes.

The reference deployment shows it. DeepSeek separates a prefill stage at EP32 across four nodes, with nine routed experts and one shared per GPU, from a decode stage at EP144 across eighteen nodes, with two routed experts and one shared per GPU. Over a day in late February 2025, the whole thing mobilized 278 nodes at peak and 226.75 on average, for an estimated daily cost of about $87,072 at two dollars per GPU-hour. SGLang reproduced the architecture on 96 H100s and publishes 22,282 tokens per second per node in decode, at $0.20 per million output tokens, roughly a fifth of the official API’s price.

Compute-communication overlap obeys the same regime logic, and it may be the clearest demonstration in the whole article. SGLang’s two-batch overlap splits the batch in two to hide one half’s communication behind the other’s compute. At batch 256 it yields 25.5% more decode throughput. At batch 32 it takes away 27%. The same optimization, on the same code, improves or degrades depending on the single parameter we have been discussing throughout. Below 64 to 128 tokens per device there is nothing to overlap: communication is not the bottleneck, and splitting the batch merely halves the useful work.

For anyone tuning a server, here is where it plays out.

What you are setting	vLLM	SGLang
Enable expert parallelism	--enable-expert-parallel	--ep
Pick the transport	--all2all-backend	--moe-a2a-backend
Transport regime	deepep_high_throughput or deepep_low_latency	--deepep-mode normal or low_latency
Compute-communication overlap	--enable-dbo	--enable-single-batch-overlap
Expert balancing	--enable-eplb	--enable-eplb

vLLM flags this whole area as experimental and warns that names may change. On the SGLang side, a documented limitation surprises many people in production: with DeepEP, Mooncake, NIXL-EP or MORI, the expert parallelism degree must equal the tensor parallelism degree.

The regime row is the command-line translation of this entire section: under disaggregation, the prefill instance runs in normal mode and the decode instance in low-latency mode. Orchestration layers add another rung above this, a subject we treated separately in our French coverage of AMD Infera, NVIDIA Dynamo and llm-d.

A closing warning: large-scale expert parallelism has a break-even point. On a modest model with few experts, communication overhead outweighs the weight-loading gain, and spreading further slows you down. NVIDIA says so in its own documentation. This mode is not deployed because you can afford it, it is deployed when the regime justifies it.

Sparsity always resurfaces

A ratio of 32.7 between total and active parameters does not stay politely inside the model card. It resurfaces in places you would not expect, and two of them are worth ending on.

Experts hold 95 to 97% of a modern MoE’s weights. The consequence fits in one sentence: everyone quantizes the experts, and nothing else. Kimi K2.6 ships in INT4 with attention, shared experts and the output head explicitly excluded. DeepSeek-V4 carries a dedicated expert_dtype field in FP4, separate from the FP8 of the rest of the model. The NVFP4 checkpoints NVIDIA publishes for V4-Pro put the experts in NVFP4 and keep attention and the dense layers in FP8. Precision has stopped being a property of the model and become a property of each component, and it is the MoE’s structure that forced it.

The second place is more unexpected. In llama.cpp, a batch triggers GPU prefill from 32 tokens. In ik_llama.cpp, when experts are offloaded to CPU memory, that threshold becomes 32 multiplied by the total expert count and divided by the active count, which is 2,048 tokens for a top-6-of-384 model. That factor is exactly the model’s ratio. Someone wrote that formula, obviously. But they did not choose it: it is the only value that makes the trade-off come out right, and the geometry of the MoE therefore surfaces intact in a runtime constant.

One avenue remains that nobody has quantified, and it could displace the imbalance problem rather than correct it. DeepSeek-V4 freezes routing on three of its layers: expert indices there are no longer produced by a learned network but read from a frozen table indexed by the input token. On those layers, each expert’s load becomes a deterministic function of the token distribution. It stays uneven, since a handful of vocabulary tokens recur thousands of times more often than the rest. But it is known in advance and invariant over time, which means an optimal placement can be computed offline, once, and never goes stale. No more profiling chasing a distribution that has already moved.

No public measurement says what that idea is worth, nor across how many layers it stays compatible with model quality. If it holds, it settles by construction what three years of balancers have been trying to patch at runtime. That is the question we would like to see measured before the next generation.

Sources and method

Dollar amounts are quoted as published. Performance figures are reported with their regime (prefill, decode or training), their batch size and their interconnect domain, without which they are not comparable with one another.

Verified facts

Model architectures. config.json files read directly on HuggingFace on July 28, 2026 for DeepSeek-V4-Pro, DeepSeek-V4-Flash, GLM-5.2, Qwen3.5-397B-A17B and Kimi-K2.6. Expert sizes are computed from those files, not taken from vendor communications.

Communication and expert parallelism. The DeepEP repository (version 1 latency and throughput tables, measured on H800 and ConnectX-7 400 Gb/s), EPLB, LPLB and profile-data. Measured communication fractions: UBEP (arXiv 2607.06202), semantic and speculative parallelism (arXiv 2503.04398), COMET (arXiv 2502.19811), Lina (arXiv 2210.17223), Megatron-Core (arXiv 2603.07685) for the comparison between the NVLink domain and the external network in training.

Deployments and throughput. open-infra-index for the EP32 prefill and EP144 decode configuration, node counts and daily cost. SGLang post of May 5, 2025 on large-scale expert-parallel serving (96 H100s, EP72 decode, two-batch overlap, EPLB gains) and the post of July 20, 2025 on Kimi K2. vLLM documentation on expert-parallel deployment plus engineering posts from December 2025 and February 2026. SGLang documentation. TensorRT-LLM technical blogs and NVIDIA’s October 20, 2025 publication on large-scale expert parallelism on NVL72, including the break-even point.

Interconnect. NVIDIA reference architectures and the Blackwell architecture page for NVLink 4 and 5, domain sizes and NVL72 aggregate rates. ConnectX-7 datasheet. UALink 200G 1.0 specification, ratified April 8, 2025.

Expert imbalance. UltraEP (arXiv 2606.04101, version 3 of June 18, 2026) for inter-rank and per-expert load ratios, and for the finding that prefill is the main source. ReLibra (arXiv 2605.08639) for the instability of the hot expert set between micro-batches. Libra (ICLR 2026, SNU-ARC/Libra repository) for worsening imbalance on recent generations. MoETuner (arXiv 2502.06643) and ReaLB (arXiv 2604.19503) in corroboration.

Memory regime. KTransformers dashboard consulted July 28, 2026 and the kvcache-ai/ktransformers repository. NVIDIA Nemotron work on latent MoE architectures (arXiv 2601.18089) for weight loading dominating low-latency serving. Tensor Economics analysis of September 2, 2025 for single-node DeepSeek serving.

The MoE’s training motivation. Clark et al., unified scaling laws for routed language models, ICML 2022 (arXiv 2202.01169). Krajewski, Ludziejewski et al., scaling laws for fine-grained MoE, ICML 2024 (arXiv 2402.07871). DeepSeekMoE (arXiv 2401.06066).

Credible estimates

The all-to-all volume of about 10 MB per token is a direct calculation from verified parameters (8 selected experts, 7,168-value hidden state, FP8 dispatch and BF16 combine, 58 MoE layers), not a measurement.

The crossover batch of roughly one thousand over RDMA and three thousand over NVLink is an estimate derived from those volumes and the achieved rates DeepEP measures on H800, corrected by HBM3 bandwidth of 3.35 TB/s. It assumes saturated experts, communication not overlapped with compute, and routing touching every expert at large batch. Any of those assumptions failing pushes the crossover higher.

Memory footprints by precision are computed by multiplying total parameter count by bytes per parameter, excluding the key-value cache and execution buffers.

The contrast between the NVLink domain and the external network is given at achieved rates (about 160 GB/s against 50 GB/s, measured by DeepEP on H800), not by comparing an advertised link speed against a measured throughput. The factor above thirty found elsewhere compares two different kinds of quantity and is not usable as such.

The daily cost of about $87,072 rests on the two-dollars-per-GPU-hour assumption DeepSeek itself uses.

Hypotheses

The link between growing expert granularity (many small experts rather than few large ones) and worsening imbalance is a line of reasoning, not a measurement. The correlation is consistent with Libra’s and UltraEP’s data, but no study isolates it as a variable.

The balancing benefit expected from DeepSeek-V4’s frozen hash routing is reasoning from the structure declared in the model’s configuration. No public measurement exists.

What we could not establish

No primary source from DeepSeek or NVIDIA publishes a measured communication fraction for a decode step with its full configuration. The best approximations available come from third parties.

No expert imbalance measurement has been published for the 2026 generation (DeepSeek-V4, GLM-5.2, Kimi K2.6, Qwen3.5-397B), precisely the models where the phenomenon should be sharpest. The measurements cited concern DeepSeek-V3, Qwen3-235B, GLM-4.5 and Mixtral.

The production figures DeepSeek publishes (per-node throughput, cost, share of tokens served from cache) are self-reported and have been subject to no independent audit. We reproduce neither the 545% margin advanced in the same document, which assumes every token would be billed at the highest rate, nor the $5.576 million training cost, which covers only the final run.

The Mixtral-8x7B figures cited in the introduction and in the imbalance section come from the architecture Mistral published and are arithmetically consistent, but their config.json was not reread during this verification pass.

DeepEP version 2 did not republish its latency tables after changing transport and removing its zero-SM low-latency mode. The figures cited are therefore version 1’s.

MoE serving: at what batch size does the network take over?