A GPU can wait on a component that computes nothing
An inference request first arrives as packets. The system must identify the tenant, enforce network policy, terminate or relay encryption, read input data, find any previously computed prefix and move useful blocks toward the engine that will consume them. A new request can join a batch only after the required steps, even while the GPU executes other batches. If the host CPU handles every policy, NVMe command and copy, cores intended for the runtime become infrastructure processors.
A Data Processing Unit moves that boundary. The card contains its own processor, memory, network interfaces and accelerators. It sits between network, storage and host, then handles locally the work that would have crossed the server kernel. It is not a second model accelerator: no LLM matrix multiplication runs there. It is an independent execution domain for the path that feeds, protects and observes inference.
BlueField-4 and Pensando Salina make this boundary visible. NVIDIA now calls the domain scale-in, while AMD describes a front-end network for AI factories. Their commercial scopes differ, but they answer the same pressure: more agents and longer contexts create more data access, security policy and state movement even when the number of computed tokens stays constant.
What happens when a request enters
Without a DPU, a packet reaches the network interface, triggers the host network stack, crosses firewall and virtualization rules, then wakes the runtime. If the request can reuse a prefix stored elsewhere, the same CPU participates in recovery, processes the storage protocol and orchestrates copies. The work may be asynchronous, but it still shares cores, caches and memory bandwidth with the serving process.
With a DPU, infrastructure operations run before traffic enters the server’s trust domain. The DPU classifies the flow, enforces tenant policy, presents a virtual network or NVMe device to the host and executes transfers supported by its hardware. The runtime retains the semantic decision, such as this prefix is worth retrieving. The DPU takes part in execution, such as move these blocks from this storage tier to that destination.
The separation also has a security property. A rule enforced by the host system can be bypassed by a compromised host. A correctly configured policy enforced on the card, before traffic reaches the host, remains outside tenant control. That independence explains why NVIDIA and AMD combine networking, encryption, access control, storage and telemetry in the same chip: the functions can observe or transform one flow without depending on the protected system.
BlueField-4 and Salina: published 800 versus 400 Gb/s
BlueField-4 advertises up to 800 Gb/s per port in Ethernet or InfiniBand, which is 100 GB/s raw in one direction. Salina exposes two physical 400 GbE ports, but its product brief limits both data-plane and adapter throughput to 400 Gb/s. In bump-in-the-wire mode, those ports carry ingress and egress for the same inline processing path. Adding them would count one flow twice. These ceilings say neither how many payload bytes remain after headers, nor how much reaches a GPU, nor how many requests stay within their latency objective.
| Characteristic | NVIDIA BlueField-4 | AMD Pensando Salina |
|---|---|---|
| Published networking | Up to 800 Gb/s per port, Ethernet or InfiniBand | 2 × 400 GbE physical; 400 Gb/s data-plane and adapter throughput |
| Host connection | PCIe 6 x16 | PCIe 5 x16 |
| Published onboard compute | Grace CPU, 64 Arm Neoverse V2 cores | 16 Arm Neoverse N1 cores in the Helios specification |
| Published memory | Up to 128 GB LPDDR5X | 64 GB DDR5-6400 ECC |
| Data plane | Inline accelerators and DOCA services | Programmable P4, C and C++ pipeline |
| Storage | DPU: NVMe-oF, file and object; CMX: separate STX processor | NVMe-oF over TCP or RoCE, NVMe virtualization |
| Highlighted AI ecosystem | DPU: DOCA; STX and CMX: DOCA Memos with Dynamo and NIXL | DPU-managed NVMe and announced Helios integration |
The host connection becomes a limit only when traffic crosses it. In host mode, PCIe Peripheral Component Interconnect Express. The bus linking the GPU to the CPU and RAM. Its bandwidth (a few tens of GB/s) is one to two orders of magnitude below the VRAM's, which makes the host↔GPU transfer a bottleneck as soon as data is moved off the card. 5 x16 reaches about 64 GB/s nominal per direction, compared with 128 GB/s for BlueField-4’s PCIe 6 x16. Packet overhead, transactions and flow control reduce useful throughput. In bump-in-the-wire mode, the Salina brief says PCIe only powers the card; traffic moves from one network port to the other. These ceilings cannot establish serving throughput or directly compare the two modes.
BlueField-4 emphasizes a substantial general-purpose domain with 64 Grace cores and inline accelerators. DOCA supplies the services and libraries that run routing, isolation, storage, telemetry and data movement. Pensando arranges its data plane around a programmable P4 pipeline supplemented by C and C++. The contrast is not programmable versus fixed. It is between two programming models, two datacenter-control integrations and two sets of deployed services.
The KV cache does not live in the DPU during decode
The active KV cache The stored key and value vectors an LLM has already computed for every token it has processed. It avoids recomputing attention over the whole history, at the cost of a memory footprint that grows with the context length. is reread by attention kernels for every token. Keeping it in DPU DDR or NVMe during that loop would force accesses across an interconnect much slower than GPU HBM. A DPU is not a VRAM substitute. It acts at cache-lifetime boundaries: when a block is evicted, shared with another engine, restored before decode or retained for a future request.
The BlueField-4 name covers two products that must remain separate here. The server-side BlueField-4 DPU handles infrastructure services and DOCA-supported transfers. The Vera BlueField-4 STX storage processor is a different platform: it combines a Vera CPU, ConnectX-9, NVMe and up to 1.6 Tb/s of Spectrum-X to build CMX. Assigning CMX storage or its 1.6 Tb/s to the PCIe 6 DPU would be incorrect.
Dynamo and its cache manager decide placement. NIXL provides a common interface for moving data across HBM, RAM and file, block or object storage. DOCA Memos organizes the distributed cache on that stack. Those layers can use BlueField, but their presence does not prove that every KV-cache block crosses the DPU. Our analysis of the KV cache as the center of serving covers that hierarchy; the question here is which processor executes each I/O.
AMD announces a different path: Salina accelerates storage access and manages NVMe intended to extend cache capacity. Its product brief confirms NVMe virtualization and NVMe-oF over TCP or RDMA Remote Direct Memory Access. The network card reads and writes a remote machine's memory directly, bypassing the CPU and the operating system's networking stack. It carries inter-node traffic (InfiniBand, RoCE): far faster than an ordinary socket, and an order of magnitude slower than a link inside the node. /RoCE. The explicit KV-cache connection comes from a July 2026 AMD post, not from a public protocol equivalent to NIXL or a reproducible serving benchmark. It is an announced capability, not yet proof of a gain on a defined model and workload.
The mechanism has a less visible consequence. As more cache leaves HBM, its index and policy become more important. The system must know a block’s owner, version, format, tenant, destination and avoided recomputation value. A DPU can enforce protection and execute the transfer. It cannot decide by itself that an 8,000-token prefix will be reused in thirty seconds. That information belongs to the router and runtime.
Published benchmarks answer different questions
AMD measures 117 million packets per second and advertises 782 Gb/s for a Salina SDN test in bump-in-the-wire mode, then compares 117 MPPS with the 80 MPPS published for BlueField-3. The public note does not state whether 782 Gb/s adds directions, ports or cards. Because the Salina brief publishes 400 Gb/s for the data plane and adapter, the result must not be presented as 782 Gb/s of useful one-direction throughput on one card. The protocol compares neither Salina with BlueField-4 nor any model, cache or TTFT.
NVIDIA publishes up to 1.45× the storage throughput of standard Ethernet for a BlueField-4 plus Spectrum-X stack, using files up to 50 GB. The exact comparator, full workload details and an isolated BlueField-4 result are not supplied in the post. The number shows the claimed effect of a co-designed stack. It cannot assign the gain to the DPU alone, much less compare it with Salina.
Claims about freed CPU cores, tokens per second or energy efficiency face the same limit. Moving network processing away from the CPU can return those cores to the runtime, but only when the CPU was the bottleneck. If decode already saturates HBM bandwidth, freeing twenty cores does not increase generated tokens. If the service spends its time recovering contexts or enforcing multi-tenant policy, the same offload can become decisive.
A useful benchmark would fix a model, runtime, prefix length, hit rate, storage and concurrency. It would publish TTFT at p50, p95 and p99, token throughput under a latency objective, bytes moved, consumed CPU cores and whole-system energy. No public result as of August 30, 2026 meets those conditions for both DPUs.
Choosing without inventing the missing benchmark
In a Vera Rubin stack already built around Dynamo, NIXL, Spectrum-X and CMX, the BlueField-4 DPU and STX processor occupy different roles inside one integrated path. The value does not come from an isolated 800 Gb/s port. Runtime, transfer library, network control and context storage share an operating model. That integration reduces assembly work at the price of stronger dependence on NVIDIA’s stack.
In a P4-controlled Ethernet infrastructure, where the operator already uses Pensando tooling or wants custom network services at every server edge, Salina follows another logic. Its brief precisely describes networking, security and storage primitives. AMD integrates it into Helios and claims DPU-managed NVMe for cache, but the complete serving layer is less thoroughly documented in public than the Dynamo and NIXL path.
In a small cluster where CPUs are not saturated, storage stays local and prefixes see little reuse, neither product guarantees an economic gain. A SmartNIC or conventional RDMA NIC may be sufficient. A DPU becomes rational when the operator must run networking, security, storage and high-throughput isolation without entrusting those functions to the system serving tenants.
The useful test therefore begins before purchase: profile host cores, network queues, storage accesses, cache transfers and latency SLOs. If the wait is in HBM, fix the model or GPU path. If it sits in the infrastructure path, the DPU finally has a concrete problem to solve.
Conclusion
BlueField-4 and Salina move a boundary more than they win a race. Network, storage and security stop being incidental host functions and become an autonomous compute domain on the path of every request and context block.
The next question will not be which DPU prints the largest port rate. It will be who controls the distributed state of inference. When one prefix exists across three memory tiers and several tenants, the decisive component is not the one that transports it fastest, but the one that can prove which block to move, for whom, and why the transfer costs less than recomputation.
Sources and method
Hardware specifications were frozen on August 30, 2026. Port rates and hardware characteristics are verified facts from official briefs. Performance gains remain vendor measurements limited to their protocols. No AMD figure is used as a BlueField-4 comparison when its source measures BlueField-3.
- NVIDIA BlueField-4 data sheet, June 2026: 64 Arm Neoverse V2 cores, up to 128 GB LPDDR5X, PCIe 6 x16, 800 Gb/s interface and the distinction between the DPU and STX storage processor. Verified facts.
- NVIDIA, BlueField-4 Powers New Scale-In Network Infrastructure for Agentic AI Factories, August 24, 2026: scale-in, security, control and storage roles. Architecture described by NVIDIA; throughput gains claimed by the vendor.
- NVIDIA, Scaling Agentic AI Factories Through Extreme Co-Design with NVIDIA BlueField, July 2026: relationship among BlueField-4, STX, DOCA Memos, NIXL and distributed context. Verified composition; production benefits claimed by the vendor.
- NVIDIA, Inference Context Memory Storage announcement, January 5, 2026: H2 2026 availability, CMX, hardware placement and NIXL/Dynamo integration. The “up to 5×” gain remains a vendor claim and is not used to compare the DPUs.
- AMD Pensando Salina product brief: two 400 GbE ports, 400 Gb/s data-plane and adapter throughput, PCIe 5 x16, 64 GB DDR5-6400 ECC, P4/C/C++, NVMe-oF and deployment modes. Verified facts.
- AMD Pensando DPU page: Salina’s 117 MPPS and 782 Gb/s test method and comparison with BlueField-3. Vendor measurement with unspecified aggregation, not transferred to BlueField-4 or token throughput.
- AMD Helios page: Salina integration and 16 Arm Neoverse N1 cores. Verified facts.
- AMD, AI Networking Built for Scale, July 23, 2026: Salina’s announced role in Helios and NVMe storage for KV cache. AMD’s announced architecture, without a published reproducible serving benchmark.
- PCI-SIG PCI Express 6.0 Specification: theoretical PCIe 5 and PCIe 6 rates. The per-direction values in this article are verifiable calculations from official rates, not application throughput.
- NVIDIA’s official NIXL repository: transfer abstraction across GPU memory, RAM, files, blocks and object storage. Verified in public code and documentation.
