Your model has outgrown one GPU
You have found a model that works well for document analysis or a team’s coding tasks. Its quantized weights no longer fit on your graphics card. Lowering precision further becomes uncomfortable, so you start looking at workstations with more memory.
Two names stand out: AMD Threadripper Halo Station and the GB300 NVIDIA DGX Station. AMD describes the former as an IFA 2026 prototype due in 2027. It is a different class of system from Ryzen AI Halo, just as DGX Station is different from the smaller DGX Spark.
AMD’s product announcement advertises up to 2.6 TB of combined memory and 3.4 times the total capacity of DGX Station. That comparison adds memory capacities. It does not measure response speed or establish how many agents can work within an acceptable latency budget.
This analysis uses specifications and documentation reviewed on September 10, 2026. LeCompute has not benchmarked either workstation. The calculations below are sizing examples, and announced specifications still need checking against shipping configurations.
Identify the memory pools before adding them
HBM High Bandwidth Memory. Stacked DRAM integrated into the accelerator package through a very wide interface. It provides several TB/s, although the exact packaging topology depends on the product. , short for high-bandwidth memory, sits close to the GPU to feed its compute units. CPU memory can hold more data, but the GPU reaches that data through a different path. Both systems need that distinction.
| Memory | Threadripper Halo Station | DGX Station GB300 |
|---|---|---|
| GPU memory | 4 × 144 GB HBM3e; 576 GB total | 252 GB HBM3e |
| HBM bandwidth | 4 TB/s per card; 16 TB/s aggregate | 7.1 TB/s |
| CPU memory | Up to 2 TB RDIMM | 496 GB LPDDR5X |
| Combined capacity | About 2.6 TB | 748 GB |
The MI350P specifications give per-card figures. NVIDIA’s DGX Station specifications explicitly separate CPU and GPU memory. Read together, they describe two different engineering tasks: distributing work among GPUs and making effective use of CPU memory from a GPU.
Several AMD cards could host independent copies of a model that fits on each card. Alternatively, a capable runtime could divide one model among them. Those are different deployment goals, as discussed in our single-card versus multi-GPU comparison.
Independent copies can process separate requests. A divided model needs intermediate results to move between the devices responsible for successive parts of the computation. Aggregate memory may solve the capacity problem while leaving latency unresolved.
Coherence simplifies addressing, not access cost
DGX Station connects Grace and Blackwell through a coherent memory system. CPU and GPU can share an address space and maintain a consistent view of data. The physical memory does not all move next to the GPU.
NVIDIA’s Grace memory-placement guidance makes locality an explicit concern. An allocation in CPU memory behaves differently from one in HBM. A convenient address space does not guarantee that a particular inference runtime uses the extra capacity efficiently.
For a deployment, ask what the runtime actually does: keep weights in HBM, access them remotely, transfer pieces as needed, or execute some operations on the CPU. These choices lead to different performance profiles. Successfully loading the model is only the start of the evaluation.
Halo Station needs a transfer map
The MI350P product brochure lists a PCIe Peripheral Component Interconnect Express. The bus linking the GPU to the CPU and RAM. Its bandwidth (a few tens of GB/s) is one to two orders of magnitude below the VRAM's, which makes the host↔GPU transfer a bottleneck as soon as data is moved off the card. 5.0 x16 interface. It does not provide a complete map of the future workstation’s motherboard connections, shared links or effective accelerator-to-accelerator paths.
Direct access between GPUs depends on the platform and software. AMD’s HIP API provides a function to query whether one device can access another device’s memory. That capability must be checked on the system; it does not establish identical paths or bandwidth for every device pair.
The reverse claim would also overreach: splitting a model is not inherently a poor solution. A runtime may keep each shard close to the GPU that computes it and limit communication. Concurrent requests can also improve utilization. The intended workload determines whether the topology helps.
Budget for weights and context
Consider a hypothetical 600-billion-parameter model, with every weight stored in four bits. In decimal units, raw weights occupy 600 billion × 4 / 8, or 300 GB. This excludes quantization scales, tensors stored at a different precision and runtime allocations.
Those weights alone exceed the main DGX Station GPU’s HBM capacity. Some data must go elsewhere, or the representation must change. The combined memory capacity gives room to investigate a solution, without telling us its generation speed.
An ideal four-way division would place 75 GB of raw weights on each AMD card. That is arithmetic, not a demonstrated deployment. Real models may have uneven shards, replicated structures and allocations that do not divide neatly.
The KV cache The stored key and value vectors an LLM has already computed for every token it has processed. It avoids recomputing attention over the whole history, at the cost of a memory footprint that grows with the context length. also needs space. It stores the keys and values attention uses to reuse earlier context. For an illustrative full-attention architecture, assume 80 layers, 8 KV heads per layer, a head dimension of 128 and 2-byte values. At 32,768 tokens, one sequence needs 2 × 80 × 8 × 128 × 32,768 × 2 bytes, about 10.7 GB. Four separate sequences without shared prefixes need 42.9 GB.
These assumptions do not describe a named checkpoint. They show why allocating the entire budget to weights can leave no room for useful context. Local attention, cache compression or shared prefixes would change the calculation. Our KV cache infrastructure guide explains the mechanisms behind those choices.
Compare the work you need to finish
A fair evaluation needs the same checkpoint, quantization, context and output requirements. Start with one request and record time to the first response, followed by generation speed. Then increase concurrency to reflect the team that will use the workstation.
Keep failed requests, latency-budget violations and peak memory in the results. High aggregate throughput can coexist with unacceptable waits for individual users. Measure wall power during the same workload; adding component power ratings does not measure system efficiency.
Pricing must cover the complete configuration and its support. The reviewed sources do not establish a final like-for-like quotation for both systems. A cost-per-token or payback claim would currently depend on assumptions too large to support a purchase recommendation.
Choose a layout your runtime can exploit
If the model already fits on one card, establish whether you need a faster individual conversation or more simultaneous conversations. If the weights exceed one card’s memory, ask for a model-placement and generation demonstration before making aggregate capacity the deciding factor.
Halo Station is worth watching for its GPU memory distributed across several accelerators. DGX Station offers a different arrangement around one GPU and coherent CPU memory. The useful decision rests on your model, runtime and acceptable latency. Today’s specifications can help design that evaluation; they cannot name its winner.
Sources and method
Announced specifications. AMD, Threadripper Halo Station, including configuration note HALO-02; MI350P specifications and product brochure. NVIDIA, DGX Station GB300 and datasheet. Reviewed September 10, 2026. These are vendor sources, not independent measurements. AMD’s maximum configuration should not be equated with every exhibited prototype, and a server-card specification alone does not define workstation cooling.
Verified mechanisms. NVIDIA, Grace Hopper/Blackwell memory placement; AMD, HIP peer-memory access API. These explain access mechanisms without validating an inference recipe on Halo Station.
Calculations and limits. Weight and cache budgets use explicit teaching assumptions and decimal units: 1 GB is one billion bytes. The cache example stores separate keys and values at every layer, without compression or sharing between sequences. Margins, distribution and performance still require measurement. No LeCompute hardware benchmark, inference speedup or price is claimed.
