From “over 600 billion parameters” to one 96 GB option
Apple’s March 2025 Mac Studio announcement made an unusual local-AI promise. An M3 Ultra configuration could keep a language model with more than 600 billion parameters entirely in memory. The enabling specification was not a neural-engine TOPS figure. It was up to 512 GB of unified memory, visible to CPU and GPU, backed by more than 800 GB/s of bandwidth.
Apple’s August 2026 specification page describes a narrower product. M3 Ultra still comes with a 28-core or 32-core CPU, a 60-core or 80-core GPU and 819 GB/s of memory bandwidth. The memory section lists 96 GB. Ars Technica recorded the removal of 512 GB in March. MacRumors recorded the removal of 256 GB in May. The silicon did not change; the purchasable configurations did.
| Launch, March 2025 | Catalog, August 2026 | |
|---|---|---|
| Unified memory | 96, 256 or 512 GB | 96 GB |
| Memory bandwidth | over 800 GB/s | 819 GB/s |
| Maximum GPU | 80 cores | 80 cores |
| Apple LLM claim | >600B resident | not repeated as an orderable capacity |
| Technical effect | very large resident models | same bandwidth, lower residency ceiling |
The timeline proves a catalog cut. It cannot, by itself, prove one cause. Apple did not publish a note assigning each removed capacity to a specific supply decision. Tim Cook acknowledged stronger-than-expected demand for Macs used with AI and agentic tools, constrained availability and substantially higher memory costs. Reporting connects those facts to the global DRAM shortage. That is a documented inference, not a configuration-by-configuration Apple explanation.
Why 96 unified GB is not ordinary system RAM
On Apple silicon, unified memory is a physical programming property. CPU and GPU access the same pool. MLX documentation shows that an array can be created once and processed by either device without an explicit host-to-VRAM copy. Its scheduler inserts dependencies when a CPU operation feeds a GPU operation.
That architecture made the 512 GB Studio a category of its own. Adding ordinary host RAM to a CUDA workstation does not let its GPU stream 400 GB of weights at local VRAM bandwidth. M3 Ultra’s installed memory was also GPU memory. The trade-off remained severe: 819 GB/s sits far below modern HBM accelerators, and enormous dense models generated slowly. The critical capability was that they ran on one desktop at all.
The 96 GB label needs a correction too. macOS, applications, MLX workspaces and the KV cache The stored key and value vectors an LLM has already computed for every token it has processed. It avoids recomputing attention over the whole history, at the cost of a memory footprint that grows with the context length. all consume the pool. MLX exposes device_info() to report both physical memory and max_recommended_working_set_size, while set_wired_limit() controls how much remains resident. Useful model capacity is not a universal fraction of 96 GB. It must be measured on the system and remain below total memory.
What still fits
Weight capacity begins with parameters × bits / 8, then adds quantization Reducing the number of bits that encode each weight of a model (from 16 bits down to 8, 4, or fewer). It shrinks the memory footprint by the same factor, at the cost of a controlled accuracy loss, without changing the parameter count. metadata, runtime buffers and context cache. The formula predicts neither model quality nor throughput. It removes physically impossible configurations.
| Class | Estimated weights | Inside 96 GB? |
|---|---|---|
| Dense 32B, 4-bit | 18 to 22 GB | yes, large margin |
| Dense 70B, 4-bit | 40 to 45 GB | yes, room for cache |
| Dense 70B, 8-bit | 72 to 80 GB | possible, system and context constrained |
| Dense 120B, 4-bit | 68 to 78 GB | tight, validate the real artifact |
| Dense 405B, 4-bit | roughly 220 to 250 GB | no |
| 600B+ MoE, 4-bit | more than 300 GB total | no, even with few active parameters |
MoE Mixture-of-Experts. An architecture where the network is split into many experts, of which a router activates only a small subset per token. Per-token compute follows the number of active parameters; memory follows the total count, since every expert must stay resident in VRAM, ready to be called. models expose the decisive distinction. Active parameter count controls part of the work per token. Total parameter count controls residency because every expert must remain callable. A 600B model activating 30B parameters can behave like a much smaller compute load after loading, while still requiring hundreds of gigabytes of weights. The 512 GB Mac attacked that capacity problem. Unified memory cannot make the same artifact fit inside 96 GB.
At the other end, a 4-bit 32B uses only a fraction of the pool. Moving that workload from 96 GB to 512 GB never increased the 819 GB/s interface or GPU core count. The larger configuration bought room for other state, not more tokens per second for that model.
Long context consumes the gain from smaller weights
The most systematic public dataset we found comes from an MLX community contributor running a 512 GB M3 Ultra Mac Studio: 276 benchmark runs, five models, six quantization levels, seven context lengths, three trials per cell, batch 1 and median throughput. It is neither an Apple benchmark nor an independent replication. It is useful because the raw data and changing variables are explicit.
Qwen 32B occupies much less than 96 GB in the test. Its context-scaling behavior therefore applies to a workload still accessible on the current catalog, after matching GPU core count and software versions.
| Format | 1K context | 32K context | 128K context |
|---|---|---|---|
| F16 | 10.4 tokens/s | 8.5 tokens/s | 5.5 tokens/s |
| Q8 | 18.3 tokens/s | 13.4 tokens/s | 7.1 tokens/s |
| Q4 | 31.2 tokens/s | 19.0 tokens/s | 8.5 tokens/s |
| Q2 | 47.6 tokens/s | 24.1 tokens/s | 9.3 tokens/s |
At 1K, Q2 delivers 4.6 times F16 throughput. At 128K, that ratio falls to 1.7. The author attributes over 70 percent of memory traffic to the FP16 cache in that regime. The mechanism fits: smaller weights reduce bytes streamed per token, then a growing cache absorbs the saved bandwidth. Our KV-cache analysis covers that crossover in depth.
The same dataset reports 4-bit Llama 405B generation falling from 2.9 tokens/s at 1K to 2.1 at 64K, with time to first token reaching minutes as prompts grow. Capacity made the experiment possible. It did not make a dense 405B conversational. A new single Mac Studio can no longer reproduce that experiment.
The adversarial case: almost nobody needed 512 GB
The benchmark supplies the strongest criticism of this article. Its author calls 512 GB excessive for common single-model inference: Qwen 32B Q4 occupies about 19 GB in that setup. Spare memory cannot raise bandwidth or add GPU cores. Local chat with a 7B, 30B or quantized 70B retains the core value of M3 Ultra at 96 GB.
That criticism is correct. The current Mac Studio is not “broken for AI.” It still offers a rare desktop capacity, MLX’s native unified-memory model and room for networks that one 32 GB consumer GPU cannot load.
What vanished is a research option rather than the median use case. The 256 GB and 512 GB machines could compare 405B quantizations, keep several models resident, hold huge caches, fine-tune larger networks or explore MoEs whose low active count hides a large total footprint. Those were the tasks that differentiated M3 Ultra from ordinary CUDA workstations. Low sales volume does not erase technical uniqueness.
How 96 GB changes the purchase decision
For quantized 7B through 70B, capacity is not the first constraint. Choose the GPU core count, measure prefill The opening phase of LLM inference: prompt tokens are processed in parallel to build the context state. That reuse raises arithmetic intensity and can make the phase compute-bound; the exact regime depends on the model, batch, context and backend. and decode The autoregressive generation phase of an LLM: one token is produced at a time while rereading weights and the relevant context state. At low batch sizes, that traffic can make decode memory-bandwidth-bound. Its share of total cost still depends on input and output lengths, batching and cache reuse. , then size context. The 96 GB machine can keep weights and a useful cache in one pool.
For dense 120B, use the actual model file. A Q4 estimate may fit, but macOS, workspaces and a long context compete for the remaining space. The simple 120 billion × 4 bits calculation omits scales, tables and cache.
For dense 405B or giant MoEs, one new Mac Studio fails on capacity. A used 512 GB configuration, several Macs or a GPU server becomes the relevant category. Thunderbolt clustering adds a distributed parallelism model; it does not recreate one simple 512 GB local pool.
For multi-model serving and fine-tuning, 96 GB closes the envelope earlier. Unified memory must carry weights, activations, gradients, optimizer state when applicable and the rest of the system. An inference workload occupying 50 GB can cross 96 GB as soon as it becomes training.
Conclusion
Apple did not make M3 Ultra slower. It made the Mac Studio less unusual. Its 819 GB/s interface, CPU-GPU memory pool and MLX stack remain a coherent local-LLM platform for workloads below the new ceiling. A well-configured 32B loses no generation throughput because the 512 GB order box disappeared.
The catalog did remove a rarer capability: opening models weighing hundreds of gigabytes on a desk, even when LPDDR bandwidth made their limits obvious. In 2025, Apple sold the option to experiment first and choose the model later. In 2026, buyers must reverse that order. Ninety-six gigabytes is neither inadequate nor equivalent to 512. It moves the first question from “how fast?” to “is it still resident?”
Sources and method
Apple launch facts come from the March 5, 2025 release, Apple unveils new Mac Studio, which states 96 GB to 512 GB, over 800 GB/s and residency for LLMs above 600 billion parameters. The current configuration comes from Apple’s live Mac Studio specifications, which listed 96 GB and 819 GB/s at publication.
The removal timeline uses Ars Technica’s report on 512 GB and MacRumors’ report on 256 GB. They also report Tim Cook’s comments on demand, supply and memory cost. We do not promote their supply-chain interpretation into a detailed official Apple explanation.
The software mechanism comes from MLX documentation on unified memory and set_wired_limit. Those pages establish the shared CPU-GPU pool and the distinction between total memory and recommended working set.
Performance measurements come from the community discussion Systematic inference benchmarks: 5 models × 6 quants × 7 context lengths on M3 Ultra. Its stated protocol covers 276 runs, three trials per configuration, batch 1 and 256 generated tokens. We did not reproduce it and Apple does not validate it, so the figures remain third-party measurements.
The model-size ranges in Table 2 are residency estimates derived from parameters × bits / 8 with allowances for weight-format overhead. They deliberately exclude a fixed KV-cache budget because cache size depends on architecture, batch, context length and precision. Validate a purchase with the target artifact size, the MLX working-set limit and a run at the intended context.
