DeepSeek's 75% HBM 'Cut' Is Narrower Than the Headline — and It Repriced an Extended Memory Rally

A headline this week said the Chinese AI lab DeepSeek had cut its memory needs for one thing by 75% and for another by 87.5%, and memory stocks promptly took it on the chin. Micron fell about 5% Friday morning, SanDisk about 4%, and Samsung and SK Hynix dropped more than 3% in Seoul. If you hold or watch any of the AI-memory names — a group that has been the single most rewarding trade of the past year — the question is whether a cheaper model is a genuine threat to memory demand or a very well-timed excuse to sell.

Start with what the number actually is, because the headline overstates its own reach. The 75% and 87.5% cuts apply to the model's KV cache, not to the memory inside a datacenter generally. The KV cache is the AI's working scratchpad — a running record of the conversation, documents, and code it has already read, which it must keep available so it can refer back to them. DeepSeek says its new V4.1-Flash cuts that global cache to 890 bytes per token, versus 3,514 for the prior V4-Flash and 48,068 for V3.2. That leaves the HBM-resident slice at a quarter of before and the persistent SSD slice at about an eighth.

That is a real engineering result, and it sits squarely inside a trend we flagged a year ago: memory is being decoupled from compute, with the model scratchpad leaking out of expensive HBM. What is notable here is where this iteration went: rather than just pushing cache down to cheaper SSD, DeepSeek shrank the cache in both tiers at once. The mechanism is a new asymmetric encoder-decoder architecture — 552 billion parameters but only 8 billion active per input token and 16 billion per output — plus sparse attention that reads only a few hundred relevant positions, and 4-bit quantization of the global cache.

The part worth flagging is how much of the SSD savings is an approximation rather than pure efficiency. DeepSeek stopped saving short-lived sliding-window context to storage, and if that state is missing it reconstructs it by replaying just the latest 128 tokens — a deliberate quality-for-memory trade the report concedes could expose capability loss in extreme conditions. There is also no independent validation yet: no third-party benchmarks of the hard-agent cases, and no disclosed cost-per-token or latency numbers. The bytes-per-token figure is a specification, not a measured economics result.

Which is the correct reading for memory demand: 75% less HBM, or hardly less?

The distinction that decides it is what the KV cache is versus what drives memory sales. The cache is one slice of the memory needed to serve these models; the other slices — the hundreds of gigabytes of weights that must sit in HBM just to run a model this size, the clusters that train it, and the raw growth in how much inference gets served — are untouched by this release. A 75% cut in the scratchpad is not a 75% cut in HBM sold, and nobody is claiming total memory demand fell by that amount. That is the first and most important correction to the headline.

The second is the direction the efficiency pushes. In pricing terms, DeepSeek's move lowers the cost of keeping an AI agent alive and reusing its context over a long session. Cheaper per-inference cost is historically what expands total compute and memory consumption, not what shrinks it. If the number of agents and users grows faster than the per-token memory each of them needs, total memory demand climbs even as each model's footprint falls. That is the mechanism the bear case has to overcome, and the report's own math concedes that a cache that is eight times smaller is not eight times cheaper once compute, network, and harness overhead are counted.

None of this makes the selloff wrong. What it makes is a reflection of what these stocks were priced for. This is a group that has already delivered absurd numbers: Micron is up roughly 240% year-to-date and its trailing revenue has nearly tripled, with gross margin above 70% and operating margin near 66%. SanDisk is up roughly 600% year-to-date, and both names trade near 12 times trailing sales. When a stock has moved that far, its valuation stops being anchored to today's teardown economics and starts leaning forward onto next quarter's growth staying intact — which is precisely the lever a scoped, unvalidated efficiency claim pokes.

So the honest answer to "what does it mean for memory" is narrower than the headline and wider than the panic. It does not prove a demand crash: the cut touches one slice of one cost line, part of it is approximation, and cheaper agents tend to breed more agents. But it does prove the derivative these valuations now rest on — how much memory gets consumed per unit of intelligence — is no longer a one-way ratchet. The resolution will show up where it always does for memory: in whether efficiency-driven agent volume outruns the shrinking per-token footprint. Until that is measured, treat the 75% as a spec sheet that repriced expectations, not as a count of bytes that vanished from the industry.