CXMT (长鑫存储, ChangXin Memory Technologies) is China’s largest DRAM company. Revenue surged in 2025, and the company is now pursuing a Shanghai IPO with a target valuation reportedly in the $42 billion range. It is already considered the world’s fourth-largest DRAM player by capacity.

The market still tends to view CXMT as just “China’s late-arriving DRAM maker.” But look at the products one by one, and the story changes. DDR5 is already shipping inside Lenovo laptops. LPDDR5X accounts for roughly 30% of the Chinese smartphone market. HBM still has a wide generational gap, but within the context of China’s domestic AI supply chain, it is not something you can dismiss simply as “behind.”

The point that deserves the most attention is 3D DRAM. In 3D DRAM, there is a real possibility that the entire competitive landscape gets redrawn.

I recently had the chance to speak separately with the head of SK hynix’s Americas operation and a current ASML employee. Both had weighty things to say about China’s 3D DRAM efforts. I will get into the details later, but after those conversations, I came away with a strong sense that China’s 3D DRAM capabilities should not be taken lightly.

In this article, I will first lay out where CXMT stands today, then break down how far it has actually gotten in DDR5 and LPDDR5X sales, why HBM still has a significant gap, and how 3D DRAM and the equipment ecosystem factor in to what kind of variable this company could become for the memory industry.

Table of Contents

1. Where CXMT Stands Today

2. Commodity DRAM: The Opening Is Already There

3. HBM, Where the Biggest Gap Remains

4. 3D DRAM, Where the Game Gets Flipped

5. What 3D DRAM Changes: Not Just CXMT, but the Entire Equipment Ecosystem

6. What Actually Changes for the Big 3

7. Threat or Illusion

This article does not constitute a recommendation to buy or sell any specific security. All investment decisions are solely the responsibility of the investor.

Where CXMT Stands Today

CXMT’s starting point was 2016. Backed by China’s Big Fund semiconductor self-sufficiency initiative, the company built a 12-inch DRAM fab and began volume production with 19nm DDR4 and LPDDR4. Wafer output, which stood at around 40,000 wafers per month in 2020, has reportedly grown to approximately 720,000 wafers per quarter by the end of 2025. The company now operates three 12-inch DRAM fabs.

To understand why that number matters, it helps to cover the basics of DRAM process technology. DRAM is a memory semiconductor that stores data, and each memory cell consists of one transistor and one capacitor in what is known as a 1T1C structure. The transistor acts as a switch for reading and writing data, while the capacitor stores charge to represent 0s and 1s. Ultimately, DRAM competitiveness comes down to how small and precisely you can make that cell.

Process shrinking means reducing the size of this cell. Smaller cells mean more cells fit into the same area, chip capacity goes up, and more chips come off a single wafer. The result is lower cost per bit. This is why Samsung, SK hynix, and Micron are obsessed with the race through 10nm-class nodes like 1a, 1b, and 1c.

The challenge is how you implement that shrink. The Big 3 use ASML’s EUV lithography tools to achieve finer patterns with relatively fewer process steps. CXMT, however, cannot access EUV equipment due to US-led export controls. So it relies on DUV tools and multi-patterning to achieve the same level of patterning. Circuits that cannot be printed in a single exposure are split across two or even four passes. This adds process steps, and each additional step accumulates overlay errors, putting CXMT at a disadvantage in yield and cost.

Despite this, CXMT has pushed its way up to its G4 node using DUV multi-patterning alone. G4 is an internal generation label. According to TechInsights analysis, the G4 feature size is around 16nm, roughly a 20% cell size reduction from the previous G3. The Big 3 were already in volume production at the 16nm level around 2018 to 2019, which puts the gap at five to six years or more. The fact that CXMT got here without EUV is precisely why the industry started taking it more seriously. It is also the backdrop to Seoul National University Professor Hwang Cheol-seong’s assessment that CXMT could produce 11nm-class DRAM without EUV.

So where does the business stand? In terms of revenue-based global DRAM market share, CXMT was at roughly 4% as of Q2 2025, still in the single digits. But its share of production capacity has already climbed into the low teens and is expected to expand further by 2027. The capacity to run wafers is already significant, but the share of products it can actually sell remains limited. Node competitiveness, yield, and customer qualification have not fully caught up yet. Once those three align, the revenue share could climb faster than people expect.

The financials reflect this structure directly. According to Reuters, citing IPO filings, CXMT posted a loss of approximately 2.3 billion yuan in the first half of 2025 despite surging revenue. The earliest breakeven is projected for 2026. Pursuing an IPO while still in the red means the market is betting on future growth rather than current profits. CXMT reportedly invested $6 to $7 billion in capex between 2023 and 2024, and it is building a new DRAM fab in Shanghai that is two to three times the size of its existing fabs. A separate HBM back-end fab is also under construction.

CXMT is not yet a finished powerhouse. Its technology trails the Big 3, its revenue share is still low, and it is losing money. But production capacity has already reached the global top four, and once product competitiveness and customer qualification fall into place, the numbers can shift quickly.

Now let’s go product by product.

Commodity DRAM: The Opening Is Already There

Commodity DRAM is the general-purpose memory that goes into PCs, laptops, smartphones, and standard servers. DDR5, LPDDR5X, and server-grade RDIMM and MRDIMM all fall into this category. Unlike HBM, this market is not decided by a single overwhelming technology edge. What ultimately matters is whether you can build products at a certain quality level, whether you can actually supply customers, and how reliably you can push volume. By that measure, CXMT has already reached a position that is hard to ignore.

The key question is not whether CXMT has caught up with Samsung or SK hynix on technology. It has not. The more important point is that while the Big 3 pour resources into HBM, CXMT has started filling the vacancies opening up in the commodity space with real volume. The threat does not come from technology brochures. It comes from shipments and adoption.

LPDDR5X: Where It First Made Its Presence Felt

LPDDR5X is where CXMT first showed real muscle. LPDDR5X is a mobile memory used in smartphones, tablets, and AI edge devices, and in the Chinese domestic market, CXMT has already built a presence that is difficult to dismiss. According to CXMT’s own disclosures, its 8533 Mbps and 9600 Mbps products are in volume production, while the 10667 Mbps product is in the customer sampling phase. Per Caixin Global reporting, CXMT already holds roughly 30% of China’s smartphone LPDDR market. It would be premature to say that every speed grade is in stable mass production, but at least within the Chinese domestic market, the company is clearly generating meaningful volume.

The reason CXMT has an edge here is less about the technology itself and more about market structure. Chinese OEMs like Xiaomi, Oppo, Vivo, and Honor ship massive smartphone volumes every year. For them, CXMT memory is not just a domestic substitute. It is a practical option that diversifies supply chains while also lowering costs. The longer US-China tensions persist, the stronger the incentive to increase adoption of domestic components.

Qualcomm’s remarks add symbolic weight here. During the Q1 FY2026 earnings call in February 2026, CEO Cristiano Amon stated that Qualcomm is in a qualified state with all memory suppliers, including CXMT. That means the technical groundwork for Chinese OEMs to pair Qualcomm APs with CXMT memory is already in place. This does not yet signal large-scale global proliferation, but it is safe to say that in LPDDR5X, CXMT has moved past the market entry stage.

DDR5: Still Short, but Already Shipping

DDR5 is also seeing clear changes. DDR5 is the standard memory spec used widely in PCs and servers, and CXMT is no longer a company that only develops products in this space. G4-node DDR5 products are being distributed through Chinese memory brands like Gloway, and at Chinese semiconductor trade shows, CXMT has shown high-speed specs approaching the Big 3’s latest. In November 2025, Caixin Global reported that CXMT was supplying DDR5 to Lenovo, and by March 2026, reports emerged that some models in the Lenovo ThinkBook 2026 series had CXMT memory modules inside. This is not a candidate under review. It is a supplier whose parts are going into products that actually ship.

That said, you cannot call it on par with the Big 3 just yet. There are clear limits in manufacturing competitiveness. According to TechInsights, CXMT’s DDR5 chip die size is approximately 67 square millimeters, about 40% larger than Samsung’s equivalent product. Because of the gap in process node, even at the same capacity, fewer chips come off each wafer, and the cost structure is inherently less favorable. Reports of early yield issues and high-temperature stability problems reinforce these weaknesses. There are assessments that quality has improved enough to be usable for PCs, but it is still hard to say CXMT has achieved Samsung-level or SK hynix-level yield and cost competitiveness.

But in the commodity market, you do not need top-tier polish to be a threat. What matters here is that CXMT is now making DDR5, selling it, and getting it into customer systems. There is a bigger gap than people realize between a company that cannot build something and a company that ships it, even imperfectly.

Server memory is an even more conservative segment. CXMT has announced RDIMM and MRDIMM lineups, but servers are not like PCs where simply working is enough. You need CPU platform compatibility, long-duration stability, the customer’s own internal validation, and reliability testing in live data center environments. The certification process is longer and far more demanding. So the server side has not been cracked as quickly as the PC side.

That said, the Chinese domestic server market could be different. For local players like Alibaba Cloud and Tencent Cloud, the motivation goes beyond price. Supply chain independence matters. Even without following the same conservative validation frameworks that global hyperscalers use, there is room for limited adoption after internal evaluation. It may not be enough to shake the global server market right now, but the Chinese domestic market alone could become a meaningful source of demand.

The Real Story Is Structure, Not Products

What becomes even more important here is not the completeness of individual products but the structure of the industry. Samsung, SK hynix, and Micron are all concentrating wafer capacity on HBM right now. But HBM DRAM dies are significantly larger than commodity DDR5 dies. The same wafer yields fewer dies, and it takes far more capacity to supply the same amount of memory. The more the Big 3 lean into HBM, the tighter commodity DRAM supply gets in relative terms.

In this environment, the rules of the game change.

The winner is not the company with the perfect product. It is the company that has something ready to ship right now. When supply itself is tight, the value of an alternative supplier grows, even if quality is a step below the leaders. This is exactly where CXMT finds an opening.

There are already signs of this playing out.

In early 2026, Reuters re-cited Nikkei Asia reporting that HP and Dell had explored qualifying CXMT DRAM, and that Acer and Asus were also looking into the possibility of adopting Chinese-made memory. It is too early to say this has reached large-scale adoption. But the fact that global OEMs have stopped treating CXMT as a non-existent player and started viewing it as an alternative supplier they could use if needed is itself significant.

Putting It Together

The commodity DRAM threat from CXMT is not arising because its products are better than the Big 3’s. It is arising because while the Big 3 focus on HBM, CXMT has started filling the gap in the commodity space with actual volume. In LPDDR5X, it has already established a meaningful presence in Chinese domestic demand. DDR5 has moved to the stage of real shipments and system integration. Servers need more time, but the Chinese domestic market can serve as a foundation.

What you should be watching right now is not technology demos. It is shipments, adoption, and supply gaps. In commodity markets, it is often not the most capable company but the one that fills the empty space first that reshapes the landscape. That space is already open, and CXMT is walking into it.

This wraps up the commodity DRAM story. LPDDR5X already accounts for 30% of the Chinese market, and DDR5 has started going into Lenovo products. The next questions become more important.

Why does such a large gap remain in HBM?

And conversely, why does China have a chance to flip the script in 3D DRAM?

If these shifts could change not just the memory makers but the winners and losers among equipment companies, who faces the most uncomfortable future?

Subscribe now

Read more

Two straight days of freefall. When a stock that’s gone up 25x in a year drops 20% in a week, the word “correction” doesn’t quite capture it. Roughly $20B in market cap evaporated in five trading days. A lukewarm reaction to the $1B Nanya investment, the “AI needs less memory” panic triggered by Google’s TurboQuant, supply glut fears after Micron’s capex announcement, lock-up expiry selling. The bad news hit all at once.

The Nasdaq rallied as high as +1.6% intraday before rolling over into the red. On a day when the broader market was already shaky, SanDisk cratered 8%. A stock with a beta of 2.74 getting hit by both market weakness and sector-specific headwinds simultaneously. The question is whether this is a memory sector problem or a SanDisk-specific problem. Whether the structural story of NAND supply tightening has broken, or whether this is simply the inevitable pullback in a 25x stock. BofA, in the middle of this selloff, raised its price target from $850 to $900. The market and the analysts are looking in opposite directions.

In this piece, I’m going to tear apart each of the past week’s catalysts one by one. I’ll separate noise from real risk, and assess whether the fundamentals are actually impaired, from the perspective of a semiconductor engineer.

Disclaimer

This article is not a buy or sell recommendation for any specific stock and does not constitute investment advice. The author does not hold a position in any of the stocks mentioned. The information contained herein is based on publicly available data and represents personal analysis only, with no guarantee of accuracy or completeness. All investment decisions and resulting gains or losses are solely the responsibility of the investor.

What Happened This Week

Feb 17–18 (background): WD secondary offering. Western Digital executed a secondary offering of its remaining SanDisk stake. Roughly 5.82 million shares were priced at $545 per share, with the deal closing on February 19. SanDisk received none of the proceeds. The ownership structure shift had already begun.

March 20: Lock-up expiry. Separately from the February offering, a lock-up period expired on March 20, making over 2 million insider-held shares eligible for sale. The stock slipped from $772 to below $740. Given that the stock had rallied 25% in the same week, the lock-up expiry became a convenient excuse for profit-taking.

March 20: Micron earnings fallout. On the same day, Micron reported quarterly results and highlighted its capex plans. Concerns about increased NAND supply spread across the entire memory sector. SanDisk fell 5%.

March 23: Profit-taking + valuation warnings. With the stock price above the average analyst target of $761, the perception that “it’s more expensive than consensus” took hold. The broader market rallied on geopolitical developments (Iran tensions), but SanDisk moved in the opposite direction. Down 3.76%.

March 25: Double punch. First, SanDisk announced a $1B equity investment in Taiwan’s Nanya Technology. Down 6%. Second, Google announced TurboQuant. “Does AI need less memory now?” The entire memory sector sold off.

March 26 (today): Selling continues. Far from recovering yesterday’s losses, the stock dropped another 8.13%, hitting $623. The Nasdaq briefly attempted a rebound intraday before rolling over again, and SanDisk caught both the broader market weakness and memory sector dumping at the same time. The aftershocks of the Nanya deal and TurboQuant hadn’t faded, and a broader risk-off move piled on top. Down 20% from the $772 high. Roughly $20B in market cap gone in one week.

Now let’s figure out what’s noise and what’s real.

Subscribe now

Noise: Lock-up, Profit-Taking, Nanya

Lock-up expiry and profit-taking are supply events. Short-term traders exiting a 25x stock is natural. It’s not structural change. The stock is still well above its 50-day moving average of $593.

The Nanya investment deserves a closer look. The market knocked the stock down 6% on this deal, so let’s be precise about what actually happened.

SanDisk, through its subsidiary SanDisk Technologies, acquired a 3.9% stake in Taiwan’s Nanya Technology for $1B. The deal involved roughly 139 million newly issued shares at a 15% discount to Nanya’s 30-day average price, structured as a private placement. Under Taiwanese securities law, there’s a 3-year lock-up. Alongside the equity investment, SanDisk and Nanya signed a multi-year strategic DRAM supply agreement.

One important piece of context. SanDisk wasn’t the only participant in this private placement.

It was part of a $2.5B (NT$78.7B) round that included SanDisk at $1B (roughly 139 million shares, 3.9%), Solidigm (SK hynix subsidiary) at $500M (roughly 71.39 million shares, 2%), Kioxia at $500M (70 million shares, 2%), and Cisco at $500M (71.5 million shares, 2%). All at the same price of NT$223.9 per share. This wasn’t an exclusive SanDisk deal. It was an industry event where the three major NAND players (SanDisk, Kioxia, Solidigm) plus a networking giant (Cisco) simultaneously locked in DRAM supply. All three SSD makers signed separate DRAM supply agreements with Nanya.

Three SSD manufacturers locking in DRAM sourcing at the same time sends a strong signal: the entire industry recognizes a DRAM bottleneck.

Why Nanya? Nanya Technology is a Taiwan-based pure-play DRAM maker. As Samsung, SK hynix, and Micron have shifted resources toward DDR5 and HBM, deprioritizing DDR4 production, Nanya has emerged as a critical supplier in the DDR4 market. TrendForce spot pricing shows DDR4/16G trading above DDR5/16G, an inversion. The older-generation DDR4 costs more because the big three have structurally reduced DDR4 output.

Nanya’s Q4 2025 results reflect this trend. Per Nanya’s own disclosures, the company posted record quarterly EPS, with gross margin climbing to 49% (up roughly 30 percentage points sequentially). DRAM ASP also rose by double digits QoQ. The company plans to significantly expand 2026 capex year over year, pushing 1C/1D node transitions and new fab equipment installation (targeting early 2027). DDR5 still accounts for only about 10% of total shipments, but next-generation DDR5 product development is underway.

According to industry reports, China’s CXMT has been moving to reduce its DDR4 production mix, a trend that is further strengthening Nanya’s position in the DDR4 market.

The market’s negative reaction makes sense. $1B exceeds last quarter’s adjusted free cash flow of $843M. Cash on hand was $1.54B, so there’s still $500M+ left after the investment, but it raises questions about short-term financial flexibility. The “why is a pure NAND company putting $1B into a DRAM company?” reaction is understandable. A 3.9% minority stake doesn’t give you much of a voice in management either.

But from an engineering standpoint, this deal has logic. Three angles.

First, DRAM cache is critical for high-performance enterprise SSDs. Enterprise SSD controllers keep the FTL (Flash Translation Layer) mapping table in DRAM to deliver random read/write performance. If DRAM supply is constrained, it creates a bottleneck for high-performance eSSD production. In the current environment of structurally tight DDR4 supply, locking in multi-year DRAM sourcing is a practical move to protect eSSD production continuity.

Second, and this is a strategic interpretation rather than confirmed fact, it could be groundwork for the HBF era. HBF stacks NAND dies like HBM and places them on an interposer next to the GPU. In this architecture, the ability to work with both DRAM and NAND becomes important. SanDisk securing a DRAM supply relationship through Nanya could serve as the foundation for a DRAM-side partnership when HBF reaches commercialization. SanDisk hasn’t officially drawn this connection, so treat this as a hypothesis for now.

Third, the valuation angle. They bought Nanya shares at a 15% discount to the 30-day average. Nanya’s earnings are surging, and DDR4 supply shortages are expected to persist through at least the first half of 2026. There’s a 3-year lock-up, but Nanya’s enterprise value is likely to be higher in three years than it is today.

Let me also flag the risks. A 3.9% minority stake means limited strategic control. There’s Taiwan geopolitical exposure. If DRAM prices roll over, the value of the Nanya stake drops. The 3-year lock-up means zero liquidity.

If it doesn’t, the “why did they spend $1B?” discomfort lingers.

All three of these (lock-up, profit-taking, Nanya) are predominantly short-term supply dynamics. Lock-up and profit-taking are temporary selling pressure. Nanya is a strategically rational deal that suffered from poor market communication. Based on currently available information, none of these events have directly impaired SanDisk’s NAND business fundamentals.

Subscribe now

Half Real: Micron Capex and Supply Concerns

Micron’s aggressive capex announcement deserves more serious consideration. The market’s worry goes like this: “Micron pours money into NAND, supply increases, prices drop, SanDisk margins collapse.”

But the narrative the market is running with and the message Micron itself delivered point in opposite directions. On the same earnings call, Micron said that “data center NAND demand exceeds supply for the foreseeable future” and projected industry NAND bit shipment growth of roughly 20% for 2026. They also noted that initial output from their new Singapore NAND fab won’t come until the second half of 2028. In other words, Micron itself is saying “even with our capex increase, supply tightness persists for the time being,” while the market looked at the capex number in isolation and panicked about oversupply.

Let’s also look at this from a technical standpoint. There are only two ways to actually increase NAND bit supply. Build a new fab, or stack more layers in existing fabs to increase bit density per die. The first takes 12 to 18 months. The second requires a process node migration, and during the transition, yield stabilization and validation can actually cause bit output to temporarily decline.

Looking at the composition of Micron’s NAND capex increase (+63% YoY), per Micron’s own disclosures, the bulk of the increase goes toward cleanroom and equipment spending for the Tongluo fab and new US fabs. This is long-term infrastructure investment, not something that translates into immediate bit output growth. The US ID1 fab won’t produce meaningful output before 2027, and the Singapore fab won’t before the second half of 2028.

The current situation across manufacturers: Samsung and SK hynix are either reducing or freezing NAND investment while concentrating capital on HBM. According to industry research (TrendForce and others), 2026 NAND capex growth is projected in the low single digits, significantly below DRAM capex growth. NAND is being intentionally deprioritized.

The state of the layer stacking race also matters. Based on each company’s public disclosures: SK hynix is in mass production at 321-layer QLC, Samsung is producing V9 (290-layer class) TLC, Micron is shipping 276-layer G9 QLC. SanDisk/Kioxia are in production with BiCS 8 (218 to 232 layers), with BiCS 9 (300-layer class) targeted for 2026 and BiCS 10 (332 layers) originally scheduled for the second half of 2027, though there are reports (Nikkei, unofficial) that AI demand is pushing the timeline forward.

It’s true that SanDisk/Kioxia trail SK hynix by roughly a year on layer count. That means a cost-per-bit disadvantage and potentially weaker positioning in eSSD price competition. But flip it around: the 218-layer to 300+ layer transition gives them significant room to increase bit density per die. When this technology transition executes, the same fabs produce more bits, cost per bit improves, and margins expand. SK hynix at 321 layers is approaching the limits of how much additional efficiency it can extract, while SanDisk/Kioxia still have steps left to climb.

This is where CBA (CMOS directly Bonded to Array) matters. In simple terms, it’s a technology that fabricates the memory array and logic circuitry on separate wafers, then bonds them together using hybrid bonding. Traditional 3D NAND built both on the same wafer. CBA separates them so each can be optimized independently. The array for high-density stacking, the logic for high-speed interfaces. This architecture enables high-speed interfaces like Toggle DDR 6.0 (4.8 GT/s). YMTC commercialized this first under the name Xtacking, and SanDisk/Kioxia adopted it starting with BiCS 8. It directly enables eSSD performance differentiation and, as I’ll discuss later, serves as a technical prerequisite for HBF.

Bottom line: Micron’s capex increase gave the market a narrative for supply concerns, but Micron itself says demand exceeds supply, and new fab output won’t arrive until 2027 at the earliest. This doesn’t change the 2026 supply-demand picture.

Subscribe now

The Real Question: Is Google TurboQuant a Threat?

Read more

I had the chance to attend a seminar by SK Hynix’s North America regional president at a Silicon Valley Korean semiconductor meetup today. The topic was memory in the AI era. It was only about 20 minutes, but it was packed with insight on the structural shifts reshaping the memory market, the technical roadmap for HBM, and things investors genuinely need to understand.

This article pulls together what I heard at the seminar, a brief 1-on-1 conversation with him afterward, and my own read on where the memory market is heading.

Still think we’re in a memory super cycle?

You’re wrong.

More precisely: memory is no longer a cyclical industry.

Table of Contents

1. This Is Not a Memory Super Cycle

2. The Three Requirements to Buy HBM

3. cHBM: Memory Goes Foundry

4. What It Takes to Build a 1GW Data Center

5. Is Hybrid Bonding Mandatory for HBM?

6. 3D DRAM and the China vs. Korea Dynamic

7. PIM’s Limits and Google

8. SK Hynix ADR, and Micron

   ---

Disclaimer

This article is based on a seminar given by SK Hynix’s North America regional president at the Silicon Valley K ASIC Seminar on March 24, 2026, and reflects my own views. It is not a recommendation to buy or sell any stock. Given that the underlying perspective comes from someone inside the memory industry, the content may lean bullish on memory. All investment decisions and their outcomes are your own responsibility.

“This Is Not a Memory Super Cycle”

The phrase “super cycle” gets thrown around a lot when people describe today’s memory market. Demand is strong, prices are up, and sentiment around HBM is running high, so the label sticks naturally. But what I heard at this seminar was a direct challenge to that framing. Super cycle is a term rooted in how people used to look at memory, and the argument is that we’ve moved into territory that framework simply can’t explain.

The old memory industry ran on textbook cyclicality. A slight supply surplus would crater prices. Prices fell, companies eventually cut investment. Supply tightened, prices bounced back. Prices rose, everyone rushed to add capacity again. This loop, supply glut, price collapse, investment pullback, supply shortage, price recovery, repeat, played out on a multi-year cadence. Memory earned its reputation as the quintessential cyclical industry.

But the three major players don’t move like that anymore. Strong demand doesn’t automatically trigger aggressive capacity expansion the way it once did. The reason is simple: they’ve all paid for it. They went big during the up-cycle, got hit with supply gluts and price collapses, and bled for it. That experience left a mark. Today, the priority has shifted from chasing market share through volume to profitability, capital efficiency, and customer mix.

This tendency gets even stronger with products like HBM, where supply is inherently constrained and customer requirements are complex. Memory makers can’t just route limited production capacity wherever demand appears. Deciding which customer gets allocated capacity, and for how long, is directly tied to margin. Long-term supply agreements, in this context, aren’t just about locking in volume. They’re a signal that memory companies are managing supply far more strategically than before. It’s no longer an industry where anyone can just make more and sell more when prices go up.

This is also where long-term contracts take on a different meaning. In the old memory market, spot prices and short-term deals drove the cycle. Now, with AI infrastructure buildout and surging demand for high-value memory like HBM, securing stable supply has become critical on the customer side. And for memory companies, with massive upfront investment, process transitions, and packaging resource allocation all in play, demand visibility matters enormously. Both sides benefit. Long-term contracts are becoming a mechanism for reducing uncertainty and stabilizing supply chains, not just a sales vehicle.

These structural shifts, stacking on top of each other, are pulling memory further away from its classic cyclical pattern. That doesn’t mean price volatility disappears. Memory is still subject to supply and demand dynamics and will continue to fluctuate with market conditions. But the character of that volatility is different now. The old dynamic, where a good upcycle would trigger industry-wide overinvestment and eventually collapse everything, is becoming far less likely.

The regional president’s view was clear. Memory isn’t in the middle of a super cycle that pops and resets. It’s transforming into an industry that can command structurally higher value on a sustained basis. In his words, it’s moving closer to a structural growth industry than a commodity business that just rides the macro. He said it half-jokingly, but he floated the idea that returns from memory investments could potentially outpace index returns going forward.

The key question in today’s memory market isn’t where this cycle peaks. It’s that the nature of the industry itself has already changed. Viewing it through the lens of past cycles means missing what’s actually happening. Memory is no longer an industry you analyze by the cycle. It’s one you read by understanding structural change.

Subscribe now

The Three Requirements to Buy HBM

HBM today is not something you can just buy because you want it. According to what I heard, if you want to purchase HBM from SK Hynix, you need all three of the following at the same time.

First, access to TSMC’s advanced packaging capacity.

Second, the data center power to actually run those chips at scale.

Third, the capital to fund all of it.

Each of these is hard on its own. The real problem is that you need all three simultaneously. Securing HBM is not the finish line. You still have to build the GPU or AI accelerator, package it, integrate it into servers and racks, and run it in a large-scale data center. HBM is not a standalone memory market. It’s a market where packaging, systems, infrastructure, and capital are all bound together.

That’s why the realistic pool of HBM buyers has effectively narrowed to hyperscalers and players at that tier. NVIDIA, Microsoft, Google, Amazon, Meta. Not just because their AI demand is large, but because they can actually claim TSMC packaging resources, absorb multi-billion dollar infrastructure investments, and secure power capacity that could eventually reach the GW scale. HBM is a technology market, yes, but it’s equally a market of supply chain control, infrastructure mobilization, and financial endurance.

This also explains why NVIDIA holds the position it does. NVIDIA’s dominance isn’t just about GPU performance. NVIDIA is the most concentrated source of demand, the most powerful ecosystem operator, and the most certain destination for HBM. From a memory company’s perspective, NVIDIA isn’t just a buyer of chips. It’s the player that can convert HBM allocation into system revenue fastest, at the largest scale, with the highest degree of certainty. In other words, the customer that generates the most impact per unit of supply.

AI chip startups, by contrast, are structurally disadvantaged. No matter how good the chip design, insufficient HBM access makes it nearly impossible to build a competitive system. Even after taping out, lack of TSMC packaging capacity delays product shipment. On top of that, there’s large-scale customer qualification and real-world data center power constraints to clear.

The implication is significant. In the AI semiconductor market, “good technology equals good investment” no longer holds as a simple formula. Technology is a necessary condition, not a sufficient one. What matters far more is who can secure stable HBM supply, who can claim packaging capacity, and who can mobilize power and capital to convert it into actual revenue. This is what HBM absolute supremacy looks like in practice.

Subscribe now

cHBM: Memory Goes Foundry

Anyone who’s read my previous articles already knows this, but memory companies including SK Hynix are engaged in active cHBM co-design discussions with hyperscalers. The reason is straightforward. In AI systems, the bottleneck is no longer compute performance alone. Data movement, power efficiency, packaging area, thermal management, and system-level optimization are all becoming critical. Producing large volumes of good memory is no longer enough. The era of deeply customizing memory to fit what each customer’s system architecture actually requires has arrived.

This is where the character of the memory industry begins to shift.

In standard product-driven memory, cost reduction and yield competition were the core game. Who could make more, cheaper, more reliably was what mattered. cHBM changes the equation. The base die takes on more logic. Custom design work increases. Packaging complexity rises. Validation scope expands. That naturally inflates development costs and NRE-type expenses, and raises the share of co-design work between customer and supplier. Memory moves away from pure manufacturing logic and starts incorporating design services, advanced packaging, and system optimization as core parts of the business.

This is why I don’t read cHBM as simply “HBM getting more sophisticated.”

It’s a signal that the memory industry is moving from commodity to solution business. Beyond that, it means memory companies are beginning to behave, in certain domains, like foundries.

Subscribe now

What It Takes to Build a 1GW Data Center

People occasionally push back with something like this: “Fine, HBM I get. But DRAM is still a commodity, and structurally memory is still a cyclical industry, isn’t it?”

It was, historically. DRAM was the textbook commodity. Supply up, prices collapse. Prices collapse, capex contracts. Supply tightens, prices recover. Prices recover, aggressive capacity expansion begins again. That’s why many people still believe that even if HBM is an exception, the broader memory market including DRAM will eventually revert to its old cyclical pattern.

Let me ask a question.

How many DRAM wafers do you think it takes to build a single 1GW data center?

According to what I heard at the seminar today, roughly 350,000 wafers.

Let that sink in.

What makes this number even more striking is that those 350,000 wafers are only the DRAM portion. Data centers don’t run on DRAM alone. You need GPUs, HBM, packaging, networking, SSDs, power infrastructure, and cooling systems, all at once. Building a 1GW data center isn’t a matter of adding a few more servers. It means simultaneously moving the entire semiconductor supply chain and the entire power infrastructure.

That’s the critical point.

The memory cycles of the past moved within the relatively predictable boundaries of IT demand driven by PCs, smartphones, and servers. Supply fluctuated against that demand, and prices swung accordingly. AI data center buildout is a different animal entirely. The absolute scale of demand is enormous, and it’s not a one-off event. It’s tied to the long-term infrastructure investment plans of the largest companies in the world. With Microsoft, Google, Amazon, and Meta all talking about GW-scale data centers, memory demand can no longer be reasonably framed as something that dips and recovers on the old pattern.

That doesn’t mean DRAM price volatility goes away. DRAM still carries commodity characteristics and will respond to market conditions. But the context driving that volatility is fundamentally different now. The magnitude of demand, the identity of the buyers, the way supply is being managed, and the structure of long-term contracts have all changed. Memory will still be volatile, but it is absolutely not a cyclical industry in the way that term used to mean.

Subscribe now

Read more

Substack is currently in a trial phase, so all premium quality articles are available for free for now. Once paid subscriptions are introduced, pricing will be set at a level comparable to professional tech writers on Substack. Early supporters will have the benefit of locking in their current pledge price for future paid access.

Subscribe now

Why Samsung Could Flip the Script on HBM4

Samsung’s HBM4 paper at ISSCC 2026 isn’t just a technical announcement. It’s a blueprint for how the company plans to claw back ground it lost to SK Hynix in the HBM market. In this article, I’ll walk through the key technologies one by one, in a way that’s accessible even if you’re not already steeped in semiconductors or HBM.

Before we dive in, let me quickly cover what HBM actually is. If you already know, feel free to skip ahead.

HBM (High Bandwidth Memory) is memory built by stacking multiple DRAM chips on top of each other. If a regular DRAM chip is a single-story house, HBM is a high-rise apartment building. The layers are connected by microscopic vertical channels called TSVs (Through-Silicon Vias) — think of them as the elevators between floors.

The reason we stack chips rather than spread them out is simple: as AI models have grown, the demand for moving massive amounts of data quickly has exploded, and going vertical is far more efficient than going horizontal.

Looking at how HBM has evolved, from HBM2 in 2018 to HBM4 in 2026, bandwidth has grown 10.8x and capacity 4.5x in just eight years. That tells you just how fast AI’s memory appetite is scaling.

Samsung’s HBM4 specs break down as follows: a 12-layer stack, 36GB capacity, 2,048 I/O pins, 3.3TB/s bandwidth (a 260% improvement over HBM3E), and a maximum speed of 13Gb/s per pin. In short — capacity and speed have both taken a massive leap.

One thing worth flagging: the official JEDEC standard for HBM4 (JESD270-4) specifies a maximum data rate of 6.4Gbps per pin, for a total bandwidth ceiling of 2TB/s. Samsung hit 13Gbps in actual testing — more than 2.0x the spec. Even accounting for JEDEC’s deliberately conservative baseline, that tells you just how aggressively Samsung is pushing performance beyond the standard.

If you would like to understand HBM manufacturing across the design, process, test, and packaging stages, please refer to the article below.

Key #1: Top and Bottom Both Changed — 4nm Logic Base Die + 1c DRAM Core Die

The most fundamental shift in Samsung’s HBM4, as shown in this paper, is a simultaneous generational upgrade to both types of chips that make up an HBM stack.

HBM consists of two distinct chip types: multiple core dies (which actually store data) stacked on top of a single base die (which handles the interface with the outside world and serves as the gateway to the GPU).

Start with the base die. Up through HBM3E, even the base die was made on DRAM process technology. DRAM processes are optimized for building memory cells, which makes them relatively limited for implementing complex logic circuits — transistors are larger, and the number of available metal routing layers is constrained. By moving the base die to Samsung Foundry’s 4nm FinFET logic process, transistors shrank dramatically, meaning more circuitry fits in the same area, switching speeds improved, power consumption dropped (VDDQ fell 32%, from 1.1V to 0.75V), and sophisticated logic features like PMBIST and automatic calibration became feasible.

But changing just the base die isn’t enough. That’s where the core die generation jump becomes decisive.

HBM3E’s core dies used a 4th-generation 10nm DRAM process, roughly the “1a” node. HBM4 jumps two generations to the 6th-generation 10nm process, “1c” — skipping 1b entirely. Moving to 1c significantly shrinks the memory cell area, opening up considerably more free space within the same die footprint. Why that free space matters is what the next section is about.

The upshot: Samsung simultaneously moved the base die to 4nm logic and the core die to 1c DRAM, improving area efficiency on both ends. This dual upgrade is what makes everything else in HBM4 possible.

There’s a strategic angle worth calling out here. Samsung owns its own foundry, and 4nm logic is a Samsung Foundry process. Switching the base die to logic is the kind of move that only makes sense if you have memory and foundry capabilities under the same roof. SK Hynix would need to source that logic process externally (e.g., from TSMC). TSMC’s process quality may well be superior, but in terms of vertical integration flexibility and supply security, Samsung’s structure has a real structural edge here.

Key #2: 4x More TSVs and Tighter Microbump Pitch

As explained above, Samsung created area headroom in both the core die (via 1c) and the base die (via 4nm logic). The most direct beneficiary of that headroom is the TSV count.

TSVs are the tiny vertical channels that connect the stacked dies and carry data between layers. HBM4 has 4x as many TSVs as HBM3E — channel count doubled (from 16 to 32 channels), and DQ TSVs per channel also doubled. The shrinking cell area of 1c DRAM freed up space to land more TSV pads on the core die, while the 4nm base die provided the matching receiver density below. The paper explicitly credits both sides, describing the 2x DQ TSV increase as a joint product of the 1c core die above and the 4nm base die below.

To explain the relationship between DQ and TSVs: DQ stands for Data Queue — these are the I/O pins through which actual data flows between memory and the outside world. In HBM, DQ signals travel vertically between layers via TSVs. More DQ TSVs means more data can move simultaneously.

In HBM3E, a single DQ TSV handled both BL0 and BL4 data using time-multiplexing — one TSV alternating between two bit-line signals, like squeezing two passengers into a single elevator car by making them take turns. In HBM4, the two TSVs are split so DQ_TSV0 is dedicated to BL0 and DQ_TSV1 handles BL4. This gives each TSV a 2tck (two clock cycle) data window, versus 1tck in HBM3E. A wider data window means the receiving end has more time to cleanly capture valid data, which is what enables stable high-speed operation.

On top of this, ABB (Adaptive Body Bias) technology is applied across the 12 stacked core dies. No two dies are identical — semiconductor manufacturing always introduces subtle variation, and some dies run fast while others run slow. ABB addresses this by adjusting the voltage applied to the body of each die’s transistors, which effectively tunes their threshold voltage (the voltage at which transistors switch on). A higher threshold voltage means slower but leakier transistors; a lower one means faster but with more leakage. By nudging slower dies down and faster dies up, ABB narrows the performance spread across the stack. The paper shows that before ABB, the delay distribution across core dies is wide with notable outliers on both ends; after ABB, the spread tightens significantly.

Key #3: tCCDR Auto Calibration — Solving HBM’s Hidden Performance Bottleneck

The most technically impressive part of the paper is the per-channel TSV RDQS timing auto-calibration. The terminology is dense, but the underlying concept is intuitive. Let me walk through it.

First, RDQS. RDQS stands for Read Data Strobe — it’s the timing reference signal that memory sends alongside data when reading out to the GPU. Without a timing reference, the receiver wouldn’t know when exactly to sample the incoming data. RDQS acts like a doorbell: “Open the door and receive the package right now.” The data is the package; RDQS is the ring of the bell.

Now, tCCDR. HBM consists of multiple stacked core dies, each with a unique identifier called a SID (Stack ID). When a GPU switches from reading SID0 to reading SID1, there’s a minimum gap required between the two commands. That minimum gap is tCCDR (Column-to-Column Delay for different Ranks) — basically, the minimum wait time required when switching between die layers.

Why does tCCDR matter? AI workloads have the GPU constantly switching between dies at high frequency. A large tCCDR means every die switch incurs a wait, dragging down real-world system performance no matter how fast the per-pin rate is. It’s rarely highlighted in spec sheets, but it’s a parameter that directly affects AI system performance.

The problem in HBM4 — with 32 channels and signals traveling through 12 die layers before reaching the base die — is that tiny timing differences between channels are unavoidable. Some channels have RDQS arriving slightly early, others slightly late. TSV path lengths vary, manufacturing tolerances vary, and temperature and voltage add further drift. From the receiver’s perspective, this is a problem: some channels are “doorbell rang, package safely delivered,” while others are “doorbell still ringing, package still in transit.” If the receiver keeps pushing speed under these conditions, the RDQS-to-data alignment slips in some channels, the valid data window narrows, and errors start appearing.

Samsung’s solution works in three stages.

In stage 1, after power-on, the Ready signal from each SID triggers the calibration process.

In stage 2, a reference signal passes through a Replica Tree — a circuit that identically mirrors the actual RDQS signal path, so any timing variation in real operation is accurately reproduced. The slowest SID’s RDQS is selected as the reference, and all other channels synchronize to it.

In stage 3, a TDC (Time-to-Digital Converter) translates each channel’s timing offset into a digital code, which passes through an Offset Controller and Encoder and is applied to each channel’s DCDL (Digitally Controlled Delay Line).

Faster channels get extra delay added; slower channels get less — aligning all channels’ RDQS arrival times.

If that went over your head, here’s a more intuitive version.

Imagine 32 people on a stage, all supposed to clap on the same beat. But everyone’s reaction time is different — one person is consistently 0.2 seconds late, another 0.1 seconds late, another 0.1 seconds early. Without calibration, the conductor has to set the tempo to accommodate the slowest person. Raise the tempo even a little, and the late clappers miss the beat, and instead of a single crisp “clap,” the audience hears a drawn-out scatter of sound. The issue isn’t just one slow person — everyone is off by varying amounts, and the faster the tempo, the more those small gaps become audible.

Now with calibration: before the performance starts, the conductor has everyone do a test clap, measures how early or late each person is, and then tells the faster people to wait — “you, wait 0.05 seconds; you, wait 0.12 seconds.” The idea isn’t to force the slow people faster; it’s to hold the fast people back just enough so everyone arrives at the same moment. Once aligned, the conductor can push the tempo without the claps falling apart, because the offsets have already been corrected.

That’s exactly what Samsung’s calibration does for HBM. The paper’s results are striking: without calibration, the maximum speed at tCCDR = 2nCK was 7.8Gb/s. With calibration applied, it rose to 9.4Gb/s — roughly a 20% improvement. This isn’t just a higher pin speed on paper; it’s a tangible improvement in the performance that an actual AI system experiences.

Key #4: WDQS 4-Phase Skew Monitoring — Catching Defects at the Wafer, Not the Package

While RDQS is the timing reference for reads, WDQS works in the opposite direction. WDQS (Write Data Strobe) is the timing reference the GPU sends alongside data when writing to memory — essentially the GPU telling the memory, “receive and store this data now.”

HBM4 uses a quarter-rate 4-Phase WDQS scheme. This means a single clock cycle is divided into four phases — I, IB, Q, QB — enabling four data transfers per clock cycle and effectively quadrupling throughput. But for this to work, the four phases have to be precisely evenly spaced. If the gap between I and IB is narrower than the gap between Q and QB, there isn’t enough time to capture data in the narrow window. This phase imbalance is called skew. At 13Gb/s, each phase window is only about 19ps (picoseconds, or trillionths of a second), so even a few ps of skew is catastrophic.

Traditionally, skew issues were only caught at the SiP (System-in-Package) final test stage — after HBM and GPU have already been assembled into a complete package. Finding a defect there means scrapping the entire assembly, GPU included. On top of that, external test equipment has bandwidth limitations that make it hard to accurately measure high-speed signals, and probing thousands of bumps adds further complexity.

Samsung’s approach is fundamentally different. They use an internal self-loop method that requires no external contact at all. A clock signal generated from a ring oscillator or PLL (Phase-Locked Loop) inside the base die is injected into the WDQS path, flows through the actual WDQS distribution network, and an internal skew monitor measures how far off each phase is. Based on those measurements, skew is automatically corrected per DWORD (32-bit data unit). The correction first aligns the WDQS_T/C (True/Complement differential pair) duty cycle to 50%, then divides the signal by 2 to generate the four phases, and finally equalizes the phase intervals to within a few picoseconds.

The real value here is that the screening point moves to the base die wafer stage. Trim values are determined for each of the 64 DWORDs and recorded via an IEEE 1500 interface. Dies where the trim saturates — meaning the skew is beyond the correctable range — are discarded at the wafer level, before any stacking even begins. That means bad base dies get filtered out before CoW (Chip-on-Wafer) or SiP packaging, avoiding expensive downstream waste. Good dies have their trim values stored and simply loaded during packaging. On-die trimming can also be re-run in the field if operating conditions change.

In actual measured results, applying wafer-level trim to a CoW proxy package running at 10Gb/s reduced RDQS deterministic jitter (DJ) from 10.6ps to 4.5ps — more than halved. DJ measures how far a signal’s timing deviates from ideal, and a lower value means a cleaner, more open data eye, enabling stable high-speed operation.

Key #5: PMBIST — A Testing Revolution Made Possible by Logic Process

HBM is a complex system with thousands of TSVs and dozens of channels, which makes testing it extremely challenging. Fail to catch defective dies early, and you don’t find out until after expensive back-end processes like CoW and SiP packaging have already been completed.

MBIST (Memory Built-In Self-Test) is a self-test circuit embedded directly inside the memory chip — no external equipment needed; the chip tests itself and reports pass or fail. In HBM3E, MBIST was limited to a fixed set of test patterns due to the area constraints of DRAM process technology. Think of it as a multiple-choice exam locked to five fixed questions: any defect that doesn’t happen to trigger those five patterns goes undetected.

HBM4’s PMBIST (Programmable MBIST) is a different beast entirely. Leveraging the area freed up by the 4nm logic process, the test program storage space is roughly 225x larger than before. That fixed five-question multiple-choice test has become a free-form question bank where you can design essays and practical exams. Better yet, test engineers can write test scenarios directly, using an SDK, just like writing code — meaning new defect patterns discovered in the field can be immediately incorporated into future testing. It’s effectively a small dedicated test processor living inside the base die.

The most meaningful practical difference is flexibility in test ordering. AI workloads have the GPU rapidly switching between dies, and that switching speed (tCCDR) directly affects real-world AI performance. In HBM3E, the fixed test ordering made it structurally impossible to test this switching scenario — there was no way to construct a pattern with rapid cross-die switching. With HBM4 PMBIST, test sequences are freely programmable. You can reproduce an actual AI workload pattern — die 0 → die 1 → die 2, switching rapidly — and test it directly. It’s like an entirely new subject being added to the curriculum.

From a DFT engineer’s perspective, this is a fundamental paradigm shift in test methodology. Defects that previously couldn’t be caught because the test repertoire was too limited can now be systematically targeted with purpose-built scenarios.

Key #6: Bank Architecture Change — Quiet but Meaningful

One more change that tends to get overlooked in generation-to-generation spec comparisons: the bank structure shifted from 4BG × 4BA to 2BG × 8BA.

BG (Bank Group) and BA (Bank) are the organizational units inside memory. Think of memory as a large library: bank groups are the floors, and banks are the reading rooms on each floor. Back-to-back accesses to the same room incur a wait, but switching to a different room allows immediate access. Switching between different floors (bank groups) is even faster.

Going from 4BG × 4BA to 2BG × 8BA means the number of bank groups halved (from 4 to 2), but the number of banks within each group doubled (from 4 to 8). The total number of banks (16) is unchanged. The benefit is that within any given bank group, there are now more rooms to pick from. AI workloads tend to involve rapid access across diverse addresses in short time windows, and having more banks per group raises the probability of being able to access sequentially without hitting a bank conflict.

Summary: The Structural Reasons Samsung Could Overtake SK hynix

Stepping back from the technical details, there’s a clear structural logic to Samsung’s HBM4 strategy.

First, this is the first real realization of foundry-memory vertical integration. HBM4 is the first Samsung product to seriously leverage the fact that the company has both foundry (4nm logic) and memory (6th-generation 10nm DRAM) under one roof. The base die logic process transition is a move that would be difficult to execute without that vertical integration — and the advanced capabilities like PMBIST and auto-calibration only become possible because of it.

Second, the aggressive DRAM process generation jump. Skipping 1b and going directly to 1c created the physical preconditions for 2x more DQ TSVs per channel — which, combined with the tighter microbump pitch, enabled HBM4’s expanded channel and I/O configuration. Without this decision, no amount of base die sophistication would have been enough to hit HBM4’s target specs.

Third, fundamental improvements in test and yield. PMBIST (225x more program space, RISC architecture, flexible looping), on-die WDQS skew correction (early screening at wafer level), and tCCDR auto-calibration — together, these create a virtuous cycle of earlier defect detection, higher yields, and optimized real-world system performance. This is Samsung’s structural attempt to reverse the yield advantage SK Hynix built up over HBM3E.

Fourth, a focus on system-level performance. The tCCDR auto-calibration improves not the per-pin speed on a spec sheet, but the performance the GPU actually experiences. Achieving 13Gbps — over 1.6x the JEDEC spec of 8Gbps — while simultaneously optimizing die-switching performance (tCCDR) is a direct shot at NVIDIA’s pain points as a customer.

Open Questions

A single paper doesn’t decide the outcome of a competition. Several critical questions remain.

What is the actual yield of Samsung Foundry’s 4nm process? Is the 1c DRAM process stable in volume production? Can the paper’s performance numbers be reproduced in mass manufacturing? And SK Hynix will almost certainly pursue its own logic base die strategy through its partnership with TSMC — how does that competitive dynamic play out?

But what this paper makes clear is this: Samsung is not simply trying to catch up with HBM4. It’s competing through a differentiated approach built on its own structural strengths. Round two of the HBM race has started.

Appendix: HBM3E vs HBM4 Key Spec Comparison

Process: HBM3E — core and base die both on 4th-gen 10nm DRAM. HBM4 — core die on 6th-gen 10nm DRAM (1c), base die on 4nm FinFET.

JEDEC official data rate: HBM3E 8.0Gbps → HBM4 6.4Gbps

Samsung measured max speed: HBM3E 10.0Gb/s → HBM4 13.0Gb/s.

Channel configuration: HBM3E 16 channels × 2PC × 32 I/O → HBM4 32 channels × 2PC × 32 I/O.

Bank structure: HBM3E 4BG × 4BA → HBM4 2BG × 8BA.

Bandwidth: HBM3E 1.3TB/s per cube → HBM4 3.3TB/s per cube.

Max capacity: HBM3E 24Gb × 16-High = 48GB → HBM4 24Gb × 12-High = 36GB.

Supply voltage (VDDC/VDDQ): HBM3E 1.1V/1.1V → HBM4 1.05V/0.75V.

Microbump pitch: HBM3E 96μm × 110μm → HBM4 70μm × 110μm.

Test approach: HBM3E MBIST (fixed patterns, 2,272 bits) → HBM4 PMBIST (programmable, 512Kbits).

Chip size: HBM3E 11mm × 11mm → HBM4 12.8mm × 11mm.

Introduction

As we enter the AI era, the importance of memory, led by HBM (High Bandwidth Memory), has never been greater. AI semiconductors from NVIDIA, AMD, and others cannot achieve their full performance without HBM. So how exactly do we test this memory?

My doctoral research lab focused on Design for Test (DFT) for semiconductor testing and was one of the world’s most renowned research groups in memory testing and repair. Today, I’ll explain the fundamentals of memory testing methodology from my doctoral research and industry experience in a way that even non-specialists can understand.

Substack is currently in a trial phase, so all premium quality articles are available for free for now. Once paid subscriptions are introduced, pricing will be set at a level comparable to professional tech writers on Substack. Early supporters will have the benefit of locking in their current pledge price for future paid access.

Subscribe now

1. What is Memory Testing?

Basic Principle: Write and Read

The most fundamental principle of memory testing is simple. Write a value to a specific address, read it back, and verify that the read value matches what was written. If the written and read values differ, the bit cell is determined to have a fault.

For example, if you write ‘1’ to address 0x1000 and read back ‘0’, that cell has a Stuck-at-0 fault. Conversely, if you write ‘0’ but read ‘1’, it’s a Stuck-at-1 fault.

Memory Test Algorithms

However, simply writing 0s and 1s to all cells and reading them back cannot detect all faults. Systematic algorithms are needed to detect interactions between memory cells, known as Coupling Faults.

March algorithms are the representative family of memory test algorithms. Various variants exist such as March C-, March LR, and March SS, each capable of detecting different fault types. The basic structure of March algorithms is as follows:

• ⇑ (Ascending): Perform operations while incrementing addresses from 0 to the end

• ⇓ (Descending): Perform operations while decrementing addresses from end to 0

• ⇕ (Either): Perform operations in either direction

For example, the March C- algorithm consists of 6 elements: {⇕(w0); ⇑(r0,w1); ⇑(r1,w0); ⇓(r0,w1); ⇓(r1,w0); ⇕(r0)}, capable of detecting Stuck-at Faults, Transition Faults, and some Coupling Faults.

Process-Specific Test Algorithm Implementation

Beyond standard March algorithms, specialized tests exist that consider memory characteristics. A representative example is VDR (Voltage Drop Retention) testing. This method tests retention characteristics while intentionally lowering the operating voltage, filtering out weak cells with insufficient voltage margins. Marginal defects that pass at normal voltage but fail at lower voltage are often the culprits behind intermittent field failures.

In actual production environments, defects often concentrate at specific locations or conditions depending on process characteristics, leading to the development of process-tailored test patterns. Ultimately, how deeply each company understands its process’s defect characteristics and possesses optimized custom algorithms determines its yield competitiveness.

2. Why is MBIST Necessary?

Memory Built-In Self-Test (MBIST) embeds memory test circuitry inside the chip, enabling self-testing of memory with minimal ATE usage. So why do we design MBIST instead of directly testing memory with ATE?

First, speed. When testing memory through ATE, the interface speed between the ATE and chip becomes a bottleneck. Modern SoC embedded SRAM operates at GHz speeds, which external testers cannot match. MBIST enables At-Speed testing inside the chip.

Second, accessibility. Modern SoCs contain hundreds of memory blocks. Directly accessing all these memories through external pins is physically impossible. MBIST can access all memories through internal test structures such as IJTAG or IEEE 1500.

Third, cost. Reducing usage time of expensive ATE equipment lowers test costs. MBIST can test multiple memories simultaneously in parallel, also shortening test time.

3. What is Memory Repair?

Unlike logic, memory can improve yield through repair. Memory repair is a technology that replaces defective rows or columns with spare rows/columns. It’s similar to installing backup elevators in a building and switching to backup when the main elevator breaks down.

Redundancy Analysis (RA)

RA is the algorithm that determines which spare row/column should replace which defective row/column. Rather than simply replacing with spares whenever a defect is found, optimization is needed to cover as many defects as possible with limited spare resources. For example, if multiple defects exist in the same column, column repair may be more efficient than row repair.

BIRA and BISR

BIRA (Built-In Redundancy Analysis) is an on-chip algorithm that analyzes defect information to find optimal repair solutions. It collects defect addresses found during testing and determines spare row/column allocation.

BISR (Built-In Self-Repair) is the stage that actually applies the repair information determined by BIRA. Typically, fuses are programmed so that accessing a defective address automatically redirects to spare rows/columns. Recently, eFuse or OTP (One-Time Programmable) memory is also used.

4. Difference Between ECC and Memory Repair

ECC (Error Correction Code) and memory repair both address memory defects, but differ in application timing and purpose.

Memory repair physically replaces defects occurring during manufacturing. Once repaired, it operates like a normal cell with no runtime overhead.

ECC detects and corrects transient errors (Soft Errors) occurring at runtime. It addresses temporary phenomena like bit flips caused by cosmic rays or alpha particles. SECDED (Single Error Correction, Double Error Detection) is representative, correcting 1-bit errors and detecting 2-bit errors.

In practice, HBM and GDDR6 use both technologies together. Repair removes Hard Faults during manufacturing, while ECC handles Soft Errors during operation. From HBM3 onward, On-Die ECC became a mandatory specification to address additional defects that can occur in stacked structures.

5. Actual Test Flow: From GO/NO-GO to Bitmap

In actual production environments, memory testing proceeds in stages. Multiple memory blocks are tested simultaneously in parallel through MBIST, with test depth varying by purpose.

GO/NO-GO Testing

The first stage is GO/NO-GO testing. It quickly determines whether memory is basically functional, simply whether it’s alive or dead. Writing 0s and 1s to all bits and reading them confirms only Pass/Fail. This stage can determine whether memory defects occur in the chip, but cannot identify exactly which memory or which memory cell is defective.

Bitmap Testing and Its Limitations

When GO/NO-GO fails, Bitmap testing is performed to precisely identify which bit cells are defective. It records Pass/Fail information individually for every bit position.

Bitmap testing is a powerful diagnostic tool but has a critical drawback. Since defective cell information for each memory must be stored sequentially, parallel testing is impossible, and test time increases exponentially for large-capacity memory. Therefore, it’s used only for Failure Analysis (FA), not production lines.

Bitmap results play a key role in identifying defect causes. If defects concentrate in specific rows or columns, Row/Column Decoder problems can be suspected. If they appear in cluster patterns, process particle contamination is likely. Based on such analysis results, new test algorithms may be developed for yield ramp-up stages. Ultimately, production uses a two-track strategy: quick screening with GO/NO-GO, with separate Bitmap analysis only for problematic chips.

Why Memory Testing Follows Logic Testing

Memory testing is generally performed after logic testing (Scan Test). This isn’t simply a matter of sequence but has technical and strategic reasons.

First, the MBIST controller is part of the logic. Testing memory requires the MBIST controller to function properly, but this controller itself is logic circuitry. If the logic has faults, MBIST won’t operate correctly, making memory test results unreliable.

Second, clarity in test result interpretation. If memory testing fails without knowing whether logic is normal, it’s impossible to distinguish between actual memory defects and MBIST controller faults. Logic must first be confirmed normal through Scan Test before memory test results can be trusted.

Third, yield strategy. Memory can be repaired through spare rows/columns, but logic faults cannot be repaired. Therefore, from a yield improvement perspective, it’s more efficient to first secure unrepairable logic yield, then proceed with memory testing and repair.

6. HBM Testing: Challenges of 3D Memory Structure

HBM is a 3D structure where multiple DRAM dies are vertically stacked and connected via TSV (Through-Silicon Via). This structure is the fundamental reason why improving HBM yield is difficult. Unlike conventional single-die memory, yield multiplies at each stage.

Stage 1: Individual DRAM Die Testing (Known Good Die)

Before stacking, each DRAM die is individually tested to select Known Good Die (KGD). For example, with 12-layer stacked HBM3E, even if individual die yield is 95%, simple calculation gives 0.95 to the 12th power, approximately 54%. If even one defective die is included, the entire HBM becomes defective, so KGD selection accuracy determines final yield.

Two test access approaches exist. The HBM JEDEC specification adopts IEEE 1500 standard-based test wrappers, enabling standardized test access. Additionally, before stacking, each die’s memory can be directly accessed and tested through DA (Direct Access) pins.

Stage 2: Post-Stacking Testing

Once dies are stacked, DA pins on memory dies can no longer be used. Testing is now only possible through the Base Die, with test structures implemented in the Base Die accessing upper dies’ memory through TSV.

The problem is that the stacking process itself creates new defects. TSV connection defects, micro bump bonding defects, and defects from thermal stress during bonding can occur. Even dies selected as KGD can become defective after stacking, making retesting essential. Considering stacking process yield, final yield drops further.

Stage 3: Post-System Integration Testing

After HBM is connected to host chips like GPUs or AI accelerators through interposers, final memory testing is performed at the system level. The entire memory path including interposer connections is tested through the host chip’s memory controller.

Yield loss occurs again at this stage. Interposer bonding defects, HBM-host interface timing issues, and signal integrity problems may be newly detected. Ultimately, final yield equals KGD yield × stacking process yield × system integration yield. This is the structural reason why HBM prices are high.

Conclusion

Memory testing and repair technologies are core technologies determining semiconductor yield. On top of the seemingly simple principle of “write and read,” various technologies are layered: March algorithms, MBIST, BIRA/BISR, and ECC.

As we enter the HBM era, test complexity continues to increase. 3D stacked structures create new defect possibilities at each stage, and test strategies must evolve accordingly. The importance of KGD selection, necessity of post-stacking retesting, and multi-stage test systems extending to system-level integration testing guarantee HBM quality.

As the AI semiconductor market grows, the role of memory test engineers becomes increasingly important. Faster, more accurate, and more efficient test technologies will enable the next generation of AI accelerators.

HBM is everywhere in the conversation right now. Stock forums, news outlets, casual dinner tables — everyone’s heard that “HBM is in short supply” or “NVIDIA can’t build enough GPUs because of HBM.”

But almost nobody explains why it’s actually hard. Most coverage stops at “they stack dies using TSVs” or “the micro bumps have to align perfectly.” That’s not an explanation. That’s a caption.

This piece walks through every stage of HBM production — design, fabrication, test, packaging — and explains what makes each one genuinely difficult. It also covers what happens after the product ships to a customer, because that’s where things get interesting in ways most people don’t expect. Everything here is based on direct experience and publicly available technical information.

Substack is currently in a trial phase, so all premium quality articles are available for free for now. Once paid subscriptions are introduced, pricing will be set at a level comparable to professional tech writers on Substack. Early supporters will have the benefit of locking in their current pledge price for future paid access.

Subscribe now

1. Design: It’s Not Just Stacked DRAM

The bus width problem

A standard DDR5 interface is 64 bits wide. HBM3E runs at 1,024 bits. HBM4 pushes that to 2,048 bits. Wider bus doesn’t just mean more wires — every I/O pin needs its own signal path, and once you account for power and control signals, a single HBM3E or HBM4 stack requires over a thousand connections to the GPU sitting next to it.

You cannot route that over a PCB. The geometry doesn’t work. That’s the fundamental reason silicon interposers exist, and why 2.5D packaging like CoWoS is non-negotiable. We’ll come back to that.

Power distribution through TSVs

One of the most underappreciated design challenges in HBM is the Power Distribution Network. Delivering clean power through TSVs across 12 or 16 stacked dies is genuinely difficult — during high-current events like refresh operations, voltage droops become a serious problem at the upper dies.

How a company arranges its TSV layout to manage this is one of the core proprietary differentiators between memory vendors. It directly determines yield and performance, which is why nobody publishes it.

HBM4’s logic base die

HBM4 changes the rules. Previous generations used DRAM process technology throughout. HBM4’s base die — the bottom layer of the stack — will be fabricated on a foundry-class logic process, with TSMC 12nm and Samsung’s SF-class node cited in industry reports. Confirmed node specifics are still pending official announcements from each company.

More importantly, this base die can carry customer-specific logic. That means NVIDIA’s HBM4 and AMD’s HBM4 will be physically different products. Memory is transitioning from a commodity component to a semi-custom one, and for the memory vendors, the design complexity that comes with that is enormous.

2. Fabrication: Yield Is the Whole Game

TSV formation

A TSV is a vertical hole etched through a silicon wafer and filled with copper. The diameter is just a few micrometers, and the aspect ratio — depth relative to width — is high enough that both the etch and the plating steps are prone to defects.

A 12-layer HBM stack contains millions of TSVs. One bad connection means a dead die. TSV yield is substantially harder to control than conventional DRAM fabrication, which is why HBM vendors build TSV repair schemes into their designs to limit the damage from individual failures.

Wafer thinning

To stack dies, each wafer has to be ground down to an extreme thinness. A 12-layer HBM requires dies at around 50 micrometers. Going to 16 layers means getting down to 30 micrometers — less than half the thickness of a human hair.

At that thickness, wafers crack easily and bow under their own stress. A bowed wafer cannot be bonded with the precision HBM requires. JEDEC package height constraints tighten further with each generation, so the pressure to thin each die only increases as layer counts go up. The 775 micrometer figure cited in some HBM4 discussions reflects draft specifications, not yet a finalized JEDEC standard.

HBM consumes far more wafer capacity than standard DRAM

On a bit-for-bit basis, HBM consumes roughly two to three times the wafer area of conventional DRAM. This is an industry estimate based on die size and process overhead, not a precise published figure — but the directional implication is real. When fabs allocate capacity to HBM, DDR5, LPDDR, and GDDR7 all get squeezed. The current tightness in those markets is a structural consequence of that tradeoff, not bad planning.

3. Test: Hard Before Stacking, Harder After

Wafer-level burn-in

Once DRAM wafers come out of fab, the first step is burn-in — running the dies at elevated temperature and voltage to screen out early-life failures. This is followed by functional testing at both low and high speeds. So far, this is similar to standard DRAM production. The hard part starts next.

KGD: the gate you can’t skip

Standard DRAM fails at the individual package level. One bad die, one scrapped unit. HBM is different. Because you’re stacking 12 dies together, every single die has to be confirmed good before assembly. This is called Known Good Die testing, and the math behind it explains why it matters so much.

Assume individual die yield is 99%. For a single chip, that’s fine. Stack 12 of them and the probability that all 12 are good drops to 88.6%. Drop per-die yield to 97% and a 12-layer stack comes out at 69.4% yield. Every bad die in the stack takes down all the good ones around it. That cost structure is what makes KGD testing worth the investment.

The testing itself isn’t straightforward either. Post-dicing dies are 30 to 50 micrometers thick — fragile enough to crack under the wrong handling. Dedicated die-level handlers are required. Probe card pitches shrink with every generation, and a single setup can cost tens of millions of won. The tradeoff between test coverage and test time has real consequences in both directions: too little coverage and defects make it into the stack, too much and throughput collapses.

Post-stack test

After the dies are stacked and connected through TSVs, an entirely new set of problems appears. Verifying bond alignment and interconnect integrity becomes exponentially harder as layer count increases. Isolating a defect to a specific die, bank, and row in a 12-layer stack — based entirely on external test access — is exactly as difficult as it sounds.

Final package test adds at-speed functional testing on top of that, and many vendors now run both ATE and System Level Test in sequence. The cost of finding a defect after full assembly justifies the redundancy.

Test infrastructure falls behind product cycles

Moving from 8 layers to 12 to 16 increases test time proportionally. But HBM generations turn over faster than conventional DRAM. Test program development, probe card design, and BIST IP updates all have to land in time for product launch. When a customer like NVIDIA revises its interface specifications, the test conditions have to be rebuilt from scratch. That infrastructure burden has pushed test cost to a significant portion of total HBM manufacturing cost.

4. Packaging: The Most Precise Assembly in Semiconductor History

Microbump alignment

Dies are connected layer to layer through microbumps. HBM3E runs at a bump pitch of around 25 micrometers. HBM4 at 16 layers is expected to push that down to 16 to 18 micrometers.

If a bump misaligns, the connection fails. There is no bump-to-bump rerouting redundancy in the HBM PHY signal path — each signal maps to exactly one bump. If that bump fails, the entire channel it serves goes down. HBM3E has eight channels, so a single bump failure doesn’t immediately kill the whole stack, but a dead channel means reduced bandwidth, and in a data center workload context, a GPU running at partial memory bandwidth is effectively unusable.

MR-MUF vs. NCF vs. hybrid bonding

SK Hynix uses MR-MUF (Mass Reflow Molded Underfill), which offers strong thermal performance and is currently the most production-proven approach. Samsung uses NCF (Non-Conductive Film) with thermocompression bonding, which provides better precision at fine pitches.

The longer-term direction is hybrid bonding, which eliminates bumps entirely. It’s already in production for stacked CMOS image sensors. The challenge for HBM is the combination of TSV integration and layer count — applying hybrid bonding across 12 or more layers introduces new yield, metrology, and long-term reliability questions that remain unsolved at production scale.

Warpage

As layers stack up, the accumulated mismatch in coefficient of thermal expansion between dies builds mechanical stress throughout the package. The result is warpage — a bow in the package that compounds with every additional layer. Simulation data consistently shows increasing residual stress as layer count grows, and that stress affects both downstream assembly and field reliability.

CoWoS: the bottleneck that outlasts the die

Finishing the HBM stack is not the end of the line. The HBM still has to be integrated with a GPU or ASIC on a silicon interposer through TSMC’s CoWoS process. As of now, TSMC’s CoWoS capacity is sold out through 2026. That single constraint is the narrowest point in the entire AI chip supply chain.

5. After Delivery: Shipping It Doesn’t Mean You’re Done

HBM ships to customers — NVIDIA, AMD, Google and others — as a standalone component. The customer assembles it alongside their GPU or ASIC into a 2.5D SiP. That’s where a new set of problems begins.

Thermal stress during customer assembly

The customer’s packaging process involves reflow heat treatment, which puts additional thermal stress on the microbumps and underfill inside the HBM stack. Parts that passed every test at the memory vendor’s facility can develop problems after going through the customer’s assembly line. It happens.

Three failure mechanisms in the field

In data centers running 24/7 at full load, three degradation mechanisms work in parallel. Electromigration moves metal atoms along fine interconnects under sustained high current density. Thermal cycling fatigues interconnects through repeated temperature swings. Creep deforms solder joints under prolonged high-temperature mechanical load. A product that passed qualification marginally — not with strong margin, just enough to pass — can fail quietly in the field months or years after shipment.

PPR and predictive maintenance

The HBM specification includes Post Package Repair, which allows a failed lane to be replaced with a spare. It works when the defect is isolated and the spare budget hasn’t been exhausted. For physical damage that has accumulated over time, hardware replacement is ultimately the only answer.

That limitation is pushing the industry toward predictive maintenance — continuous monitoring of signal quality during normal operation, so degradation can be detected before it turns into a system failure. Synopsys and proteanTecs are among the companies building solutions in this space.

What this actually looks like — from my time at AMD

I dealt with this firsthand at AMD. When an HBM-related failure surfaces at the product level, you immediately run into three walls.

The first is attribution. Is this an HBM failure or a GPU failure? Since both are integrated into the same package, there’s no way to probe internally. You’re working from symptoms and trying to reason backward to a cause.

The second is test methodology. Even once you’ve narrowed it to HBM, you have to figure out how to retest it at the system level. The test environment a memory vendor uses before shipment and the environment a customer can access on a finished SiP are completely different. There’s no pre-existing playbook. You build the methodology from scratch, under schedule pressure.

The third is the vendor interface. Getting useful analysis from the memory vendor requires giving them the right data in a format they can actually work with. That interface — what information to share, in what structure — is rarely established in advance. And the vendors’ own field support capacity is limited. When something goes wrong in production, the engineers who can actually help are already stretched thin. The result is that field failures at this level generate a disproportionate amount of wasted time and resources on both sides.

The Bottom Line: Why HBM Is Expensive and Why It’s Scarce

If there’s one sentence that captures why HBM is so hard, it’s this: it’s the only product in the semiconductor value chain where every single stage is running at maximum difficulty at the same time.

And even after clearing all of that to actually produce HBM, you still have to go through CoWoS packaging to get to a finished product — and that line is sold out through 2026.

That’s why HBM is expensive, why it’s scarce, and why memory company stock prices keep going up.

Executive Summary

High Bandwidth Flash (HBF) emerged with an ambitious promise to solve AI’s memory bottleneck. The idea was to stack NAND flash like HBM, delivering HBM-level bandwidth with 16x more capacity. SK hynix and SanDisk are developing this technology with a target of sampling in late 2026, and on the surface, it looks like the perfect answer to LLM inference memory capacity problems.

However, a close analysis of the H³ paper published by SK hynix researchers reveals that HBF faces much higher barriers to practical implementation than initially apparent. This article examines the assumptions and limitations of the H³ architecture from a practicing semiconductor engineer’s perspective, reviews the technical and economic challenges HBF will face during commercialization, and looks at alternative technology solutions currently being developed in the industry.

Key findings:

The read-only workload assumptions in H³ are very limited in actual LLM inference scenarios. Overcoming NAND’s fundamental physical limitations (1-2 orders of magnitude latency difference) requires a 40MB SRAM buffer, DRAM, and complex controllers, which undermines the initial promise of “cheap NAND.” Practical barriers like production yield, thermal management, and reliability validation are higher than expected. Alternative solutions like CXL Memory, HBM4, and software optimizations are maturing faster.

1. Background of HBF: The Memory Capacity Crisis Created by AI

The bottleneck in AI workloads is no longer compute performance. To keep up with the 989 TFLOPS that NVIDIA’s H100 can deliver, memory needs to supply data just as fast. HBM3 meets this requirement with 819GB/s bandwidth and 100ns access latency, but it has a critical weakness: This is capacity. At maximum 192GB per GPU (B200), HBM is woefully insufficient for running large models like Llama 3.1 405B (about 405GB in FP8) on a single GPU.

The bigger problem is KV cache. For Llama 3.1 405B with 1M token context, pre-computed KV cache alone reaches about 540GB. Extending to 10M tokens means 5.4TB. Handling this with HBM alone would require dozens of GPUs, driving up both cost and power consumption proportionally.

This is the background for what SanDisk and SK hynix call the “memory wall” and their proposal of HBF.

HBF’s promise is clear: stack NAND flash with TSV (Through Silicon Via) like HBM to provide 16x the capacity of HBM (~3TB) at equivalent bandwidth (8TB/s). Since NAND costs about 1/5 of HBM, the economics work out too. On the surface, it’s a perfect solution. But we need to look more closely.

2. The H³ Paper’s Approach: Hybrid Architecture and Core Assumptions

2.1 H³ Architecture Overview

The H³ (Hybrid architecture using HBM and HBF) proposed by SK hynix researchers acknowledges HBF’s limitations when used alone and takes a hybrid approach combining HBM and HBF. The core design works like this:

HBM connects directly to the GPU shoreline for maximum bandwidth. HBF connects through the HBM base die in a daisy-chain configuration. An address decoder and router inside the HBM base die separates HBM/HBF access. A 40MB SRAM-based Latency Hiding Buffer (LHB) mitigates NAND’s slow access latency.

In this structure, the GPU sees both HBM and HBF as main memory through a unified address space. Read-only data (model weights, pre-computed KV cache) goes in HBF, while dynamically generated KV cache stays in HBM.

2.2 Core Assumptions

H³’s performance claims rest on several important assumptions:

Workload assumption: Most LLM inference data is read-only. Model weights and shared pre-computed KV cache don’t change during the entire inference period.

Access pattern assumption: LLM inference is deterministic and sequential. Therefore, needed data can be accurately predicted and prefetched in advance.

Performance assumption: The 40MB SRAM buffer achieves a high enough hit rate (not explicitly stated in the paper, but implicitly requires 80%+ ) so that HBF’s 20μs latency stays hidden most of the time.

Latency hiding assumption: Since LLM inference is memory bandwidth-bound, the hop latency from HBF access can be sufficiently hidden.

Cost assumption: Since NAND dies are cheap, the total system remains economical compared to HBM-only even after adding extra components (SRAM, DRAM, controller, TSV).

The paper’s simulation results are impressive. 1.25x throughput improvement for 1M token cases, 6.14x for 10M token cases. Throughput per power reaches up to 2.69x. Even cutting HBF bandwidth in half still beats HBM-only.

But these results are only achievable when all the above assumptions are met. And that’s where the problem lies.

3. Limits of Assumptions and Technical Feasibility Issues

3.1 Limitations of the Read-Only Workload Assumption

The paper classifies model weights and shared pre-computed KV cache as “read-only,” but how valid is this assumption in actual production LLM services?

Reality of Model Weights:

Fine-tuning and PEFT: In production environments, parameter-efficient fine-tuning like LoRA and QLoRA is common. Adapter weights are small but frequently updated.

Model versioning: A/B testing or gradual rollout scenarios involve serving multiple model versions simultaneously. How does HBF handle model swaps?

Quantization changes: Dynamic switching between INT8, FP8, and FP16 is a common production optimization technique.

Reality of KV Cache:

Scope of pre-computed cache: The cache-augmented generation (CAG) that the paper presents is a valid use case, but it covers only a tiny fraction of total LLM inference. General-purpose conversational services like ChatGPT and Claude generate new KV cache for every request.

Cache invalidation: When shared documents get updated, how do you refresh pre-computed cache? Given HBF’s low write endurance, this is a critical issue.

Cache eviction: Managing hundreds of GB of shared cache pool requires replacement policies like LRU, which involve writes.

3.2 NAND’s Physical Limits: An Unavoidable Wall

Even if the read-only assumption holds, a more fundamental problem remains. The latency difference between NAND cells and DRAM cells isn’t something architectural tricks can solve. This comes from differences in physical laws:

DRAM cell: Reads and writes charge in a capacitor. Only requires electrical switching. 10-20ns.

NAND cell: Moves electrons to a floating gate through tunneling. Requires high voltage and long time. 25-100μs.

That’s a 1-2 order of magnitude difference. This is also the fundamental reason Intel Optane (3D XPoint) couldn’t replace DRAM. Even Optane’s ~100ns latency couldn’t match DRAM’s 10-20ns. HBF’s 20μs? That’s a 1000x difference.

The 40MB SRAM buffer “mitigates” this difference but doesn’t “solve” it. The moment an SRAM miss occurs, that difference is fully exposed.

In other words, “pure read-only” workloads are very limited in practice, and even when read-only is guaranteed, NAND’s physical latency limits can’t be overcome. The paper’s simulation represents an idealized scenario.

4. Cost Structure Reality: The “Cheap NAND” Trap

HBF advocates emphasize that “NAND wafers cost much less than HBM.” True. Looking at just the NAND die itself, it’s definitely cheaper than an HBM die. But what about total cost?

For HBM, the main costs are memory dies, TSV stacking, and packaging. The controller is integrated inside the GPU, and no separate buffer or intermediate layer is needed. The structure is relatively simple.

HBF starts with cheap NAND dies, but it doesn’t end there. To hide NAND’s slow latency requires a 40MB SRAM buffer. This isn’t a small cache memory but a substantial amount of high-speed memory. SRAM costs far more per unit area than NAND.

Add to this the separate DRAM needed to run the FTL (Flash Translation Layer). Like an SSD controller stores metadata, HBF needs working memory for address mapping and wear leveling. This DRAM is an additional cost.

TSV stacking and bonding itself is an expensive process. TSV involves drilling microscopic holes vertically through silicon and filling them with metal. It requires expensive equipment and precise process control, and a single mistake can render the entire die defective. HBM uses homogeneous stacking of identical memory dies, so the process is relatively standardized. HBF uses heterogeneous stacking mixing NAND, SRAM, and controller logic.

Aligning and bonding dies with different characteristics is much trickier. They have different thermal expansion coefficients, different electrical properties, and different reliability requirements. This significantly increases process complexity, leading directly to lower yields and higher costs. Problems like die cracking when drilling TSVs, misalignment during bonding, or delamination from thermal stress become more likely.

Packaging and testing also get more complex. For HBM, memory access testing is the main validation item. For HBF, you need to verify SRAM buffer hit rates, FTL accuracy, wear leveling algorithms, ECC (Error Correction Code) operation, garbage collection efficiency, and more. It’s essentially SSD controller-level complexity.

The cost of the controller logic itself that manages all this can’t be ignored either. HBM is a simple memory interface, but HBF needs a sophisticated controller performing complex address translation, prefetching, cache management, wear leveling, and garbage collection.

More importantly, there’s development cost and risk. HBF is a completely new architecture. It requires massive investment and time for R&D, standardization work, software ecosystem building, and customer validation. HBM has already been through all this. Yields are stable and the ecosystem is mature. HBF has to start from scratch.

Low yields in early production also matter. HBM initially had low yields and high cost per unit. It improved gradually over several generations. HBF uses more complex heterogeneous stacking than HBM, so initial yields will likely be lower. If yield is half, cost is effectively double.

Software integration costs can’t be overlooked either. PyTorch, TensorFlow, CUDA, and all AI frameworks were designed assuming HBM and DRAM. To efficiently use HBF’s SRAM buffer requires optimizing memory allocation strategy, data placement, prefetching hints, and more at the software level. This takes significant engineering resources.

So while “NAND is cheap” is true, “HBF systems are cheap” needs verification. Starting with cheap materials, all the elements added in making it into a practical product drive up total cost. For HBF to prove economic advantage, it needs to demonstrate not just $/GB but actual TCO (Total Cost of Ownership) in real workloads.

5. Alternative Technology Solutions and Market Dynamics

While HBF aims to sample in 2026-2027, other technologies are already maturing rapidly. The industry is approaching the same problem of memory capacity expansion in different ways.

5.1 HBM4: The Power of Conventional Evolution

SK hynix, Samsung, and Micron are focusing on HBM4 development. Production target: 2025-2026.

Bandwidth: 1.5TB/s per cube (50% improvement over HBM3e)

Capacity: 32-48GB per cube (improved stacking technology)

Reliability: Maintains proven HBM reliability

Ecosystem: Perfect compatibility with existing software stack

If 384GB per GPU (8 cubes × 48GB) becomes possible, HBF’s “capacity advantage” shrinks. Plus HBM4 is proven in latency, reliability, and ecosystem.

HBM-PIM (Processing-In-Memory): Samsung’s HBM-PIM performs simple operations (vector addition, activation) inside the memory. It reduces data movement to increase effective bandwidth. More innovative than HBF while building on the existing HBM ecosystem.

5.2 CXL Memory: A New Paradigm in Scalability

Compute Express Link (CXL) is a standard connecting CPU/GPU and memory over PCIe. CXL 2.0/3.0 supports memory pooling:

Capacity expansion: Multiple servers access a shared memory pool. TB-scale scaling possible.

Flexibility: Allocate only what’s needed. Better utilization.

Bandwidth: CXL 3.0 is PCIe 6.0-based at 256GB/s (x16 lane). Lower than HBF, but overwhelming in capacity scalability.

Ecosystem: Intel, AMD, NVIDIA all support it. Industry standard.

Samsung, SK hynix, and Micron are already mass-producing CXL Memory modules. This addresses the “large capacity memory expansion” need that HBF targets, but in a different way.

5.3 Software Optimization: Reducing the Problem Itself

Instead of increasing memory with hardware, approaches to reduce memory usage through software are also developing rapidly:

FlashAttention-3: Optimizes KV cache access patterns to reduce memory bandwidth requirements. FlashDecoding++ cuts latency to 1/3 in long context inference.

Grouped-Query Attention (GQA): Adopted by latest models like Llama 3. Reduces KV cache size 4-8x while maintaining performance.

Quantization: FP8 and INT4 quantization cut memory footprint to half or less. NVIDIA H100/B200 natively support FP8.

vLLM, TensorRT-LLM: Inference engines that optimize memory management. Paged Attention reduces memory waste, continuous batching increases utilization.

These software optimizations reduce memory pressure without hardware investment. By the time HBF reaches production, we might not need that much memory in the first place.

5.4 Different Strategic Choices in the Market

SanDisk and SK hynix are leading HBF development. Interestingly, other major memory vendors chose different technology paths:

Samsung: #1 in HBM market share. Focused on HBM4 and HBM-PIM development. No official HBF announcement.

Micron: Started HBM3e supply. Expanding CXL Memory product lineup. No HBF mention.

NVIDIA: B200, GB200 roadmap adopts HBM3e and NVLink-based memory expansion strategy.

AMD, Intel: Focusing on building CXL ecosystem.

This shows the industry is trying to solve the same “large capacity memory expansion” problem through different technical approaches. HBF is a NAND-based innovative approach, HBM4 is gradual improvement of proven technology, and CXL focuses on system-level scalability. Each company appears to have chosen the optimal strategy based on their technical capabilities and market positioning.

Which approach ultimately wins in the market will be determined by actual production performance, economics, and customer adoption rates.

6. Why HBF Still Matters: Memory Companies’ Platform Strategy

Despite the challenges HBF faces, there’s a fundamental business strategy shift in the memory industry behind why this technology gets attention. Traditionally, memory companies were commodity suppliers. Whether DRAM or NAND, they produced to standardized specs and competed on price for market share. The problem with this model is difficulty in differentiation and low margins.

HBM’s emergence started changing this dynamic. HBM isn’t just a memory chip but a system component that needs tight integration with GPUs. It requires complex engineering for TSV stacking, thermal management, power management, and interface optimization with GPUs. This opened opportunities for memory companies to provide higher value and secure better margins.

HBF is an extension of this trend. Beyond simply supplying memory, it proposes a solution that redesigns the entire memory hierarchy. The HBF system is a complex platform integrating NAND, SRAM buffer, DRAM, controller, and interface logic. This enables memory companies to:

• Collaborate with customers at the system architecture level

• Extend influence into the software stack (prefetching hints, data placement optimization, etc.)

• Create technical differentiation beyond simple price competition

• Accumulate IP and know-how to raise entry barriers

Samsung’s HBM-PIM follows the same context. By adding compute functions inside memory, memory companies transform from simple storage device suppliers into part of the computing architecture. Micron’s CXL Memory is similar. It’s a platform solution that changes how server systems manage memory.

From this perspective, HBF’s technical challenges don’t necessarily mean project failure. Even if HBF doesn’t replace HBM in the general AI accelerator market, memory companies gain from this process:

• Heterogeneous memory stacking technology

• Know-how for using NAND as memory

• System-level collaboration experience with GPU/AI accelerator vendors

• IP accumulation usable for future platform product development

SK hynix collaborating with SanDisk to develop HBF can be understood in this context. SK hynix dominates DRAM and HBM, while SanDisk dominates NAND. Their collaboration is a strategic move experimenting with memory technology convergence and platformization beyond single product success.

Ultimately, HBF isn’t just “memory with more capacity than HBM” but an example of the memory industry’s efforts to transition from commodity business to platform business. Whether this specific product succeeds or fails in the market, this direction itself matters as a long-term survival strategy for the memory industry.

Conclusion: Technology’s Promise and Reality’s Wall

High Bandwidth Flash started with an attractive promise. HBM-level bandwidth with 16x capacity and cheap NAND cost. It looked like it would solve AI’s memory bottleneck in one shot.

But analyzing the H³ paper from SK hynix researchers in depth reveals the price required to keep that promise:

• Physical limits: NAND cell’s 1-2 order latency difference can’t be overcome architecturally. The 40MB SRAM buffer is mitigation, not solution.

• Exploding complexity: Adding 40MB SRAM, tens of GB DRAM, complex FTL controller, and difficult TSV stacking to “cheap NAND” makes the total system anything but cheap.

• Fragile assumptions: The pure read-only workload and deterministic access patterns that H³ assumes are extremely limited in actual production.

• Reliability concerns: Whether NAND’s aging, read disturb, and endurance issues in GPU thermal environments can meet production requirements remains unknown.

• Market indifference: With major players silent and alternative technologies maturing quickly, HBF’s market positioning looks unclear.

Does this mean HBF will fail?

No. SanDisk and SK hynix’s technical capabilities are beyond question. They’ll prove this technology works at the 2026-2027 sample stage. But “it works” and “it succeeds” are different questions.

HBF’s future is probably:

• A niche solution effective in highly specialized CAG workloads

• Special markets where power and capacity balance is critical, like Edge AI devices

• A complement targeting “the gap between HBM and SSD,” not an HBM replacement

Technology starts with promise, but must overcome reality’s wall to reach the market. HBF is standing before that wall now.

Disclaimer

This article is a technical analysis based on published papers, technical documents, and general semiconductor industry principles. HBF technology’s final performance and market success can only be judged with post-production empirical data.

This article is not intended as investment advice or evaluation of specific companies (SanDisk, SK hynix, Samsung, Micron, etc.). There is no intention to disparage the technical capabilities and R&D efforts of the mentioned companies.

Technology evolves rapidly. Some content may change due to technical developments after this article’s writing (February 2026). Readers should verify the latest information.