Technology

I don't know if you're saying this, so my apologies if I'm misunderstanding what you're saying, but this isn't principally ECC DIMMs that are being produced.

I suppose that a small portion of AI-related sales might go to ECC DDR5 DIMMs, because some of that hardware will probably use it, but what they're really going to be using in bulk is high-bandwidth-memory (HBM), which is going to be non-modular, connected directly to the parallel compute hardware.

HBM achieves higher bandwidth than DDR4 or GDDR5 while using less power, and in a substantially smaller form factor.[13] This is achieved by stacking up to eight DRAM dies and an optional base die which can include buffer circuitry and test logic.[14] The stack is often connected to the memory controller on a GPU or CPU through a substrate, such as a silicon interposer.[15][16] Alternatively, the memory die could be stacked directly on the CPU or GPU chip. Within the stack the dies are vertically interconnected by through-silicon vias (TSVs) and microbumps. The HBM technology is similar in principle but incompatible with the Hybrid Memory Cube (HMC) interface developed by Micron Technology.[17]

The HBM memory bus is very wide in comparison to other DRAM memories such as DDR4 or GDDR5. An HBM stack of four DRAM dies (4‑Hi) has two 128‑bit channels per die for a total of 8 channels and a width of 1024 bits in total. A graphics card/GPU with four 4‑Hi HBM stacks would therefore have a memory bus with a width of 4096 bits. In comparison, the bus width of GDDR memories is 32 bits, with 16 channels for a graphics card with a 512‑bit memory interface.[18] HBM supports up to 4 GB per package.

I have been in a few discussions as to whether it might be possible to use, say, discarded PCIe-based H100s as swap (something for which there are existing, if imperfect, projects for Linux) or directly as main memory (which apparently there are projects to do with some older video cards using Linux's HMM, though there's a latency cost in that point due to needing to traverse the PCIe bus...it's going to be faster than swap, but still have some performance hit relative to a regular old DIMM, even if the throughput may be reasonable).

It's also possible that one could use the hardware as parallel compute hardware, I guess, but the power and cooling demands will probably be problematic for many home users.

In fact, there have been articles up as to how existing production has been getting converted to HBM production

there was an article up a while back about how a relatively-new factory that had been producing chips aimed at DDR4 had just been purchased and was being converted over by...it was either Samsung or SK Hynix...to making stuff suitable for HBM, which was faster than them building a whole new factory from scratch.

It's possible that there may be economies of scale that will reduce the price of future hardware, if AI-based demand is sustained (instead of just principally being part of a one-off buildout) and some fixed costs of memory chip production are mostly paid by AI users, where before users of DIMMs had to pay them. That'd, in the long run, let DIMMs be cheaper than they otherwise would be...but I don't think that financial gains for other users are principally going to be via just throwing secondhand memory from AI companies into their traditional, home systems.