remember we got Hands-On with NVIDIA's AI accelerator the h100. But this little one is pretty important too. It's Micron's HBM 3E, a high bandwidth memory chip used to power AI workloads. The Tech's revolutionary for the industry because it allows for high-speed data transfers and more capacity, both bottlenecks in developing AI. Producing the chips is complex, but the payoff is great for Micron, Samsung, and other players. The Micron Chip's smaller and thinner than a penny. It's key to train llms for generative AI. Basically, the more data you can serve up more quickly, the better the model gets. There are eight layers of DRAM or memory stacked on top of each other. Each layer is 3 GB, so together you get 24 GB. The reason that Micron's HBM3, or high bandwidth memory, is important is that in the AI context, it solves for a problem called the memory wall. To train the llms behind chat GPT, for example, you need to be able to process a large amount of data very

quickly, AI workloads increase, and Nvidia releases new higher performance AI chips like Blackwell. The memory demands also increase. This year, Micron also released a higher capacity chip that stacks 12 layers of dam, so 36 GB of memory. The two look identical when placed side by side. Each layer is made thinner to provide more memory without taking up more physical space.

New Tech: First Look at Micron's New HBM3E Chip

Welcome to the Bloomberg Technology YouTube channel. Subscribe for original series and mini-docs on game-changing technologies, profiles on innovators, live conversations with experts, and breaking technology news & analysis from Bloomberg TV.

New Tech: First Look at Micron's New HBM3E Chip

New Tech: First Look at Micron's New HBM3E Chip

Scripts