HBM
Also called: HBM3e, High Bandwidth Memory
Very high bandwidth memory, stacked in layers right next to the GPU. HBM3e is used on AI accelerators such as the NVIDIA H200.
HBM stands for High Bandwidth Memory. Instead of chips spread across the card as with GDDR, HBM stacks several layers of memory on top of each other and places them right next to the graphics processor, on the same package. The generations have followed one another: HBM2, HBM2e, HBM3, then HBM3e, used on the latest AI accelerators.
This proximity allows an extremely wide interface, several thousand bits, where a consumer card is limited to a few hundred. The result is bandwidth measured in terabytes per second. The NVIDIA H200, for example, carries 141 GB of HBM3e with a bandwidth of about 4.8 TB/s. For AI this is decisive: training and inference of large models spend most of their time reading and writing data in memory.
HBM is expensive to manufacture and reserved for data center GPUs and AI servers, such as HGX platforms. On a product page, compare the total capacity, the HBM generation and the bandwidth. For a language model, the capacity tells you what size of model can be loaded, and the bandwidth how fast it will be able to generate text.
Our NVIDIA H200 page presents this card and its HBM3e memory in detail.