FP8 / FP4 precision
Also called: FP16, FP32, BF16
8-bit or 4-bit number formats used in AI. Fewer bits mean less memory and more operations per second, at the cost of a little precision.
A GPU computes with floating-point numbers, stored on a larger or smaller number of bits. FP32, on 32 bits, is classic single precision, FP64 is used for scientific computing, and FP16 and BF16 use 16 bits. For AI, NVIDIA added FP8 with the Hopper and Ada Lovelace architectures, then FP4 with Blackwell.
Halving the number of bits halves the memory used and, on compatible Tensor Cores, roughly doubles the number of operations per second. A neural network tolerates this loss of precision well, especially for inference: a model properly converted to FP8 gives almost identical answers. FP4 goes further and requires calibration techniques to limit the loss of quality. Training is most often done in mixed precision, in BF16 or even FP8.
On a product page, performance in TFLOPS is always given for a specific precision. An FP4 figure can be several times, or even dozens of times, higher than the FP32 figure: so compare two cards at the same precision, and look out for the “with sparsity” mention, which doubles the result again. Also check that the target precision is actually supported by the hardware: at NVIDIA, native FP4 only appears with Blackwell.
We help you choose between an AI workstation and a server based on the precision and size of the models you want to run.