Multi-GPU
Using several graphics cards in the same machine to combine their memory and computing power, for AI as well as mining.
A multi-GPU setup is a machine fitted with several graphics cards working together. It ranges from a two-card workstation to an eight-GPU server, and even clusters of networked servers. The principle is simple: combine the memory and computing power of several GPUs.
In mining, each card in a rig works independently: the total hashrate is simply the sum of the cards' hashrates. In AI it is more subtle. A model too large for a single GPU is split across several cards, layer by layer or within each layer, and the GPUs then have to exchange data. Two 16 GB cards can thus load a model of around 25 GB, but the speed does not double: everything depends on the link between them, PCIe or NVLink, and on the software used.
Before choosing a multi-GPU machine, check the number of PCIe lanes available for each card, the power supply capacity, the cooling and the spacing between the cards. On the software side, Ollama automatically spreads a model across several GPUs, and vLLM can make them work in parallel to serve several users. Several mid-range cards often offer more memory for the same budget than a single high-end card.
Our AI Pro workstation combines two 16 GB cards for 32 GB of video memory, and we offer servers with up to 8 GPUs on quotation.