Fine-tuning

Fine-tuning starts from a model already trained on enormous amounts of text or images, then trains it again on a smaller, targeted dataset. The model keeps its general knowledge but learns a tone, an answer format or a specialist vocabulary: legal, medical, technical, or simply that of your company.

Retraining all the parameters of a large model requires a lot of memory and several high-end GPUs. That is why methods such as LoRA are widely used: they freeze the original model and train only small matrices added alongside it, a few million parameters instead of several billion. The result fits in a small file that is loaded on top of the base model. Fine-tuning is not always the right answer: to make an AI know your documents, RAG is often simpler and easier to keep up to date.

On the hardware side, the amount of VRAM is the first criterion. A LoRA on a 7 to 8 billion parameter model loaded in a quantized version remains within reach of a 16 GB card. Full fine-tuning of the same model needs far more, often several GPUs linked together. Also look at memory bandwidth and support for recent compute formats such as BF16.

We build GPU workstations and rigs suited to both training and inference, and we help you size the memory for the models you are aiming for.