RAG
Also called: retrieval-augmented generation, document assistant
RAG (retrieval-augmented generation) lets an AI answer from your documents: it finds the relevant passages, then writes its answer citing them.
RAG stands for Retrieval-Augmented Generation. The principle is simple: before answering, the AI searches a document base for the passages related to the question, then uses them to write its answer. This is the technique used by most corporate document assistants.
It works in two stages. First, indexing: the documents are split into chunks, each chunk is converted into an embedding, a vector that sums up its meaning, and then stored in a vector database. Then, for each question, the question itself is also turned into a vector, the closest chunks are retrieved and passed to the LLM within its context window. The model no longer answers from memory: it relies on your texts, can cite its sources and makes far fewer mistakes. Adding a document is enough to update the assistant, with no retraining.
Quality depends as much on preparation as on hardware: clean documents, scans run through OCR, sensible chunking and a good embedding model. On the machine side, you need to run both the embedding model and the LLM. A 16 GB card is enough to get started; more VRAM allows a more capable model and a longer context to cite more passages. Open WebUI already includes a RAG feature for initial testing.
We install private AI assistants connected to your documents on your premises, with the hardware, installation and support, on quotation.