Context window
Also called: context, context length
The context window is the maximum amount of text, counted in tokens, that an AI model takes into account at once: instructions, documents and answer.
A language model cannot read an unlimited amount of text. With each request, it receives a set of tokens: the instructions, the conversation history, any attached documents, then the answer it generates. The context window is the maximum size of that set. It is expressed in tokens, for example 8,000, 32,000 or 128,000. A common word is often a single token, while a long or rare word takes several.
Anything beyond the window is ignored: in a long conversation, the model eventually “forgets” the beginning. It is working memory, not permanent memory. A large window makes it possible to analyse an entire contract or a large codebase in one go, but it is costly in computation and memory, because for each token the model keeps intermediate data called the key-value cache.
On a local machine, it is often the VRAM that limits the usable window, well before the model's theoretical limit. A model that fits in 16 GB with a short context may no longer fit with a 64,000-token context. Tools such as Ollama or vLLM let you adjust this length. To work with large volumes of documents, RAG avoids loading everything at once by passing only the relevant passages to the model.
When we configure an AI workstation, we take into account the context length you need when choosing the amount of GPU memory.