How does an LLM like ChatGPT work? Explained in 3D

What is an LLM? A machine for guessing the next word

ChatGPT, Gemini, Mistral and Llama are LLMs, short for Large Language Models. Behind their multi-page answers, the mechanism is disconcertingly simple: the model reads a text and calculates, for every possible word, the probability that it comes next. It picks one, appends it to the text, and starts again. Everything else, grammar, facts, reasoning, emerges from this single task repeated billions of times during training.

The visualization at the top of this page is not a conceptual diagram but a real model: Qwen2.5-0.5B, an open model released by Alibaba, with 494 million parameters, reading the sentence « Le bitcoin a été créé par Satoshi » (“Bitcoin was created by Satoshi”). Every colored cell in the 3D view is a value actually computed by the model: positive numbers in green, negative ones in orange, learned weights in cyan and purple. Hover over a block to read the exact value.

Step 1: splitting the sentence into tokens

A model reads neither letters nor words, but tokens: chunks of text drawn from a fixed dictionary of about 150,000 entries. Frequent words have their own token, rare words are split into pieces. Our sentence gives 7 tokens, including « ·bitcoin » and « ·Satoshi » (the dot represents the space that precedes the word). « Eiffel », on the other hand, becomes « ·E », « iff » and « el », and the model will write « Nakamoto » in two steps: « ·Nak » then « amoto ».

Each token is replaced by its number in the dictionary: « ·bitcoin » is token number 17,873. From then on, the model handles nothing but numbers.

Step 2: embeddings, words turned into vectors

The number serves as an address in a large table of 151,936 rows by 896 columns, the embedding table. Each token retrieves its row from it: a list of 896 numbers that sums up what the model has learned about it. Tokens with similar meanings have similar vectors. These 136 million numbers are not written by hand: they were adjusted during training, like all the other weights.

Our sentence thus becomes a block of 7 rows by 896 columns. This block travels through the entire model and is enriched at every layer: it is called the residual stream, the glowing column in the 3D view.

Step 3: attention, or how words look at each other

To understand « créé par Satoshi » (“created by Satoshi”), each token must be able to fetch information from the others. This is the role of attention, the mechanism at the heart of the transformer, the architecture behind almost every LLM since 2017. Each vector is first normalized, then multiplied by three weight matrices to produce three new vectors:

• a query (Q): what the token is looking for;
• a key (K): what it offers the others;
• a value (V): the information it will pass on if it is chosen.

Each token compares its query with the keys of all the preceding tokens. The larger the product, the more it “looks at” that token, and it retrieves a mix of the values weighted by these scores. It can never look at the following tokens, which gives the attention maps their triangular shape. The position of each word is injected by rotating the vectors by an angle that depends on their rank (the RoPE technique).

The model performs this calculation 14 times in parallel, with 14 heads that each learn to spot something different. In our example, at the 3^rd layer, head no. 1 puts 87% of the attention of « ·Satoshi » on « ·par ». We also observe a phenomenon well known to researchers: on average, 59% of the last token's attention goes to the very first one, « Le », which serves as a resting point when there is nothing useful to read.

Step 4: the MLP, the model's memory

After attention, each token goes on its own through an MLP network: its vector of 896 numbers is projected to 4,864 dimensions, filtered by an activation function (SwiGLU), then brought back to 896. These matrices hold nearly two thirds of the model's parameters. Interpretability research suggests that this is largely where learned associations are stored, such as « Satoshi » leading to « Nakamoto ».

The output of attention, like that of the MLP, never overwrites the vector: it is added to it. Each layer writes its contribution into the residual stream, which accumulates information from bottom to top.

Step 5: 24 layers, and an answer that only arrives at the end

Attention followed by the MLP forms a block. Qwen2.5-0.5B stacks 24 blocks that are identical in shape, each with its own weights. Very large models stack 80 or more, with vectors of more than 8,000 numbers.

We can spy on the model along the way: at the output of each layer, we translate the last token's vector into a word, as if the model had to answer right away. This is the logit lens, shown to the right of each layer in the 3D view. The result is striking: for 22 layers, the suggestions make no sense (code fragments, Chinese characters), then the answer suddenly appears. « ·Nak » only appears at the 24^th and final layer; for « La tour Eiffel se trouve à » (“The Eiffel Tower is located in”), « ·Paris » appears at the 23^rd.

Step 6: choosing the next word, and temperature

The final vector is compared with each of the 151,936 rows of the embedding table, the same one as at the start: one per token in the dictionary. Each token gets a score, which is turned into a probability. Here, « ·Nak » wins with 96.4%, far ahead of « ·nak » (0.43%). On the Eiffel Tower, the model hesitates more: « ·Paris » at 34.4%, ahead of « ·la » at 13.5%.

A chatbot does not always take the first choice: it draws at random according to these probabilities. The temperature controls this randomness. Set low, it makes the model predictable and repetitive; set high, it flattens the probabilities and makes the model more creative, up to the point of incoherence. The slider in the 3D view recalculates the bars live.

Step 7: generating, one token after another

The chosen token is added to the sentence, and everything starts again: tokenization, embeddings, 24 layers, probabilities. After « ·Nak », the model is 99.5% sure that « amoto » comes next, then it writes « en 2008 » (“in 2008”). A 500-word ChatGPT answer means about 700 complete passes through the model. To avoid recalculating everything each time, the keys and values already computed are kept in memory (the KV cache).

What about training?

Nobody programmed the 494 million weights. At the start, they are random numbers. The model is shown huge amounts of text (up to 18 trillion tokens for the Qwen2.5 family), asked at each position to guess the next token, and each weight is corrected slightly in the direction that would have reduced the error. Repeated trillions of times, this correction is enough for language, and a good share of knowledge, to emerge. An assistant like ChatGPT then adds a second training phase on conversations, to learn to follow instructions.

Why AI needs GPUs

Each generated token requires about two operations per parameter: one billion operations for our small model, 140 billion for a model with 70 billion parameters. Above all, all the weights must be read from memory again for every token. It is memory bandwidth, more than computing power, that limits the speed of an LLM.

This is exactly why AI GPUs carry HBM memory: the NVIDIA H200 offers 141 GB at 4.8 TB/s, enough to fit a 70-billion-parameter model in FP8 on a single card. GPT-3 (175 billion parameters) takes up 350 GB in BF16 and needs at least three H200s just for its weights. It is also why a mining chip cannot run an AI: an ASIC is hard-wired for a single calculation, whereas an LLM chains matrix multiplications whose weights change at every layer. We explain this in detail in ASIC vs GPU: what's the difference?

Running a model in-house, on your own servers, keeps your data within the company. We design these machines, from workstations to 8-GPU servers: see our GPU servers for AI and our private AI assistant.

The models behind ChatGPT are much larger and their details are not public, but they belong to the same family as the one on this page, the decoder-only transformer, which was publicly documented up to GPT-3. A question about AI or the hardware that runs it? Call us on +33 6 32 49 26 41.