What is a large language model (LLM) and how does it write? Explained without maths

When you type a question into ChatGPT, Claude or Gemini, the answer appears word by word, as if someone were typing on the other side. That picture is a little misleading. Nobody is searching an archive or checking an encyclopedia. A large language model (LLM) is doing one single thing, many times in a row: deciding which piece of text is most likely to come next.

This guide explains that mechanism without formulas. It is not theory for its own sake: almost everything that frustrates you about AI (that it invents facts, forgets what you said at the start, answers differently each time) makes sense once you see how it writes.

The core idea: predicting the next piece

Think of your phone's autocomplete. You type "see you" and it suggests "tomorrow". An LLM does the same thing, at a vastly larger scale and with far more sophistication: it does not look only at the last two words but at the whole text in front of it, which can be an entire conversation or a document many pages long.

The process goes like this:

1. It receives your text (your instructions, the conversation so far and any document you pasted).
2. It calculates, for every possible piece, how likely it is to come next.
3. It picks one of those pieces.
4. It adds it to the text and goes back to step 2.

It repeats that loop hundreds or thousands of times until the answer is complete. That is why you see it appear gradually: it is not "written" in advance.

What a token is

Those pieces are not exactly words. They are called tokens, and a token can be a short word, part of a long word, a punctuation mark or a space. "Cat" may be one token; "unconstitutionally" will probably be split into several. As a rough guide, OpenAI estimates that in English one token is about three quarters of a word; other languages, such as Spanish, usually need somewhat more tokens per word.

Tokens matter for three practical reasons: providers charge per token, length limits are measured in tokens, and the model "sees" text through them. That last point is why it sometimes fails at tasks that look trivial, such as counting the letters in a word: it does not see letters, it sees chunks.

How a language model learns

An LLM goes through two big stages before it reaches your screen.

Stage one: reading an enormous amount of text

In pre-training, the model reads a gigantic amount of text (web pages, books, code, articles) and practises a single exercise: hide the next piece and try to guess it. Each time it gets it wrong, it slightly adjusts its parameters, millions or billions of internal numbers that work like dials.

After billions of attempts, those adjustments capture very rich patterns: grammar, common facts, writing styles, reasoning structures, how to write a Python function or a formal email. It is important to understand what does not happen here: the model does not keep the documents it read as if it were a library. It keeps statistical regularities. That is why it can explain a concept in new words, and also why it can invent a quotation that "sounds" like a real one.

The architecture that makes all of this possible is called the Transformer, introduced in 2017 in the research paper "Attention Is All You Need". Its key component is the attention mechanism: when deciding the next word, it lets the model weigh which parts of the earlier text matter most. In "the cat did not cross the street because it was tired", attention helps link "tired" to "cat" rather than to "street".

Stage two: learning to be an assistant

A model that has only been pre-trained knows how to continue text, but not necessarily how to follow instructions. If you write "Write a poem about the sea", it might continue with another similar instruction, as if it were completing a list of exercises.

To turn it into an assistant, it is trained again on examples of well-answered instructions and on human ratings: people compare answers and mark which is better, and the model is adjusted to produce more of the preferred kind. This technique is known as reinforcement learning from human feedback (RLHF), and OpenAI described it in detail in 2022 in its work on InstructGPT. Providers today combine variations of these techniques with their own safety rules.

The result is the assistant you know: it follows instructions, adopts formats and declines some requests. Underneath, though, it is still the same mechanism: predicting the most likely continuation, now in the style of "a helpful assistant".

What this explains about its behaviour

With the mechanism in mind, several behaviours stop being mysterious.

Why it invents facts with total confidence

The model has no internal truth detector. If you ask for the source of an unusual claim, the most likely continuation is something shaped like a source: author, title, year. Whether that source exists is not part of the calculation. This is called a hallucination. It is more frequent the rarer or more specific the fact you ask for (an exact figure, a court ruling, the biography of a little-known person).

The practical consequence is simple: anything you are going to publish, decide on or hand over gets checked. How to check an AI answer before you use it gives you a step-by-step method.

Why it does not know what happened yesterday

What the model knows about the world was fixed when its training ended, at what is called the knowledge cutoff. If the app does not connect it to a search engine or to your documents, it has no way of knowing anything later, and it may give outdated information as if it were current. When an assistant does cite recent pages, it is because the app ran a search and passed the results to it as context.

Why it "forgets" the start of a long conversation

The model only sees what fits in its context window: the maximum number of tokens it can process at once, adding up your messages, its answers and any documents. When a conversation goes beyond that, the app trims or summarises the oldest part. And even within the limit, information in the middle of a very long text tends to be used less well than information at the beginning or the end.

Why the same question gets different answers

The model does not always choose the most likely token: it samples among the candidates with some randomness, governed by a setting called temperature. At low temperature answers are steadier; at high temperature they vary more. In chat apps you usually cannot control it, but it explains why asking again can give you another version.

What this changes about how you ask

If the model writes what is most likely given the input, your input is the main lever. Some practical consequences:

• Give it context it cannot guess. Who you are, who you are writing for, what you already tried. Without that, the most likely answer is the generic one.
• Show it examples. One or two examples of the format you want change the result far more than three paragraphs describing it.
• Ask for the format explicitly. "Three-column table", "120 words maximum", "bullet points".
• Paste the source when you have it. If you give it the document, it works from that instead of from its statistical memory, and the risk of invention drops sharply.
• Allow it to say it does not know. An instruction such as "if you are not sure, say so and explain what is missing" reduces made-up answers.

A before-and-after example:

Before:
Write me an email for a customer.

After:
Write a 120-word email to a customer who paid for an order 10 days ago and
still has not received it. We are a small coffee shop in Medellín.
Acknowledge the delay without excuses, explain that the parcel ships tomorrow
with a tracking number and offer 10% off their next purchase. Warm tone, no
template phrases. If any detail is missing, leave it in square brackets.

The second version is not "magic": it simply leaves much less room for the generic continuation.

What an LLM is not

To finish, three common confusions:

• It is not a search engine. A search engine returns existing documents; an LLM generates new text. Some apps combine both, but they are different pieces.
• It is not a database. You cannot be sure it "remembers" a specific fact, even if that fact is on the internet.
• It is not a person. It can sound empathetic, confident or sorry, because that is how people write in the texts it learned from. That does not imply intentions or consciousness.

None of this makes it less useful. A language model is an extraordinary tool for drafting, summarising, explaining, coding and thinking out loud. It just performs better when you know what kind of tool it is.

Keep reading

• Tokens and the context window: why AI "forgets" and how to work with long documents without losing information.
• Why AI makes things up: what hallucinations are and five habits to reduce them.
• How to write a prompt that works the first time: a five-part structure for asking better.
• How to choose an AI model without looking at the leaderboard: how to compare models with your own tests.
• If you are starting from zero, follow the AI from scratch path and look terms up in the glossary.

Sources

• Vaswani, A. et al. (2017). Attention Is All You Need. arXiv.
• Ouyang, L. et al. (2022). Training language models to follow instructions with human feedback. arXiv.
• OpenAI Help Center. What are tokens and how to count them?
• Liu, N. F. et al. (2023). Lost in the Middle: How Language Models Use Long Contexts. arXiv.