Intermediate level8 min readTo learn at another level, choose it before you start the course.
After this explainer you canExplain how a model writes an answer one token at a time, why a sure answer can still be wrong, and how apps and agents use models.
The short answerAn AI model is a computer program that learned from a huge number of examples. You give it a question, and it gives you an answer.
In simple words
- It learned from examples, not from rules someone wrote.
- It writes by guessing the next token, a word or part of a word, again and again.
- It can sound sure and still be wrong.
- Chat apps are built around a model; an agent can also act for you.
The short answerAn AI model is a program that learned patterns from huge amounts of data. It turns an input, such as a question, into an output, such as an answer, a picture or a decision.
In simple words
- A model learns from examples instead of following rules a programmer wrote.
- Language models such as GPT, Gemini and Claude learned from vast amounts of text.
- They write by predicting the next token, a small piece of text, again and again.
- Apps and agents are built on top of a model; the model is the engine.
The short answerA modern language model is a transformer network whose parameters were first fitted to predict the next token, then adjusted by post-training to follow instructions and earn rewards. At use time it outputs a probability distribution over its vocabulary, and a decoding rule turns those probabilities into text.
Key points
- Most language models use the transformer design from 2017, built around attention.
- A causal mask lets each token attend only to itself and earlier tokens.
- Each step gives a probability for every token in the vocabulary; decoding settings such as temperature turn them into text.
- The EU AI Act calls these general-purpose AI models and gives their makers duties.
1 One token at a time
Each bar shows how likely a token is to come next. Press the button to add the most likely one.
A language model writes by
- predicting51%
- guessing22%
- choosing13%
- copying8%
- all other tokens6%
2 Model, product, agent
Words to know
- Model
- A computer program that learned from examples.
- Token
- A small piece of text, often a word or part of a word.
- Agent
- An AI tool that can act for you, such as filling in forms.
Learning from examples
Most computer programs follow exact steps that a person wrote. An AI model is different. It learned by looking at a huge amount of text, pictures or code, a bit like learning a language by reading a giant library.
Nobody writes its answers in advance. It makes each answer new from what it learned.
It only knows what was in its examples, up to a certain date. Some chat apps can also search the web to find newer facts.
Guessing the next token
When you ask a question, the model writes the answer one small piece at a time. These pieces are called tokens. A token is often a word, and sometimes part of a word. Each time, the model picks a token that is likely to come next.
That is why its answers sound natural. It is also why it can be wrong and still sound sure. It picks likely words. It does not check facts the way a careful person would.
The app around the model
Apps such as ChatGPT, Gemini and Claude are built around models. Gemini and Claude are also the names of the models inside. The app adds the chat window, memory of past chats, safety filters and extra tools.
An agent is a step further: it can also do things for you, such as open websites or fill in forms.
Try it yourself
Ask a chat app about something you know very well, such as your town. Check each fact: did a wrong fact sound sure?
Check yourself3 quick questions
A chat app gives you a long, well-written answer. How did the model make it?
A model’s answer sounds very sure. What does that tell you?
Which of these is an agent, not just a chat app?
Words to know
- Model
- A computer program that learned from examples.
- Token
- A small piece of text, often a word or part of a word.
- Agent
- An AI tool that can act for you, such as filling in forms.
Learning instead of rules
Ordinary software follows steps that a programmer writes down. An AI model works differently: it is shown a huge number of examples and adjusts itself until its answers fit them.
What it learns is stored as numbers called parameters. Large models have billions of them. Nobody writes these numbers by hand; training sets them.
How a language model writes
Language models split text into tokens: small pieces of text, often a whole word or part of a longer word. Given the tokens so far, the model predicts which token is likely to come next, adds it, and repeats.
This simple step, done thousands of times, produces answers, summaries and code. It also explains a known weakness: a model predicts likely words rather than looking up facts, so it can sound sure and still be wrong.
A model also has a training cut-off date. It knows nothing newer unless the app gives it web search or documents.
Model, product, agent
A model such as GPT-6 is the engine. A product such as ChatGPT wraps it with an app, safety filters and extra tools.
An agent goes one step further: it lets the model take actions, such as browsing, clicking or running code, to reach a goal. Many entries in our AI incident log involve agents.
How the law describes it
The OECD, an international organisation of governments, defines an AI system by what it does. It works out from its input how to produce outputs. These can be predictions, content, recommendations or decisions. The EU AI Act builds on the same idea.
The EU also has a separate term, the general-purpose AI model, for models that can do many different tasks. Their makers have their own duties, which the last explainers of this path cover.
Try it yourself
Ask a chat app about something you know very well, such as your town. Check each fact: did a wrong fact sound sure?
Check yourself3 quick questions
A model gives a wrong date for a famous event, but it sounds sure. What is the most likely reason?
A news story says a model has “billions of parameters”. What are parameters?
Which of these is an agent?
The transformer and attention
Most of today’s language models use the transformer, introduced in the 2017 paper “Attention Is All You Need”. Its key part, attention, lets each token weigh itself and all earlier tokens in the context. A causal mask hides later tokens, so the model never sees the token it must predict.
Each token enters the model as a vector, a list of numbers. Attention compares a token’s query vector with the key vectors of the tokens it can see, and the scores decide how much of each token’s value vector to mix in.
Because earlier tokens never look ahead, their keys and values can be stored and reused while the model writes; explainer 3 calls this the KV cache.
Stacking many attention and feed-forward layers gives the model its capacity. The weights of these layers are the parameters. Closed labs often keep the count secret: OpenAI’s GPT-4 report gave no model size.
From probabilities to text
For each position, the model outputs a probability for every token in its vocabulary. Llama 3’s vocabulary, for example, has about 128,000 tokens. A decoding rule then picks one token.
Greedy decoding always takes the most likely token. Sampling draws a token at random, weighted by the probabilities. A higher temperature flattens them, so less likely tokens are picked more often and the same prompt can return different answers.
Why models make things up
Next-token training rewards fluent, likely text, not checked truth. A fact that appears only once in the training data, such as a little-known person’s birthday, is easy to get wrong.
Many tests also score a wrong guess the same as “I don’t know”, so models learn that guessing pays. Kalai and others (2025) argue that changing how tests are scored is part of the fix.
Context and the system around the model
The context window is the most tokens the model can take into account at once, prompt and answer together. Attention compares tokens with each other, so its computing cost grows with the square of the context length.
Products wrap the model in system instructions, retrieval of documents, tool calls and safety classifiers. An agent runs a loop: plan, call a tool, read the result, repeat until the goal is met or a limit stops it.
Legal definitions
The EU AI Act defines a general-purpose AI model by what it can do. It shows significant generality, can competently perform a wide range of distinct tasks and can be built into many other systems.
Training on large data with self-supervision is the typical case, not a condition. Models used only for research, development or prototyping before release are excluded (Article 3(63)).
That definition, not the brand name, decides whether a maker has the duties of Articles 53 to 55, covered in the training and frontier explainers.
Try it yourself
Ask a chat app about something you know very well, such as your town. Check each fact: did a wrong fact sound sure?
Check yourself3 quick questions
In a model that writes text, which tokens can attention use when it processes the 10th token of a prompt?
A writer wants more varied story ideas from the same prompt. Which setting should they change?
A start-up trains a model that can only spot defects in photos of one car part. Is it a general-purpose AI model under the AI Act?
Sources
- Updated definition of an AI system (OECD.AI)
- AI Act, Article 3: definitions (EU AI Act Service Desk)
- Attention Is All You Need (Vaswani and others, 2017, arXiv)
- Generation strategies: greedy decoding and sampling (Hugging Face Transformers documentation)
- Introducing Meta Llama 3 (Meta)
- Why Language Models Hallucinate (Kalai and others, 2025, arXiv)
- Training language models to follow instructions with human feedback (Ouyang and others, 2022, arXiv)
- GPT-4 Technical Report (OpenAI, 2023, arXiv)
- Models overview, with training data cut-off dates (Claude documentation)
- Web search tool: answers beyond the knowledge cut-off (Claude documentation)
This explainer was generated by AI systems under the editorial rules of Silicon AI News and checked against the sources it lists. The live parts come from our checked stories, trackers, model comparison and rules checker.