What Is a Large Language Model?

Finish this sentence: “Selamat hari ___.” You probably thought “raya” before you finished reading. You did not look it up; years of hearing Malaysian greetings made one ending far likelier than the others. Now imagine a machine that has read a large fraction of the public internet, plus millions of books and articles, and does nothing but play this game — guess the next bit of text — at astonishing scale. That machine is a large language model.

A very good guesser of the next word

A large language model (LLM) is a neural network trained on a vast amount of text to predict what comes next. Give it “The capital of Malaysia is” and it calculates, for every possible next word, how likely it is. “Kuala Lumpur” gets a high score; “banana” gets almost none.

To write a whole answer, it repeats the trick: pick a word, add it to the text, predict the next one, and so on — one piece at a time, like a very fast typist who decides each word only after seeing the ones before. Your question, plus everything it has written so far, is the input for each new guess.

It sounds too simple to be impressive. But to predict the next word well across every kind of writing — recipes, legal contracts, poetry, maths homework, Malay pantun — a model has to pick up a great deal about grammar, facts, styles and how arguments are put together. Prediction turns out to be a powerful teacher.

Tokens: how a model reads

Models do not actually read whole words. They read tokens — chunks of text that may be a whole word, part of a word, a space or a punctuation mark. Common English words are often one token; long or rare words get split into pieces. On average, a token is roughly three-quarters of an English word. Languages that were less common in the training data, including Bahasa Melayu and Tamil, often need more tokens for the same meaning, which can make them slightly slower and costlier to process. Type your own sentence into the box below.

Tokens matter to you for two reasons. Services measure limits and prices in tokens, and models sometimes stumble on tasks that depend on individual letters — such as counting the r’s in “strawberry” — because they never “see” letters, only chunks.

Choosing the next word: temperature

The model does not produce a single answer; it produces a list of possible next words, each with a probability. Something then has to choose. Always taking the top word gives safe, repetitive text. Sometimes taking a less likely word gives variety. A setting called temperature controls that balance.

In the demo below, the bars are made-up but realistic scores for the next word. Press “Sample a word” a few times. Then drag the temperature down to 0.1 and sample again; then push it up to 2.0.

At low temperature the model almost always picks the favourite (“sambal”): focused and predictable — good for facts and summaries. At high temperature the odds even out, and occasionally you get “durian”: varied, creative and sometimes silly. Chat apps usually choose this setting for you, which is why asking the same question twice can give you two different answers.

From text predictor to helpful assistant

A model that has only learned to predict text is not yet a helpful assistant. Ask it “What is the capital of Malaysia?” and it might continue with “What is the capital of Thailand? What is the capital of…”, because lists of quiz questions are common online. Turning a predictor into an assistant takes several stages.

The first stage is pretraining; the next two are forms of fine-tuning, called instruction tuning and RLHF. OpenAI described this recipe in its 2022 InstructGPT paper, months before ChatGPT launched in November 2022; most assistants today use variations of it. The story is told in The Generative Era.

Size, memory and the calendar

Why “large”?

“Large” refers to the number of parameters — the adjustable knobs from How Machines Learn — and the amount of text used in training. GPT-3, released by OpenAI in 2020, had 175 billion parameters, which was startling at the time. Some openly released models today have several hundred billion, a few around a trillion, and many companies no longer reveal the size of their biggest models As of 2026. Training data is measured in trillions of tokens. Researchers found that, within limits, bigger models trained on more data with more computing power get predictably better — the scaling laws — which is why companies have spent such vast sums on them. Bigger is not everything, though: smaller, well-trained models can now match the giants of a few years ago.

The context window: short-term memory

A model can only consider a limited amount of text at once: your messages, any documents you paste and its own replies. This is its context window, measured in tokens. Early GPT-3 could handle about 2,000 tokens — a few pages. Many assistants today handle hundreds of thousands of tokens, and some around a million — several long books As of 2026. When a very long conversation exceeds the limit, the oldest parts drop out or get summarised, which is why an assistant may “forget” something you said much earlier. Starting a fresh chat for a new task is a good habit. Working with Context shows how to make the most of the window.

The knowledge cutoff: a model frozen in time

Training stops on a certain date. After that, the model knows nothing new unless it is given a search tool or you paste in the information. That date is its knowledge cutoff. Ask a model without search about last week’s news, this year’s budget or today’s exchange rate, and it may give you an out-of-date answer — or confidently invent one. Many assistants now search the web for some questions; look for links to sources in the answer, and check them.

Fluent, yet sometimes wrong

Here is the most important practical lesson on this page. An LLM is trained to produce text that sounds like a likely, good answer. Most of the time, sounding right and being right go together. But not always. When the model does not “know” something, the most likely-sounding continuation may still be a confident, specific, well-written answer that is simply false: a book that does not exist, a wrong date, a made-up court case, a misquoted law. This is called a hallucination.

Consensus All current LLMs hallucinate sometimes; newer models generally do so less often, especially when they can search, but the problem has not gone away. Fluency is not evidence. For anything that matters — health, money, law, work — check important facts against a reliable source. What AI Gets Wrong goes further.

Does it understand anything?

This is one of the liveliest arguments in AI. Debated Is an LLM merely a very sophisticated autocomplete, or does it build some genuine understanding of the world on the way to predicting text?

In 2021 the linguist Emily M. Bender, the computer scientist Timnit Gebru and colleagues published a paper called “On the Dangers of Stochastic Parrots”. They argued that language models stitch together word patterns from their training data without any access to meaning — like a parrot repeating phrases — and that their fluency makes people wrongly assume there is a mind behind them. The paper also raised concerns about biased training data, environmental cost and who benefits. Its phrase “stochastic parrot” (stochastic means “involving chance”) became a rallying cry for sceptics.

Others push back. They point out that today’s models can solve new problems not in their training data, explain their reasoning, and work with images and code; and that researchers peering inside models have found internal structures that seem to track things like positions on a game board or facts about the world. Who is right depends partly on what you mean by “understand” — a question philosophers have not settled for humans, either.

This guide’s framing For practical purposes, the useful stance sits between the camps: use LLMs as capable, tireless helpers for drafting, explaining and brainstorming, while never assuming that confident words mean checked facts.

Key terms on this page

Sources & further reading

• Ashish Vaswani et al., “Attention Is All You Need”, NeurIPS (2017) — the Transformer.
• Tom Brown et al., “Language Models are Few-Shot Learners” (2020) — GPT-3.
• Long Ouyang et al., “Training language models to follow instructions with human feedback” (2022) — InstructGPT.
• Emily M. Bender, Timnit Gebru, Angelina McMillan-Major and Margaret Mitchell (as “Shmargaret Shmitchell”), “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?”, FAccT (2021).
• Jared Kaplan et al., “Scaling Laws for Neural Language Models” (2020).
• Melanie Mitchell and David Krakauer, “The debate over understanding in AI’s large language models”, PNAS (2023).