Welcome back!
You chat with them every day — Claude, ChatGPT, Copilot — but what actually is a large language model (LLM)? Today we'll take a friendly, jargon-free tour: how text becomes numbers, what "prediction" really means, and why these models sometimes confidently make things up.
Step one: text becomes tokens
Computers don't read words — they crunch numbers. So the first thing an LLM does is chop your text into tokens: small chunks that each get a number. A token is often a whole word, but it can be part of a word or even punctuation. Here's how a model might split a sentence:
Each common word got its own token. But a rarer word — say, giggled — might get split into pieces:
Common words get their own token; rare words get built from fragments, like spelling with LEGO bricks. This is why LLMs can handle words they've never seen — including your variable names.
The core trick: predicting the next token
Ready for the surprisingly simple secret? An LLM does one thing: given some tokens, predict which token comes next. That's it. It predicts one token, adds it to the text, then predicts the next, over and over, until a whole answer has poured out.
Think of your phone keyboard's autocomplete, which guesses your next word from the last few you typed. Now supercharge it: instead of glancing at three words, the model considers everything in the conversation, and instead of a pocket-sized memory of phrases, it has patterns absorbed from a huge slice of human writing — books, articles, and mountains of code. At that scale, "guess the next token" stops feeling like autocomplete and starts feeling like understanding.
Training vs. using
There are two very different phases in a model's life, and mixing them up causes a lot of confusion:
| Training | Using (inference) | |
|---|---|---|
| What happens | The model reads vast amounts of text, adjusting billions of internal dials to get better at next-token prediction | The dials are frozen; the model just predicts tokens for your prompt |
| How often | Once, over months, on enormous computer clusters | Every time you press Enter |
| Does it learn? | Yes — this is where all learning happens | No — your chat doesn't change the model itself |
This explains something important: when you chat with an LLM, it isn't "learning" from you in real time, and it doesn't know about events after its training ended. It's a very knowledgeable library with a printing date.
Context windows, in plain words
The context window is the model's working memory: the maximum number of tokens it can consider at once — your conversation, your pasted code, its own replies, everything. Picture a desk: the model can only reason about papers currently on the desk. If the conversation grows too long, the earliest pages slide off the edge, and the model genuinely no longer sees them.
Ever had a long chat where the AI "forgot" your earlier instructions? Now you know why. It wasn't being careless — those tokens simply fell off the desk.
Why models make things up
Here's the part every developer needs to internalize. An LLM's job is to produce plausible next tokens — not true ones. Truth and plausibility usually overlap, because the model learned from mostly-accurate text. But when they don't overlap, the model doesn't say "I'm not sure." It fluently generates something that sounds right: a function that doesn't exist, a citation to a paper never written, a config setting from a parallel universe. We call these hallucinations.
Tip: Hallucinations are most likely at the edges of the model's knowledge — niche libraries, very recent changes, or oddly specific facts. The more obscure your question, the more skeptical you should be of a smooth, confident answer.
What this means when you code with AI
- Verify names and APIs. A hallucinated method looks exactly like a real one until the compiler disagrees. Run the code; let errors be your fact-checker.
- Feed the desk. The model only knows what's in its context window. Paste the relevant code, the error message, and the goal — don't make it guess.
- Restate what matters in long chats. If a conversation drags on, briefly repeat your key requirements so they stay "on the desk."
- Expect a knowledge cutoff. For anything newer than the model's training, share documentation yourself or use tools that fetch it.
Note: None of this makes LLMs less impressive — a next-token predictor that can explain recursion and debug your SQL is astonishing. It just means you should treat it as a brilliant, occasionally overconfident collaborator rather than an oracle.
You now understand these tools better than most people who use them daily — nicely done! Next step: put that knowledge to work by writing prompts that play to the model's strengths in prompt engineering basics for developers, or see how systems ground models in real data with what is RAG?