Prediction, Not Understanding
Your tool completes code the way your phone completes sentences: by predicting what usually comes next. That one fact explains its most dangerous habit.
Time to look under the hood, because one fact explains almost everything strange these tools do.
An AI coding tool is powered by a large language model: a program trained on an enormous amount of text, including millions of public code repositories. Its one job: given some text, predict what plausibly comes next, one small chunk at a time.
Code, to the model, is just text with very strong patterns.
// You type:
function getUserById(
// The model has seen this shape a million times, so it continues:
function getUserById(id) {
return users.find(user => user.id === id);
}It is not reading your requirements. It is not reasoning about your database. It is completing a pattern, the way your phone keyboard completes "See you" with "soon". The completion is drawn from what code usually looks like in this situation, across everything the model was trained on.
When "what usually comes next" matches what you need, the tool looks telepathic. When it doesn't, you get code that looks exactly right and isn't.
When an AI tool completes your half-written function, what is the completion fundamentally based on?
- AStatistical patterns: what code most plausibly continues this text, learned from training data
- BA static analysis of your whole program to derive the logically correct continuation
- CExecuting your code in a sandbox and completing it so the tests pass
- DA lookup in an index of verified, human-reviewed code snippets
Here is the trap hidden inside pattern prediction: the strongest pattern wins, even when your actual intent points the other way.
Function names, common idioms, and famous code shapes are extremely strong patterns. Your comments and specs are weaker ones. When they conflict, guess which side the prediction lands on.
Watch this setup. A developer is implementing a spec that is deliberately unusual:
// PROJ-114: legacy quirk. This must return true when n is ODD.
// (The name is historical and cannot be changed without breaking callers.)
function isEven(n) {A human reads the comment, winces at the name, and writes return n % 2 !== 0;.
The model has seen function isEven followed by the classic even-check body tens of thousands of times. The comment above is one weak signal against a tidal wave of pattern. This is not a rare edge case; it is the everyday shape of AI mistakes: your code's names and shapes pull the prediction toward the usual thing, not your thing.
A developer pauses after the opening brace and lets the completion tool fill in the body. Given how pattern prediction weighs a famous function name against an unusual comment, what body does the tool most likely produce?
// PROJ-114: legacy quirk. This must return true when n is ODD.
// (The name is historical and cannot be changed.)
function isEven(n) {- Areturn n % 2 === 0; (the classic isEven body, contradicting the comment)
- Breturn n % 2 !== 0; (correctly following the spec in the comment)
- CIt refuses to complete a function whose name contradicts its spec
- Dthrow new Error("Name conflicts with specification");
Let's name the distinction this lesson turns on, because the whole course leans on it.
- Plausible code looks like code that solves this kind of problem. It is idiomatic, fluent, well-shaped. Pattern prediction produces plausible code by construction: that is literally what it optimizes.
- Correct code does what YOUR situation requires: your spec, your edge cases, your data.
Most of the time these overlap, which is why the tools are genuinely useful. But nothing in the machinery checks correctness. There is no step where the model verifies its output against reality. Fluency is the product; correctness is a frequent side effect.
This has a sharp practical consequence: you cannot judge AI code by how it reads. Reading quality is exactly the thing the model is superhuman at producing. The tells of wrong AI code are not awkwardness or hesitation; they are mismatches with facts the model didn't weigh: your spec, your versions, your edge cases. Finding those mismatches is a learnable skill, and Modules 4 and 5 drill it.
A teammate says: "The generated function matches our codebase's style perfectly, so the tool clearly understands our system." What is the sharpest correction from this lesson?
- AMatching style shows strong pattern prediction, not understanding; the same machinery happily produces idiomatic code that violates your spec
- BThe teammate is right: style matching is only possible with genuine comprehension of the system
- CThe tool copied the style from a database of verified snippets, so the logic is verified too
- DStyle matching is a bug; correct tools should output a neutral generic style
- AI coding tools are powered by models that predict the most plausible continuation of text; code is text with strong patterns.
- The strongest pattern wins: famous names and idioms outweigh your comments and unusual specs, which is where everyday AI bugs come from.
- Plausible is not correct. Fluency is produced by construction; correctness happens only when pattern and intent coincide.
- You cannot judge AI code by how it reads; judge it against your spec, your tests, and real documentation.
Next: prediction is the engine, but it is wrapped in three very different vehicles. Autocomplete, chat, and agents see different things, control different things, and fail differently.