Why AI Coding Tools Hallucinate APIs (and How to Catch It)

Ask an AI assistant to flatten a nested array in JavaScript and sooner or later you will be handed something like this:

const result = nested.flatten();

It looks perfect. It is confident. It is also not JavaScript. Run it and you get:

TypeError: nested.flatten is not a function

The real method is flat(). There is no flatten() on JavaScript arrays; it never shipped in the language standard. The model did not misremember it the way a person does. It generated a plausible method name, because generating plausible text is the only thing a language model ever does.

This article explains why AI coding tools invent APIs, the four distinct flavors the problem comes in (one of which is a genuine security threat), and a practical workflow for catching hallucinations before they cost you an afternoon.

The short answer

AI coding assistants are next-token predictors. They generate the most statistically likely continuation of your code based on patterns in their training data, without looking anything up. When your request lands in territory the training data covered densely, likely text and correct text are usually the same thing. When it lands in sparse territory, the model still produces likely-looking text, and there is no internal alarm that distinguishes "I know this" from "this resembles things I have seen." A hallucinated API is not a malfunction. It is the system working as designed, on a question where design and truth part ways.

If you understand that one paragraph, everything else about working with these tools follows from it. Let us unpack it properly.

Why prediction produces fake APIs

A language model trained on billions of lines of code has absorbed enormous statistical regularities: what function names look like, which words co-occur, how a library call is shaped. That is why it is so good at real APIs. flat(), map(), and reduce() appear millions of times in training data, so predicting them is easy and almost always right.

But the same machinery keeps producing output where the data is thin. Ask about an obscure library, an unusual parameter combination, or a task at the intersection of two frameworks, and the model composes an answer from adjacent patterns. Lots of languages have a flatten (Ruby does, Kotlin does, lodash does). JavaScript itself named the proposal flatten before renaming it flat() in 2018 to avoid breaking old websites. So nested.flatten() is a very probable string of tokens. Probability is the model's only compass, and probability points at a method that does not exist.

Three properties of this failure make it dangerous rather than merely annoying:

• No confidence signal. The model states its best guess and its worst guess in exactly the same fluent tone. Confidence in the prose tells you nothing about reliability of the content.
• No lookup step. Unless the tool is explicitly wired to search docs or run code, nothing in generation consults reality. It is pattern completion all the way down.
• Plausibility is optimized for. The output is specifically shaped to look right to a human reader. Hallucinations are camouflaged by construction.

The four flavors of hallucinated code

Naming the variants makes them much easier to spot in the wild.

1. Invented methods and functions

The flatten() case: a method that simply does not exist on the object. These are the friendliest hallucinations because they usually fail loudly and immediately with a TypeError (JavaScript) or AttributeError (Python). The error message itself is the diagnosis: the API is fictional, not misused.

2. Real API, wrong details

Nastier: the function exists but the model gets the signature, parameter names, default values, or return type wrong. It may pass a keyword argument the function never accepted, or assume a return value is a list when it is an iterator. Some of these fail loudly. The worst ones run fine and behave subtly differently from what the model told you, which means the bug reaches you as wrong behavior rather than as an error, sometimes much later. A section of our classic-bugs field guide is devoted to this shape.

3. Version blending

Training data contains every version of every library, all flattened together. The model has seen your framework's v2 syntax and v5 syntax in roughly the same neighborhood, so it happily writes code mixing both, or hands you a pattern that was idiomatic in 2019 and deprecated since. If you have ever received an answer using an API "that was removed two major versions ago," this is why. It also blends across libraries: a little pandas idiom in your polars code, an Express pattern in your Fastify handler.

4. Invented packages, which is now a security problem

Sometimes the model hallucinates an entire dependency: npm install some-plausible-name for a package nobody ever published. This used to be a harmless dead end. It no longer is. Because models hallucinate the same plausible names repeatedly, attackers register those names on npm and PyPI and fill them with malware, then wait for developers to install what the AI recommended. The technique is called slopsquatting, and it converts a model quirk into a supply-chain attack. The rule it forces is simple: an AI recommendation is not a reason to trust a package. Check the registry, the repo, the download count, and the age of any dependency you did not already know, before it touches your machine.

Why "just ask the model" does not work

The natural instinct is to ask, "are you sure flatten() exists?" Sometimes that helps. But the model generates its answer to that question the same way it generated the code: by producing likely text. It can double down on a fiction with a fluent, detailed justification, including citations to documentation pages that do not exist. It can also do the opposite and cave instantly on code that was correct, because you sounded doubtful and agreement is a very likely continuation.

Interrogating the model is still useful, but as a probe rather than a verdict: ask it to explain what a line does and where the API is documented, and treat a vague or shifting answer as a signal to check reality directly. The verdict has to come from outside the model. Which brings us to the workflow.

The catching workflow

You do not need to fact-check every line you generate. You need a layered net so that hallucinations get caught cheaply and early.

Layer 1: Let your tooling read first. A typed language or a good editor catches flavor 1 and much of flavor 2 before you even run anything. TypeScript red-squiggles a nonexistent method instantly. Python type checkers and IDE autocomplete do the same. This is a real, underrated argument for types in the AI era: they turn silent fiction into immediate visual noise.

Layer 2: Treat unfamiliar APIs as unverified. While reviewing a diff (here is the full reviewing checklist), any function, parameter, or package you do not personally recognize gets thirty seconds against the actual documentation. Not a vibe check. The docs page, for the version you are running. This single habit catches flavors 2 and 3 almost completely.

Layer 3: Run it before you trust it. A hallucinated method dies on its first execution. Run generated code on a real input in a scratch file or REPL before it goes anywhere important. Loud, early failure is the good outcome. Design for it.

Layer 4: Gate every new dependency. Before installing anything an AI suggested: does the package exist, does it have a real repository, real maintenance activity, and a plausible install count? Two minutes, and it closes the slopsquatting door.

Reducing hallucinations at the prompt

You cannot prompt hallucination away, but you can shrink the target:

• State your versions. "Node 22, Express 5" in the prompt reduces version blending, because it concentrates prediction on the right region of training data.
• Paste real signatures. If the task involves a library the model may know thinly, paste the actual function signature or a snippet of the docs into the context. Models are dramatically more accurate about text that is in front of them than text they must reconstruct from training.
• Constrain the toolbox. "Standard library only" or "use only the packages already in package.json" removes the invented-dependency failure mode outright.
• Prefer tools that can look things up. Assistants that search documentation or execute code before answering close part of the gap by construction. The reflex to verify is being built into the tools themselves, which tells you what the vendors think of raw generation too.

The mindset that makes this easy

Here is the reframe that turns all of the above from a chore into a habit: stop treating the assistant as an oracle and treat it as an extremely fast, extremely well-read collaborator who does not check their work. You would never merge a contractor's changes because they sounded confident. You check the work, and the work is usually good, and the checking is fast once you know where failures cluster.

That skill, knowing where AI code fails and verifying efficiently, transfers across every tool and every model generation. Hallucination rates have dropped as models improved, but the failure mode is inherent to prediction without lookup: thin training data plus fluent output. The developers who internalize that are the ones who get the speed of AI coding without inheriting its fiction.

Frequently asked questions

Do the newest AI models still hallucinate APIs?

Yes, less often but not rarely enough to skip verification. Better training and doc-search integrations have reduced the rate, especially for popular libraries. But the underlying mechanism, prediction without an authoritative lookup, has not changed, so sparse territory still produces confident fiction. Treat improvement as a lower error rate, not an error rate of zero.

What is slopsquatting?

Registering a package name that AI models frequently hallucinate, and filling it with malicious code, so that developers who install an AI-recommended dependency without checking it get compromised. It is typosquatting's AI-era sibling: instead of exploiting human spelling errors, it exploits statistically predictable model errors. Defense is unchanged: verify a package exists, is maintained, and is widely used before installing it.

Why does the AI apologize and then hallucinate again?

Because the apology is also just likely text. Correcting the model in a conversation does not update its knowledge; it only adds context to this session. If the underlying training distribution points at a wrong answer, the model can drift back to it, sometimes in the same thread. For anything that matters, put the correct information directly in your prompt rather than relying on the model having "learned" from your correction.

Can hallucinations be fixed for good?

Not within pure next-token generation, is the honest answer. What does eliminate them in practice is grounding: tools that retrieve real documentation, run code, and check results before presenting them. As assistants become more agentic, more of that grounding happens automatically. Until you can see it happening, the grounding step is yours.

Keep going

Spotting hallucinated APIs is one lesson from the failure-zoo module of our AI-Assisted Development course: 61 interactive lessons on directing AI tools, reviewing what they produce, and verifying the result before it ships, including hands-on challenges where you catch planted hallucinations in realistic diffs. The first three lessons are free to preview, no account needed.