Tokens: How the Model Reads
A system that writes working code can fail kindergarten spelling. Tokens explain why, and they are also the meter that bills you.
Ask a model to count the letters in "strawberry" and it may confidently answer 2 r's. A system that writes working code fails kindergarten spelling. Why?
Because the model never sees letters. Before your prompt reaches the model, a tokenizer chops it into tokens: chunks of text, usually 3 to 4 characters or one common word each. The model reads a sequence of token IDs, not characters.
"The quick brown fox" → ["The", " quick", " brown", " fox"] (4 tokens)
"strawberry" → ["straw", "berry"] (2 tokens)
"antidisestablishment" → ["ant", "idis", "establish", "ment"] (4 tokens)
(splits from one common tokenizer; every model's tokenizer cuts differently,
so the exact pieces and counts vary between models)To the model, "strawberry" is a couple of opaque puzzle pieces. How many r's are inside a puzzle piece? It has to guess from patterns, and patterns about spelling are weak. Tokens are the model's syllables: it hears your text, it never sees it written down.
Which of these strings almost certainly splits into the MOST tokens?
- Afloccinaucinihilipilification
- Bthe cat sat on the mat
- Chello hello hello hello
- Dimport numpy
Tokenization explains a whole family of famous failures. They all share one shape: the task lives at the character level, but the model lives at the token level.
- Counting letters in a word 😖
- Reversing a string exactly
- "Words that end in -ing" lists with wrong entries
- Arithmetic on long digit strings (numbers split into odd chunks like
123+456)
The model isn't being lazy. You are asking it to inspect the inside of pieces it cannot see inside. It answers from pattern memory instead, which is confident and often wrong.
With tool use disabled, you ask a small non-reasoning model: "Reverse the string 'refrigerator' exactly." What is the most likely outcome?
Reverse the string 'refrigerator' exactly.- AA fluent, confident answer that is close but has letters wrong or misplaced
- BA flat refusal, since string reversal is against policy
- CA perfect reversal every single time
- DAn error from the API, because models cannot process quoted strings
Tokens are not just a quirk. They are the meter that bills you ⚠️
Every API charges per token: everything you send (input tokens) plus everything the model produces (output tokens). Latency scales with tokens too; the model generates output one token at a time.
A support bot's system prompt: 2,000 tokens
Traffic: 100,000 requests/day
→ 200 MILLION input tokens/day, before anyone even asks a questionOne bloated paragraph, multiplied by production traffic, is real money. Output is usually billed several times higher than input, so an unnecessarily chatty answer format costs you twice over: money AND user waiting time.
This is why professionals develop token awareness: not miserly, just deliberate. Every sentence on the intern's desk should be earning its place. Module 10 turns this instinct into a full cost-engineering toolkit (caching, routing, output caps).
Complete the cost rule every production prompt engineer knows by heart: API usage is billed per ____, on both what you send and what the model generates.
Billing unit = the ____ (singular; input + output)- A tokenizer splits your text into tokens (about 3 to 4 characters or one common word each) before the model sees anything. The model reads token IDs, never letters.
- Common words are single cheap tokens; rare words shatter into many fragments.
- Character-level tasks (counting letters, exact reversal) fight the representation and fail in a characteristic way: fluent, confident, almost right. Do exact character work in code.
- Tokens are the unit of cost and latency: input + output, with output typically pricier. Token awareness is a professional habit.
Next: where all those tokens actually live, and why the model forgets you between sessions: the context window.