What Are Large Language Models?
Understand what LLMs are, how one simple training trick produces them, and what they fundamentally can and cannot do.
Welcome to LLM Fundamentals 🤖
You've probably already used an AI assistant to explain an error message, draft an email, or write a function. It can feel like magic. This course opens the box: by the end, you'll understand what's actually happening inside, and you'll be able to design real AI features on top of it.
The journey:
- How LLMs work (this module): tokens, cost, context windows
- Prompting essentials: getting reliable output on purpose
- Embeddings and RAG: connecting models to your data
- Fine-tuning: when to customize the model itself
- AI safety: hallucinations, attacks, privacy
- Final project: design a complete AI Knowledge Assistant
No machine learning background is required. A little Python appears along the way, and every snippet is short and readable.
Start with the single most important fact in this entire course.
A large language model (LLM) is a neural network (a program that learns patterns from examples instead of following hand-written rules) trained on massive amounts of text to answer one question, over and over: given everything so far, what small chunk of text most plausibly comes next? (Each small chunk is called a token; a whole lesson on them is coming shortly.)
Input: "The capital of France is"
Output: " Paris" (by far the most likely continuation)That's it. It is not looking anything up in a database. It is not searching the internet. It learned statistical patterns from trillions of words, and it continues your text the way those patterns suggest.
The surprise of the last few years: practicing this one prediction task at enormous scale produces grammar, world knowledge, working code, and multi-step reasoning as emergent side effects. Think of an LLM as the world's most powerful autocomplete: so good at "what comes next" that it can finish a legal brief, a poem, or a Python function.
What is the core training objective of a large language model?
- APredicting the next token given all the previous tokens
- BRetrieving facts from a structured knowledge database
- CClassifying text into predefined categories
- DMemorizing question-and-answer pairs word for word
Here's the flip side of "powerful autocomplete", and it explains most AI failures you'll ever debug.
Because the model predicts plausible text rather than true text, three limitations are built in:
- It can be confidently wrong: a false statement that sounds plausible is generated with exactly the same fluent confidence as a true one
- It has a knowledge cutoff: training ended on a specific date, so it knows nothing after that
- It cannot access the internet or your files unless the application explicitly wires in tools to do so
An LLM is not a database that returns errors when a record is missing. When it doesn't know, it usually produces its best-sounding guess anyway. Module 6 is devoted to handling this, but keep it in mind from day one: plausible is not the same as true.
You ask an LLM about your company's internal vacation policy. The policy was never on the public internet. What most likely happens?
- AIt generates a plausible-sounding generic policy that may not match yours at all
- BIt returns an error because the document is not in its database
- CIt secretly searches your company's intranet for the document
- DIt refuses, because models never answer questions about companies
Which models are we talking about? Specific model names change every few months, so it pays to think in families. As of mid-2026, the major ones are:
- GPT (OpenAI): general-purpose, multimodal
- Claude (Anthropic): reasoning, safety, long documents
- Gemini (Google): very long context windows
- Llama (Meta): open weights you can run on your own hardware
All of them are LLMs built on the same next-token principle. What they're good at in practice:
- Answering questions and explaining concepts across domains
- Writing, summarizing, translating, and editing text
- Generating, explaining, and debugging code
- Reasoning through multi-step problems
- Taking on specific personas and formats when instructed
A model states, fluently and confidently: "The Eiffel Tower was completed in 1901." (It was 1889.) What best explains how this happens?
- AA wrong year can still be a statistically plausible continuation, and the model generates plausible text, not verified facts
- BThe model's fact database has a corrupted record for the Eiffel Tower
- CThe model is programmed to occasionally lie to seem more human
- DThis can't happen: confident phrasing means the model verified the fact
- A large language model (LLM) is a neural network trained on one task: predict the next chunk of text given everything so far
- Language skill, world knowledge, code, and reasoning are emergent side effects of that training, not hand-programmed features
- The model generates plausible text, which is not the same as true text, and it sounds equally confident either way
- Built-in limits: it can be confidently wrong, it has a knowledge cutoff, and it can't reach the internet or your files on its own
- Think in model families (GPT, Claude, Gemini, Llama), because individual model names change constantly
Next up: watching the model actually generate an answer, one token at a time, and the dial that controls how random it gets.