Learning path

Builder

Write the code: a tiny GPT from scratch, then an autonomous loop that fixes itself.

28 lessons · 175 min left

Start path → What a large language model does

FoundationsLesson 2: What a large language model doesAn LLM is autocomplete that swallowed a library. Understanding that one sentence explains most of its strengths and every one of its weaknesses.Beginner4 min read🧪 Lab📝 TestUp next
Build an LLMLesson 36: Which LLM can you actually build?There are five honest tiers, separated by three or four orders of magnitude in cost. Pick yours before you write a line of code.Beginner6 min read🧪 Lab📝 Test
Build an LLMLesson 37: Step 1 · The tokenizerBefore a model sees language, something must chop text into pieces. Get this wrong and everything downstream is quietly worse.Middle6 min read🧪 Lab📝 Test
Build an LLMLesson 38: Step 2 · Embeddings and positionTurning token ids into vectors, and telling the model what order they came in. Two lookup tables, one subtle idea.Middle5 min read🧪 Lab📝 Test
Build an LLMLesson 39: Step 3 · Attention, derived slowlyThe one idea the whole field rests on. Three vectors per token, one dot product, one softmax. Click the diagram to walk through it.Middle7 min read🧪 Lab📝 Test
Build an LLMLesson 40: Step 4 · The transformer blockAttention plus a small feed-forward network plus two normalisations plus two residual connections. Repeat N times. That is the entire model.Middle5 min read🧪 Lab📝 Test
Build an LLMLesson 41: Step 5 · A complete tiny GPTEvery piece assembled into a model you can actually train tonight. Under 100 lines, and structurally identical to a frontier model.Middle7 min read🧪 Lab📝 Test
Build an LLMLesson 42: Step 6 · The training loop that convergesThe model is the easy part. This module is the one that decides whether your run works or wastes a week.Middle7 min read🧪 Lab📝 Test
Build an LLMLesson 43: Step 7 · Data is the productArchitecture is nearly free — you can copy it. Data is where models actually differ, and where almost all the work is.Middle6 min read🧪 Lab📝 Test
Build an LLMLesson 44: Step 8 · Scaling laws and the compute budgetHow to decide model size and dataset size before spending money. Move the sliders and watch the bill.Advanced6 min read🧪 Lab📝 Test
Build an LLMLesson 45: Step 9 · Training across many GPUsOne model no longer fits on one card. Four ways to split it, and the rule for choosing between them.Advanced6 min read🧪 Lab📝 Test
Build an LLMLesson 46: Step 10 · Making it fastSame model, same GPUs, three times the throughput. Most training runs leave that on the table.Advanced7 min read🧪 Lab📝 Test
Build an LLMLesson 47: Step 11 · Modern architecture upgradesWhat separates a 2019 transformer from a 2026 one. Six changes, each small, together substantial.Advanced6 min read🧪 Lab📝 Test
Build an LLMLesson 48: Step 12 · Long contextGoing from 4k to 1M tokens is not one trick. It is four, and each one costs something.Advanced6 min read🧪 Lab📝 Test
Build an LLMLesson 49: Step 13 · Teaching it to follow instructionsA base model completes text. It does not answer questions. Supervised fine-tuning is the step that turns one into the other.Middle6 min read🧪 Lab📝 Test
Build an LLMLesson 50: Step 14 · Preference optimisationSFT teaches the model what a good answer looks like. Preference training teaches it which of two good answers is better — and that is where character comes from.Advanced7 min read🧪 Lab📝 Test
Build an LLMLesson 51: Step 15 · Reasoning modelsThe newest chapter. Instead of buying capability with a bigger model, buy it with more thinking at the moment of answering.Advanced6 min read🧪 Lab📝 Test
Build an LLMLesson 52: Step 16 · LoRA, the path most people takeTrain 0.1% of the parameters, get most of the benefit, on one consumer GPU. This is the module with the highest practical value in the whole track.Middle6 min read🧪 Lab📝 Test
Build an LLMLesson 53: Step 17 · Evaluate, shrink, serveA model that is not measured is not finished. A model that costs too much to run never ships. Both problems, in one module.Middle6 min read🧪 Lab📝 Test
Build an AGILesson 61: Three tiers of digital cognitionA model that talks, a model that checks itself, and a system that acts on its own. Most confusion about AGI comes from mixing these three up.Beginner7 min read🧪 Lab📝 Test
Build an AGILesson 62: The model proposes, the compiler disposesThe single sentence that turns an unreliable text generator into a system you can trust with real work.Middle6 min read🧪 Lab📝 Test
Build an AGILesson 63: The OODA loop and the four pillarsThe architecture. Four software components in a continuous cycle — and the reason it is a cycle rather than a pipeline.Middle7 min read🧪 Lab📝 Test
Build an AGILesson 64: Hands on: the mini-AGI engineOne file, about a hundred lines. It plans, writes code, runs it, reads the error, and fixes itself. Everything above, made real.Middle8 min read🧪 Lab📝 Test
Build an AGILesson 65: The bug hiding in your verifierThe loop above declares victory as soon as the code runs without crashing. Running is not the same as being right, and that gap is where self-improving systems go wrong.Advanced7 min read🧪 Lab📝 Test
Build an AGILesson 66: Making the executor a real sandboxThe tutorial's executor runs attacker-influenceable code on your machine with your credentials. Here is what has to change before it touches anything you care about.Advanced7 min read🧪 Lab📝 Test
Build an AGILesson 67: Working memory that survives the loopA growing string is fine for three iterations and useless for three hundred. Splitting state from retrieval is what lets an agent work for hours instead of minutes.Advanced7 min read🧪 Lab📝 Test
Build an AGILesson 68: Teaching this to the next buildersHow to mentor someone — a curious teenager or a junior engineer — so they build systems that matter instead of another wrapper around a chat box.Beginner6 min read🧪 Lab📝 Test
Agent securityLesson 18: Sandboxing: the room with no windowsYou cannot stop the agent being fooled. You can decide what room it is standing in when it happens.Middle5 min read🧪 Lab📝 Test