Concept
The model has no memory — only whatever fits in the window.
Every turn re-sends the conversation. When it no longer fits, something has to be dropped.
Concept
Tokens & Context Windows
Learn what a token is, how context windows fill up, and why cost and quality both hinge on context. Free, one sitting.
Start Quick LessonFree to explore · ~3 minutes
Why this matters
Teams routinely ship AI features without knowing what a request costs or why quality falls apart in long conversations. Both trace to the same thing: the context window. Understand tokens and you can forecast spend, spot the moment a chat starts losing the thread, and design around it.
What you'll cover
What a Token Is
- Tokens are not words
- How text becomes tokens
- Code and non-English text
- Counting tokens before you send
- Why letter counting is hard
The Context Window
- Everything the model can see
- There is no memory outside it
- Input and output both count
- Window size across models
- Long context is not free
Where the Money Goes
- Input tokens vs output tokens
- The cost of a long conversation
- Cached tokens
- Forecasting a monthly bill
- Finding your expensive calls
When the Window Fills
- What gets dropped
- Why long chats lose the thread
- Summarising history
- Position within the window
- Detecting context loss
Budgeting Context
- Allocating the window
- Trimming retrieved content
- Shortening system prompts
- Choosing output length
- A context budget in practice
What you'll understand
- Explain what a token is and how text maps to tokens
- Estimate the cost of a request before you send it
- Say why long conversations lose earlier detail
- Design around context limits instead of hitting them
A look inside
Three moments from this Quick Lesson
The real thing — not a mockup of it.
How it fits
Input tokens + Output tokens = Your bill
Both are metered, usually at different rates, on every single call.
Apply
A long chat starts forgetting earlier details. Why?
The model is tired
Earlier turns fell outside the context window
The temperature is too high
How it works
01
Understand the concept
A plain-language walkthrough of the idea itself, no prior context assumed.
02
See it in practice
A simple diagram or example showing how it actually fits together.
03
Apply what you learned
One quick check that you can recognise it, not just recall it.
Useful for
DevelopersProduct managersFoundersFinance and ops partners
Ready to understand Tokens & Context Windows?
Want to go deeper? Explore AI Product Manager →
Frequently asked
Is a token the same as a word?+
No. A token is roughly three-quarters of an English word on average, but punctuation, code, and non-English text tokenize very differently.
What will I learn?+
How tokens are counted, how context windows fill and truncate, and how to forecast cost and quality from both.
Is this Quick Lesson free?+
Yes. Quick Lessons are free, short, and do not require a paid plan — sign in only to save your progress.