Context Window

Imagine you are about to answer a complex question. You spread relevant documents across your desk to reference as you work. Your desk has a fixed surface area - at some point, you simply cannot fit any more papers on it. Anything that falls off the edge is gone from your view. An AI's context window is exactly that desk.

Every time you send a message to an AI, it reads the entire conversation from the beginning before it responds. It sees your latest message, everything you said before, everything it said before, and any documents you pasted in. But only up to a limit. Beyond that limit, earlier parts of the conversation simply disappear from the AI's view - it is not that it forgets in the way a human forgets, it is more that those things are no longer in the room when it is thinking.

The size of a context window is measured in units called tokens, which are roughly equivalent to common words. A small context window of around 8,000 tokens can hold about 30 pages of text. A large one of 200,000 tokens - like the one in some of the latest Claude models - can hold the entire text of a long novel. The bigger the window, the more the AI can hold in view at once.

Bigger windows do come with trade-offs. Running AI with a very large context window is slower and more expensive. And research has found that AI models do not pay equal attention to everything across a very long context - they tend to focus better on content near the beginning and end, and sometimes miss important details buried in the middle of a very long document.

For most everyday uses - asking questions, drafting emails, summarising short reports - context window size does not matter much. But for tasks like analysing a long legal contract, reviewing an entire codebase, or having a very long research conversation without losing track of earlier details, the size of the context window determines what is and is not possible.