What Is an LLM Context Window?
The context window is the maximum number of tokens a language model can process in a single request. That total includes everything: your system prompt, the conversation history you send, any data you inject (retrieved chunks, JSON payloads, documents), and the model's own response. Exceed the window and the model either truncates the input or returns an error — neither is fun to debug. Planning the budget before you build avoids the surprise.
Context Window Sizes by Model (mid-2026)
| Model | Context window | Max output |
|---|---|---|
| GPT-4o | 128K | 16K |
| GPT-4.1 | 1M | 32K |
| Claude Sonnet 4 | 200K | 32K |
| Claude Opus 4 | 200K | 32K |
| Gemini 2.5 Pro | 1M | 65K |
| Llama 4 Scout | 10M | varies |
These figures are approximate as of mid-2026 and may have changed since. Check your provider's current docs for exact limits.
How to Budget Your Context Window
- System prompt — keep it under 500 tokens if you can. Every token here is consumed on every request, so a bloated system prompt scales your bill linearly with call volume.
- Conversation history — grows with each turn. Implement a sliding window that drops the oldest turns once you cross a threshold, or summarize earlier turns into a single compressed message.
- Data / context — RAG chunks, retrieved documents, structured payloads. This is typically the biggest and most variable section. Compress structured data with a token-efficient format like TOON — it saves 30–60% on uniform-shape data with no loss.
- Output— reserve enough room for the response you actually expect. If the user asks for "a detailed analysis," budget 2,000–4,000 tokens; if you're classifying to a single word, 100 is plenty.
Related Tools
- Count exact tokens with the Token Counter.
- Reduce data payload tokens with the JSON ↔ TOON Converter.
- Format your system/user/assistant messages for the API with the Prompt Formatter.
Privacy
Your text stays in your browser. Tokenization runs entirely client-side; nothing is sent to any server or API.