Generative AI · Fast-moving · Beginner

Context Window

The maximum number of tokens a model can consider at once, covering the system prompt, conversation, retrieved documents and its own reply.

What Context Window is

The context window is the model's working memory for a single request. Anything outside it simply does not exist for that call, which is why long chats lose earlier detail.

How it works

Window sizes are architectural and serving decisions; longer windows raise memory and latency costs. Applications manage the budget with summarisation, truncation, retrieval and structured memory rather than pushing everything in.

Why it matters

Most disappointing assistant behaviour in production is a context management problem, not a model quality problem.

Common uses

→Long document analysis
→Multi-turn conversation design
→Codebase-wide reasoning
→RAG budget planning

Strengths

✓Bigger windows simplify document workflows

Watch for

✓Accuracy can fall for facts buried mid-context
✓Cost and latency rise with length

Continue exploring

Tokenization Retrieval-Augmented Generation AI Memory Attention Mechanism KV Cache