Context rot is the degradation of an agent's performance as its context window fills up — even when the total token count stays well under the model's technical limit. Task coherence erodes as older but still-relevant information gets buried under newer tokens or quietly drops out at the edges of the model's attention. This is measured, not anecdotal: Chroma's "Context Rot" technical report evaluated 18 models (including GPT-4.1, Claude 4, Gemini 2.5) and found performance degrades — increasingly unevenly — as input length grows, even on tasks as simple as repeating text, and the "Lost in the Middle" study showed accuracy drops sharply when relevant information sits in the middle of a long context rather than at the edges. In practice, agents start failing well before the advertised window is full.
Why it happens
Attention is not uniform across a long window. Models reliably attend to the start and end of the context ("primacy" and "recency") but degrade in the middle — the "lost in the middle" effect. As an agent loop appends tool results, retrieved documents, and prior turns, the instructions and facts that mattered on turn 3 sink into that low-attention middle by turn 30. Nothing was truncated, so it looks fine — but the model has effectively stopped using it.
How to prevent it
1. Compaction over accumulation. When the window approaches a threshold (not the hard limit), summarize the older turns into a compact state and reinitialize with that summary. Keep verbatim only what must stay verbatim — open sub-goals, the active plan, unresolved errors.
2. Externalize memory. Offload durable facts to a store (files, vector DB, scratchpad) and retrieve on demand. The window should hold the working set for the current step, not the entire history.
3. Structure the window by position. Pin the system prompt and current task at the top; keep the most recent tool output at the bottom. Put reference material that must survive in those high-attention zones, not the middle.
4. Evict aggressively. Drop stale tool outputs, resolved errors, and superseded plans. A smaller, cleaner window outperforms a larger, noisier one — fewer distractor tokens means sharper attention on what matters.
5. Delegate to subagents. Hand a bounded sub-task to a fresh agent with its own clean window, and return only the result. This is compaction by architecture: the parent never sees the sub-task's intermediate churn.
The one-line rule
Context engineering asks what tokens should occupy the window at every moment — including what to evict, compress, or delegate. Treat the window as a scarce, actively-managed resource, not an append-only log, and context rot stops being your top failure mode.
Sources: Chroma — Context Rot: How Increasing Input Tokens Impacts LLM Performance · Liu et al. — Lost in the Middle (TACL 2024) · Anthropic — Effective context engineering for AI agents.
Related: context compaction, token budgets, agent memory tiers, multi-agent orchestration.