The State of Context Engineering
dbreunig.com ·
Context engineering has evolved from manually selecting prompt information to building software harnesses that retrieve, filter, prune, summarize, and isolate information for AI models. Dave Breunig argues that the central task is deciding which behavior should be enforced by deterministic code and which should be delegated to probabilistic model calls. Long context windows, which expanded from roughly 128,000 tokens to millions beginning with Gemini 1.5, can accommodate more data but also more errors and conflicting signals.
Breunig identifies context poisoning, distraction, confusion, and clash as major failure modes. Google reported that Gemini 2.5 playing the original Pokémon Red repeatedly pursued tea, an objective from Pokémon FireRed embedded in its training data, despite the older game requiring lemonade. Research by Carolina Camassa and Derek Shiller found that contradictory examples in a conversation can increasingly override explicit instructions. Google also reported that Gemini 2.5 began repeating historical actions rather than making new plans once Pokémon-agent context grew well beyond 100,000 tokens.
Harness techniques include retrieval-augmented generation, selective tool loading, fresh-context subagents, history pruning, external storage, and compaction. Breunig says agentic models introduce tool-environment biases and “context inertia,” in which agents persist with an unproductive hypothesis. Anthropic’s Memento tests and research from Microsoft and UC Berkeley found that periodically clearing conversational context while retaining work artifacts can restore exploration and improve results.
What people are saying
The thread zeroed in on a less glamorous reality of context engineering: giant windows do not automatically produce smarter agents.
What readers liked
- Provider-managed compaction is seen as a practical default because it offloads session-state bookkeeping while preserving a usable long-running…
- Dynamic compaction and eviction can keep local agents responsive around a bounded context size, enabling one agent to move across several repository…
Concerns raised
- Long conversations can make even frontier chat models act less capable, turning accumulated history from useful memory into a source of degraded…
- Pre-pruning a middle turn can invalidate the prefix KV cache for later turns, sacrificing the inference savings that made retaining context…
Read full discussion analysis
Source: Hacker News · Based on 4 sampled comments · Analyzed
Discussion overview
The thread zeroed in on a less glamorous reality of context engineering: giant windows do not automatically produce smarter agents. Several commenters report frontier chat models becoming noticeably less coherent after enough turns, while coding agents can also misbehave when they are starved of crucial repository or task history. The practical conclusion is that context needs active management, not indiscriminate accumulation or aggressive deletion.
A major split concerned whether a smaller decision model should prune context before compaction. Skeptics argued that this can be economically backward: the pruning model must ingest the same material, and removing a middle turn breaks prefix-based KV-cache reuse for every later turn. Others pointed out that local inference changes the tradeoff. Keeping active context near a manageable size can improve speed and let one agent work across multiple tasks, while eviction and compaction can fight the context contamination that caching alone cannot solve. Hosted provider compaction was treated as a credible default for users who prioritize simplicity, although it was not presented as a cure for all failures.
Commenters also pushed for conceptual precision: bad behavior caused by information learned during training belongs to the model’s weights, whereas poisoned context is harmful material supplied during the current interaction. Overall, the room sees context length as a powerful but fragile control knob, with a useful middle range rather than a simple more-is-better rule.
Points of disagreement
The sharpest dispute is whether pre-pruning context with a fast model is worth it. Critics say it duplicates token-loading costs and destroys valuable prefix-cache continuity, so hosted compaction or retaining history wins unless a much stronger compactor is essential. Supporters of dynamic local compaction say bounded active context materially improves inference speed and reduces context-driven failures, making eviction worthwhile despite lost cache reuse.
What readers liked
- Provider-managed compaction is seen as a practical default because it offloads session-state bookkeeping while preserving a usable long-running agent workflow.
- Dynamic compaction and eviction can keep local agents responsive around a bounded context size, enabling one agent to move across several repository tasks without carrying every past detail.
- Treating context as a deliberately tuned working set, rather than permanent transcript storage, gives coding-agent builders a concrete lever for more reliable behavior.
Concerns raised
- Long conversations can make even frontier chat models act less capable, turning accumulated history from useful memory into a source of degraded reasoning.
- Pre-pruning a middle turn can invalidate the prefix KV cache for later turns, sacrificing the inference savings that made retaining context attractive.
- An overly sparse context fails too: agents lose essential task or codebase information and start producing the same strange behavior blamed on excessive context.
- Confusing training-weight behavior with poisoned prompt context muddies diagnosis, risking the wrong fix for an agent that is failing.
Keep reading
Sources and details
dbreunig.com · Published
The State of Context Engineering
Read report →
Community analysis available for 1 of 1 linked discussions.
Additional discussion searches are still incomplete.
Coverage details and dates
Latest available community analysis:
Summaries: 1 current · 0 awaiting update · 0 not verified
Latest summary update:
Hacker News · dbreunig.com