The idea
AI agents working for hours fill their context window. Today's fixes come from outside the model: automatic summaries, rule-based compaction, external memory and RAG. A paper posted to arXiv on September 29, 2026 by researchers from the University of Washington, Meta Superintelligence Labs, MIT and Trillium Labs, covered by InfoQ on October 11, proposes something else: Context Language Models (CLMs).
In a CLM, context is treated as a file the model can rewrite freely: delete useless intermediate results, keep key facts, write progress notes, log failed experiments. The model learns what's worth keeping instead of following a fixed rule. In multi-agent systems, each agent's context is a file.
Code is open on GitHub (facebookresearch/context-language-models, ~680 stars).
Reported results
Zero-shot, with existing models:
- BrowseComp-Plus: +11.4% accuracy with 21.5% fewer FLOPs.
- 12-hour EdgeBench: +5% score with 59% fewer FLOPs.
- 24-hour multi-repo agent-swarm task: 65% more improvement at the same compute.
Also:
- Natural-language instructions refined in a skill-optimization loop improved accuracy by up to 35.9 points on a context-management task.
- Reinforcement learning took Qwen3.5-9B from 28.8% to 42.5% on BrowseComp-Plus (+47.6% relative) with 12% fewer FLOPs.
- A serving mechanism, Suffix Cache Reuse, cut server compute 35% versus standard SGLang.
Caveats
The authors note:
- The model may discard important information it can't later recover.
- Editable context becomes a new prompt-injection channel: malicious or self-generated instructions can persist across turns.
- More control doesn't guarantee better decisions about what to keep.
Critics add that gains come from specific tasks and that rewriting context invalidates the model cache, raising production inference cost; the authors' caching trick helps, with drawbacks.
Why it matters
Alongside MemTensor's Metis and other proposals this season, CLMs show a clear trend: memory and context are moving from the harness into the model. For agent builders:
- Context is cost: measure tokens and FLOPs per task as carefully as accuracy.
- Test context strategies (summaries, progress notes, pruning) on your own workflows; the paper shows good instructions alone help.
- Treat context as attack surface: if an agent can rewrite its notes, malicious content can stick. Validate and log those edits.
- Watch the cache: rewriting early context can wipe out prompt-caching discounts.
Sources
- arXiv 2609.37725 (Context Language Models): https://arxiv.org/abs/2609.37725
- GitHub, facebookresearch/context-language-models: https://github.com/facebookresearch/context-language-models
- InfoQ (2026-10-11): https://www.infoq.com/news/2026/10/context-language-models/
Want to optimize the cost and reliability of your AI agents? KGA IT offers a free consultation: kga-it.com.