Context Compaction: Query-Based and Objective Modes | Morph
Compaction drops 50-70% of an agent’s context while keeping every surviving line verbatim. Two modes: objective compaction strips filler with no guida...
AI Research
Summary
Factory Research evaluated context compression for AI agents using probe-based tests. They found structured summarization preserves more details than OpenAI or Anthropic methods during long sessions.
Highlights
- Traditional metrics like ROUGE fail to measure functional context preservation.
- Probe-based tests verify if agents recall specific details after compression.
- Structured summarization outperformed OpenAI and Anthropic in debugging tasks.
- Optimizing for tokens per task improves agent productivity over tokens per request.
- Testing covered debugging, code review, and ML research scenarios.
auto-generated
Context
Audience
AI Engineers and Developers
DomainArtificial Intelligence
Formatlong-form article
Accessfree online
Source Overview
Factory Research built an evaluation framework using probe-based tests to measure context preservation in compression strategies for long-running AI agent sessions in tasks like debugging and code review.
They tested structured summarization, OpenAI's compact endpoint, and Anthropic's SDK, finding Factory's anchored iterative summarization retains more details on facts, files, planning, and decisions. In a debugging example, Factory scored 4.8/5 for recalling the exact 401 error on /api/auth/login and root cause, outperforming Anthropic (3.9/5) and OpenAI (3.2/5).
Topics
Related
Context CompressionAI AgentsStructured SummarizationOpenAIAnthropicEvaluation Frameworks
