Context Optimization Techniques

Optimization Strategies

Compaction

if context_tokens / context_limit > 0.8:
    context = compact_context(context)

1. Tool outputs → replace with summaries
2. Old turns → summarize early conversation
3. Retrieved docs → summarize if recent versions exist
4. Never compress system prompt

• Tool outputs: Preserve findings, metrics, conclusions
• Conversational: Preserve decisions, commitments, context shifts
• Documents: Preserve key facts, remove supporting evidence

Observation Masking

if len(observation) > max_length:
    ref_id = store_observation(observation)
    return f"[Obs:{ref_id} elided. Key: {extract_key(observation)}]"

• Never mask: Current task critical, most recent turn, active reasoning
• Consider: 3+ turns old, key points extractable, purpose served
• Always mask: Repeated outputs, boilerplate, already summarized

KV-Cache Optimization

# Cache-friendly ordering: stable content first
context = [
    system_prompt,      # Cacheable
    tool_definitions,   # Cacheable
    reused_templates,   # Reusable
    unique_content      # Unique
]

• Avoid dynamic content (timestamps)
• Use consistent formatting
• Keep structure stable across sessions

Context Partitioning

# Each sub-agent has clean, focused context
results = await gather(
    research_agent.search("topic A"),
    research_agent.search("topic B"),
    research_agent.search("topic C")
)
# Coordinator synthesizes without carrying full context
synthesized = await coordinator.synthesize(results)

Budget Management

context_budget = {
    "system_prompt": 2000,
    "tool_definitions": 3000,
    "retrieved_docs": 10000,
    "message_history": 15000,
    "reserved_buffer": 2000
}
# Monitor and trigger optimization at 70-80%

When to Optimize

Performance Targets

• Compaction: 50-70% reduction, <5% quality loss
• Masking: 60-80% reduction in masked observations
• Cache: 70%+ hit rate for stable workloads

Best Practices

1. Measure before optimizing
2. Apply compaction before masking
3. Design for cache stability
4. Partition before context becomes problematic
5. Monitor effectiveness over time
6. Balance token savings vs quality
7. Test at production scale
8. Implement graceful degradation