Compaction illustrated

Budget Reduction + Microcompact + Snip implemented Context Collapse / Reactive pending
Code: context/engine.py, context/budget.py, context/microcompact.py, context/tool_aging/, context/snip.py
Reference: Claude Code compaction reference(five-tier cascade, based on reverse engineering)

Current rolling-summary constraints

Compaction uses the same DAG visibility, tool-result rendering and aging rules as model requests. Top-level conversation defines only the replaceable range; each range's budget includes actually visible subcalls. Always retain the latest complete user turn, and retain the latest two by default; the token budget may allow fewer than twenty top-level messages. Short history already below the retention budget does not invoke the summary model.

When keeping two turns leaves more than the requested token budget, select the next complete user-turn boundary and retain the latest turn. This avoids a large completed tool-heavy turn making compaction ineffective. Never split an active user turn or discard its tool evidence without including it in the summary. The completion event reports both token counts and whether a summary was actually applied.

Summary input starts at the first message of the selected range and includes original user requests, the prior summary and visible tool results. Summary input and covers_ids describe the same range; no removed prefix is omitted from model input. The previous summary comes only from records applicable to the current branch, not a session cache from another branch.

After generation, perform the same replacement and rendering in the in-memory graph and persist only if estimated tokens strictly decrease. Estimates include summary markers and tool results; compaction markers and the context panel use the same message rendering. Failure, cancellation, empty summary, no gain or content changes during generation neither commit a summary nor increment compaction count. Failures do not substitute a structural summary that discards content.

Storage retains one rolling summary and existing covers_ids without migrating or deleting history. Repeated compaction incorporates the currently applicable summary; existing DAG rendering rules preserve branch isolation, unchanged HEAD and tool-call/result pairing. Historical incorrect summaries are not automatically rewritten.

Acceptance and boundaries

With large tool results among sixteen top-level messages, summary coverage includes complete older turns and actual context shrinks.
The summary receives every replaced user request and visible tool result; recent turns stay unchanged.
Equal-length or longer summaries, cancellation, provider errors and edits during asynchronous work do not write.
Repeated compaction, branch switching, occupancy statistics and final requests render consistently.

Reference sources remain those already compared on this page; this change adds no external implementation or changes to other multi-tier compaction policies.

Implementation status

These corrections are implemented. Regression tests cover complete summary input, retaining the latest two turns, rejecting no-gain summaries, cancellation/concurrent changes, repeated compaction and statistics consistency. Session-graph compaction scope uses the same message snapshot, avoiding inconsistent summary state if compaction repeats during loading.

Node types 🔵 Conversation node 🟢 Function node 🟡 Tool node

1. Overview: routine operation + five-tier cascade

Aligned with Claude Code, with two categories:

Routine operation(before every LLM call, regardless of context length): Budget Reduction

Cascading compaction(escalate through Tiers 1→2→3→4→5, cheapest first, advancing only if the previous tier is insufficient):

	Name	Action	Trigger	Calls LLM	Invalidates cache	Status
Routine	Budget Reduction	Truncate oversized individual outputs	Individual tool_result > 4000 characters	No	No	✅
Tier 1	Microcompact	Clean up old tool outputs	Tool calls ≥ 50 calls(then every 25 calls) / idle for 90 minutes	No	Cache-aware preserves cache	✅
Tier 2	Snip	Remove the oldest turns	Still over budget after Tier 1	No	Yes	✅
Tier 3	Context Collapse	Segment summaries	Still over after Tier 2, ~90%	Yes (once per segment)	Yes	⏳
Tier 4	Auto-Compact	Full summary	Still over after Tier 3 / user `/compact`	Yes (once)	Yes	✅
Tier 5	Reactive	Emergency compaction	API returns 413	Yes	Yes	⏳

Cascade logic: before every LLM call, check from Tier 1. Execute when conditions match, then recheck the budget. Advance to the next tier if still over.

Note: reverse engineering describes Claude Code Tiers 3 and 4 as overlapping strategies, not necessarily a strict cascade./compact The manual command directly triggers Tier 4, skipping Tiers 1–3.

2. Our addition: function-call node lifecycle

Claude Code uses linear conversations without a DAG and does not need this layer. Our function-call nodes (🟢) require additional handling.

2.1 While a function runs

Trigger: Function called, status="running"
Action: Internal subtree fully expanded in context
Visibility: Controlled by render_context callers/subcalls/expose
Node state: Internal view: full children.External view: expose=io by default; callers see input and output only
Relationship to compaction: While the function runs, internal 🟡 tool outputs still undergo Budget Reduction (§3) and Microcompact (Tier 1). They do not participate in Snip because active content cannot be deleted.

2.2 Function completes → expose=io, then normal cascade

Trigger: Function returns, status="completed"/"error"
Action: Expose=io naturally hides the internal subtree with no extra operation. Children remain inspectable in the DAG.
Afterward: The function node (input + output) enters the normal cascade of §3-§4 , treated the same as 🔵 Conversation node . No separate policy is needed.

3. Routine operation: Budget Reduction ✅ Implemented

Matches Claude Code: runs before every LLM call, regardless of context length.

Action: Truncate oversized individual 🟡 tool outputs. Does not process 🔵 user/assistant messages
Threshold: 4000 characters(about 1000 tokens)
Truncation method: Retain the first 2400 characters(60%) and last 1600 characters(40%), discarding the middle
Code: context/tool_aging/truncate.py: middle_truncate, context/tool_aging/policy.py: MAX_TOOL_RESULT_CHARS = 4000

4. Cascading compaction (Tiers 1→5)

After Budget Reduction, check tiers in order. Advance only if the preceding tier leaves context over threshold.

Tier 1: Microcompact ✅ Implemented

Matches Claude Code: clean up old 🟡 tool outputs.

Trigger: Cache-aware: 50 tool calls trigger the first pass, then every 25 calls. Time-based: idle for ~90 minutes
Processes only: FileRead, Shell, Grep, Glob, WebSearch, WebFetch, FileEdit, FileWrite
Two paths: Cache-aware(Anthropic provider): clear old tool_result on the server through the Context Editing API (cache_edits, beta header context-management-2025-06-27). Client messages stay unchanged,cache remains valid. Sentinel bytes are normalized for byte stability; repeated operations do not change cached content.
Time-based(other providers / 90-minute idle): replace client-side content with placeholder text (~20 tokens).Invalidates cache
Compaction effect: Each tool output: original size → 0 tokens (cache-aware) or ~20 tokens (time-based)
Code: context/microcompact.py, context/rules/microcompact.py

Tier 2: Snip ✅ Implemented

Matches Claude Code: directly remove the oldest turns without summaries or LLM calls.

Trigger: Still over budget after Tier 1
Removal unit: Complete turns(user + assistant + all tool calls in the turn removed together)
How much to remove: Remove turns oldest first and recalculate tokens after each until below threshold; no fixed N-turn count
Cost: Information is completely lost and the cache prefix is invalidated, but no LLM call is needed.
Function nodes: Completed 🟢 Function node participate in Snip chronologically. Running nodes are retained.
Code: context/snip.py

Tier 3: Context Collapse ⏳ Pending

Matches Claude Code: segmented LLM summaries, retaining original messages for rollback.

Trigger: Still over after Tier 2 Snip (about 90%; mandatory blocking at 95%)
Segmentation: 5–10 turns per segment, independently summarized by the LLM (about 100–300 tokens each)
LLM calls: Count = number of segments, e.g. 30 turns in 3 segments = 3 LLM calls
Reversible: Original messages remain in the collapse store through read-time projection

Tier 4: Auto-Compact ✅ Implemented

Matches Claude Code: full LLM summary replacing all old messages, without rollback.

Trigger: Still over after Tier 3, or the user manually invokes /compact (jump directly here)
LLM calls: 1 call(full summary)
Summary size: About 2000-5000 tokens
Retain: The latest 1–2 turns in full. Reload the system prompt from configuration rather than relying on the summary.
/compact command: /compact or /compact retain the database-migration discussion. Manual use is recommended at 60% .
Chained compaction: Continue after compaction; when context fills again, summarize using the previous summary. Information degrades over successive summaries.
Code: context/engine.py: compact()

Tier 5: Reactive ⏳ Pending

Emergency compaction when the API returns 413 (prompt too long), at most once per turn. Stop if all attempts fail.

5. Execution flow

Before every LLM call:

1. Budget Reduction → truncate oversized tool outputs (every turn)

2. Cascade checks (tier order):
   Tier 1: Microcompact condition met? → clean old tool outputs
   Tier 2: Still over threshold? → Snip older turns
   Tier 3: Still over? → Context Collapse segment summaries
   Tier 4: Still over? → Auto-Compact full summary

3. Call the LLM

4. If the API returns 413:
   Tier 5: Reactive → emergency compaction → retry
   → stop on failure

6. Complete example

Scenario: 200K context, several hours of work, with research_agent invoked midway.

Stage 1 (0–30%): normal conversation

🔵 user → 🔵 assistant; Budget Reduction checks tool outputs and skips those within limits. No cascade tier triggers.

Stage 2 (30–50%): Tier 1 Microcompact begins

Tool calls exceed 50. Cache-aware Microcompact triggers for the first time, cleaning old outputs and freeing about 15K tokens. It repeats every 25 calls. Tiers 2–5 do not trigger.

Stage 3: invoke 🟢 research_agent

While the function runs, internal 🟡 tool calls expand. Budget Reduction truncates oversized outputs. On completion, expose=io hides the internal subtree and the node enters normal cascading.

Stage 4 (over threshold): Tier 2 Snip triggers

Microcompact leaves context over budget. Snip removes the oldest five turns, including completed function nodes, bringing it below threshold. Tiers 3–5 do not trigger.

Stage 5 (extreme): Tier 3/4 trigger

After repeated Snip, little removable content remains. Context Collapse summarizes segments; if insufficient, Auto-Compact summarizes everything. This is rare.

Stage 6: continue using the system

Conversation continues normally after compaction. Repeat the cascade when over threshold again. The model can inspect old function internals by reading DAG children through tools.

7. Mapping to Claude Code

Ours	Claude Code	Description
§2 Function-node lifecycle	None	Our addition (DAG + expose=io)
§3 Budget Reduction	Budget Reduction	Matches (4000-character threshold, 60/40 truncation)
Tier 1 Microcompact	Tier 1 Microcompact	Matches; Anthropic uses the cache-aware path
Tier 2 Snip	Tier 2 Snip	Matches (remove whole turns)
Tier 3 Context Collapse	Tier 3 Context Collapse	Matches (segment summaries with rollback)
Tier 4 Auto-Compact / /compact	Tier 4 Auto-Compact / /compact	Matches (full summary)
Tier 5 Reactive	Tier 5 Reactive	Matches (413 emergency compaction)

Source:Claude Code compaction reference(based on reverse engineering, not official documentation)

8. Implementation plan

Step	Action	Status	Code
1	Budget Reduction — truncate oversized outputs (§3)	✅	`context/tool_aging/truncate.py`
2	Tier 1 Microcompact — clean old tool outputs	✅	`context/microcompact.py`
3	Tier 2 Snip — remove oldest turns	✅	`context/snip.py`
4	Tier 3 Context Collapse — segmented LLM summaries	⏳
5	Tier 4 Auto-Compact — full LLM summary	✅	`context/engine.py`
6	Tier 5 Reactive — 413 emergency compaction	⏳
7	/compact command	⏳
8	/context command (token distribution)	✅	`cli/repl/handlers.py`
9	Anthropic Context Editing API integration	⏳

Stage	Behavior	Boundary
Budget	Count system prompt, tools, response schema, output reserve and history before every model request.	Recalculate for fallback models.
Tool evidence	Summarize complete text-only tool results first.	Preserve call identifiers and error flags.
Checkpoint	When still over budget, replace an older complete text-only execution prefix with a task summary; retain every user message verbatim and the latest assistant segment.	Never split tool groups or discard multimodal or unresolved content.
Summary request	Process all source text in budgeted chunks with rolling summaries. Retry a provider overflow using a smaller chunk.	Bound calls and time; cancellation, errors and exhausted budgets preserve original history.
Continue	Reuse checkpoints only for an identical source prefix and model. Send only a verified smaller request.	Keep raw execution messages and DAG nodes unchanged.