Compaction#

Compaction keeps a long conversation inside the model's context window by replacing the oldest turns with one LLM-written summary. This document is the single authority on how compaction stores its result, how the result changes what the model reads, how it composes with branches and with repeated compaction, and which invariants protect it. The DAG's visual treatment of a summary (the capsule) is specified in dag/rendering.md §9; this document defines the data and semantics that rendering consumes.

1. The model: a rolling summary, exactly one active#

A session has at most one active summary at a time. Compacting again does not stack a second summary on top of the first: the summariser receives the previous summary text as input, absorbs it, and produces a replacement. The session's extra_meta._last_summary_id names the active summary; extra_meta._last_summary_text carries its text for the next chaining. Every older summary node stays on disk as an inert relic, flagged for the graph as superseded_summary and never consulted again.

The next LLM request therefore always has the shape

[system prompt] [active summary] [kept tail, verbatim] [new user message]

— one summary, never a stack, followed by the turns it did not eat.

2. Data model: an append-only stand-in#

Compaction writes exactly one node and mutates nothing:

Rules that follow from append-only:

• No clones. The kept tail keeps its ids and predecessors untouched. A cloned tail would mint a second id space every consumer must translate.
• No edge rewrites. The covered nodes stay on the chain exactly as written; the first kept node still points at the last covered node. The pre-compaction view is always reconstructible by ignoring the summary.
• No head movement. Compaction is a pure insert. HEAD stays on the branch tip it was on; the summary changes what a render of that branch produces, not which branch is active.
• Ids, not seq intervals. covers_ids is the record of what was summarised. A seq interval cannot express this in a DAG — seqs of sibling branches interleave, so any interval sweep drags dead forks into the coverage and its answer changes when HEAD moves. The interval form (metadata.covers = [first_seq, last_seq]) does not exist in this design; nothing reads or writes it.
• covers_ids is a contiguous chain segment of real turns. It never contains another summary node. When a re-compaction eats "the previous summary plus k more turns", the new node's covers_ids is the previous segment extended by those k turns' ids — coverage is always expressed in terms of the underlying conversation, and the old summary is retired via _last_summary_id, not via nesting.

3. The rendering rule: segment substitution#

render_context (context/nodes.py) is the one place that decides what the model reads, for chat and for runtime.exec alike. Compaction enters it as a single rule:

Let S be the session's active summary and L = covers_ids(S), a contiguous segment of a conversation chain. When rendering from head H: if every node of L lies on H's predecessor spine, drop L from the rendering and admit S at L's position (S's own splice point — its predecessor — puts it exactly where the segment began). Otherwise render the spine raw.

Properties this buys, each of which is a requirement, not a side effect:

• The summary reaches the prompt. S is admitted by rule, not by hoping the spine walk stumbles onto a node nothing points to. The rendered id list for a compacted branch is [ROOT, S, kept tail…].
• Branch isolation is automatic. A fork whose spine does not contain the whole covered segment — a retry from inside the covered range, a dead sibling from the same era — fails the ⊆ test and renders raw. Its context was never compacted, and it does not inherit a summary of turns it never had.
• HEAD-independence of storage. Checking out any branch, at any time, yields a deterministic rendering from data alone. No render result depends on where HEAD happened to be when something else ran.
• Superseded summaries are invisible here. Only the active summary is consulted; relics never elide anything.

The same rule, stated over the same covers_ids, drives the DAG's capsule fold — the graph shows the folded capsule on exactly the branches whose context carries the summary, and shows raw turns on branches that render raw. One fact, two projections.

4. The compaction pipeline#

trigger_compaction (manual /compact), auto-compact (budget ≥ 80% before a turn) and reactive compact (provider overflow error) all run the same engine.compact pipeline:

1. Freeze the rendered input. compaction_view.load_compaction_view uses render_context and render_dag_messages, including tool results, expose, spill references and the current aging boundary. Top-level turns define coverage; their visible child calls contribute to input and token budgets.
2. Select whole turns. The retained-token target is 10% of the model window, bounded to 8,000–40,000 tokens (an explicit override takes priority). The default prefers at least four conversational messages at a user boundary. A history already below the target is a no-op. If the last two turns exceed the target, the cut can advance to the latest user boundary: the completed earlier turn enters the summary and the latest user turn stays intact. If that latest turn alone exceeds the target, it stays intact.
3. Summarize the entire covered prefix. Every replaced turn, including the initial user request and any applicable previous summary, enters the model input together with its rendered child calls. Session-global summary caches are not injected into another branch. Non-text media is represented by a reference notice; compaction does not ask the summary model to interpret it.
4. Validate before writing. Reject empty, failed, cancelled, oversized or nonreducing summaries. Render a candidate summary in an in-memory copy with the same aging boundary and require a strictly smaller local token estimate. Recheck the source graph and head after the asynchronous model call; a changed source is a no-op and can be retried. Provider failure preserves the original history instead of replacing it with a lossy structural fallback.
5. Persist and report. Write one summary node; covers_ids expands any previous summary to the underlying real turn IDs. Report that same coverage count, with local before/after estimates using the provider-message renderer shared by the context panel. No-op paths write no node and do not increment compaction usage. Original history and HEAD remain unchanged.

The context display uses provider input usage for a completed request. Each request also records a local estimate of those same dispatched messages, so a later graph edit can scale its local estimate with a matched measured/estimated ratio. That scaled value remains labelled as an estimate until the next provider request reports usage. Legacy usage without a matched request estimate is not used to calibrate later graph states.

5. HEAD integrity#

Compaction was one of several writers that could move HEAD as a side effect. The design allows exactly one mover:

• Single writer. SessionStore.set_head is the only way HEAD changes, and it is called only by explicit user-facing moves: send-turn advance, retry/edit fork, checkout, rewind, branch delete. Compaction, session load, worker restart, model switch and meta saves never call it.
• Append advances HEAD only on chain extension. append_message moves HEAD to the new node only when the node's predecessor equals the current HEAD — the natural "conversation grew" case. Any other insert (a summary splice, a side-branch write, a relic) leaves HEAD alone. This replaces the old unconditional auto-advance plus per-caller snapshot/restore compensation.
• Spawned turns never move HEAD. A same-session sub-agent turn (task / send_message) runs with TurnRequest.advance_head=False: the spawn branch opens without registering itself as head, and every write the inner dispatcher makes (branch root, placeholder, reply, finalize, error) is head-neutral. The transcript follows HEAD, so a stolen head switched the user's window to the agent's conversation mid-run and mixed the two dialogues. Cross-session sends still advance the target session's own head — there the turn IS that conversation growing.
• The turn's head policy is one object. dispatcher/turn_writer.py's TurnWriter performs every chain write a turn makes and alone applies advance_head. The invariant is structural: inside the dispatcher package, set_head / update_session(head_id=…) appear only in that file (plus the manual function-run path in forced_tool.py, a user-initiated move by definition).
• Mirrors are read-only, and the transcript has one source. The webui keeps an in-memory conv mirror for sidebar metadata plus a one-shot messages snapshot taken at load_session; nothing writes to that snapshot incrementally. The live transcript is the React session store alone — stream deltas, turn results (upserted onto the <user_msg_id>_reply row) and tree hydrations all write there. The mirror never syncs back to the store or the disk: save_meta carries no head_id, and no mirror row can ever become a stored head or a stored node. Storage stays upstream of every mirror, across restarts.

6. What the graph shows#

Defined in dag/rendering.md §9; the wire contract from this side:

• The active summary row carries covers_ids — verbatim from metadata.covers_ids, extended with the caller subtrees of covered turns (a covered turn folds together with its tool calls), minus ids that no longer exist.
• Superseded summary rows carry superseded_summary: true and no covers_ids.
• The graph builder does no seq arithmetic and no head-dependent filtering; everything it says about coverage restates covers_ids.

7. Extension points#

The rolling-single-summary policy matches the reference tools (Claude Code, Codex CLI, Gemini CLI) and keeps the prompt-cache prefix stable. It is a policy, not a property of the storage: every alternative compaction scheme differs only in which summaries count as active (a policy field) and how the renderer substitutes them (the §3 rule). The append-only stand-in node is common to all of them, so switching schemes never migrates data:

• Segmented summaries (several compaction nodes kept live): N summary nodes covering disjoint chain segments; §3 applies per summary and the trunk carries N capsules. Replace _last_summary_id with an active set.
• Nested summaries (a summary of summaries): relax the "covers_ids names real turns only" rule to admit summary ids, and make substitution recursive.
• External-memory schemes (summary retrieved on demand instead of inlined): the node is stored identically; only the renderer stops inlining it.

Beneath all of these sits the contract that survives even a fully arbitrary context — one assembled by retrieval, cross-branch selection, or any future policy rather than a spine walk:

1. The DAG is the ledger, not the context. Nodes record what happened, append-only; context is a deterministic view function over them. Changing how context is built changes the view function, never the data. The renderer already deviates from the pure chain today (render_range, expose, attach/merge, memory prefetch) — each deviation is data, not hidden state.
2. Provenance is mandatory. Whatever the view function produces, the ids whose content actually entered a call's prompt are stamped on that call (reads). Replay, audit and the graph's per-node context marking depend on this record — not on the view function staying simple.

A summary node is the first instance of a view node — a node that stands in for other content in renders. Retrieved memory snippets, injected documents and cross-session references generalise it; the capsule's visual grammar (stand-in in place, expand to see the original) is the generic treatment for the class. No generic view-composition framework is built ahead of a concrete second scheme.

8. Invariants and their tests#

The scenario suite runs these flows end-to-end on a real SessionStore (chat → fork → checkout → compact → chat → compact → chat → restart-load), checking head and rendering after every step — the class of cross-module side-effect bug involved here does not show up in unit tests of the parts.

Implementation status#

The storage and rendering contracts are implemented. The revised selection and validation policy is under verification:

• §2 node shape and §4 pipeline — context/persistence.py (insert_summary_node, covered_chain_ids, rendered_history); the covers seq interval no longer exists anywhere.
• §3 segment substitution — render_context in context/nodes.py (active_summary, summary_covers_ids).
• §5 HEAD integrity — the chain-extension append rule in store/session/session_store.py; apps/server/openprogram_server/_webui/persistence.py save_meta strips head_id unconditionally and save_messages is gone; the CLI turn path writes rows through db.append_message.
• §6 graph contract — apps/server/openprogram_server/_webui/graph_builder.py.
• §8 invariants — tests/unit/context/test_compaction_covers.py, tests/unit/dag/test_graph_builder_covers.py, tests/integration/dag/test_dag_mutation_scenarios.py.

Compaction inside one user message#

When no output cap is specified, the request uses the smallest of the model's output capability, the standard output reserve (16,384 tokens), and one quarter of its context window. The same resolved cap is used for input budgeting and provider dispatch. Explicit caps remain unchanged; an oversized protected request is rejected rather than silently reducing the requested output.

Before every AgentLoop provider request, including chat tool continuations and function-local llm() / agent() calls, the runtime budgets the converted messages, actual tools, system prompt, output cap, and a safety margin. When needed, it summarizes text-only results of fully completed tool groups without waiting for another user message. User messages, tool arguments, result IDs, error flags, and multimodal content remain unchanged.

This is a request-local transformation. Original execution messages and the DAG remain intact; it does not create a durable conversation-summary node. An invocation-local cache matches original result content and model settings. Summary calls have their own input/output budgets, at most 32 requests per preparation and a 60-second deadline, and bypass AgentLoop to avoid recursive compaction. Token counts are local estimates, not a guarantee of provider accounting or semantic equivalence. If required content cannot fit, the runtime does not dispatch a request it estimates to be over budget.

Request-local checkpoints#

When tool-result summaries are insufficient, older complete text-only execution history can be replaced in the next request by a task summary. All user messages and the latest assistant segment remain verbatim. Incomplete tool groups, multimodal input and signed reasoning are not eligible for this checkpoint. Checkpoints are reused only within the current invocation for the same model and unchanged source prefix; the DAG remains the durable record.

Summary requests process long text in bounded chunks and retry context overflow with smaller chunks. They share a finite call/time budget; failures leave the original history intact. If protected content alone exceeds the window, the request fails explicitly instead of silently dropping user instructions.