# Overview **Assembles conversation history + current input into what gets fed to the LLM on each call.** Upstream of [`../providers/`](../providers/): context produces a `Context` (system / messages / tools), and providers translate it into each vendor's wire request. Contract = `Context` (system / messages / tools, where content blocks may carry `cache_control`). This layer decides **what to feed and how to layer it**; providers decide **how to send it to a given vendor**. The two layers are decoupled. > For how each call is layered by stability and how the model is told its situation, see > [`composition.md`](composition.md); this document covers the storage and > compaction mechanism underneath it. --- ## 1. Assembly Pipeline The main chat path has one context-assembly engine: ``` dispatcher.process_user_turn → engine.prepare() ← core: 6 steps producing TurnPrep 1. reference scan (references) 2. select history messages ── commit-chain (default) or dag (flag), pick one 3. assemble system prompt (identity + workspace files + skills + memory) 4. compute token budget (four-way split: system / history / tools / output reserve) 5. usage reconciliation (provider-measured + local estimate) 6. return TurnPrep → should_auto_compact? → compact() (optional, inline LLM-summarized) → agent_loop → Context(system, messages, tools) → provider ``` Thresholds: 80% auto-compaction / 95% emergency (`engine.py`). **Main path vs. auxiliary paths**: outside the main path, 8–10 features each make their own single LLM call (summarization / memory sleep / branch summary / mixture-of-agents / agentic runtime). They bypass engine.prepare and hand-roll a minimal `Context` (essentially just system + one user message, no tools). This is intentional layering — these calls don't need the full chat context. --- ## 2. Storage Model: DAG Conversation memory splits into two layers: - **DAG**: the true conversation history. Append-only, never rewritten. Stores user / assistant / tool / runtime nodes, plus the topology of branch / retry / attach / merge. - **ContextCommit**: an immutable snapshot of the "LLM input context" under a given head (the result after compaction, aging, summary, and attach expansion). See §3. ### Nodes ```python @dataclass class Call: id: str seq: int # monotonically increasing within a session; the sole ordering key (≠ wall-clock) created_at: float role: Literal["user", "assistant", "tool", "system"] name: str # tool name / model name / "" input: Any # tool args / system text output: Any # tool result / assistant content / user content (permanent original text) predecessor: Optional[str] # conversation edge (user/assistant chain) ┐ mutually exclusive caller: Optional[str] # call edge (assistant → tool → sub-llm) ┘ reads: list[str] # declares which nodes were read metadata: dict ``` Constraints: `predecessor`/`caller` are mutually exclusive (guaranteed on the write side); `output` is permanent original text, and no aging/compact ever touches it. ### The git metaphor: retry / edit / fork are the same operation Each turn is a "commit", and `predecessor` is the object it responds to. retry / edit / fork are essentially identical — hanging a divergent sibling under some node's parent, differing only in how they're triggered: | Operation | Meaning | |---|---| | Send a new message | append a node under the current HEAD, HEAD advances | | retry | a same-content sibling (assistant re-sampling / function re-run) | | edit | a different-content sibling (forks under the same parent) | | switch version `< N/M >` | checkout to a sibling, **display-only, no re-run** | | branch into a new session | new Session, HEAD points at some node, diverges going forward | Key points: **checkout never triggers a re-run** (switching HEAD only re-renders); **edit/retry are forbidden while an agent is running** (to avoid hanging the active execution tree on a node that's about to "go stale"); **workdir does not roll back with checkout** (side effects belong to the user, just as `git checkout` won't re-run your tests). --- ## 3. ContextCommit Immutable Snapshot An immutable snapshot of the "context fed to the LLM" under a given head. Current implementation: git-backed JSON (`/context/commits/.json`), with code in `openprogram/context/commit/`. ### Data Structures ```python @dataclass class ContextCommit: id: str session_id: str parent_ids: list[str] # single parent for a normal turn; multiple parents for a merge turn created_at: float head_node_id: str rules_version: str total_tokens: int items: list[ContextItem] summary: str = "" @dataclass class ContextItem: source_node_id: str # corresponding DAG node (summaries use a virtual sm_) role: str state: Literal["full", "aged", "cleared", "summarized", "summary"] locked: bool # True = rules no longer touch it rendered: str tokens: int reason: str # "new" / "tail_window" / "idle_60min" / "attached_from:X" merged_into: Optional[str] is_anchor: bool # high-value original text retained during summarization attached_from: Optional[str] # source commit this came from via an attach expansion ``` state: `full` (original content) / `aged` (tool result replaced with short text) / `cleared` (old tool result replaced with a fixed placeholder, cache-friendly) / `summarized` (folded into some summary, not rendered) / `summary` (the synthesized summary item). ### Generation `ensure_latest_commit()`: finds the nearest commit on the head's ancestor chain; if it already corresponds to the current head, returns directly; if there are new nodes, runs `generate_commit()` (copy parent items → convert new nodes to `full` items → run `RULE_PIPELINE` → compute tokens, save JSON). Rules only modify unlocked items, never write back to the DAG, and summary items exist only in the commit (using `sm_`). ### Compaction: Three Rule Classes | Rule | Trigger | Effect | |---|---|---| | **tool aging** | tool result exceeds the tail window (default `tail_turns=3`) | full → aged, replaced with `[tool ] output: … ` | | **idle clearing** | aged and untouched for 60 min | aged → cleared fixed placeholder, more stable cache prefix | | **summarize** | `total_tokens > threshold` (~70–80% budget) | the oldest run of consecutive items is folded into one `summary`; folded items are marked `summarized`; high-value ones (cited/pinned/anchor) stay full | ### Rendering `render_commit(commit)` → provider messages: `summarized` is skipped; `summary` is rendered as assistant text prefixed with `[Summary]`; tool items are downgraded to user text (to avoid protocol errors when pairing information is missing). **A pure function, same input same output** — easy to test, debug, and predict cache hits. --- ## 4. Attach / Merge (Cross-Branch References and Aggregation) attach and merge reuse the same mechanism — expanding another branch's ContextCommit into a set of items within the current commit. The difference: **attach is passive** (stages, waits for the next LLM run), **merge is active** (immediately triggers an LLM turn and writes a multi-parent commit). - **Attach**: the attach pointer is a DAG node with `function="attach"` (its metadata contains the source session/head/commit). The generator reads `source_commit_id`, expands its items, wraps them with open/close markers, and tags each with `attached_from`; deduplication is by `attached_from` (**expanded only once across turns**); if it can't be loaded, it falls back to a single user item. Expanded items default to `full/unlocked`, passing through the rules on equal footing with native turns (**they can be compacted**); summarize respects attach boundaries. - **Merge**: `process_merge_turn()` writes N temporary attach pointers after the target head (each pointing at a peer's terminal commit) + one merge instruction → runs `process_user_turn()` → saves a **multi-parent** commit (`parent_ids=[target_prev, peer_1, …]`) → marks the peer heads merged. The `base_peer`'s attach block is marked `locked=True` (guaranteeing the merge agent sees the original text), while the other peers can be summarized. Multi-parent commits come only from merge. --- ## 5. Cross-Turn Tool Context **Problem**: if `role=tool` rows in history are dropped, the model can't see across turns which tools it called, with what arguments, and what they returned — leading to repeated calls and hallucinating files it "already checked". This is distinct from summarizing the whole history (per-item tool aging vs. whole-segment semantic compaction). **Strategy** (tool aging, i.e. the first rule in §3, refined): - **tail-window full text**: the tool_use / tool_result of the most recent `TAIL_TURNS=3` assistant turns is retained in full. - **old-turn aging**: for earlier turns, the tool_use head is kept (args truncated to `MAX_TOOL_ARGS_CHARS=200`), and the tool_result is swapped for a one-line semantic stub. - **per-item cap**: any tool_result over `MAX_TOOL_RESULT_CHARS=4000` is truncated at head and tail (truncated even within the tail, to prevent a single turn from blowing up). - **critical-tool protection**: `PRUNE_PROTECTED_TOOLS={todo_create, todo_update, todo_list, web_search}` do not participate in aging. Data flow: `get_branch` → hang caller-children tool rows back onto the assistant → tool aging (full in tail / stub for old ones) → per-item truncation → (when over threshold) whole-segment summarize → render into ToolUse + ToolResult content blocks. Code: `openprogram/context/tool_aging/`. Thresholds are taken from reference frameworks: tail-window (OpenCode), per-item stub (Hermes), per-item truncation + critical-tool protection (OpenCode). --- ## 6. Invariants 1. The DAG is append-only; rules never change node content. 2. A ContextCommit is not written back once saved; new rules only affect subsequent commits. 3. A ContextItem's compaction state tightens monotonically, never reverting to a more complete state. 4. `locked=True` is not modified by rules; `state=full` is the default. 5. summary items are not written to the DAG. 6. attach expansion is deduplicated by `attached_from`, expanded only once. 7. Multi-parent commits come only from merge. 8. `render_commit` is a pure function. 9. checkout is display-only, does not re-run, does not roll back the workdir. --- ## 7. UI - **History view**: the full DAG, with nodes = circle/triangle/square, showing all tool calls / retry branches / merge confluences (drawing forks from `parent_ids`). - **ContextCommit Timeline** (right column): `apps/web/components/right-sidebar/context-commit-timeline/` + `ws_actions/context_commits.py`. Lists the current commit's items with state badges + token counts (full/aged/cleared/summary, with their `attached_from` source). ### Occupancy: one number, two readers Two widgets report how full the window is — the progress ring in the composer (`apps/web/components/chat/context-badge.tsx`) and the `/context` breakdown panel it opens (`apps/web/components/chat/context-breakdown-panel.tsx`). Both render the same server-computed record, so they agree at every moment: ```python {"window": int, "total_used": int, "basis": "measured" | "estimated", "estimated": int, "calibration": float} # calibration only when measured ``` `openprogram/context/session_stats.py` builds it; `server.session_context_stats` serves it to `/context` and `server._broadcast_context_stats` pushes it over the `context_stats` WS frame. The ring reads `total_used / window` from the store; the panel reads the same fields for its headline and derives `Free space` from them, so its per-category rows never contradict the total. The canonical WebSocket chat runner records each provider request's prompt size as it finishes, so the ring and panel update between tool calls. At turn end it refreshes the active branch head and persists the final measurement. Session loading can recover a newer provider usage record when an older chat turn finished before this broadcast was implemented; it only does so when no later session edit has superseded that usage. **The number always describes now.** Two bases produce it: | Basis | When | `total_used` | |---|---|---| | `measured` | a real request just completed | what the provider billed for that call's prompt (`input_tokens + cache_read`) | | `estimated` | the graph moved since that request | recomputed from the current branch with the local tokenizer | A measured reading also records `calibration` — measured ÷ estimated for the same graph — which says how far the local estimate drifts from what the provider charges. **Graph-change triggers.** Anything that changes what the next request will carry re-estimates and re-broadcasts through `server.refresh_context_stats`, so the ring follows the graph instead of lagging one request behind: | Change | Site | |---|---| | compaction lands mid-turn | `apps/server/openprogram_server/_webui/_execute/chat.py` (`compaction_finished`) | | model switch | `apps/server/openprogram_server/_webui/routes/execution/runtime.py`, `apps/server/openprogram_server/_webui/ws_actions/runtime.py` | | branch checkout / delete | `apps/server/openprogram_server/_webui/ws_actions/branch.py` | | sibling checkout | `apps/server/openprogram_server/_webui/_chat_routes.py` | A measurement belongs to the branch it was taken on: `session_context_stats` keeps the `measured` basis only while HEAD has not moved, and re-estimates for any other head. **Window resolution.** One path only — `get_model(provider, model)` + `real_context_window()`, splitting `"provider:model"` on the colon (`server._resolve_context_window`). The session's picker override wins over its persisted model, so a switch takes effect before the next save. --- ## 8. Relationship to the Three-Layer Composition [`composition.md`](composition.md) layers each LLM call into three tiers by how often the content changes, which serves the cache (stable content up front) and also tells the model its situation. This document's mechanism supplies the material for those tiers: | Tier | Content | Change frequency | Where it comes from | |---|---|---|---| | **L0 constant** | identity / global instructions / tool list | unchanged for the whole session | assembled by the engine's system-prompt step (§1) | | **L1 situation** | which function I'm a part of / who called me / which step of the program / where the output goes | changes on each new frame | the call tree, rendered from the DAG (§2) | | **L2 task** | progress within the frame / inherited upstream results / current input / output format | changes each step | history selection + compaction (§1, §3) | --- ## Appendix: Implementation Notes - Two history-rendering paths coexist, commit-chain and dag, each with its own traversal logic. They are candidates for convergence into one: a change to either is easy to forget in the other, which has produced a real defect before (ThinkingContent handled on the dag path and missed on the commit path). - The auxiliary paths' system prompt does not follow the agent. Summarization and branch summary use a hard-coded `SUMMARIZATION_SYSTEM_PROMPT` that does not track the user's AGENTS.md, identity, or skills. - pinning, dedup, and user-driven manual pin/unpin are not available under ContextCommit. An earlier design used a SQLite `node_annotations` derived model: DAG as truth plus one recomputable Annotation per node, updated by an annotator pipeline each turn and rendered by a `build_view` pure function. It and ContextCommit are two ways of landing the same idea — annotation as derived, discardable and recomputable state in a SQL table, versus ContextCommit as an immutable snapshot in git JSON. ContextCommit is what the system uses; the ideas that carried over (compaction tightens monotonically, the view is a pure function, summaries are not written to the DAG) are described above. Current context occupancy uses the last measured request, never the sum of requests in a turn. Empty terminal events preserve the last measurement. If no measurement exists, the UI uses a local estimate. Persisted usage and calibration from the older cumulative accounting format are invalidated; cumulative billing totals remain unchanged.