Which turns have been written to memory

A session is a DAG, not a straight line. Every branch's conversation should enter memory, while shared prefixes should not enter twice. This page illustrates the fact that determines both. The main document is docs/reference/design/memory/written-marker.md, with its Chinese counterpart at written-marker.zh.md; the surrounding memory subsystem is described in overview.md and its mechanism diagram in architecture.html.

Layer 1 · Current running mechanism

Position cursor: one integer per session, stored inruntime.json. After branching, it omits an entire branch or writes only a fragment starting mid-conversation while reporting success.

Layer 3 · Finalized, not yet implemented

Mark nodes as written and walk backward from each branch tip until the first marked node. Seven files, about 200 lines changed and 130 added, plus one-time migration.

Layer 4 · Unscheduled

Store no separate tracking state; determine pending work from memory's own content. Indexing and invalidation rules are the obstacle, larger than all of layer 3.

01 Current mechanism 02 Other implementations 03 Planned changes 04 Ideal design

01Layer 1 · Pre-migration implementation

Progress is one integer keyed by session id. Six branches in a session all share that integer.

Same session: after writing the trunk, the user returns to m3 twice to ask new questions Nine trunk turns are already in memory and ordinal=9 is stored. Two branches fork from m3 and continue for five and eleven turns respectively. Trunk m1m2 m3 m4m5m6 m7m8m9 These nine turns are in memory Cursor {"ordinal": 9} Branch A Five turns, none written Branch B Eleven turns, none written Reading first flattens the chain into a list, then assigns ordinals get_branch(branch A tip) 012 345 67 ordinal > 9 before claiming Claims 0 turns The entire branch is treated as already written get_branch(branch B tip) The portion still numbered ≤ 9 Tail beyond 9 ordinal > 9 before claiming Claims 2 turns (measured) Writes a partial fragment, marks the full range Worse: It reports success. No code path reports a problem, and the earlier portion is never retrieved again.
Diagram details: Writes a fragment starting mid-conversation, then marks the entire range as written. Ordinals are produced by flattening; they are not message properties._records contains for index, message in enumerate(messages): the number exists only when the chain becomes a list and disappears with that list. One message reached along two paths can receive different numbers; different messages at equal depth can receive the same number.
The ordinal sequence also includes rows memory does not record._records numbers a row by its position across the whole branch, including skipped rows. Tool-call rows and runtime-scheduled turns consume ordinals, so rows beyond the cursor and corresponding turns are not a simple subtraction. The two counts above were measured rather than calculated.
This representation even prevents correction.advance_cursorRaises cursor cannot move backwards when a new ordinal is lower than the stored one, but moving backward is precisely what branching requires. A monotonic counter cannot be repaired into a branching-conversation representation.

02Layer 2 · Other implementations

references/ contains nine directories;claude-code/ contains only five files from one tool, leaving eight frameworks with meaningful answers. Four have genuine long-term memory.

Framework Conversation shape Memory Tracking state determining which turns to write Fork behavior claude-code-leaked DAG Yes Message-UUID cursor lastMemoryMessageUuid An in-process closure variable, never persisted Copies preserve original UUIDs, so the cursor still resolves When the message is missing, recount all messages rather than returning zero codex-cli Linear · Fork creates a new file Yes One row per thread: watermark + lease Separate memories_1.sqlite keyed by thread_id A new thread_id has no row, so extraction restarts from the beginning No ancestry join exists, so the shared prefix is extracted twice openclaw DAG Yes Line-number cursor lastContentLine, one per session file promotedAt Yes, but on derived candidate records New-file line numbers restart at 0 and copied prefixes are ingested again Ingestion reads the file flat without following parent pointers hermes-agent Linear · Session-level fork Yes No memory cursor; turn order determines progress Disk flushing has a separate in-process index _last_flushed_db_idx Every fork manually realigns the index Fork hook on_session_switch Unimplemented by seven of eight providers pi-mono DAG · Strictly append-only No At branch switch, compute summary coverage directly from the tree without storing tracking state Annotations append a label; moving a leaf appends a leaf Nothing goes stale because every switch recomputes it opencode SQLite No Not applicable revert Mark a point first; the next message hard-deletes everything after it, retaining no older branches pi-ai No storage No Stateless; caller sends all messages each time No branch concept weclaw Delegated No CLI/ACP: session ids; HTTP: twenty turns in memory No fork concept No per-message memory marker Openclaw comes closest, but marks derived candidate records rather than transcript rows.

Only two actual patterns

Choose a unit coarse enough that forks no longer matter codex-cli uses threads, openclaw session files, hermes-agent turns Before fork Whole file / thread = one key After fork New key, progress starts from zero Cost: the three green turns are copied conversation extracted a second time No turns are lost. Deduplication is violated, and duplication is recoverable. Compute this increment directly from the tree pi-mono: collect to common ancestor; no stored state Nearest common ancestor Abandoned leaf New leaf Summarize two nodes from old leaf to common ancestor Its storage supports this because stored records cannot be modified Annotation = append a target-referencing label; active-leaf movement = append a node-referencing leaf. A proven third option; Layer 3 explains its rejection.
Diagram details: Memory tracks nothing: it runs after each turn, so execution order determines progress. Stateless streaming client; caller supplies the entire message array each time. CLI/ACP backends cache only downstream session ids; HTTP retains the latest twenty turns in memory. None writes “memory consumed this message” onto the conversation record. Diagram details: pi-mono walks to the nearest common ancestor and collects the intervening segment without storing state. Walk from the abandoned leaf to the common ancestor; the two intervening nodes require summaries. This is a real third option besides mutating nodes. Layer 3 rejects it for reasons, not because it lacks precedent. Others duplicate at forks; we omit.Codex-cli reextracts a forked thread from the beginning, and openclaw rescans a new file from line 0. Neither loses turns; both reread consumed content. Duplication is recoverable: nightly organization merges passages describing the same fact, and writing a turn twice costs one model call. Omission is not recoverable because no process revisits those turns.
Two useful behaviors to adopt.① claude-code-leaked recounts all messages if the cursor target is missing; its comment explains that returning zero would permanently stop extraction for the rest of the session. An adjacent counter lacks that fallback and silently remains zero, so this is a deliberate decision rather than a default. ② pi-mono's append-only marker records.

03Layer 3 · Current proposal: node markers

The boundary must use something that identifies a message: the node itself. Record that memory processed a turn on that turn's node.

Same session and fork: walk backward from the branch tip and stop at the first marked turn All nine trunk turns have markers; neither branch does. Traversal needs no ordinal and neither branch needs to know the other. Trunk m1m2 m3 m4m5m6 m7m8m9 All nine have markers Branch A Walk backward from the tip, collecting until marked m3, then stop Returns all 5 turns Branch B Same traversal and stop point Returns all 11 turns Branches need no ordering: Whichever writes first marks the shared turns; the other stops at those markers when it arrives.

Where and when to mark

Where history/0007-u-a3f1c2.json {"id": "a3f1c2", "role": "user", "predecessor": "9d0e77", "metadata": {"rewound": null, "memory_written_scriptorium": "w-4f21c8e0"}} The key includes provider name because the interface is pluggable; a shared boolean becomes incorrect as soon as a second provider exists The value is memory-workspace identity because markers travel with nodes; copying to a machine with empty memory would otherwise claim every turn was written Traversal stops only when a marker identifies its own workspace When: order the three steps by replayability 1 Archive evidence Content-addressed; append-only 2 Install topic files Block IDs + footnotes; all-or-none Were files changed? Topic-file paths marked commit/ok in the audit; an empty list means none 3 Write markers Only after successful transaction installation with actual file changes Crashes at these points leave unreferenced evidence and no markers; the same batch remains pending This requiresredoing the work. Marking first turns the same crash into silent turn loss. If even batches changing no files were marked Twenty rejected edits; false success; one-byte topic Two forbidden side effects: ① Do not change updated_at; the idle watchdog uses it to determine whether a session was processed. Changing it triggers another forced write and model call. ② Do not use mark_synced(); that fingerprint would cover writes the marking process never read, so indexes missing those nodes would never rebuild.
Diagram details: Append-only and content-addressed; archiving twice is byte-identical. Contain block IDs and footnotes; partial installation corrupts state, so install all or none. Measured: twenty turns hit the same rejected edit and reported success while the topic file contained one byte. All required mechanisms already exist.metadata is an open dictionary. The message adapter puts unknown fields there and readers expand keys back to top level, so keys must not collide with message fields (id role content predecessor caller timestamp status are already reserved). Mutating stored nodes is also established:rewind sets rewound, revert setsreverted, and turn finalization rewrites node files in place with checkpoint stamps;SessionNodeWriter.update already performs the operation markers need, first passing through SessionStore._open to rebuild stale indexes, then usingatomic_write_text to rewrite directly, avoiding both forbidden effects above.
Measured cost.The session index cache rebuilds once; 289 nodes and 4.2 MB take14–50 milliseconds and the cost is self-limiting. Git grows because the session directory isgit add -A each turn, somark only this batch, never rescan and rewrite the whole session; otherwise every turn adds the entire session's size, producing quadratic growth.

Files to change

In dependency order. Items 1–6 form one coherent batch; 7 may follow separately; 8 must be correct on its first execution. 1 · session_store.py Add merge_node_metadata　About 20 added lines 2 · session_node_writer.py Delegate update to item 1　About 12 removed lines 3 · memory/store.py Add workspace_id()　About 12 added lines 4 · runtime/state.py Remove cursors / advance_cursor　About 10 removed lines 5 · runtime/online.py pending → unwritten_turns, add mark callback　About 25 changed lines 6 · memory/writing.py Traversal and writer returning changed files　About 40 changed lines 7 · Query every branch at session boundaries list_branches + once per tip　About 30 added lines 8 · mark_archived_turns.py Seed from sources/ once　About 50 added lines Tests test_memory_writing.py (existing) Change _pending assertions to read markers test_memory_write_timing.py (existing) Threshold and idle behavior remain unchanged Add test_memory_written_marker.py Two measured cases, shared prefix written once, No rejected markers; migration seed Total: About 200 changed and 130 added lines across seven files, plus a new test file. The only runtime cost worth remeasuring afterward is the index rebuild.

One-time migration: mark only verifiable contiguous prefixes

Archives first produce candidate IDs; candidates cannot become markers directly legacy Accept only the first valid header group immediately after the file title No body length: later headers may be user text; rewrite the remainder. v2 Read the valid prefix from a strict record-lines parser Skip bodies by line count; the first invalid or truncated frame ends parsing, without restarting later. The old ordinal bug may leave: m1 ✓ m2 ✓ m3 ✓ b1 × b2 × b3 ✓ Shared prefix + unarchived middle + archived tail Marking all candidates directly causes permanent omissions After b3 is marked, backward traversal from the tip stops immediately. b1 and b2 never enter another write batch. Current migration: retain contiguous prefixes on the real DAG Reuse writing._records filtering; start from the first writable record and stop at the b1 gap. Mark only m1–m3 Write b1, b2 and b3 together next time; b3 may duplicate, but the gap is not skipped. Order: Delete cursors only after safe prefixes for all sessions are successfully marked; any failed batch retains cursors for retry.

Counterargument: an external set is exact; traversal is not

Mark nodesMemory stores its own ID set
How to find unwritten turnsWalk backward from the tip until the first markerTraverse the entire branch and subtract the set
At a forkReach the shared prefix, find a marker and stopSkip prefix IDs already in the set
What to storeOne additional field on an existing nodeOne file per thread containing every written ID
GrowthNo growthOne entry per message retained for the session's lifetime
Read costA few steps backward from the tipThe thread's entire inventory
MigrationMark only DAG-contiguous prefixes from trusted archive candidatesSeed the set with the same safe prefix
Lost marker or entryTraversal stops early and all older turns become unreachableThat turn is retrieved again
CouplingMemory writes a field into session storageMemory owns its tracking state
Diagram details: No body length exists; a second valid header could be verbatim user content, so rewrite subsequent records. This argument is valid:The position cursor is an O(1) small-file read entirely outside the session tree. Node markers record the same fact in another location whose directory mtime is used elsewhere for change detection. A set is exact; traversal assumes markers form a branch prefix. Oldest-first batches establish that assumption, but it is not a validated invariant.
The other half determines the choice:External tracking is cheap only while it is one number. To remain correct after forks, it must become an ID set, introducing file placement, migration, permanent ID retention and read cost growing with the session. Session storage already records a turn's origin, whether it is runtime self-talk, whether it is a branch root, and whetherrewind has rolled it back. A memory-written field belongs to that same category.

04Layer 4 · Ideal design

Layer 3 answers what to implement next; this layer asks what a redesign would look like. The following are not phased versions of markers, but another formulation of the same problem.

The same fact is recorded in three places All describe which turns memory consumed, and all can disagree. ① Position cursor cursors in runtime.json Layer 3 replaces these with node markers ② Source-id set in source archives sources/<provider>/<thread>.md Read on every write to skip already archived turns ③ Footnotes in topic files Each topics/*.md record links to its source Only this record proves content actually entered a file Redesign: store nothing elsewhere; pending work is a query Unwritten = reachable from any live tip and not referenced by any memory record. The set in ② is already computed on every write; it is an answer to this question that is not used as such. It cannot disagree with memory because it is memory Remove a passage from a topic file, and its corresponding turns become pending again, correctly Moving sessions needs no extra state No workspace identity or import path, no migration and no lost-marker case Three obstacles Archives are regex-parsed Markdown, costing O(file) per read; an index needs invalidation rules; archived does not imply referenced
A record knows its source branch Now Branch A Branch B Same batch of topic files Retrieval returns both to a reader on only one branch Ideal Records carry source branches; retrieval prefers the current branch Missing: topic field, branch filtering, alternative-branch semantics Memory receives notifications instead of polling Now Check thresholds after every turn Watchdog scans every five minutes Result: a branch abandoned an hour ago waits until the whole session becomes idle before writing Ideal Session storage reports that this tip stopped Memory responds Five head-write sites; shared function drops old value; no event. Adding the event is small; routing all five paths through it is not, and waiting currently loses no data.
Diagram details: Three gaps: topic format lacks the field, retrieval lacks branch filtering, and nightly organization lacks mutually alternative branch semantics. Obstacle: head writes are spread across five sites; their sole shared function discards the old value, and no event carries the movement. Implementation status (2026-08-11).Layer 1 records pre-migration implementation, layer 2 audits open-source frameworks, and layer 3 is implemented: session nodes usemetadata.memory_written_scriptorium = <workspace-id> to record writing state; the current branch derives its unwritten suffix from markers. Mark only successful batches that actually modify memory; session completion processes current head before other live branches. RuntimeState.cursors andadvance_cursor are removed. First write on legacy installations validates legacy headers and the strict valid prefix of sources/openprogram/_v2/*.md: legacy accepts only the first valid header after the title; v2 stops at invalid frames. Candidate IDs undergo the same record filtering as online writing on real DAG paths; mark only the contiguous prefix before the first gap, allowing later tails to be rewritten. Delete oldcursors only after all session markers succeed, retaining it on failure for retry. Structured speaker fields, trusted v2 frames and speaker filtering are also implemented. Historicalmemory backfill deliberately ignores node markers and selects only trusted Sources not referenced by Topics, excluding pending entries. It neither adds nor removes session markers. Node markers record online-writer processing state; Topic references record whether historical Sources entered retrievable content. These definitions are not interchangeable. Historical backfill has not run in the canonical workspace. Layer 4 remains unimplemented.