Mechanism diagrams

How a conversation becomes topic files on disk, what happens if a write fails midway, how forks track previously written messages, and who maintains the small memory block injected into every session. The main text is in docs/reference/design/memory/overview.md; the complete design for section 04 is in written-marker.md. This page illustrates mechanisms only. Comparisons with other implementations and the rationale for our choices are in Memory writing: comparison of eight frameworks.

Running implementation

Two write entry points, five write steps, staged transactions, nightly organization, the silence-means-success return-value contract, and Section 05 and its resident-block derived view: core.md is rendered from topics/core.md; exceeding the budget no longer rejects transactions.

Designed, not yet implemented

Section 04’s write cursor: the running implementation still uses positions, which can omit an entire fork or write only a fragment starting mid-conversation. The planned approach marks nodes as written and walks backward from head to the first marked node.

Interface present but not connected

name is_available initialize shutdown extract_before_discard: five methods have no runtime callers.

01 From conversation to long-term memory 02 Five write steps 03 Transactions: stage, validate, install 04 Cursor: identity versus position 05 Maintaining the resident block 06 Nine interface methods 07 Failure behavior 08 Unresolved issues 09 Web page status 10 Unified modification interface

01 From conversation to long-term memory

Two entry points share one write implementation. They differ only in the force switch: one waits for enough content; the other writes everything remaining.

From conversation to long-term memory Conversation content is not buffered in process memory for writing. Every write rereads session storage, so process restarts lose nothing. Sessions Persistent and ordered; stable ID per message User Assistant User Assistant User Assistant … get_branch(session_id) walks backward along predecessor edges, returning the head branch in order Only user messages and assistant replies count. Tool calls/results, runtime-scheduled turns such as subagent reports or branch-merge notices, and empty messages never enter memory. Entry one · end of each turn agent/dispatcher write(session_id=…) force=False Read stored session messages Called every turn; usually does nothing Entry two · session becomes idle memory/session_watcher write(messages, …) force=True Scan every five minutes; idle after 30 Write all remaining content Daily 03:00 · organize memory/scheduler reorganize(model=…) Reorganize disk content; no new content writing.write() Both paths converge here force=False Run once and return. Below threshold, write nothing. force=True Repeat passes while content remains until nothing remains. No next batch after session end; partial writes leave content unwritten. One write_session() pass Five steps; see section 02 1 Select unwritten messages 2 Accumulate 16k tokens 3 Model edits staged files 4 Organize at the threshold 5 Validate and install Memory repository <state>/memory/ topics/ Only model-editable files sources/ Append-only source archive core.md Injected into every session timeline/ + two locations Derived views rebuilt after writes .scriptorium/ Cursor, write lock, writer history One commit per successful write When forced: repeat while content remains Organization bypasses ingestion: it rearranges topics/, leaves sources/ unchanged, and does not advance the cursor

Diagram details: Session storage. No messages passed; read session storage. Scan every five minutes; idle after 30 minutes. Write everything remaining regardless of batch threshold. Reorganize existing disk content; add nothing new. While content remains, run successive passes. After a session ends there is no next batch; stopping halfway leaves content unwritten forever. Select unwritten messages not covered by the cursor. Batch by accumulating 16k tokens. Call the model to edit staged files. Organize when enough content accumulates. Commit: install everything after validation. Topic files; the model edits only these. timeline/ and two other locations.

Reading the diagram: three entries on the left, shared logic in the center, disk on the right. Green triggers every turn but writes only after 16k tokens accumulate. Blue triggers after idleness and loops to write everything remaining. Purple produces no new content.
Why the per-turn entry passes no messages: session storage is already persistent and ordered, with stable message IDs. A duplicate process buffer would be lost on restart and produce positions that vary between runs.

02 Five write steps

What happens in one pass. The threshold determines whether to call a model, not how much a single call writes; a day-long conversation may need several passes.

1 Select unwritten content Every message on the branch minus cursor-covered messages = remaining session content runtime.pending() Also exclude tool calls and runtime-scheduled turns 2 Batch Accumulate from the start Cut a batch at 16k tokens That segment is the current batch Leave the rest for the next pass; one model call cannot contain a whole day’s conversation 3 Call the model to edit topics Writer receives batch source text and the current directory structure File tools edit topics/ Edits affect copies in the staging directory; see section 03. Runs using the user’s own login 4 Organize After five batches or 40k tokens → reorganize once One day after global organization → reorganize again One action, two triggers; skip when neither applies 5 Commit the transaction Validation passes → install all Validation fails → discard the batch Advance cursor after installation This batch’s source text is archived before the model call, so the archive never falls behind the cursor A cross-process write lock covers the entire pass. Give up if acquisition takes over one second: active conversations commonly write memory, so retrying next turn is preferable to delaying the user.

Diagram details: minus messages covered by the cursor. Leave the remainder for the next pass; After one day since global organization. Validation passes → install everything. Advance the cursor only after installation.

Diagram details: Accumulate messages from the beginning. The writer receives this batch’s source text.

Topic normalization rejects substantive uncited prose with INVALID_TOPIC_FORMAT before pruning empty sections or topics. Changes, patches, editor/CLI staged edits and writer shell commits share this validation. Rejection preserves committed topic/source bytes, Runtime state, revision and Git history. Cited native/legacy records and genuinely empty topics retain their existing rules.

An instance-owned trusted pre-install callback records autosave or restore history only after validation and stage preparation, before committed files are moved. Invalid edits never create a manual checkpoint; callback failure never installs. A valid edit still records the exact pre-edit snapshot and the existing checkpoint deadline. Tools and request text cannot supply this callable.

03 Transactions: stage, validate, install

The model never edits the repository directly. It edits a temporary copy, installed only after complete validation. Failed validation discards the copy without changing a byte in the repository.

Complete write-transaction lifecycle Memory repository topics/ sources/ core.md Read-only during the transaction; a reference that nobody edits directly Copy everything Staging System temporary directory topics/ Writable; all edits this turn go here sources/ chmod 0444 Filesystem rejects writes immediately Read the repository baseline before copying. A later baseline would miss deleted block IDs. Four validations 1 sources/ fingerprint unchanged Source archive is append-only 2 Every pre-existing block ID remains Rewrite, merge, or move paragraphs 3 Cross-topic links target #^block IDs 4 Referenced sources actually exist core.md budget; valid block links Pass · install everything Back up existing data Install full staged copy Rebuild views; commit Fail · discard the batch Create fresh staging Repository unchanged Raise the rejection reason After rejection: one repair attempt only First rejection Return rejected text and rule-specific repair guidance Workspace restored to its pre-turn state Retry the edit Same staging and four validations Take a fresh baseline before repair; a later baseline may contain corrupt text. Second rejection → do not write this batch COMMIT_REJECTED; cursor unchanged; turns pending. False success would skip content never installed on disk. The same pipeline serves human edits: manually edited topics use identical staging and validation. Invalid edits are rejected immediately without changing committed files. Block IDs are how other paragraphs, timelines, and relation graphs locate a paragraph. Content may change but IDs must remain: this is the strongest validation constraint.

Diagram details: Read-only throughout the transaction; Paragraphs may be rewritten, merged, or moved.

Diagram details: Staging directory. Created under the system temporary directory. Writable; every writer edit this turn goes here. Filesystem-level denial prevents writes immediately. Read the baseline from the repository before copying. Reading it after copying. would compare an edit with itself and treat lost block IDs as though they never existed. core.md stays within budget; no dangling block links. Move existing data to backup. Install the complete staged copy. Rebuild derived views and commit. Recreate staging from a fresh copy. Leave the repository unchanged. Return the rejected text and a specific. instruction for correcting this rule to the writer. The workspace has already been restored to its pre-turn state. Take a fresh baseline before repair; taking it afterward would. parse text the model has just corrupted. Return COMMIT_REJECTED, leave the cursor unchanged, and keep these turns pending. Reporting success here would let the caller advance past content never installed on disk.

04 Cursor: identity versus position

A session is a graph, not a linear sequence. Re-asking, retrying a reply, or spawning a subagent creates a branch. Every branch’s messages should enter memory, but shared prefixes should not be written twice.

One session forks from message three after the main branch was written Nine main-branch messages are already in memory. The user returns to m3 and asks again; six more messages follow, with head now at b9. Main branch m1 m2 m3 m4 m5 m6 m7 m8 m9 These nine are already written Fork from m3 New branch b4 b5 b6 b7 b8 b9 Head is here These six have never been written Shared prefix Current · record the written position Cursor: ordinal=9; accept larger values 45 67 89 All six are ≤ 9 Write none Planned · mark nodes as written Walk from head to first marked node UnmarkedUnmarkedUnmarked UnmarkedUnmarkedUnmarked All six remain unwritten Stop at marked m3 Alternative · memory stores an ID set Traverse branch; subtract stored IDs New IDsNew IDsNew IDs New IDsNew IDsNew IDs All six remain unwritten Same result, different cost Why the prefix is not duplicated: Walking backward reaches the same m3 message with its marker, and traversal stops there. The two branches need no ordering or knowledge of each other. The first writer marks shared messages; the next traversal stops at those markers. Where the marker lives Node metadata, with a provider-qualified key Value identifies the memory workspace; exported markers retain workspace identity. Rewind’s rewound marker uses the same mechanism When to mark Two conditions: successful transaction installation, and actual file changes in this pass Observed: twenty turns encountered the same rejected edit, returned success, and left a one-byte topic file Status of this section Design settled; implementation still uses a positional cursor. Red crosses show actual pre-change behavior Observed: position 9 skips a five-turn branch; only the last two of eleven turns are selected.

Diagram details: Cursor stores ordinal=9; accept only larger ordinals. Walk backward from head, collect unmarked nodes, stop at the first marker. Traverse the whole branch and subtract IDs in the set. Value identifies the memory workspace; exported sessions. retain markers without being mistaken for another workspace. Observed: stored position 9 treats a five-turn branch as empty,. and an eleven-turn branch as its last two turns.

Why position cannot represent identity: ordinals are assigned when a linked structure is expanded into a list. They describe that expansion, not messages. Before expansion there is no ordinal; afterward it has no relationship to the message ID. The same message reached by two paths can receive different ordinals, and different messages at equal depths can share one. Furthermore, get_branch filters turns marked rewind , and compaction inserts summary nodes, so even one path changes between reads.
The representation also rejects correction: advance_cursor raises cursor cannot move backwards when a new ordinal is below the stored one, although moving backward is exactly what branching requires. A monotonic counter cannot be corrected into a representation of branching conversations.
The two failures differ in visibility: before the branch exceeds the stored height, nothing is claimed and the whole branch is omitted. Afterward, only the excess is claimed: a fragment beginning mid-conversation is written, then the entire branch is marked written. This second case returns success without any indication of failure.
First · current implementation Positional cursor:runtime.json stores one {message_id, ordinal}per thread, claiming only larger ordinals Ordinals come from enumerate(get_branch(...)), and advance_cursor explicitly forbids moving backward, preventing correction Second · other implementations Nine reference frameworks were read.Three store branching conversations(Claude-code, Openclaw, Pi-mono: append-only files with parent pointers); three are linear; one rewrites the entire session on retry and destroys the old branch; two do not store conversations. Linear histories make positions sufficient for identity, so most never encounter this problem.None marks conversation turns as already consumed by memory. Codex-cli forks into a new file; extraction tracks thread-level watermarks and leases,using a coarser unit where branching no longer matters Pi-mono storage cannot modify a record. Annotations and cursor moves append references; branch summaries compute coverage from the tree at switching time,deriving increments from topology Claude-code keeps its cursor in process memory and reprocesses the whole session when the referenced message is missing. Openclaw writes promotedAton derived records, while its memory hook reads the last fifteen conversation lines as a flat list Third · two approaches Mark nodes Pending content: walk backward from head until the first marker Storage: one additional field on an existing node Growth: no separate growth; each read traverses a few nodes Migration: mark nodes covered by archived sources Missing marker: traversal stops too early and older content remains pending Coupling: memory writes a field into session storage Memory stores its own ID set Pending content: traverse the whole branch and subtract the set Storage: one file per thread containing every written ID Growth: one entry per message; reads load the complete list Migration: seed the set from the same archive Missing entry: precisely that turn becomes pending again Coupling: memory owns bookkeeping, including files, migration, and append-only retention Fourth · choice Node markers. External ID sets require storage, migration, permanent retention, and growing read cost. The tradeoff is precise sets versus approximate traversal. Traversal assumes markers form a prefix, relying on every batch selecting the oldest turns; this is an assumption, not an invariant. Session storage already records provenance, display kind, branch roots, and rewind state; another field is the same kind of metadata.

Diagram details: Use node markers. External bookkeeping that remains correct after forks requires a set, introducing file placement, migration, permanent ID retention, and read cost that grows with the conversation.

Perform the idempotent step first: archive evidence, write topics, then mark nodes. Archives are append-only and content-addressed; archiving the same batch twice is byte-identical. Topics have block IDs and footnotes, so partial writes would corrupt them; transactions install all or nothing. If the writer dies between steps, it leaves unreferenced evidence, no markers, and the same pending batch for the next retry. Marking first would turn that crash into silent loss of those turns.
Completion needs verifiable output: marking requires successful transaction installationandactual file changes. Nightly organization must report changed files, not merely inspected files. Absence of an error is not an output.
Verified cost to session storage: sessions have no whole-tree hash. The sole byte-level tree hash is the memory-workspace revision rooted at <state>/memory, separate from <state>/sessions , so node markers are not mistaken for concurrent memory writes. The session-index cache is affected: history-directory mtime changes on rewrites just as on appends. Another process rebuilds the index; measured cost for 289 nodes and 4.2 MB is 14–50 ms and is self-limiting. Git is what grows: the session directory runs git add -A per turn, and changed nodes become new blobs. Therefore, mark only the current batch; never rescan and rewrite every node, or each turn adds the entire session and total growth becomes quadratic. Two additional constraints: markers must not change updated_at(the idle watcher uses it to decide whether a session was handled; changing it would trigger another forced write), and must not use the “mark session synchronized” path.

05 Maintaining the resident block

core.md is the small block injected at each session’s start. It is now rendered from topics/core.md; 2000 tokens is a rendering budget, not a transaction gate. Previously it was the writable source of truth: once full, it stopped changing and rejected every later transaction.

One file, two roles: edited source versus rendered output Left is the old behavior; right is the running implementation. The distinction is where the source of truth lives. Previously · core.md was authoritative The writer agent edited it directly prompts/write.py · prompts/system.py A 2000-token limit raised _synchronize The entire transaction was rejected, including topic changes Repair guidance only said: core.md is full; leave it unchanged and put this fact in a topic file. Every later stable fact was redirected elsewhere Nothing shortened it: nightly organization listed only topics/**.md. The limit measured all staged core.md, not the current change An already oversized core.md therefore rejected every later transaction, even topic-only edits and no-op transactions. Yet it was ordinary Markdown that humans could edit. Observed: after a manual enlargement, a no-op transaction reported Core Memory exceeds 2000 tokens: 4004 After two rejections, neither this session nor the next enters memory COMMIT_REJECTED is not retryable; the watcher marks the session handled. Only manually shortening the file resolves this state. Now · topics/core.md is authoritative; core.md is rendered Authoritative topics/core.md Topic file; no size limit Paragraph ^a1 Paragraph ^b2 Paragraph ^c3 · omitted this time Both writers organize this source Render Derived core.md Rebuilt after installation Paragraph ^a1 Paragraph ^b2 Include up to 2000 tokens Omitted blocks yield IDs The budget limits rendering, not writes Omitted source paragraphs remain indexed for search and memory_get. Omission loses visibility, not content, so clipping does not need to distinguish authorship. To keep a paragraph visible, move it earlier in the source through ordinary editing. Why clipping first requires a derived view Transactions require every pre-existing block ID to remain discoverable. Clipping unique core.md blocks could reject a transaction. Rendered IDs are a subset of source IDs, so omitted blocks remain in the source. Status of this section Implemented. The left-hand behavior no longer occurs. Observed: 11,883-token core.md allows unrelated writes; 1,983 tokens rendered. The older implementation already maintained this:memory/core.pyrefresh_from_wiki rebuilt core.md from a designated wiki page, clipping at heading boundaries to the budget, called by sleep’s deep stage. That process was removed when switching to topics/sources. The right-hand design restores it with a topic file replacing the designated wiki page.

Diagram details: management/block_views.py _synchronize. Both writers and nightly organization use this file.

Diagram details: It reports COMMIT_REJECTED, outside the retryable set, so the watcher still marks the session handled. Ordinary topic file without a size limit. Both writers and nightly organization use only this file. Rebuilt after each successful installation. Include content up to 2000 tokens. Omitted blocks produce IDs, not errors. Omitted paragraphs remain in the source, indexed and discoverable through search and memory_get. A unique block in core.md could make clipping reject the entire transaction. Rendered IDs are always a subset. Observed: an 11,883-token handwritten core.md no longer blocks an unrelated write; rendering produced 1,983 tokens.

Difference between the approaches: the current description’s old path rejects on the write path, preventing content from entering; the planned replacement omits on the rendering path, making content invisible only for that injection. Why not add another gate: another gate still blocks writes and stable facts. Other implementations must distinguish human and automatic content before clipping because their resident file is the only copy: deleting a line destroys it. Here the complete source remains, so authorship does not need classification.

Web defaults to the actual injected block

GET /api/memory/core returns both topics/core.md the authoritative source, root-level core.md rendered output, and effective injected content after considering memory.core.inject. Core defaults to a read-only effective-injection view with actual tokens and the 2000-token budget. “Full records” opens the editable source. Existing content/size/mtime continues to mean authoritative source for compatibility.

Writer and Organizer govern content purpose

Writer reads complete input but records only facts useful later, excluding repeated status checks, unchanged measurements, one-off tool processes, and details recoverable from project files. Project progress belongs in its project Topic. When Organizer handles topics/core.md, it moves project-specific, stale, or duplicate content to ordinary Topics or deletes it, retaining only rules, preferences, and identity facts needed by most future sessions.
Acceptance entryRequirement
GET /api/memory/corereads a complete snapshot under the workspace lock; it retains authoritative content/size/mtime and additionally returns rendered content, effective injection, injection switch, token count, and budget.
Memory → CoreDefaults to actual injected content and token budget, explicitly showing empty injection when disabled. Users can switch to the full source and save through existing transactions. New drafts during save are not overwritten by refresh; all Markdown previews use shared sanitization.
Writer promptNo longer requires every repeated process to become a Topic; Core excludes ordinary project progress.
Organizer promptExplicitly organizes project processes, duplicates, and stale states in Core; deleted content remains in Git history.
Implementation status (2026-08-17): implemented; awaiting review after fixes. Web defaults to effective Core after the injection switch. Full-record saves synchronize Runtime-normalized source through compare-and-set; Markdown is sanitized; GET reads only complete snapshots during transaction installation. Writer/Organizer include these content-purpose constraints. Structured Memory transactions reduced the production workspace from 186 records to 15 and nine Topic files to four; topics/core.md was reduced from 31 records/about 15,305 tokens to three/about 429 tokens, with about 425 injected tokens. Repeated probes, old tool lists, repeated project checks, and invalid states were removed from current views but remain in Memory Git history. Automatic writing now starts at 8000 tokens. Retrieval uses organized Topics without directly mixing Source text. Current fixes cover special-token counting and continued edits during save; review conclusions are recorded after validation.

06 Nine interface methods

This is the sole interface between memory and agent runtime. Every method except name has a default implementation, so replacement memory systems implement only methods they need. The diagram follows the session lifecycle and explicitly marks unconnected methods.

Nine methods arranged by lifecycle Above the line: actual runtime calls. Below: interface exists, but no runtime caller. system_prompt() Inject core.md at session start Wrap in memory-context distinct from current user input search(query) Search before each turn Default BM25, top five results Tight time budget; must be fast write(…) Once per turn, once on idleness Only force differs, so this is one method, not two reorganize() Reorganize daily at 03:00 Also manually triggered: openprogram memory sleep Selection Session start Before each turn Context compaction End of each turn Session end 03:00 nightly name is_available() Two operations select a memory backend. No current selection: direct construction initialize(…) Once at session start Built-in use: store session ID; fallback unreachable extract_before_discard() Stores nothing; selects what to retain. Compactor supplies discarded text Memory selects summary content shutdown() Should run after the final turn Idle watcher finalizes through write(force=True) Five methods lack runtime call sites. Third-party extensions: connections, final writes, and preservation before compaction. The built-in implementation needs none of these, hiding the missing calls. Integrating a system such as mem0 first requires connecting them.

Diagram details: selection step: it is constructed directly. a session ID, so that fallback path is unreachable. Compactor supplies messages to discard. Memory selects what the summary retains.

Diagram details: so it is not mistaken for current user input. Two operations for selecting. a memory backend. Currently there is no. Should run once at session start. Consequence: the built-in implementation uses it to store. Opposite direction from the others: it stores nothing. The compactor has messages about to be discarded. and asks memory which should remain in the summary. Currently the idle watcher handles finalization. These five methods are not redundant; the runtime call sites are missing. They are extension points for third-party memory systems, which often need connections, session-end flushing, and pre-compaction preservation.

Two other conventions: recalled text is already wrapped in <memory-context> where it is produced; wrapping again at output would strip the inner layer and leave an empty wrapper. Also, the interface has no tool-registration method: memory systems with tools are plugins, already able to register commands, skills, MCP, hooks, and agents. A private route would bypass that mechanism.

07 Failure behavior

Writes have three return outcomes interpreted by the watcher. Silence means success; failures distinguish transient conditions from those that will not change on retry.

write() return values determine idle-watcher behavior write() One call Three answers Return nothing Write complete, or below threshold Both mean “no problem to report” Watcher: mark handled Do not revisit this session Skip on the next poll Why the convention cannot be reversed Forgetting a return also returns nothing. Reversing it causes endless retries Still pending · worth retrying Another writer holds the lock Git commit failed or vector backend unavailable Model unreachable or writer process cannot start Watcher: leave unmarked Retry unchanged at the next poll Conversation data remains safe in session storage What these failures share The condition can disappear: the lock releases, the model recovers, and the same content may succeed next time Still pending · retry will not help Both writer edits were rejected by the transaction Unavailable source or invalid patch No session ID was provided Watcher: still mark handled and publish the failure reason memory.ingest_ended ok: false Why mark an incomplete write handled Same content, same rejection; retries waste quota. Invisible failures are never corrected Per-turn writes use the same returns. Previously, lock contention was misreported as an unmet threshold, silently leaving a turn unwritten. The per-turn path logs pending content at debug level; the next turn naturally retries. The idle watcher is responsible for session finalization and therefore decides whether to mark it handled. Memory failures never fail the conversation: every hook catches and logs its own exceptions. Catching an exception does not mean pretending nothing happened.

Diagram details: Write complete, or below threshold with nothing due. Patch cites an unavailable source or has invalid formatting. and publish the reason on the event bus. endless retries only consume model quota. Invisible.

Diagram details: Writing finished, or the threshold was not reached and nothing is due. The opposite interpretation would turn that omission into endless retries. Sending identical content again yields identical rejection; The per-turn path uses the same returns and can also encounter a held lock. Previously lock contention was swallowed as “not enough content,” indistinguishable from an unmet threshold, leaving an unwritten turn without any report.

08 Branch-related implementation status

Node write markers and live-branch enumeration at session boundaries are implemented. Remaining issues concern mutually exclusive branch semantics, cross-session spawn relationships, and compaction-summary deduplication, not positional bookkeeping.

A branch is revisited only at a session boundary Head: will be written Wait for idle Within a turn, both write paths request the branch ending at head. Only session boundaries enumerate every branch.

Diagram details: Wait until the session is idle.

Implemented. Forced boundary writes process current head first, then other unarchived branch tips; node markers deduplicate shared prefixes. The idle watcher still triggers this, not head-change events.
Mutually exclusive branches, one topic file Branch A: use Flask Branch B: FastAPI topics/projects/x.md This project uses Flask This project uses FastAPI Adjacent text Neither says they conflict Retrieval returns both; nightly organization may even merge them.

Diagram details: Branch B: switch to FastAPI. Two adjacent paragraphs. they are mutually exclusive.

The topic format has no expression for “these are mutually exclusive options”, so records from different branches can only coexist side by side.
Cross-session spawn: independent bookkeeping Session A · main conversation cursors/openprogram/A.json Session B · subagent branch cursors/openprogram/B.json spawn Thread keys are session IDs, so the halves use separate cursor files; neither traversal enters the other graph. No loss. Missing: a link identifying the continued conversation.

Diagram details: No duplication or omission. What is missing is the link saying this branch continues that conversation.

Spawn branches within one session share its cursor and archive without extra machinery.For cross-session spawn, memory cannot identify the two parts as related.
When a compaction summary becomes head Original turns, already written Compaction summary Uses the predecessor of the first summarized turn, so normally it is a side branch If head moves to the summary branch, the summary becomes an unwritten assistant turn: its content repeats existing memory, but the cursor has never seen its ID.
Consequently, the same facts are written again in different wording. Normally this does not happen because summaries are side branches rather than mainline nodes.

09 Memory Web page

The page uses OpenProgram’s existing top title bar, secondary sidebar, and content area. Clear hierarchy, consistent row heights, readable content width, and stable narrow-window behavior improve the UI without new data layers or decorative dashboards.

Current problem

The four sections, context lists, and Topic rows have similar visual weight; before selection, the right side is nearly empty. Core explanation/view switching sits on the left while content/token status sits on the right, obscuring their relationship.

Structural principles

The top bar provides page identity and settings only. The secondary sidebar selects a section, then lists its items or filters. The content header consolidates name, metadata, and actions; body text has a controlled reading width.

Unchanged

Retain Topics, Timeline, Recent, Core, and existing APIs. No writer status bar, Commitments, Scheduler status, statistics dashboard, new dependency, or new persisted UI state.

Target page · wide layout

Click the four left-hand sections to inspect their lists and content states.

Current data: 0 Topics / 3 Core records / 2 Timeline dates

Memory
Memory settings
Topics
Durable project, preference, and relationship memory
▧

No topics yet

Durable memory created by conversations and the background writer will appear here. Core rules are managed separately.

Open Memory settings
August 15, 2026
2 memory records · derived from Topics
Timeline

Saturday

Timeline remains a derived chronological view. It presents the Markdown generated for the selected date without adding event times or change metadata.

Core · 2026-08-15

Never fabricate tool calls or tool results. Report only observed results from available tools.

Core · 2026-08-15

Use plain “you” with no invented honorific unless the owner explicitly chooses one.

Recent memory
3 / 50 records · newest durable records first · derived from Topics
CORE2026-08-15

Never fabricate tool calls or tool results. Report observed results and state when no available tool returns a result.

CORE2026-08-15

Use plain “you” with no invented honorific unless the owner explicitly chooses one.

CORE2026-08-10

Use Chinese by default when discussing Memory and other operational work.

Core memory
Injection enabled · 425 / 2,000 tokens
Prompt previewSource records
Read-only prompt preview

Core memory

Generated from Source records and inserted into new system prompts. Edit the source records to change this preview.

Language·Aug 10

Use Chinese by default when discussing Memory and other operational work.

Address·Aug 15

Use plain “you” with no invented honorific unless the owner explicitly chooses one.

Tool reporting·Aug 15

Never fabricate tool calls or tool results. Report observed results and state when no available tool returns a result.

Narrow width · interactive prototype

Below 720px, do not stack section navigation, list, and detail vertically. Click a Topic for details; Back returns to the list.

Memory
⌕ Search topics
▧

No topics yet

New durable topics appear here.

Narrow-width rules

Section navigation appears only in list mode. Detail mode uses the full content width and an explicit Back button. Core, Timeline, and Recent enter content mode directly when there is no file list. Window width never rewrites desktop-sidebar expansion state.

Width priority Content detail > context list > section labels. Hide section text first, retaining icons; do not compress body text below readable width.
AreaTarget structureProblem avoided
Top title barPage title and Memory settingsNo totals that existing APIs cannot accurately calculate; no writer/refresh status or duplicate explanations
Secondary sidebarFolder, Topic, Timeline year/month/day, Recent, and Core entries all share 36px row height, 10px horizontal padding, 8px radius, 13px text, and selection background. Folder and Timeline year/month reuse the same 14×14 linear ChevronRight SVG/path/rotation and numeric badges. Timeline expresses hierarchy through indentation only. Topics does not duplicate topics/core.mdmanaged by Core. Expansion controls avoid tiny filled triangles, and Timeline uses the same count presentation as Topics
Content headerName, type, path, modification time, view, and editing actions in one 76px areaRemove a separate metadata strip and scattered Core status
BodyPreview/Timeline/Recent/Core use a maximum 780px reading area; Edit remains a full-width monospace editorAvoid excessively wide text and large unstructured empty regions
Markdown previewTopics, Timeline, and Core interpret Memory’s Obsidian Flavored Markdown through existing marked and HTML sanitization. Named [^id] become sequential [1] superscripts; definitions form a separated ordered list at the end, supporting two-space continuation and ^[inline footnote]. Paragraph-end or standalone ^block-id becomes an invisible anchor only, accepting Latin letters, digits, and hyphens in IDs. Fenced code, four-space-indented code, and backslash-escaped footnote syntax remain literalDo not expose internal evidence/block IDs in prose; do not change editable disk source
Narrow widthSection icons plus single-page list/detail switchingDo not auto-collapse desktop sidebars, stack three panes vertically, or allow overlap
Upstream basis:Obsidian Flavored Markdown defines the combination of CommonMark, GFM, LaTeX, and Obsidian extensions; Footnotes defines numbered named footnotes, two-space continuation, and inline footnotes; Internal links defines block-ID placement and character set. Obsidian’s main application is closed source, and public API/help repositories do not expose its core renderer. This implementation follows official syntax and observable Reading/Publish output rather than copying unpublished code.

Implementation scope

Page layout is mainly maintained in apps/web/components/memory/index.tsx, parts.tsx and memory-page.module.css . Memory-specific reading transformations live in a dedicated Markdown helper, reusing existing marked and HTML sanitization without changing editable content, APIs, writer, or disk format.

Explicit exclusions

No home dashboard, charts, Topic-creation wizard, drag sorting, bulk editing, writer status, automatic sidebar collapse, new APIs, or dependencies. Footnote sources retain existing URLs; no new source-browsing API or Memory syntax in Chat Markdown. Without selected content, show only a concise selection prompt and existing settings entry.
Acceptance boundaries: Public entry /memory shows a fixed secondary sidebar and complete content area at desktop width. Recent/Core do not repeat title, count, view switch, or injection status in the middle sidebar; Core controls/status appear once in the right content header. Recent’s header shows current records and live memory.recent.limit, for example 7 / 50 records, never confusing record count with token budget. The top bar does not call truncated Recent counts total Memory. Preview, Timeline, Recent, and Core use centered document areas; empty Topics has its own empty-state card. Defined named and inline footnotes render in occurrence order as [1] superscripts and an end-of-document list. Two-space continuations belong to the preceding footnote; body text hides evidence IDs and valid paragraph-end block IDs. Links, repeated refs, and undefined refs behave predictably. Inline/fenced/list-fenced/four-space-indented code stays literal, as do escaped references. Arbitrary input cannot collide with internal placeholders. Timeline collapses by year/month/day and never uses derived-file rebuild times as record times. Core distinguishes read-only Prompt preview from editable Source records; Topics excludes core.md. Below 720px, no horizontal overlap; editor actions occupy a separate wrappable row. The page neither requests nor renders writer status, pending counts, Commitments, or Scheduler state.
Implementation status for this iteration: Recent and Core no longer duplicate context blocks in the middle sidebar. Recent title/capacity appear only in the right content header, reading current memory.recent.limit from Recent API response headers and showing “current records / configured limit.” Core Prompt preview / Source records, injection status, and tokens also appear once on the right. Timeline year/month/day collapse remains. Structured transactions removed survey-test project records from production Memory; normal background writers may create new Topics. Centered documents, card empty states, a single 76px header, 780px reading width, and wrapping narrow-window edit actions are in production code. Topic rows and Timeline date entries share 36px height, 10px horizontal padding, 8px radius, and 13px text; Folder and Timeline year/month share 14×14 linear ChevronRight and numeric badges, with indentation-only hierarchy. Memory Preview converts named/inline footnotes to Obsidian Reading/Publish-style numbered superscripts and ordered end lists, hiding valid trailing block IDs while preserving editor/disk source. Complete mutually exclusive list/detail modes below 720px remain outside this iteration’s production scope.

Implementation evidence

Commit 5f1f06c4 implements document areas, empty states, and compact context sidebars; 1c770a4e merges path metadata into the 76px header and standardizes 36px/780px dimensions; b583833d removes inaccurate top totals and adds narrow-window action wrapping; 7ef440c6 unifies secondary-sidebar rows for Topics, Timeline, Recent, and Core.check-memory-status changed from RED on the parent commit to GREEN. Memory settings, Markdown sanitization, TypeScript, production build, HTML, links, and diff checks pass. Memory Markdown tests cover named/repeated/inline footnotes, two-space continuations, literal inline/fenced/list-fenced/four-space code, escapes, placeholder collisions, hidden block IDs, and sanitization. A real Topic in the default port-18100 App displays [1]–[4] and an end footnote list with 1.–4. markers, without internal IDs in the body.

Compatibility requirements

Preserve four sections, existing requests, Core compare-and-set saves, Markdown sanitization, partial Timeline dates, keyboard focus, Core exclusion from Topics, and settings navigation. Footnote transformation occurs before sanitization; generated links/anchors pass through the same sanitizer. No new component library, API, database fields, automatic sidebar state, or test Memory.

10 Unified modification interface

Memory supports both direct Topic-file edits and structured calls by models or other programs. Neither replaces the other; they differ only in how callers express changes. All modifications use the same Runtime staging, validation, derived views, atomic installation, and Git process.

Method one · edit files directly

Background writers or organizing models create, rewrite, move, or delete topics/**/*.md inside Runtime-provided staging. Each Source in new footnotes is written as [short keyword](source_ref). The model chooses keywords and an existing Source, not the final URL. After editing it must not overwrite production Memory directly; it calls commit_edits so Runtime allocates/preserves IDs, resolves source_ref into archive links, validates Source/block links, rebuilds Timeline/Recent/Relations/Core, and installs everything. Web/CLI file edits use the same staged_edit entry.

Method two · structured interface

Model tools and owner REST accept memory_changes[]. Content operations are create_record, update_record, delete_record; structural operations are move_records. Runtime renders them into Topic Markdown and uses the same installation path as direct edits.

Source identity

source is immutable and must be verified. Runtime uses it to generate links, inspect trust state, and restore derived views. Display text cannot change the target.

Reference keyword

label is local display text for this citation, generated in the same Memory write call, not a permanent Source title. Memory-agent prompts and tool schemas require short context-relevant keywords and discourage Markdown, URLs, Source IDs, speaker names, or time templates. Apart from a nonempty-string Markdown requirement, Runtime does not assess keyword quality, truncate/rewrite labels, or add a model call for them.
Canonical output:{"content":"...","sources":[{"source":"provider/thread/message","label":"research scope"}]}. Topic files retain standard Markdown: [research scope](../../sources/...#source-...). The parser recovers Source identity from the link target, never from bracketed display text.
Reference interfaceReusable conventionProject choice
RFC 6902 JSON Patchadd/remove/replace/move are separate; no semantic merge/split.Preserve content CRUD versus structural move, targeting stable memory_id rather than exposing Markdown path pointers.
Notion block childrenContent creation also specifies insertion position, owned by the parent structure.create_record.destination describes Topic, heading chain, and position together.
ProseMirror TransformHigh-level join/split intentions decompose into verifiable transaction steps.Models plan merges/splits as update/create/delete/move; do not add ambiguous Runtime primitives.
Mem0 / LettaPublic Memory write APIs primarily create/update/delete records or blocks.Retain record-level CRUD and add batch moves required by multi-file Topics.
Structured operationRequired dataRuntime behavior
create_recordFinal content, time, nonempty sources[{source,label}], and destination: topic_path, headings, position; before/after additionally requires anchor_memory_id. Model schema uses undated to represent no date; owner REST also accepts null.Validate Topic, position, and every Source; retain label verbatim as display text, write canonical Markdown, and allocate block/evidence IDs.
update_recordExisting memory_id plus complete final content/time/sources.Update only record content, preserving block ID and position; replace the paragraph and evidence definition with final values.
delete_recordExisting memory_id.Delete the record and unused evidence definition; reject the whole transaction if block links would dangle.
move_recordsOrdered, nonempty memory_ids, and the same destination.as creation. Move records in the specified order, preserving IDs, content, time, Sources, and creation order. Reorder within a section or move across sections/Topics.
Shared transaction: Both methods edit staging, normalize Topics, validate Sources/relations, rebuild Timeline/Recent/Relations/Core, atomically install, and then record Git. Pre-installation errors cannot change production Memory. Post-installation Git failures must not pretend rollback occurred; results distinguish memory_committed and git_committed.
Compatible entry points:memory_update and POST /api/memory/changes Add memory_changes. Existing whole-Topic changes[path/action/content] and legacy unified diff patch remain supported. Each request selects exactly one of the three formats; all eventually call MemoryWorkspace.update.
Source input:memory_changes[].sources[].source may reference an existing verifiable archived Source, or request-level sources using a new-source-* label; Runtime converts the latter to a formal Source ID first. Callers cannot submit provenance, trust state, or owner identity. Prerelease source_refs: [string] remains accepted, using the original string as a compatibility display label; new requests expose only sources.
Compatibility and migration: Existing display labels remain parseable. Runtime does not guess keywords without a semantic model. New writes and explicit updates use labels supplied by the Memory agent; labels survive moves, derived-view rebuilds, and body updates unchanged. Source archives, real Source IDs, evidence IDs, and block IDs remain unchanged.
Position contract:destination.position Accept only start, end, before, after. before/after anchors must belong to the destination Topic/heading chain and cannot belong to the move set. Final order exactly matches memory_ids. Each ID accepts at most one content operation and one move; delete and move cannot target the same ID.
Combined operations: A semantic merge consists of one update_record, several delete_record , and any required move_records; a semantic split consists of one update, several creates, and required moves. Section/file merges and splits require only batch move. Runtime provides neither merge_records, split_record, place_records nor reorder_records.
Structural cleanup: After moves/deletes, Runtime removes heading chains without Memory records and deletes Topic files that contain no records. Derived Timeline, Recent, Relations, and Core rebuild in the same transaction; callers do not edit them.
Legacy-format boundary:mem_* records retain their IDs. Structural operations cannot split multiple old IDs or canonical aliases sharing one physical paragraph. Moves must include all IDs of that paragraph contiguously in original paragraph order in one move_records.
Evidence conflicts: When moving across Topics, if the destination or another source Topic in the batch has the same evidence label with different time/Source semantics, Runtime atomically rejects everything with INVALID_ARGUMENT . Identical semantics with different relative link paths are not conflicts.
Prerelease compatibility: Earlier structured-interface create/update/delete inputs are converted to new names by Runtime. Model schemas and this document expose only four official operations; legacy aliases are not extended.
Scheduler boundary: Local-owner prompt tasks may explicitly call memory_status, memory_get and memory_update to maintain long-term records linked by MemoryRef; Scheduler service does not implicitly dual-write. Deleted records leave no placeholder or ID forwarding; querying an old memory_id returns NOT_FOUND. External token capabilities and non-owner mutation scope remain deferred.
Model mutation scope: Interactive models, background writers, and organizers can rewrite, merge, split, or delete current Topics. Deleting a block requires removing/updating Topic links to it in the same transaction; Source archives remain append-only. Incremental writes and organization both attempt Git commits.
Acceptance entryObservable results
After direct file editing, commit_edits[research scope](source_ref) normalizes to a real archive link with the same keyword. Production Topic, Timeline, Recent, Relations, and Core reflect the edits together; the parser recovers full Source IDs. Validation failure leaves production files unchanged.
MemoryWorkspace.update(memory_changes=...)create_record/update_record accept sources[{source,label}] and preserve caller labels; legacy source_refs still works. The four operations work independently or together; any failure leaves revision and all Memory bytes unchanged.
memory_updatememory_changes, whole-file changes , and legacy unified diff share transaction behavior and return fields.
POST /api/memory/changesaccepts the same memory_changes and returns revision, changed_files, block/evidence IDs, and Git commit information.
Implementation brief: Scope is Writer Prompt, Topic footnote normalization/parser, structured memory_changes, Topic moves, and Timeline derived display. Source archives, retrieval identity, Preview renderer, database, and model-call count stay unchanged. Public-entry RED covers direct file editing and MemoryWorkspace.update(memory_changes=...); affected validation covers Topic parsing, structured modifications, Writer prompt, derived views, owner REST, HTML, and documentation links.
Source-label implementation status: implemented. Writer generates short keywords in the same call following prompts/schemas. Runtime preserves and validates full Source IDs, renders agent labels verbatim, and retains real identities during moves, Timeline rebuilds, and retrieval indexing. Structured interfaces expose sources[{source,label}]; legacy source_refs, pre-v2 provider archives, and Source filenames with encoded characters remain readable and writable. Each Source archive is scanned once per Topic-tree parse. Runtime no longer classifies, truncates, or substitutes labels semantically.
Retrieval performance and trust: Production BM25 uses a versioned disk cache and reuses in-process indexes by Topic/Source file signatures; embedding also reuses parsed events and document vectors. Changes to time, size, or inode invalidate caches, so content changes and Source promotion re-evaluate trust. BM25, Embedding, and Hybrid resolve trusted/pending from every Source reference before returning a Topic; pending-backed Topics never enter automatic context.
Implementation status (2026-08-15): Model tools and owner REST expose the four official operations. Whole-file changes and legacy unified diff remain compatible. Direct edits and structured operations share staging, validation, empty-structure cleanup, Timeline/Recent/Core derivation, atomic installation, and Git history. Public-entry regression tests cover batch moves, position/shared-paragraph checks, Recent creation-order preservation on first structural change, and cross-Topic evidence conflict rejection. Scheduler’s internal Agent lifecycle writes follow its design document; deleted records leave old memory_id returns NOT_FOUND, without retained records or old-ID mappings. External token capability remains deferred as specified here.
Implementation status (2026-08-11): per-turn and idle writes share one transaction; nodes store workspace write markers; session boundaries enumerate live branches; the resident block is rendered from topics/core.md; background writers use the default chat agent with live memory.writer.model updates. Unclassified exceptions are non-retryable by default.memory.backend=none disables prompts, recall, automatic writing, organization, and memory threads, and rejects all CLI memory verbs and /api/memory/* routes before workspace creation. Tools, CLI, and Web expose recent writer success, failure category, retryability, and pending counts. One-shot memory backfill is implemented for trusted Sources not referenced by Topics, excluding pending Sources and reusing normal writer transactions by batch. Component tests cover SessionDB, default-model resolution, managed tools, staging installation, Topics, markers, and watcher state. Production-workspace acceptance parsed all 23 old v1 Source files (154 frames) through read-only compatibility from origin_scope to authority_tier. Two pending messages produced three Topic files and five unique blocks; six Source refs and relation targets validated; both node markers were written. A second scan processed zero items without changing revision. Historical backfill was not run, leaving 152 unreferenced frames. Still unimplemented: unauthorized-request hold queue, requester-level filtering, mutually exclusive branch semantics, cross-session spawn relationships, and direct head-change-triggered writing. Failure reason codes are a closed enum; restricted writer phases cannot see Source archives.