Storage consistency

This document defines timestamp, snapshot, merged-head and lock-scope contracts for SessionStore. See Session storage for the data model. Each contract has its own acceptance boundary; implementation evidence is collected at the end.

Registry timestamps and snapshots

updated_at is the last content-append time, not the time of an arbitrary metadata write. Registry reads, mutations and snapshots share a state lock. Disk publication is serialized separately; filesystem I/O never holds that state lock. Deferred flush must preserve mutations newer than its snapshot.

Operation	Contract
_persist_meta	Rename, pin, read markers, branch markers and HEAD persistence preserve the content timestamp. Append explicitly advances it.
_index and list_sessions()	Readers get a point-in-time copy of each row. Concurrent creation, mutation and deletion cannot tear fields or invalidate iteration.
Deferred flush	An older snapshot cannot overwrite a newer one. Mutations during publication remain dirty and are saved by a subsequent flush.

Mechanisms and alternatives

Reuse the SessionMemoryIndex boundary: update state under its lock and perform file I/O after release. Use standard threading locks: one registry state lock for dictionaries, dirty state and timers, and one write lock for publication ordering. Existing atomic_write_text provides atomic replacement.

Adopt explicit content timestamps, shallow row snapshots and serialized atomic writes. Remove implicit timestamp advancement from metadata persistence. Reject a global lock around every store operation, synchronous registry writes for every append, a new timestamp field requiring consumer migration, a database, cross-process registry locks and new background services.

State flow

1. Appending a message or node generates one timestamp for both session metadata and its registry entry.
2. Rename, pin, read and branch/head changes persist the existing timestamp.
3. Registry mutations hold the state lock. Listing copies each row under the same lock, then filters and sorts outside it.
4. A writer acquires the write lock, then captures JSON state and clears only the covered dirty state under the state lock. It releases the state lock before atomic disk publication.
5. A concurrent mutation marks the registry dirty again. A later timer or exit flush publishes that subsequent state.

Failure and compatibility

• Disk failures retain the existing warning behavior. In-memory state remains available; a subsequent mutation/flush can attempt publication again.
• Returned rows are copies: caller mutations cannot modify the registry.
• Creation preserves an explicitly supplied historical timestamp. Rebuilding an index does not migrate its schema.
• Sort by descending updated_at, treating absent values as zero. Wire shape and fields remain unchanged.
• Registry work does not alter the node DAG concurrency model, credentials, authentication or orphan cleanup. Merged heads and location publication have the separate contracts below.

Registry acceptance

Public entry	Scenario	Observable result
update_session	Rename, pin or mark a session read	Metadata/index timestamps and list ordering stay unchanged.
append_message	Append content	Metadata/index advance together and the list uses the new timestamp.
list_sessions	Mutate a returned row	A subsequent list is unaffected.
Deferred flush	Mutate while an earlier snapshot writes	A subsequent flush retains the new fields in index.json.
Concurrent update/list/flush	Run readers and writers together	No iteration errors; JSON remains parseable; final memory and disk state agree.

Atomic merged-head updates

mark_merged(session_id, head_ids) is a set-union operation. Child-agent completion, task runners and Web branch actions can call it concurrently; both updates must survive.

Property	Requirement
Atomicity	Read the existing list, deduplicate/append, and update idx.meta in one idx._lock critical section.
Ordering	Preserve existing order; append new, nonempty, stripped IDs in each call's input order.
Idempotency	Repeated IDs create no duplicates. No new values means no disk write.
Persistence	Update memory under lock, then call _persist_meta outside it. At least one final snapshot from concurrent calls contains the complete union.

This contract is limited to the read-modify-write operation. It does not introduce a transaction API or alter branch naming, DAG deletion or the session-index flush protocol.

Acceptance: two concurrent callers appending different heads preserve both in memory and after reload; whitespace and duplicate inputs preserve normalization and idempotency; persistence runs outside idx._lock; exceptions and caller behavior remain unchanged.

Store lock scope

SessionStore._lock protects location mappings and the LRU cache only. Directory scans, atomic writes, fsync and recursive deletion must not hold it, so slow I/O for one session does not block unrelated sessions.

Resource	Lock	Requirement
_locations, _sessions, LRU order	_lock	Read, mutate or copy memory only; no filesystem I/O.
Initial load, rebuild, deletion and cache invalidation for a session	Per-session RLock	Serialize the same session; allow different sessions to proceed concurrently. No generic transaction layer.
locations.json publication	Dedicated write lock	Serialize snapshot capture and publication; capture under the global state lock, then release that state lock before writing. An old snapshot cannot overwrite a newer one.
SessionMemoryIndex persistence	_persist_lock then idx._lock	Preserve lock ordering and stale-rebuild semantics. No I/O under idx._lock.

This boundary does not change file formats, LRU capacity, deletion semantics, cross-process stale detection or Windows behavior.

• While session A blocks in read_meta/list_history, session B's cached read completes in bounded time.
• While A's recursive deletion blocks, B's read does not wait for A.
• Concurrent first loads of one session publish only one cached object; deletion and loading cannot republish a deleted object.
• After concurrent location updates for two sessions, reloaded locations.json contains both.

Cache snapshot freshness

Warm rebuilds and first loads capture the disk fingerprint before reading history and metadata. The rebuilt index records that captured fingerprint. A concurrent atomic replacement changes the fingerprint, so the next public read rebuilds again. An old snapshot must never be marked with a newer disk fingerprint. This preserves dynamic DAG context and terminal-state visibility without new storage or global locks.

Acceptance uses two real SessionStore instances. A history or metadata write completes after a reader has loaded the prior snapshot and before it marks synchronization. The next get_nodes or get_session call sees the new value, for both cached and first-load paths. Cancellation retry keeps its existing deadline and observes the actual cancelled file.

Windows session directory removal

Explicit deletion, expired-session cleanup and capacity cleanup share the platform removal adapter. If removing a regular Git object fails because its Windows read-only attribute is set, clear that attribute and retry the failed removal once. Other permission failures retain each caller's best-effort behavior. Do not change ACLs or follow links to change target permissions. Missing-history eligibility and archive retention rules are unchanged.

Acceptance uses public SessionStore reopen and delete_session entries with a read-only Git object. The native Windows regression checks that the directory is actually gone, not merely absent from the index. This follows the standard shutil read-only removal recipe, with Python 3.11-compatible error callbacks and a read-only attribute check.

Implementation status and evidence

Source	Responsibility
openprogram/store/session/session_store.py	Timestamps, registry state/write locks, snapshots and deferred flush.
openprogram/store/session/session_node_writer.py	One activity timestamp per append, shared by metadata and index.
tests/component/store/test_session_index_consistency.py	Public regressions for ordering timestamps, snapshots and concurrent writes.

The registry timestamp/snapshot implementation and the Windows removal adapter have independent specification and quality review evidence recorded in their task ledgers. Windows acceptance requires native Windows CI; local simulation is insufficient. The merged-head and lock-scope sections preserve their design and acceptance contracts; consolidating these documents does not add implementation or platform-acceptance evidence.