File Modification Management — Industry Analysis and OpenProgram Design#

This document describes how OpenProgram tracks and rolls back file changes made by the agent: the two industry approaches, the four layers OpenProgram runs, and how they coordinate. Related: agent-worktree.md, overview.md (entity layer), git-as-entity-memory.md. Code: store/snapshot/checkpoint/, store/shadow_git/, store/snapshot/read_tracking.py, agent/internals/_revert.py, worktree/.

1. Two Approaches: Snapshot vs Sandbox#

For managing file modifications, AI coding agents split into two approaches across the industry:

It is not an either/or choice. Leading frameworks are already moving toward running both side by side — snapshots address the day-to-day rollback experience (second-level recovery), and sandboxes address security isolation (full bash coverage).

2. Industry Comparison#

2.1 Full Landscape Comparison#

2.2 Three Ways to Implement Snapshots#

There are three concrete approaches to snapshot mechanisms in the industry:

The three can coexist — they solve different problems:

• File copy: the fastest undo (no dependency on git; works for non-git projects too)
• Git commit: permanent history (queryable via git log, but pollutes user git)
• Shadow git: balances diff traceability with not touching user git

2.3 Terminology#

3. OpenProgram's Approach#

3.1 Design Principles#

1. Don't touch user git. Neither Claude Code nor Hermes writes commits into the user's repository; this is an industry consensus.
2. Unified entry trigger. Following Hermes, the checkpoint is triggered uniformly before all tool executions, covering bash.

3.2 The Four-Layer Mechanism#

Each of the four layers handles one concern, with no overlap. Removing any one of them leaves a capability gap.

                ┌─── read-before-edit (concurrency guard, the gate in front of all writes) ───┐
                │  before the agent writes: never read / disk changed after read → reject, make it re-read │
                └───────────────────────────┬──────────────────────────┘
                                             ▼

 ┌──────────────── Snapshot ────────────────┐   ┌──────────── Sandbox ─────────────┐
 │                                                  │   │                                         │
 │  ①  Checkpoint          ②  Shadow git            │   │  ③  Worktree        ④  System-level sandbox        │
 │     "rollback"                 "history"                 │   │     "file isolation"          "permission restriction"         │
 │     turn-level, temporary          independent store, persistent        │   │     independent copy            restricts bash scope     │
 │     doesn't touch user git           doesn't touch user git            │   │     agent enters explicitly      config toggle           │
 │     always on, automatic           on by default, automatic            │   │     on demand                workspace-write default │
 │                                                  │   │                                         │
 └──────────────────────────────────────────────────┘   └─────────────────────────────────────────┘

3.3 Division of Labor Between ① Checkpoint and ② Shadow git#

Both run simultaneously; it is not an either/or choice:

/rewind integrates the two on rollback: it first restores files from the checkpoint (fastest), and the shadow git records remain queryable.

4. Unified Entry Trigger — How bash Is Covered#

4.1 Implementation#

The three edit tools — write / edit / apply_patch — each call checkpoint_before_edit internally for a precise single-file backup.

bash coverage sits in _execute_tool_calls (agent_loop.py, the single entry for all tools):

#### agent_loop.py — inside _execute_tool_calls
pre_snapshot = _snapshot_cwd(tool_call.name)   # returns None for non-bash-like tools
result = tool.execute(...)                     # write/edit already have a precise backup internally
_checkpoint_changed_files(tool_call.name, pre_snapshot)  # compare, supplement a checkpoint for changed files

_snapshot_cwd(tool_name): only takes effect for bash-like tools (_BASH_LIKE_TOOLS). It scans the working directory recursively (using the worktree path when a worktree is active), records {path: (mtime_ns, size)}, and copies each file's pre-execution content into a temporary staging directory — otherwise, backing up from the original path after the command has run would already store the modified content. Files larger than _STAGE_MAX_BYTES skip staging. _checkpoint_changed_files: compares the before/after snapshots and checkpoints added/modified files from their staging copies.

5. Concurrency Guard: read-before-edit (the Front Gate)#

store/snapshot/read_tracking.py. This is the safety foundation of the whole mechanism: the agent never blindly overwrites a file that the user changed but the agent hasn't read since. Every entry that lands in ① checkpoint and ② shadow git is therefore a clean agent change, and rollback cannot damage the user's own edits.

It follows Claude Code's Edit/Write contract:

• read records the baseline — when reading a file, record its fingerprint (mtime_ns, size, sha1).
• Verify before writing — edit / write overwriting an existing file / apply_patch Update compares before writing:
  • Never read (NEVER_READ) → reject, prompt to read first.
  • Read but the disk changed (STALE, modified by user/linter/another process) → reject, prompt to re-read. The change is not written to disk.
• New files are skipped — write a new file / apply_patch Add doesn't require reading first; record the baseline after writing.
• A successful write refreshes the baseline — the same file can be modified again without re-reading.

It uses a content hash rather than just mtime: when the user types fast, a change may land in the same mtime tick, and looking only at the timestamp would miss it. The session is resolved via the _store ContextVar; outside of a turn (unit tests / standalone calls) the whole guard is a no-op (UNTRACKED → allow).

6. Implementation Details#

6.1 Coordination Rules#

Rule A: When a worktree is active, the shadow git commit yields

shadow_git.commit_turn_changes first checks find_active_for_session(sid):

• Active worktree exists → skip (the agent's changes are in the worktree copy; committing the original directory is wrong/empty).
• None → proceed as usual.

Rule B: /rewind = restore from checkpoint + keep shadow git queryable

When /rewind reverts:

1. Restore files from the checkpoint (the fastest path).
2. The shadow git history is not rolled back — it stays queryable, so the user can diff to see what the agent changed.
3. gitignored files / non-git folders: the checkpoint is the only fallback.

There's no need to decide between git reset vs revert; the rollback logic stays simple.

6.2 Full Lifecycle#

Take, as an example, a session bound to a real project directory:

session start
  │
  ├─ [turn N starts]
  │
  ├─ agent wants to edit a.py
  │     [front gate] read-before-edit verifies a.py freshness (never read / already changed → reject)
  │     [unified entry] checkpoint(turn=N, a.py)   ← copy old content into <session>/checkpoints/N/ before editing
  │     ↓ actually write a.py → refresh read baseline
  │
  ├─ agent runs bash "sed -i 's/old/new/' b.py"
  │     [unified entry] record the hash of the working directory's file state
  │     ↓ actually run bash
  │     [unified entry] compare hashes, b.py changed → supplement checkpoint(turn=N, b.py)
  │
  ├─ [turn N ends]
  │     · session-git commit (conversation DAG)              ← always happens
  │     · ② shadow git commit:                      ← on by default
  │         - active worktree? → skip (Rule A)
  │         - otherwise commit all file changes of this turn to the shadow store
  │     · GC: gc_evict_old(session) ← delete old checkpoints beyond the cap
  │
  ├─ user clicks ↩ rewind or types /rewind
  │     → Rule B: restore files from checkpoint, keep shadow git queryable
  │
  └─ ……

6.3 Releasing Checkpoints#

Checkpoints are stored in <session>/checkpoints/<turn_id>/, released as follows:

Note: a checkpoint is a full file copy (shutil.copy2), deliberately not a hardlink (the agent's open(w) truncates the inode, and a shared hardlink would lose the original content). Disk cost is linear in files×turns, capped by the GC limit.

6.4 Shadow git Storage#

Storage location: ~/.openprogram/shadow-git/<project-hash>/

• One shadow git store per project directory (distinguished by path hash)
• Doesn't touch the user's .git, fully independent
• Auto-commits all file changes of a turn at the end of the turn
• Supports git diff, git log, single-file recovery
• No GC needed (git compresses objects naturally)

6.5 The Relationship Among the Four#

The four layers are independent and solve different problems:

Coordination points:

• Checkpoint ↔ Shadow git: run simultaneously, coordinated uniformly by undo (Rule B). Checkpoint handles fast recovery, Shadow git handles permanent history.
• Worktree ↔ Shadow git: Rule A, Shadow git yields when a Worktree is active.
• System-level sandbox: fully orthogonal to the other three layers — it restricts what bash can touch, without affecting the operation of snapshots and file isolation.

6.6 Ad-hoc (Default Project) Sessions#

For casual chats not bound to a real directory:

• ① Checkpoint: as usual.
• ② Shadow git: as usual (the shadow store doesn't depend on the user's repository).
• ③ Worktree: not applicable (no source repo).
• ④ System-level sandbox: as usual (restricting bash scope is independent of project binding).

7. User Decision Matrix#

8. Usage Guide#

8.1 Web (webui)#

Rollback operations:

• Every user message has a ↩ button in its top-right corner ("Rewind to here") — after clicking it:
  1. The message's text returns to the input box (re-editable)
  2. This message and all subsequent conversation are removed from the interface
  3. Files are restored to their state before this message (via checkpoint)
  4. The old conversation is preserved in the DAG as a history branch, not lost
• Type /rewind in the chat box — lists the recent rollback points (up to 10), each showing a summary and timestamp
• Type /rewind N in the chat box — roll back to the N-th rollback point (N chosen from the list)

Sandbox:

• The system-level sandbox starts in workspace-write: writes stay inside the workspace and configured writable roots, configured credential paths are unreadable, and network access is disabled.
• Type /sandbox in the chat box to toggle between workspace-write and an explicit danger-full-access.

8.2 CLI#

• /rewind — list rollback points
• /rewind N — roll back to the N-th point
• /sandbox — toggle the sandbox

8.3 TUI Terminal Interface#

• /rewind, /rewind N — same as the CLI
• /sandbox — same as the CLI

8.4 Features That Take Effect Automatically (No Manual Action Needed)#

These features run automatically in the background, and the user doesn't need to do anything:

8.5 Tools Available to the Agent#

The only tools the agent (the LLM) can call are the worktree series, used for file isolation:

Checkpoint, Shadow git, and the sandbox are all automatically running low-level mechanisms, not exposed as tools. The user interacts with them through the /rewind and /sandbox commands.

9. Sandbox Isolation — ③ Worktree + ④ System-level Sandbox#

The three mechanisms below have different execution environments and operational requirements:

9.1 ③ Worktree — File Isolation#

The agent calls worktree_create to create an independent copy of the working directory; if it works out, worktree_merge; if it goes wrong, worktree_discard.

Limitation: it isolates only files, not processes or the network — bash can still rm -rf /, read ~/.ssh/, and access the network. It protects against breaking the code, not against bash acting outside its intended scope.

9.2 ④ System-level Sandbox — Permission Restriction#

Uses OS kernel mechanisms to restrict what the bash process can do:

• File system: reads are broadly available except configured deny_read paths; writes stay inside cwd and configured writable roots
• Network: disabled by default
• Implementation: Seatbelt (sandbox-exec) on macOS, bubblewrap on Linux
• Code: openprogram/sandbox/__init__.py (resolve_policy + wrap_command), backend/local.py (_invocation integration)
• Command: /sandbox toggle (CLI _cli_chat/handlers.py + webui ws_actions/chat.py)

This is the selected OpenProgram runtime boundary because commands use the host's actual Git, Python, Conda, npm, and compiler installations. It is implemented separately for macOS and Linux. Unsupported platforms refuse sandboxed execution by default; Docker is not an automatic fallback.

9.3 The Relationship Between ③ and ④#

The two solve different problems and can be combined:

• Using ③ alone: changes happen in a copy, but bash is unrestricted
• Using ④ alone: changes happen in the original directory, but bash is scope-restricted
• Combined: changes happen in a copy and bash is also restricted by the host-native policy

9.4 Optional Container Backend (Not in the Current Scope)#

OpenProgram does not require Docker for unattended execution. Authority and pairing decide which senders may enter the agent; the sandbox limits side effects made by an admitted agent while preserving the host development environment. Docker may be added later as an explicitly selected second backend when a concrete product requirement needs a separate Linux environment. It is not implemented now and will not replace or automatically back up the native sandbox.

Appendix: Implementation Status#

All four layers — ① Checkpoint, ② Shadow git, ③ Worktree, ④ System-level sandbox — are implemented, along with the read-before-edit guard and the /rewind and /sandbox commands across webui, CLI, and TUI.

Separate work: