OpenProgram / Provider design / implementation handoff

Request recording and strict offline replay

Design audit baseline: 990bfe36e625579443e1cb01a1b3266c8dbd0e87; lifecycle implementation review: 99252205957a2850e1203a888a0fa0b403b0c3f7. This page defines product entry points, the provider runtime lifecycle, registry wrappers, file safety, strict offline behavior, and verification contracts, with separate design and implementation evidence.

01 · Goals, non-goals, and security boundaries

02 · Current baseline implementation and gaps

03 · Corresponding design in the reference corpus

The repository reference snapshot contains only references/pi-ai. This section describes that snapshot alone, without claiming coverage of other versions or the entire open-source ecosystem.

Rejected approaches: per-vendor recorders, raw HTTP proxy capture, and test mocks as substitutes for product entry points. These duplicate implementation, expose unredacted wire data, or fail to cover actual worker configuration.

04 · Architecture

05 · CLI and configuration contracts

CLI syntax

openprogram recordings status [--json]
openprogram recordings record [--name NAME]
openprogram recordings replay <ID-or-PATH>
openprogram recordings off
openprogram recordings list [--json]
openprogram recordings show <ID-or-PATH> [--json] [--content]
openprogram recordings delete <ID> [--yes]
openprogram recordings prune --older-than-days N [--dry-run] [--yes]

Configuration schema

{
  "record_replay": {
    "mode": "off",          // off | record | replay
    "file": ""              // managed ID or explicit filesystem path
  }
}

openprogram config set can still edit each setting separately; the recommended recordings record/replay/off uses setup.update_config() to write both fields at once, avoiding intermediate states. Settings are read only when initialize_provider_runtime() moves from NEW to INITIALIZING, never after READY or FAILED. Changes require restarting the affected process, which command output must state explicitly. Management commands inspect/update settings without runtime initialization.

06 · Registry wrappers and lifecycle

Import contract

import openprogram.providers only defines Python APIs, exports model APIs, and loads existing static model modules. It must not call register_builtins(), read record_replay settings, open recording files, install a transform, or mutate the credential registry through auth-adapter imports. Isolated subprocess tests verify this contract; comments are insufficient. Removing other existing read-only initialization in model catalog modules is outside this task.

Responsibilities and interfaces

api_registry retains register_api_provider(api, provider), get_api_provider(api) and a once-only transform. The transform covers registrations made both before and after initialization.

configure_provider_transform(transform)
  - Set once per process; the same value is a no-op, conflicts raise RuntimeError
  - Retain the original provider; do not wrap an already transformed provider again
  - Atomically replace all current registry values
  - Later register_api_provider(api, original) stores transform(api, original) automatically

record:
  shared_sink = RecordingSink(path)
  transform(api, original) -> RecordingProvider(original, shared_sink)

replay:
  shared_replay = ReplayProvider(path)
  transform(api, original) -> shared_replay
  original only confirms that the API is registered; ReplayProvider neither retains it nor delegates to it

All recording wrappers share one sink; all replay APIs share one ReplayProvider, so call_index represents the total order of provider API calls in the process, rather than a separate sequence starting at 0 for each API. Direct library use still supports independent constructors; product activation must not require callers to enumerate APIs.

Process-wide state

NEW
  └─ initialize_provider_runtime() ─→ INITIALIZING
       ├─ success ─→ READY
       └─ any incomplete exit ─→ FAILED (immutable failure descriptor)

• State and waiting use one threading.Condition; one thread initializes at a time, while others wait for READY or FAILED.
• READY never rereads settings; changes continue to follow next_start.
• FAILED occurs only during built-in registration: the runtime stores the stage, cause type, and redacted cause message. Each caller receives a fresh ProviderRuntimeInitializationError with identical fields and message; do not reuse exception objects carrying old tracebacks.
• Record/replay activation failure does not set runtime FAILED: log the error, atomically replace the registry transform with a fail-closed _BlockedRecordReplayProvider, and enter READY with snapshot.mode == "blocked". Every provider call raises the same diagnostic, directing the user to openprogram recordings status or openprogram recordings off; no credential resolution or network access occurs.
• KeyboardInterrupt/SystemExit and similar control exceptions store a initialization_interrupted failure descriptor under the lock and notify_all(); the initializing thread rethrows the original control exception, while waiters and later callers receive a stable initialization failure. No in-process retry; restart begins at NEW.
• Construct RecordingSink/ReplayProvider and complete validation before publishing the transform; publication failure must not leave a partially replaced registry.
• Use fresh subprocesses to isolate off/record/replay/blocked/FAILED tests; add no production API that resets global lifecycle state.

Visibility of built-in registration

register_builtins() imports and validates built-in modules under its own lock, builds a local API-provider batch, and publishes it once through the registry. Failure before publication leaves the API registry unchanged; mark registered only after complete success. Anthropic/Codex/Gemini auth adapters retain their idempotent registration functions, moved from package import to this explicit phase. If an auth adapter fails, the provider runtime enters FAILED; public get_api_provider() cannot observe or use published providers because of the lifecycle gate. A process restart rebuilds them. Anthropic/OpenAI/Google SDKs are core dependencies, so wrapping the entire provider import in except ImportError must no longer hide internal defects.

Explicit startup and shared guarantees

Web and worker explicitly call initialize_provider_runtime() before starting recovery threads, dispatchers, or provider warm-up threads, so invalid replay is reported during single-threaded startup. Public get_api_provider() still performs an idempotent check, covering Python library calls and future entry points. Lazy auth-adapter registration calls only register_builtins(); a corrupt replay file cannot cause it to fail.

Runtime construction and credential boundary

Strict offline behavior must cover create_runtime(), not just stream.py. The current factory and subscription constructors read API keys/OAuth before get_api_provider(). The target does not refactor live runtimes: off/record retain the existing factory; replay adds a credential-free branch accepting explicit provider/model choices, or AGENTIC_PROVIDER/AGENTIC_MODEL and the existing non-credential selection in configuration default_provider/default_model. Without a selection, immediately return a replay configuration error; do not proceed to CLI or AuthStore detection.

The replay factory does not construct a subscription runtime. It uses pure provider-to-model-namespace metadata from PROVIDERS to construct a base Runtime. Plain providers already have this field; add model_namespace: openai-codex→openai-codex, gemini-cli→gemini-subscription to the subscription entries. API-routed providers retain their model-catalog namespace. This branch does not infer the original factory provider from the recording: v1 requests store only the wire-level Model, which does not encode the original runtime factory choice. Nor can the first model in a multi-API file define all subsequent runtime identities. Once a request enters ReplayProvider, existing full request comparison validates the final Model, context, tools, and options; mismatches use the existing ReplayMismatch diagnostic.

Relationship to the OpenProgram execution model

record/replay supports provider testing and failure reproduction, not workflow definition. Agent loops, Runtime retries, tools, Session persistence, and DAG state execute normally; only the registry provider changes from live/record to ReplayProvider. Replay can therefore verify upper-layer behavior against the same model-event sequence, but it does not restore historical Sessions, replay tool side effects, roll back files, or define execution steps.

07 · Recording files, strict parsing, and concurrent writes

Format policy

New writers continue to emit format_version=1 and the existing four line types.call_end.outcome is optional diagnostic metadata: absence means complete, preserving compatibility with old v1 files and readers. It does not change event models or request matching. Headers may gain optional metadata; v1 readers must ignore unknown header fields. Bump the version only when required request/event fields or semantics change.

{"type":"header","format_version":1,
 "recording_id":"20260811T134502Z-f7a29c1d",
 "created_at":"2026-08-11T13:45:02Z","redaction_version":1}
{"type":"request","call_index":0,"model":{...},"context":{...},"options":{...}}
{"type":"event","call_index":0,"event_index":0,"event":{...}}
{"type":"call_end","call_index":0,"event_count":1,"outcome":"complete"}

recording_id, created_at and redaction_version support management and diagnostics only, not request comparison. Old v1 files without these fields remain replayable; list/show derive missing information from stat and content.

Strict-reader invariants

Concurrent-write contract

1. RecordingSink holds a process-local threading.RLock; serialize and append each JSONL line within the same critical section.
2. Each append opens the data file with os.open(O_WRONLY | O_CREAT | O_APPEND, 0o600) and uses openprogram._compat.flock(fd, LOCK_EX) for interprocess exclusion.
3. Create the initial header and allocate the next call_index under the file lock. The minimal implementation may scan the maximum request index at begin_call: accept O(n) allocation on this diagnostic path, adding a sidecar counter only if measurements show large files are a bottleneck.
4. Write a complete UTF-8 JSON record plus newline using a os.write loop; do not assume one write completes it.
5. Do not force fsync after each request/event append; fsync after call_end. A crash may leave an incomplete call that the strict parser detects.
6. Events may interleave across threads/processes; call_index from event_index groups each call completely. Replay order follows request call_index.

08 · 0700/0600, path constraints, and redaction

Filesystem

• The managed root is get_state_dir() / "recordings"; add get_recordings_dir() and tighten it to 0700 after creation.
• On POSIX, tighten the managed recording root to 0700 whether new or existing. A library-level explicit external RecordingSink(path) does not change parent-directory permissions or follow parent symlinks to chmod. Create data files as 0600 on first write. After each open of an existing record target, call openprogram._compat.restrict_to_user(path) to repair overly broad permissions, then verify mode 0600 with fstat; reject recording if it cannot be guaranteed.
• Replay distinguishes managed and explicit external paths: managed files may be tightened to 0600; external files must already be regular files with mode 0600 and are not chmod-modified. Explicitly reject overly broad external permissions for the owner to resolve.
• Create lock files as 0600 too. list/show/delete/prune use lstat to reject symlinks and verify managed-root containment after resolution.
• Recording uses O_NOFOLLOW where supported and O_EXCL to initialize a new file; never truncate an existing recording. Name conflicts fail immediately. Without O_NOFOLLOW, open the basename relative to a managed-root directory fd, then compare lstat/fstat device, inode, and regular-file type against the opened result.
• delete/prune delete only regular files with the .jsonl suffix; no recursion, symlink following, or explicit external deletion.

Redaction contract

• remove_secret_values() runs before any serialization; record mode has no disable option.
• Retain the current case-insensitive sensitive-field set, including nested dicts/lists. Field names recognize -/_ and camelCase boundaries. Compound header/option names ending in token, key, secret, password are fully redacted too, such as X-Session-Token, service_api_key, refreshToken, clientSecret. Do not match non-credential counters such as token_count/tokenCount.
• Value patterns cover common Authorization schemes (Bearer, Basic, Bot, token, Key), URL userinfo, and the currently implemented query keys api_key/apikey/access_token/token; adding a query key requires implementation and tests together, not a design-only claim.
• Do not use generic high-entropy-string rules: they unpredictably change normal prompts and cause replay mismatches. New provider credential forms require deterministic patterns and regression tests.
• record and replay must dispatch using the same redaction_version. Treat existing v1 files without this field as redaction v1.

09 · Strict offline replay contract

Structural constraints enforce replay isolation; it does not depend on callers voluntarily avoiding network access.

1. The replay transform returns the same ReplayProvider.
2. ReplayProvider retains no original provider, imports no vendor transport, and offers no fallback.
3. stream.py must obtain the provider first; when the provider declares requires_credentials = False, do not call resolve_provider_key().
4. Fully parse and validate the file before transform installation; corrupt files block worker startup rather than failing midway through startup.
5. Unknown APIs still produce the registry missing-provider error; never dynamically load live providers for replay.
6. Request mismatches, exhausted recordings, unknown events, and event-validation failures raise directly; no failure may switch to a live provider.

Tests replace both socket.socket.connect, socket.create_connection and shared HTTP-client constructors with failing sentinels, asserting no network attempt on successful replay, mismatch, corrupt-file, and exhaustion paths.

10 · Recording-file management

list/show use the same pure inspect_recording(path) function, and replay activation uses the same strict parser. Do not add a permissive management parser. prune uses stat mtime for age; header created_at is display-only, preventing header edits from changing retention.

11 · Failure semantics

12 · Compatibility and migration rules

• Library API: RecordingProvider(provider, path), ReplayProvider(path), recording_path, call_count and existing exception attributes remain.
• Existing v1 files: Continue reading them; stricter checks reject truncated, orphaned, or duplicate lines previously ignored by permissive readers. This fail-closed behavior is intentional.
• Old writers: Headers containing only type/version remain valid; derive management metadata from stat/content and mark it legacy-v1.
• Default execution: mode=off takes one configuration snapshot on first actual provider lookup; later requests add no config-file I/O. Provider identity and credential-resolution order remain unchanged.
• Configuration: A missing record_replay subtree means off. Environment variables do not automatically enable it, and no scan selects the “latest recording.”
• Import: import openprogram.providers no longer guarantees a populated built-in registry; actual provider lookup and explicit register_builtins() take that responsibility. Public stream/complete behavior is unchanged.
• Management recovery: Remove _OPENPROGRAM_RECORDINGS_MANAGEMENT; structural separation guarantees status/off recovery, independent of import order.
• Markdown: record-replay.md/.zh.md tracks this page’s current CLI, strict offline guarantees, and lifecycle boundaries, replacing the obsolete test-library-only description.
• Future formats: Use separate parsers for version upgrades; never rewrite user files in place. Offer an explicit migrate command only for defined field-semantic changes.

13 · Minimal implementation files

Do not change vendor providers, off/record live factories, Web routes, Node CLI, or the recording format. Add only the replay branch to the runtime factory. Calls bypassing get_api_provider() or plugins bypassing register_api_provider() must return to the shared boundary; do not add vendor-specific cases in the recorder or lifecycle module.

14 · Acceptance and test matrix

Minimum post-implementation verification commands

pytest -q tests/component/providers/registry/test_provider_initialization.py tests/component/providers/registry/test_record_replay.py tests/component/providers/registry/test_record_replay_registry.py tests/component/providers/registry/test_record_replay_cli.py
  pytest -q tests/component/providers/adapters/test_scripted_provider.py tests/component/providers/registry/test_registry_from_config.py tests/unit/runtime/test_usage_stream_chokepoint.py tests/unit/providers/auth/test_auth_methods.py tests/unit/webui/test_web_config_schema.py tests/component/config/test_config_write_safety.py
python -m openprogram recordings --help
python -m openprogram config get record_replay.mode
python -m scripts.docs_site.build
python -m scripts.docs_site.checklinks
git diff --check

15 · Implementation and verification record

Design sources: current OpenProgram provider implementation, CLI/config entries, tests, and the in-repository references/pi-ai snapshot. Implementation-status updates must cite code symbols, tests, or commits; a written design is not an implemented capability.