# Runtime

Agent execution runtime — the run loop, worktrees, async tasks, streaming/resume, the DAG model, and revert layers.

For API usage, see [Runtime API](../../api/runtime.md). The following sections explain the runtime contracts and their design choices.

<div id="runtime-design-rationale"></div>

## 1 runtime = 1 session

Each `Runtime` instance is bound to one provider session, with a 1:1 lifecycle:

```
create_runtime()      = open session
runtime.exec()        = send one request within the session
runtime.close()       = close session
```

There is no `reset()` / `new_session()`; to get a new session, call `create_runtime()` again.

**Why not let one Runtime reuse multiple sessions?**

- For CLI providers (Claude Code / Codex / Gemini CLI), session state lives in the subprocess.
  Reusing would require hanging mutable state like "which session id is current" on the Runtime, introducing concurrency races.
- API providers are themselves stateless; doing "multiple sessions" is just an upper-layer dict, equivalent to "multiple Runtimes" —
  no new capability.
- ContextVar auto-injection (next item) relies on the simple model of "the runtime is bound to the current function tree";
  multiple sessions would complicate the injection semantics.

The cost: a user who wants to run two independent conversations has to manage two runtime objects. This is acceptable, because such scenarios are rare.

## ContextVar auto-injects the runtime

The Agent method execution reads the `_current_runtime` ContextVar; if the current function
was not passed a `runtime=` argument, it uses that. If the entry function has none either, it automatically calls `create_runtime()`.

**Why not have functions declare the runtime explicitly?**

With explicit declaration, every agentic function would have to add `runtime: Runtime` to its signature,
and every nested call would have to explicitly pass `runtime=runtime` through — pure plumbing
boilerplate unrelated to the function logic. ContextVar hides it away: a child function naturally inherits its parent's
runtime, one is created automatically at the entry point, and it is closed automatically at exit.

**Why not use a module-level singleton?**

A singleton is shared across threads / coroutines, so two concurrent agents would step on each other's session state.
ContextVar is isolated per thread + coroutine, making it naturally concurrency-safe.

## Session-provider and API-provider share one abstraction

Whether the backend is the Claude Code CLI (has a session) or the Anthropic API (no session),
`Agent` method authors see the same interface, `llm(...)` / `agent(...)`, both of which end in `runtime.exec`.
The framework uses the `has_session` attribute to distinguish the two provider classes and take different internal paths:

| | session provider (CLI) | API provider |
|---|---|---|
| Conversation memory | managed by the subprocess itself | none; each exec is independent |
| Context injection | skips DAG render, sends only the docstring + the current content | assembles history from the DAG via `render_context` + `render_dag_messages` |
| `render_range.subcalls` | no effect (the session remembers the conversation itself) | takes effect (used to bound the window of injected history) |

**Why not split into two separate Runtime classes?**

When writing `gui_agent`, the author should not care which kind of provider the backend is. Forcing a split would require two
implementations per function, violating the layering of "functions describe the task, providers describe the execution channel." The cost of the shared abstraction is
the single `has_session` conditional branch — worth it.

## Retry lives in the runtime layer, not the provider layer

`exec()` / `async_exec()` have a built-in `max_retries`, defaulting to 2, so any provider
gets retries.

**Why not add retry to each provider class separately?**

- The retry policy is consistent across all providers (network timeout / rate limit / 5xx); no need to duplicate it
- Failure reports have a uniform format (`Attempt N: ErrorType: msg`), which aids debugging
- Programming errors like `TypeError` / `NotImplementedError` are uniformly not retried (only the
  runtime layer knows "this is a provider implementation bug, retrying won't help")

The provider layer only cares about "send the request out, get the reply back." Cross-cutting concerns like retry / throttling / caching
all live in the runtime.

## DAG writes: entering and exiting a function both write a code node, exec writes an llm node

```
enter Agent method       → write a code node (status=running)
                              → set the _call_id ContextVar to point at this node
runtime.exec() in the body    → write an llm node under the current _call_id
                              → the node's caller = _call_id
exit the function (return / except) → backfill the same code node's output / status
```

When `expose="hidden"`, the code node write is skipped (but a phantom `_call_id`
is still set, so that LLM calls inside the function body have a frame to reference).

**Why not write a single completed-state node only on exit?**

- While the function is still running, the webui visualizer needs to immediately show "it's running" (display a spinner)
- On an exceptional exit there must also be a node present (so the error information can be recorded)

Writing twice (entry + exit backfill) suits real-time observability better than writing once (on completion).

## Embedding seams

The core runs embedded in a host that is not OpenProgram — somebody else's
service or agent framework using `Agent` method + the execution DAG as a
component (see
[Embedding in your own stack](../../../capabilities/agentic-programming/embedding-in-your-own-stack.md)).
The contract, executed by `tests/component/runtime/test_standalone_embed.py` rather than
stated in prose only:

- **The library face never imports a UI surface.** `import openprogram` and a
  full `Runtime(call=...).exec()` round trip must not pull in `webui`, FastAPI,
  Uvicorn or Textual.
- **No implicit state paths.** With `SessionStore(root_path=...)` +
  `session_scope(store, session_id)` the run never consults
  `openprogram.paths` (`~/.openprogram`); with no store bound it runs and
  persists nothing.

Where the platform needs to reach into the core, the direction is inverted
into registration hooks, mirroring `add_pre_invocation_hook`:

| Seam | Core default | Platform registration |
|---|---|---|
| `set_cancellation_check` | no-op — nothing cancels | `openprogram.agent.run_control` registers the exact execution check at import |
| `set_session_id_provider` | `None` — `runtime.can_ask()` is `False` | `openprogram.agent.run_control` provides the active session id |
| `session_scope(store, id)` | unset — no persistence | the dispatcher binds the per-turn store itself |

**Why hooks instead of a web UI cancellation module?** The previous
session-scoped module duplicated cancellation state and made the core depend
on the Web UI. The hook keeps the dependency one-way while `run_control`
checks the exact execution token.

**`ImportError` during `exec()` is permanent, not transient.** The retry loop
classifies it as non-retryable: a missing subsystem does not appear by
retrying, and burning the full backoff schedule turned a 0-second failure into
a ~52-second one.

## Related implementation files

- `openprogram/agentic_programming/runtime.py` — Runtime base class, `exec` / `_call` protocol, retry loop
- `openprogram/agentic_programming/call_state.py` — decorator / `_inject_runtime` / `_call_id` / `_current_runtime` ContextVar
- `openprogram/providers/__init__.py` — `detect_provider` / `create_runtime` auto-detection
- `openprogram/providers/<vendor>/runtime.py` — each provider's `_call` implementation
- `openprogram/context/nodes.py` `render_context` — DAG → reads computation (the actual semantics of `render_range`)
- `openprogram/context/render.py` `render_dag_messages` — reads → provider messages conversion

## Related designs

- [`dag/overview.md`](dag/overview.md) — **authoritative**: the execution-record data model (one single graph / three node roles user·llm·code / caller+predecessor edges / render_context retrieval)
- [`dag/rendering.md`](dag/rendering.md) — **authoritative rendering spec**: layout / edges / legend / default visibility, 12 scenarios
- [`dag/branch-collaboration.md`](dag/branch-collaboration.md) — branch collaboration (communication / dispatch / merge) design and implementation steps
- [`execution/agent-call-flow.md`](execution/agent-call-flow.md) — the call-flow skeleton (turn / loop, orthogonal to the node model)
- [`execution/agent-worktree.md`](execution/agent-worktree.md)
- [`execution/async-job-lifecycle.md`](execution/async-job-lifecycle.md)
- [`execution/control.html`](execution/control.html) — **authoritative**: unified pause, continue, step, steering, cancellation, checkpoint, revision, and recovery contract
- [`execution/dispatcher-split.md`](execution/dispatcher-split.md) — break `agent/dispatcher.py` into a responsibility-scoped package (no-1000-line rule)
- [`operations/file-management.html`](operations/file-management.html) — authoritative file attribution, Review, Undo, Restore, branch alignment, and multi-agent ownership
- [`session/`](session/) — the session subsystem: data model, storage, naming, listing, lifecycle, broadcast
- [Nested `llm()` streaming design](https://github.com/fzkuji-neo/OpenProgram/blob/main/docs/reference/design/runtime/agentic-llm-streaming.zh.html) — **Full design** (zh): per-node token streaming (protocol, FE buffer, persistence, cancel/reconnect, in-process+subprocess)
- [`operations/streaming-resume.md`](operations/streaming-resume.md)
- [`operations/user-input-requests.md`](operations/user-input-requests.md) — pause a running function to ask the user (`runtime.ask`/`confirm`), question registry + WS/REST protocol + subprocess bridge
- [`agent-collaboration.md`](agent-collaboration.md) — **authoritative**: agent collaboration as one cross-branch communication primitive — the four domains, the tool surface, the three budgets ([the tool surface rendered](agent-collab-architecture.html), [the eight reference implementations compared](agent-collab-comparison.html))
- [`sandbox-architecture.html`](sandbox-architecture.html) — the canonical execution-security design: authority tiers, permission modes and approval, sandbox enforcement, framework comparison, and implementation evidence. [`permission-model.md`](sandbox-architecture.html) and [`sandbox.md`](sandbox-architecture.html) remain stable link targets.

- [Nested LLM node content (ordered chat blocks)](https://github.com/fzkuji-neo/OpenProgram/blob/main/docs/reference/design/runtime/nested-llm-node-content.zh.html)