Agent Resource Quotas and Task Lifecycle Governance

04 Inspect and configure global defaults and session overrides

Current behavior

The config schema includes collaboration limits and agent.resource_limits; the resolver handles global/session/parent/job sources, and environment configuration controls worker capacity.

User impact

Configuration presence does not establish verified enforcement at every entry point or UI. Show configured values, sources, and effective values without presenting incomplete enforcement as complete.

Gap

Multiple equivalent global, parent, and child settings make precedence difficult to explain; using 0 for both unlimited and disabled perpetuates ambiguity.

Target behavior

One minimal agent.resource_limits object holds global defaults; session metadata overrides matching fields. APIs always return source, configured, and effective values. Tasks may only tighten effective session limits.

agent.resource_limits:
  max_live_per_session: integer | null
  max_queued_per_session: integer | null
  max_jobs_per_session: integer | null
  max_total_tokens: integer | null
  max_cost_usd: decimal-string | null
  max_runtime_seconds: integer | null
  idle_timeout_seconds: integer | null

All non-null values must exceed 0; null means unset/inherit. For compatibility, queued, cumulative, and the four budget types are disabled by default.max_live_per_session defaults to current scheduler capacity when unset; otherwise its effective value is the lesser of configuration and scheduler capacity.OPENPROGRAM_JOB_WORKERS remains the sole source of global execution capacity, exposed through the read-only scheduler_capacity field.

Only the owner may change global defaults and session overrides. Agent/task requests may supply only stricter token/cost/runtime/idle budgets, without changing live, queued, cumulative limits or increasing any owner limit. Extend existing config get/set with these fields and reuse session metadata; do not create a second quota configuration service.

max_total_tokens and max_cost_usd count governed agent/Job invocations carrying job_id within the session; ordinary foreground main conversations are excluded.max_runtime_seconds and idle_timeout_seconds is a time ceiling for each Job and its ancestor scopes. Changes to live/queued/cumulative limits do not terminate existing live/queued work, but immediately constrain new admission or queued → live; token/cost changes apply the new session ceiling at the next reservation. Runtime/idle defaults affect only subsequently admitted Jobs, avoiding retroactive deadlines.

05 Admission and atomic reservation when an agent creates a task

Current behavior

Synchronous/asynchronous agent tools and busy-target dispatch call runner/governor admission. The governor checks depth, fan-out, queue, and cumulative counts transactionally; the model tool retains a process-local fan-out precheck. Ordinary busy-target messages enter the inbox before Job creation on delivery.

User impact

Identical requests can differ by entry point; concurrent frontends/processes may pass non-atomic checks simultaneously; failed creation may leave Job, branch, inbox, or count residue.

Gap

Unified admission, cross-process transactions, and idempotent identity exist in source. Rejection without Job-creation side effects and notification of admission refusal after ordinary message queueing still require entry-point verification.

Target behavior

Every execution-task creation path must call ResourceGovernor.admit_job. Perform pure validation first, atomically check depth/fan-out and resource limits and reserve in SQLite, then create the Job, any new branch, and Job-bound inbox entry and publish execution-accepted events only on success. Rejection creates no Job. An independently authorized ordinary message receipt may precede admission.

Unified coverage

• Synchronous and asynchronous agent(); agent(to=...) busy-target precreation; run_agent_turn_async.
• WebSocket/API spawn; model tools; any future direct JobRunner.spawn_job entry point.
• Retries with the same job_id and request fingerprint are idempotent reads and cannot increase fan-out, cumulative counts, or queue reservation again; differing content returns quota.admission_conflict.
• send_message creates no Agent branch, but a new Job it triggers requires admission and counts against the execution target session’s cumulative jobs and resource budgets. Ordinary inbox receipt has not yet created a Job and must not claim execution admission succeeded.

Durable intent across SQLite and jobs.json

1. Generate job_id, admission_id and request fingerprint; validate arguments, target, and permissions without mutation.
2. In the usage DB’s first BEGIN IMMEDIATE transaction, check depth, per-turn fan-out, and session/parent remaining capacity, insert a preparing admission, reserve one queue slot plus provisional fan-out and cumulative increments, then commit.preparing must be durable before jobs.json writes; every concurrent admission check includes its provisional occupancy to prevent cross-process overallocation.
3. Use openprogram._compat.flock to protect per-session task-file read-modify-write and persist the Job. Do not broadcast or create other irreversible side effects yet.
4. In a second BEGIN IMMEDIATE transaction, change admission from preparing to queued, finalize provisional fan-out/cumulative counts, and commit; only then create the branch and Job-bound inbox entry and broadcast execution-accepted status. An existing ordinary message receipt is linked to the admitted Job at delivery.
5. Any failure triggers idempotent compensation. On restart, a durable preparing with a matching Job is finalized in the second transaction; without one, release queue and provisional counts transactionally. SQLite transactions must not span jobs.json file I/O.

06 Queueing and live concurrency

Current behavior

The runner uses durable admissions and a dispatcher. The governor checks session/global live capacity before claiming queued work; queued work has no live claim.

User impact

Queue/cumulative configuration and live claims exist, but complete cross-entry enforcement, fairness and queue diagnostics still require verified evidence.

Gap

A durable dispatcher and atomic resource claims exist in source; fairness, worker-loss recovery and cross-process admission/release require the full acceptance gate.

Target behavior

Admission first enters a durable queue. The dispatcher submits only when global worker and session live capacity are available, releasing queue and acquiring live in one transaction. Queued work consumes no live slot.

Compatible Job states remain pending/queued/running/completed/cancelled/errored; resource state persists separately.stopping is especially important: a Job may display cancelled, but its live lease cannot return until worker/thread/runtime termination is confirmed. This prevents overallocation when the 30-second watchdog only changes status.

The dispatcher selects the earliest globally ordered admitted_seq task whose session currently has live capacity, skipping temporarily ineligible sessions so their head task cannot block others. The global worker pool still determines total execution capacity.

Only the single execution process holding the existing worker lock may call queued → live and submit to the executor. Web/TUI/CLI/API processes only admit and durably enqueue, preserving the single-worker topology and preventing separate global thread pools per process. Multiple execution workers would require a separate scheduling design.

07 Parent-child budget allocation and tightening-only inheritance

Current behavior

Jobs carry budget_scope_id and effective_limits. The governor creates parent-linked scopes, resolves ancestor limits, and checks ancestor balances transactionally during provider reservations; §01 records provider integration status.

User impact

The scope model supports shared parent ceilings rather than independent child allowances. Users need the effective ceiling and shared_remaining to distinguish a local limit from currently available shared capacity.

Gap

The primitives and production reservation path exist; explicit-child allocation, sibling contention, and all entry-point propagation still require the full acceptance matrix.

Target behavior

A task request may carry one limits object containing only token/cost/runtime/idle budgets. Explicit child limits cannot exceed effective session limits, parent-task limits, or current parent-scope remaining capacity.

Deterministic rule without explicit child allocation

Do not divide the parent budget equally in advance or predict future work from fan-out. Children join the shared parent budget scope. Every provider/tool reservation atomically checks the task’s own ceiling, every ancestor scope, and session scope; the first successful durable reservation acquires capacity. This avoids idle static shares and does not misrepresent current remaining capacity as a guaranteed child allotment.

The UI labels shared_remaining as a shared live balance, not an allotment. Explicit child budgets create stricter local ceilings without permanently deducting from the parent in advance; actual reservations consume parent balance.

08 Metering and enforced stopping for tokens, cost, wall-clock, and idle

Current behavior

Governed provider calls reserve through BudgetedRequest before credentials or network access, clamp output, and settle terminal usage. The runner has a budget loop and bounded-operation checks; ungoverned legacy paths retain best-effort usage recording.

User impact

Protection depends on Job attribution and an audited provider/operation boundary. Unknown cost must remain distinguishable from zero, and a timeout result alone does not prove a synchronous thread stopped.

Gap

Reservation, attribution, unknown-cost handling, activity tracking, and operation boundaries exist in source. Adapter coverage, late settlement, and operation-specific stop confirmation require runtime acceptance; those gates were not rerun here.

Target behavior

The existing usage ledger alone records actual usage; add reservation state in the same DB. Atomically reserve before model/tool calls, settle using final provider usage afterward. Active timers and boundary checks jointly trigger existing cascading cancellation.

Explicit bound for non-preemptible operations

Enforce exhaustion at model-request/tool-call boundaries, with active timers triggering cancellation promptly. Do not describe an executing operation as instantly terminated. Its maximum non-preemptible interval must equal the minimum of declared operation timeout, remaining task runtime, and system cancel grace. Synchronous in-process operations that cannot meet this must move to terminable subprocesses or be rejected for strict-budget tasks; asyncio.wait_for returning does not mean the executor thread has ended.

Accounting delay and crashes

• Reservation states are reserved → started → settled, or, before provider I/O begins, reserved → released. Release not-yet-started reservations if credential resolution, request configuration, or provider-generator creation fails; after successful start, cancellation, transport errors, or missing terminal usage retain conservative exposure for late settlement of the same reservation. Absence of a first stream event does not prove the provider never received the request.
• If a request started but final usage is unavailable, retain conservative unsettled exposure; do not fabricate actual tokens/cost or release immediately and allow repeated overspending.
• Budgeted tasks fail closed if the accounting DB is inaccessible before calling. Post-call settlement failure marks the task errored, stops further calls, and retains the reservation for reconciliation.
• Existing best-effort recording may remain for non-budgeted legacy paths; governed calls require acknowledged reserve/settle protocols.

09 Slot release and cumulative semantics after cancellation, failure, timeout, and completion

Current behavior

Job terminal state and resource admission/lease state are separate. The governor retains cumulative admissions and supports stopping and confirmed release; runner completion and recovery use these primitives.

User impact

A cancelled Job can still occupy live capacity while its execution is stopping. The UI must distinguish requested cancellation from confirmed exit and quota release.

Gap

The state separation and cumulative-retention rules are defined in code. Watchdog, late completion, and crash interleavings still need full-matrix verification; absence of this review evidence is not absence of the primitives.

Target behavior

Release queue/live only when execution termination is confirmed or lease-recovery rules hold. Increase cumulative count once on successful admission and never return it for any terminal state.

Cumulative counts last for the session lifecycle. There is no implicit reset, and session archiving does not return quota. Users can create a new session or have the owner explicitly raise its override. Lowering limits cannot delete existing tasks; usage above the new limit only rejects later admissions.

10 Restart recovery, cross-process concurrency, and diagnosis

Current behavior

Job storage uses cross-process flock. Runner startup and its reconcile loop use durable admissions, owner leases, and ResourceSaga recovery; legacy unlinked Jobs follow the existing orphan policy. The worker lock keeps execution on one worker.

User impact

Admission and recovery span SQLite, Job files, and runtime ownership. Diagnosis must distinguish restored queued work, worker loss, and released resources while retaining the original execution identity.

Gap

Owner/lease, preparing reconciliation, file locking, and structured reason primitives exist. Process-death timing, simultaneous frontend writes, and repeated recovery still require the acceptance matrix; this review did not rerun it.

Target behavior

Single-worker execution remains supported; multiple UI/API processes compete through SQLite admission transactions. Each live admission carries owner/lease; reconciliation uses intent, Job, and lease facts to prevent permanent quota leaks.

Recovery rules

• preparing + Job exists: validate admission/job ids, then finalize as queued;preparing + no Job: roll back queue and provisional cumulative delta.
• queued re-enters the durable dispatcher after restart without consuming live capacity.
• live/stopping with an absent owner and expired lease: first mark the Job errored under the existing orphan policy with reason error.worker_lost, then release live; another instance cannot claim an unexpired lease. Initially renew every 10 seconds with a 30-second TTL, persisted using UTC wall-clock. If the original owner still holds the worker lock, an apparently expired wall-clock lease must not be released. These are not user quota settings.
• A lease proves worker ownership, not rollback of external side effects. Preserve worker-lost diagnosis in Job output.
• All jobs.json read-modify-writes use cross-process flock; SQLite uses WAL, BEGIN IMMEDIATE, and unique keys for idempotent admission/reservation.

This design adds no cluster consensus or remote scheduler. If multiple workers are later allowed, the same job_admissions lease/owner contract may extend, but leader election, clock assumptions, and external-runtime fencing require a separate design.

11 User-visible state and reason codes in Web, TUI, and CLI

Current behavior

JobResourceView and stable resource reasons exist, with Web subscriptions, terminal/CLI adapters, and list_jobs/job_output projections. Cancellation uses canonical execution control.

User impact

Users need the same queue, cumulative, cost-known, and stopping explanations across surfaces. Data projection alone does not establish that every field is correctly displayed in every state.

Gap

The shared DTO exists; cross-interface field/reason parity, legacy records, unknown-cost display, and recovery states still require runtime and visual acceptance. No standalone Tasks panel is claimed.

Target behavior

Web, TUI, CLI, and model tools consume the same JobResourceView; wording may vary by interface, but fields and reason codes must match.

Stable reason code