OpenProgram · Provider / Runtime design · implemented and reviewed · 2026-08-12

JSON Schema Structured Output

Design baseline: main@708bfbab. Implementation evidence: Tasks 1–6; final review commit 2fb471b3. Sections 2–15 retain the approved design and implementation checklist; §16 separates implemented contracts, verification evidence and unmet complete gates.

1. Scope, goals and boundaries

Function-tool arguments use the canonical tool schema for local validation. Strict provider export represents optional fields as required nullable fields; shared argument normalization restores those added nulls to omitted object properties before canonical validation. Required fields and originally nullable values retain their semantics. Only nulls accepted by the actual strict property schema are eligible, so enum, conditional and combinator constraints remain enforced. Normalization copies inputs, preserves array positions and applies the same optional-default semantics to non-strict callers. It does not modify function signatures or weaken canonical schemas.

Excluded: lossy schema-to-provider-subset rewriting, automatic Pydantic types, cross-language code generation, tokenwise incremental JSON AST, Web schema editor, or changed ordinary-text returns/events.

2. Baseline audit (main@708bfbab)

This section records project-start facts and gaps as the basis for design decisions; it does not describe the current state of 2fb471b3 .

Call and return chain

Schema must travel through this existing chain. Do not add a structured provider client bypassing the Agent loop, which would create separate tool-calling, deadline, record/replay, DAG and persistence semantics.

3. Public API and normalized types

Retain the parameter name response_format because it is already public and documented. Support bare JSON Schema and an envelope containing OpenProgram controls. Normalize exactly once.

JsonValue = None | bool | int | float | str | list["JsonValue"] | dict[str, "JsonValue"]

class JsonSchemaOutput(BaseModel):
    type: Literal["json_schema"] = "json_schema"
    schema: dict[str, Any]                  # Original caller schema; never mutate in place
    name: str = "response"                 # ^[A-Za-z_][A-Za-z0-9_-]{0,63}$
    description: str | None = None
    strict: bool = True
    fallback: Literal["auto", "none", "prompt"] = "auto"
    max_validation_retries: Literal[0, 1] = 1

# Existing-call compatibility: a bare dict is the schema, not a provider payload.
Runtime.exec(content=..., response_format={"type":"object", ...}) -> JsonValue
Runtime.exec(content=..., response_format=JsonSchemaOutput(...)) -> JsonValue
Runtime.exec(content=..., response_format=None) -> str

• Use overloads for return types; ordinary text calls still return str. Successful structured calls do not return JSON strings.
• async_exec, exec, AgentSession and dispatcher use the same JsonSchemaOutput; provider-private response_format payload.
• is not accepted. Before any network request, invoke validator_for(schema).check_schema(schema). Unknown meta-schemas or invalid schemas raise StructuredOutputSchemaError without consuming a provider request.
• Declare pyproject.toml directly in jsonschema>=4.23. An existing transitive lockfile dependency is not a core-feature contract.

Public provider fields

class Model(BaseModel):
    structured_output: bool | None = None   # models.dev: true / false / unknown

class StreamOptions(BaseModel):
    ...
    response_format: JsonSchemaOutput | None = None

class AssistantMessage(BaseModel):
    ...
    structured_output: JsonValue | None = None
    structured_output_mode: Literal["native", "tool", "prompt"] | None = None
    structured_output_attempt: int | None = None

None means unknown capability, not False. Raw JSON text remains in assistant text content for history, DAG, audit and replay; program calls return structured_output.

4. Capability negotiation

The model catalog says only whether the model claims structured-output support. API adapters must also specify payload dialect, streaming and tool-combination constraints; negotiation checks both.

class StructuredOutputCapabilities(BaseModel):
    native: Literal["supported", "unsupported", "unknown"]
    dialect: Literal["openai_chat", "openai_responses", "anthropic",
                     "google", "bedrock"] | None
    streaming: bool
    with_tools: bool
    schema_profile: str                 # Provider-subset checker name

negotiate(model, api_provider, options, tools) -> StructuredOutputPlan
# plan.mode: native | tool | prompt
# plan.provider_schema: Semantically equivalent projected copy
# plan.original_schema: Used for final local validation

1. Invalid schema: fail immediately.
2. If the adapter explicitly supports the API, the model field is not False, schema projects losslessly and current tools are supported, choose native.
3. fallback="auto" and the provider supports strict tool schemas, schema projects losslessly, and tool_choice does not prohibit/force another tool: choose a hidden tool.
4. fallback="prompt": explicit prompt fallback guarantees only local validation, not provider-level strictness.
5. Otherwise raise StructuredOutputUnsupportedError before sending, reporting provider, model, conflicting capabilities and fallback reasons.

Provider mappings and initial boundaries

External contract references: OpenAI Structured Outputs, Anthropic structured outputs, Google GenerateContent API, Amazon Bedrock structured outputs, Azure OpenAI structured outputs. These interfaces change; adapter contracts are required before enabling capabilities.

5. Native mode and fallbacks

Native schema

Adapters only map normalized configuration to provider payloads. Existing providers/_schema may supply traversal/dialect detection, but transformations must preserve semantics, such as key naming or JSON-string serialization. Dropping constraints, relaxing additionalProperties, rewriting unions or any other lossy transform reports the exact path and disables native mode. Final validation always uses the caller’s original schema.

Hidden-tool fallback

{
  "name": "__openprogram_submit_json",
  "description": "Submit the final response matching the required schema.",
  "parameters": <provider_schema>,
  "strict": true
}

• Send the hidden tool alongside caller tools. Ordinary tool calls execute normally and continue to the next round.
• The hidden tool must appear alone. Its arguments are the final candidate; do not execute it or generate ToolResultMessage.
• The fallback round sets parallel_tool_calls=False. If the caller explicitly requests parallel execution, tool_choice="none" or forces another tool, negotiation fails rather than overriding settings.
• Plain text without a hidden-tool call is classified as missing_submission and may enter one repair request.

Prompt fallback

Enable only when the caller explicitly sets fallback="prompt" : add a compact schema and JSON-only requirement to the system layer, then parse/validate final text identically. This is not strict and is never silently used in auto . This distinguishes provider constraints from purely local detection.

6. Streaming events and incremental text

Structured output does not incrementally validate schemas. Forward raw text deltas per attempt so consumers may display generated JSON; parse only after final text_end . Add two framework events and optional attempt metadata on text events to prevent invalid attempts becoming final messages.

EventTextStart/TextDelta/TextEnd.output_attempt: int | None = None

EventStructuredOutputRetry {
  type: "structured_output_retry",
  attempt: 1,
  next_attempt: 2,
  issues: [{code, path, schema_path, message}],
}

EventStructuredOutputEnd {
  type: "structured_output_end",
  attempt: 1 | 2,
  mode: "native" | "tool" | "prompt",
  value: JsonValue,
}

Provider raw done is consumed by the Agent loop; expose only the structured transaction’s final done. Ordinary text retains its event sequence. CLI/TUI/Web safely ignore unknown events for rolling upgrades.

7. Strict parsing, validation and bounded retries

Parsing rules

• Direct text uses json.loads(raw, parse_constant=reject_nonfinite); reject Markdown fences, surrounding explanations, trailing content, NaN/Infinity.
• Parse hidden-tool arguments as complete JSON text too; never use utils/json_parse.py partial/forgiving parsing.
• Use jsonschema.validators.validator_for(original_schema) to create the validator, with a registry prohibiting external-resource access. Sort errors by instance path, schema path and message.
• Schema preflight rejects all $ref/$dynamicRef/$recursiveRef that cannot resolve to embedded resources in the current schema. Fragment references and relative references defined by embedded $id remain valid, so Runtime, CLI, Web and replay never initiate DNS, HTTP or file reads for local validation.
• Canonical schema JSON is limited to 1 MiB, 4096 container nodes, 8192 container edges and depth 100. Exceeding any limit returns stable invalid_schema.
• before provider calls. Fragment-only references allow local JSON Pointers/anchors; direct reference self/mutual cycles fail preflight to prevent validator recursion overflow. Finite recursive schemas advancing through instance structure remain supported.
• Transport at most 20 errors with messages of at most 500 characters each, avoiding copying complete sensitive output into logs or WS errors.

Retry states

Shared call budget

max_validation_retries and max_retries are two limits, not multiplicative loops. The former caps semantic repairs after initial-candidate failure; the latter caps total provider calls initiated at the Runtime boundary by one synchronous/asynchronous exec , including the initial call, semantic repair and calls following transport failures. Repair requires both an available validation retry and remaining shared call budget.

Therefore Runtime-boundary provider calls never exceed max_retries. For example, max_retries=1 causes an invalid first candidate to immediately return StructuredOutputValidationError; max_retries=2 with one default validation retry permits only initial call plus one repair. A repair transport failure already consumes budget; the outer layer cannot regain a full budget. Adapter/SDK transport retries remain governed by existing deadlines/transport settings, not validation retry counts.

structured_output_retry.attempt and next_attempt continues identifying structured candidate attempts, not transport attempts. Emit only after Runtime confirms remaining shared budget and is about to repair; never emit false retry events on exhaustion.

Repair prompts contain only failure type and bounded issues, without reinterpreting schemas or calling another LLM. Repair context lasts only for this exec; session history/final DAG keep only the valid final assistant, while model-call records retain both provider calls for audit.

8. Coexistence with tool calling

Explicitly copy structured configuration during agent_loop._stream_assistant_response() Reconstructing SimpleStreamOptions . Final validation belongs at the Agent-loop no-tool terminal boundary, not duplicated inside provider parsers.

9. CLI, WebSocket/API and error semantics

Python API

value = runtime.exec(content=..., response_format=schema)  # Python JsonValue

except StructuredOutputError as e:
    e.code       # invalid_schema | unsupported | invalid_json | validation_failed |
                 # missing_submission | refusal | incomplete
    e.provider
    e.model
    e.attempts
    e.issues     # bounded machine-readable list

StructuredOutputError uses the existing LLM-error boundary without pretending to be a transport error.retryable means whether a new exec/budget could succeed; validation-retry exhaustion is indicated by attempts .

CLI

Add to existing one-shot openprogram --print the option --json-schema PATH; initially leave interactive REPL/TUI input unchanged. Success writes canonical UTF-8 compact JSON plus newline to stdout and exits 0. Errors go to stderr: usage/invalid schema exits 2, unsupported 3, generation/final-validation failure 4. Existing provider/auth/cancel semantics remain. Persist compact JSON in session assistant content with parsed value/mode in metadata.

WebSocket and asynchronous HTTP API

// request: existing chat action and POST /api/function/{name} body
response_format?: JsonSchemaOutput | JSONSchema

// stream
{type:"chat_response", data:{type:"stream_event", event:{
  type:"structured_output_retry", attempt:1, next_attempt:2, issues:[...]}}}

// success
{type:"chat_response", data:{type:"result", content:"{...}",
  structured_output:<JsonValue>, structured_output_mode:"native", attempt:1}}

// terminal error
{type:"chat_response", data:{type:"error", code:"validation_failed",
  content:"Structured output failed validation", attempts:2, issues:[...]}}

• POST /api/function/{name} Continue returning {session_id,msg_id} acknowledgements immediately; deliver results through the same WS/event chain instead of synchronous HTTP.
• Web chat sends no schema by default. Display schema results as canonical JSON text, retaining typed values for API consumers; retry clears candidate text.
• Server json.dumps(..., default=str) must not serialize structured values; validated values already belong to the JSON data model, otherwise this is an internal error.

10. Record/replay and persistence

• RECORDING_FORMAT_VERSION moves from 1 to 2. v2 request options record complete normalized response_format; explicitly reject old v1 to avoid ambiguous mismatches from default None fields.
• Record raw provider events, not locally derived validation events. Replay returns identical raw events and the Agent loop deterministically reconstructs retry/end/error.
• Each validation repair creates a new provider call and call_index; recordings must contain two request/call_end pairs. Missing or extra replay calls are mismatches.
• After normalization, recursively sort schema keys and compute SHA-256 for logs, DAG metadata and error correlation. Recordings retain the full schema for strict comparison; hashes do not replace it.
• Session assistant content stores final canonical JSON; metadata stores mode, attempt and schema hash. Invalid candidates never enter the session message chain.

11. Session-title migration

After core capabilities are ready, migrate agent/dispatcher/titles.py as the first internal consumer:

TITLE_SCHEMA = {
  "type": "object",
  "properties": {"title": {"type": "string", "minLength": 1, "maxLength": 80}},
  "required": ["title"],
  "additionalProperties": false,
}

result = runtime.exec(content=prompt, toolset="none", max_iterations=2,
  response_format=JsonSchemaOutput(
    schema=TITLE_SCHEMA, name="session_title", max_validation_retries=1,
))
title = result["title"]

Retain Unicode trimming and final database length protection; remove ONLY-title-text instructions and quote, Title: and first-line format cleanup. Title generation stays best-effort: structured errors retain the phase-1 placeholder without affecting the main turn. Update session/name.md and feature matrix to distinguish implemented from designed behavior.

12. References: adopt, modify, reject

Repository references/pi-ai already has shared stream options, provider-local payload builders, tool schemas and supportsStrictMode, but strict applies only to tool definitions. It lacks generic response schemas, final JSON validation and structured retries.

13. Exact files to change

14. Test matrix and acceptance

15. Implementation order and completion

1. Public types, direct dependency, schema preflight/validator/errors; establish pure unit tests first.
2. Capability registry, models.dev tri-state retention and provider schema profiles.
3. Provider payload mappings and adapter contracts; unverified adapters stay unknown.
4. Agent-loop terminal validation, hidden tools, events and one repair; Runtime parsed returns.
5. Record/replay v2, CLI and WS/API consumers; verify old text behavior unchanged.
6. Migrate session titles; update feature matrix and naming documentation.

Provider capabilities, contracts and title consumer are implemented. Feature matrix retains the user-approved score 78.5 and existing status cells; complete regression gates and specific trust-boundary evidence remain separately recorded here. Adding payload fields alone is not feature completion.

16. Implementation and verification evidence

Commit evidence: Task 1 00945c85..27a5aa8a; Task 2 da4ececd..8ca7b04c; Task 3 6f8c80b7..546c2bc7; Task 4 27e4764f..510f60ee; Task 5 9743f3e1..86ca66d5; Task 6 5bba3e64..2fb471b3. Final scoped SPEC and QUALITY reviews for Tasks 1–6 are PASS with remaining 0.

Verification state: Task 6 affected selection: 219 passed, 1 skipped; Task 7 focused selection: 226 passed, 1 skipped; documentation build: 453 pages, link check: 0 broken. The complete Python selection on the final Task 7 tree produced 3290 passed, 5 skipped, 1 xfailed, 2 failed; the two failures were the existing tool-cancellation ExecInterrupt assertion and memory-writer turn-order assertion, independently reproduced before implementation at baseline 86ca66d5 . They are not attributed to this structured-output diff, but do not satisfy complete-gate release requirements.

Subsequent quality fixes: Global preflight now enforces 1 MiB/4096 nodes/8192 edges/100 depth budgets and rejects external resources/direct cycles. Local validation uses a no-retrieval registry; structured streams commit only after a provider raw terminal. Scoped SPEC/QUALITY conclusions will be recorded after review.