Providers#

Design docs for the LLM provider layer. Providers translate the framework's internal unified context (Context: system / messages / tools) into each vendor's API request, handling authentication, caching, errors, and the model catalog.

The docs are organized into four groups by responsibility:

Translation + Caching (core)#

How the provider-agnostic unified format is translated into each vendor's wire format, and how prompt caching is implemented per provider — the core mechanism of the providers layer.

• request-build — Overall design: the unified-format Context, per-provider translation, and the caching modes.
• cache-control-passthrough (in docs/plans/) — per-block passthrough of Anthropic cache_control.
• record-replay — recording provider calls to a redacted JSONL recording file and replaying them offline for deterministic tests.
• For upstream (how content is layered and assembled, L0/L1/L2) see context/composition.md.

auth/ — Credentials · Authentication · Accounts#

Resolution, validation, and storage of API keys and subscription OAuth, plus the multi-account pool and rotation.

• credential-validation-unification — The credential-validation entry point
• credential-status-redesign — Credential status (usable or stopped)
• api-key-resolution-unification — The API key resolution chain
• unified-auth-storage — Self-contained auth storage
• unified-account-management — Account management + pool rotation/fallback
• claude-code-direct-oauth — claude-code subscription OAuth direct connection

reliability/ — Fault tolerance · Errors · Retries · Timeouts#

Classification, retries, timeouts, and upward propagation of errors when a model call fails.

• llm-fault-tolerance — Overall design for fault tolerance and timeouts
• error-retry — Error handling and retry decisions
• error-taxonomy-propagation — Structured errors propagated all the way to the UI
• error-and-timeout-mechanism.html — Visualization of the error/timeout mechanism

models/ — Model catalog · Capabilities#

The data layout and configuration structure of the model list, plus the declarative mapping of capabilities like thinking/effort. Every model is bound to the provider it belongs to, so it lives under providers.

• models — Model catalog and provider configuration (data layout, fetch, merge)
• thinking-effort — The thinking/effort subsystem (declarative per-provider mapping)
• fast-tier — the Fast tier: two-tier detection (hand-written subscription entries / models.dev auto), storage, wires