Model Catalog and Provider Configuration#

This document describes the runtime logic of the model catalog: where data lives, how files interact with code, and how the backend and frontend each consume it. Thinking-effort parameter details are covered in thinking-effort.md.

1. Architecture in one sentence#

The system only remembers the models the user enabled. The settings page is hierarchical — providers first, then models: the first level shows the provider list; only after opening one provider does the page query, live, which models that provider offers — and that query is never persisted. The act of enabling copies a model's full spec, as of that moment, into config.json. The runtime registry ENABLED_MODELS is those few dozen config rows — what get_model() resolves, what the chat page shows, and what the user checked are physically the same data.

Core invariant: selectable in chat = enabled = resolvable by the backend. Not because a merge pipeline keeps two lists aligned, but because there is only one list.

Properties that follow automatically:

• No big files: no full catalog is stored (models.dev has 151 providers, thousands of largely duplicated models); config holds only the handful the user enabled.
• Nothing goes stale: staleness requires storage. The available list is queried live, so it is always current; enabled models' specs are overwritten on demand via the settings page "Refresh" — refreshing only what the user actually uses.
• git stays clean: the program writes only the user's config; the repository holds only hand-written provider.json; the installed package directory is read-only at runtime.

2. Data layout (split by author)#

openprogram/providers/                 ← all git-tracked, read-only at runtime
├── deepseek/
│   ├── provider.json                  ← all hand-written config for this provider (section 3)
│   └── deepseek.py                    ← wire/stream implementation (only for dedicated protocols)
├── enabled_models.py                  ← ENABLED_MODELS: loads config + endpoint fill + thinking derivation
└── models.py                          ← get_model / get_providers / get_models

~/.openprogram/
└── config.json                        ← the only user-side persistence

Naming rule: nothing here is called a "catalog". One word for five different things is what made the naming confusing, so each module is named for what it holds: enabled_models.py for the registry, storage.py for config persistence, thinking_spec.py for the thinking declarations, and webui's _model_listing/ for the presentation layer. The name ENABLED_MODELS is accurate only because the dict holds enabled models and nothing else.

Providers without a directory (fireworks, together, …): models.dev lists them live; the user enters a key, browses, enables — no file in the package is ever needed.

3. provider.json: the only hand-written file#

All human configuration for a provider in one file; every field optional:

{
  "id": "deepseek",
  "endpoints": {
    "default": {"api": "openai-completions", "base_url": "https://api.deepseek.com/v1"}
  },
  "thinking": {
    "wire_format": "effort_string",
    "effort_map": {"minimal": "minimal", "low": "low", "medium": "medium", "high": "high", "max": "max"},
    "default_effort": "medium"
  },
  "cache": {"mode": "none"},
  "model_overrides": {
    "some-model": {"headers": {"X-Foo": "1"}, "compat": {"no_stream_options": true}}
  },
  "models_from": null
}

The test: if a machine can obtain a field, a human doesn't write it. provider.json contains no model list — the list is browsed live, the spec is copied at enable time.

Directory names use underscores (amazon_bedrock/); id keeps the hyphenated name (amazon-bedrock). Same-service-multiple-protocols (Bailian's OpenAI-compatible + Anthropic-compatible endpoints) = two endpoints of one provider, not two providers.

4. The two actions: browse, enable#

4.1 Browse (live, never persisted)#

Browsing has two levels. Level one: the provider list (settings landing view) = providers with a local provider.json ∪ models.dev's provider index — names and configuration status only, no model names. Level two: the model list — only after the user opens one specific provider does the page query that provider:

list_available_models(provider_id)
  = the provider's official source (one fetcher, see below)
  ⊕ models.dev (fills price/capabilities; full fallback when there is no key)
  → merged in memory, returned straight to the frontend

Fetcher placement rule: a provider whose interface departs from the standard OpenAI shape carries its own fetcher in its own directory. Each source has a different shape but returns the same contract: success → list[dict] (each row has at least id/name), failure → {"error": ...}.

Convention loading: a provider whose interface departs from standard ships a list_models.py in its own directory exposing fetch(provider_id, timeout); the dispatcher's _load_fetcher __import__s it by directory name — the exact mechanism probe_thinking.probe() uses, so adding a provider needs no central edit. A standard-interface provider ships no such file and uses the generic openai_compat. Whether the file exists = whether the provider's interface departs from standard — a natural, on-demand test.

Whatever the source, fetch_and_normalize is the single normalisation choke point: it collapses each fetcher's disparate keys (context_length / context_window / contextWindow, …) into one entry dict, then layers models.dev on top. Downstream only ever sees the normalised row — never the source differences.

Ordinary provider results use the short in-memory browse cache. Account-scoped subscription catalogues also keep an atomic last-known-good copy under the profile state directory. A failed request can therefore show the last good rows as stale data without treating them as a successful authoritative refresh. An empty result or an unconfigured provider preserves saved model selections; only a non-empty current official catalogue can retire subscription models.

4.2 Enable (copy the spec into config)#

The user checks a model in the browse list:

enable_model(provider_id, row)
  → spec = browse row ⊕ provider.json.model_overrides[id] ⊕ api/base_url from endpoints
  → thinking levels derived from provider.json.thinking, written alongside
  → appended to config.json providers.<p>.models
  → ENABLED_MODELS reloads

• Disable = delete the row from config. Subscription providers also record an id tombstone so an automatic refresh does not re-enable it.
• Refresh = re-run browse for enabled models and overwrite their specs (handles spec drift over time; touches only what the user actually uses).
• Manually adding a model (one the provider doesn't list) = the user fills in a row in the same form — it writes to the same list; the old custom_models concept dissolves.
• Subscription catalogues = after login, at worker startup when stale, on the six-hour background cadence, and on manual Refresh, read the account's official model table. New ids are enabled automatically, changed capability rows are refreshed, removed ids retire, and explicit disables stay disabled. Adding a future Codex or Grok model does not require a source-code list edit. When the persisted enabled rows actually change, the worker broadcasts a provider_models_changed invalidation hint. Connected web and desktop clients then re-read the provider and enabled-model endpoints; reconnecting clients invalidate the same queries to cover events missed while offline.

5. How the backend uses it#

# openprogram/providers/enabled_models.py
ENABLED_MODELS: dict[str, Model]   # key = "<prefix>/<id>", content = config specs + derived fields

Loaded from config at startup (tens of rows, instant), reloaded when config changes. The three query functions get_model / get_providers / get_models keep their interfaces — the 20+ runtime callers (agent, runtime, failover, …) change nothing. get_model falls back through auth.aliases on a miss.

Contract: the system only knows enabled models. Failover chains and agent configs must reference enabled models; referencing a non-enabled model is a configuration error whose message points to the settings page. Old sessions referencing a deleted model still display history; they just cannot continue with that model.

6. How the frontend uses it#

The webui presentation layer (_model_listing/) does no merging or derivation — browse merging is one function (4.1), spec merging happens at enable time. webui imports providers; providers never import webui.

End to end: enter a key → browse (live list shows deepseek-v4-flash) → check it (full spec written to config; ENABLED_MODELS["deepseek/deepseek-v4-flash"] appears) → pick it in chat and send (get_model hits the same config row). At every moment the system holds exactly one copy of model data.

7. Invariants (check before changing code)#

1. Persist only what's enabled: the only persisted model data is the enabled specs in config. Any second persisted list (full snapshot, fetch cache file, hand-written catalog) is a violation.
2. Browsing never persists: the available list is a live query + in-memory cache, never written to a file.
3. Split by author: human data in git (provider.json); program data in user config; the package directory is read-only at runtime.
4. Minimal hand-writing: provider.json stores only what machines cannot obtain, and never a model list.
5. One-way layering: openprogram.providers never imports openprogram.webui.
6. Key compatibility: "<prefix>/<id>", alias fallback, key_prefix (gemini-subscription dual keys) preserved; the registry stays one mutable dict.
7. Many sources, one shape: whether a row comes from an official /v1/models, an account-level private endpoint (codex), the models.dev community catalogue, or a hand-typed entry, it is normalised into one row shape by the single fetch_and_normalize function; enabling routes through the single _upsert_spec_row choke point into config (_normalize_spec_row fills the Model-schema keys); reading routes through the single _build_model_from_row converter into a Model. Each of the three choke points is exactly one; nothing may bypass them to mint its own shape.

8. Implementation status#

The design above is what the code does. The registry is ENABLED_MODELS, defined in enabled_models.py; the webui presentation layer is _model_listing/; enabling a model copies its full spec into config; browsing is live with no disk persistence; and thinking.json and cache.json are folded into provider.json, so _default_api_for / _resolve_base_url read endpoints directly and the providers layer no longer reads the registry back.

Properties that any change here must preserve:

• Enablement survives: a config row must resolve through get_model to the same Model it did before, for every provider.
• Aliases and dual keys: gemini-subscription's google-gemini-cli/* and gemini-subscription/* are 10 keys with distinct names. Enabled rows carry their own key and name, and the alias fallback stays in place.
• The claude-code borrowing chain: browse data borrowed from anthropic via models_from, 3 models auto-enabled after login, and its own fetcher.
• Field-by-field fidelity: nested cost, multimodal input, headers (copilot depends on it), and compat all travel in the spec written to config at enable time.
• Verification granularity: multi-wire providers need one exec per (api, base_url, headers, compat) combination.
• tests/unit/providers/registry/test_provider_wire_invariants.py and tests/unit/providers/registry/test_model_fetch_routing.py stay green.