OpenProgram Docs
Token tracking#
Every vendor reports token usage with different field names and different conventions (some include cached tokens in the input count, some don't). The provider layer normalizes all of them into one usage record per assistant message, so the UI and session records never deal with vendor formats.
Unified format#
Each assistant message carries a Usage record (defined in openprogram/providers/types.py):
| Field | Meaning |
|---|---|
input |
Non-cached input tokens (cached tokens are subtracted where the vendor reports an inclusive total) |
output |
Output tokens |
cache_read |
Input tokens served from the prompt cache |
cache_write |
Input tokens written to the prompt cache (providers with explicit caching only) |
total_tokens |
The vendor-reported total for the call |
cost |
USD cost, computed from the enabled model's per-million-token prices (input / output / cache read / cache write) |
The convention follows Anthropic: input excludes cached tokens. Vendors whose input count includes cached tokens (the OpenAI protocols) are converted by subtracting the cached figure.
What each provider reports#
The streaming implementations extract usage from the vendor's final event or last chunk:
| Protocol (providers) | Raw fields | Cache stats |
|---|---|---|
Anthropic Messages (anthropic, claude-code, and Anthropic-protocol gateways such as minimax, kimi_coding, vercel_ai_gateway) |
input_tokens, output_tokens, cache_read_input_tokens, cache_creation_input_tokens |
read + write |
OpenAI Responses (openai, openai_codex, azure_openai_responses, github_copilot) |
input_tokens (includes cached — subtracted), output_tokens, input_tokens_details.cached_tokens, total_tokens |
read only |
OpenAI Completions (deepseek, groq, mistral, openrouter, and other compatible endpoints) |
prompt_tokens, completion_tokens (reasoning tokens from completion_tokens_details are split out of the output count), total_tokens |
none |
Google Generative AI (google) |
prompt_token_count, candidates_token_count, total_token_count |
none |
Cloud Code Assist (gemini_subscription, google_gemini_cli) |
promptTokenCount, candidatesTokenCount + thoughtsTokenCount, cachedContentTokenCount, totalTokenCount |
read only |
Bedrock Converse Stream (amazon_bedrock) |
inputTokens, outputTokens, cacheReadInputTokens, cacheWriteInputTokens, totalTokens |
read + write |
Cost is computed right after extraction, from the model entry's pricing. On the OpenAI Responses protocol the response's actual service_tier additionally adjusts the price, so fast-tier requests are costed at the priority rate.
Where usage shows up#
- Per message: each assistant message stores its own usage and cost; they persist with the session.
- Chat badge: the badge by the input box shows the latest call's usage (
11.2k in · 450 out). No accumulation across calls — the last call's input token count is what the current context occupies, which is the number that matters. The tooltip breaks the input down: base / cache write / cache hit for Claude-protocol providers, base / cached for Codex, a "% cached" figure for the rest. Counts format as1.2kabove a thousand and1.0mabove a million. - Context bar: the server pushes a
context_statsevent over the chat WebSocket after each turn, carrying the chat usage plus the model's context window; the UI renders the fill percentage from it. - Function executions: each function run uses its own runtime, and its usage is reported per execution on the function card — independent of the chat conversation's numbers.
Providers report usage per call, so there is no per-provider accumulation logic anywhere: the latest value always describes the latest request.