llm-router
v1.4.37One front door + provider protocol in front of every LLM provider.
Install
bash
iii trigger compose::add worker=llm-routerRun in your project with the iii engine running. Exact versions are immutable and digest-pinned.
Agent-ready brief for v1.4.37
Install, config, dependencies, readme, skills and API reference, all in one place. Fetch as agent-context.md for an LLM to consume.
Full markdown
The same content rendered as discrete blocks below is exposed as a single markdown document at
/registry/workers/llm-router.md. Paste it into an LLM prompt or pipe it through curl from a worker.Install
iii trigger compose::add worker=llm-router@1.4.37Dependencies
Dependency · 3Version
Readme
README.md
llm-router
One front door for every LLM provider. The router owns routing, the provider registry, credential resolution, the model catalog, streaming relay, retries, and a single failure contract — consumers call one chat surface and never talk to a provider directly.
llm-router is a standalone iii worker. Providers plug in as separate workers
at runtime through a self-registration protocol
(iii trigger compose::add worker=provider-); the router never compiles against a provider, and removing a
provider worker removes the provider.
Install
iii trigger compose::add worker=llm-routerQuickstart
A consumer streams a turn by creating an iii channel, handing the router the
channel's write endpoint, and reading frames from the read endpoint
while router::chat runs. Any SDK works; Node shown:
import { createChannel } from 'iii-sdk';
const { reader, writerRef } = await createChannel(iii);
reader.onMessage((frame) => {
const event = JSON.parse(frame); // AssistantMessageEvent
if (event.type === 'text_delta') process.stdout.write(event.delta);
});
const res = await iii.trigger('router::chat', {
writer_ref: writerRef, // direction "write"
model: 'claude-sonnet-4',
messages: [{ role: 'user', content: [{ type: 'text', text: 'Hello' }], timestamp: Date.now() }],
}, { timeout_ms: 620_000 }); // outer timeout ≥ the router's 600s stream budget
// res: { ok, provider, model, stop_reason, usage }The streaming contract: every stream ends with exactly one terminal frame
(done or error). When the router has to kill a stream itself (idle
timeout, provider crash), it synthesizes the terminal frame and attaches the
partial content, so consumers never hang on a half-open stream.
Functions
Consumer surface
| Function | Purpose |
|---|---|
router::chat |
Stream a turn into the caller's channel; returns the turn summary. |
router::complete |
Non-streaming convenience over the same pipeline; returns the final message. |
router::abort |
Cancel an in-flight turn by request_id. |
router::route |
Read-only routing preview: {model, provider?} → {provider, candidates}, same rules and error codes as router::chat. Pin the result as the explicit provider on the chat call when you need the provider before streaming. |
router::models::list |
List catalog models, filterable by provider / capability / modality (chat by default, stt, tts, any). |
router::models::get |
Fetch one model record (null when unknown). |
router::models::supports |
Check one capability flag for one model. |
router::embed |
Batch text embeddings through the first embed-capable provider in the registry ({model?, provider?, input[]} → {provider, model, embeddings[][]}); providers without an embed surface are skipped. |
router::transcribe |
Speech to text: {model?, provider?, audio_base64, mime?, language?, prompt?} → {provider, model, text, segments[]?, language?, duration_secs?}. The provider comes from the named provider, else from the catalog owner of the stt model, else the first provider that declared one. |
router::speak |
Text to speech: {model?, provider?, text, voice?, format?, language?, speed?} → {provider, model, audio_base64, mime, voice?, duration_secs?}. Same resolution over tts models. |
router::count_tokens |
Count prompt tokens: {model, provider?, system_prompt?, tools?, messages} → {provider, model, tokens, estimator}, resolved with the same routing rules as router::chat and forwarded to provider::. Never runs the model and costs nothing; estimator is provider (metering API) or tiktoken (local tokenizer). A provider without the surface is a typed router/no_token_counter error, so callers can fall back to their own estimate. |
router::provider::list |
Registered providers with configured / available status, where each credential comes from (credential_source, credential_ref, credential_error — never a value), and each provider's starting point: default_model and default_thinking_level (see Default model). |
Only the read surface is agent-callable (router::models::list / get /
supports, router::provider::list); everything else is denied to in-run
agents — see Security model.
Error conventions
Errors use stable router/ identifiers. Streaming failures return an
error object with a user-facing message, the shared kind, retryable,
and an optional diagnostic detail. User interfaces should show message
and keep code / detail behind technical details; automation should branch
on code and retryable, never parse the prose.
The main recovery codes are router/provider_unavailable,
router/capacity_exceeded, router/stream_idle_timeout,
router/stream_incomplete, router/provider_auth_expired,
router/provider_rate_limited, router/context_overflow, and
router/provider_rejected.
Provider protocol
Token-gated after the first declare: the response to register carries a
registration token, and every later protocol call must present it.
| Function | Purpose |
|---|---|
router::provider::register |
Self-declaration at attach time; idempotent re-declare with the token. |
router::provider::resolve |
Per-request credential + endpoint resolution (config > secret:// / env:// reference > env > none, see Credentials). |
router::provider::update_credential |
Persist a refreshed credential (OAuth write-back). |
router::models::reconcile |
Replace the provider's catalog slice in one write. |
The provider worker itself exposes provider:: and, when
capable, provider:: (model discovery),
provider:: (prompt token counting),
provider:: (embeddings), and the speech pair
provider:: / provider::.
router::chat accepts an optional structured prompt next to the flat
system_prompt: system_sections (ordered {text, cache_boundary} blocks)
and cache_intent ({surface_digest}, the caller's sha256: identity for
the text before the boundary). A sections-only request is flattened into
system_prompt (sections joined with \n\n); when both are sent they must
agree byte for byte or the request is rejected as invalid_request. Both are
forwarded verbatim to provider:: alongside the flat string, so a
provider that knows nothing about sections keeps reading system_prompt.
Cache-aware providers put their prefix marker on the boundary block
(Anthropic, Claude Code) or derive a shared prompt_cache_key from the digest
(OpenAI, Codex) so every session on the same frozen prefix lands on one cache
entry. The digest is a routing hint, never evidence of a hit — read
usage.cache_read for that.
Configuration
All operator configuration lives in the engine's llm-router configuration
entry — no env vars, no config file. The entry schema is composed at runtime
from each registered provider's declaration. The router fetches the complete
entry at boot and keeps an in-memory snapshot synchronized by the
configuration:updated trigger; request handlers never poll
configuration::get.
{
"default_provider": "anthropic",
"providers": {
"anthropic": {
"api_key": "secret://ANTHROPIC_API_KEY",
"api_url": "https://api.anthropic.com/v1/messages",
"max_tokens": 8192
}
},
"routing_heuristics": [{ "pattern": "^gpt-", "provider": "openai" }],
"settings": {
"stream_timeout_ms": 600000,
"idle_timeout_ms": 120000,
"retry_max": 2,
"output_token_max": 32000
}
}| Setting | Default | Meaning |
|---|---|---|
stream_timeout_ms |
600000 |
Hard budget for one streamed turn. |
idle_timeout_ms |
120000 |
Max silence between provider frames before the attempt is cut. |
retry_max |
2 |
Retries per turn for retryable failures before the first forwarded frame (0–10). |
output_token_max |
unset | Optional ceiling on max_output_tokens. Unset forwards each model's own output ceiling, so providers run without an artificial limit; set it only to cap spend. |
Pasting a key into a provider's slice is the whole onboarding flow: the
router diffs the changed slice, debounces ~2 s, and kicks that provider's
provider:: discovery; discovered models land in the
catalog via router::models::reconcile and show up in router::models::list
within seconds — no restart.
Credentials and secret references
The configuration service's default filesystem adapter persists this entry at
./config/ (llm-router unless III_CONFIG_NAME overrides it),
a folder meant to be committed, so keep keys out of it: keep the key in the
secrets worker, which llm-router depends on, and put a versionable
reference in the slice:
"api_key": "secret://ANTHROPIC_API_KEY"— encrypted in the secrets worker's vault;"api_key": "env://ANTHROPIC_API_KEY"— the variable in the project's env file (.env, or the secrets worker'senv_filefor this namespace; else the secrets worker's environment), read by the secrets worker each time, so an edit to the file applies without restarting the router.
Provider keys reuse their env var name; NAME matches
^[A-Za-z_][A-Za-z0-9_.-]{0,127}$. Either way the secret's consumers must
list llm-router; the Console's key field does this for you.
A provider's credential is the first of:
- the slice's
credentialobject (OAuth write-back viaupdate_credential); - a literal
api_key; - an
api_keyreference (secret://orenv://), resolved throughsecrets::resolve; - the provider's
credential_env_var, read in the router's process; - none.
A reference is never forwarded to a provider as a key. When it does not
resolve, the provider is not configured, and the error says why: secret NAME not found in the secrets worker, environment variable NAME is not set in the project's .env or the secrets worker's environment, llm-router is not allowed to read secret NAME; add llm-router to the secret's consumers, or
secrets worker is not running. The env var is deliberately not used as a fallback past an
explicit reference: you chose the stored secret, so a missing one is reported
rather than silently replaced by a different key. Providers built on the
shared scaffold skip their own env fallback in that case too, and
router::chat fails before dispatch with router/not_configured and the same
reason.
router::provider::list entries and the router::provider::resolve response
carry credential_source (config | env | secret | none),
credential_ref (the secret:// or env:// reference, when the slice uses
one; credential_source is secret for both, as the secrets worker resolved
it) and
credential_error. All three are optional on the wire. The older source
field keeps its config | env | none values: a resolved reference reports
config there.
Resolved values live only in the router's memory: never in the entry, logs or traces, and a cached value is wiped when replaced. They are re-read:
- on
secrets::changed(the router binds it at boot; without a secrets worker the engine holds the binding until the type registers). A rotated, deleted or revoked secret then nudges every provider whose slice references it throughprovider::, so the provider drops its cached resolve, re-declares and refreshes its models;::on_router_ready - when a slice that references it changes, before the debounced
refresh_modelsfires, so a pasted reference is resolvable by the time discovery asks for it; - when the secrets worker registers after the router, so a router that booted first recovers without a restart;
- on demand after 5 minutes (10 seconds for a failure), bounding staleness if an event is ever missed. While the secrets worker is unreachable, a value already resolved keeps being served: secrets cannot rotate or be revoked while it is down.
The reference itself is not secret: it is committed, listed and logged. Never
paste a key after secret:// or env://.
Operational notes
- Secret references resolve as
llm-router. The secrets worker checks the router's engine-stamped worker identity against the secret'sconsumers, not the provider's. - Env-var credential fallback resolves in the router's process. Without
a reference, a provider's
credential_env_var(e.g.ANTHROPIC_API_KEY) is read by the llm-router binary, not by the provider worker — launch the router with those variables set, or put a reference in the entry. A key present only in another worker's environment shows up asconfigured: false.env://NAMEis different: the secrets worker reads it from the project's.env, so the router needs noenv_filefor it. - Registration-token recovery. Re-registering a provider id without its
original token is rejected (anti-takeover). If a provider durably lost its
token, delete the router's registry state (state scope
llm-router, keyregistry) and restart the affected providers to re-bind; pasted credentials in the configuration entry are unaffected.
Security model
The agent-callable surface is governed by the engine-wide permission policy
(repo-root iii-permissions.yaml), not a per-worker
file. In-run agents may only read the catalog and provider list:
- Allowed:
router::models::list,router::models::get,router::models::supports,router::provider::list. - Denied to agents: the chat/spend surface (
router::chat,router::complete,router::abort,router::route), the whole provider/credential protocol (router::provider::resolve/register/update_credential,router::models::reconcile), and directprovider::*calls.
Worker-to-worker calls bypass the agent gate, so the harness, context-manager,
and provider workers reach the full surface — only in-run agents are
restricted. Provider credentials live in the configuration entry, or behind a
secret:// or env:// reference in the secrets worker, and are never readable back
through any allowed function: router::provider::list reports where a
credential comes from, never its value.
Events
The router registers three custom trigger types and fans out to every bound handler. Bind with the standard two-step pattern; the handler receives the payload verbatim (no envelope).
| Trigger type | Fires when | Payload |
|---|---|---|
router::models::changed |
a provider reconciles its catalog slice | { "provider": " |
router::provider::changed |
the registry changes (declare / availability flip) | { "provider": " |
router::ready |
the router finishes booting; providers re-declare on it | {} |
The router itself binds the engine's configuration trigger (its entry) and,
when the optional secrets worker is installed, secrets::changed
(router::on_secret_changed).
iii.registerFunction('my-worker::on-models-changed', async (payload) => {
console.log('catalog changed:', payload); // { provider, count }
return {};
});
iii.registerTrigger({
type: 'router::models::changed',
function_id: 'my-worker::on-models-changed',
config: {},
});Writing a provider worker
A provider worker must:
- Register
provider::honouring the channel-writer contract: forward upstream output as::stream AssistantMessageEventframes into thewriter_refit receives, ending with one terminal frame. - Declare itself at startup via
router::provider::register— retrying with backoff until acknowledged (covers provider-before-router boot order) — and re-declare on therouter::readyevent after a router restart. The declaration may carryicon_svg: one self-contained, monochromedocument (squareviewBox, no fixed size, at most 32 KiB). The router echoes it inrouter::provider::listand the console paints it as acurrentColormask beside the provider's models; a missing or malformed mark falls back to the provider's initial. The declaration may also carrydefault_models, the provider's recommended starting models in preference order (the current mid-range model first, then its predecessors), anddefault_thinking_level, the level to pair with it; see Default model. - Resolve credentials per request via
router::provider::resolve; never read keys directly. A response withcredential_source: "secret"and no credential is an unresolvablesecret://orenv://reference: report itscredential_error, do not substitute a key of your own. - Treat closure of its stream channel as cancellation: abort the upstream request and stop writing frames.
- Map upstream failures to the shared
ErrorKindtaxonomy on itserrorframes. Transport retries (429 / 5xx / connect) are the router's job, not the provider's.
A provider may also register provider:: ({model, system_prompt?, tools?, messages} → {model, tokens, estimator}) to serve
router::count_tokens: an exact count from a provider metering API
(estimator: "provider") or a local tokenizer estimate (estimator: "tiktoken"). Providers without it simply make router::count_tokens return
a typed router/no_token_counter error for that provider.
Default model
router::provider::list reports one default_model per provider so that a
caller with no model choice of its own (a fresh harness::send naming only
a provider, a console opening its first chat) starts on something sensible.
It is resolved at read time against the provider's current catalog slice:
- the model variant first. A variant is the first all-letter id segment
after the family (
terraingpt-6.1-terra,sonnetinclaude-sonnet-5-5), ranked in the orderdefault_modelsfirst names it. Within a ranked variant the newest version in the slice wins, listed or not, so a newer Luna never beats a Terra when Terra is listed first. A model with a ranked variant counts even outside the first preference's family, as long as it has the samevendor/prefix, so Copilot ranks GPT and Claude models in one list; - ids with no ranked variant come last: the first id in the declared
default_modelsthe slice holds, else the slice model sharing the longest id prefix with the first preference, as long as they share the family (the id up to its first-:claude,gpt,codex/gpt). On a tie the highest version wins (gpt-4.1overgpt-4o), then the plain id over a dated snapshot or a-minivariant; - otherwise absent, and the caller keeps its previous behaviour.
A provider that declares no default_models (local model servers, speech
providers) never reports one. default_thinking_level is copied from the
declaration; absent means the caller should omit the level and let the
provider apply its own default.
A declaration may also carry context_overflow_hint: one or two plain
sentences a console shows under a context-overflow failure on one of the
provider's models, for causes compacting cannot fix (a self-hosted server
started with a small window, say), optionally with a link to the provider's
docs. router::provider::list reports it verbatim; absent means the console
shows only its generic steps.
A provider that works without a key (a local server started without one)
declares credential_optional: true. The router then reports it
configured when no credential resolves; a key that is set is still
resolved and sent. Absent or false means a key is required.
Thinking levels
router::chat takes thinking_level: off, minimal, low, medium,
high or xhigh. Each provider maps a level onto its own knob and warns
when a model cannot honour it. off asks for no reasoning at all and is
honoured only where the model's descriptor says supports_thinking_off: true (a native switch exists: OpenAI reasoning_effort: none on the
models that take it, DeepSeek and GLM thinking.type: disabled, Anthropic
Sonnet 5.5 between_tools). false means the model always reasons or the
provider has no switch; absent means the provider does not know. Consoles
offer, hide or disable the Off choice accordingly. A provider that cannot
switch reasoning off sends its lowest effort instead and reports a warning.
Speech providers
A speech provider is the same kind of worker with no chat stream: it
declares its models with a speech block and serves one or both speech
surfaces. router::models::list hides speech models from chat pickers
unless asked (modality: "stt" | "tts" | "any"), and router::chat
refuses them with a message that names the right door.
{ "id": "scribe-v1", "provider": "elevenlabs", "context_window": 0, "max_output_tokens": 0,
"speech": { "modality": "stt", "languages": ["en", "hi"], "streaming": true } }| Function | Request | Response |
|---|---|---|
provider:: |
{model?, audio_base64, mime, language?, prompt?} |
{model, text, segments[{text, start_secs?, end_secs?}]?, language?, duration_secs?} |
provider:: |
{model?, text, voice?, format, language?, speed?} |
{model, audio_base64, mime, voice?, duration_secs?} |
model arrives as the id the caller named, or null for the provider's
default. format is what the caller asked for (mp3, wav, pcm16,
opus); answer with the container you produced and say so in mime.
Credentials resolve through router::provider::resolve exactly as for
chat providers.
The first real provider implementing this protocol is
provider-anthropic/ — useful as a reference
implementation alongside the scripted provider in the integration tests.
provider-openai/ follows the same structure for the
OpenAI Chat Completions API (native structured output, reasoning_effort).
provider-opencode-go/ follows the same structure for the
OpenCode Go Chat Completions API (curated model metadata, reasoning_effort).
Local development & testing
cargo test # unit suite, no engine needed
cargo test --test integration # engine-backed suite; self-skips without an engineThe integration suite spawns a throwaway engine per test when iii is on
PATH (or III_ENGINE_BIN points at a binary) and covers the chat relay,
cancellation, abort, restart recovery, registration token gating, paste-a-key
discovery (literal keys, secret:// and env:// references, against a fake
secrets worker), and event delivery end to end.
To run the worker locally against an engine:
cargo run -- --url ws://127.0.0.1:49134--url defaults to ws://127.0.0.1:49134 and honours the III_WS_URL
environment variable when the flag is not set. --config is accepted per
the standard worker CLI but ignored with a warning — operator config lives
in the engine's llm-router configuration entry (see Configuration above).
API reference (JSON)
agent-api-reference.json
{
"functions": [
{
"description": "Serve the llm-router worker's injected console UI assets (content function for its console:script / console:style / console:module triggers).",
"metadata": {
"internal": true
},
"name": "llm-router::ui-content",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of the content function: the console asks for one asset by path.",
"properties": {
"path": {
"description": "The asset path from the trigger config (e.g. `state/page.js`).",
"type": "string"
}
},
"required": [
"path"
],
"title": "UiContentInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of the content function.",
"properties": {
"content": {
"description": "The asset source, verbatim.",
"type": "string"
},
"content_type": {
"description": "MIME type the console should serve the asset with.",
"type": "string"
}
},
"required": [
"content",
"content_type"
],
"title": "UiContentResult",
"type": "object"
}
},
{
"description": "Abort an in-flight router::chat/complete by request_id; reports whether a live request was cancelled.",
"metadata": {
"internal": true
},
"name": "router::abort",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of the `router::abort` iii function.",
"properties": {
"request_id": {
"type": "string"
}
},
"required": [
"request_id"
],
"title": "AbortRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"aborted": {
"type": "boolean"
}
},
"required": [
"aborted"
],
"title": "AbortResponse",
"type": "object"
}
},
{
"description": "Stream a chat completion: route {model, provider?} to a provider, relay assistant frames to writer_ref, and return the terminal response.",
"metadata": {
"internal": true
},
"name": "router::chat",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ChannelDirection": {
"enum": [
"read",
"write"
],
"type": "string"
},
"PromptCacheIntent": {
"description": "Why a request's stable prefix is shareable across sessions: `surface_digest` is the caller's content identity for the text before the boundary (`sha256:<hex>`). A routing/accounting hint only — never a provider cache handle, and never evidence of a cache hit.",
"properties": {
"surface_digest": {
"type": "string"
}
},
"required": [
"surface_digest"
],
"type": "object"
},
"PromptSection": {
"description": "One ordered system-prompt section. `cache_boundary` marks the end of a cumulative prefix worth its own provider cache entry (the frozen agent-profile surface). Adapters without a boundary concept read the sections joined with \"\\n\\n\", which must equal `system_prompt` when both forms are sent.",
"properties": {
"cache_boundary": {
"default": false,
"type": "boolean"
},
"text": {
"type": "string"
}
},
"required": [
"text"
],
"type": "object"
},
"StreamChannelRef": {
"properties": {
"access_key": {
"type": "string"
},
"channel_id": {
"type": "string"
},
"direction": {
"$ref": "#/definitions/ChannelDirection"
}
},
"required": [
"access_key",
"channel_id",
"direction"
],
"type": "object"
}
},
"description": "Input of the `router::chat` iii function: a [`ChatCall`] plus the caller's write channel. The handler relays assistant frames to `writer_ref` and also returns the terminal [`ChatResponse`].",
"properties": {
"cache_intent": {
"anyOf": [
{
"$ref": "#/definitions/PromptCacheIntent"
},
{
"type": "null"
}
],
"default": null
},
"max_output_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"messages": true,
"metadata": {
"default": null
},
"model": {
"type": "string"
},
"provider": {
"default": null,
"type": [
"string",
"null"
]
},
"provider_options": {
"default": null
},
"request_id": {
"default": null,
"type": [
"string",
"null"
]
},
"response_format": {
"default": null
},
"session_id": {
"default": null,
"description": "Stable conversation identity, forwarded separately from the per-turn request id.",
"type": [
"string",
"null"
]
},
"system_prompt": {
"default": null,
"type": [
"string",
"null"
]
},
"system_sections": {
"default": null,
"items": {
"$ref": "#/definitions/PromptSection"
},
"type": [
"array",
"null"
]
},
"thinking_level": {
"default": null
},
"tools": {
"default": null
},
"writer_ref": {
"allOf": [
{
"$ref": "#/definitions/StreamChannelRef"
}
],
"description": "The caller's write channel (direction \"write\"); frames are relayed here."
}
},
"required": [
"messages",
"model",
"writer_ref"
],
"title": "ChatFnInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ErrorKind": {
"enum": [
"auth_expired",
"rate_limited",
"context_overflow",
"transient",
"permanent"
],
"type": "string"
},
"ErrorShape": {
"properties": {
"code": {
"type": "string"
},
"detail": {
"description": "Provider/transport diagnostics for logs and expandable UI details. `message` remains the stable, user-facing explanation.",
"type": [
"string",
"null"
]
},
"kind": {
"anyOf": [
{
"$ref": "#/definitions/ErrorKind"
},
{
"type": "null"
}
]
},
"message": {
"type": "string"
},
"retryable": {
"type": [
"boolean",
"null"
]
}
},
"required": [
"code",
"message"
],
"type": "object"
},
"StopReason": {
"enum": [
"end",
"length",
"function_call",
"aborted",
"error"
],
"type": "string"
},
"Usage": {
"properties": {
"cache_read": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cache_write": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"reasoning": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
}
},
"properties": {
"error": {
"anyOf": [
{
"$ref": "#/definitions/ErrorShape"
},
{
"type": "null"
}
]
},
"model": {
"type": "string"
},
"ok": {
"type": "boolean"
},
"provider": {
"type": "string"
},
"stop_reason": {
"anyOf": [
{
"$ref": "#/definitions/StopReason"
},
{
"type": "null"
}
]
},
"usage": {
"anyOf": [
{
"$ref": "#/definitions/Usage"
},
{
"type": "null"
}
]
}
},
"required": [
"model",
"ok",
"provider"
],
"title": "ChatResponse",
"type": "object"
}
},
{
"description": "Non-streaming convenience over router::chat: run the turn on an internal channel and return the final assistant message + usage.",
"metadata": {
"internal": true
},
"name": "router::complete",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"PromptCacheIntent": {
"description": "Why a request's stable prefix is shareable across sessions: `surface_digest` is the caller's content identity for the text before the boundary (`sha256:<hex>`). A routing/accounting hint only — never a provider cache handle, and never evidence of a cache hit.",
"properties": {
"surface_digest": {
"type": "string"
}
},
"required": [
"surface_digest"
],
"type": "object"
},
"PromptSection": {
"description": "One ordered system-prompt section. `cache_boundary` marks the end of a cumulative prefix worth its own provider cache entry (the frozen agent-profile surface). Adapters without a boundary concept read the sections joined with \"\\n\\n\", which must equal `system_prompt` when both forms are sent.",
"properties": {
"cache_boundary": {
"default": false,
"type": "boolean"
},
"text": {
"type": "string"
}
},
"required": [
"text"
],
"type": "object"
}
},
"description": "ChatRequest minus writer_ref (the sink arrives separately — the function handler wraps the looked-up writer, complete wraps its own).\n\n`messages` / `tools` / `response_format` / `thinking_level` / `provider_options` stay `Value`: they are forwarded to the provider verbatim, so the router intentionally does not re-validate their shape. The struct still derives `JsonSchema` so the SDK emits a real request schema (the freeform sub-fields surface as permissive sub-schemas).",
"properties": {
"cache_intent": {
"anyOf": [
{
"$ref": "#/definitions/PromptCacheIntent"
},
{
"type": "null"
}
],
"default": null
},
"max_output_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"messages": true,
"metadata": {
"default": null
},
"model": {
"type": "string"
},
"provider": {
"default": null,
"type": [
"string",
"null"
]
},
"provider_options": {
"default": null
},
"request_id": {
"default": null,
"type": [
"string",
"null"
]
},
"response_format": {
"default": null
},
"session_id": {
"default": null,
"description": "Stable conversation identity, forwarded separately from the per-turn request id.",
"type": [
"string",
"null"
]
},
"system_prompt": {
"default": null,
"type": [
"string",
"null"
]
},
"system_sections": {
"default": null,
"items": {
"$ref": "#/definitions/PromptSection"
},
"type": [
"array",
"null"
]
},
"thinking_level": {
"default": null
},
"tools": {
"default": null
}
},
"required": [
"messages",
"model"
],
"title": "ChatCall",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"AssistantMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"error_kind": {
"anyOf": [
{
"$ref": "#/definitions/ErrorKind"
},
{
"type": "null"
}
]
},
"error_message": {
"type": [
"string",
"null"
]
},
"model": {
"type": "string"
},
"native_stop_reason": {
"type": [
"string",
"null"
]
},
"provider": {
"type": "string"
},
"role": {
"$ref": "#/definitions/AssistantRoleTag"
},
"stop_reason": {
"$ref": "#/definitions/StopReason"
},
"timestamp": {
"format": "int64",
"type": "integer"
},
"usage": {
"anyOf": [
{
"$ref": "#/definitions/Usage"
},
{
"type": "null"
}
]
},
"warnings": {
"items": {
"type": "string"
},
"type": [
"array",
"null"
]
}
},
"required": [
"content",
"model",
"provider",
"role",
"stop_reason",
"timestamp"
],
"type": "object"
},
"AssistantRoleTag": {
"enum": [
"assistant"
],
"type": "string"
},
"ContentBlock": {
"description": "Content blocks — the atomic units of message content (README § Content blocks).",
"oneOf": [
{
"properties": {
"text": {
"type": "string"
},
"type": {
"enum": [
"text"
],
"type": "string"
}
},
"required": [
"text",
"type"
],
"type": "object"
},
{
"properties": {
"attachment_id": {
"description": "Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.",
"type": [
"string",
"null"
]
},
"data": {
"type": "string"
},
"mime": {
"type": "string"
},
"type": {
"enum": [
"image"
],
"type": "string"
}
},
"required": [
"data",
"mime",
"type"
],
"type": "object"
},
{
"description": "Reference to an original upload in the session-manager attachment store. A REFERENCE ONLY (never inline bytes); the harness strips it before `router::chat`. Tolerated here so a stray block deserializes instead of failing the request; every counter/mapping emits nothing for it.",
"properties": {
"attachment_id": {
"type": "string"
},
"mime": {
"type": "string"
},
"name": {
"type": "string"
},
"size": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": {
"enum": [
"file"
],
"type": "string"
}
},
"required": [
"attachment_id",
"mime",
"name",
"size",
"type"
],
"type": "object"
},
{
"properties": {
"signature": {
"type": [
"string",
"null"
]
},
"text": {
"type": "string"
},
"type": {
"enum": [
"thinking"
],
"type": "string"
}
},
"required": [
"text",
"type"
],
"type": "object"
},
{
"description": "Opaque redacted thinking payload — replayed verbatim on the Anthropic wire.",
"properties": {
"data": {
"type": "string"
},
"type": {
"enum": [
"redacted_thinking"
],
"type": "string"
}
},
"required": [
"data",
"type"
],
"type": "object"
},
{
"properties": {
"arguments": true,
"function_id": {
"type": "string"
},
"id": {
"type": "string"
},
"type": {
"enum": [
"function_call"
],
"type": "string"
}
},
"required": [
"arguments",
"function_id",
"id",
"type"
],
"type": "object"
},
{
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"function_call_id": {
"type": "string"
},
"is_error": {
"type": [
"boolean",
"null"
]
},
"type": {
"enum": [
"function_result"
],
"type": "string"
}
},
"required": [
"content",
"function_call_id",
"type"
],
"type": "object"
}
]
},
"ErrorKind": {
"enum": [
"auth_expired",
"rate_limited",
"context_overflow",
"transient",
"permanent"
],
"type": "string"
},
"StopReason": {
"enum": [
"end",
"length",
"function_call",
"aborted",
"error"
],
"type": "string"
},
"Usage": {
"properties": {
"cache_read": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cache_write": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"reasoning": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
}
},
"description": "Output of the `router::complete` iii function (non-streaming convenience).",
"properties": {
"message": {
"$ref": "#/definitions/AssistantMessage"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"usage": {
"anyOf": [
{
"$ref": "#/definitions/Usage"
},
{
"type": "null"
}
]
}
},
"required": [
"message",
"model",
"provider"
],
"title": "CompleteResponse",
"type": "object"
}
},
{
"description": "Count prompt tokens for {model, provider?, system_prompt?, tools?, messages} through the resolved provider's provider::<id>::count_tokens surface; never runs the model and costs nothing.",
"metadata": {
"internal": true
},
"name": "router::count_tokens",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"AgentFunction": {
"description": "Function invocation schema — what a provider sees as a `tools` array entry (README § Function invocation schema; adapter boundary). These describe iii functions exposed to the model, not provider-native tools.",
"properties": {
"description": {
"type": "string"
},
"execution_mode": {
"type": [
"string",
"null"
]
},
"label": {
"type": [
"string",
"null"
]
},
"name": {
"type": "string"
},
"parameters": true
},
"required": [
"description",
"name",
"parameters"
],
"type": "object"
},
"AgentMessage": {
"anyOf": [
{
"$ref": "#/definitions/AssistantMessage"
},
{
"$ref": "#/definitions/FunctionResultMessage"
},
{
"$ref": "#/definitions/CustomMessage"
},
{
"$ref": "#/definitions/UserMessage"
}
],
"description": "The canonical transcript message union. Untagged: the single-variant role tags disambiguate deserialization."
},
"AssistantMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"error_kind": {
"anyOf": [
{
"$ref": "#/definitions/ErrorKind"
},
{
"type": "null"
}
]
},
"error_message": {
"type": [
"string",
"null"
]
},
"model": {
"type": "string"
},
"native_stop_reason": {
"type": [
"string",
"null"
]
},
"provider": {
"type": "string"
},
"role": {
"$ref": "#/definitions/AssistantRoleTag"
},
"stop_reason": {
"$ref": "#/definitions/StopReason"
},
"timestamp": {
"format": "int64",
"type": "integer"
},
"usage": {
"anyOf": [
{
"$ref": "#/definitions/Usage"
},
{
"type": "null"
}
]
},
"warnings": {
"items": {
"type": "string"
},
"type": [
"array",
"null"
]
}
},
"required": [
"content",
"model",
"provider",
"role",
"stop_reason",
"timestamp"
],
"type": "object"
},
"AssistantRoleTag": {
"enum": [
"assistant"
],
"type": "string"
},
"ContentBlock": {
"description": "Content blocks — the atomic units of message content (README § Content blocks).",
"oneOf": [
{
"properties": {
"text": {
"type": "string"
},
"type": {
"enum": [
"text"
],
"type": "string"
}
},
"required": [
"text",
"type"
],
"type": "object"
},
{
"properties": {
"attachment_id": {
"description": "Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.",
"type": [
"string",
"null"
]
},
"data": {
"type": "string"
},
"mime": {
"type": "string"
},
"type": {
"enum": [
"image"
],
"type": "string"
}
},
"required": [
"data",
"mime",
"type"
],
"type": "object"
},
{
"description": "Reference to an original upload in the session-manager attachment store. A REFERENCE ONLY (never inline bytes); the harness strips it before `router::chat`. Tolerated here so a stray block deserializes instead of failing the request; every counter/mapping emits nothing for it.",
"properties": {
"attachment_id": {
"type": "string"
},
"mime": {
"type": "string"
},
"name": {
"type": "string"
},
"size": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": {
"enum": [
"file"
],
"type": "string"
}
},
"required": [
"attachment_id",
"mime",
"name",
"size",
"type"
],
"type": "object"
},
{
"properties": {
"signature": {
"type": [
"string",
"null"
]
},
"text": {
"type": "string"
},
"type": {
"enum": [
"thinking"
],
"type": "string"
}
},
"required": [
"text",
"type"
],
"type": "object"
},
{
"description": "Opaque redacted thinking payload — replayed verbatim on the Anthropic wire.",
"properties": {
"data": {
"type": "string"
},
"type": {
"enum": [
"redacted_thinking"
],
"type": "string"
}
},
"required": [
"data",
"type"
],
"type": "object"
},
{
"properties": {
"arguments": true,
"function_id": {
"type": "string"
},
"id": {
"type": "string"
},
"type": {
"enum": [
"function_call"
],
"type": "string"
}
},
"required": [
"arguments",
"function_id",
"id",
"type"
],
"type": "object"
},
{
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"function_call_id": {
"type": "string"
},
"is_error": {
"type": [
"boolean",
"null"
]
},
"type": {
"enum": [
"function_result"
],
"type": "string"
}
},
"required": [
"content",
"function_call_id",
"type"
],
"type": "object"
}
]
},
"CustomMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"custom_type": {
"type": "string"
},
"details": true,
"display": {
"type": [
"string",
"null"
]
},
"role": {
"$ref": "#/definitions/CustomRoleTag"
},
"timestamp": {
"format": "int64",
"type": "integer"
}
},
"required": [
"content",
"custom_type",
"role",
"timestamp"
],
"type": "object"
},
"CustomRoleTag": {
"enum": [
"custom"
],
"type": "string"
},
"ErrorKind": {
"enum": [
"auth_expired",
"rate_limited",
"context_overflow",
"transient",
"permanent"
],
"type": "string"
},
"FunctionResultMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"details": true,
"function_call_id": {
"type": "string"
},
"function_id": {
"type": "string"
},
"is_error": {
"type": "boolean"
},
"role": {
"$ref": "#/definitions/FunctionResultRoleTag"
},
"timestamp": {
"format": "int64",
"type": "integer"
}
},
"required": [
"content",
"details",
"function_call_id",
"function_id",
"is_error",
"role",
"timestamp"
],
"type": "object"
},
"FunctionResultRoleTag": {
"enum": [
"function_result"
],
"type": "string"
},
"StopReason": {
"enum": [
"end",
"length",
"function_call",
"aborted",
"error"
],
"type": "string"
},
"Usage": {
"properties": {
"cache_read": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cache_write": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"reasoning": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"UserMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"role": {
"$ref": "#/definitions/UserRoleTag"
},
"timestamp": {
"format": "int64",
"type": "integer"
}
},
"required": [
"content",
"role",
"timestamp"
],
"type": "object"
},
"UserRoleTag": {
"description": "Single-variant role tags: exact-match on deserialize, correct wire string on serialize, and they let `AgentMessage` be an untagged union.",
"enum": [
"user"
],
"type": "string"
}
},
"properties": {
"messages": {
"description": "Wire agent messages, the same shape `router::chat` accepts. Must be non-empty.",
"items": {
"$ref": "#/definitions/AgentMessage"
},
"type": "array"
},
"model": {
"default": null,
"description": "Model id the prompt targets; routes exactly like `router::chat` and selects the provider's tokenizer. Optional only when `provider` pins the destination.",
"type": [
"string",
"null"
]
},
"provider": {
"default": null,
"description": "Pin an explicit provider, bypassing heuristics (optional).",
"type": [
"string",
"null"
]
},
"system_prompt": {
"default": null,
"description": "System prompt counted as part of the request (optional).",
"type": [
"string",
"null"
]
},
"tools": {
"default": null,
"description": "Function invocation schemas the turn would carry; their serialized schemas count toward the total (optional).",
"items": {
"$ref": "#/definitions/AgentFunction"
},
"type": [
"array",
"null"
]
}
},
"required": [
"messages"
],
"title": "RouterCountTokensRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"estimator": {
"description": "`provider` when a provider metering API produced the count, `tiktoken` when a local tokenizer estimated it.",
"type": "string"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"tokens": {
"description": "Prompt tokens the provider counted for the assembled request.",
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"estimator",
"model",
"provider",
"tokens"
],
"title": "RouterCountTokensResponse",
"type": "object"
}
},
{
"description": "Batch text embeddings through a provider's provider::<id>::embed surface. Names a provider or discovers the first embed-capable one from the live registry; one vector per input, order preserved.",
"metadata": {
"internal": true
},
"name": "router::embed",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"input": {
"description": "Texts to embed, one vector returned per input, order preserved.",
"items": {
"type": "string"
},
"type": "array"
},
"model": {
"default": null,
"description": "Embedding model id; the provider's default when omitted.",
"type": [
"string",
"null"
]
},
"provider": {
"default": null,
"description": "Provider id (e.g. `openai`); the first embed-capable provider in the live registry when omitted.",
"type": [
"string",
"null"
]
}
},
"required": [
"input"
],
"title": "RouterEmbedRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"embeddings": {
"description": "One embedding per input, in input order.",
"items": {
"items": {
"format": "float",
"type": "number"
},
"type": "array"
},
"type": "array"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
}
},
"required": [
"embeddings",
"model",
"provider"
],
"title": "RouterEmbedResponse",
"type": "object"
}
},
{
"description": "Resolve the effective model output budget using the same precedence as router::chat.",
"metadata": {
"internal": true
},
"name": "router::models::budget",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::models::budget`.",
"properties": {
"id": {
"default": "",
"description": "Model id to budget.",
"type": "string"
},
"max_output_tokens": {
"description": "Optional caller-requested output budget. When absent, the same provider and router defaults used by `router::chat` apply.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"provider": {
"default": "",
"description": "Provider id that owns the model. Empty resolves an unambiguous model id.",
"type": "string"
}
},
"title": "ModelBudgetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"anyOf": [
{
"$ref": "#/definitions/ModelBudgetResponse"
},
{
"type": "null"
}
],
"definitions": {
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"reasoning_efforts": {
"items": {
"$ref": "#/definitions/ReasoningEffort"
},
"type": [
"array",
"null"
]
},
"speech": {
"anyOf": [
{
"$ref": "#/definitions/SpeechModel"
},
{
"type": "null"
}
],
"description": "Set on speech models only; see [`SpeechModel`]."
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_thinking_off": {
"description": "Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.",
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"ModelBudgetResponse": {
"description": "Effective limits `router::chat` will use for this model and request.",
"properties": {
"effective_max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"model": {
"$ref": "#/definitions/Model"
}
},
"required": [
"effective_max_output_tokens",
"model"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"ReasoningEffort": {
"description": "One provider-native reasoning effort advertised for a specific model.\n\nValues intentionally remain strings: provider catalogs can add efforts without requiring a router-wide enum release first.",
"properties": {
"description": {
"type": [
"string",
"null"
]
},
"effort": {
"type": "string"
}
},
"required": [
"effort"
],
"type": "object"
},
"SpeechModality": {
"description": "What a speech model does. Chat models carry no `speech` block.",
"oneOf": [
{
"description": "Speech to text, served through `router::transcribe`.",
"enum": [
"stt"
],
"type": "string"
},
{
"description": "Text to speech, served through `router::speak`.",
"enum": [
"tts"
],
"type": "string"
}
]
},
"SpeechModel": {
"description": "Facts about a speech model. Present only on models served through `router::transcribe` / `router::speak`; such models report `context_window` and `max_output_tokens` as 0.",
"properties": {
"languages": {
"description": "BCP-47 tags of the languages the model handles; empty when the provider does not say.",
"items": {
"type": "string"
},
"type": "array"
},
"modality": {
"$ref": "#/definitions/SpeechModality"
},
"streaming": {
"default": false,
"description": "Realtime input (stt) or streamed audio output (tts) is available.",
"type": "boolean"
}
},
"required": [
"modality"
],
"type": "object"
}
},
"title": "Nullable_ModelBudgetResponse"
}
},
{
"description": "Read one catalog model by {provider, id}; null when the model is not registered.",
"metadata": {},
"name": "router::models::get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::models::get`.",
"properties": {
"id": {
"default": "",
"description": "Model id to look up.",
"type": "string"
},
"provider": {
"default": "",
"description": "Provider id that owns the model.",
"type": "string"
}
},
"title": "ModelGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"anyOf": [
{
"$ref": "#/definitions/ModelGetResponse"
},
{
"type": "null"
}
],
"definitions": {
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"reasoning_efforts": {
"items": {
"$ref": "#/definitions/ReasoningEffort"
},
"type": [
"array",
"null"
]
},
"speech": {
"anyOf": [
{
"$ref": "#/definitions/SpeechModel"
},
{
"type": "null"
}
],
"description": "Set on speech models only; see [`SpeechModel`]."
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_thinking_off": {
"description": "Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.",
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"ModelGetResponse": {
"description": "Output of `router::models::get` (the function returns `null` when the model is not registered — the cold-window signal).",
"properties": {
"model": {
"$ref": "#/definitions/Model"
}
},
"required": [
"model"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"ReasoningEffort": {
"description": "One provider-native reasoning effort advertised for a specific model.\n\nValues intentionally remain strings: provider catalogs can add efforts without requiring a router-wide enum release first.",
"properties": {
"description": {
"type": [
"string",
"null"
]
},
"effort": {
"type": "string"
}
},
"required": [
"effort"
],
"type": "object"
},
"SpeechModality": {
"description": "What a speech model does. Chat models carry no `speech` block.",
"oneOf": [
{
"description": "Speech to text, served through `router::transcribe`.",
"enum": [
"stt"
],
"type": "string"
},
{
"description": "Text to speech, served through `router::speak`.",
"enum": [
"tts"
],
"type": "string"
}
]
},
"SpeechModel": {
"description": "Facts about a speech model. Present only on models served through `router::transcribe` / `router::speak`; such models report `context_window` and `max_output_tokens` as 0.",
"properties": {
"languages": {
"description": "BCP-47 tags of the languages the model handles; empty when the provider does not say.",
"items": {
"type": "string"
},
"type": "array"
},
"modality": {
"$ref": "#/definitions/SpeechModality"
},
"streaming": {
"default": false,
"description": "Realtime input (stt) or streamed audio output (tts) is available.",
"type": "boolean"
}
},
"required": [
"modality"
],
"type": "object"
}
},
"title": "Nullable_ModelGetResponse"
}
},
{
"description": "List catalog models, optionally filtered by provider, a capability flag, and modality (chat by default; stt, tts, or any).",
"metadata": {},
"name": "router::models::list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ModalityFilter": {
"description": "Model family selector for `router::models::list`. `chat` is the default so pickers built before speech models existed keep listing only what `router::chat` can run.",
"enum": [
"chat",
"stt",
"tts",
"any"
],
"type": "string"
}
},
"description": "Input of `router::models::list`.",
"properties": {
"capability": {
"description": "Keep only models that support this capability flag (optional).",
"type": [
"string",
"null"
]
},
"modality": {
"anyOf": [
{
"$ref": "#/definitions/ModalityFilter"
},
{
"type": "null"
}
],
"description": "Model family: `chat` (default), `stt`, `tts`, or `any`."
},
"provider": {
"description": "Filter to a single provider id (optional).",
"type": [
"string",
"null"
]
}
},
"title": "ModelsListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"reasoning_efforts": {
"items": {
"$ref": "#/definitions/ReasoningEffort"
},
"type": [
"array",
"null"
]
},
"speech": {
"anyOf": [
{
"$ref": "#/definitions/SpeechModel"
},
{
"type": "null"
}
],
"description": "Set on speech models only; see [`SpeechModel`]."
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_thinking_off": {
"description": "Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.",
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"ReasoningEffort": {
"description": "One provider-native reasoning effort advertised for a specific model.\n\nValues intentionally remain strings: provider catalogs can add efforts without requiring a router-wide enum release first.",
"properties": {
"description": {
"type": [
"string",
"null"
]
},
"effort": {
"type": "string"
}
},
"required": [
"effort"
],
"type": "object"
},
"SpeechModality": {
"description": "What a speech model does. Chat models carry no `speech` block.",
"oneOf": [
{
"description": "Speech to text, served through `router::transcribe`.",
"enum": [
"stt"
],
"type": "string"
},
{
"description": "Text to speech, served through `router::speak`.",
"enum": [
"tts"
],
"type": "string"
}
]
},
"SpeechModel": {
"description": "Facts about a speech model. Present only on models served through `router::transcribe` / `router::speak`; such models report `context_window` and `max_output_tokens` as 0.",
"properties": {
"languages": {
"description": "BCP-47 tags of the languages the model handles; empty when the provider does not say.",
"items": {
"type": "string"
},
"type": "array"
},
"modality": {
"$ref": "#/definitions/SpeechModality"
},
"streaming": {
"default": false,
"description": "Realtime input (stt) or streamed audio output (tts) is available.",
"type": "boolean"
}
},
"required": [
"modality"
],
"type": "object"
}
},
"description": "Output of `router::models::list`.",
"properties": {
"models": {
"items": {
"$ref": "#/definitions/Model"
},
"type": "array"
}
},
"required": [
"models"
],
"title": "ModelsListResponse",
"type": "object"
}
},
{
"description": "Replace a provider's catalog slice — the only catalog write path (token-gated).",
"metadata": {
"internal": true
},
"name": "router::models::reconcile",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"DiscoveryCode": {
"description": "Explicit allowlist: arbitrary error strings cannot cross the router boundary.",
"enum": [
"permission-denied",
"invalid-api-key",
"insufficient-credits",
"spending-limit-exceeded",
"rate-limit-exceeded"
],
"type": "string"
},
"DiscoveryOutcome": {
"description": "Safe discovery categories. No upstream message, team id or URL is retained.",
"enum": [
"success",
"empty",
"not_configured",
"billing",
"authentication",
"permission",
"rate_limit",
"unavailable",
"invalid_response"
],
"type": "string"
},
"DiscoveryReport": {
"additionalProperties": false,
"properties": {
"attempt": {
"type": "string"
},
"code": {
"anyOf": [
{
"$ref": "#/definitions/DiscoveryCode"
},
{
"type": "null"
}
]
},
"http_status": {
"format": "uint16",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"outcome": {
"$ref": "#/definitions/DiscoveryOutcome"
}
},
"required": [
"attempt",
"outcome"
],
"type": "object"
},
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"reasoning_efforts": {
"items": {
"$ref": "#/definitions/ReasoningEffort"
},
"type": [
"array",
"null"
]
},
"speech": {
"anyOf": [
{
"$ref": "#/definitions/SpeechModel"
},
{
"type": "null"
}
],
"description": "Set on speech models only; see [`SpeechModel`]."
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_thinking_off": {
"description": "Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.",
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"ReasoningEffort": {
"description": "One provider-native reasoning effort advertised for a specific model.\n\nValues intentionally remain strings: provider catalogs can add efforts without requiring a router-wide enum release first.",
"properties": {
"description": {
"type": [
"string",
"null"
]
},
"effort": {
"type": "string"
}
},
"required": [
"effort"
],
"type": "object"
},
"SpeechModality": {
"description": "What a speech model does. Chat models carry no `speech` block.",
"oneOf": [
{
"description": "Speech to text, served through `router::transcribe`.",
"enum": [
"stt"
],
"type": "string"
},
{
"description": "Text to speech, served through `router::speak`.",
"enum": [
"tts"
],
"type": "string"
}
]
},
"SpeechModel": {
"description": "Facts about a speech model. Present only on models served through `router::transcribe` / `router::speak`; such models report `context_window` and `max_output_tokens` as 0.",
"properties": {
"languages": {
"description": "BCP-47 tags of the languages the model handles; empty when the provider does not say.",
"items": {
"type": "string"
},
"type": "array"
},
"modality": {
"$ref": "#/definitions/SpeechModality"
},
"streaming": {
"default": false,
"description": "Realtime input (stt) or streamed audio output (tts) is available.",
"type": "boolean"
}
},
"required": [
"modality"
],
"type": "object"
}
},
"description": "Input of `router::models::reconcile` — the only catalog write path.",
"properties": {
"discovery": {
"anyOf": [
{
"$ref": "#/definitions/DiscoveryReport"
},
{
"type": "null"
}
],
"description": "Optional token-gated discovery completion. Failures preserve the slice."
},
"models": {
"default": [],
"description": "The full replacement set of models for this provider.",
"items": {
"$ref": "#/definitions/Model"
},
"type": "array"
},
"provider": {
"default": "",
"description": "Provider whose catalog slice is being replaced.",
"type": "string"
},
"token": {
"description": "Registration token gating the write (optional).",
"type": [
"string",
"null"
]
}
},
"title": "ModelsReconcileRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of the `router::models::reconcile` iii function.",
"properties": {
"count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"provider": {
"type": "string"
}
},
"required": [
"count",
"provider"
],
"title": "ModelsReconcileResponse",
"type": "object"
}
},
{
"description": "Check whether a model supports a capability flag (fails open for unknown models).",
"metadata": {},
"name": "router::models::supports",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::models::supports`.",
"properties": {
"capability": {
"default": "",
"description": "Capability flag to check (e.g. `structured_output`, `vision`).",
"type": "string"
},
"id": {
"default": "",
"description": "Model id to check.",
"type": "string"
},
"provider": {
"default": "",
"description": "Provider id that owns the model.",
"type": "string"
}
},
"title": "ModelsSupportsRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of `router::models::supports`.",
"properties": {
"supported": {
"type": "boolean"
}
},
"required": [
"supported"
],
"title": "ModelsSupportsResponse",
"type": "object"
}
},
{
"description": "Internal: reactively reload the in-memory configuration snapshot and fan out provider model discovery after configuration changes.",
"metadata": {
"internal": true,
"trace_hidden": true
},
"name": "router::on_config_changed",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Advisory configuration-change event delivered to `router::on_config_changed`. The handler ignores event values and re-fetches the authoritative entry before replacing its in-memory snapshot.",
"properties": {
"id": {
"description": "Configuration id that changed (advisory).",
"type": [
"string",
"null"
]
}
},
"title": "ConfigChangedEvent",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Generic acknowledgement returned by trigger-bound handlers whose result is not consumed by callers (kept typed so the response schema is concrete).",
"properties": {
"ok": {
"type": "boolean"
}
},
"required": [
"ok"
],
"title": "RouterAck",
"type": "object"
}
},
{
"description": "Internal: a worker's function registrations changed — re-discover live providers and nudge them to re-declare, so a provider that reconnected is resolvable again without waiting for its own catalog timer.",
"metadata": {
"internal": true,
"trace_hidden": true
},
"name": "router::on_functions_changed",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Advisory `engine::functions-available` event delivered to `router::on_functions_changed`. The engine sends `{ event: \"functions_changed\", functions: [...] }` with the full current function list; it does not say which worker or function changed. The handler ignores event values (the `functions` list is not deserialized) and re-fetches the authoritative registry before nudging live providers.",
"properties": {
"event": {
"description": "Engine event tag, `\"functions_changed\"` (advisory).",
"type": [
"string",
"null"
]
}
},
"title": "FunctionsChangedEvent",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Generic acknowledgement returned by trigger-bound handlers whose result is not consumed by callers (kept typed so the response schema is concrete).",
"properties": {
"ok": {
"type": "boolean"
}
},
"required": [
"ok"
],
"title": "RouterAck",
"type": "object"
}
},
{
"description": "Internal: a secret changed in the secrets worker — re-resolve the secret:// or env:// credential reference and refresh the providers that use it.",
"metadata": {
"internal": true,
"trace_hidden": true
},
"name": "router::on_secret_changed",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Advisory `secrets::changed` event delivered to `router::on_secret_changed`. It never carries a value; the handler re-reads the reference through `secrets::resolve`. Other payload fields (`fingerprint`, `updated_at`) are ignored.",
"properties": {
"action": {
"description": "`created`, `rotated`, `deleted` or `access_changed` (advisory).",
"type": [
"string",
"null"
]
},
"name": {
"description": "Secret name (advisory).",
"type": [
"string",
"null"
]
},
"ref": {
"description": "The same secret as a `secret://NAME` or `env://NAME` reference; it names the store (advisory).",
"type": [
"string",
"null"
]
}
},
"title": "SecretChangedEvent",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Generic acknowledgement returned by trigger-bound handlers whose result is not consumed by callers (kept typed so the response schema is concrete).",
"properties": {
"ok": {
"type": "boolean"
}
},
"required": [
"ok"
],
"title": "RouterAck",
"type": "object"
}
},
{
"description": "List registered providers with their configured/available status, where each credential comes from (credential_source, credential_ref, credential_error; never a value), and the provider's starting point (default_model: the newest model of the best-ranked variant its preferences name, else its first declared preference present in the catalog, else the closest same-family model; default_thinking_level).",
"metadata": {},
"name": "router::provider::list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::provider::list` — takes no arguments. A struct (rather than `Value`) keeps the request schema concrete; unknown fields (e.g. the engine-injected `_caller_worker_id`) are ignored.",
"title": "ProviderListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"CredentialOrigin": {
"description": "`credential_source`: where the precedence found the provider's credential. `secret` is a `secret://NAME` or `env://NAME` reference in the slice, resolved through the `secrets` worker; `configured` says whether it actually yielded one.",
"enum": [
"config",
"env",
"secret",
"none"
],
"type": "string"
},
"DiscoveryCode": {
"description": "Explicit allowlist: arbitrary error strings cannot cross the router boundary.",
"enum": [
"permission-denied",
"invalid-api-key",
"insufficient-credits",
"spending-limit-exceeded",
"rate-limit-exceeded"
],
"type": "string"
},
"DiscoveryOutcome": {
"description": "Safe discovery categories. No upstream message, team id or URL is retained.",
"enum": [
"success",
"empty",
"not_configured",
"billing",
"authentication",
"permission",
"rate_limit",
"unavailable",
"invalid_response"
],
"type": "string"
},
"DiscoveryStatus": {
"additionalProperties": false,
"properties": {
"checked_at_ms": {
"format": "int64",
"type": "integer"
},
"code": {
"anyOf": [
{
"$ref": "#/definitions/DiscoveryCode"
},
{
"type": "null"
}
]
},
"http_status": {
"format": "uint16",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"outcome": {
"$ref": "#/definitions/DiscoveryOutcome"
},
"stale": {
"description": "A transient failure left a previous catalog in place.",
"type": "boolean"
}
},
"required": [
"checked_at_ms",
"outcome",
"stale"
],
"type": "object"
},
"ProviderInfo": {
"description": "One `router::provider::list` entry.",
"properties": {
"available": {
"type": "boolean"
},
"configured": {
"type": "boolean"
},
"context_overflow_hint": {
"description": "Provider-specific guidance a console appends to a context-overflow failure on one of this provider's models; copied from the declaration. Absent when the provider declared none.",
"type": [
"string",
"null"
]
},
"credential_env_var": {
"description": "Environment variable declared by API-key providers. `None` means the provider owns authentication (OAuth, local app login, device flow).",
"type": [
"string",
"null"
]
},
"credential_error": {
"description": "Why the configured credential cannot be used, in words an operator can act on (e.g. the secret is missing or this worker may not read it).",
"type": [
"string",
"null"
]
},
"credential_ref": {
"description": "The slice's `secret://NAME` or `env://NAME` reference, when it uses one.",
"type": [
"string",
"null"
]
},
"credential_source": {
"anyOf": [
{
"$ref": "#/definitions/CredentialOrigin"
},
{
"type": "null"
}
],
"description": "Where the credential comes from: `config`, `env`, `secret` or `none`."
},
"default_model": {
"description": "The model a consumer should start with when nothing else names one: the newest model of the best-ranked variant its declared `default_models` name (Terra, Sol, Astra, Luna for GPT providers; Sonnet, Opus, Fable, Haiku for Claude providers), else the first declared id in its catalog slice, else the closest same-family model, else absent (see `registry::availability::resolve_default_model`).",
"type": [
"string",
"null"
]
},
"default_thinking_level": {
"anyOf": [
{
"$ref": "#/definitions/ThinkingLevel"
},
{
"type": "null"
}
],
"description": "The thinking level a consumer should start with when nothing else names one; copied from the declaration. Absent means omit the level and let the provider apply its own default."
},
"discovery": {
"anyOf": [
{
"$ref": "#/definitions/DiscoveryStatus"
},
{
"type": "null"
}
],
"description": "Last completed discovery, independent of configuration and availability."
},
"display_name": {
"type": "string"
},
"icon_svg": {
"description": "The provider's mark as inline SVG, copied from its declaration. Absent when the provider declared none.",
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"supports_model_listing": {
"type": "boolean"
}
},
"required": [
"available",
"configured",
"display_name",
"id",
"supports_model_listing"
],
"type": "object"
},
"ThinkingLevel": {
"description": "\"off\" asks the provider not to reason at all and is honoured only where the model advertises `Model::supports_thinking_off` (elsewhere the provider falls back to its own lowest setting, or omits the parameter, and warns). \"minimal\" requests the lowest reasoning effort and needs only `thinking` support; levels map to provider-native knobs via `Model::thinking_budgets`.",
"enum": [
"off",
"minimal",
"low",
"medium",
"high",
"xhigh"
],
"type": "string"
}
},
"properties": {
"providers": {
"items": {
"$ref": "#/definitions/ProviderInfo"
},
"type": "array"
}
},
"required": [
"providers"
],
"title": "ProviderListResponse",
"type": "object"
}
},
{
"description": "Provider self-declaration at attach time (token-gated upsert); composes the configuration entry schema and reconciles static models.",
"metadata": {
"internal": true
},
"name": "router::provider::register",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"reasoning_efforts": {
"items": {
"$ref": "#/definitions/ReasoningEffort"
},
"type": [
"array",
"null"
]
},
"speech": {
"anyOf": [
{
"$ref": "#/definitions/SpeechModel"
},
{
"type": "null"
}
],
"description": "Set on speech models only; see [`SpeechModel`]."
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_thinking_off": {
"description": "Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.",
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"ProviderDefaults": {
"additionalProperties": true,
"properties": {
"api_url": {
"type": [
"string",
"null"
]
},
"max_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"ReasoningEffort": {
"description": "One provider-native reasoning effort advertised for a specific model.\n\nValues intentionally remain strings: provider catalogs can add efforts without requiring a router-wide enum release first.",
"properties": {
"description": {
"type": [
"string",
"null"
]
},
"effort": {
"type": "string"
}
},
"required": [
"effort"
],
"type": "object"
},
"SpeechModality": {
"description": "What a speech model does. Chat models carry no `speech` block.",
"oneOf": [
{
"description": "Speech to text, served through `router::transcribe`.",
"enum": [
"stt"
],
"type": "string"
},
{
"description": "Text to speech, served through `router::speak`.",
"enum": [
"tts"
],
"type": "string"
}
]
},
"SpeechModel": {
"description": "Facts about a speech model. Present only on models served through `router::transcribe` / `router::speak`; such models report `context_window` and `max_output_tokens` as 0.",
"properties": {
"languages": {
"description": "BCP-47 tags of the languages the model handles; empty when the provider does not say.",
"items": {
"type": "string"
},
"type": "array"
},
"modality": {
"$ref": "#/definitions/SpeechModality"
},
"streaming": {
"default": false,
"description": "Realtime input (stt) or streamed audio output (tts) is available.",
"type": "boolean"
}
},
"required": [
"modality"
],
"type": "object"
},
"ThinkingLevel": {
"description": "\"off\" asks the provider not to reason at all and is honoured only where the model advertises `Model::supports_thinking_off` (elsewhere the provider falls back to its own lowest setting, or omits the parameter, and warns). \"minimal\" requests the lowest reasoning effort and needs only `thinking` support; levels map to provider-native knobs via `Model::thinking_budgets`.",
"enum": [
"off",
"minimal",
"low",
"medium",
"high",
"xhigh"
],
"type": "string"
}
},
"description": "Input of `router::provider::register` — a provider worker's declaration plus the optional re-registration token.",
"properties": {
"config_schema": true,
"context_overflow_hint": {
"description": "Guidance a console shows under a context-overflow failure on one of this provider's models, for causes the console cannot fix by compacting (a self-hosted server started with a small window, say). One or two plain sentences, optionally with a link to the provider's docs. `router::provider::list` carries it verbatim.",
"type": [
"string",
"null"
]
},
"credential_env_var": {
"type": [
"string",
"null"
]
},
"credential_optional": {
"description": "`true` when the provider works without a credential (a local server started without a key): the router then reports it `configured` with no key. A key that is set is still resolved and sent. Absent means a key is required.",
"type": [
"boolean",
"null"
]
},
"default_models": {
"description": "Chat model ids the provider recommends as a starting point. The order in which the list first names each variant sets the variant rank. `router::provider::list` reports the newest catalog model of the best-ranked variant as `default_model` (see `registry::availability::resolve_default_model`).",
"items": {
"type": "string"
},
"type": [
"array",
"null"
]
},
"default_thinking_level": {
"anyOf": [
{
"$ref": "#/definitions/ThinkingLevel"
},
{
"type": "null"
}
],
"description": "The thinking level to use with the default model when a caller names none. Absent means omit the level."
},
"defaults": {
"anyOf": [
{
"$ref": "#/definitions/ProviderDefaults"
},
{
"type": "null"
}
]
},
"display_name": {
"type": [
"string",
"null"
]
},
"icon_svg": {
"description": "The provider's mark: one self-contained `<svg>` document, monochrome, drawn with a square `viewBox` and no fixed size. Consoles paint it as a `currentColor` mask next to the provider's models, so colour and scripts are ignored. Keep it under [`PROVIDER_ICON_SVG_MAX_BYTES`]; the router drops anything larger or not starting with `<svg`.",
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"models": {
"items": {
"$ref": "#/definitions/Model"
},
"type": [
"array",
"null"
]
},
"supports_model_listing": {
"type": [
"boolean",
"null"
]
},
"token": {
"description": "Registration token proving ownership on re-register (omit on first declare).",
"type": [
"string",
"null"
]
},
"worker_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"id"
],
"title": "ProviderRegisterRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "`registration_token` is the provider ownership credential; only its sha256 hash is persisted. Engine caller metadata is not an authorization identity because worker names are self-reported.",
"properties": {
"id": {
"type": "string"
},
"ok": {
"type": "boolean"
},
"registration_token": {
"type": "string"
}
},
"required": [
"id",
"ok",
"registration_token"
],
"title": "ProviderRegisterResponse",
"type": "object"
}
},
{
"description": "Resolve a provider's effective credential + api_url + max_tokens (token-gated); a secret://NAME or env://NAME api_key resolves through the secrets worker.",
"metadata": {
"internal": true
},
"name": "router::provider::resolve",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::provider::resolve`.",
"properties": {
"begin_discovery": {
"default": false,
"description": "Begin a latest-wins discovery attempt. Older providers omit this.",
"type": "boolean"
},
"id": {
"default": "",
"description": "Provider id to resolve credentials/config for.",
"type": "string"
},
"token": {
"description": "Registration token gating the resolve (optional).",
"type": [
"string",
"null"
]
}
},
"title": "ProviderResolveRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Credential": {
"oneOf": [
{
"properties": {
"key": {
"type": "string"
},
"type": {
"enum": [
"api_key"
],
"type": "string"
}
},
"required": [
"key",
"type"
],
"type": "object"
},
{
"properties": {
"access_token": {
"type": "string"
},
"expires_at": {
"format": "int64",
"type": [
"integer",
"null"
]
},
"provider_extra": true,
"refresh_token": {
"type": [
"string",
"null"
]
},
"scopes": {
"items": {
"type": "string"
},
"type": [
"array",
"null"
]
},
"type": {
"enum": [
"oauth"
],
"type": "string"
}
},
"required": [
"access_token",
"type"
],
"type": "object"
}
]
},
"CredentialOrigin": {
"description": "`credential_source`: where the precedence found the provider's credential. `secret` is a `secret://NAME` or `env://NAME` reference in the slice, resolved through the `secrets` worker; `configured` says whether it actually yielded one.",
"enum": [
"config",
"env",
"secret",
"none"
],
"type": "string"
},
"CredentialSource": {
"description": "Where the resolved credential came from, as providers read it. A resolved `secret://` or `env://` reference reports `config` (the reference lives in the configuration entry); `credential_source` tells them apart.",
"enum": [
"config",
"env",
"none"
],
"type": "string"
}
},
"description": "Output of the `router::provider::resolve` iii function: the effective credential and settings plus the credential diagnostics.",
"properties": {
"api_url": {
"type": [
"string",
"null"
]
},
"configured": {
"type": "boolean"
},
"credential": {
"anyOf": [
{
"$ref": "#/definitions/Credential"
},
{
"type": "null"
}
]
},
"credential_error": {
"description": "Why the configured credential cannot be used, in words an operator can act on (e.g. the secret is missing or this worker may not read it).",
"type": [
"string",
"null"
]
},
"credential_ref": {
"description": "The slice's `secret://NAME` or `env://NAME` reference, when it uses one.",
"type": [
"string",
"null"
]
},
"credential_source": {
"anyOf": [
{
"$ref": "#/definitions/CredentialOrigin"
},
{
"type": "null"
}
],
"description": "Where the credential comes from: `config`, `env`, `secret` or `none`."
},
"discovery_attempt": {
"description": "Opaque lease returned only to a token-authenticated discovery caller.",
"type": [
"string",
"null"
]
},
"max_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"source": {
"$ref": "#/definitions/CredentialSource"
}
},
"required": [
"configured",
"source"
],
"title": "ProviderResolveOutput",
"type": "object"
}
},
{
"description": "OAuth write-back: store a provider credential in the configuration entry under the entry write lock (token-gated).",
"metadata": {
"internal": true
},
"name": "router::provider::update_credential",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::provider::update_credential` (OAuth write-back).",
"properties": {
"credential": {
"default": null,
"description": "The credential object to store (provider-specific shape)."
},
"id": {
"default": "",
"description": "Provider id whose credential slice is being written.",
"type": "string"
},
"token": {
"description": "Registration token gating the write (optional).",
"type": [
"string",
"null"
]
}
},
"title": "UpdateCredentialRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of `router::provider::update_credential`.",
"properties": {
"ok": {
"type": "boolean"
}
},
"required": [
"ok"
],
"title": "UpdateCredentialResponse",
"type": "object"
}
},
{
"description": "Read-only routing preview: resolve {model, provider?} to the chosen provider + ordered candidates without streaming.",
"metadata": {
"internal": true
},
"name": "router::route",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::route` — the read-only routing preview.",
"properties": {
"model": {
"default": "",
"description": "Model id to route.",
"type": "string"
},
"provider": {
"description": "Pin an explicit provider, bypassing heuristics (optional).",
"type": [
"string",
"null"
]
}
},
"title": "RouteRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of `router::route`.",
"properties": {
"candidates": {
"items": {
"type": "string"
},
"type": "array"
},
"provider": {
"type": "string"
}
},
"required": [
"candidates",
"provider"
],
"title": "RouteResponse",
"type": "object"
}
},
{
"description": "Text to speech through provider::<id>::speak: {model?, provider?, text, voice?, format?} to {provider, model, audio_base64, mime}. Resolves the provider from the catalog's tts models when none is named.",
"metadata": {
"internal": true
},
"name": "router::speak",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"format": {
"default": null,
"description": "Audio container wanted: `mp3` (default), `wav`, `pcm16`, `opus`. Providers answer with what they can and say so in `mime`.",
"type": [
"string",
"null"
]
},
"language": {
"default": null,
"description": "BCP-47 language hint for multilingual voices.",
"type": [
"string",
"null"
]
},
"model": {
"default": null,
"description": "Text-to-speech model id (a `tts` model from `router::models::list`); the provider's default when omitted.",
"type": [
"string",
"null"
]
},
"provider": {
"default": null,
"description": "Provider id; resolved from the model's catalog owner when omitted.",
"type": [
"string",
"null"
]
},
"speed": {
"default": null,
"description": "Speaking-rate multiplier; 1.0 is the voice's own pace.",
"format": "double",
"type": [
"number",
"null"
]
},
"text": {
"description": "Text to speak.",
"type": "string"
},
"voice": {
"default": null,
"description": "Voice id or name as the provider knows it; the provider's default when omitted.",
"type": [
"string",
"null"
]
}
},
"required": [
"text"
],
"title": "RouterSpeakRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"audio_base64": {
"description": "The audio, base64.",
"type": "string"
},
"duration_secs": {
"format": "double",
"type": [
"number",
"null"
]
},
"mime": {
"description": "MIME type of the audio, e.g. `audio/mpeg`.",
"type": "string"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"voice": {
"description": "Voice the provider used.",
"type": [
"string",
"null"
]
}
},
"required": [
"audio_base64",
"mime",
"model",
"provider"
],
"title": "RouterSpeakResponse",
"type": "object"
}
},
{
"description": "Speech to text through provider::<id>::transcribe: {model?, provider?, audio_base64, mime?, language?} to {provider, model, text, segments?}. Resolves the provider from the catalog's stt models when none is named.",
"metadata": {
"internal": true
},
"name": "router::transcribe",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"audio_base64": {
"description": "The audio, base64. WAV unless `mime` says otherwise.",
"type": "string"
},
"language": {
"default": null,
"description": "BCP-47 language hint; the model detects the language when omitted.",
"type": [
"string",
"null"
]
},
"mime": {
"default": null,
"description": "MIME type of the audio: `audio/wav` (default), `audio/mpeg`, `audio/webm`, `audio/ogg`, `audio/flac`.",
"type": [
"string",
"null"
]
},
"model": {
"default": null,
"description": "Speech-to-text model id (an `stt` model from `router::models::list`); the provider's default when omitted.",
"type": [
"string",
"null"
]
},
"prompt": {
"default": null,
"description": "Vocabulary or context hint for the recognizer, when the provider takes one.",
"type": [
"string",
"null"
]
},
"provider": {
"default": null,
"description": "Provider id; resolved from the model's catalog owner when omitted.",
"type": [
"string",
"null"
]
}
},
"required": [
"audio_base64"
],
"title": "RouterTranscribeRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"TranscriptSegment": {
"description": "One timed span of a transcript; times are absent when the provider gives none.",
"properties": {
"end_secs": {
"format": "double",
"type": [
"number",
"null"
]
},
"start_secs": {
"format": "double",
"type": [
"number",
"null"
]
},
"text": {
"type": "string"
}
},
"required": [
"text"
],
"type": "object"
}
},
"properties": {
"duration_secs": {
"format": "double",
"type": [
"number",
"null"
]
},
"language": {
"description": "Language the provider detected or was told (BCP-47).",
"type": [
"string",
"null"
]
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"segments": {
"description": "Timed spans when the provider produces them.",
"items": {
"$ref": "#/definitions/TranscriptSegment"
},
"type": "array"
},
"text": {
"description": "The whole transcript.",
"type": "string"
}
},
"required": [
"model",
"provider",
"text"
],
"title": "RouterTranscribeResponse",
"type": "object"
}
}
],
"triggers": [
{
"description": "A provider reconciled its catalog slice. Payload: { provider, count }.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by all three router event bindings. No filters in v1.",
"title": "EventBindingConfig",
"type": "object"
},
"metadata": {},
"name": "router::models::changed",
"return_schema": {}
},
{
"description": "The provider registry changed (declare / availability flip). Payload: { provider, op }.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by all three router event bindings. No filters in v1.",
"title": "EventBindingConfig",
"type": "object"
},
"metadata": {},
"name": "router::provider::changed",
"return_schema": {}
},
{
"description": "The router finished booting; providers bind here and re-declare after a restart.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by all three router event bindings. No filters in v1.",
"title": "EventBindingConfig",
"type": "object"
},
"metadata": {},
"name": "router::ready",
"return_schema": {}
}
]
}