One front door + provider protocol in front of every LLM provider.
iii trigger compose::add worker=llm-routerRun in your project with the iii engine running. Exact versions are immutable and digest-pinned.
Serve the llm-router worker's injected console UI assets (content function for its console:script / console:style / console:module triggers).
The asset path from the trigger config (e.g. `state/page.js`).
The asset source, verbatim.
MIME type the console should serve the asset with.
Abort an in-flight router::chat/complete by request_id; reports whether a live request was cancelled.
Stream a chat completion: route {model, provider?} to a provider, relay assistant frames to writer_ref, and return the terminal response.
Stable conversation identity, forwarded separately from the per-turn request id.
The caller's write channel (direction "write"); frames are relayed here.
Provider/transport diagnostics for logs and expandable UI details. `message` remains the stable, user-facing explanation.
Non-streaming convenience over router::chat: run the turn on an internal channel and return the final assistant message + usage.
Stable conversation identity, forwarded separately from the per-turn request id.
Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.
Count prompt tokens for {model, provider?, system_prompt?, tools?, messages} through the resolved provider's provider::<id>::count_tokens surface; never runs the model and costs nothing.
Wire agent messages, the same shape `router::chat` accepts. Must be non-empty.
Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.
Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.
Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.
Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.
Single-variant role tags: exact-match on deserialize, correct wire string on serialize, and they let `AgentMessage` be an untagged union.
Model id the prompt targets; routes exactly like `router::chat` and selects the provider's tokenizer. Optional only when `provider` pins the destination.
Pin an explicit provider, bypassing heuristics (optional).
System prompt counted as part of the request (optional).
Function invocation schemas the turn would carry; their serialized schemas count toward the total (optional).
`provider` when a provider metering API produced the count, `tiktoken` when a local tokenizer estimated it.
Prompt tokens the provider counted for the assembled request.
Batch text embeddings through a provider's provider::<id>::embed surface. Names a provider or discovers the first embed-capable one from the live registry; one vector per input, order preserved.
Texts to embed, one vector returned per input, order preserved.
Embedding model id; the provider's default when omitted.
Provider id (e.g. `openai`); the first embed-capable provider in the live registry when omitted.
One embedding per input, in input order.
Resolve the effective model output budget using the same precedence as router::chat.
Model id to budget.
Optional caller-requested output budget. When absent, the same provider and router defaults used by `router::chat` apply.
Provider id that owns the model. Empty resolves an unambiguous model id.
The capability record (README § Model descriptor).
Set on speech models only; see [`SpeechModel`].
BCP-47 tags of the languages the model handles; empty when the provider does not say.
What a speech model does. Chat models carry no `speech` block.
Realtime input (stt) or streamed audio output (tts) is available.
Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.
Read one catalog model by {provider, id}; null when the model is not registered.
Model id to look up.
Provider id that owns the model.
The capability record (README § Model descriptor).
Set on speech models only; see [`SpeechModel`].
BCP-47 tags of the languages the model handles; empty when the provider does not say.
What a speech model does. Chat models carry no `speech` block.
Realtime input (stt) or streamed audio output (tts) is available.
Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.
List catalog models, optionally filtered by provider, a capability flag, and modality (chat by default; stt, tts, or any).
Keep only models that support this capability flag (optional).
Model family: `chat` (default), `stt`, `tts`, or `any`.
Filter to a single provider id (optional).
Set on speech models only; see [`SpeechModel`].
BCP-47 tags of the languages the model handles; empty when the provider does not say.
What a speech model does. Chat models carry no `speech` block.
Realtime input (stt) or streamed audio output (tts) is available.
Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.
Replace a provider's catalog slice — the only catalog write path (token-gated).
Optional token-gated discovery completion. Failures preserve the slice.
Safe discovery categories. No upstream message, team id or URL is retained.
The full replacement set of models for this provider.
Set on speech models only; see [`SpeechModel`].
BCP-47 tags of the languages the model handles; empty when the provider does not say.
What a speech model does. Chat models carry no `speech` block.
Realtime input (stt) or streamed audio output (tts) is available.
Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.
Provider whose catalog slice is being replaced.
Registration token gating the write (optional).
Check whether a model supports a capability flag (fails open for unknown models).
Capability flag to check (e.g. `structured_output`, `vision`).
Model id to check.
Provider id that owns the model.
Internal: reactively reload the in-memory configuration snapshot and fan out provider model discovery after configuration changes.
Configuration id that changed (advisory).
Internal: a worker's function registrations changed — re-discover live providers and nudge them to re-declare, so a provider that reconnected is resolvable again without waiting for its own catalog timer.
Engine event tag, `"functions_changed"` (advisory).
Internal: a secret changed in the secrets worker — re-resolve the secret:// or env:// credential reference and refresh the providers that use it.
`created`, `rotated`, `deleted` or `access_changed` (advisory).
Secret name (advisory).
The same secret as a `secret://NAME` or `env://NAME` reference; it names the store (advisory).
List registered providers with their configured/available status, where each credential comes from (credential_source, credential_ref, credential_error; never a value), and the provider's starting point (default_model: the newest model of the best-ranked variant its preferences name, else its first declared preference present in the catalog, else the closest same-family model; default_thinking_level).
Provider-specific guidance a console appends to a context-overflow failure on one of this provider's models; copied from the declaration. Absent when the provider declared none.
Environment variable declared by API-key providers. `None` means the provider owns authentication (OAuth, local app login, device flow).
Why the configured credential cannot be used, in words an operator can act on (e.g. the secret is missing or this worker may not read it).
The slice's `secret://NAME` or `env://NAME` reference, when it uses one.
Where the credential comes from: `config`, `env`, `secret` or `none`.
The model a consumer should start with when nothing else names one: the newest model of the best-ranked variant its declared `default_models` name (Terra, Sol, Astra, Luna for GPT providers; Sonnet, Opus, Fable, Haiku for Claude providers), else the first declared id in its catalog slice, else the closest same-family model, else absent (see `registry::availability::resolve_default_model`).
The thinking level a consumer should start with when nothing else names one; copied from the declaration. Absent means omit the level and let the provider apply its own default.
Last completed discovery, independent of configuration and availability.
Safe discovery categories. No upstream message, team id or URL is retained.
A transient failure left a previous catalog in place.
The provider's mark as inline SVG, copied from its declaration. Absent when the provider declared none.
Provider self-declaration at attach time (token-gated upsert); composes the configuration entry schema and reconciles static models.
Guidance a console shows under a context-overflow failure on one of this provider's models, for causes the console cannot fix by compacting (a self-hosted server started with a small window, say). One or two plain sentences, optionally with a link to the provider's docs. `router::provider::list` carries it verbatim.
`true` when the provider works without a credential (a local server started without a key): the router then reports it `configured` with no key. A key that is set is still resolved and sent. Absent means a key is required.
Chat model ids the provider recommends as a starting point. The order in which the list first names each variant sets the variant rank. `router::provider::list` reports the newest catalog model of the best-ranked variant as `default_model` (see `registry::availability::resolve_default_model`).
The thinking level to use with the default model when a caller names none. Absent means omit the level.
The provider's mark: one self-contained `<svg>` document, monochrome, drawn with a square `viewBox` and no fixed size. Consoles paint it as a `currentColor` mask next to the provider's models, so colour and scripts are ignored. Keep it under [`PROVIDER_ICON_SVG_MAX_BYTES`]; the router drops anything larger or not starting with `<svg`.
Set on speech models only; see [`SpeechModel`].
BCP-47 tags of the languages the model handles; empty when the provider does not say.
What a speech model does. Chat models carry no `speech` block.
Realtime input (stt) or streamed audio output (tts) is available.
Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.
Registration token proving ownership on re-register (omit on first declare).
Resolve a provider's effective credential + api_url + max_tokens (token-gated); a secret://NAME or env://NAME api_key resolves through the secrets worker.
Begin a latest-wins discovery attempt. Older providers omit this.
Provider id to resolve credentials/config for.
Registration token gating the resolve (optional).
Why the configured credential cannot be used, in words an operator can act on (e.g. the secret is missing or this worker may not read it).
The slice's `secret://NAME` or `env://NAME` reference, when it uses one.
Where the credential comes from: `config`, `env`, `secret` or `none`.
Opaque lease returned only to a token-authenticated discovery caller.
Where the resolved credential came from, as providers read it. A resolved `secret://` or `env://` reference reports `config` (the reference lives in the configuration entry); `credential_source` tells them apart.
OAuth write-back: store a provider credential in the configuration entry under the entry write lock (token-gated).
The credential object to store (provider-specific shape).
Provider id whose credential slice is being written.
Registration token gating the write (optional).
Read-only routing preview: resolve {model, provider?} to the chosen provider + ordered candidates without streaming.
Model id to route.
Pin an explicit provider, bypassing heuristics (optional).
Text to speech through provider::<id>::speak: {model?, provider?, text, voice?, format?} to {provider, model, audio_base64, mime}. Resolves the provider from the catalog's tts models when none is named.
Audio container wanted: `mp3` (default), `wav`, `pcm16`, `opus`. Providers answer with what they can and say so in `mime`.
BCP-47 language hint for multilingual voices.
Text-to-speech model id (a `tts` model from `router::models::list`); the provider's default when omitted.
Provider id; resolved from the model's catalog owner when omitted.
Speaking-rate multiplier; 1.0 is the voice's own pace.
Text to speak.
Voice id or name as the provider knows it; the provider's default when omitted.
The audio, base64.
MIME type of the audio, e.g. `audio/mpeg`.
Voice the provider used.
Speech to text through provider::<id>::transcribe: {model?, provider?, audio_base64, mime?, language?} to {provider, model, text, segments?}. Resolves the provider from the catalog's stt models when none is named.
The audio, base64. WAV unless `mime` says otherwise.
BCP-47 language hint; the model detects the language when omitted.
MIME type of the audio: `audio/wav` (default), `audio/mpeg`, `audio/webm`, `audio/ogg`, `audio/flac`.
Speech-to-text model id (an `stt` model from `router::models::list`); the provider's default when omitted.
Vocabulary or context hint for the recognizer, when the provider takes one.
Provider id; resolved from the model's catalog owner when omitted.
Language the provider detected or was told (BCP-47).
Timed spans when the provider produces them.
The whole transcript.
A provider reconciled its catalog slice. Payload: { provider, count }.
The provider registry changed (declare / availability flip). Payload: { provider, op }.
The router finished booting; providers bind here and re-declare after a restart.
Find workers, kits and executables, or jump to a page.