Skip to content
Sign in

llm-router

v1.5.4

One front door + provider protocol in front of every LLM provider.

license: Apache-2.0Used in 1 kitllmmodelsprovidersrouter

Install

bash
iii trigger compose::add worker=llm-router@1.5.4

Run in your project with the iii engine running. Exact versions are immutable and digest-pinned.

Functions24

llm-router::ui-content

Function

Serve the llm-router worker's injected console UI assets (the target of its ade::ui::asset observers; the console calls it through each asset's observer with {path}).

Request
  • pathstringrequired

    The asset path from the observer's args (e.g. `state/page.js`).

Response
  • contentstringrequired

    The asset source, verbatim.

  • content_typestringrequired

    MIME type the console should serve the asset with.

router::abort

Function

Abort an in-flight router::chat/complete by request_id; reports whether a live request was cancelled.

Request
  • request_idstringrequired
Response
  • abortedbooleanrequired

router::chat

Function

Stream a chat completion: route {model, provider?} to a provider, relay assistant frames to writer_ref, and return the terminal response.

Request
  • cache_intentany of
    any of (2)
    Variant 1
    • surface_digeststringrequired
    Variant 2
    valuenull
  • max_output_tokensinteger· uint64min 0
  • messagesunknownrequired
  • metadataunknown
  • modelstringrequired
  • providerstring
  • provider_optionsunknown
  • request_idstring
  • response_formatunknown
  • retry_maxinteger· uint32min 0

    Maximum retries after the first provider attempt. Omitted uses the router's configured limit; zero disables retries. Can only reduce that limit.

  • session_idstring

    Stable conversation identity, forwarded separately from the per-turn request id.

  • system_promptstring
  • system_sectionsobject[]
    • cache_boundaryboolean
    • textstringrequired
  • thinking_levelunknown
  • toolsunknown
  • writer_refall ofrequired

    The caller's write channel (direction "write"); frames are relayed here.

    all of (1)
    Variant 1
    • access_keystringrequired
    • channel_idstringrequired
    • directionstringrequiredenum read, write
Response
  • errorany of
    any of (2)
    Variant 1
    • codestringrequired
    • detailstring

      Provider/transport diagnostics for logs and expandable UI details. `message` remains the stable, user-facing explanation.

    • kindany of
      Show 2 nested
      any of (2)
      Variant 1
      valuestringenum auth_expired, rate_limited, context_overflow, transient, permanent
      Variant 2
      valuenull
    • messagestringrequired
    • retryableboolean
    Variant 2
    valuenull
  • modelstringrequired
  • okbooleanrequired
  • providerstringrequired
  • stop_reasonany of
    any of (2)
    Variant 1
    valuestringenum end, length, function_call, aborted, error
    Variant 2
    valuenull
  • usageany of
    any of (2)
    Variant 1
    • cache_readinteger· uint64min 0
    • cache_writeinteger· uint64min 0
    • cost_usdnumber· double
    • inputinteger· uint64min 0
    • outputinteger· uint64min 0
    • reasoninginteger· uint64min 0
    Variant 2
    valuenull

router::complete

Function

Non-streaming convenience over router::chat: run the turn on an internal channel and return the final assistant message + usage.

Request
  • cache_intentany of
    any of (2)
    Variant 1
    • surface_digeststringrequired
    Variant 2
    valuenull
  • max_output_tokensinteger· uint64min 0
  • messagesunknownrequired
  • metadataunknown
  • modelstringrequired
  • providerstring
  • provider_optionsunknown
  • request_idstring
  • response_formatunknown
  • retry_maxinteger· uint32min 0

    Maximum retries after the first provider attempt. Omitted uses the router's configured limit; zero disables retries. Can only reduce that limit.

  • session_idstring

    Stable conversation identity, forwarded separately from the per-turn request id.

  • system_promptstring
  • system_sectionsobject[]
    • cache_boundaryboolean
    • textstringrequired
  • thinking_levelunknown
  • toolsunknown
Response
  • messageobjectrequired
    • contentone of[]required
      one of (7)
      Variant 1
      • textstringrequired
      • typestringrequiredenum text
      Variant 2
      • attachment_idstring

        Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.

      • datastringrequired
      • mimestringrequired
      • typestringrequiredenum image
      Variant 3
      • attachment_idstringrequired
      • mimestringrequired
      • namestringrequired
      • sizeinteger· uint64requiredmin 0
      • typestringrequiredenum file
      Variant 4
      • signaturestring
      • textstringrequired
      • typestringrequiredenum thinking
      Variant 5
      • datastringrequired
      • typestringrequiredenum redacted_thinking
      Variant 6
      • argumentsunknownrequired
      • function_idstringrequired
      • idstringrequired
      • typestringrequiredenum function_call
      Variant 7
      • contentunknown[]required
      • function_call_idstringrequired
      • is_errorboolean
      • typestringrequiredenum function_result
    • error_kindany of
      any of (2)
      Variant 1
      valuestringenum auth_expired, rate_limited, context_overflow, transient, permanent
      Variant 2
      valuenull
    • error_messagestring
    • modelstringrequired
    • native_stop_reasonstring
    • providerstringrequired
    • rolestringrequiredenum assistant
    • stop_reasonstringrequiredenum end, length, function_call, aborted, error
    • timestampinteger· int64required
    • usageany of
      any of (2)
      Variant 1
      • cache_readinteger· uint64min 0
      • cache_writeinteger· uint64min 0
      • cost_usdnumber· double
      • inputinteger· uint64min 0
      • outputinteger· uint64min 0
      • reasoninginteger· uint64min 0
      Variant 2
      valuenull
    • warningsstring[]
  • modelstringrequired
  • providerstringrequired
  • usageany of
    any of (2)
    Variant 1
    • cache_readinteger· uint64min 0
    • cache_writeinteger· uint64min 0
    • cost_usdnumber· double
    • inputinteger· uint64min 0
    • outputinteger· uint64min 0
    • reasoninginteger· uint64min 0
    Variant 2
    valuenull

router::count_tokens

Function

Count prompt tokens for {model, provider?, system_prompt?, tools?, messages, thinking_level?, response_format?} through the resolved provider's provider::<id>::count_tokens surface; never runs the model and costs nothing.

Request
  • messagesany of[]required

    Wire agent messages, the same shape `router::chat` accepts. Must be non-empty.

    any of (4)
    Variant 1
    • contentone of[]required
      Show 7 nested
      one of (7)
      Variant 1
      • textstringrequired
      • typestringrequiredenum text
      Variant 2
      • attachment_idstring

        Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.

      • datastringrequired
      • mimestringrequired
      • typestringrequiredenum image
      Variant 3
      • attachment_idstringrequired
      • mimestringrequired
      • namestringrequired
      • sizeinteger· uint64requiredmin 0
      • typestringrequiredenum file
      Variant 4
      • signaturestring
      • textstringrequired
      • typestringrequiredenum thinking
      Variant 5
      • datastringrequired
      • typestringrequiredenum redacted_thinking
      Variant 6
      • argumentsunknownrequired
      • function_idstringrequired
      • idstringrequired
      • typestringrequiredenum function_call
      Variant 7
      • contentunknown[]required
      • function_call_idstringrequired
      • is_errorboolean
      • typestringrequiredenum function_result
    • error_kindany of
      Show 2 nested
      any of (2)
      Variant 1
      valuestringenum auth_expired, rate_limited, context_overflow, transient, permanent
      Variant 2
      valuenull
    • error_messagestring
    • modelstringrequired
    • native_stop_reasonstring
    • providerstringrequired
    • rolestringrequiredenum assistant
    • stop_reasonstringrequiredenum end, length, function_call, aborted, error
    • timestampinteger· int64required
    • usageany of
      Show 2 nested
      any of (2)
      Variant 1
      • cache_readinteger· uint64min 0
      • cache_writeinteger· uint64min 0
      • cost_usdnumber· double
      • inputinteger· uint64min 0
      • outputinteger· uint64min 0
      • reasoninginteger· uint64min 0
      Variant 2
      valuenull
    • warningsstring[]
    Variant 2
    • contentone of[]required
      Show 7 nested
      one of (7)
      Variant 1
      • textstringrequired
      • typestringrequiredenum text
      Variant 2
      • attachment_idstring

        Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.

      • datastringrequired
      • mimestringrequired
      • typestringrequiredenum image
      Variant 3
      • attachment_idstringrequired
      • mimestringrequired
      • namestringrequired
      • sizeinteger· uint64requiredmin 0
      • typestringrequiredenum file
      Variant 4
      • signaturestring
      • textstringrequired
      • typestringrequiredenum thinking
      Variant 5
      • datastringrequired
      • typestringrequiredenum redacted_thinking
      Variant 6
      • argumentsunknownrequired
      • function_idstringrequired
      • idstringrequired
      • typestringrequiredenum function_call
      Variant 7
      • contentunknown[]required
      • function_call_idstringrequired
      • is_errorboolean
      • typestringrequiredenum function_result
    • detailsunknownrequired
    • function_call_idstringrequired
    • function_idstringrequired
    • is_errorbooleanrequired
    • rolestringrequiredenum function_result
    • timestampinteger· int64required
    Variant 3
    • contentone of[]required
      Show 7 nested
      one of (7)
      Variant 1
      • textstringrequired
      • typestringrequiredenum text
      Variant 2
      • attachment_idstring

        Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.

      • datastringrequired
      • mimestringrequired
      • typestringrequiredenum image
      Variant 3
      • attachment_idstringrequired
      • mimestringrequired
      • namestringrequired
      • sizeinteger· uint64requiredmin 0
      • typestringrequiredenum file
      Variant 4
      • signaturestring
      • textstringrequired
      • typestringrequiredenum thinking
      Variant 5
      • datastringrequired
      • typestringrequiredenum redacted_thinking
      Variant 6
      • argumentsunknownrequired
      • function_idstringrequired
      • idstringrequired
      • typestringrequiredenum function_call
      Variant 7
      • contentunknown[]required
      • function_call_idstringrequired
      • is_errorboolean
      • typestringrequiredenum function_result
    • custom_typestringrequired
    • detailsunknown
    • displaystring
    • rolestringrequiredenum custom
    • timestampinteger· int64required
    Variant 4
    • contentone of[]required
      Show 7 nested
      one of (7)
      Variant 1
      • textstringrequired
      • typestringrequiredenum text
      Variant 2
      • attachment_idstring

        Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.

      • datastringrequired
      • mimestringrequired
      • typestringrequiredenum image
      Variant 3
      • attachment_idstringrequired
      • mimestringrequired
      • namestringrequired
      • sizeinteger· uint64requiredmin 0
      • typestringrequiredenum file
      Variant 4
      • signaturestring
      • textstringrequired
      • typestringrequiredenum thinking
      Variant 5
      • datastringrequired
      • typestringrequiredenum redacted_thinking
      Variant 6
      • argumentsunknownrequired
      • function_idstringrequired
      • idstringrequired
      • typestringrequiredenum function_call
      Variant 7
      • contentunknown[]required
      • function_call_idstringrequired
      • is_errorboolean
      • typestringrequiredenum function_result
    • rolestringrequiredenum user

      Single-variant role tags: exact-match on deserialize, correct wire string on serialize, and they let `AgentMessage` be an untagged union.

    • timestampinteger· int64required
  • modelstring

    Model id the prompt targets; routes exactly like `router::chat` and selects the provider's tokenizer. Optional only when `provider` pins the destination.

  • providerstring

    Pin an explicit provider, bypassing heuristics (optional).

  • response_formatany of

    The turn's `router::chat` response format (optional), for providers that render it into the prompt they bill.

    any of (2)
    Variant 1
    • schemaunknown
    • typestringrequired
    Variant 2
    valuenull
  • system_promptstring

    System prompt counted as part of the request (optional).

  • thinking_levelany of

    The turn's `router::chat` thinking level (optional). Providers whose prompt template depends on it (a reasoning-effort line, a chat vs thinking transition) count the prompt that level renders.

    any of (2)
    Variant 1
    valuestringenum off, minimal, low, medium, high, xhigh
    Variant 2
    valuenull
  • toolsobject[]

    Function invocation schemas the turn would carry; their serialized schemas count toward the total (optional).

    • descriptionstringrequired
    • execution_modestring
    • labelstring
    • namestringrequired
    • parametersunknownrequired
Response
  • estimatorstringrequired

    `provider` when a provider metering API produced the count, `tiktoken` when a local tokenizer estimated it.

  • modelstringrequired
  • providerstringrequired
  • tokensinteger· uint64requiredmin 0

    Prompt tokens the provider counted for the assembled request.

router::embed

Function

Batch text embeddings through a provider's provider::<id>::embed surface. Names a provider or discovers the first embed-capable one from the live registry, then from embed-only workers; one vector per input, order preserved.

Request
  • inputstring[]required

    Texts to embed, one vector returned per input, order preserved.

  • modelstring

    Embedding model id; the provider's default when omitted.

  • providerstring

    Provider id (e.g. `openai`); the first embed-capable provider in the live registry when omitted.

Response
  • embeddingsnumber[][]required

    One embedding per input, in input order.

  • modelstringrequired
  • providerstringrequired

router::models::budget

Function

Resolve the effective model output budget using the same precedence as router::chat.

Request
  • idstring

    Model id to budget.

  • max_output_tokensinteger· uint64min 0

    Optional caller-requested output budget. When absent, the same provider and router defaults used by `router::chat` apply.

  • providerstring

    Provider id that owns the model. Empty resolves an unambiguous model id.

Response
any of (2)
Variant 1
  • effective_max_output_tokensinteger· uint64requiredmin 0
  • modelobjectrequired

    The capability record (README § Model descriptor).

    • context_windowinteger· uint64requiredmin 0
    • display_namestring
    • idstringrequired
    • input_limitinteger· uint64min 0
    • max_output_tokensinteger· uint64requiredmin 0
    • pricingany of
      Show 2 nested
      any of (2)
      Variant 1
      • cache_readnumber· double
      • cache_writenumber· double
      • inputnumber· double
      • outputnumber· double
      Variant 2
      valuenull
    • providerstringrequired
    • reasoning_effortsobject[]
      Show 2 nested
      • descriptionstring
      • effortstringrequired
    • speechany of

      Set on speech models only; see [`SpeechModel`].

      Show 2 nested
      any of (2)
      Variant 1
      • languagesstring[]

        BCP-47 tags of the languages the model handles; empty when the provider does not say.

      • modalityone ofrequired

        What a speech model does. Chat models carry no `speech` block.

        Show 2 nested
        one of (2)
        Variant 1
        valuestringenum stt
        Variant 2
        valuestringenum tts
      • streamingboolean

        Realtime input (stt) or streamed audio output (tts) is available.

      Variant 2
      valuenull
    • supports_cacheboolean
    • supports_structured_outputboolean
    • supports_thinkingboolean
    • supports_thinking_offboolean

      Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.

    • supports_toolsboolean
    • supports_visionboolean
    • supports_xhighboolean
    • thinking_budgetsobject
Variant 2
valuenull

router::models::get

Function

Read one catalog model by {provider, id}; null when the model is not registered.

Request
  • idstring

    Model id to look up.

  • providerstring

    Provider id that owns the model.

Response
any of (2)
Variant 1
  • modelobjectrequired

    The capability record (README § Model descriptor).

    • context_windowinteger· uint64requiredmin 0
    • display_namestring
    • idstringrequired
    • input_limitinteger· uint64min 0
    • max_output_tokensinteger· uint64requiredmin 0
    • pricingany of
      Show 2 nested
      any of (2)
      Variant 1
      • cache_readnumber· double
      • cache_writenumber· double
      • inputnumber· double
      • outputnumber· double
      Variant 2
      valuenull
    • providerstringrequired
    • reasoning_effortsobject[]
      Show 2 nested
      • descriptionstring
      • effortstringrequired
    • speechany of

      Set on speech models only; see [`SpeechModel`].

      Show 2 nested
      any of (2)
      Variant 1
      • languagesstring[]

        BCP-47 tags of the languages the model handles; empty when the provider does not say.

      • modalityone ofrequired

        What a speech model does. Chat models carry no `speech` block.

        Show 2 nested
        one of (2)
        Variant 1
        valuestringenum stt
        Variant 2
        valuestringenum tts
      • streamingboolean

        Realtime input (stt) or streamed audio output (tts) is available.

      Variant 2
      valuenull
    • supports_cacheboolean
    • supports_structured_outputboolean
    • supports_thinkingboolean
    • supports_thinking_offboolean

      Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.

    • supports_toolsboolean
    • supports_visionboolean
    • supports_xhighboolean
    • thinking_budgetsobject
Variant 2
valuenull

router::models::list

Function

List catalog models, optionally filtered by provider, a capability flag, and modality (chat by default; stt, tts, or any).

Request
  • capabilitystring

    Keep only models that support this capability flag (optional).

  • modalityany of

    Model family: `chat` (default), `stt`, `tts`, or `any`.

    any of (2)
    Variant 1
    valuestringenum chat, stt, tts, any
    Variant 2
    valuenull
  • providerstring

    Filter to a single provider id (optional).

Response
  • modelsobject[]required
    • context_windowinteger· uint64requiredmin 0
    • display_namestring
    • idstringrequired
    • input_limitinteger· uint64min 0
    • max_output_tokensinteger· uint64requiredmin 0
    • pricingany of
      any of (2)
      Variant 1
      • cache_readnumber· double
      • cache_writenumber· double
      • inputnumber· double
      • outputnumber· double
      Variant 2
      valuenull
    • providerstringrequired
    • reasoning_effortsobject[]
      • descriptionstring
      • effortstringrequired
    • speechany of

      Set on speech models only; see [`SpeechModel`].

      any of (2)
      Variant 1
      • languagesstring[]

        BCP-47 tags of the languages the model handles; empty when the provider does not say.

      • modalityone ofrequired

        What a speech model does. Chat models carry no `speech` block.

        Show 2 nested
        one of (2)
        Variant 1
        valuestringenum stt
        Variant 2
        valuestringenum tts
      • streamingboolean

        Realtime input (stt) or streamed audio output (tts) is available.

      Variant 2
      valuenull
    • supports_cacheboolean
    • supports_structured_outputboolean
    • supports_thinkingboolean
    • supports_thinking_offboolean

      Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.

    • supports_toolsboolean
    • supports_visionboolean
    • supports_xhighboolean
    • thinking_budgetsobject

router::models::on-changed

Function

A provider reconciled its catalog slice. Event: { provider, count }. Takes no args; filter on input.provider.

Request
  • countinteger· uintrequiredmin 0

    How many models the slice holds.

  • providerstringrequired

    The provider whose slice changed.

Response
  • countinteger· uintrequiredmin 0

    How many models the slice holds.

  • providerstringrequired

    The provider whose slice changed.

router::models::reconcile

Function

Replace a provider's catalog slice — the only catalog write path (token-gated).

Request
  • discoveryany of

    Optional token-gated discovery completion. Failures preserve the slice.

    any of (2)
    Variant 1
    • attemptstringrequired
    • codeany of
      Show 2 nested
      any of (2)
      Variant 1
      valuestringenum permission-denied, invalid-api-key, insufficient-credits, spending-limit-exceeded, rate-limit-exceeded
      Variant 2
      valuenull
    • http_statusinteger· uint16min 0
    • outcomestringrequiredenum success, empty, not_configured, billing, authentication, permission, …

      Safe discovery categories. No upstream message, team id or URL is retained.

    Variant 2
    valuenull
  • modelsobject[]

    The full replacement set of models for this provider.

    • context_windowinteger· uint64requiredmin 0
    • display_namestring
    • idstringrequired
    • input_limitinteger· uint64min 0
    • max_output_tokensinteger· uint64requiredmin 0
    • pricingany of
      any of (2)
      Variant 1
      • cache_readnumber· double
      • cache_writenumber· double
      • inputnumber· double
      • outputnumber· double
      Variant 2
      valuenull
    • providerstringrequired
    • reasoning_effortsobject[]
      • descriptionstring
      • effortstringrequired
    • speechany of

      Set on speech models only; see [`SpeechModel`].

      any of (2)
      Variant 1
      • languagesstring[]

        BCP-47 tags of the languages the model handles; empty when the provider does not say.

      • modalityone ofrequired

        What a speech model does. Chat models carry no `speech` block.

        Show 2 nested
        one of (2)
        Variant 1
        valuestringenum stt
        Variant 2
        valuestringenum tts
      • streamingboolean

        Realtime input (stt) or streamed audio output (tts) is available.

      Variant 2
      valuenull
    • supports_cacheboolean
    • supports_structured_outputboolean
    • supports_thinkingboolean
    • supports_thinking_offboolean

      Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.

    • supports_toolsboolean
    • supports_visionboolean
    • supports_xhighboolean
    • thinking_budgetsobject
  • providerstring

    Provider whose catalog slice is being replaced.

  • tokenstring

    Registration token gating the write (optional).

Response
  • countinteger· uintrequiredmin 0
  • providerstringrequired

router::models::supports

Function

Check whether a model supports a capability flag (fails open for unknown models).

Request
  • capabilitystring

    Capability flag to check (e.g. `structured_output`, `vision`).

  • idstring

    Model id to check.

  • providerstring

    Provider id that owns the model.

Response
  • supportedbooleanrequired

router::on-ready

Function

The router is ready: each observer receives the event once the router has booted and the observer is accepted, once per router start, and once more when its worker reconnects within the observer grace period. Providers observe it to re-declare. Takes no args; the event is {}.

Request
Empty object
Response
Empty object

router::on_config_changed

Function

Internal: reactively reload the in-memory configuration snapshot and fan out provider model discovery after configuration changes.

Request
  • idstring

    Configuration id that changed (advisory).

Response
  • okbooleanrequired

router::on_functions_changed

Function

Internal: a worker's function registrations changed — re-discover live providers and nudge them to re-declare, so a provider that reconnected is resolvable again without waiting for its own catalog timer.

Request
  • eventstring

    Engine event tag, `"functions_changed"` (advisory).

Response
  • okbooleanrequired

router::on_secret_changed

Function

Internal: a secret changed in the secrets worker — re-resolve the secret:// or env:// credential reference and refresh the providers that use it.

Request
  • outputany of

    The `secrets::on-changed` event (the observed call's output).

    any of (2)
    Variant 1
    • actionstring

      `created`, `rotated`, `deleted` or `access_changed` (advisory).

    • namestring

      Secret name (advisory).

    • refstring

      The same secret as a `secret://NAME` or `env://NAME` reference; it names the store (advisory).

    Variant 2
    valuenull
Response
  • okbooleanrequired

router::provider::list

Function

List registered providers with their configured/available status, where each credential comes from (credential_source, credential_ref, credential_error; never a value), and the provider's starting point (default_model: the newest model of the best-ranked variant its preferences name, else its first declared preference present in the catalog, else the closest same-family model; default_thinking_level).

Request
Empty object
Response
  • providersobject[]required
    • availablebooleanrequired
    • configuredbooleanrequired
    • context_overflow_hintstring

      Provider-specific guidance a console appends to a context-overflow failure on one of this provider's models; copied from the declaration. Absent when the provider declared none.

    • credential_env_varstring

      Environment variable declared by API-key providers. `None` means the provider owns authentication (OAuth, local app login, device flow).

    • credential_errorstring

      Why the configured credential cannot be used, in words an operator can act on (e.g. the secret is missing or this worker may not read it).

    • credential_refstring

      The slice's `secret://NAME` or `env://NAME` reference, when it uses one.

    • credential_sourceany of

      Where the credential comes from: `config`, `env`, `secret` or `none`.

      any of (2)
      Variant 1
      valuestringenum config, env, secret, none
      Variant 2
      valuenull
    • default_modelstring

      The model a consumer should start with when nothing else names one: the newest model of the best-ranked variant its declared `default_models` name (Terra, Sol, Astra, Luna for GPT providers; Sonnet, Opus, Fable, Haiku for Claude providers), else the first declared id in its catalog slice, else the closest same-family model, else absent (see `registry::availability::resolve_default_model`).

    • default_thinking_levelany of

      The thinking level a consumer should start with when nothing else names one; copied from the declaration. Absent means omit the level and let the provider apply its own default.

      any of (2)
      Variant 1
      valuestringenum off, minimal, low, medium, high, xhigh
      Variant 2
      valuenull
    • discoveryany of

      Last completed discovery, independent of configuration and availability.

      any of (2)
      Variant 1
      • checked_at_msinteger· int64required
      • codeany of
        Show 2 nested
        any of (2)
        Variant 1
        valuestringenum permission-denied, invalid-api-key, insufficient-credits, spending-limit-exceeded, rate-limit-exceeded
        Variant 2
        valuenull
      • http_statusinteger· uint16min 0
      • outcomestringrequiredenum success, empty, not_configured, billing, authentication, permission, …

        Safe discovery categories. No upstream message, team id or URL is retained.

      • stalebooleanrequired

        A transient failure left a previous catalog in place.

      Variant 2
      valuenull
    • display_namestringrequired
    • icon_svgstring

      The provider's mark as inline SVG, copied from its declaration. Absent when the provider declared none.

    • idstringrequired
    • supports_model_listingbooleanrequired

router::provider::on-changed

Function

The provider registry changed: a declaration (op register), a discovery result (discovery) or an availability flip (available, unavailable). Event: { provider, op }. Takes no args; filter on input.provider and input.op.

Request
  • opstringrequired

    `register`, `discovery`, `available` or `unavailable`.

  • providerstringrequired

    The provider the change is about.

Response
  • opstringrequired

    `register`, `discovery`, `available` or `unavailable`.

  • providerstringrequired

    The provider the change is about.

router::provider::register

Function

Provider self-declaration at attach time (token-gated upsert); composes the configuration entry schema and reconciles static models.

Request
  • config_schemaunknown
  • context_overflow_hintstring

    Guidance a console shows under a context-overflow failure on one of this provider's models, for causes the console cannot fix by compacting (a self-hosted server started with a small window, say). One or two plain sentences, optionally with a link to the provider's docs. `router::provider::list` carries it verbatim.

  • credential_env_varstring
  • credential_optionalboolean

    `true` when the provider works without a credential (a local server started without a key): the router then reports it `configured` with no key. A key that is set is still resolved and sent. Absent means a key is required.

  • default_modelsstring[]

    Chat model ids the provider recommends as a starting point. The order in which the list first names each variant sets the variant rank. `router::provider::list` reports the newest catalog model of the best-ranked variant as `default_model` (see `registry::availability::resolve_default_model`).

  • default_thinking_levelany of

    The thinking level to use with the default model when a caller names none. Absent means omit the level.

    any of (2)
    Variant 1
    valuestringenum off, minimal, low, medium, high, xhigh
    Variant 2
    valuenull
  • defaultsany of
    any of (2)
    Variant 1
    • api_urlstring
    • max_tokensinteger· uint64min 0
    Variant 2
    valuenull
  • display_namestring
  • icon_svgstring

    The provider's mark: one self-contained `<svg>` document, monochrome, drawn with a square `viewBox` and no fixed size. Consoles paint it as a `currentColor` mask next to the provider's models, so colour and scripts are ignored. Keep it under [`PROVIDER_ICON_SVG_MAX_BYTES`]; the router drops anything larger or not starting with `<svg`.

  • idstringrequired
  • modelsobject[]
    • context_windowinteger· uint64requiredmin 0
    • display_namestring
    • idstringrequired
    • input_limitinteger· uint64min 0
    • max_output_tokensinteger· uint64requiredmin 0
    • pricingany of
      any of (2)
      Variant 1
      • cache_readnumber· double
      • cache_writenumber· double
      • inputnumber· double
      • outputnumber· double
      Variant 2
      valuenull
    • providerstringrequired
    • reasoning_effortsobject[]
      • descriptionstring
      • effortstringrequired
    • speechany of

      Set on speech models only; see [`SpeechModel`].

      any of (2)
      Variant 1
      • languagesstring[]

        BCP-47 tags of the languages the model handles; empty when the provider does not say.

      • modalityone ofrequired

        What a speech model does. Chat models carry no `speech` block.

        Show 2 nested
        one of (2)
        Variant 1
        valuestringenum stt
        Variant 2
        valuestringenum tts
      • streamingboolean

        Realtime input (stt) or streamed audio output (tts) is available.

      Variant 2
      valuenull
    • supports_cacheboolean
    • supports_structured_outputboolean
    • supports_thinkingboolean
    • supports_thinking_offboolean

      Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.

    • supports_toolsboolean
    • supports_visionboolean
    • supports_xhighboolean
    • thinking_budgetsobject
  • supports_model_listingboolean
  • tokenstring

    Registration token proving ownership on re-register (omit on first declare).

  • worker_idstring
Response
  • idstringrequired
  • okbooleanrequired
  • registration_tokenstringrequired

router::provider::resolve

Function

Resolve a provider's effective credential + api_url + max_tokens (token-gated); a secret://NAME or env://NAME api_key resolves through the secrets worker.

Request
  • begin_discoveryboolean

    Begin a latest-wins discovery attempt. Older providers omit this.

  • idstring

    Provider id to resolve credentials/config for.

  • tokenstring

    Registration token gating the resolve (optional).

Response
  • api_urlstring
  • configuredbooleanrequired
  • credentialany of
    any of (2)
    Variant 1
    one of (2)
    Variant 1
    • keystringrequired
    • typestringrequiredenum api_key
    Variant 2
    • access_tokenstringrequired
    • expires_atinteger· int64
    • provider_extraunknown
    • refresh_tokenstring
    • scopesstring[]
    • typestringrequiredenum oauth
    Variant 2
    valuenull
  • credential_errorstring

    Why the configured credential cannot be used, in words an operator can act on (e.g. the secret is missing or this worker may not read it).

  • credential_refstring

    The slice's `secret://NAME` or `env://NAME` reference, when it uses one.

  • credential_sourceany of

    Where the credential comes from: `config`, `env`, `secret` or `none`.

    any of (2)
    Variant 1
    valuestringenum config, env, secret, none
    Variant 2
    valuenull
  • discovery_attemptstring

    Opaque lease returned only to a token-authenticated discovery caller.

  • max_tokensinteger· uint64min 0
  • sourcestringrequiredenum config, env, none

    Where the resolved credential came from, as providers read it. A resolved `secret://` or `env://` reference reports `config` (the reference lives in the configuration entry); `credential_source` tells them apart.

router::provider::update_credential

Function

OAuth write-back: store a provider credential in the configuration entry under the entry write lock (token-gated).

Request
  • credentialunknown

    The credential object to store (provider-specific shape).

  • idstring

    Provider id whose credential slice is being written.

  • tokenstring

    Registration token gating the write (optional).

Response
  • okbooleanrequired

router::route

Function

Read-only routing preview: resolve {model, provider?} to the chosen provider + ordered candidates without streaming.

Request
  • modelstring

    Model id to route.

  • providerstring

    Pin an explicit provider, bypassing heuristics (optional).

Response
  • candidatesstring[]required
  • providerstringrequired

router::speak

Function

Text to speech through provider::<id>::speak: {model?, provider?, text, voice?, format?} to {provider, model, audio_base64, mime}. Resolves the provider from the catalog's tts models when none is named.

Request
  • formatstring

    Audio container wanted: `mp3` (default), `wav`, `pcm16`, `opus`. Providers answer with what they can and say so in `mime`.

  • languagestring

    BCP-47 language hint for multilingual voices.

  • modelstring

    Text-to-speech model id (a `tts` model from `router::models::list`); the provider's default when omitted.

  • providerstring

    Provider id; resolved from the model's catalog owner when omitted.

  • speednumber· double

    Speaking-rate multiplier; 1.0 is the voice's own pace.

  • textstringrequired

    Text to speak.

  • voicestring

    Voice id or name as the provider knows it; the provider's default when omitted.

Response
  • audio_base64stringrequired

    The audio, base64.

  • duration_secsnumber· double
  • mimestringrequired

    MIME type of the audio, e.g. `audio/mpeg`.

  • modelstringrequired
  • providerstringrequired
  • voicestring

    Voice the provider used.

router::transcribe

Function

Speech to text through provider::<id>::transcribe: {model?, provider?, audio_base64, mime?, language?} to {provider, model, text, segments?}. Resolves the provider from the catalog's stt models when none is named.

Request
  • audio_base64stringrequired

    The audio, base64. WAV unless `mime` says otherwise.

  • languagestring

    BCP-47 language hint; the model detects the language when omitted.

  • mimestring

    MIME type of the audio: `audio/wav` (default), `audio/mpeg`, `audio/webm`, `audio/ogg`, `audio/flac`.

  • modelstring

    Speech-to-text model id (an `stt` model from `router::models::list`); the provider's default when omitted.

  • promptstring

    Vocabulary or context hint for the recognizer, when the provider takes one.

  • providerstring

    Provider id; resolved from the model's catalog owner when omitted.

Response
  • duration_secsnumber· double
  • languagestring

    Language the provider detected or was told (BCP-47).

  • modelstringrequired
  • providerstringrequired
  • segmentsobject[]

    Timed spans when the provider produces them.

    • end_secsnumber· double
    • start_secsnumber· double
    • textstringrequired
  • textstringrequired

    The whole transcript.

Search the registry

Find workers, kits and executables, or jump to a page.