Skip to content
Registry

llm-router

v1.3.2

One front door + provider protocol in front of every LLM provider.

llmmodelsprovidersrouter

Install

bash
iii trigger compose::add worker=llm-router@1.3.2

Run in your project with the iii engine running. Exact versions are immutable and digest-pinned.

Functions16

router::abort

Function

Abort an in-flight router::chat/complete by request_id; reports whether a live request was cancelled.

Request
  • request_idstringrequired
Response
  • abortedbooleanrequired

router::chat

Function

Stream a chat completion: route {model, provider?} to a provider, relay assistant frames to writer_ref, and return the terminal response.

Request
  • max_output_tokensinteger· uint64min 0
  • messagesunknownrequired
  • metadataunknown
  • modelstringrequired
  • providerstring
  • provider_optionsunknown
  • request_idstring
  • response_formatunknown
  • system_promptstring
  • thinking_levelunknown
  • toolsunknown
  • writer_refall ofrequired

    The caller's write channel (direction "write"); frames are relayed here.

    all of (1)
    Variant 1
    • access_keystringrequired
    • channel_idstringrequired
    • directionstringrequiredenum read, write
Response
  • errorany of
    any of (2)
    Variant 1
    • codestringrequired
    • messagestringrequired
    Variant 2
    valuenull
  • modelstringrequired
  • okbooleanrequired
  • providerstringrequired
  • stop_reasonany of
    any of (2)
    Variant 1
    valuestringenum end, length, function_call, aborted, error
    Variant 2
    valuenull
  • usageany of
    any of (2)
    Variant 1
    • cache_readinteger· uint64min 0
    • cache_writeinteger· uint64min 0
    • cost_usdnumber· double
    • inputinteger· uint64min 0
    • outputinteger· uint64min 0
    • reasoninginteger· uint64min 0
    Variant 2
    valuenull

router::complete

Function

Non-streaming convenience over router::chat: run the turn on an internal channel and return the final assistant message + usage.

Request
  • max_output_tokensinteger· uint64min 0
  • messagesunknownrequired
  • metadataunknown
  • modelstringrequired
  • providerstring
  • provider_optionsunknown
  • request_idstring
  • response_formatunknown
  • system_promptstring
  • thinking_levelunknown
  • toolsunknown
Response
  • messageobjectrequired
    • contentone of[]required
      one of (6)
      Variant 1
      • textstringrequired
      • typestringrequiredenum text
      Variant 2
      • datastringrequired
      • mimestringrequired
      • typestringrequiredenum image
      Variant 3
      • signaturestring
      • textstringrequired
      • typestringrequiredenum thinking
      Variant 4
      • datastringrequired
      • typestringrequiredenum redacted_thinking
      Variant 5
      • argumentsunknownrequired
      • function_idstringrequired
      • idstringrequired
      • typestringrequiredenum function_call
      Variant 6
      • contentunknown[]required
      • function_call_idstringrequired
      • is_errorboolean
      • typestringrequiredenum function_result
    • error_kindany of
      any of (2)
      Variant 1
      valuestringenum auth_expired, rate_limited, context_overflow, transient, permanent
      Variant 2
      valuenull
    • error_messagestring
    • modelstringrequired
    • native_stop_reasonstring
    • providerstringrequired
    • rolestringrequiredenum assistant
    • stop_reasonstringrequiredenum end, length, function_call, aborted, error
    • timestampinteger· int64required
    • usageany of
      any of (2)
      Variant 1
      • cache_readinteger· uint64min 0
      • cache_writeinteger· uint64min 0
      • cost_usdnumber· double
      • inputinteger· uint64min 0
      • outputinteger· uint64min 0
      • reasoninginteger· uint64min 0
      Variant 2
      valuenull
    • warningsstring[]
  • modelstringrequired
  • providerstringrequired
  • usageany of
    any of (2)
    Variant 1
    • cache_readinteger· uint64min 0
    • cache_writeinteger· uint64min 0
    • cost_usdnumber· double
    • inputinteger· uint64min 0
    • outputinteger· uint64min 0
    • reasoninginteger· uint64min 0
    Variant 2
    valuenull

router::embed

Function

Batch text embeddings through a provider's provider::<id>::embed surface. Names a provider or discovers the first embed-capable one from the live registry; one vector per input, order preserved.

Request
  • inputstring[]required

    Texts to embed, one vector returned per input, order preserved.

  • modelstring

    Embedding model id; the provider's default when omitted.

  • providerstring

    Provider id (e.g. `openai`); the first embed-capable provider in the live registry when omitted.

Response
  • embeddingsnumber[][]required

    One embedding per input, in input order.

  • modelstringrequired
  • providerstringrequired

router::models::budget

Function

Resolve the effective model output budget using the same precedence as router::chat.

Request
  • idstring

    Model id to budget.

  • max_output_tokensinteger· uint64min 0

    Optional caller-requested output budget. When absent, the same provider and router defaults used by `router::chat` apply.

  • providerstring

    Provider id that owns the model. Empty resolves an unambiguous model id.

Response
any of (2)
Variant 1
  • effective_max_output_tokensinteger· uint64requiredmin 0
  • modelobjectrequired

    The capability record (README § Model descriptor).

    • context_windowinteger· uint64requiredmin 0
    • display_namestring
    • idstringrequired
    • input_limitinteger· uint64min 0
    • max_output_tokensinteger· uint64requiredmin 0
    • pricingany of
      Show 2 nested
      any of (2)
      Variant 1
      • cache_readnumber· double
      • cache_writenumber· double
      • inputnumber· double
      • outputnumber· double
      Variant 2
      valuenull
    • providerstringrequired
    • reasoning_effortsobject[]
      Show 2 nested
      • descriptionstring
      • effortstringrequired
    • supports_cacheboolean
    • supports_structured_outputboolean
    • supports_thinkingboolean
    • supports_toolsboolean
    • supports_visionboolean
    • supports_xhighboolean
    • thinking_budgetsobject
Variant 2
valuenull

router::models::get

Function

Read one catalog model by {provider, id}; null when the model is not registered.

Request
  • idstring

    Model id to look up.

  • providerstring

    Provider id that owns the model.

Response
any of (2)
Variant 1
  • modelobjectrequired

    The capability record (README § Model descriptor).

    • context_windowinteger· uint64requiredmin 0
    • display_namestring
    • idstringrequired
    • input_limitinteger· uint64min 0
    • max_output_tokensinteger· uint64requiredmin 0
    • pricingany of
      Show 2 nested
      any of (2)
      Variant 1
      • cache_readnumber· double
      • cache_writenumber· double
      • inputnumber· double
      • outputnumber· double
      Variant 2
      valuenull
    • providerstringrequired
    • reasoning_effortsobject[]
      Show 2 nested
      • descriptionstring
      • effortstringrequired
    • supports_cacheboolean
    • supports_structured_outputboolean
    • supports_thinkingboolean
    • supports_toolsboolean
    • supports_visionboolean
    • supports_xhighboolean
    • thinking_budgetsobject
Variant 2
valuenull

router::models::list

Function

List catalog models, optionally filtered by provider and/or a capability flag.

Request
  • capabilitystring

    Keep only models that support this capability flag (optional).

  • providerstring

    Filter to a single provider id (optional).

Response
  • modelsobject[]required
    • context_windowinteger· uint64requiredmin 0
    • display_namestring
    • idstringrequired
    • input_limitinteger· uint64min 0
    • max_output_tokensinteger· uint64requiredmin 0
    • pricingany of
      any of (2)
      Variant 1
      • cache_readnumber· double
      • cache_writenumber· double
      • inputnumber· double
      • outputnumber· double
      Variant 2
      valuenull
    • providerstringrequired
    • reasoning_effortsobject[]
      • descriptionstring
      • effortstringrequired
    • supports_cacheboolean
    • supports_structured_outputboolean
    • supports_thinkingboolean
    • supports_toolsboolean
    • supports_visionboolean
    • supports_xhighboolean
    • thinking_budgetsobject

router::models::reconcile

Function

Replace a provider's catalog slice — the only catalog write path (token-gated).

Request
  • modelsobject[]

    The full replacement set of models for this provider.

    • context_windowinteger· uint64requiredmin 0
    • display_namestring
    • idstringrequired
    • input_limitinteger· uint64min 0
    • max_output_tokensinteger· uint64requiredmin 0
    • pricingany of
      any of (2)
      Variant 1
      • cache_readnumber· double
      • cache_writenumber· double
      • inputnumber· double
      • outputnumber· double
      Variant 2
      valuenull
    • providerstringrequired
    • reasoning_effortsobject[]
      • descriptionstring
      • effortstringrequired
    • supports_cacheboolean
    • supports_structured_outputboolean
    • supports_thinkingboolean
    • supports_toolsboolean
    • supports_visionboolean
    • supports_xhighboolean
    • thinking_budgetsobject
  • providerstring

    Provider whose catalog slice is being replaced.

  • tokenstring

    Registration token gating the write (optional).

Response
  • countinteger· uintrequiredmin 0
  • providerstringrequired

router::models::supports

Function

Check whether a model supports a capability flag (fails open for unknown models).

Request
  • capabilitystring

    Capability flag to check (e.g. `structured_output`, `vision`).

  • idstring

    Model id to check.

  • providerstring

    Provider id that owns the model.

Response
  • supportedbooleanrequired

router::on_config_changed

Function

Internal: reactively reload the in-memory configuration snapshot and fan out provider model discovery after configuration changes.

Request
  • idstring

    Configuration id that changed (advisory).

Response
  • okbooleanrequired

router::provider::list

Function

List registered providers with their configured/available status.

Request
Empty object
Response
  • providersobject[]required
    • availablebooleanrequired
    • configuredbooleanrequired
    • display_namestringrequired
    • idstringrequired
    • supports_model_listingbooleanrequired

router::provider::register

Function

Provider self-declaration at attach time (token-gated upsert); composes the configuration entry schema and reconciles static models.

Request
  • config_schemaunknown
  • credential_env_varstring
  • defaultsany of
    any of (2)
    Variant 1
    • api_urlstring
    • max_tokensinteger· uint64min 0
    Variant 2
    valuenull
  • display_namestring
  • idstringrequired
  • modelsobject[]
    • context_windowinteger· uint64requiredmin 0
    • display_namestring
    • idstringrequired
    • input_limitinteger· uint64min 0
    • max_output_tokensinteger· uint64requiredmin 0
    • pricingany of
      any of (2)
      Variant 1
      • cache_readnumber· double
      • cache_writenumber· double
      • inputnumber· double
      • outputnumber· double
      Variant 2
      valuenull
    • providerstringrequired
    • reasoning_effortsobject[]
      • descriptionstring
      • effortstringrequired
    • supports_cacheboolean
    • supports_structured_outputboolean
    • supports_thinkingboolean
    • supports_toolsboolean
    • supports_visionboolean
    • supports_xhighboolean
    • thinking_budgetsobject
  • supports_model_listingboolean
  • system_promptstring
  • tokenstring

    Registration token proving ownership on re-register (omit on first declare).

  • worker_idstring
Response
  • idstringrequired
  • okbooleanrequired
  • registration_tokenstringrequired

router::provider::resolve

Function

Resolve a provider's effective credential + api_url + max_tokens (token-gated).

Request
  • idstring

    Provider id to resolve credentials/config for.

  • tokenstring

    Registration token gating the resolve (optional).

Response
  • api_urlstring
  • configuredbooleanrequired
  • credentialany of
    any of (2)
    Variant 1
    one of (2)
    Variant 1
    • keystringrequired
    • typestringrequiredenum api_key
    Variant 2
    • access_tokenstringrequired
    • expires_atinteger· int64
    • provider_extraunknown
    • refresh_tokenstring
    • scopesstring[]
    • typestringrequiredenum oauth
    Variant 2
    valuenull
  • max_tokensinteger· uint64min 0
  • sourcestringrequiredenum config, env, none

router::provider::update_credential

Function

OAuth write-back: store a provider credential in the configuration entry under the entry write lock (token-gated).

Request
  • credentialunknown

    The credential object to store (provider-specific shape).

  • idstring

    Provider id whose credential slice is being written.

  • tokenstring

    Registration token gating the write (optional).

Response
  • okbooleanrequired

router::route

Function

Read-only routing preview: resolve {model, provider?} to the chosen provider + ordered candidates without streaming.

Request
  • modelstring

    Model id to route.

  • providerstring

    Pin an explicit provider, bypassing heuristics (optional).

Response
  • candidatesstring[]required
  • providerstringrequired

router::system_prompt::get

Function

Effective identity prompt for {provider?} (default_provider when omitted): operator override when set, else provider-declared; null when absent.

Request
  • providerstring
Response
  • providerstring
  • system_promptstring

Triggers3

router::models::changed

Trigger

A provider reconciled its catalog slice. Payload: { provider, count }.

Invocation
Empty object
Return
valueunknown

router::provider::changed

Trigger

The provider registry changed (declare / availability flip). Payload: { provider, op }.

Invocation
Empty object
Return
valueunknown

router::ready

Trigger

The router finished booting; providers bind here and re-declare after a restart.

Invocation
Empty object
Return
valueunknown

Search the registry

Find workers, kits and executables, or jump to a page.