# provider-anthropic > Anthropic Messages API provider worker; implements provider::anthropic::stream and provider::anthropic::refresh_models behind llm-router. | field | value | |-------|-------| | version | 1.2.28 | | type | binary | | license | Apache-2.0 | | repo | https://github.com/iii-hq/workers | | supported_targets | x86_64-apple-darwin, aarch64-apple-darwin, x86_64-pc-windows-msvc, aarch64-pc-windows-msvc, x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-unknown-linux-musl, armv7-unknown-linux-gnueabihf | | author | iii | ## installation ```sh iii trigger compose::add worker=provider-anthropic@1.2.28 ``` ## dependencies - `llm-router` @ `latest` - `state` @ `latest` ## readme # provider-anthropic Claude models behind [llm-router](https://github.com/iii-hq/workers/tree/main/llm-router). Install this worker next to the router, give it an API key, and the Anthropic catalog — streaming, extended thinking, tool use, vision, automatic prompt caching — appears behind the router's single front door (`router::chat` / `router::complete`). You never call this worker directly: the router invokes it worker-to-worker, and `iii-permissions.yaml` blocks agent access to `provider::anthropic::*`. ## Install ```bash iii trigger compose::add worker=provider-anthropic ``` The provider does nothing on its own — it plugs into the router: ```bash iii trigger compose::add worker=llm-router ``` `iii trigger compose::add` resolves each worker and its dependencies, writes exact declarations to `worker-compose.yaml`, and reconciles the Compose project. ## Quickstart Give the router a credential: paste a key into the `anthropic` slice of the engine's `llm-router` configuration entry, or set `ANTHROPIC_API_KEY` in the router's environment. ```json { "providers": { "anthropic": { "api_key": "sk-ant-…" } } } ``` The router picks up the change and kicks model discovery; Claude models land in the catalog (`router::models::list`). Then make a first call — Node shown, any SDK works: ```ts const res = await iii.trigger('router::complete', { model: 'claude-sonnet-4-6', messages: [{ role: 'user', content: [{ type: 'text', text: 'Hello' }], timestamp: Date.now() }], }, { timeout_ms: 320_000 }); // outer timeout ≥ the router's 300s stream budget // res: { message, usage, provider, model } — message is the final AssistantMessage ``` For token-by-token streaming, call `router::chat` with an iii channel — the walkthrough lives in [llm-router's Quickstart](https://github.com/iii-hq/workers/blob/main/llm-router/README.md). Request extended thinking with `thinking_level`; the worker maps the level to an Anthropic thinking budget from the model's catalog record. `xhigh` degrades to `high` on models that don't support it, and the level is dropped on models that don't support thinking at all. ## Configuration All operator configuration lives in the router's `llm-router` entry — this worker keeps no config of its own: ```jsonc "anthropic": { "api_key": "sk-ant-…", // or ANTHROPIC_API_KEY in the router's env "api_url": "https://api.anthropic.com/v1/messages", // override for proxies / gateways "max_tokens": 8192 // output ceiling when a request sets none } ``` Worker-side environment variables: | Variable | Default | Meaning | |---|---|---| | `PROVIDER_ANTHROPIC_CACHE` | enabled | `0`/`false` disables automatic prompt-cache markers | | `PROVIDER_ANTHROPIC_CACHE_TTL` | `1h` | TTL of the shared-prefix markers on the sectioned path; `5m` restores the default cache | | `PROVIDER_ANTHROPIC_PREFIX_MISMATCH` | `drop_block` | `thinking.block_binding.prefix_mismatch_behavior` sent (with the `thinking-binding-controls-2026-08-01` beta) on every thinking request: `drop_block` drops a replayed thinking block whose prefix was edited, `error` returns the 400 instead, `off` (also `0`/`false`/`none`) sends neither the field nor the beta (for gateways that reject it). Case-insensitive; any other value logs one warning and uses `drop_block`. **On accounts created before 2026-08-31, sending the field is what turns enforcement on**: after a prefix edit, replayed thinking blocks that sit after the edited element and were minted before the edit (every such block, for a `system`/`tools` edit), which still reached the model without the field, are now dropped on every later request; blocks before the edited element and blocks minted after the edit survive; `off` restores the old behaviour. Dropped blocks become one `anthropic dropped replayed thinking blocks: […]` warning on the message and a worker-log `warn` with the session id and model when the list arrives (aborted and errored responses included; grep the reason, e.g. `prefix_binding_mismatch`) | | `PROVIDER_ANTHROPIC_CAPTURE_DIR` | unset | Debug only, absolute path (a relative one logs one warning and captures nothing): append every request that has a session id to `/.jsonl` (bytes outside `[a-z0-9.-]`, `_` and uppercase included, written as `_xx` hex; the body plus `anthropic-beta`; auth headers are never written; on Unix the dir it creates is `0700` and each file `0600`). **The files hold user content, tool output, and images**: keep them out of the repo and delete them after use | | `III_WS_URL` | `ws://127.0.0.1:49134` | engine WebSocket to attach to when `--url` is not set | The binary also takes the standard worker CLI flags: `--url` (engine WebSocket), `--manifest` (print the registry manifest and exit), and `--config` (accepted but ignored with a warning — provider config comes from the `llm-router` configuration entry). Prompt caching needs no setup: markers go on the system prompt, the tools tail, and the last stable assistant turn whenever the prefix is big enough to be worth a cache write. When the router forwards `system_sections`, the `system` field becomes one text block per section and the marker moves to the block flagged `cache_boundary` (the frozen agent-profile prefix) whatever its size — Anthropic applies its per-model token minimum over the whole prefix, tools included, and silently skips a short one — so the per-session tail after it no longer invalidates the shared entry. At most two system blocks are marked, keeping the total at Anthropic's four. On that sectioned path the boundary block (and the tools marker ahead of it) use the 1-hour cache (`ttl: "1h"`, 2x base on the one write, reads unchanged) so a profile stays warm across sessions up to an hour apart; the per-turn messages anchor keeps the 5-minute default, which also satisfies the longer-before-shorter TTL rule. `PROVIDER_ANTHROPIC_CACHE_TTL=5m` goes back to 5 minutes everywhere. ## Models The catalog slice is **live** `GET /v1/models`: model ids, display names, context windows, output ceilings, and the capability flags (adaptive thinking, `xhigh` effort, vision) all come from the API on every refresh, so a newly shipped model (Opus 5.5, Sonnet 5, Fable 5.1, …) appears in the picker as soon as the API lists it — no code or SDK change required. Rows the API marks as thinking-capable but *not* adaptive-capable (the pre-4.6 generation) are dropped because this provider only implements adaptive thinking; the haiku family is the exception and stays with thinking gated off. The API does not publish pricing, so that is the one hand-maintained table: [`src/curated.rs`](src/curated.rs), USD per MTok keyed by base model id (date suffixes stripped). A model missing there still routes and shows up; it only loses cost enrichment, and the harness refuses `max_cost_usd` budgets on it. Update the table against [platform.claude.com/docs/en/about-claude/pricing](https://platform.claude.com/docs/en/about-claude/pricing) when Anthropic ships or reprices a model. ## Notes - **Token counting:** `provider::anthropic::count_tokens` (behind `router::count_tokens`) posts the assembled prompt to the messages endpoint's `count_tokens` sibling for an exact provider-metered count; it never runs the model and costs nothing. - **Structured output:** the Messages API has no native JSON mode; every catalog record declares `supports_structured_output: false`, and a forwarded `response_format` is reported in `warnings` and ignored. - **Errors:** 401/403 → `auth_expired`, 429 → `rate_limited`, 413/context → `context_overflow`, 5xx/network → `transient`, other 4xx → `permanent`. The worker never retries — the router owns retry policy. - **Credentials:** resolved per request via `router::provider::resolve` (config slice → `ANTHROPIC_API_KEY` on the router → none). Both `api_key` (x-api-key) and `oauth` (Bearer) shapes work; v1 performs no OAuth refresh. - **Identity binding:** the router issues a `registration_token` on first registration, persisted in state (scope `provider-anthropic`). If that state is lost the router rejects re-registration — clear the binding on the router side to re-pair. ## Tests ```bash cargo test # unit (pure modules + TCP stubs) III_ENGINE_BIN=$(which iii) cargo test --test integration -- --test-threads=1 ``` The integration suite spawns a real engine, the real router (path dep), this provider, and a local stub upstream — no external API calls anywhere. ## api reference ```json { "functions": [ { "description": "Cancel the in-flight upstream stream for a request_id (router::abort fan-out), stopping billed generation immediately.", "metadata": { "internal": true }, "name": "provider::anthropic::abort", "request_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "description": "Input of a provider's `provider::::abort`: actively cancel the in-flight upstream stream for `request_id` (the router's `request_id`, delivered to the provider as `resolution_key`) so billed generation stops immediately instead of waiting for the provider to notice the closed channel on its next write.", "properties": { "request_id": { "type": "string" } }, "required": [ "request_id" ], "title": "ProviderAbortRequest", "type": "object" }, "response_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "description": "Output of `provider::::abort`. `aborted: false` means the request was unknown — already finished, never started, or aborted before (idempotent).", "properties": { "aborted": { "type": "boolean" } }, "required": [ "aborted" ], "title": "ProviderAbortResponse", "type": "object" } }, { "description": "Count prompt tokens for {model, system_prompt?, tools?, messages} via the upstream count_tokens metering endpoint; never runs the model and costs nothing.", "metadata": { "internal": true }, "name": "provider::anthropic::count_tokens", "request_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "definitions": { "AgentFunction": { "description": "Function invocation schema — what a provider sees as a `tools` array entry (README § Function invocation schema; adapter boundary). These describe iii functions exposed to the model, not provider-native tools.", "properties": { "description": { "type": "string" }, "execution_mode": { "type": [ "string", "null" ] }, "label": { "type": [ "string", "null" ] }, "name": { "type": "string" }, "parameters": true }, "required": [ "description", "name", "parameters" ], "type": "object" }, "AgentMessage": { "anyOf": [ { "$ref": "#/definitions/AssistantMessage" }, { "$ref": "#/definitions/FunctionResultMessage" }, { "$ref": "#/definitions/CustomMessage" }, { "$ref": "#/definitions/UserMessage" } ], "description": "The canonical transcript message union. Untagged: the single-variant role tags disambiguate deserialization." }, "AssistantMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "error_kind": { "anyOf": [ { "$ref": "#/definitions/ErrorKind" }, { "type": "null" } ] }, "error_message": { "type": [ "string", "null" ] }, "model": { "type": "string" }, "native_stop_reason": { "type": [ "string", "null" ] }, "provider": { "type": "string" }, "role": { "$ref": "#/definitions/AssistantRoleTag" }, "stop_reason": { "$ref": "#/definitions/StopReason" }, "timestamp": { "format": "int64", "type": "integer" }, "usage": { "anyOf": [ { "$ref": "#/definitions/Usage" }, { "type": "null" } ] }, "warnings": { "items": { "type": "string" }, "type": [ "array", "null" ] } }, "required": [ "content", "model", "provider", "role", "stop_reason", "timestamp" ], "type": "object" }, "AssistantRoleTag": { "enum": [ "assistant" ], "type": "string" }, "ContentBlock": { "description": "Content blocks — the atomic units of message content (README § Content blocks).", "oneOf": [ { "properties": { "text": { "type": "string" }, "type": { "enum": [ "text" ], "type": "string" } }, "required": [ "text", "type" ], "type": "object" }, { "properties": { "attachment_id": { "description": "Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.", "type": [ "string", "null" ] }, "data": { "type": "string" }, "mime": { "type": "string" }, "type": { "enum": [ "image" ], "type": "string" } }, "required": [ "data", "mime", "type" ], "type": "object" }, { "description": "Reference to an original upload in the session-manager attachment store. A REFERENCE ONLY (never inline bytes); the harness strips it before `router::chat`. Tolerated here so a stray block deserializes instead of failing the request; every counter/mapping emits nothing for it.", "properties": { "attachment_id": { "type": "string" }, "mime": { "type": "string" }, "name": { "type": "string" }, "size": { "format": "uint64", "minimum": 0, "type": "integer" }, "type": { "enum": [ "file" ], "type": "string" } }, "required": [ "attachment_id", "mime", "name", "size", "type" ], "type": "object" }, { "properties": { "signature": { "type": [ "string", "null" ] }, "text": { "type": "string" }, "type": { "enum": [ "thinking" ], "type": "string" } }, "required": [ "text", "type" ], "type": "object" }, { "description": "Opaque redacted thinking payload — replayed verbatim on the Anthropic wire.", "properties": { "data": { "type": "string" }, "type": { "enum": [ "redacted_thinking" ], "type": "string" } }, "required": [ "data", "type" ], "type": "object" }, { "properties": { "arguments": true, "function_id": { "type": "string" }, "id": { "type": "string" }, "type": { "enum": [ "function_call" ], "type": "string" } }, "required": [ "arguments", "function_id", "id", "type" ], "type": "object" }, { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "function_call_id": { "type": "string" }, "is_error": { "type": [ "boolean", "null" ] }, "type": { "enum": [ "function_result" ], "type": "string" } }, "required": [ "content", "function_call_id", "type" ], "type": "object" } ] }, "CustomMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "custom_type": { "type": "string" }, "details": true, "display": { "type": [ "string", "null" ] }, "role": { "$ref": "#/definitions/CustomRoleTag" }, "timestamp": { "format": "int64", "type": "integer" } }, "required": [ "content", "custom_type", "role", "timestamp" ], "type": "object" }, "CustomRoleTag": { "enum": [ "custom" ], "type": "string" }, "ErrorKind": { "enum": [ "auth_expired", "rate_limited", "context_overflow", "transient", "permanent" ], "type": "string" }, "FunctionResultMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "details": true, "function_call_id": { "type": "string" }, "function_id": { "type": "string" }, "is_error": { "type": "boolean" }, "role": { "$ref": "#/definitions/FunctionResultRoleTag" }, "timestamp": { "format": "int64", "type": "integer" } }, "required": [ "content", "details", "function_call_id", "function_id", "is_error", "role", "timestamp" ], "type": "object" }, "FunctionResultRoleTag": { "enum": [ "function_result" ], "type": "string" }, "StopReason": { "enum": [ "end", "length", "function_call", "aborted", "error" ], "type": "string" }, "Usage": { "properties": { "cache_read": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "cache_write": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "cost_usd": { "format": "double", "type": [ "number", "null" ] }, "input": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "output": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "reasoning": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] } }, "type": "object" }, "UserMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "role": { "$ref": "#/definitions/UserRoleTag" }, "timestamp": { "format": "int64", "type": "integer" } }, "required": [ "content", "role", "timestamp" ], "type": "object" }, "UserRoleTag": { "description": "Single-variant role tags: exact-match on deserialize, correct wire string on serialize, and they let `AgentMessage` be an untagged union.", "enum": [ "user" ], "type": "string" } }, "properties": { "messages": { "description": "Wire agent messages, the same shape `provider::anthropic::stream` accepts. Must be non-empty.", "items": { "$ref": "#/definitions/AgentMessage" }, "type": "array" }, "model": { "description": "Model id the prompt targets (required by the upstream endpoint).", "type": "string" }, "system_prompt": { "default": null, "description": "System prompt counted as the wire `system` field when present.", "type": [ "string", "null" ] }, "tools": { "default": null, "description": "Function invocation schemas; mapped to the wire `tools` array.", "items": { "$ref": "#/definitions/AgentFunction" }, "type": [ "array", "null" ] } }, "required": [ "messages", "model" ], "title": "CountTokensRequest", "type": "object" }, "response_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "properties": { "estimator": { "description": "Always `provider`: the count came from the upstream metering API.", "type": "string" }, "model": { "type": "string" }, "tokens": { "description": "Prompt tokens the upstream endpoint counted for the assembled request.", "format": "uint64", "minimum": 0, "type": "integer" } }, "required": [ "estimator", "model", "tokens" ], "title": "CountTokensResponse", "type": "object" } }, { "description": "Internal: router::ready subscriber that re-declares this provider and refreshes its catalog.", "metadata": { "internal": true }, "name": "provider::anthropic::on_router_ready", "request_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "description": "Event delivered to a provider's `provider::::on_router_ready` (the `router::ready` trigger payload, currently `{}`). Unknown fields are ignored.", "title": "RouterReadyEvent", "type": "object" }, "response_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "description": "Ack returned by a provider's `provider::::on_router_ready`.", "properties": { "ok": { "type": "boolean" } }, "required": [ "ok" ], "title": "ProviderReadyAck", "type": "object" } }, { "description": "Refresh the Anthropic catalog slice from the upstream models endpoint and reconcile it through the router; returns the model count written.", "metadata": { "internal": true }, "name": "provider::anthropic::refresh_models", "request_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "description": "Input of a provider's `provider::::refresh_models` — takes no arguments. A struct (not `Value`) keeps the request schema concrete; unknown fields (e.g. the engine-injected `_caller_worker_id`) are ignored.", "title": "RefreshModelsRequest", "type": "object" }, "response_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "description": "Output of `provider::::refresh_models`.", "properties": { "count": { "format": "uint", "minimum": 0, "type": "integer" }, "ok": { "type": "boolean" } }, "required": [ "count", "ok" ], "title": "RefreshModelsResponse", "type": "object" } }, { "description": "Stream an Anthropic chat completion: resolve credentials, call the upstream Messages API, and relay AssistantMessageEvent frames to writer_ref.", "metadata": { "internal": true }, "name": "provider::anthropic::stream", "request_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "definitions": { "AgentFunction": { "description": "Function invocation schema — what a provider sees as a `tools` array entry (README § Function invocation schema; adapter boundary). These describe iii functions exposed to the model, not provider-native tools.", "properties": { "description": { "type": "string" }, "execution_mode": { "type": [ "string", "null" ] }, "label": { "type": [ "string", "null" ] }, "name": { "type": "string" }, "parameters": true }, "required": [ "description", "name", "parameters" ], "type": "object" }, "AgentMessage": { "anyOf": [ { "$ref": "#/definitions/AssistantMessage" }, { "$ref": "#/definitions/FunctionResultMessage" }, { "$ref": "#/definitions/CustomMessage" }, { "$ref": "#/definitions/UserMessage" } ], "description": "The canonical transcript message union. Untagged: the single-variant role tags disambiguate deserialization." }, "AssistantMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "error_kind": { "anyOf": [ { "$ref": "#/definitions/ErrorKind" }, { "type": "null" } ] }, "error_message": { "type": [ "string", "null" ] }, "model": { "type": "string" }, "native_stop_reason": { "type": [ "string", "null" ] }, "provider": { "type": "string" }, "role": { "$ref": "#/definitions/AssistantRoleTag" }, "stop_reason": { "$ref": "#/definitions/StopReason" }, "timestamp": { "format": "int64", "type": "integer" }, "usage": { "anyOf": [ { "$ref": "#/definitions/Usage" }, { "type": "null" } ] }, "warnings": { "items": { "type": "string" }, "type": [ "array", "null" ] } }, "required": [ "content", "model", "provider", "role", "stop_reason", "timestamp" ], "type": "object" }, "AssistantRoleTag": { "enum": [ "assistant" ], "type": "string" }, "ChannelDirection": { "enum": [ "read", "write" ], "type": "string" }, "ContentBlock": { "description": "Content blocks — the atomic units of message content (README § Content blocks).", "oneOf": [ { "properties": { "text": { "type": "string" }, "type": { "enum": [ "text" ], "type": "string" } }, "required": [ "text", "type" ], "type": "object" }, { "properties": { "attachment_id": { "description": "Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.", "type": [ "string", "null" ] }, "data": { "type": "string" }, "mime": { "type": "string" }, "type": { "enum": [ "image" ], "type": "string" } }, "required": [ "data", "mime", "type" ], "type": "object" }, { "description": "Reference to an original upload in the session-manager attachment store. A REFERENCE ONLY (never inline bytes); the harness strips it before `router::chat`. Tolerated here so a stray block deserializes instead of failing the request; every counter/mapping emits nothing for it.", "properties": { "attachment_id": { "type": "string" }, "mime": { "type": "string" }, "name": { "type": "string" }, "size": { "format": "uint64", "minimum": 0, "type": "integer" }, "type": { "enum": [ "file" ], "type": "string" } }, "required": [ "attachment_id", "mime", "name", "size", "type" ], "type": "object" }, { "properties": { "signature": { "type": [ "string", "null" ] }, "text": { "type": "string" }, "type": { "enum": [ "thinking" ], "type": "string" } }, "required": [ "text", "type" ], "type": "object" }, { "description": "Opaque redacted thinking payload — replayed verbatim on the Anthropic wire.", "properties": { "data": { "type": "string" }, "type": { "enum": [ "redacted_thinking" ], "type": "string" } }, "required": [ "data", "type" ], "type": "object" }, { "properties": { "arguments": true, "function_id": { "type": "string" }, "id": { "type": "string" }, "type": { "enum": [ "function_call" ], "type": "string" } }, "required": [ "arguments", "function_id", "id", "type" ], "type": "object" }, { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "function_call_id": { "type": "string" }, "is_error": { "type": [ "boolean", "null" ] }, "type": { "enum": [ "function_result" ], "type": "string" } }, "required": [ "content", "function_call_id", "type" ], "type": "object" } ] }, "CustomMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "custom_type": { "type": "string" }, "details": true, "display": { "type": [ "string", "null" ] }, "role": { "$ref": "#/definitions/CustomRoleTag" }, "timestamp": { "format": "int64", "type": "integer" } }, "required": [ "content", "custom_type", "role", "timestamp" ], "type": "object" }, "CustomRoleTag": { "enum": [ "custom" ], "type": "string" }, "ErrorKind": { "enum": [ "auth_expired", "rate_limited", "context_overflow", "transient", "permanent" ], "type": "string" }, "FunctionResultMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "details": true, "function_call_id": { "type": "string" }, "function_id": { "type": "string" }, "is_error": { "type": "boolean" }, "role": { "$ref": "#/definitions/FunctionResultRoleTag" }, "timestamp": { "format": "int64", "type": "integer" } }, "required": [ "content", "details", "function_call_id", "function_id", "is_error", "role", "timestamp" ], "type": "object" }, "FunctionResultRoleTag": { "enum": [ "function_result" ], "type": "string" }, "Model": { "description": "The capability record (README § Model descriptor).", "properties": { "context_window": { "format": "uint64", "minimum": 0, "type": "integer" }, "display_name": { "type": [ "string", "null" ] }, "id": { "type": "string" }, "input_limit": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "max_output_tokens": { "format": "uint64", "minimum": 0, "type": "integer" }, "pricing": { "anyOf": [ { "$ref": "#/definitions/Pricing" }, { "type": "null" } ] }, "provider": { "type": "string" }, "reasoning_efforts": { "items": { "$ref": "#/definitions/ReasoningEffort" }, "type": [ "array", "null" ] }, "speech": { "anyOf": [ { "$ref": "#/definitions/SpeechModel" }, { "type": "null" } ], "description": "Set on speech models only; see [`SpeechModel`]." }, "supports_cache": { "type": [ "boolean", "null" ] }, "supports_structured_output": { "type": [ "boolean", "null" ] }, "supports_thinking": { "type": [ "boolean", "null" ] }, "supports_thinking_off": { "description": "Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.", "type": [ "boolean", "null" ] }, "supports_tools": { "type": [ "boolean", "null" ] }, "supports_vision": { "type": [ "boolean", "null" ] }, "supports_xhigh": { "type": [ "boolean", "null" ] }, "thinking_budgets": { "additionalProperties": { "format": "uint64", "minimum": 0, "type": "integer" }, "type": [ "object", "null" ] } }, "required": [ "context_window", "id", "max_output_tokens", "provider" ], "type": "object" }, "Pricing": { "properties": { "cache_read": { "format": "double", "type": [ "number", "null" ] }, "cache_write": { "format": "double", "type": [ "number", "null" ] }, "input": { "format": "double", "type": [ "number", "null" ] }, "output": { "format": "double", "type": [ "number", "null" ] } }, "type": "object" }, "PromptCacheIntent": { "description": "Why a request's stable prefix is shareable across sessions: `surface_digest` is the caller's content identity for the text before the boundary (`sha256:`). A routing/accounting hint only — never a provider cache handle, and never evidence of a cache hit.", "properties": { "surface_digest": { "type": "string" } }, "required": [ "surface_digest" ], "type": "object" }, "PromptSection": { "description": "One ordered system-prompt section. `cache_boundary` marks the end of a cumulative prefix worth its own provider cache entry (the frozen agent-profile surface). Adapters without a boundary concept read the sections joined with \"\\n\\n\", which must equal `system_prompt` when both forms are sent.", "properties": { "cache_boundary": { "default": false, "type": "boolean" }, "text": { "type": "string" } }, "required": [ "text" ], "type": "object" }, "ReasoningEffort": { "description": "One provider-native reasoning effort advertised for a specific model.\n\nValues intentionally remain strings: provider catalogs can add efforts without requiring a router-wide enum release first.", "properties": { "description": { "type": [ "string", "null" ] }, "effort": { "type": "string" } }, "required": [ "effort" ], "type": "object" }, "ResponseFormat": { "properties": { "schema": true, "type": { "type": "string" } }, "required": [ "type" ], "type": "object" }, "SpeechModality": { "description": "What a speech model does. Chat models carry no `speech` block.", "oneOf": [ { "description": "Speech to text, served through `router::transcribe`.", "enum": [ "stt" ], "type": "string" }, { "description": "Text to speech, served through `router::speak`.", "enum": [ "tts" ], "type": "string" } ] }, "SpeechModel": { "description": "Facts about a speech model. Present only on models served through `router::transcribe` / `router::speak`; such models report `context_window` and `max_output_tokens` as 0.", "properties": { "languages": { "description": "BCP-47 tags of the languages the model handles; empty when the provider does not say.", "items": { "type": "string" }, "type": "array" }, "modality": { "$ref": "#/definitions/SpeechModality" }, "streaming": { "default": false, "description": "Realtime input (stt) or streamed audio output (tts) is available.", "type": "boolean" } }, "required": [ "modality" ], "type": "object" }, "StopReason": { "enum": [ "end", "length", "function_call", "aborted", "error" ], "type": "string" }, "StreamChannelRef": { "properties": { "access_key": { "type": "string" }, "channel_id": { "type": "string" }, "direction": { "$ref": "#/definitions/ChannelDirection" } }, "required": [ "access_key", "channel_id", "direction" ], "type": "object" }, "ThinkingLevel": { "description": "\"off\" asks the provider not to reason at all and is honoured only where the model advertises `Model::supports_thinking_off` (elsewhere the provider falls back to its own lowest setting, or omits the parameter, and warns). \"minimal\" requests the lowest reasoning effort and needs only `thinking` support; levels map to provider-native knobs via `Model::thinking_budgets`.", "enum": [ "off", "minimal", "low", "medium", "high", "xhigh" ], "type": "string" }, "Usage": { "properties": { "cache_read": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "cache_write": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "cost_usd": { "format": "double", "type": [ "number", "null" ] }, "input": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "output": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "reasoning": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] } }, "type": "object" }, "UserMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "role": { "$ref": "#/definitions/UserRoleTag" }, "timestamp": { "format": "int64", "type": "integer" } }, "required": [ "content", "role", "timestamp" ], "type": "object" }, "UserRoleTag": { "description": "Single-variant role tags: exact-match on deserialize, correct wire string on serialize, and they let `AgentMessage` be an untagged union.", "enum": [ "user" ], "type": "string" } }, "description": "Input of a provider worker's `provider::::stream` iii function — what the router forwards per attempt. (No `PartialEq`: `iii_sdk::StreamChannelRef` doesn't implement it.)", "properties": { "cache_intent": { "anyOf": [ { "$ref": "#/definitions/PromptCacheIntent" }, { "type": "null" } ] }, "max_output_tokens": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "messages": { "items": { "$ref": "#/definitions/AgentMessage" }, "type": "array" }, "model": { "type": "string" }, "model_meta": { "anyOf": [ { "$ref": "#/definitions/Model" }, { "type": "null" } ] }, "provider_options": true, "resolution_key": { "type": [ "string", "null" ] }, "response_format": { "anyOf": [ { "$ref": "#/definitions/ResponseFormat" }, { "type": "null" } ] }, "session_id": { "description": "Stable conversation identity for provider cache affinity.", "type": [ "string", "null" ] }, "system_prompt": { "type": [ "string", "null" ] }, "system_sections": { "description": "Ordered sections behind `system_prompt` (always forwarded alongside the flat string, which equals the sections joined with \"\\n\\n\").", "items": { "$ref": "#/definitions/PromptSection" }, "type": [ "array", "null" ] }, "thinking_level": { "anyOf": [ { "$ref": "#/definitions/ThinkingLevel" }, { "type": "null" } ] }, "tools": { "items": { "$ref": "#/definitions/AgentFunction" }, "type": [ "array", "null" ] }, "writer_ref": { "$ref": "#/definitions/StreamChannelRef" } }, "required": [ "messages", "model", "writer_ref" ], "title": "ProviderStreamInput", "type": "object" }, "response_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "description": "Output of a provider's `provider::::stream` (spec § stream contract): the function streams frames to `writer_ref` and returns this ack.", "properties": { "ok": { "type": "boolean" } }, "required": [ "ok" ], "title": "ProviderStreamOutput", "type": "object" } } ], "triggers": [] } ```