# provider-deepseek > DeepSeek Chat Completions provider worker; implements provider::deepseek::stream and provider::deepseek::refresh_models behind llm-router. | field | value | |-------|-------| | version | 0.1.21 | | type | binary | | license | Apache-2.0 | | repo | https://github.com/iii-hq/workers | | supported_targets | x86_64-apple-darwin, aarch64-apple-darwin, x86_64-pc-windows-msvc, aarch64-pc-windows-msvc, x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-unknown-linux-musl, armv7-unknown-linux-gnueabihf | | author | iii | ## installation ```sh iii trigger compose::add worker=provider-deepseek@0.1.21 ``` ## dependencies - `llm-router` @ `latest` - `state` @ `latest` ## readme # provider-deepseek DeepSeek Chat Completions provider worker behind [llm-router](https://github.com/iii-hq/workers/tree/main/llm-router). Implements the provider protocol from `tech-specs/2026-06-agentic/llm-router.md`: `provider::deepseek::stream` (SSE chunks → `AssistantMessageEvent` frames into a router-owned channel) and `provider::deepseek::refresh_models` (upstream `GET /models`, enriched with local metadata → `router::models::reconcile`), plus `provider::deepseek::count_tokens` (prompt token counting behind `router::count_tokens`, with DeepSeek's own published vocabulary rather than a borrowed one; never runs the model and costs nothing). Default upstream: `https://api.deepseek.com/chat/completions` — DeepSeek's OpenAI-compatible surface is rooted at the bare host, with no `/v1` segment. Override `api_url` to point at any other OpenAI-compatible endpoint; the models listing is always read from that endpoint's `/models` sibling. ## Behavior - **Registration:** self-declares via `router::provider::register` with backoff until acked, and re-declares on the `router::ready` trigger type. The declaration carries no models and `credential_env_var: DEEPSEEK_API_KEY`; the post-register refresh discovers the catalog, gated on a configured credential (no key → empty slice, so the picker never shows unusable rows). - **Identity binding:** the router returns a `registration_token` on first registration; it is persisted in state (scope `provider-deepseek`, key `registration_token`) and presented on every later `register`/`resolve`/`reconcile`. If that state is lost the router rejects re-registration — the operator must clear the binding on the router side. - **Credentials:** resolved per request via `router::provider::resolve` (config slice → `DEEPSEEK_API_KEY` env on the router → none). Both `api_key` and `oauth` credential shapes are sent as `Authorization: Bearer`; v1 performs no OAuth refresh. - **Catalog:** `GET /models` owns the id list; `src/curated.rs` supplies what the listing does not carry — display names, context windows, output ceilings, pricing (USD per MTok) and the `reasoning_effort` vocabulary from api-docs.deepseek.com (snapshot 2026-09-10). The live ids are `deepseek-flash` (DeepSeek-V4.1-Flash) and `deepseek-v4-pro`; the retired `deepseek-v4-flash` names are still accepted by the API and get the Flash row's metadata under their own id. An id the table does not know still lands in the catalog on conservative defaults (64K context, 8K output, no pricing) rather than disappearing, so a model DeepSeek ships tomorrow is routable today. Prices are the **peak** rate — DeepSeek's list price; off-peak (every hour outside 01:00–04:00 and 06:00–10:00 UTC, Monday to Friday) is half of it, so reported cost is exact at peak and a 2x ceiling off-peak — a time-of-day multiplier is not something the catalog can express. DeepSeek retires V4 Pro on 2026-09-14 12:00 Beijing time: from then `deepseek-v4-pro` requests are answered by V4.1 Flash and billed at the Flash price until V4.1 Pro ships; the row stays while the id is listed. - **Liveness:** `ping` at least every 30s of upstream silence; a failed channel write (caller gone / `router::abort`) drops the SSE receiver and aborts the in-flight HTTP request. DeepSeek holds an overloaded request open with `: keep-alive` SSE comments, which the decoder ignores. Generation must start within **120s** of the upstream request, including the wait for HTTP headers. Keepalives, empty deltas, and usage-only chunks cannot extend that deadline. On expiry the provider closes the HTTP request and emits one `transient` error explaining that generation did not start; the model selection and catalog stay unchanged. The first text, reasoning, or tool-call content disarms this startup deadline, so a long answer continues under the normal router stream/idle budgets. Set `PROVIDER_READ_TIMEOUT_SECS` on the **provider worker** to change both this startup budget and the existing HTTP read-silence timeout (for example, when a self-hosted `api_url` needs a longer prompt evaluation). - **Errors:** 401/403 → `auth_expired`, **402 (out of balance) → `permanent`** (a billing wall retry cannot fix), 429 → `rate_limited`, 400/422 → `permanent` unless the message says the prompt did not fit, `context_length_exceeded` → `context_overflow`, 500/503 and network → `transient`. `finish_reason: insufficient_system_resource` — the upstream running out of capacity mid-generation — terminates the stream as a `transient` error so the router retries instead of returning a silently truncated answer. No transport retries here: the router owns retry policy. Pre-generation diagnostics stay in `error_message`, with empty assistant content. A read failure after generation starts retains the streamed content, usage, and warnings so the Harness can preserve and resume the partial response. - **Structured output:** `json_object` mode only — DeepSeek documents no strict json_schema mode. A `response_format` schema is dropped with a report-and-continue warning; every catalog record declares `supports_structured_output: false`. The caller must mention "json" in the prompt per DeepSeek's rules. - **Reasoning:** with a requested thinking level the request carries `thinking: {type: enabled}` plus the top-level `reasoning_effort` param — the router's five levels collapse onto DeepSeek's three-wide vocabulary as `minimal`/`low` → `low`, `medium`/`high` → `high`, `xhigh` → `max` (`src/reasoning.rs`; the API would coerce a literal `xhigh` down to `high`, so the mapping is done here). With **no** level both params are omitted, so each model runs its own documented default: the V4 family reasons at `high` effort — an unconfigured console chat streams its chain of thought out of the box, with reasoning tokens billed as output — while a legacy non-thinking alias (`deepseek-chat`) keeps the behavior its name encodes. **Off** is the one knob the ladder cannot express: pass `provider_options: { deepseek: { thinking: "disabled" } }` on the send and the request carries `thinking: {type: disabled}` with no `reasoning_effort` (a `thinking_level` on the same send is reported as ignored). It is never synthesised — an off-by-default would blank the console's thinking pane — and any value other than `enabled`/`disabled` fails the turn instead of silently running at full effort. Each documented catalog row advertises the effort vocabulary as `reasoning_efforts` (`low`, `high`, `max`) so `router::models::get` shows what a level lands on. Reasoning models stream their chain of thought as `reasoning_content` deltas, which the worker surfaces as `thinking` blocks (`src/sse.rs`), and `completion_tokens_details.reasoning_tokens` lands on `usage.reasoning`. Assistant thinking is **replayed** to the API as `reasoning_content` only on tool-calling messages — DeepSeek 400s a tool round whose intermediate reasoning was dropped — and nowhere else: the API documents replayed reasoning as ignored outside tool rounds, where it would only re-bill the whole chain as input on every later turn. - **Block ordering:** the assembled message is a *sequence* of blocks in the order the model produced them, not one merged block per kind. A turn that reasons, answers, reasons again and answers again lands as four blocks, and a tool call sits between the blocks it actually fell between — matching what the `thinking_start`/`text_start`/… frames already described on the wire, and the same shape `provider-anthropic` assembles. DeepSeek's own response carries one `reasoning_content` and one `content` per message, so in practice a turn is one thinking block then one text block; the ordering matters behind an `api_url` override onto a gateway that genuinely interleaves. On the turns that replay at all (tool-calling ones — see Reasoning), replay flattens the blocks back to the two scalar fields the request shape has (thinking blocks joined, text blocks joined) — the only representation DeepSeek accepts. - **Text only:** DeepSeek documents no multimodal content-part array, so image blocks are replaced with a text marker instead of being sent as `image_url` parts (which would fail the whole turn); the caller gets a report-and-continue warning and every known catalog record declares `supports_vision: false`. - **Prompt caching:** on by default for every account, with no request markers (context caching on disk). Hits land on shared request *prefixes*, so the wire layer keeps a session's transcript append-only by construction: every serialization rule depends only on a message's own content (never on what follows it), which makes turn N's request body a byte-stable prefix of turn N+1's; ignorable reasoning is dropped instead of resent (see Reasoning); and no `user_id` is sent, so all traffic shares one account-wide cache instead of partitioning it. The two deliberate exceptions mutate history for correctness and cost one cache bust each: a late tool result replacing its orphan placeholder, and latest-wins dedup of a duplicated result. DeepSeek reports the resulting prompt split directly, and it maps straight onto the spec's disjoint usage splits: `prompt_cache_hit_tokens` → `usage.cache_read` (billed at the cached-input rate) and `prompt_cache_miss_tokens` → `usage.input` (billed at the input rate). Because the router's cost fill adds the two, `input` is deliberately the miss slice rather than the `prompt_tokens` total — with a ~120x cache discount, billing the cached prefix at both rates would roughly double the reported cost of a long agent loop. ## Tests ```bash cargo test # unit (pure modules + TCP stubs) III_ENGINE_BIN=$(which iii) cargo test --test integration -- --test-threads=1 ``` The integration suite spawns a real engine, the real router (path dep), this provider, and a local stub upstream serving both `/models` and `/chat/completions` — no external API calls anywhere. ## Running The binary takes the standard worker CLI flags: `--url` (engine WebSocket, default `ws://127.0.0.1:49134`, falls back to the `III_URL` environment variable), `--manifest` (print the registry manifest and exit), and `--config` (accepted but ignored with a warning — provider config comes from the `llm-router` configuration entry). ## api reference ```json { "functions": [ { "description": "Cancel the in-flight upstream stream for a request_id (router::abort fan-out), stopping billed generation immediately.", "metadata": { "internal": true }, "name": "provider::deepseek::abort", "request_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "description": "Input of a provider's `provider::::abort`: actively cancel the in-flight upstream stream for `request_id` (the router's `request_id`, delivered to the provider as `resolution_key`) so billed generation stops immediately instead of waiting for the provider to notice the closed channel on its next write.", "properties": { "request_id": { "type": "string" } }, "required": [ "request_id" ], "title": "ProviderAbortRequest", "type": "object" }, "response_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "description": "Output of `provider::::abort`. `aborted: false` means the request was unknown — already finished, never started, or aborted before (idempotent).", "properties": { "aborted": { "type": "boolean" } }, "required": [ "aborted" ], "title": "ProviderAbortResponse", "type": "object" } }, { "description": "Count prompt tokens for {model, system_prompt?, tools?, messages} locally with DeepSeek's own published vocabulary; never runs the model and costs nothing.", "metadata": { "internal": true }, "name": "provider::deepseek::count_tokens", "request_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "definitions": { "AgentFunction": { "description": "Function invocation schema — what a provider sees as a `tools` array entry (README § Function invocation schema; adapter boundary). These describe iii functions exposed to the model, not provider-native tools.", "properties": { "description": { "type": "string" }, "execution_mode": { "type": [ "string", "null" ] }, "label": { "type": [ "string", "null" ] }, "name": { "type": "string" }, "parameters": true }, "required": [ "description", "name", "parameters" ], "type": "object" }, "AgentMessage": { "anyOf": [ { "$ref": "#/definitions/AssistantMessage" }, { "$ref": "#/definitions/FunctionResultMessage" }, { "$ref": "#/definitions/CustomMessage" }, { "$ref": "#/definitions/UserMessage" } ], "description": "The canonical transcript message union. Untagged: the single-variant role tags disambiguate deserialization." }, "AssistantMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "error_kind": { "anyOf": [ { "$ref": "#/definitions/ErrorKind" }, { "type": "null" } ] }, "error_message": { "type": [ "string", "null" ] }, "model": { "type": "string" }, "native_stop_reason": { "type": [ "string", "null" ] }, "provider": { "type": "string" }, "role": { "$ref": "#/definitions/AssistantRoleTag" }, "stop_reason": { "$ref": "#/definitions/StopReason" }, "timestamp": { "format": "int64", "type": "integer" }, "usage": { "anyOf": [ { "$ref": "#/definitions/Usage" }, { "type": "null" } ] }, "warnings": { "items": { "type": "string" }, "type": [ "array", "null" ] } }, "required": [ "content", "model", "provider", "role", "stop_reason", "timestamp" ], "type": "object" }, "AssistantRoleTag": { "enum": [ "assistant" ], "type": "string" }, "ContentBlock": { "description": "Content blocks — the atomic units of message content (README § Content blocks).", "oneOf": [ { "properties": { "text": { "type": "string" }, "type": { "enum": [ "text" ], "type": "string" } }, "required": [ "text", "type" ], "type": "object" }, { "properties": { "attachment_id": { "description": "Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.", "type": [ "string", "null" ] }, "data": { "type": "string" }, "mime": { "type": "string" }, "type": { "enum": [ "image" ], "type": "string" } }, "required": [ "data", "mime", "type" ], "type": "object" }, { "description": "Reference to an original upload in the session-manager attachment store. A REFERENCE ONLY (never inline bytes); the harness strips it before `router::chat`. Tolerated here so a stray block deserializes instead of failing the request; every counter/mapping emits nothing for it.", "properties": { "attachment_id": { "type": "string" }, "mime": { "type": "string" }, "name": { "type": "string" }, "size": { "format": "uint64", "minimum": 0, "type": "integer" }, "type": { "enum": [ "file" ], "type": "string" } }, "required": [ "attachment_id", "mime", "name", "size", "type" ], "type": "object" }, { "properties": { "signature": { "type": [ "string", "null" ] }, "text": { "type": "string" }, "type": { "enum": [ "thinking" ], "type": "string" } }, "required": [ "text", "type" ], "type": "object" }, { "description": "Opaque redacted thinking payload — replayed verbatim on the Anthropic wire.", "properties": { "data": { "type": "string" }, "type": { "enum": [ "redacted_thinking" ], "type": "string" } }, "required": [ "data", "type" ], "type": "object" }, { "properties": { "arguments": true, "function_id": { "type": "string" }, "id": { "type": "string" }, "type": { "enum": [ "function_call" ], "type": "string" } }, "required": [ "arguments", "function_id", "id", "type" ], "type": "object" }, { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "function_call_id": { "type": "string" }, "is_error": { "type": [ "boolean", "null" ] }, "type": { "enum": [ "function_result" ], "type": "string" } }, "required": [ "content", "function_call_id", "type" ], "type": "object" } ] }, "CustomMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "custom_type": { "type": "string" }, "details": true, "display": { "type": [ "string", "null" ] }, "role": { "$ref": "#/definitions/CustomRoleTag" }, "timestamp": { "format": "int64", "type": "integer" } }, "required": [ "content", "custom_type", "role", "timestamp" ], "type": "object" }, "CustomRoleTag": { "enum": [ "custom" ], "type": "string" }, "ErrorKind": { "enum": [ "auth_expired", "rate_limited", "context_overflow", "transient", "permanent" ], "type": "string" }, "FunctionResultMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "details": true, "function_call_id": { "type": "string" }, "function_id": { "type": "string" }, "is_error": { "type": "boolean" }, "role": { "$ref": "#/definitions/FunctionResultRoleTag" }, "timestamp": { "format": "int64", "type": "integer" } }, "required": [ "content", "details", "function_call_id", "function_id", "is_error", "role", "timestamp" ], "type": "object" }, "FunctionResultRoleTag": { "enum": [ "function_result" ], "type": "string" }, "StopReason": { "enum": [ "end", "length", "function_call", "aborted", "error" ], "type": "string" }, "Usage": { "properties": { "cache_read": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "cache_write": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "cost_usd": { "format": "double", "type": [ "number", "null" ] }, "input": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "output": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "reasoning": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] } }, "type": "object" }, "UserMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "role": { "$ref": "#/definitions/UserRoleTag" }, "timestamp": { "format": "int64", "type": "integer" } }, "required": [ "content", "role", "timestamp" ], "type": "object" }, "UserRoleTag": { "description": "Single-variant role tags: exact-match on deserialize, correct wire string on serialize, and they let `AgentMessage` be an untagged union.", "enum": [ "user" ], "type": "string" } }, "properties": { "messages": { "description": "Wire agent messages, the same shape `provider::deepseek::stream` accepts. Must be non-empty.", "items": { "$ref": "#/definitions/AgentMessage" }, "type": "array" }, "model": { "description": "Model id the prompt targets. Reported back unchanged; every DeepSeek chat model shares the one vocabulary.", "type": "string" }, "system_prompt": { "default": null, "description": "System prompt counted as its own wire message when present.", "type": [ "string", "null" ] }, "tools": { "default": null, "description": "Function invocation schemas; each serialized schema counts toward the total.", "items": { "$ref": "#/definitions/AgentFunction" }, "type": [ "array", "null" ] } }, "required": [ "messages", "model" ], "title": "CountTokensRequest", "type": "object" }, "response_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "properties": { "estimator": { "description": "Always `tokenizer`: the model's own vocabulary produced the count.", "type": "string" }, "model": { "type": "string" }, "tokens": { "description": "Prompt tokens for the assembled request, by DeepSeek's vocabulary.", "format": "uint64", "minimum": 0, "type": "integer" } }, "required": [ "estimator", "model", "tokens" ], "title": "CountTokensResponse", "type": "object" } }, { "description": "Internal: router::ready subscriber that re-declares this provider and refreshes its catalog.", "metadata": { "internal": true }, "name": "provider::deepseek::on_router_ready", "request_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "description": "Event delivered to a provider's `provider::::on_router_ready` (the `router::ready` trigger payload, currently `{}`). Unknown fields are ignored.", "title": "RouterReadyEvent", "type": "object" }, "response_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "description": "Ack returned by a provider's `provider::::on_router_ready`.", "properties": { "ok": { "type": "boolean" } }, "required": [ "ok" ], "title": "ProviderReadyAck", "type": "object" } }, { "description": "Reconcile the DeepSeek catalog slice through the router: list the upstream models, enrich each with local metadata, and return the model count written.", "metadata": { "internal": true }, "name": "provider::deepseek::refresh_models", "request_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "description": "Input of a provider's `provider::::refresh_models` — takes no arguments. A struct (not `Value`) keeps the request schema concrete; unknown fields (e.g. the engine-injected `_caller_worker_id`) are ignored.", "title": "RefreshModelsRequest", "type": "object" }, "response_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "description": "Output of `provider::::refresh_models`.", "properties": { "count": { "format": "uint", "minimum": 0, "type": "integer" }, "ok": { "type": "boolean" } }, "required": [ "count", "ok" ], "title": "RefreshModelsResponse", "type": "object" } }, { "description": "Stream a DeepSeek chat completion: resolve credentials, call the upstream Chat Completions API, and relay AssistantMessageEvent frames to writer_ref.", "metadata": { "internal": true }, "name": "provider::deepseek::stream", "request_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "definitions": { "AgentFunction": { "description": "Function invocation schema — what a provider sees as a `tools` array entry (README § Function invocation schema; adapter boundary). These describe iii functions exposed to the model, not provider-native tools.", "properties": { "description": { "type": "string" }, "execution_mode": { "type": [ "string", "null" ] }, "label": { "type": [ "string", "null" ] }, "name": { "type": "string" }, "parameters": true }, "required": [ "description", "name", "parameters" ], "type": "object" }, "AgentMessage": { "anyOf": [ { "$ref": "#/definitions/AssistantMessage" }, { "$ref": "#/definitions/FunctionResultMessage" }, { "$ref": "#/definitions/CustomMessage" }, { "$ref": "#/definitions/UserMessage" } ], "description": "The canonical transcript message union. Untagged: the single-variant role tags disambiguate deserialization." }, "AssistantMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "error_kind": { "anyOf": [ { "$ref": "#/definitions/ErrorKind" }, { "type": "null" } ] }, "error_message": { "type": [ "string", "null" ] }, "model": { "type": "string" }, "native_stop_reason": { "type": [ "string", "null" ] }, "provider": { "type": "string" }, "role": { "$ref": "#/definitions/AssistantRoleTag" }, "stop_reason": { "$ref": "#/definitions/StopReason" }, "timestamp": { "format": "int64", "type": "integer" }, "usage": { "anyOf": [ { "$ref": "#/definitions/Usage" }, { "type": "null" } ] }, "warnings": { "items": { "type": "string" }, "type": [ "array", "null" ] } }, "required": [ "content", "model", "provider", "role", "stop_reason", "timestamp" ], "type": "object" }, "AssistantRoleTag": { "enum": [ "assistant" ], "type": "string" }, "ChannelDirection": { "enum": [ "read", "write" ], "type": "string" }, "ContentBlock": { "description": "Content blocks — the atomic units of message content (README § Content blocks).", "oneOf": [ { "properties": { "text": { "type": "string" }, "type": { "enum": [ "text" ], "type": "string" } }, "required": [ "text", "type" ], "type": "object" }, { "properties": { "attachment_id": { "description": "Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.", "type": [ "string", "null" ] }, "data": { "type": "string" }, "mime": { "type": "string" }, "type": { "enum": [ "image" ], "type": "string" } }, "required": [ "data", "mime", "type" ], "type": "object" }, { "description": "Reference to an original upload in the session-manager attachment store. A REFERENCE ONLY (never inline bytes); the harness strips it before `router::chat`. Tolerated here so a stray block deserializes instead of failing the request; every counter/mapping emits nothing for it.", "properties": { "attachment_id": { "type": "string" }, "mime": { "type": "string" }, "name": { "type": "string" }, "size": { "format": "uint64", "minimum": 0, "type": "integer" }, "type": { "enum": [ "file" ], "type": "string" } }, "required": [ "attachment_id", "mime", "name", "size", "type" ], "type": "object" }, { "properties": { "signature": { "type": [ "string", "null" ] }, "text": { "type": "string" }, "type": { "enum": [ "thinking" ], "type": "string" } }, "required": [ "text", "type" ], "type": "object" }, { "description": "Opaque redacted thinking payload — replayed verbatim on the Anthropic wire.", "properties": { "data": { "type": "string" }, "type": { "enum": [ "redacted_thinking" ], "type": "string" } }, "required": [ "data", "type" ], "type": "object" }, { "properties": { "arguments": true, "function_id": { "type": "string" }, "id": { "type": "string" }, "type": { "enum": [ "function_call" ], "type": "string" } }, "required": [ "arguments", "function_id", "id", "type" ], "type": "object" }, { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "function_call_id": { "type": "string" }, "is_error": { "type": [ "boolean", "null" ] }, "type": { "enum": [ "function_result" ], "type": "string" } }, "required": [ "content", "function_call_id", "type" ], "type": "object" } ] }, "CustomMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "custom_type": { "type": "string" }, "details": true, "display": { "type": [ "string", "null" ] }, "role": { "$ref": "#/definitions/CustomRoleTag" }, "timestamp": { "format": "int64", "type": "integer" } }, "required": [ "content", "custom_type", "role", "timestamp" ], "type": "object" }, "CustomRoleTag": { "enum": [ "custom" ], "type": "string" }, "ErrorKind": { "enum": [ "auth_expired", "rate_limited", "context_overflow", "transient", "permanent" ], "type": "string" }, "FunctionResultMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "details": true, "function_call_id": { "type": "string" }, "function_id": { "type": "string" }, "is_error": { "type": "boolean" }, "role": { "$ref": "#/definitions/FunctionResultRoleTag" }, "timestamp": { "format": "int64", "type": "integer" } }, "required": [ "content", "details", "function_call_id", "function_id", "is_error", "role", "timestamp" ], "type": "object" }, "FunctionResultRoleTag": { "enum": [ "function_result" ], "type": "string" }, "Model": { "description": "The capability record (README § Model descriptor).", "properties": { "context_window": { "format": "uint64", "minimum": 0, "type": "integer" }, "display_name": { "type": [ "string", "null" ] }, "id": { "type": "string" }, "input_limit": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "max_output_tokens": { "format": "uint64", "minimum": 0, "type": "integer" }, "pricing": { "anyOf": [ { "$ref": "#/definitions/Pricing" }, { "type": "null" } ] }, "provider": { "type": "string" }, "reasoning_efforts": { "items": { "$ref": "#/definitions/ReasoningEffort" }, "type": [ "array", "null" ] }, "speech": { "anyOf": [ { "$ref": "#/definitions/SpeechModel" }, { "type": "null" } ], "description": "Set on speech models only; see [`SpeechModel`]." }, "supports_cache": { "type": [ "boolean", "null" ] }, "supports_structured_output": { "type": [ "boolean", "null" ] }, "supports_thinking": { "type": [ "boolean", "null" ] }, "supports_thinking_off": { "description": "Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.", "type": [ "boolean", "null" ] }, "supports_tools": { "type": [ "boolean", "null" ] }, "supports_vision": { "type": [ "boolean", "null" ] }, "supports_xhigh": { "type": [ "boolean", "null" ] }, "thinking_budgets": { "additionalProperties": { "format": "uint64", "minimum": 0, "type": "integer" }, "type": [ "object", "null" ] } }, "required": [ "context_window", "id", "max_output_tokens", "provider" ], "type": "object" }, "Pricing": { "properties": { "cache_read": { "format": "double", "type": [ "number", "null" ] }, "cache_write": { "format": "double", "type": [ "number", "null" ] }, "input": { "format": "double", "type": [ "number", "null" ] }, "output": { "format": "double", "type": [ "number", "null" ] } }, "type": "object" }, "PromptCacheIntent": { "description": "Why a request's stable prefix is shareable across sessions: `surface_digest` is the caller's content identity for the text before the boundary (`sha256:`). A routing/accounting hint only — never a provider cache handle, and never evidence of a cache hit.", "properties": { "surface_digest": { "type": "string" } }, "required": [ "surface_digest" ], "type": "object" }, "PromptSection": { "description": "One ordered system-prompt section. `cache_boundary` marks the end of a cumulative prefix worth its own provider cache entry (the frozen agent-profile surface). Adapters without a boundary concept read the sections joined with \"\\n\\n\", which must equal `system_prompt` when both forms are sent.", "properties": { "cache_boundary": { "default": false, "type": "boolean" }, "text": { "type": "string" } }, "required": [ "text" ], "type": "object" }, "ReasoningEffort": { "description": "One provider-native reasoning effort advertised for a specific model.\n\nValues intentionally remain strings: provider catalogs can add efforts without requiring a router-wide enum release first.", "properties": { "description": { "type": [ "string", "null" ] }, "effort": { "type": "string" } }, "required": [ "effort" ], "type": "object" }, "ResponseFormat": { "properties": { "schema": true, "type": { "type": "string" } }, "required": [ "type" ], "type": "object" }, "SpeechModality": { "description": "What a speech model does. Chat models carry no `speech` block.", "oneOf": [ { "description": "Speech to text, served through `router::transcribe`.", "enum": [ "stt" ], "type": "string" }, { "description": "Text to speech, served through `router::speak`.", "enum": [ "tts" ], "type": "string" } ] }, "SpeechModel": { "description": "Facts about a speech model. Present only on models served through `router::transcribe` / `router::speak`; such models report `context_window` and `max_output_tokens` as 0.", "properties": { "languages": { "description": "BCP-47 tags of the languages the model handles; empty when the provider does not say.", "items": { "type": "string" }, "type": "array" }, "modality": { "$ref": "#/definitions/SpeechModality" }, "streaming": { "default": false, "description": "Realtime input (stt) or streamed audio output (tts) is available.", "type": "boolean" } }, "required": [ "modality" ], "type": "object" }, "StopReason": { "enum": [ "end", "length", "function_call", "aborted", "error" ], "type": "string" }, "StreamChannelRef": { "properties": { "access_key": { "type": "string" }, "channel_id": { "type": "string" }, "direction": { "$ref": "#/definitions/ChannelDirection" } }, "required": [ "access_key", "channel_id", "direction" ], "type": "object" }, "ThinkingLevel": { "description": "\"off\" asks the provider not to reason at all and is honoured only where the model advertises `Model::supports_thinking_off` (elsewhere the provider falls back to its own lowest setting, or omits the parameter, and warns). \"minimal\" requests the lowest reasoning effort and needs only `thinking` support; levels map to provider-native knobs via `Model::thinking_budgets`.", "enum": [ "off", "minimal", "low", "medium", "high", "xhigh" ], "type": "string" }, "Usage": { "properties": { "cache_read": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "cache_write": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "cost_usd": { "format": "double", "type": [ "number", "null" ] }, "input": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "output": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "reasoning": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] } }, "type": "object" }, "UserMessage": { "properties": { "content": { "items": { "$ref": "#/definitions/ContentBlock" }, "type": "array" }, "role": { "$ref": "#/definitions/UserRoleTag" }, "timestamp": { "format": "int64", "type": "integer" } }, "required": [ "content", "role", "timestamp" ], "type": "object" }, "UserRoleTag": { "description": "Single-variant role tags: exact-match on deserialize, correct wire string on serialize, and they let `AgentMessage` be an untagged union.", "enum": [ "user" ], "type": "string" } }, "description": "Input of a provider worker's `provider::::stream` iii function — what the router forwards per attempt. (No `PartialEq`: `iii_sdk::StreamChannelRef` doesn't implement it.)", "properties": { "cache_intent": { "anyOf": [ { "$ref": "#/definitions/PromptCacheIntent" }, { "type": "null" } ] }, "max_output_tokens": { "format": "uint64", "minimum": 0, "type": [ "integer", "null" ] }, "messages": { "items": { "$ref": "#/definitions/AgentMessage" }, "type": "array" }, "model": { "type": "string" }, "model_meta": { "anyOf": [ { "$ref": "#/definitions/Model" }, { "type": "null" } ] }, "provider_options": true, "resolution_key": { "type": [ "string", "null" ] }, "response_format": { "anyOf": [ { "$ref": "#/definitions/ResponseFormat" }, { "type": "null" } ] }, "session_id": { "description": "Stable conversation identity for provider cache affinity.", "type": [ "string", "null" ] }, "system_prompt": { "type": [ "string", "null" ] }, "system_sections": { "description": "Ordered sections behind `system_prompt` (always forwarded alongside the flat string, which equals the sections joined with \"\\n\\n\").", "items": { "$ref": "#/definitions/PromptSection" }, "type": [ "array", "null" ] }, "thinking_level": { "anyOf": [ { "$ref": "#/definitions/ThinkingLevel" }, { "type": "null" } ] }, "tools": { "items": { "$ref": "#/definitions/AgentFunction" }, "type": [ "array", "null" ] }, "writer_ref": { "$ref": "#/definitions/StreamChannelRef" } }, "required": [ "messages", "model", "writer_ref" ], "title": "ProviderStreamInput", "type": "object" }, "response_schema": { "$schema": "http://json-schema.org/draft-07/schema#", "description": "Output of a provider's `provider::::stream` (spec § stream contract): the function streams frames to `writer_ref` and returns this ack.", "properties": { "ok": { "type": "boolean" } }, "required": [ "ok" ], "title": "ProviderStreamOutput", "type": "object" } } ], "triggers": [] } ```