provider-gemini
v0.2.0Google Gemini Interactions API provider worker; implements provider::gemini::stream and provider::gemini::refresh_models behind llm-router.
Install
iii trigger compose::add worker=provider-geminiRun in your project with the iii engine running. Exact versions are immutable and digest-pinned.
Full markdown
/registry/workers/provider-gemini.md. Paste it into an LLM prompt or pipe it through curl from a worker.Install
iii trigger compose::add worker=provider-gemini@0.2.0Dependencies
Readme
provider-gemini
Google Gemini models behind
llm-router, on the
Gemini Interactions API. Install this worker next to the router, provide a
Gemini API key, and the live Gemini chat catalog appears behind router::chat
and router::complete: streaming text, thought summaries, function calls
(with images in function results), structured JSON output and implicit prompt
caching. The router invokes the provider functions worker-to-worker, and
iii-permissions.yaml blocks agents from calling them directly.
Install
iii call compose::add worker=provider-gemini worker=llm-routeriii call compose::add resolves the workers and their dependencies, writes
exact declarations to worker-compose.yaml, and reconciles the Compose project.
Quickstart
Create a key at Google AI Studio and put
it in the gemini slice of the engine's llm-router configuration entry, or
export GEMINI_API_KEY where the router runs:
{ "providers": { "gemini": { "api_key": "AIza…" } } }The provider discovers the catalog as soon as a key is configured. Then call the router with a Gemini model id:
const result = await iii.call({
function_id: 'router::complete',
payload: {
model: 'gemini-3.8-flash',
thinking_level: 'low',
messages: [
{
role: 'user',
content: [{ type: 'text', text: 'Summarise this diff in one line.' }],
timestamp: Date.now(),
},
],
},
timeoutMs: 320_000,
});For token-by-token output, call router::chat with an iii channel as described
in llm-router's Quickstart.
router::provider::list reports a default_model for gemini: the newest
catalog model of the best-ranked variant, flash first, then hydrogen.
Configuration
All provider configuration lives in the router's llm-router entry:
"gemini": {
"api_key": "AIza…", // or GEMINI_API_KEY
"api_url": "https://generativelanguage.googleapis.com/v1beta/interactions", // full endpoint URL
"max_tokens": 65536 // thinking + output; capped at 65536
}The models listing is read from the /models sibling of api_url, so a proxy
that serves both routes works with one override, as long as api_url ends in
/interactions. Any other path gets no model listing, and the key is never
sent to the public host in its place. The default is v1beta
because v1 does not serve the preview models or the -latest aliases.
Worker environment variables:
| Variable | Default | Meaning |
|---|---|---|
GEMINI_API_KEY |
unset | Credential fallback declared to llm-router |
PROVIDER_READ_TIMEOUT_SECS |
900 |
Upstream read timeout and the deadline for generation to start; Google asks for at least 15 minutes on hard Gemini 4 prompts |
III_URL |
ws://127.0.0.1:49134 |
Engine WebSocket when --url is not set |
The binary also accepts the standard --url, --manifest, and --config
flags; --config is ignored with a warning.
Models, thinking and accounting
GET /models is the source of truth for ids, context windows, output limits
and display names. Non-chat models (speech, image, embedding, live, robotics
and similar) are filtered out. Prices come from Google's pricing page and are
kept in src/curated.rs;
the -latest aliases have no price.
Every listed model thinks. thinking_level maps one to one onto Gemini's
levels, clamped to what the model accepts, with a warning when it changes:
gemini-4-neon and gemini-4-hydrogen take low to xhigh;
gemini-3.6-flash, gemini-3.5-flash, gemini-3.5-flash-lite,
gemini-3.1-flash-lite and gemini-3-flash-preview take minimal to high;
everything else takes low to high. With no level the model's own default applies. Thought summaries are
always requested, so the console shows the thinking as it streams.
Requests are stateless (store: false): the router's history is sent in full
every turn, and Google's implicit cache bills the repeated prefix at the cached
rate. Usage maps total_cached_tokens to cache_read, the rest of the input
to input, and output plus thought tokens to output (Google bills thinking
at the output rate); thought tokens are also reported as reasoning.
Known limits
- API key only. OAuth credentials are refused with a clear error; the Gemini API needs a project header the router's OAuth credential does not carry.
- No token counter.
router::count_tokensanswersno_token_counterfor Gemini models and callers use their own estimate. - Thinking cannot be switched off.
thinking_level: offruns at the model's lowest level and says so in a warning. - No long decode.
max_output_tokensis capped at 65,536; Gemini's continuation loop for longer outputs is not implemented. gemini-2.5-*is not listed. Those ids answer "no longer available to new users" for new keys.- Messages from another provider are replayed behind a placeholder thought step, because Gemini rejects a function call in the current turn that has no thought at the start of its model run. Gemini's own thought signatures are replayed as received; thinking from other providers is dropped.
- The router ends any provider stream after its
stream_timeout_ms(600 s by default); raise it for prompts that need Gemini 4's full 15 minutes.
Tests
cargo test
GEMINI_API_KEY=… cargo test --test live_smoke -- --ignored --nocaptureThe unit suite replays trimmed captures of the live API from
tests/fixtures/ and makes no external calls. The integration suite runs a
real engine, the real router and this provider against a local stub upstream.
The live suite (ignored by default, five short requests) checks a text turn,
an agent_call tool loop, a tool call made by another provider and a
history that opens on a tool call.
API reference (JSON)
{
"functions": [
{
"description": "Cancel the in-flight upstream stream for a request_id (router::abort fan-out), stopping billed generation immediately.",
"metadata": {
"internal": true
},
"name": "provider::gemini::abort",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of a provider's `provider::<id>::abort`: actively cancel the in-flight upstream stream for `request_id` (the router's `request_id`, delivered to the provider as `resolution_key`) so billed generation stops immediately instead of waiting for the provider to notice the closed channel on its next write.",
"properties": {
"request_id": {
"type": "string"
}
},
"required": [
"request_id"
],
"title": "ProviderAbortRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of `provider::<id>::abort`. `aborted: false` means the request was unknown — already finished, never started, or aborted before (idempotent).",
"properties": {
"aborted": {
"type": "boolean"
}
},
"required": [
"aborted"
],
"title": "ProviderAbortResponse",
"type": "object"
}
},
{
"description": "Internal: re-declares this provider and refreshes its catalog on every router::on-ready event.",
"metadata": {
"internal": true
},
"name": "provider::gemini::on_router_ready",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "The router's ready event: what `router::on-ready` returns, and what a provider's `provider::<id>::on_router_ready` receives. Currently `{}`; unknown fields are ignored.",
"title": "RouterReadyEvent",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Ack returned by a provider's `provider::<id>::on_router_ready`.",
"properties": {
"ok": {
"type": "boolean"
}
},
"required": [
"ok"
],
"title": "ProviderReadyAck",
"type": "object"
}
},
{
"description": "Reconcile the Google Gemini catalog slice through the router: list the upstream models, enrich each with local metadata, and return the model count written.",
"metadata": {
"internal": true
},
"name": "provider::gemini::refresh_models",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of a provider's `provider::<id>::refresh_models` — takes no arguments. A struct (not `Value`) keeps the request schema concrete; unknown fields (e.g. the engine-injected `_caller_worker_id`) are ignored.",
"title": "RefreshModelsRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of `provider::<id>::refresh_models`.",
"properties": {
"count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"ok": {
"type": "boolean"
}
},
"required": [
"count",
"ok"
],
"title": "RefreshModelsResponse",
"type": "object"
}
},
{
"description": "Stream a Google Gemini turn: resolve credentials, call the upstream Interactions API, and relay AssistantMessageEvent frames to writer_ref.",
"metadata": {
"internal": true
},
"name": "provider::gemini::stream",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"AgentFunction": {
"description": "Function invocation schema — what a provider sees as a `tools` array entry (README § Function invocation schema; adapter boundary). These describe iii functions exposed to the model, not provider-native tools.",
"properties": {
"description": {
"type": "string"
},
"execution_mode": {
"type": [
"string",
"null"
]
},
"label": {
"type": [
"string",
"null"
]
},
"name": {
"type": "string"
},
"parameters": true
},
"required": [
"description",
"name",
"parameters"
],
"type": "object"
},
"AgentMessage": {
"anyOf": [
{
"$ref": "#/definitions/AssistantMessage"
},
{
"$ref": "#/definitions/FunctionResultMessage"
},
{
"$ref": "#/definitions/CustomMessage"
},
{
"$ref": "#/definitions/UserMessage"
}
],
"description": "The canonical transcript message union. Untagged: the single-variant role tags disambiguate deserialization."
},
"AssistantMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"error_kind": {
"anyOf": [
{
"$ref": "#/definitions/ErrorKind"
},
{
"type": "null"
}
]
},
"error_message": {
"type": [
"string",
"null"
]
},
"model": {
"type": "string"
},
"native_stop_reason": {
"type": [
"string",
"null"
]
},
"provider": {
"type": "string"
},
"role": {
"$ref": "#/definitions/AssistantRoleTag"
},
"stop_reason": {
"$ref": "#/definitions/StopReason"
},
"timestamp": {
"format": "int64",
"type": "integer"
},
"usage": {
"anyOf": [
{
"$ref": "#/definitions/Usage"
},
{
"type": "null"
}
]
},
"warnings": {
"items": {
"type": "string"
},
"type": [
"array",
"null"
]
}
},
"required": [
"content",
"model",
"provider",
"role",
"stop_reason",
"timestamp"
],
"type": "object"
},
"AssistantRoleTag": {
"enum": [
"assistant"
],
"type": "string"
},
"ChannelDirection": {
"enum": [
"read",
"write"
],
"type": "string"
},
"ContentBlock": {
"description": "Content blocks — the atomic units of message content (README § Content blocks).",
"oneOf": [
{
"properties": {
"text": {
"type": "string"
},
"type": {
"enum": [
"text"
],
"type": "string"
}
},
"required": [
"text",
"type"
],
"type": "object"
},
{
"properties": {
"attachment_id": {
"description": "Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.",
"type": [
"string",
"null"
]
},
"data": {
"type": "string"
},
"mime": {
"type": "string"
},
"type": {
"enum": [
"image"
],
"type": "string"
}
},
"required": [
"data",
"mime",
"type"
],
"type": "object"
},
{
"description": "Reference to an original upload in the session-manager attachment store. A REFERENCE ONLY (never inline bytes); the harness strips it before `router::chat`. Tolerated here so a stray block deserializes instead of failing the request; every counter/mapping emits nothing for it.",
"properties": {
"attachment_id": {
"type": "string"
},
"mime": {
"type": "string"
},
"name": {
"type": "string"
},
"size": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": {
"enum": [
"file"
],
"type": "string"
}
},
"required": [
"attachment_id",
"mime",
"name",
"size",
"type"
],
"type": "object"
},
{
"properties": {
"signature": {
"type": [
"string",
"null"
]
},
"text": {
"type": "string"
},
"type": {
"enum": [
"thinking"
],
"type": "string"
}
},
"required": [
"text",
"type"
],
"type": "object"
},
{
"description": "Opaque redacted thinking payload — replayed verbatim on the Anthropic wire.",
"properties": {
"data": {
"type": "string"
},
"type": {
"enum": [
"redacted_thinking"
],
"type": "string"
}
},
"required": [
"data",
"type"
],
"type": "object"
},
{
"properties": {
"arguments": true,
"function_id": {
"type": "string"
},
"id": {
"type": "string"
},
"type": {
"enum": [
"function_call"
],
"type": "string"
}
},
"required": [
"arguments",
"function_id",
"id",
"type"
],
"type": "object"
},
{
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"function_call_id": {
"type": "string"
},
"is_error": {
"type": [
"boolean",
"null"
]
},
"type": {
"enum": [
"function_result"
],
"type": "string"
}
},
"required": [
"content",
"function_call_id",
"type"
],
"type": "object"
}
]
},
"CustomMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"custom_type": {
"type": "string"
},
"details": true,
"display": {
"type": [
"string",
"null"
]
},
"role": {
"$ref": "#/definitions/CustomRoleTag"
},
"timestamp": {
"format": "int64",
"type": "integer"
}
},
"required": [
"content",
"custom_type",
"role",
"timestamp"
],
"type": "object"
},
"CustomRoleTag": {
"enum": [
"custom"
],
"type": "string"
},
"ErrorKind": {
"enum": [
"auth_expired",
"rate_limited",
"context_overflow",
"transient",
"permanent"
],
"type": "string"
},
"FunctionResultMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"details": true,
"function_call_id": {
"type": "string"
},
"function_id": {
"type": "string"
},
"is_error": {
"type": "boolean"
},
"role": {
"$ref": "#/definitions/FunctionResultRoleTag"
},
"timestamp": {
"format": "int64",
"type": "integer"
}
},
"required": [
"content",
"details",
"function_call_id",
"function_id",
"is_error",
"role",
"timestamp"
],
"type": "object"
},
"FunctionResultRoleTag": {
"enum": [
"function_result"
],
"type": "string"
},
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"reasoning_efforts": {
"items": {
"$ref": "#/definitions/ReasoningEffort"
},
"type": [
"array",
"null"
]
},
"speech": {
"anyOf": [
{
"$ref": "#/definitions/SpeechModel"
},
{
"type": "null"
}
],
"description": "Set on speech models only; see [`SpeechModel`]."
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_thinking_off": {
"description": "Whether `thinking_level: off` can switch the model's reasoning off. `Some(true)`: a native off switch exists. `Some(false)`: the model always reasons, or the provider has no switch. `None`: the provider does not know; consoles show the choice disabled.",
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"PromptCacheIntent": {
"description": "Why a request's stable prefix is shareable across sessions: `surface_digest` is the caller's content identity for the text before the boundary (`sha256:<hex>`). A routing/accounting hint only — never a provider cache handle, and never evidence of a cache hit.",
"properties": {
"surface_digest": {
"type": "string"
}
},
"required": [
"surface_digest"
],
"type": "object"
},
"PromptSection": {
"description": "One ordered system-prompt section. `cache_boundary` marks the end of a cumulative prefix worth its own provider cache entry (the frozen agent-profile surface). Adapters without a boundary concept read the sections joined with \"\\n\\n\", which must equal `system_prompt` when both forms are sent.",
"properties": {
"cache_boundary": {
"default": false,
"type": "boolean"
},
"text": {
"type": "string"
}
},
"required": [
"text"
],
"type": "object"
},
"ReasoningEffort": {
"description": "One provider-native reasoning effort advertised for a specific model.\n\nValues intentionally remain strings: provider catalogs can add efforts without requiring a router-wide enum release first.",
"properties": {
"description": {
"type": [
"string",
"null"
]
},
"effort": {
"type": "string"
}
},
"required": [
"effort"
],
"type": "object"
},
"ResponseFormat": {
"properties": {
"schema": true,
"type": {
"type": "string"
}
},
"required": [
"type"
],
"type": "object"
},
"SpeechModality": {
"description": "What a speech model does. Chat models carry no `speech` block.",
"oneOf": [
{
"description": "Speech to text, served through `router::transcribe`.",
"enum": [
"stt"
],
"type": "string"
},
{
"description": "Text to speech, served through `router::speak`.",
"enum": [
"tts"
],
"type": "string"
}
]
},
"SpeechModel": {
"description": "Facts about a speech model. Present only on models served through `router::transcribe` / `router::speak`; such models report `context_window` and `max_output_tokens` as 0.",
"properties": {
"languages": {
"description": "BCP-47 tags of the languages the model handles; empty when the provider does not say.",
"items": {
"type": "string"
},
"type": "array"
},
"modality": {
"$ref": "#/definitions/SpeechModality"
},
"streaming": {
"default": false,
"description": "Realtime input (stt) or streamed audio output (tts) is available.",
"type": "boolean"
}
},
"required": [
"modality"
],
"type": "object"
},
"StopReason": {
"enum": [
"end",
"length",
"function_call",
"aborted",
"error"
],
"type": "string"
},
"StreamChannelRef": {
"properties": {
"access_key": {
"type": "string"
},
"channel_id": {
"type": "string"
},
"direction": {
"$ref": "#/definitions/ChannelDirection"
}
},
"required": [
"access_key",
"channel_id",
"direction"
],
"type": "object"
},
"ThinkingLevel": {
"description": "\"off\" asks the provider not to reason at all and is honoured only where the model advertises `Model::supports_thinking_off` (elsewhere the provider falls back to its own lowest setting, or omits the parameter, and warns). \"minimal\" requests the lowest reasoning effort and needs only `thinking` support; levels map to provider-native knobs via `Model::thinking_budgets`.",
"enum": [
"off",
"minimal",
"low",
"medium",
"high",
"xhigh"
],
"type": "string"
},
"Usage": {
"properties": {
"cache_read": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cache_write": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"reasoning": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"UserMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"role": {
"$ref": "#/definitions/UserRoleTag"
},
"timestamp": {
"format": "int64",
"type": "integer"
}
},
"required": [
"content",
"role",
"timestamp"
],
"type": "object"
},
"UserRoleTag": {
"description": "Single-variant role tags: exact-match on deserialize, correct wire string on serialize, and they let `AgentMessage` be an untagged union.",
"enum": [
"user"
],
"type": "string"
}
},
"description": "Input of a provider worker's `provider::<id>::stream` iii function — what the router forwards per attempt. (No `PartialEq`: `iii_sdk::StreamChannelRef` doesn't implement it.)",
"properties": {
"cache_intent": {
"anyOf": [
{
"$ref": "#/definitions/PromptCacheIntent"
},
{
"type": "null"
}
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"messages": {
"items": {
"$ref": "#/definitions/AgentMessage"
},
"type": "array"
},
"model": {
"type": "string"
},
"model_meta": {
"anyOf": [
{
"$ref": "#/definitions/Model"
},
{
"type": "null"
}
]
},
"provider_options": true,
"resolution_key": {
"type": [
"string",
"null"
]
},
"response_format": {
"anyOf": [
{
"$ref": "#/definitions/ResponseFormat"
},
{
"type": "null"
}
]
},
"session_id": {
"description": "Stable conversation identity for provider cache affinity.",
"type": [
"string",
"null"
]
},
"system_prompt": {
"type": [
"string",
"null"
]
},
"system_sections": {
"description": "Ordered sections behind `system_prompt` (always forwarded alongside the flat string, which equals the sections joined with \"\\n\\n\").",
"items": {
"$ref": "#/definitions/PromptSection"
},
"type": [
"array",
"null"
]
},
"thinking_level": {
"anyOf": [
{
"$ref": "#/definitions/ThinkingLevel"
},
{
"type": "null"
}
]
},
"tools": {
"items": {
"$ref": "#/definitions/AgentFunction"
},
"type": [
"array",
"null"
]
},
"writer_ref": {
"$ref": "#/definitions/StreamChannelRef"
}
},
"required": [
"messages",
"model",
"writer_ref"
],
"title": "ProviderStreamInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of a provider's `provider::<id>::stream` (spec § stream contract): the function streams frames to `writer_ref` and returns this ack.",
"properties": {
"ok": {
"type": "boolean"
}
},
"required": [
"ok"
],
"title": "ProviderStreamOutput",
"type": "object"
}
}
]
}