# provider-elevenlabs

> ElevenLabs speech provider worker; implements provider::elevenlabs::transcribe and provider::elevenlabs::speak behind llm-router's router::transcribe and router::speak.

| field | value |
|-------|-------|
| version | 0.1.1 |
| type | binary |
| license | Apache-2.0 |
| repo | https://github.com/iii-hq/workers |
| supported_targets | x86_64-apple-darwin, aarch64-apple-darwin, x86_64-pc-windows-msvc, aarch64-pc-windows-msvc, x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-unknown-linux-musl, armv7-unknown-linux-gnueabihf |
| author | iii |

## installation

```sh
iii trigger compose::add worker=provider-elevenlabs@0.1.1
```

## dependencies

- `llm-router` @ `latest`
- `state` @ `latest`

## readme

# provider-elevenlabs

ElevenLabs speech behind [llm-router](https://github.com/iii-hq/workers/tree/main/llm-router): speech to text with
Scribe and text to speech with the Eleven voices. The worker implements the
speech half of the provider protocol, `provider::elevenlabs::transcribe`
and `provider::elevenlabs::speak`, so `router::transcribe` and
`router::speak` reach ElevenLabs without any caller knowing its API. The
voice worker uses them with `stt.backend: router` / `tts.backend: router`.

There is no chat surface: the provider declares only speech models, so
chat routing never lands here and chat pickers never list its models.

## Install

```bash
iii trigger compose::add worker=provider-elevenlabs
```

Then put the key where the router reads it: `ELEVENLABS_API_KEY` in the
router's environment, or paste it into the `elevenlabs` block of the
llm-router entry in the console's Settings. The catalog fills the moment a
key lands.

## Models

| Family | Ids | Notes |
|---|---|---|
| Speech to text | `scribe_v1`, `scribe_v1_experimental` | 99 languages, detected automatically; word timings become sentence segments. |
| Text to speech | every model `GET /v1/models` marks `can_do_text_to_speech` (`eleven_multilingual_v2`, `eleven_flash_v2_5`, `eleven_v3`, ...) | Each carries the language ids it speaks. `eleven_multilingual_v2` is the default. |

`router::models::list` with `modality: "stt"` or `"tts"` lists them;
`router::models::supports` answers `stt`, `tts` and `streaming` (no: this
worker serves batch requests only).

## Surfaces

| Function | Request | Response |
|---|---|---|
| `provider::elevenlabs::transcribe` | `{model?, audio_base64, mime?, language?}` | `{model, text, segments[{text, start_secs, end_secs}], language, duration_secs}` |
| `provider::elevenlabs::speak` | `{model?, text, voice?, format?, language?, speed?}` | `{model, audio_base64, mime, voice}` |
| `provider::elevenlabs::refresh_models` | `{}` | `{ok, count}` |

- `voice` is a voice id or the name of a voice on the account (`George`,
  `Sarah`); names resolve through `GET /v1/voices`. Omitted, George speaks.
- `format` is `mp3` (default, `audio/mpeg`), `wav` (16 kHz PCM wrapped in a
  WAV container, `audio/wav`), `pcm16` (`audio/pcm`) or `opus` (`audio/ogg`).
- `language` on transcribe is an ISO 639-1 or 639-3 hint; on speak it is
  only sent to models that accept one.
- Errors carry stable prefixes: `provider/not_configured`,
  `provider/auth_expired`, `provider/quota_exceeded`, `provider/rate_limited`,
  `provider/invalid_input`, `provider/upstream_transient`.

## Configuration

Nothing in this worker's own config file. The credential and `api_url`
(default `https://api.elevenlabs.io/v1`; the EU, India and Singapore
residency hosts work too) resolve per request through
`router::provider::resolve`.

## Tests

```bash
cargo test                      # unit tests + wire-schema goldens
UPDATE_GOLDENS=1 cargo test     # regenerate tests/golden after a surface change
```

## Running

```bash
cargo run -- --url ws://127.0.0.1:49134
```

## api reference

```json
{
  "functions": [
    {
      "description": "Internal: re-declare with llm-router after it restarts (bound to the router::ready trigger).",
      "metadata": {
        "internal": true
      },
      "name": "provider::elevenlabs::on_router_ready",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Event delivered to a provider's `provider::<id>::on_router_ready` (the `router::ready` trigger payload, currently `{}`). Unknown fields are ignored.",
        "title": "RouterReadyEvent",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Ack returned by a provider's `provider::<id>::on_router_ready`.",
        "properties": {
          "ok": {
            "type": "boolean"
          }
        },
        "required": [
          "ok"
        ],
        "title": "ProviderReadyAck",
        "type": "object"
      }
    },
    {
      "description": "Refresh the ElevenLabs catalog slice from GET /v1/models (text-to-speech models with their languages) plus the Scribe speech-to-text models, through router::models::reconcile.",
      "metadata": {
        "internal": true
      },
      "name": "provider::elevenlabs::refresh_models",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Input of a provider's `provider::<id>::refresh_models` — takes no arguments. A struct (not `Value`) keeps the request schema concrete; unknown fields (e.g. the engine-injected `_caller_worker_id`) are ignored.",
        "title": "RefreshModelsRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Output of `provider::<id>::refresh_models`.",
        "properties": {
          "count": {
            "format": "uint",
            "minimum": 0,
            "type": "integer"
          },
          "ok": {
            "type": "boolean"
          }
        },
        "required": [
          "count",
          "ok"
        ],
        "title": "RefreshModelsResponse",
        "type": "object"
      }
    },
    {
      "description": "Text to speech on the ElevenLabs voices behind router::speak: text and a voice id or name in, base64 audio out (mp3, wav, pcm16, opus).",
      "metadata": {
        "internal": true
      },
      "name": "provider::elevenlabs::speak",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "format": {
            "default": null,
            "description": "`mp3` (default), `wav`, `pcm16`, `opus`.",
            "type": [
              "string",
              "null"
            ]
          },
          "language": {
            "default": null,
            "description": "ISO 639-1 language to enforce; ignored by models that pick it from the text.",
            "type": [
              "string",
              "null"
            ]
          },
          "model": {
            "default": null,
            "description": "Text-to-speech model id; `eleven_multilingual_v2` when omitted.",
            "type": [
              "string",
              "null"
            ]
          },
          "speed": {
            "default": null,
            "description": "Speaking-rate multiplier, 0.7 to 1.2; 1.0 is the voice's own pace.",
            "format": "double",
            "type": [
              "number",
              "null"
            ]
          },
          "text": {
            "description": "Text to speak.",
            "type": "string"
          },
          "voice": {
            "default": null,
            "description": "Voice id, or the name of a voice on the account; George when omitted.",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "required": [
          "text"
        ],
        "title": "SpeakRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "audio_base64": {
            "type": "string"
          },
          "mime": {
            "type": "string"
          },
          "model": {
            "type": "string"
          },
          "voice": {
            "description": "The voice id used.",
            "type": "string"
          }
        },
        "required": [
          "audio_base64",
          "mime",
          "model",
          "voice"
        ],
        "title": "SpeakResponse",
        "type": "object"
      }
    },
    {
      "description": "Speech to text on ElevenLabs Scribe behind router::transcribe: base64 audio in, text with timed segments and the detected language out.",
      "metadata": {
        "internal": true
      },
      "name": "provider::elevenlabs::transcribe",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "audio_base64": {
            "description": "The audio, base64. Any common container; `mime` names it.",
            "type": "string"
          },
          "language": {
            "default": null,
            "description": "ISO 639-1 or 639-3 language code; detected when omitted.",
            "type": [
              "string",
              "null"
            ]
          },
          "mime": {
            "default": null,
            "description": "MIME type of the audio; `audio/wav` when omitted.",
            "type": [
              "string",
              "null"
            ]
          },
          "model": {
            "default": null,
            "description": "Speech-to-text model id; `scribe_v1` when omitted.",
            "type": [
              "string",
              "null"
            ]
          },
          "prompt": {
            "default": null,
            "description": "Accepted for the shared contract; Scribe takes no prompt.",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "required": [
          "audio_base64"
        ],
        "title": "TranscribeRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "TranscriptSegment": {
            "properties": {
              "end_secs": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "start_secs": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "text": {
                "type": "string"
              }
            },
            "required": [
              "text"
            ],
            "type": "object"
          }
        },
        "properties": {
          "duration_secs": {
            "format": "double",
            "type": [
              "number",
              "null"
            ]
          },
          "language": {
            "description": "Language Scribe detected (ISO 639-3, e.g. `eng`).",
            "type": [
              "string",
              "null"
            ]
          },
          "model": {
            "type": "string"
          },
          "segments": {
            "description": "Sentences, cut at punctuation or a pause, with word timings.",
            "items": {
              "$ref": "#/definitions/TranscriptSegment"
            },
            "type": "array"
          },
          "text": {
            "type": "string"
          }
        },
        "required": [
          "model",
          "text"
        ],
        "title": "TranscribeResponse",
        "type": "object"
      }
    }
  ],
  "triggers": []
}
```