# judge-openai

> OpenAI Decisions evaluation (judge-openai::evaluate), model listing (judge-openai::models::list) and caller-scoped cancellation (judge-openai::cancel), with typed usage, provider errors, per-call retries and worker-owned credentials.

| field | value |
|-------|-------|
| version | 0.2.3 |
| type | binary |
| license | Apache-2.0 |
| repo | https://github.com/iii-hq/workers |
| supported_targets | x86_64-apple-darwin, aarch64-apple-darwin, x86_64-pc-windows-msvc, aarch64-pc-windows-msvc, x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-unknown-linux-musl, armv7-unknown-linux-gnueabihf |
| author | iii |

## installation

```sh
iii trigger compose::add worker=judge-openai@0.2.3
```

## configuration

```yaml
- api_key: null
  max_request_bytes: 8388608
  max_response_bytes: 8388608
  max_timeout_ms: 300000
  model: gpt-6-luna
```

## dependencies

- `configuration` @ `latest`

## readme

# judge-openai

OpenAI Decisions API (`POST /v1/decisions`) provider for the
[`judge`](https://github.com/iii-hq/workers/tree/main/judge) hub. It turns JSON
state and Noul, Choice or Score questions into typed, validated answers from
`gpt-6-luna`, sharing the OpenAI credential and HTTP capacity across callers,
with model listing and caller-owned cancellation. Callers go through
`judge::evaluate` (select it with `provider: "openai"` or the session's judge
picker); the `judge-openai::*` functions below are the same contract addressed
directly. They are registered as internal, so default discovery
(`engine::functions::list`, `directory::search_functions`) shows only the hub.

The Decisions API is in public beta. The reply format may still change; this
worker ignores unknown reply fields and rejects anything it cannot validate as
`invalid_response`.

## Install

```bash
iii trigger compose::add worker=judge-openai
```

Compose starts the provider with its `configuration` dependency; add the
[`judge`](https://github.com/iii-hq/workers/tree/main/judge) hub as well to call
it through `judge::evaluate`. Use engine **`iii/v0.24.0-rc.2`**, the verified
release with `configuration::ensure` used by this repository's CI. Wait for the
provider's functions to register before making a call.

## Quickstart

In the Console, open **Settings → Workers → Judge OpenAI** and set **API key**
(the form checks the saved key against `judge-openai::models::list` and fills the
**Default model** select from it), preferably as a `secret://OPENAI_API_KEY`
reference to the
[`secrets`](https://github.com/iii-hq/workers/tree/main/secrets) worker. See
[Configuration](#configuration) for the environment fallback and precedence.

Ask whether a support ticket needs urgent attention:

```bash
iii trigger judge::evaluate --timeout-ms 65000 --json '{
  "provider": "openai",
  "timeout_ms": 60000,
  "evaluations": [{
    "id": "ticket-42",
    "state": {"ticket": "Production checkout is down for all customers."},
    "questions": {
      "urgent": {
        "type": "noul",
        "instructions": "Does this describe an urgent production outage?",
        "criteria": {"true": "A critical production workflow is blocked."}
      }
    }
  }]
}'
```

A successful response looks like this; values, usage and timing are illustrative:

```json
{
  "status": "ok", "model": "gpt-6-luna",
  "results": {"ticket-42": {
    "answers": {"urgent": {"type": "noul", "noul": 0.98}},
    "usage": {"input_tokens": 120, "output_tokens": 0}
  }},
  "stats": {"attempts": 1, "requests": 1, "questions": 1,
    "input_tokens": 120, "output_tokens": 0, "elapsed_ms": 250,
    "usage_complete": true}
}
```

Check `status` before reading answers. Errors return `status: "error"` and a
`code` with no partial results. See the hub's
[mixed Noul/Choice/Score example](https://github.com/iii-hq/workers/blob/main/judge/reference.md#evaluate),
[result and error handling](https://github.com/iii-hq/workers/blob/main/judge/reference.md#handle-results-and-failures),
[model listing](https://github.com/iii-hq/workers/blob/main/judge/reference.md#list-models) and
[cancellation](https://github.com/iii-hq/workers/blob/main/judge/reference.md#cancellation).

## How questions map to Decisions

Each evaluation is one `POST https://api.openai.com/v1/decisions` with all of its
questions; evaluations run in parallel on at most four connections per worker.
Each question is sent with `name` set to its question id, and answers are matched
back by name.

| judge | Decisions request | Answer |
| --- | --- | --- |
| `state` | `input`: a string as is; an object or array as compact JSON text | — |
| `noul` | `predicate`; `criteria.true` / `criteria.false` are appended to the instructions as `True means: …` / `False means: …` | `probability` → `noul` |
| `choice` | `choice` with `choices: [{value: key, description}]` in key order | `choice`, `probabilities` and `confidence` as returned |
| `score` | `score` with `levels: [{label}]` in order; a blank level is labelled with its index | `score`, `probabilities` keyed `"0"`…`"n-1"`, `confidence` as returned, `legend` = the request's levels |

Text is trimmed, structured instructions and descriptions are sent as compact
JSON, and blank or null instructions use a default (`Is this true of the input?`,
`Which choice best fits the input?`, `Which level best describes the input?`).
The API's probabilities, confidence and choice are never renormalized or
corrected; every answer must still pass the contract's checks (an entry for
every option, a sum within 0.02 of 1, the score in range), and a missing entry
is `invalid_response`, never filled in.

- **One-option choices.** The API takes 2 to 255 choices. A Choice with a single
  option is answered here, without HTTP, as that option with probability 1 and
  confidence 1. An evaluation made only of such questions makes no call and
  reports zero usage; a request still needs a key.
- **Refusals.** When the model declines any question, the whole call fails with
  `invalid_response`: no partial answers, no invented probability.
  `provider_error.message` reads `OpenAI refused N question(s) in evaluation
  <id>`, and `provider_error.detail` holds `refused` (at most 32 question ids,
  `truncated: true` beyond that) and that reply's `usage`. Consumers such as
  directory search and `browser::run` pause the provider for 30 seconds after
  such a failure.
- **Usage.** `stats` counts tokens and `requests` only from validated replies,
  including those accepted before another evaluation failed; refused or invalid
  replies are not counted there.
- **Models.** Only `gpt-6-luna` is supported. A request naming another model is
  `invalid_request` before any HTTP. `models::list` reads `GET /v1/models` and
  returns `gpt-6-luna` only when the key can see it, with a 922,000-token
  `context_window` (its maximum input); an empty list means the key cannot use
  the Decisions model.
- **Errors.** 408, 429 and 5xx are retried twice, honoring `Retry-After`; a hint
  longer than the remaining budget returns `http` with `retry_after_ms` at once,
  and a 429 for exhausted quota or spend limits is final. A 401 or 403 keeps
  only the OpenAI error `type` and `code`, since its message can quote part of
  the key.

The endpoint is fixed; there is no endpoint setting.

## Configuration

The Console entry defaults to `judge-openai`; set `III_CONFIG_NAME` in the worker
environment to use another entry. The form masks the API key and exposes these
defaults:

```yaml
api_key: null                  # secret://OPENAI_API_KEY, or fall back to the worker's OPENAI_API_KEY.
model: gpt-6-luna               # The only supported model.
max_request_bytes: 8388608      # Maximum JSON bytes per Decisions request.
max_response_bytes: 8388608     # Maximum bytes per response.
max_timeout_ms: 300000          # Whole-call timeout ceiling in milliseconds.
```

The key a call uses:

1. A configured `api_key`. Store the key in the `secrets` worker with
   `judge-openai` among its `consumers` and set `api_key:
   secret://OPENAI_API_KEY`. The reference is resolved through
   `secrets::resolve` at call time, kept only in memory, and re-read when
   `secrets::changed` names it, after 5 minutes, or 10 seconds after a failure.
   A reference that does not resolve returns `missing_key` with
   `provider_error.message` naming the fix, and never falls back to the
   environment. Any other `scheme://` value is rejected the same way. A literal
   key also works but should stay out of committed configuration.
2. Without `api_key`, `OPENAI_API_KEY` from the worker's environment at boot.
   This is an operator opt-in: llm-router uses the same variable name, so a
   worker started with a shared environment file answers with that key. Leave
   the variable out of this worker's environment to require the configured key.

All configuration fields hot-reload for new calls; in-flight calls keep their
snapshot, and rejected reloads keep the last valid value. Changing the process
environment requires a restart.

## Privacy

The worker never logs keys, inputs or answers. A retried attempt logs one
warning with its error code, HTTP status and timings. The worker also sets
`III_DISABLE_TRACE_PAYLOADS=1` at startup so its own OpenTelemetry spans carry
no request or response payloads. Spans recorded by the `judge` hub and by the
calling workers still include the payload; set the variable on those workers
too if that matters.

## Consumer compatibility

Callers use the shared `judge-contract` request and response types. See
[shared-contract compatibility](https://github.com/iii-hq/workers/blob/main/judge/reference.md#limits-and-compatibility)
when updating existing consumers. Each caller owns its thresholds, ranking
decisions and fallback policy.

For the full API, read the hub's
[reference.md](https://github.com/iii-hq/workers/blob/main/judge/reference.md).
For source builds, UI work and local test commands, read
[CONTRIBUTING.md](CONTRIBUTING.md).

## api reference

```json
{
  "functions": [
    {
      "description": "Signal cancellation of an active evaluation or model listing owned by the calling worker. Returns whether a signal was accepted; does not roll back provider work. Requires the same worker replica as the original call.",
      "metadata": {
        "internal": true
      },
      "name": "judge-openai::cancel",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "properties": {
          "request_id": {
            "maxLength": 128,
            "minLength": 1,
            "type": "string"
          }
        },
        "required": [
          "request_id"
        ],
        "title": "CancelRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "ErrorCode": {
            "oneOf": [
              {
                "enum": [
                  "invalid_request",
                  "missing_key",
                  "payload_too_large",
                  "deadline",
                  "attempt_timeout",
                  "cancelled",
                  "http",
                  "transport",
                  "invalid_response"
                ],
                "type": "string"
              },
              {
                "description": "The selected `judge-<provider>` worker is not registered on the engine, or (a local provider) its model failed to load.",
                "enum": [
                  "provider_unavailable"
                ],
                "type": "string"
              }
            ]
          }
        },
        "oneOf": [
          {
            "additionalProperties": false,
            "properties": {
              "cancelled": {
                "type": "boolean"
              },
              "status": {
                "enum": [
                  "ok"
                ],
                "type": "string"
              }
            },
            "required": [
              "cancelled",
              "status"
            ],
            "type": "object"
          },
          {
            "additionalProperties": false,
            "properties": {
              "code": {
                "$ref": "#/definitions/ErrorCode"
              },
              "status": {
                "enum": [
                  "error"
                ],
                "type": "string"
              }
            },
            "required": [
              "code",
              "status"
            ],
            "type": "object"
          }
        ],
        "title": "CancelResponse"
      }
    },
    {
      "description": "Returns this worker instance's effective configuration entry ID, without its value.",
      "metadata": {
        "internal": true
      },
      "name": "judge-openai::configuration-id",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Empty user request. Ignore transport metadata such as `_caller_worker_id`, which the engine adds to routed calls; no payload field selects the entry.",
        "title": "ConfigurationIdentityRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Only the entry ID is exposed; configuration values and secrets stay in the service.",
        "properties": {
          "id": {
            "type": "string"
          }
        },
        "required": [
          "id"
        ],
        "title": "ConfigurationIdentityResponse",
        "type": "object"
      }
    },
    {
      "description": "Evaluate Noul, Choice and Score questions against arbitrary JSON state using the OpenAI Decisions API. Results are atomic; stats retain known accepted usage and mark unknown counters incomplete. No credentials are accepted in the request.",
      "metadata": {
        "internal": true
      },
      "name": "judge-openai::evaluate",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "Content": {
            "anyOf": [
              {
                "type": "string"
              },
              {
                "additionalProperties": true,
                "type": "object"
              },
              {
                "items": true,
                "type": "array"
              },
              {
                "type": "null"
              }
            ],
            "description": "Structured provider descriptions; top-level numbers/bools are not descriptions."
          },
          "Evaluation": {
            "additionalProperties": false,
            "properties": {
              "id": {
                "type": "string"
              },
              "questions": {
                "additionalProperties": {
                  "$ref": "#/definitions/Question"
                },
                "minProperties": 1,
                "type": "object"
              },
              "state": {
                "$ref": "#/definitions/ScoreLevel"
              }
            },
            "required": [
              "id",
              "questions",
              "state"
            ],
            "type": "object"
          },
          "Question": {
            "oneOf": [
              {
                "additionalProperties": false,
                "properties": {
                  "criteria": {
                    "anyOf": [
                      {
                        "additionalProperties": false,
                        "properties": {
                          "false": {
                            "$ref": "#/definitions/Content"
                          },
                          "true": {
                            "$ref": "#/definitions/Content"
                          }
                        },
                        "type": "object"
                      },
                      {
                        "type": "null"
                      }
                    ]
                  },
                  "instructions": {
                    "allOf": [
                      {
                        "$ref": "#/definitions/Content"
                      }
                    ],
                    "default": null
                  },
                  "type": {
                    "enum": [
                      "noul"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "type"
                ],
                "type": "object"
              },
              {
                "additionalProperties": false,
                "properties": {
                  "criteria": {
                    "additionalProperties": {
                      "$ref": "#/definitions/Content"
                    },
                    "maxProperties": 255,
                    "minProperties": 1,
                    "type": "object"
                  },
                  "instructions": {
                    "allOf": [
                      {
                        "$ref": "#/definitions/Content"
                      }
                    ],
                    "default": null
                  },
                  "type": {
                    "enum": [
                      "choice"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "criteria",
                  "type"
                ],
                "type": "object"
              },
              {
                "additionalProperties": false,
                "properties": {
                  "criteria": {
                    "items": {
                      "$ref": "#/definitions/ScoreLevel"
                    },
                    "maxItems": 10,
                    "minItems": 2,
                    "type": "array"
                  },
                  "instructions": {
                    "allOf": [
                      {
                        "$ref": "#/definitions/Content"
                      }
                    ],
                    "default": null
                  },
                  "type": {
                    "enum": [
                      "score"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "criteria",
                  "type"
                ],
                "type": "object"
              }
            ]
          },
          "RequestOptions": {
            "additionalProperties": false,
            "properties": {
              "attempt_timeout_ms": {
                "description": "Bounds each network attempt separately from the whole-call deadline.",
                "format": "uint64",
                "minimum": 1,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          },
          "ScoreLevel": {
            "anyOf": [
              {
                "type": "string"
              },
              {
                "additionalProperties": true,
                "type": "object"
              },
              {
                "items": true,
                "type": "array"
              }
            ],
            "description": "Score levels require a description. Unlike instructions, Choice option descriptions and Noul criteria, the provider rejects a null level (HTTP422)."
          }
        },
        "properties": {
          "evaluations": {
            "items": {
              "$ref": "#/definitions/Evaluation"
            },
            "maxItems": 512,
            "minItems": 1,
            "type": "array"
          },
          "expires_at_unix_ms": {
            "format": "uint64",
            "minimum": 0,
            "type": [
              "integer",
              "null"
            ]
          },
          "model": {
            "type": [
              "string",
              "null"
            ]
          },
          "options": {
            "allOf": [
              {
                "$ref": "#/definitions/RequestOptions"
              }
            ],
            "default": {}
          },
          "request_id": {
            "maxLength": 128,
            "minLength": 1,
            "type": [
              "string",
              "null"
            ]
          },
          "timeout_ms": {
            "format": "uint64",
            "minimum": 1,
            "type": "integer"
          }
        },
        "required": [
          "evaluations",
          "timeout_ms"
        ],
        "title": "EvaluateRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "Answer": {
            "oneOf": [
              {
                "additionalProperties": false,
                "properties": {
                  "noul": {
                    "format": "double",
                    "type": "number"
                  },
                  "type": {
                    "enum": [
                      "noul"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "noul",
                  "type"
                ],
                "type": "object"
              },
              {
                "additionalProperties": false,
                "properties": {
                  "choice": {
                    "type": "string"
                  },
                  "confidence": {
                    "format": "double",
                    "type": "number"
                  },
                  "probabilities": {
                    "additionalProperties": {
                      "format": "double",
                      "type": "number"
                    },
                    "type": "object"
                  },
                  "type": {
                    "enum": [
                      "choice"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "choice",
                  "confidence",
                  "probabilities",
                  "type"
                ],
                "type": "object"
              },
              {
                "additionalProperties": false,
                "properties": {
                  "confidence": {
                    "format": "double",
                    "type": "number"
                  },
                  "legend": {
                    "additionalProperties": {
                      "$ref": "#/definitions/ScoreLevel"
                    },
                    "type": "object"
                  },
                  "probabilities": {
                    "additionalProperties": {
                      "format": "double",
                      "type": "number"
                    },
                    "type": "object"
                  },
                  "score": {
                    "format": "double",
                    "type": "number"
                  },
                  "type": {
                    "enum": [
                      "score"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "confidence",
                  "legend",
                  "probabilities",
                  "score",
                  "type"
                ],
                "type": "object"
              }
            ]
          },
          "ErrorCode": {
            "oneOf": [
              {
                "enum": [
                  "invalid_request",
                  "missing_key",
                  "payload_too_large",
                  "deadline",
                  "attempt_timeout",
                  "cancelled",
                  "http",
                  "transport",
                  "invalid_response"
                ],
                "type": "string"
              },
              {
                "description": "The selected `judge-<provider>` worker is not registered on the engine, or (a local provider) its model failed to load.",
                "enum": [
                  "provider_unavailable"
                ],
                "type": "string"
              }
            ]
          },
          "EvaluationResult": {
            "additionalProperties": false,
            "properties": {
              "answers": {
                "additionalProperties": {
                  "$ref": "#/definitions/Answer"
                },
                "type": "object"
              },
              "usage": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/Usage"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "required": [
              "answers"
            ],
            "type": "object"
          },
          "ProviderError": {
            "additionalProperties": false,
            "description": "Bounded provider diagnostics, with credentials and supplied header values redacted.",
            "properties": {
              "detail": true,
              "message": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "truncated": {
                "default": false,
                "type": "boolean"
              }
            },
            "type": "object"
          },
          "ScoreLevel": {
            "anyOf": [
              {
                "type": "string"
              },
              {
                "additionalProperties": true,
                "type": "object"
              },
              {
                "items": true,
                "type": "array"
              }
            ],
            "description": "Score levels require a description. Unlike instructions, Choice option descriptions and Noul criteria, the provider rejects a null level (HTTP422)."
          },
          "Stats": {
            "additionalProperties": false,
            "description": "Known accepted usage, never a billing estimate. Unreported counters or failure/cancellation make usage_complete false.",
            "properties": {
              "attempts": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "elapsed_ms": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "questions": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "requests": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "usage_complete": {
                "type": "boolean"
              }
            },
            "required": [
              "attempts",
              "elapsed_ms",
              "input_tokens",
              "output_tokens",
              "questions",
              "requests",
              "usage_complete"
            ],
            "type": "object"
          },
          "Usage": {
            "additionalProperties": false,
            "description": "Each absent/null counter is unknown, independently of the other counter.",
            "properties": {
              "input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          }
        },
        "oneOf": [
          {
            "additionalProperties": false,
            "properties": {
              "model": {
                "type": "string"
              },
              "results": {
                "additionalProperties": {
                  "$ref": "#/definitions/EvaluationResult"
                },
                "type": "object"
              },
              "stats": {
                "$ref": "#/definitions/Stats"
              },
              "status": {
                "enum": [
                  "ok"
                ],
                "type": "string"
              }
            },
            "required": [
              "model",
              "results",
              "stats",
              "status"
            ],
            "type": "object"
          },
          {
            "additionalProperties": false,
            "properties": {
              "code": {
                "$ref": "#/definitions/ErrorCode"
              },
              "http_status": {
                "format": "uint16",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "provider_error": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ProviderError"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "retry_after_ms": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "stats": {
                "$ref": "#/definitions/Stats"
              },
              "status": {
                "enum": [
                  "error"
                ],
                "type": "string"
              }
            },
            "required": [
              "code",
              "stats",
              "status"
            ],
            "type": "object"
          }
        ],
        "title": "EvaluateResponse"
      }
    },
    {
      "description": "List the Decisions models this key can use (only supported ones), with descriptions and release dates; empty when none is visible. Shares evaluation credentials, transport limits and permits; performs no inference.",
      "metadata": {
        "internal": true
      },
      "name": "judge-openai::models::list",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "RequestOptions": {
            "additionalProperties": false,
            "properties": {
              "attempt_timeout_ms": {
                "description": "Bounds each network attempt separately from the whole-call deadline.",
                "format": "uint64",
                "minimum": 1,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          }
        },
        "properties": {
          "expires_at_unix_ms": {
            "format": "uint64",
            "minimum": 0,
            "type": [
              "integer",
              "null"
            ]
          },
          "options": {
            "allOf": [
              {
                "$ref": "#/definitions/RequestOptions"
              }
            ],
            "default": {}
          },
          "request_id": {
            "maxLength": 128,
            "minLength": 1,
            "type": [
              "string",
              "null"
            ]
          },
          "timeout_ms": {
            "default": 30000,
            "format": "uint64",
            "minimum": 1,
            "type": "integer"
          }
        },
        "title": "ModelsRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "ErrorCode": {
            "oneOf": [
              {
                "enum": [
                  "invalid_request",
                  "missing_key",
                  "payload_too_large",
                  "deadline",
                  "attempt_timeout",
                  "cancelled",
                  "http",
                  "transport",
                  "invalid_response"
                ],
                "type": "string"
              },
              {
                "description": "The selected `judge-<provider>` worker is not registered on the engine, or (a local provider) its model failed to load.",
                "enum": [
                  "provider_unavailable"
                ],
                "type": "string"
              }
            ]
          },
          "ModelCard": {
            "properties": {
              "context_window": {
                "description": "Tokens one evaluation row can hold (question, options and state), when the provider has a fixed window. Callers size their states from it.",
                "format": "uint32",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "description": {
                "type": "string"
              },
              "max_options": {
                "description": "Most options one Choice question (or levels one Score) can offer, when the provider has a fixed limit below the contract's 255. Callers size their option groups from it.",
                "format": "uint32",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "name": {
                "type": "string"
              },
              "release_date": {
                "type": "string"
              }
            },
            "required": [
              "description",
              "name",
              "release_date"
            ],
            "type": "object"
          },
          "ProviderError": {
            "additionalProperties": false,
            "description": "Bounded provider diagnostics, with credentials and supplied header values redacted.",
            "properties": {
              "detail": true,
              "message": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "truncated": {
                "default": false,
                "type": "boolean"
              }
            },
            "type": "object"
          },
          "Stats": {
            "additionalProperties": false,
            "description": "Known accepted usage, never a billing estimate. Unreported counters or failure/cancellation make usage_complete false.",
            "properties": {
              "attempts": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "elapsed_ms": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "questions": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "requests": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "usage_complete": {
                "type": "boolean"
              }
            },
            "required": [
              "attempts",
              "elapsed_ms",
              "input_tokens",
              "output_tokens",
              "questions",
              "requests",
              "usage_complete"
            ],
            "type": "object"
          }
        },
        "oneOf": [
          {
            "additionalProperties": false,
            "properties": {
              "models": {
                "items": {
                  "$ref": "#/definitions/ModelCard"
                },
                "type": "array"
              },
              "stats": {
                "$ref": "#/definitions/Stats"
              },
              "status": {
                "enum": [
                  "ok"
                ],
                "type": "string"
              }
            },
            "required": [
              "models",
              "stats",
              "status"
            ],
            "type": "object"
          },
          {
            "additionalProperties": false,
            "properties": {
              "code": {
                "$ref": "#/definitions/ErrorCode"
              },
              "http_status": {
                "format": "uint16",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "provider_error": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ProviderError"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "retry_after_ms": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "stats": {
                "$ref": "#/definitions/Stats"
              },
              "status": {
                "enum": [
                  "error"
                ],
                "type": "string"
              }
            },
            "required": [
              "code",
              "stats",
              "status"
            ],
            "type": "object"
          }
        ],
        "title": "ModelsResponse"
      }
    },
    {
      "description": "Internal: reload OpenAI judge configuration from authoritative storage.",
      "metadata": {
        "internal": true
      },
      "name": "judge-openai::on-config-change",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Trigger payload for `<worker>::on-config-change`. Advisory only: handlers re-fetch the authoritative value and ignore it, so a direct call can never inject config.",
        "properties": {
          "id": {
            "default": null,
            "description": "Configuration id that changed.",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "OnConfigChangeEvent",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Ack returned by the internal `<worker>::on-config-change` handler.",
        "properties": {
          "ok": {
            "type": "boolean"
          }
        },
        "required": [
          "ok"
        ],
        "title": "OnConfigChangeResponse",
        "type": "object"
      }
    },
    {
      "description": "Internal: drop a cached secret:// API key after the secrets worker changed it.",
      "metadata": {
        "internal": true
      },
      "name": "judge-openai::on-secret-change",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Advisory `secrets::changed` payload; it never carries a value. Other fields (`action`, `fingerprint`, `updated_at`) are ignored.",
        "properties": {
          "name": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          },
          "ref": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "SecretChangedEvent",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "ok": {
            "type": "boolean"
          }
        },
        "required": [
          "ok"
        ],
        "title": "SecretChangedResponse",
        "type": "object"
      }
    },
    {
      "description": "Whether evaluations hold a usable key (api_key, a secret:// reference or OPENAI_API_KEY at boot), with the fix when they do not. No network call; never returns the key.",
      "metadata": {
        "internal": true
      },
      "name": "judge-openai::status",
      "request_schema": {
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "description": "`judge-<provider>::status`, answered without network calls or model loads.",
        "properties": {
          "configured": {
            "description": "An evaluation would get past credentials: a hosted provider holds a usable key. Says nothing about whether the upstream accepts it.",
            "type": "boolean"
          },
          "reason": {
            "description": "Why not, naming the fix and never a value.",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "required": [
          "configured"
        ],
        "title": "StatusResponse",
        "type": "object"
      }
    },
    {
      "description": "Serve the judge-openai worker's injected console UI assets (content function for its console:script / console:style / console:module triggers).",
      "metadata": {
        "internal": true
      },
      "name": "judge-openai::ui-content",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Input of the content function: the console asks for one asset by path.",
        "properties": {
          "path": {
            "description": "The asset path from the trigger config (e.g. `state/page.js`).",
            "type": "string"
          }
        },
        "required": [
          "path"
        ],
        "title": "UiContentInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Output of the content function.",
        "properties": {
          "content": {
            "description": "The asset source, verbatim.",
            "type": "string"
          },
          "content_type": {
            "description": "MIME type the console should serve the asset with.",
            "type": "string"
          }
        },
        "required": [
          "content",
          "content_type"
        ],
        "title": "UiContentResult",
        "type": "object"
      }
    }
  ],
  "triggers": []
}
```