# eval

> Session monitor for the Harness (deterministic diagnostics, Jev triage through judge, investigation by the user-chosen LLM, E2E validation plans; inactive until configured).

| field | value |
|-------|-------|
| version | 0.3.1 |
| type | binary |
| license | Apache-2.0 |
| repo | https://github.com/iii-hq/workers |
| supported_targets | x86_64-apple-darwin, aarch64-apple-darwin, x86_64-pc-windows-msvc, aarch64-pc-windows-msvc, x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-unknown-linux-musl, armv7-unknown-linux-gnueabihf |
| author | iii |

## installation

```sh
iii trigger compose::add worker=eval@0.3.1
```

## dependencies

- `cron` @ `latest`
- `iii-observability` @ `latest`
- `judge` @ `latest`
- `queue` @ `latest`
- `state` @ `latest`

## readme

# eval

`eval` monitors finished Harness sessions and proposes Harness improvements
backed by evidence and an E2E validation plan.

The monitor is **inactive until configured**. It observes, analyzes and
suggests; it never edits code, opens PRs or changes the observed session, and
starts E2E executions only when a person calls `eval::start-validation` (with a
code directory configured, the investigating LLM
may only call read-only functions: see [Code access](#code-access)). The
behavior is specified in
[SPECIFICATION.md](SPECIFICATION.md) and the technical plan in
[IMPLEMENTATION.md](IMPLEMENTATION.md).

## How an analysis works

1. **Admission.** A terminal `harness::on-turn-completed` event of a root session (or
   a manual `eval::analyze-session`) admits one analysis per session turn.
   Automatic observation admits only the user's chats. From `session::get` the
   session must have kind `user` (a record without a kind counts as `user`), a
   chat `metadata.surface` (`console`, `slack` or `telegram`, which each client
   writes on every send) and no `metadata` key starting with `e2e_`.
   Everything else answers `not_user_chat` and can still be analyzed by hand:
   E2E runs (kind `e2e`, or `e2e_*` metadata on the ones recorded before the
   kind existed), automations (sentinel investigations and the monitor's own,
   which it opens as `automation`; the console stamps `console` on those too,
   so the kind is what excludes them), scripted and sub-agent sessions (kind
   `user` but no surface) and sessions whose record cannot be read. Progress
   events,
   descendants and the monitor's own sessions are never admitted; a
   redelivered event returns the existing analysis.
2. **Collection.** The monitor waits until the turn is definitive and every
   descendant has finished (`harness::metrics.complete`), then reads every
   transcript page with `include_custom: true`, failing on malformed pages
   instead of skipping evidence. The window is the observed turn plus earlier
   root turns no analysis covered that started while the monitor was already
   observing (a wake continuation); history from before it is never reported
   as new. Descendants
   linked to other turns are listed but not read. Status and tree membership
   are checked again at the end; a changed session fails the analysis rather
   than mixing turns.
3. **Diagnostics.** Deterministic rules run on the complete transcripts
   before anything is reduced:
   - `repeated_contract_discovery`: a successful `engine::functions::info`
     whose every contract came back `unchanged_in_context`, each traced to an
     earlier successful result that supplied that schema. A `registry-changed`
     notice between the sources and the repeat marks it
     `harness_notice_correlated` (correlation, not cause).
   - `repeated_tool_error`: consecutive calls in one turn to the same target
     with structurally equal payloads, where the first failure was visible
     before the retry and both failed with the same explicit code. Corrected
     arguments, a successful retry, parallel calls in one response, other
     turns and, in transcripts recorded against a 0.25 engine, its
     default-namespace `engine::triggers::info` `NOT_FOUND` probe are not
     flagged.

   Each occurrence has a fingerprint (rule, session, turn, call ids).
4. **Triage (Jev).** `judge::evaluate` with `provider: typesafe` asks one
   Choice question — `needs_investigation`, `expected_behavior` or
   `insufficient_evidence` — over facts computed in code (no transcript
   text). The effective model, probabilities, confidence, criteria hash and
   `Stats` are stored.
5. **Routing.** The LLM investigates if and only if Jev answered
   `needs_investigation`, for automatic and manual analyses alike
   (`routing.reasons` is `["needs_investigation"]`, else `[]`). A deterministic
   finding, `insufficient_evidence`, a low confidence, insufficient coverage, a
   manual request and a sample of quiet sessions do not send a session there on
   their own; the findings stay in the result either way. Analyses recorded
   under the earlier routing keep their `diagnostics`, `insufficient_evidence`,
   `low_confidence`, `coverage_insufficient`, `audit_sample` and
   `manual_request` reasons; none of them is produced now.
6. **Investigation.** `harness::send` opens `eval_monitor_<evaluation_id>`
   (`kind: automation`, `metadata.origin: eval_monitor`) with the frozen
   model, an overriding system prompt and a JSON output contract (at most
   three suggestions; an empty list is valid). Without a code directory (the
   default) it is one turn with deny-all functions. With one, it reads the
   code (see [Code access](#code-access)). The bundle also carries
   `monitor.e2e_scenarios` (`{total, scenarios: [{id, title, summary}]}`, from
   `e2e::dashboard::tests-list`, one page of at most 100 scenarios, summaries cut
   at 160 characters, reused for ten minutes per process) and the prompt tells
   the LLM to set `validation.scenario_id` to one of those ids when it
   reproduces the problem, and to keep `null` when a new case is needed. When
   the E2E does not answer, the list and that instruction are omitted and the
   analysis goes on. The backend drops any suggestion
   that cites an entry absent from the evidence shown, cites code that does
   not exist, or has an incomplete E2E plan, and records why.
7. **Result.** `completed` means the monitor finished, with or without
   suggestions. It is never a validated improvement.

States: `queued → collecting → judging → (investigating) → completed`, or
`failed` (with the failing stage and code) or `cancelled`.

A failure's `code` names the cause. Jev failures are `judge_<provider code>`
(`judge_missing_key`, `judge_provider_unavailable`…), except that HTTP 402 from
the provider is `judge_out_of_credits` (the message keeps the provider's own
text, and `assets.triage_failure` keeps `code: http` and `http_status: 402`).
The investigation fails with `analyst_failed`, `analyst_step_cap` or
`analyst_output_invalid`; the others are `coverage_insufficient`,
`source_advanced`, `external_outcome_unknown`, `deadline` and `cost_cap` (an
automatic investigation that found the daily cost cap reached when it was about
to start).

Each record also keeps three facts for comparing analyses and releases:
`supersedes` (the earlier `evaluation_id` of the same turn, set when the
analysis is a reanalysis), `harness_version` (the version of this worker's
Harness that `engine::workers::list` reported when a live event admitted the
analysis, read from the engine's `default` namespace: the turn just ended on
it. A manual analysis, which may be of an older session, has none, and neither
has one admitted while the engine did not answer, which never blocks admission) and
`signals` (occurrences per deterministic pattern the collection found, keyed
`<rule_id>:<target>`; empty both when nothing was found and before collection,
and `coverage` tells which).

## Configure and use

The monitor reuses the existing providers; it never receives a TypeSafe or
LLM key. Jev credentials stay in `judge-typesafe`; the LLM's in its provider.

```bash
# 1. Choose the investigating model (validated against router::models::list)
#    and keep it paused.
iii call eval::configure --json '{
  "enabled": false,
  "model": {"model": "<model-id>", "provider": "<provider-id>"}
}'

# 2. Analyze one finished root session by hand (works while paused).
iii call eval::analyze-session --json '{"session_id": "<session>"}'

# 3. Follow it, then read evidence, triage and suggestions.
iii call eval::status --json '{"evaluation_id": "<id>"}'
iii call eval::result --json '{"evaluation_id": "<id>"}'

# 4. Enable automatic observation once the manual results look right.
iii call eval::configure --json '{
  "enabled": true,
  "model": {"model": "<model-id>", "provider": "<provider-id>"}
}'
```

`eval::config` shows the configuration and whether the
`harness::on-turn-completed` observer is active (`observer_bound`); when it is
false (no harness has registered the function, or the engine refused the
observer, with the reason in `observer_error`), automatic observation is
unavailable even if enabled.

`thinking_level`, `provider_options` (namespaced by the selected provider),
`code_repository` and `daily_cost_cap_usd` are optional. A configuration
change applies to new analyses only: each record keeps its model, its code
directory and its `config_revision`.

## Cost

Only LLM cost is in dollars: the investigation's (`usage.llm_cost_usd`, from
`harness::metrics`, and known only once the investigation ends) and the
`eval::reproduce` samples' (`cost_usd` of each reply). Jev's usage, in a
triage or in classifying a replay's replies, is reported in tokens and never
priced. A missing cost is unknown, never zero, and is never estimated: a
replay sample without a cost, one that failed included (the provider may still
bill a request the bus gave up on), adds nothing to the sums, is counted in the
reproduction's `cost_unknown_samples` (which makes its `cost_usd` a lower
bound) and in the day's `today_replay_unknown`. Each sample is added to the day
as it is saved, so a run that stops partway has already counted what it kept.

The day's spend has two buckets. **Capture** is what an analysis spends (the
investigation; Jev's triage is in tokens). **Replay** is what `eval::reproduce`
spends. Only capture is compared with the daily cap, so a manual replay never
stops automatic observation. The spend persisted before the buckets (one total)
counts as capture for its day.

`eval::config` returns a `cost` block to decide before enabling:

- `today_capture_usd` and `today_unknown`: the known capture cost since `since`
  (the start of the current **UTC** day: the monitor has no timezone setting),
  and how many analyses of the day started an investigation but reported no cost
  (they add nothing to the sum; their cost is unknown). The cost is added to a
  persisted daily spend as each investigation reports it, and
  `today_capture_usd` is the larger of that and the stored analyses' sum, so
  deleting analyses (or retention) does not give the budget back.
- `today_replay_usd` and `today_replay_unknown`: the replay bucket, the known
  cost of the day's `eval::reproduce` samples and how many had no cost
  (failed samples included). Reported, never capped.
- `today_usd`: both buckets together, everything known to be spent. **The cap
  does not compare it.**
- `cap_usd` and `capped`: the optional `daily_cost_cap_usd` and whether
  `today_capture_usd`, plus the median cost of each investigation still
  running, reached it.
- `per_analysis`: `count`, `min`, `median` and `max` of the known cost of the
  completed analyses that investigated with the configured model, provider and
  code access (or without it), and `unknown`, how many of those reported no
  cost. Without history, `count` is 0 and the statistics are absent.

`daily_cost_cap_usd` (a number above 0; absent means no cap, and then the
configuration revision is what it was) pauses **automatic** observation for the
rest of the UTC day once `today_capture_usd` reaches it: the turn is not admitted,
`eval::on-turn-completed` answers `cost_cap`, and `eval::config`'s
`last_rejection` records the turn with `reason: cost_cap` (`at_capacity` for the
unfinished-analyses cap). A manual `eval::analyze-session` is never refused by
the cap, and a redelivered event of an already admitted turn is still reused.
The cap is checked twice: at admission and again where the money is spent,
right before an automatic analysis first calls the model (under one lock, so
investigations starting together see each other); one that finds it reached
fails with `cost_cap` without calling the model; a manual reanalysis is not held
back by the cap and investigates if Jev answers `needs_investigation` again. A cost is known only when an investigation ends, so the investigations
already running are counted at the median cost of the history. With no history
there is nothing to estimate by and a first burst can overshoot by the
investigations that run at once (the queue runs 8 steps together). Raising or
removing the cap resumes observation at once.

## Code access

By default the investigating LLM sees only the evidence bundle. Set
`code_repository` to the codebase directory (normally the iii workers
repository, e.g. `/home/layon/workspaces/workers`) and the investigation
becomes a chat with that directory selected:

```bash
iii call eval::configure --json '{
  "enabled": false,
  "model": {"model": "<model-id>", "provider": "<provider-id>"},
  "code_repository": "/home/layon/workspaces/workers"
}'
```

- The path must be absolute and an existing directory on the host that runs
  `eval` (checked when it changes; blank means off). The monitor stays
  disabled by default. It reads the **current** working tree, uncommitted
  changes included: there is no pinned copy, no tag lookup and no requirement
  that the directory is a git clone. To implement a suggestion, create a
  worktree or work however you prefer.
- Each analysis freezes the directory at admission (`record.code_root`,
  echoed in `investigation.code_root`); changing the configuration later never
  alters a running analysis.
- The send carries `{"fs_scope": {"root": "<dir>"}}` in the session metadata
  and in `options.metadata`, exactly what the ADE chat sends for a selected
  directory: the Harness scopes every `coder::*` and `shell::*` call to it and
  the console shows the directory selected when the investigation session is
  opened. The policy is an explicit read-only allowlist,
  `allow: ["coder::search", "coder::tree", "coder::read-file",
  "session::messages", "github::pr::list", "engine::functions::info"]`, with
  `deny: ["eval::*", "e2e::dashboard::execution-*"]` kept as well (no new bus
  function: the LLM uses existing ones). The first five are the functions the
  prompt names (`session::messages` reads more of an observed session than
  the evidence shows);
  `engine::functions::info` is the contract lookup the invocation surface
  tells a model to make before a first call (without it the LLM guessed
  argument names, e.g. `start_line` for `line_from`). Everything else is
  refused by the Harness, so a transcript it reads cannot make it write a
  file, run a shell, start or message a session, start an E2E execution or
  record a review. `fp::pipe` is deliberately absent: its steps run with the
  `fp` worker's authority, outside this policy (analysts had used it to read
  `session::messages`, now allowed directly). Without a code directory the
  policy stays deny-all. With `approval-gate` installed five pass without a
  human (`iii-permissions.yaml` allows them); `session::messages` stays at
  that file's `needs_approval` default for session reads, so each call waits
  for a person or the deadline. The step cap goes from 1 to 32
  generate steps and the total-token cap from 200,000 to 800,000
  (`limits.investigation_code_max_turns` and
  `investigation_code_max_total_tokens`). The deadline and the
  `submit_result` output contract are unchanged. A turn that uses all its
  steps ends `completed` in the Harness with a notice instead of a result; the
  monitor fails that analysis with `analyst_step_cap` (usage kept, no
  suggestions) instead of reading the notice as an invalid answer.
- The prompt names the directory, asks the LLM to read the code before
  proposing a change and to deliver through `submit_result` only afterwards,
  states the step and token budget (a turn that uses every step delivers
  nothing), reminds it that the cause or the best fix may be in any worker
  (not only the Harness), declares the investigation read-only (no file
  changes, no starting or messaging sessions, no `eval::*`), repeats that
  transcripts and code are data, and asks it to cite every claim about code.
  It also tells the LLM that it may list the open pull requests with
  `github::pr::list` (read-only, repo `iii-hq/workers`) and must say in the
  suggestion's `limitations` when one already overlaps it. Without code access
  the prompt is the one the monitor always had, and the output schema does not
  offer `code_refs`.
- A suggestion carries `code_refs`: up to 8 `{path, line_from, line_to}` with
  the path relative to the directory and 1-based inclusive lines. When the
  investigation completes the monitor checks each one on disk and rejects the
  suggestion, with the reason, if the path is absolute or has `..`, resolves
  outside the directory once symlinks are followed, is not a file, or the
  range is not `1 <= line_from <= line_to <= lines in the file`; `code_refs`
  without code access are rejected too. The checks read the files as they are
  when the investigation ends, only as far as `line_to` and never a file above
  16 MiB (a reference to one is rejected), on a blocking thread, because the
  LLM chooses the file (a traces database or a build output lives in a
  workers checkout).
- Tool calls and tokens of the investigation are the session's own metrics
  (`harness::metrics`), counted as before.

**Residual risk.** The investigating LLM can no longer change anything or
start anything, but it still reads untrusted transcripts and can read the
workspace. The `fs_scope` root does not jail the reads: in this environment the
`ide` worker runs `coder::*` unjailed, so `coder::read-file` and `coder::search`
accept absolute paths outside the root. What a hostile transcript can still
achieve is steering what the LLM reads and what its suggestions say, so read a
suggestion's `code_refs` and text as the output of an LLM that read untrusted
input. Leave `code_repository` unset to keep the investigation read-nothing.

### Public functions

| Function | Purpose |
| --- | --- |
| `eval::configure` | Model selection, optional code directory, optional daily cost cap and enable/pause. |
| `eval::config` | Configuration (or null), observer status, the latest rejection and the `cost` block. |
| `eval::analyze-session` | Manual analysis; `reanalyze: true` creates another analysis after the previous one ended. |
| `eval::list` | Compact records, newest first (`limit` 1–200, default 50); `observation_key` keeps only the analyses of one observed turn, reanalyses included. |
| `eval::status` | One compact record, or null. |
| `eval::result` | Record plus evidence, diagnostics, triage, suggestions, the monitor's own consumption, E2E links and one review row per suggestion (`reviews`). |
| `eval::cancel` | Signals the Jev call (`judge::cancel`) and stops the investigation (`harness::stop`); never touches the observed session. |
| `eval::delete` | Deletes a terminal analysis; the turn stays marked as analyzed until retention. |
| `eval::attach-validation` | Links baseline and candidate E2E executions to a suggestion and computes the pair's evidence (`dry_run: true` only looks them up). |
| `eval::start-validation` | Explicitly starts a baseline and a candidate E2E execution (Docker) of one scenario, pinned to two pushed commits. Spends model tokens. |
| `eval::review` | `set_lifecycle`, `set_criterion` or `set_verdict` on one suggestion of a terminal analysis. |
| `eval::reviews` | The stored review rows and, per analysis, its suggestions counted by lifecycle status. |
| `eval::recurrence` | For a suggestion shipped in a version, how often its patterns appeared per analysis before and from that version. |
| `eval::reproduce` | Replays a suggestion's decision point: rebuilds the request the model received at that step (optionally edited), samples the next reply N times without running any function and reads the signal in each. `dry_run` only rebuilds and checks fidelity. See [VALIDATION.md](VALIDATION.md). |

Internal: `eval::step` (queue `eval-run`, FIFO per analysis, concurrency 8),
`eval::on-turn-completed` (the target of the worker's observer of
`harness::on-turn-completed`, delivered the event) and `eval::sweep` (the
target of its observer of `cron::on-tick` with args
`{ expression: "*/15 * * * * *" }`).

### Events

`eval::on-completed` is a private function: `eval` calls it when an analysis
reaches a terminal status and something observes it; anyone may observe it.
It takes no args; select one analysis with a filter on `input.evaluation_id`.
Its handler returns the event:

```ts
interface CompletedEventV1 {
  evaluation_id: string
  status: 'completed' | 'failed' | 'cancelled'
  timestamp: number // Unix ms
}
```

Delivery is iii's default: every call reaches each observer whose filters
match. `completed` describes the monitor, not a validated improvement.

```ts
iii.observe(
  { function_id: 'eval::on-completed', filters: { 'input.evaluation_id': { $eq: id } }, deliver: 'output' },
  'my-worker::on-analysis-done',
)
```

## Evidence handling

Snapshots keep, per session, the entry count, the SHA-256 of the collected
JSON (not of stored bytes) and masked previews. Previews favor diagnostic
evidence, the request that started the window, the final answer, notices and
a short tail, within the model-context limit (below); omitted and reduced entries are
counted. Diagnostics take at most 64 KiB of that context (all
of them stay in the snapshot); evidence that does not fit makes coverage
`insufficient`, and a context above the limit is never sent. Raw result `details` and opaque reasoning payloads are
dropped; JSON carried as text is decoded before the Harness masker hides
secret-named keys and shortens long strings. Masking is key-based: secrets
in free text are **not** guaranteed to be removed.

Metrics come from `harness::metrics` and are cumulative over the session tree
and all turns (`metrics_scope: session_tree`). The monitor's own Jev `Stats`
and investigation metrics are stored separately from the observed task.
Unknown cost stays unknown.

## Limits

Constants of this version, returned by `eval::config` as `limits` so the
console never restates them:

| Limit | Value |
| --- | --- |
| Analysis budget from admission | 30 min (queue wait, descendants and investigation included) |
| Bus calls for collection | 10 s each, within the budget |
| Jev | 60 s provider timeout, 70 s bus timeout |
| Model context | 192 KiB of serialized JSON (about 50k tokens); diagnostics take at most 64 KiB |
| Assets per analysis | 2 MiB; above it the analysis fails with `coverage_insufficient` before any model call. The turn record kept in `assets.capture.record` is the first thing left out to stay under it |
| Investigation | 1 turn, 16,384 output tokens, 200,000 total tokens (the model's own caps still apply) |
| Investigation with a code directory | 32 generate steps, 16,384 output tokens, 800,000 total tokens, only the six read-only functions of the Code access allowlist (`investigation_code_max_turns`, `investigation_code_max_total_tokens`) |
| Queue | `eval-run`, FIFO per analysis, 8 steps at once |
| Unfinished analyses | 500 |
| Retention | 30 days and 1,000 terminal analyses |

## Recovery and its limits

Records, indexes and assets live in `state` scopes `eval_monitor`,
`eval_observation`, `eval_analysis` and `eval_analysis_assets`; the review rows
in `eval_suggestion` (below). A record is
saved before its index; startup reconciles missing indexes before the
turn observer is created, and the sweep re-enqueues pending work and
fails analyses past their deadline.

- Assets are written before a stage advances, so a restart repeats
  collection, never a recorded model answer.
- A Jev call that started before a restart without a saved answer fails with
  `external_outcome_unknown`; it is not repeated automatically (the provider
  has no durable response cache). Request a reanalysis.
- A lost `harness::send` reply is resent with the same idempotency key.
- Events emitted while `eval` was offline are not recovered; analyze those
  sessions manually.
- Recovery assumes `state` kept its records. The default `kv` adapter
  flushes to disk every `save_interval_ms` (5 s by default), so a crash of the
  state worker can lose the last writes; there is no exactly-once guarantee
  under storage loss.
- Locks are per process: run a single `eval` instance.
- Persisted records reject fields they do not know, so state written by a newer
  `eval` (a capture's `record`, a reproduction's `cost_unknown_samples`) cannot
  be read back by an older one. Rolling back is not supported once new analyses
  or replays exist.

## Validation by replay

The first check of a suggestion is a replay of the step where its behavior
happened ([VALIDATION.md](VALIDATION.md)). Each suggestion carries a `check`:
the `decision_point` (an assistant entry of its evidence), the `signal` (a rule,
`contract_rediscovery` or `repeated_error_call`, or a yes/no `question` Jev
answers per reply) and the proposed `change` as edits of what the model saw.
Older suggestions have none; `eval::reproduce` then takes `check` in the request.

`eval::reproduce` rebuilds the request the Harness sent at that step: the
window from the durable log (`harness::window::build`, notices included), the
frozen runtime context, the turn's system prompt and skills baseline (copied
into the analysis as `assets.capture`, because the Harness keeps only a
session's latest turn record), the `agent_call` tool and `context::assemble`.
`assets.capture.record` also keeps the Harness's whole turn record as read (a
later fork of the session needs it, and the next turn replaces it); when it
would take the assets over their limit it is left out and
`assets.capture.record_omitted` says why, while the options and digest above
stay.
It counts the result with `router::count_tokens` (the context manager's estimate
when the provider has no counter) against the recorded usage: `exact` when the
difference equals the fixed overhead measured at the turn's first step,
`approximate` otherwise, with the reasons. It then samples `router::complete`
(one warm-up call, then four at a time), never runs a function, reads each
reply's signal and stores everything in the suggestion's review row
(`reproductions[]`). `extend` adds replies; `samples: 0` finishes one that
failed. A restart marks running replays `failed` (interrupted). Not supported
yet: native function exposure, output contracts, and windows the context
manager would prune or compact.

## E2E validation

A suggestion carries a reproduction, task invariants, a primary effort metric,
the expected baseline-versus-candidate comparison and non-regression
controls. Turning it into proof is a separate, explicit step: fix the
scenario version and criteria, run baseline and candidate with
`harness-e2e`, then link both executions:

```bash
iii call eval::attach-validation --json '{
  "evaluation_id": "<id>", "suggestion_index": 0,
  "baseline_execution_id": "<baseline>", "candidate_execution_id": "<candidate>"
}'
```

The link stores what `e2e::dashboard::execution-get` reports (status, report
availability, evidence errors). Attaching the same pair again refreshes its link
but keeps the first `attached_at`, which still says when results first existed. Each scenario keeps its `run_count` and
`measures` (the execution's own averages) and, when
`plan_execution.measurements.cohorts[]` has exactly one cohort for it, that
cohort's own aggregates: `pass_rate`, `mean_score`, `completed_runs`,
`planned_runs`, `cost_usd_per_run` and `total_tokens_per_run` (the cohort's
totals divided by its completed runs; the totals also cover failed attempts),
`median_wall_time_ms` and `p50_function_calls`. Each is absent when the E2E did
not report it (a count may arrive as a float), never zero. It is a reference,
not a verdict: improvement,
no improvement, regression or inconclusive belong to the E2E comparison and its
criteria.

## Review and validation

What people decide about a suggestion lives in its own row, in the
`eval_suggestion` state scope keyed `<evaluation_id>:<suggestion_index>`
(`SuggestionReviewV1`), never in the suggestion, whose schema is what the
investigating LLM must return. The row copies the suggestion's `title`,
`harness_component`, plan `scenario_id` and its `patterns` (`<rule_id>:<target>`
of the diagnostics the suggestion's evidence cites), so it outlives the
analysis's retention and `eval::delete`. A suggestion nobody acted on has no
stored row and reads as `new` in `eval::result.reviews` (one row per
suggestion) with `lifecycle.history: []`.

`eval::review {evaluation_id, suggestion_index, action, ...}` needs the
analysis to be terminal and the suggestion to exist. The author of every change
is `by` when the request names one, else the user the eval worker runs as
(`$USER`, else `$LOGNAME`), else the caller's identity (`_caller_worker_id`):
a browser's worker id is an opaque uuid, so it comes last. The console sends no
`by`, so what it records is the host's user.

- **`set_lifecycle`** with `status` `accepted`, `in_progress`, `shipped`,
  `rejected` or `duplicate` (never back to `new`; a shipped suggestion stays
  shipped, to complete its `pr` or `version`). `shipped` needs a `pr` (kept from
  an earlier status when omitted) and may carry the `version` it shipped in, a
  semantic version like `1.8.43`; `rejected` needs a `reason`; `duplicate`
  needs `duplicate_of`. `lifecycle.history` keeps `{status, at, by, note?}` for
  every change.
- **`set_criterion`** registers what a validation is judged by, before its
  results: a `metric` (`signal_per_run` with one of the row's `patterns`,
  `pass_rate`, `cost_usd`, `duration`, `tokens` or `function_calls`), a
  `direction`, a `min_effect` (the smallest difference between the baseline and
  candidate means that counts, in the metric's own unit: signals per run, USD,
  seconds of wall time, function calls or tokens; percentage points, 0-100, for
  `pass_rate`. `1` is one signal per run, never 1%) and `min_runs` (1-20
  completed runs each side), on the plan's scenario unless `scenario_id` names
  another. The same criterion again keeps its registration time. Once a pair
  is attached or a run started, an existing criterion is frozen
  (`criterion_frozen:`): one chosen after seeing results proves nothing.
- **`set_verdict`** records the person's `outcome` (`validated_improvement`,
  `no_improvement`, `regression` or `inconclusive`), `rationale` and
  `controls_checked`, with a snapshot of the criterion. `validated_improvement`
  is refused (`verdict_refused:`) unless a criterion was registered no later than
  the first moment results could exist (the earliest of each attached
  execution's own `started_at`, the attach, and the executions this worker
  started, also those a restart replaced), so an old pair attached after its
  criterion does not count as unseen, and the evidence has at least `min_runs`
  measured runs on both sides. The computed outcome (below) is not
  a precondition: the person may disagree, and both are kept.

### Evidence

Whenever a pair is attached (by hand or by a started run) and the suggestion's
scenario is known (the criterion's, else the plan's), the runs of that scenario
in both executions (`detail.reports[].report.scenarios[].runs[]`) become the
row's `evidence`: per run its `completed` flag (the E2E's `completion` is
`completed` and `technical` is `valid`, both kept as reported, so infrastructure
failures and incomplete runs are listed, never silently dropped), `status`,
cost, wall time, tokens, function calls, and the occurrences of each of the
row's patterns that `diagnostics::detect` finds in the run's own transcript
(`transcript.messages`; a run without transcript has no count, 0 is a
measurement). Per side `n` is the measured runs with a value of the criterion's
metric and `mean` their mean. A run is measured when its `technical` is `valid`:
one the agent left unfinished (`completion: task_incomplete`) is the Harness's
result, so it counts, as a fail of the pass rate whatever its `status` says,
while an infrastructure failure measured nothing and stays out. `computed_outcome` is decided in code, with a
`reason`: `validated_improvement` when the candidate moves the baseline mean by
at least `min_effect` (in its unit) in the wanted direction, `regression` when it
moves the wrong way by that much, `no_improvement` otherwise, and `inconclusive`
when there is no criterion, a side has no run of the scenario, more than half of
a side's runs did not complete (technical failures and unfinished runs) or a
side has fewer than `min_runs` measured runs with the metric. Only the root
session of each run is
read (a Docker run's sessions never reach this engine); every metric but
`signal_per_run` is a mean over the same runs, not the E2E cohort aggregate.

### Starting a validation

`eval::start-validation {evaluation_id, suggestion_index, scenario_id,
candidate_ref, baseline_ref?, runs?, model, provider, criterion}` replaces the
manual steps of linking two E2E runs. It runs only when called: it spends model
tokens and Docker time on two executions whose cost depends on the scenario and
`runs` (a past cohort of the same scenario in the E2E is the best estimate).

With `dry_run: true` (default `false`) it stops after the checks that need no
E2E: the refs resolve (step 1), both commits are pushed and differ, and the
scenario and runs are valid (and `criterion.min_runs <= runs`, when a criterion
is sent; `model`, `provider` and `criterion` are otherwise only required for a
real start). It answers `{baseline, candidate, warnings}`, each side
`{commit, short, branch}` (`short` is 12 characters, `branch` a remote branch
that holds the commit, the one named like the ref when several do), and
registers nothing, records no run and calls no E2E function. A baseline given
that is not an ancestor of the candidate is a warning, not an error. The
console asks it 400 ms after the last keystroke and when a field loses focus,
shows `Resolved · <sha> · pushed to <branch>` or the refusal under each field,
and keeps Start disabled until a dry run succeeded for exactly the current
refs, scenario and runs.

1. Needs `code_repository` configured (`code_repository_required:`). The refs
   are resolved with `git` in it: `candidate_ref` and `baseline_ref` may be a
   plain branch, tag or commit name; the baseline defaults to the merge base of
   the candidate and `origin/main`. Each commit must be on a remote branch of
   the clone (`git branch -r --contains`, nothing is fetched: run `git fetch`
   after a push) because the E2E builds the Harness from GitHub. Errors:
   `git_unavailable:`, `git_ref_invalid:` (also when both resolve to the same
   commit), `commit_not_pushed:`.
2. Each stack is the YAML of the E2E's stack built on the `harness` template
   (`e2e::dashboard::stacks-list`) with only its `harness` container replaced by
   `{worker: package://harness, commit: <sha>, repository: iii-hq/workers}`.
   Everything else, including the `latest` versions the base declares, is the
   E2E's own (a comparability check on the attached pair shows when two
   executions resolved different ones).
3. Nothing changed so far; now the criterion is registered and the run recorded
   as `starting` (`run` in the row: both commits, scenario, runs, model,
   provider, `started_at`), and two executions start with
   `e2e::dashboard::execution-start` (`where: docker`, one technical retry,
   `runs` 1-20, default 5, label `eval <evaluation_id> S<n> baseline|candidate`
   with `n` the suggestion number from 1). The row goes to `running` with both
   execution ids. A refusal is `e2e_busy:` or `e2e_unavailable:`; the run is
   then `failed` with the error, naming the baseline execution when it had
   already started (cancel it in the E2E). A start the E2E never answers
   (`e2e_start_unconfirmed:`, after 30 s) may still have started an execution,
   so the run stays `starting`, with the baseline id when it had one, and no
   second start can duplicate it: look for the label in the E2E; the sweep fails
   the run after 5 minutes. One run per suggestion at a time.
4. The sweep (every 15 s) polls `e2e::dashboard::executions-list {ids}` for
   running runs. When both executions are `completed`, `interrupted`,
   `cancelled` or `failed` the run is `finished` and the pair is attached with
   the evidence computed (`attached`), also when the executions ended badly: the
   evidence then says why it cannot judge. An E2E that does not answer leaves it
   `finished` with the error and the next sweep tries again, for 30 minutes
   after both ended (`finished_at`), then it is `failed` with `attach_gave_up:`
   and the last error, and the pair can be attached by hand or started again; an
   execution the E2E no longer lists, or an analysis deleted meanwhile, makes it
   `failed` at once. A `starting` run older than 5 minutes (a restart between the
   request and its answer, or an E2E that never answered) is `failed`. Startup
   recovery does not wait on the E2E: the first sweep, 15 s later, does.

### After the release

`eval::recurrence {evaluation_id, suggestion_index}` for a suggestion marked
`shipped` with a `version` returns, per row pattern, the occurrences, the
analyses with at least one and the occurrences per analysis for the analyses on
Harness versions before that version (`before`) and from it on
(`from_version`). Only analyses whose collection finished count (`signals` is
empty both when nothing was found and before collection), and analyses without a
semantic `harness_version` are left out and counted in `without_version`: the
manual ones, among them every reanalysis, so one turn is never counted twice.
Analyses are kept 30 days, so the earlier window is bounded by retention.

## Migration from prompt experiments

`eval::start`, `eval::rerun` and `eval::assert::*` are removed, and so is the
session comparison (`eval::compare-sessions` and its console tab), dropped on
request; the specification still lists it. The prompt experiments' records
stay in the `eval_job` and `eval_session` scopes untouched. Old `eval::step`
messages still queued are skipped (no matching analysis). Finish or cancel
old experiments before upgrading if their results matter.

The console page still shows the previous interface; it will be replaced by the
monitor interface in a separate change.

## Checks

```bash
cargo fmt --manifest-path eval/Cargo.toml --all -- --check
cargo test --locked --manifest-path eval/Cargo.toml --all-targets
cargo clippy --locked --manifest-path eval/Cargo.toml --all-targets -- -D warnings
# Replay the detectors over exported session::messages entries:
cargo run --manifest-path eval/Cargo.toml --example replay -- <session>=<entries.json>
```

`tests/flow.rs` runs the whole flow over the SDK wire protocol against an
in-memory engine (mocked state, queue, Harness, session-manager, router,
judge and E2E). It proves the monitor's mechanics, not the quality of its
suggestions or any Harness improvement.

## api reference

```json
{
  "functions": [
    {
      "description": "Analyze the latest definitive turn of a finished root Harness session. Returns the existing analysis of that turn unless reanalyze is true; reanalysis needs the previous analysis to be terminal. Works while automatic observation is paused.",
      "metadata": {},
      "name": "eval::analyze-session",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "reanalyze": {
            "default": false,
            "description": "Explicitly create another analysis of an already analyzed turn.",
            "type": "boolean"
          },
          "session_id": {
            "type": "string"
          }
        },
        "required": [
          "session_id"
        ],
        "title": "AnalyzeSessionRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "EvalStatusV1": {
            "enum": [
              "queued",
              "collecting",
              "judging",
              "investigating",
              "completed",
              "failed",
              "cancelled"
            ],
            "type": "string"
          }
        },
        "properties": {
          "evaluation_id": {
            "type": "string"
          },
          "reused": {
            "description": "True when an existing analysis of the same session turn was returned.",
            "type": "boolean"
          },
          "status": {
            "$ref": "#/definitions/EvalStatusV1"
          }
        },
        "required": [
          "evaluation_id",
          "reused",
          "status"
        ],
        "title": "AnalyzeSessionResponseV1",
        "type": "object"
      }
    },
    {
      "description": "Link a baseline and a candidate E2E execution (read through e2e::dashboard::execution-get) to one suggestion of a terminal analysis. The link records report availability and, when the suggestion's scenario is known, the evidence computed from the runs; it is not a verdict and starts no campaign.",
      "metadata": {},
      "name": "eval::attach-validation",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "baseline_execution_id": {
            "type": "string"
          },
          "candidate_execution_id": {
            "type": "string"
          },
          "dry_run": {
            "default": false,
            "description": "Look both executions up and return the link without saving it.",
            "type": "boolean"
          },
          "evaluation_id": {
            "type": "string"
          },
          "suggestion_index": {
            "format": "uint",
            "minimum": 0,
            "type": "integer"
          }
        },
        "required": [
          "baseline_execution_id",
          "candidate_execution_id",
          "evaluation_id",
          "suggestion_index"
        ],
        "title": "AttachValidationRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "ComparabilityCheckV1": {
            "additionalProperties": false,
            "properties": {
              "baseline": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "candidate": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "field": {
                "type": "string"
              },
              "matches": {
                "type": "boolean"
              }
            },
            "required": [
              "field",
              "matches"
            ],
            "type": "object"
          },
          "ComparabilityV1": {
            "additionalProperties": false,
            "description": "The conditions that must match for a fair comparison; only the Harness version is meant to differ.",
            "properties": {
              "checks": {
                "items": {
                  "$ref": "#/definitions/ComparabilityCheckV1"
                },
                "type": "array"
              },
              "comparable": {
                "type": "boolean"
              }
            },
            "required": [
              "checks",
              "comparable"
            ],
            "type": "object"
          },
          "E2eAssessmentsV1": {
            "additionalProperties": false,
            "properties": {
              "passed": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "total": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "passed",
              "total"
            ],
            "type": "object"
          },
          "E2eExecutionRefV1": {
            "additionalProperties": false,
            "description": "What `e2e::dashboard::execution-get` reported at lookup time.",
            "properties": {
              "assessments": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/E2eAssessmentsV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "availability": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "conclusion": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "e2e_revision": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "engine_version": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "evidence_error": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "execution_id": {
                "type": "string"
              },
              "harness_version": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "label": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "model": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "provider": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "reports": {
                "items": {
                  "$ref": "#/definitions/E2eReportRefV1"
                },
                "type": "array"
              },
              "reports_available": {
                "description": "True only when the execution reported retained reports.",
                "type": "boolean"
              },
              "scenarios": {
                "items": {
                  "$ref": "#/definitions/E2eScenarioV1"
                },
                "type": "array"
              },
              "started_at": {
                "description": "When the E2E started the execution (ms since the Unix epoch): its results cannot have existed before.",
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "status": {
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "execution_id",
              "reports",
              "reports_available",
              "scenarios"
            ],
            "type": "object"
          },
          "E2eMeasureV1": {
            "additionalProperties": false,
            "properties": {
              "average": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "samples": {
                "description": "Runs that reported this measure; fewer than `run_count` means some runs reported nothing (never counted as zero).",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "samples"
            ],
            "type": "object"
          },
          "E2eReportRefV1": {
            "additionalProperties": false,
            "properties": {
              "available": {
                "type": "boolean"
              },
              "scenario_id": {
                "type": "string"
              },
              "subject_id": {
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "available",
              "scenario_id"
            ],
            "type": "object"
          },
          "E2eScenarioV1": {
            "additionalProperties": false,
            "description": "One scenario's identity and the execution's own averages, as the E2E computed them.",
            "properties": {
              "behavior_sha256": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "completed_runs": {
                "format": "uint32",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "contract_fingerprint": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "cost_usd_per_run": {
                "description": "The cohort's total cost divided by its completed runs; the total also covers failed attempts.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "mean_score": {
                "description": "Mean score of the scored runs (0 to 100).",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "measures": {
                "additionalProperties": {
                  "$ref": "#/definitions/E2eMeasureV1"
                },
                "description": "`function_calls`, `function_call_errors`, `tokens`, `duration_seconds` and `cost_usd`, when reported.",
                "type": "object"
              },
              "median_wall_time_ms": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "p50_function_calls": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "pass_rate": {
                "description": "The scenario's cohort in `plan_execution.measurements.cohorts[]`, when the execution has exactly one for it. Each value below is absent when the E2E did not report it, never zero.\n\nShare of the planned runs that passed (0 to 1).",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "planned_runs": {
                "format": "uint32",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "run_count": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "scenario_id": {
                "type": "string"
              },
              "total_tokens_per_run": {
                "description": "The cohort's total tokens divided by its completed runs.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              }
            },
            "required": [
              "measures",
              "run_count",
              "scenario_id"
            ],
            "type": "object"
          },
          "ValidationLinkV1": {
            "additionalProperties": false,
            "description": "Baseline and candidate executions linked to one suggestion. A link is a reference, not a verdict: the improvement claim belongs to the E2E comparison and its criteria.",
            "properties": {
              "attached_at": {
                "format": "int64",
                "type": "integer"
              },
              "baseline": {
                "$ref": "#/definitions/E2eExecutionRefV1"
              },
              "candidate": {
                "$ref": "#/definitions/E2eExecutionRefV1"
              },
              "comparability": {
                "$ref": "#/definitions/ComparabilityV1"
              },
              "suggestion_index": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "attached_at",
              "baseline",
              "candidate",
              "comparability",
              "suggestion_index"
            ],
            "type": "object"
          }
        },
        "properties": {
          "link": {
            "$ref": "#/definitions/ValidationLinkV1"
          },
          "saved": {
            "type": "boolean"
          }
        },
        "required": [
          "link",
          "saved"
        ],
        "title": "AttachValidationResponseV1",
        "type": "object"
      }
    },
    {
      "description": "Cancel an analysis: signals its Jev call and stops its investigation session. Never touches the observed session or the captured evidence.",
      "metadata": {},
      "name": "eval::cancel",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "evaluation_id": {
            "type": "string"
          }
        },
        "required": [
          "evaluation_id"
        ],
        "title": "EvaluationIdRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "EvalStatusV1": {
            "enum": [
              "queued",
              "collecting",
              "judging",
              "investigating",
              "completed",
              "failed",
              "cancelled"
            ],
            "type": "string"
          }
        },
        "properties": {
          "cancelled": {
            "type": "boolean"
          },
          "status": {
            "$ref": "#/definitions/EvalStatusV1"
          }
        },
        "required": [
          "cancelled",
          "status"
        ],
        "title": "EvalCancelResponseV1",
        "type": "object"
      }
    },
    {
      "description": "Read the monitor configuration (null until configured), whether the harness::on-turn-completed observer is active, the latest rejection (capacity or daily cost cap), the day's known investigation cost against the cap with what one analysis has cost, and, with check_providers, whether the Jev triage provider answers.",
      "metadata": {},
      "name": "eval::config",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "check_providers": {
            "default": false,
            "description": "Also ask the triage provider whether it answers (`judge::models::list`).",
            "type": "boolean"
          }
        },
        "title": "MonitorStateRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "AnalysisCostStatsV1": {
            "additionalProperties": false,
            "description": "Cost of the completed analyses that investigated with the configured model, provider and code access (or lack of it).",
            "properties": {
              "count": {
                "description": "Analyses with a known cost; `min`, `median` and `max` come from them.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "max": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "median": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "min": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "unknown": {
                "description": "Analyses that investigated but reported no cost.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "count",
              "unknown"
            ],
            "type": "object"
          },
          "CapacityRejectionV1": {
            "additionalProperties": false,
            "properties": {
              "at": {
                "format": "int64",
                "type": "integer"
              },
              "reason": {
                "allOf": [
                  {
                    "$ref": "#/definitions/RejectionReasonV1"
                  }
                ],
                "default": "at_capacity",
                "description": "Why the turn was not admitted; rows saved before the cost cap existed read as `at_capacity`."
              },
              "session_id": {
                "type": "string"
              },
              "turn_id": {
                "type": "string"
              }
            },
            "required": [
              "at",
              "session_id",
              "turn_id"
            ],
            "type": "object"
          },
          "MonitorConfigV1": {
            "additionalProperties": false,
            "properties": {
              "code_repository": {
                "description": "The codebase directory the investigation reads (see `ConfigureRequestV1`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "daily_cost_cap_usd": {
                "description": "The daily cap on known investigation cost (see `ConfigureRequestV1`).",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "enabled": {
                "type": "boolean"
              },
              "enabled_since": {
                "description": "When automatic observation last went from paused to on; analyses use it to bound which earlier turns they may report. Unchanged by edits that keep observation on.",
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "model": {
                "$ref": "#/definitions/MonitorModelV1"
              },
              "revision": {
                "description": "SHA-256 of the effective configuration; analyses keep the revision they were admitted with.",
                "type": "string"
              },
              "updated_at": {
                "format": "int64",
                "type": "integer"
              }
            },
            "required": [
              "enabled",
              "model",
              "revision",
              "updated_at"
            ],
            "type": "object"
          },
          "MonitorCostV1": {
            "additionalProperties": false,
            "description": "The monitor's own spend. Only the investigation's LLM cost is in dollars; Jev's usage is reported in tokens on each record.",
            "properties": {
              "cap_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "capped": {
                "description": "A cap is set and `today_capture_usd`, plus the median cost of each investigation still running, reached it: automatic observation admits and investigates nothing until the next UTC day.",
                "type": "boolean"
              },
              "per_analysis": {
                "allOf": [
                  {
                    "$ref": "#/definitions/AnalysisCostStatsV1"
                  }
                ],
                "description": "What one investigation has cost, from the history."
              },
              "since": {
                "description": "Start (ms since the Unix epoch) of the UTC day the `today_*` values cover. The monitor has no timezone setting, so a day is a UTC day.",
                "format": "int64",
                "type": "integer"
              },
              "today_capture_usd": {
                "description": "The capture bucket, the only one the cap compares: the larger of the day's persisted capture spend (deleting an analysis does not give it back) and the sum of the stored analyses' `usage.llm_cost_usd`. A day persisted before the buckets counts entirely as capture.",
                "format": "double",
                "type": "number"
              },
              "today_replay_unknown": {
                "description": "Replay samples of the day without a cost, failed ones included. They add nothing to `today_replay_usd`, but their cost is unknown, not zero.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "today_replay_usd": {
                "description": "The replay bucket: the known cost of the day's `eval::reproduce` samples. Never capped.",
                "format": "double",
                "type": "number"
              },
              "today_unknown": {
                "description": "Analyses of the day whose investigation started but reported no cost. They add nothing to `today_capture_usd`, but their cost is unknown, not zero.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "today_usd": {
                "description": "Everything the monitor is known to have spent since `since`: `today_capture_usd` + `today_replay_usd`. The cap does not compare this.",
                "format": "double",
                "type": "number"
              }
            },
            "required": [
              "capped",
              "per_analysis",
              "since",
              "today_capture_usd",
              "today_replay_unknown",
              "today_replay_usd",
              "today_unknown",
              "today_usd"
            ],
            "type": "object"
          },
          "MonitorLimitsV1": {
            "additionalProperties": false,
            "properties": {
              "analysis_budget_ms": {
                "description": "Whole-analysis budget from admission, queue wait included.",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "assets_bytes": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "investigation_code_max_total_tokens": {
                "description": "Total-token cap of an investigation that has code access.",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "investigation_code_max_turns": {
                "description": "Generate-step cap of an investigation that has code access.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "investigation_max_output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "investigation_max_total_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "investigation_max_turns": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "judge_timeout_ms": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "max_active_analyses": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "model_context_bytes": {
                "description": "Serialized JSON shown to a model (bytes, not tokens).",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "queue_concurrency": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "retention_days": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "retention_max_terminal": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "analysis_budget_ms",
              "assets_bytes",
              "investigation_code_max_total_tokens",
              "investigation_code_max_turns",
              "investigation_max_output_tokens",
              "investigation_max_total_tokens",
              "investigation_max_turns",
              "judge_timeout_ms",
              "max_active_analyses",
              "model_context_bytes",
              "queue_concurrency",
              "retention_days",
              "retention_max_terminal"
            ],
            "type": "object"
          },
          "MonitorModelV1": {
            "additionalProperties": false,
            "description": "The Harness model the user chose for investigations. Credentials stay in the provider worker; this request never carries a key.",
            "properties": {
              "model": {
                "type": "string"
              },
              "provider": {
                "type": "string"
              },
              "provider_options": {
                "additionalProperties": true,
                "description": "Provider-native options, namespaced by the selected provider id.",
                "type": [
                  "object",
                  "null"
                ]
              },
              "thinking_level": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ThinkingLevel"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "required": [
              "model",
              "provider"
            ],
            "type": "object"
          },
          "RejectionReasonV1": {
            "oneOf": [
              {
                "description": "The unfinished-analyses cap was reached.",
                "enum": [
                  "at_capacity"
                ],
                "type": "string"
              },
              {
                "description": "The day's known cost reached `daily_cost_cap_usd`.",
                "enum": [
                  "cost_cap"
                ],
                "type": "string"
              }
            ]
          },
          "ThinkingLevel": {
            "oneOf": [
              {
                "enum": [
                  "minimal",
                  "low",
                  "medium",
                  "high",
                  "xhigh"
                ],
                "type": "string"
              },
              {
                "description": "No reasoning at all; honoured only where the model advertises `supports_thinking_off` (elsewhere the provider falls back to its own lowest setting, or omits the parameter, and warns).",
                "enum": [
                  "off"
                ],
                "type": "string"
              }
            ]
          },
          "TriageAvailabilityV1": {
            "additionalProperties": false,
            "description": "What `judge::models::list` answered for the triage provider. The key stays in the provider; this only reports whether it answers.",
            "properties": {
              "available": {
                "type": "boolean"
              },
              "checked_at": {
                "format": "int64",
                "type": "integer"
              },
              "code": {
                "description": "The provider's error code (`missing_key`, `provider_unavailable`…), or `unreachable` when the hub did not answer.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "models": {
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "provider": {
                "type": "string"
              }
            },
            "required": [
              "available",
              "checked_at",
              "models",
              "provider"
            ],
            "type": "object"
          }
        },
        "properties": {
          "config": {
            "anyOf": [
              {
                "$ref": "#/definitions/MonitorConfigV1"
              },
              {
                "type": "null"
              }
            ],
            "description": "`null` until the user configures the monitor."
          },
          "cost": {
            "allOf": [
              {
                "$ref": "#/definitions/MonitorCostV1"
              }
            ],
            "description": "What the investigations cost so far, to decide before enabling."
          },
          "last_rejection": {
            "anyOf": [
              {
                "$ref": "#/definitions/CapacityRejectionV1"
              },
              {
                "type": "null"
              }
            ],
            "description": "The latest turn not admitted: the monitor was at capacity or the daily cost cap was reached (`reason`)."
          },
          "limits": {
            "allOf": [
              {
                "$ref": "#/definitions/MonitorLimitsV1"
              }
            ],
            "description": "The limits this version enforces, so the console never restates them."
          },
          "observer_bound": {
            "description": "True while the `harness::on-turn-completed` observer is active. False while it is pending (no harness has registered the function) or after the engine refused it (`observer_error` says why): automatic observation is unavailable even if `config.enabled` is true.",
            "type": "boolean"
          },
          "observer_error": {
            "type": [
              "string",
              "null"
            ]
          },
          "triage": {
            "anyOf": [
              {
                "$ref": "#/definitions/TriageAvailabilityV1"
              },
              {
                "type": "null"
              }
            ],
            "description": "Present when the request asked to check the triage provider."
          }
        },
        "required": [
          "cost",
          "limits",
          "observer_bound"
        ],
        "title": "MonitorStateResponseV1",
        "type": "object"
      }
    },
    {
      "description": "Choose the Harness model/provider the session monitor uses for investigations and enable or pause automatic observation. A new selection is checked against router::models::list; credentials stay in the providers and are never accepted here. An optional code_repository (an absolute directory on this host, normally the iii workers repository) lets the investigation read that code like a chat with the directory selected, with only read-only functions allowed (coder::search, coder::tree, coder::read-file, github::pr::list and engine::functions::info). An optional daily_cost_cap_usd pauses automatic observation for the rest of the UTC day once the known investigation cost reaches it (manual analyses are never refused, and replay spend is reported but not counted). Analyses already admitted keep the configuration they started with.",
      "metadata": {},
      "name": "eval::configure",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "MonitorModelV1": {
            "additionalProperties": false,
            "description": "The Harness model the user chose for investigations. Credentials stay in the provider worker; this request never carries a key.",
            "properties": {
              "model": {
                "type": "string"
              },
              "provider": {
                "type": "string"
              },
              "provider_options": {
                "additionalProperties": true,
                "description": "Provider-native options, namespaced by the selected provider id.",
                "type": [
                  "object",
                  "null"
                ]
              },
              "thinking_level": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ThinkingLevel"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "required": [
              "model",
              "provider"
            ],
            "type": "object"
          },
          "ThinkingLevel": {
            "oneOf": [
              {
                "enum": [
                  "minimal",
                  "low",
                  "medium",
                  "high",
                  "xhigh"
                ],
                "type": "string"
              },
              {
                "description": "No reasoning at all; honoured only where the model advertises `supports_thinking_off` (elsewhere the provider falls back to its own lowest setting, or omits the parameter, and warns).",
                "enum": [
                  "off"
                ],
                "type": "string"
              }
            ]
          }
        },
        "properties": {
          "_caller_worker_id": {
            "description": "Stamped by the engine on every invocation; callers omit it. Listed so the closed schema still accepts the engine's metadata.",
            "type": [
              "string",
              "null"
            ],
            "writeOnly": true
          },
          "code_repository": {
            "description": "Absolute path of the codebase directory the investigation reads and runs in (normally the iii workers repository, checked out on the monitor's host); absent means no code access.",
            "type": [
              "string",
              "null"
            ]
          },
          "daily_cost_cap_usd": {
            "description": "US dollars of known investigation cost per UTC day after which automatic observation admits nothing until the next day; absent means no cap. A manual `eval::analyze-session` is never refused by it.",
            "format": "double",
            "type": [
              "number",
              "null"
            ]
          },
          "enabled": {
            "type": "boolean"
          },
          "model": {
            "$ref": "#/definitions/MonitorModelV1"
          }
        },
        "required": [
          "enabled",
          "model"
        ],
        "title": "ConfigureRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "MonitorModelV1": {
            "additionalProperties": false,
            "description": "The Harness model the user chose for investigations. Credentials stay in the provider worker; this request never carries a key.",
            "properties": {
              "model": {
                "type": "string"
              },
              "provider": {
                "type": "string"
              },
              "provider_options": {
                "additionalProperties": true,
                "description": "Provider-native options, namespaced by the selected provider id.",
                "type": [
                  "object",
                  "null"
                ]
              },
              "thinking_level": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ThinkingLevel"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "required": [
              "model",
              "provider"
            ],
            "type": "object"
          },
          "ThinkingLevel": {
            "oneOf": [
              {
                "enum": [
                  "minimal",
                  "low",
                  "medium",
                  "high",
                  "xhigh"
                ],
                "type": "string"
              },
              {
                "description": "No reasoning at all; honoured only where the model advertises `supports_thinking_off` (elsewhere the provider falls back to its own lowest setting, or omits the parameter, and warns).",
                "enum": [
                  "off"
                ],
                "type": "string"
              }
            ]
          }
        },
        "properties": {
          "code_repository": {
            "description": "The codebase directory the investigation reads (see `ConfigureRequestV1`).",
            "type": [
              "string",
              "null"
            ]
          },
          "daily_cost_cap_usd": {
            "description": "The daily cap on known investigation cost (see `ConfigureRequestV1`).",
            "format": "double",
            "type": [
              "number",
              "null"
            ]
          },
          "enabled": {
            "type": "boolean"
          },
          "enabled_since": {
            "description": "When automatic observation last went from paused to on; analyses use it to bound which earlier turns they may report. Unchanged by edits that keep observation on.",
            "format": "int64",
            "type": [
              "integer",
              "null"
            ]
          },
          "model": {
            "$ref": "#/definitions/MonitorModelV1"
          },
          "revision": {
            "description": "SHA-256 of the effective configuration; analyses keep the revision they were admitted with.",
            "type": "string"
          },
          "updated_at": {
            "format": "int64",
            "type": "integer"
          }
        },
        "required": [
          "enabled",
          "model",
          "revision",
          "updated_at"
        ],
        "title": "MonitorConfigV1",
        "type": "object"
      }
    },
    {
      "description": "Delete a terminal analysis and its stored previews. The turn stays marked as analyzed until retention, so it is not re-admitted automatically.",
      "metadata": {},
      "name": "eval::delete",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "evaluation_id": {
            "type": "string"
          }
        },
        "required": [
          "evaluation_id"
        ],
        "title": "EvaluationIdRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "properties": {
          "deleted": {
            "type": "boolean"
          }
        },
        "required": [
          "deleted"
        ],
        "title": "EvalDeleteResponseV1",
        "type": "object"
      }
    },
    {
      "description": "List recent monitor analyses as compact records, newest first, without evidence; observation_key keeps only the analyses of one observed turn.",
      "metadata": {},
      "name": "eval::list",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "limit": {
            "default": null,
            "format": "uint32",
            "minimum": 0,
            "type": [
              "integer",
              "null"
            ]
          },
          "observation_key": {
            "default": null,
            "description": "Only the analyses of this observed turn (the `observation_key` of a record), reanalyses included.",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "EvalListRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "AnalysisCountersV1": {
            "additionalProperties": false,
            "properties": {
              "diagnostics": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "entries": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "rejected_suggestions": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "sessions": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "suggestions": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "validations": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "diagnostics",
              "entries",
              "rejected_suggestions",
              "sessions",
              "suggestions",
              "validations"
            ],
            "type": "object"
          },
          "AnalysisOriginV1": {
            "enum": [
              "automatic",
              "manual"
            ],
            "type": "string"
          },
          "AnalysisRecordV1": {
            "additionalProperties": false,
            "description": "The compact analysis record listed by `eval::list`; evidence and model outputs live separately in the analysis assets.",
            "properties": {
              "analyst": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/AnalystRefV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "code_root": {
                "description": "Frozen at admission with the model: the directory the investigation reads and runs in, absent when code access was off.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "completed_at": {
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "config_revision": {
                "type": "string"
              },
              "counters": {
                "$ref": "#/definitions/AnalysisCountersV1"
              },
              "coverage": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/CoverageLevelV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "created_at": {
                "format": "int64",
                "type": "integer"
              },
              "criteria_version": {
                "type": "string"
              },
              "deadline": {
                "format": "int64",
                "type": "integer"
              },
              "evaluation_id": {
                "type": "string"
              },
              "failure": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/FailureV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "harness_version": {
                "description": "The Harness version the engine reported when a live event admitted the analysis (the turn just ended on it); absent on a manual analysis, which may be of an older session, and when the engine did not say.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "judge_call": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/JudgeCallV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "model": {
                "allOf": [
                  {
                    "$ref": "#/definitions/MonitorModelV1"
                  }
                ],
                "description": "Frozen at admission: later configuration changes never alter it."
              },
              "observation_key": {
                "type": "string"
              },
              "observe_since": {
                "description": "Earlier root turns join the window only if they started after this instant: the monitor was observing then. Turns from before the monitor was enabled are never reported as new.",
                "format": "int64",
                "type": "integer"
              },
              "origin": {
                "$ref": "#/definitions/AnalysisOriginV1"
              },
              "pending_reason": {
                "description": "Why collection is still waiting (descendants running, metrics incomplete).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "routing": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/RoutingV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "rules_version": {
                "type": "string"
              },
              "schema_version": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "session_id": {
                "type": "string"
              },
              "signals": {
                "additionalProperties": {
                  "format": "uint32",
                  "minimum": 0,
                  "type": "integer"
                },
                "description": "Occurrences of each deterministic pattern the collection found, keyed `<rule_id>:<target>` (a target may itself contain colons). Empty both when nothing was found and before collection (`coverage` is set once collection read the evidence).",
                "type": "object"
              },
              "source_title": {
                "description": "The session's title, once collection has read it.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "stages": {
                "description": "When each status began, oldest first.",
                "items": {
                  "$ref": "#/definitions/StageTimeV1"
                },
                "type": "array"
              },
              "status": {
                "$ref": "#/definitions/EvalStatusV1"
              },
              "step": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "supersedes": {
                "description": "The earlier analysis of the same turn this reanalysis was requested over (same `observation_key`); absent on a first analysis.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "turn_id": {
                "type": "string"
              },
              "updated_at": {
                "format": "int64",
                "type": "integer"
              },
              "usage": {
                "allOf": [
                  {
                    "$ref": "#/definitions/MonitorUsageV1"
                  }
                ],
                "description": "The monitor's own consumption, apart from the observed task."
              }
            },
            "required": [
              "config_revision",
              "counters",
              "created_at",
              "criteria_version",
              "deadline",
              "evaluation_id",
              "model",
              "observation_key",
              "observe_since",
              "origin",
              "rules_version",
              "schema_version",
              "session_id",
              "stages",
              "status",
              "step",
              "turn_id",
              "updated_at",
              "usage"
            ],
            "type": "object"
          },
          "AnalystRefV1": {
            "additionalProperties": false,
            "properties": {
              "sent_at": {
                "format": "int64",
                "type": "integer"
              },
              "session_id": {
                "type": "string"
              },
              "turn_id": {
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "sent_at",
              "session_id"
            ],
            "type": "object"
          },
          "CoverageLevelV1": {
            "oneOf": [
              {
                "description": "Every in-scope transcript was read and the model context holds every diagnostic's evidence.",
                "enum": [
                  "complete"
                ],
                "type": "string"
              },
              {
                "description": "Previews omit or shorten entries; the full transcripts were read.",
                "enum": [
                  "partial"
                ],
                "type": "string"
              },
              {
                "description": "Required evidence did not fit; no conclusion of healthy behavior.",
                "enum": [
                  "insufficient"
                ],
                "type": "string"
              }
            ]
          },
          "EvalStatusV1": {
            "enum": [
              "queued",
              "collecting",
              "judging",
              "investigating",
              "completed",
              "failed",
              "cancelled"
            ],
            "type": "string"
          },
          "FailureV1": {
            "additionalProperties": false,
            "description": "The stage that failed and why. `stage` is the status the analysis was in.",
            "properties": {
              "code": {
                "type": "string"
              },
              "message": {
                "type": "string"
              },
              "stage": {
                "$ref": "#/definitions/EvalStatusV1"
              }
            },
            "required": [
              "code",
              "message",
              "stage"
            ],
            "type": "object"
          },
          "JudgeCallV1": {
            "additionalProperties": false,
            "description": "A Jev call started by this analysis. Persisted before the call so a restart can tell a lost answer from one never requested.",
            "properties": {
              "deadline": {
                "format": "int64",
                "type": "integer"
              },
              "request_id": {
                "type": "string"
              },
              "started_at": {
                "format": "int64",
                "type": "integer"
              }
            },
            "required": [
              "deadline",
              "request_id",
              "started_at"
            ],
            "type": "object"
          },
          "MonitorModelV1": {
            "additionalProperties": false,
            "description": "The Harness model the user chose for investigations. Credentials stay in the provider worker; this request never carries a key.",
            "properties": {
              "model": {
                "type": "string"
              },
              "provider": {
                "type": "string"
              },
              "provider_options": {
                "additionalProperties": true,
                "description": "Provider-native options, namespaced by the selected provider id.",
                "type": [
                  "object",
                  "null"
                ]
              },
              "thinking_level": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ThinkingLevel"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "required": [
              "model",
              "provider"
            ],
            "type": "object"
          },
          "MonitorUsageV1": {
            "additionalProperties": false,
            "description": "Known usage only: absent counters are unknown, never zero.",
            "properties": {
              "judge_calls": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "judge_input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "judge_output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "judge_usage_complete": {
                "description": "False when Jev reported incomplete usage or failed.",
                "type": "boolean"
              },
              "llm_cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "llm_input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "llm_output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "required": [
              "judge_calls",
              "judge_input_tokens",
              "judge_output_tokens",
              "judge_usage_complete"
            ],
            "type": "object"
          },
          "RoutingReasonV1": {
            "oneOf": [
              {
                "enum": [
                  "needs_investigation"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced; kept so stored records still deserialize.",
                "enum": [
                  "diagnostics"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "insufficient_evidence"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "low_confidence"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "coverage_insufficient"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "audit_sample"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "manual_request"
                ],
                "type": "string"
              }
            ]
          },
          "RoutingV1": {
            "additionalProperties": false,
            "properties": {
              "investigate": {
                "type": "boolean"
              },
              "reasons": {
                "items": {
                  "$ref": "#/definitions/RoutingReasonV1"
                },
                "type": "array"
              }
            },
            "required": [
              "investigate",
              "reasons"
            ],
            "type": "object"
          },
          "StageTimeV1": {
            "additionalProperties": false,
            "properties": {
              "at": {
                "format": "int64",
                "type": "integer"
              },
              "status": {
                "$ref": "#/definitions/EvalStatusV1"
              }
            },
            "required": [
              "at",
              "status"
            ],
            "type": "object"
          },
          "ThinkingLevel": {
            "oneOf": [
              {
                "enum": [
                  "minimal",
                  "low",
                  "medium",
                  "high",
                  "xhigh"
                ],
                "type": "string"
              },
              {
                "description": "No reasoning at all; honoured only where the model advertises `supports_thinking_off` (elsewhere the provider falls back to its own lowest setting, or omits the parameter, and warns).",
                "enum": [
                  "off"
                ],
                "type": "string"
              }
            ]
          }
        },
        "properties": {
          "evaluations": {
            "items": {
              "$ref": "#/definitions/AnalysisRecordV1"
            },
            "type": "array"
          }
        },
        "required": [
          "evaluations"
        ],
        "title": "EvalListResponseV1",
        "type": "object"
      }
    },
    {
      "description": "Called when a session-monitor analysis reaches a terminal status (completed, failed or cancelled). No args; filter on input.evaluation_id or input.status. Completion describes the monitor, not a validated improvement.",
      "metadata": {},
      "name": "eval::on-completed",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "EvalStatusV1": {
            "enum": [
              "queued",
              "collecting",
              "judging",
              "investigating",
              "completed",
              "failed",
              "cancelled"
            ],
            "type": "string"
          }
        },
        "description": "The `eval::on-completed` event.",
        "properties": {
          "evaluation_id": {
            "type": "string"
          },
          "status": {
            "$ref": "#/definitions/EvalStatusV1"
          },
          "timestamp": {
            "description": "Unix milliseconds when the analysis ended.",
            "format": "int64",
            "type": "integer"
          }
        },
        "required": [
          "evaluation_id",
          "status",
          "timestamp"
        ],
        "title": "CompletedEventV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "EvalStatusV1": {
            "enum": [
              "queued",
              "collecting",
              "judging",
              "investigating",
              "completed",
              "failed",
              "cancelled"
            ],
            "type": "string"
          }
        },
        "description": "The `eval::on-completed` event.",
        "properties": {
          "evaluation_id": {
            "type": "string"
          },
          "status": {
            "$ref": "#/definitions/EvalStatusV1"
          },
          "timestamp": {
            "description": "Unix milliseconds when the analysis ended.",
            "format": "int64",
            "type": "integer"
          }
        },
        "required": [
          "evaluation_id",
          "status",
          "timestamp"
        ],
        "title": "CompletedEventV1",
        "type": "object"
      }
    },
    {
      "description": "Internal: admission and wake-up for harness::on-turn-completed.",
      "metadata": {
        "internal": true,
        "trace_hidden": true
      },
      "name": "eval::on-turn-completed",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "The `harness::on-turn-completed` event; extra fields are tolerated.",
        "properties": {
          "parent": {
            "default": null
          },
          "parent_session_id": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          },
          "session_id": {
            "default": "",
            "type": "string"
          },
          "terminal": {
            "default": false,
            "type": "boolean"
          },
          "timestamp": {
            "default": null,
            "format": "int64",
            "type": [
              "integer",
              "null"
            ]
          },
          "turn_id": {
            "default": "",
            "type": "string"
          }
        },
        "title": "WakeEventV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "WakeOutcomeV1": {
            "enum": [
              "ignored",
              "progress",
              "descendant",
              "monitor_session",
              "disabled",
              "stale",
              "admitted",
              "reused",
              "at_capacity",
              "cost_cap",
              "not_user_chat"
            ],
            "type": "string"
          }
        },
        "properties": {
          "evaluation_id": {
            "type": [
              "string",
              "null"
            ]
          },
          "outcome": {
            "$ref": "#/definitions/WakeOutcomeV1"
          }
        },
        "required": [
          "outcome"
        ],
        "title": "WakeResponseV1",
        "type": "object"
      }
    },
    {
      "description": "For a suggestion marked shipped with its version, how often its patterns were found per analysis on Harness versions before it against from it on. Analyses without a semantic Harness version are left out and counted.",
      "metadata": {},
      "name": "eval::recurrence",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "evaluation_id": {
            "type": "string"
          },
          "suggestion_index": {
            "format": "uint",
            "minimum": 0,
            "type": "integer"
          }
        },
        "required": [
          "evaluation_id",
          "suggestion_index"
        ],
        "title": "RecurrenceRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "RecurrencePatternV1": {
            "additionalProperties": false,
            "description": "One pattern over a set of analyses.",
            "properties": {
              "analyses_with": {
                "description": "Analyses with at least one occurrence.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "occurrences": {
                "description": "Occurrences summed over the analyses.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "pattern": {
                "type": "string"
              },
              "per_analysis": {
                "description": "Occurrences per analysis; absent when there is no analysis.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              }
            },
            "required": [
              "analyses_with",
              "occurrences",
              "pattern"
            ],
            "type": "object"
          },
          "RecurrenceWindowV1": {
            "additionalProperties": false,
            "properties": {
              "analyses": {
                "description": "Analyses whose collection finished and whose Harness version is known.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "patterns": {
                "items": {
                  "$ref": "#/definitions/RecurrencePatternV1"
                },
                "type": "array"
              }
            },
            "required": [
              "analyses",
              "patterns"
            ],
            "type": "object"
          }
        },
        "description": "Whether the suggestion's patterns came back: the analyses on Harness versions before the one that shipped it against the ones from it on. Analyses are kept 30 days, so the earlier window is bounded by retention.",
        "properties": {
          "before": {
            "$ref": "#/definitions/RecurrenceWindowV1"
          },
          "evaluation_id": {
            "type": "string"
          },
          "from_version": {
            "$ref": "#/definitions/RecurrenceWindowV1"
          },
          "suggestion_index": {
            "format": "uint",
            "minimum": 0,
            "type": "integer"
          },
          "version": {
            "description": "The version the suggestion shipped in, where the windows split.",
            "type": "string"
          },
          "without_version": {
            "description": "Analyses left out because they carry no (or no semantic) Harness version.",
            "format": "uint32",
            "minimum": 0,
            "type": "integer"
          }
        },
        "required": [
          "before",
          "evaluation_id",
          "from_version",
          "suggestion_index",
          "version",
          "without_version"
        ],
        "title": "RecurrenceResponseV1",
        "type": "object"
      }
    },
    {
      "description": "Replay the decision point of a suggestion: rebuild the request the model received at that step of the observed turn (optionally with edits that stand for the proposed change), sample the next reply N times (default 20) without running any function, and read the suggestion's signal in each reply. Spends model tokens; with dry_run it only rebuilds the request, checks its fidelity against the recorded usage and reports the original reply. extend adds replies to an earlier reproduction, or with samples 0 finishes one that failed. Results land in eval::result reviews.",
      "metadata": {},
      "name": "eval::reproduce",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "ChangeEditV1": {
            "additionalProperties": false,
            "description": "One edit of what the model saw. `target` is an entry id of the conversation before the decision point, or `system_prompt`. An edit either replaces `find` with `replace` in the target's text or removes the entry.",
            "properties": {
              "find": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "remove": {
                "type": "boolean"
              },
              "replace": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "target": {
                "type": "string"
              }
            },
            "required": [
              "target"
            ],
            "type": "object"
          },
          "ReproduceChangeV1": {
            "description": "What a reproduction changes in the request.",
            "oneOf": [
              {
                "properties": {
                  "kind": {
                    "enum": [
                      "none"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "kind"
                ],
                "type": "object"
              },
              {
                "properties": {
                  "kind": {
                    "enum": [
                      "proposed"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "kind"
                ],
                "type": "object"
              },
              {
                "properties": {
                  "edits": {
                    "items": {
                      "$ref": "#/definitions/ChangeEditV1"
                    },
                    "type": "array"
                  },
                  "kind": {
                    "enum": [
                      "custom"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "edits",
                  "kind"
                ],
                "type": "object"
              }
            ]
          },
          "SignalRuleV1": {
            "oneOf": [
              {
                "description": "The reply asks `engine::functions::info` only for contracts an earlier result in the context already supplied.",
                "enum": [
                  "contract_rediscovery"
                ],
                "type": "string"
              },
              {
                "description": "The reply repeats the last call that failed: same target, equal payload.",
                "enum": [
                  "repeated_error_call"
                ],
                "type": "string"
              }
            ]
          },
          "SignalV1": {
            "additionalProperties": false,
            "description": "How to recognize the behavior in one reply: exactly one of a rule computed in code or a yes/no question Jev answers about the reply.",
            "properties": {
              "question": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "rule": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/SignalRuleV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "type": "object"
          },
          "SuggestionCheckV1": {
            "additionalProperties": false,
            "description": "How a suggestion is checked at the step where its behavior happened.",
            "properties": {
              "change": {
                "default": [],
                "description": "The proposed change as edits of what the model saw before that step; empty when it cannot be expressed as text.",
                "items": {
                  "$ref": "#/definitions/ChangeEditV1"
                },
                "type": "array"
              },
              "decision_point": {
                "description": "The assistant entry where the behavior happened (`…_<step>_assistant`), one of the suggestion's `evidence` entries.",
                "type": "string"
              },
              "signal": {
                "$ref": "#/definitions/SignalV1"
              }
            },
            "required": [
              "decision_point",
              "signal"
            ],
            "type": "object"
          }
        },
        "description": "`eval::reproduce`.",
        "properties": {
          "_caller_worker_id": {
            "default": null,
            "description": "Stamped by the engine on every invocation; callers omit it.",
            "type": [
              "string",
              "null"
            ]
          },
          "by": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          },
          "change": {
            "allOf": [
              {
                "$ref": "#/definitions/ReproduceChangeV1"
              }
            ],
            "default": {
              "kind": "none"
            }
          },
          "check": {
            "anyOf": [
              {
                "$ref": "#/definitions/SuggestionCheckV1"
              },
              {
                "type": "null"
              }
            ],
            "default": null,
            "description": "The decision point and signal, for a suggestion without a `check` (an older analysis) or to override it."
          },
          "dry_run": {
            "default": false,
            "description": "Rebuild the request and check its fidelity without sampling: nothing is spent or stored.",
            "type": "boolean"
          },
          "evaluation_id": {
            "type": "string"
          },
          "extend": {
            "default": null,
            "description": "Add replies to this earlier reproduction of the suggestion, with its change; with `samples: 0`, only finish what a failure left undone.",
            "type": [
              "string",
              "null"
            ]
          },
          "samples": {
            "default": null,
            "description": "Replies to sample, 1–50; default 20.",
            "format": "uint32",
            "minimum": 0,
            "type": [
              "integer",
              "null"
            ]
          },
          "suggestion_index": {
            "format": "uint",
            "minimum": 0,
            "type": "integer"
          }
        },
        "required": [
          "evaluation_id",
          "suggestion_index"
        ],
        "title": "ReproduceRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "ChangeEditV1": {
            "additionalProperties": false,
            "description": "One edit of what the model saw. `target` is an entry id of the conversation before the decision point, or `system_prompt`. An edit either replaces `find` with `replace` in the target's text or removes the entry.",
            "properties": {
              "find": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "remove": {
                "type": "boolean"
              },
              "replace": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "target": {
                "type": "string"
              }
            },
            "required": [
              "target"
            ],
            "type": "object"
          },
          "CriterionMetricV1": {
            "description": "What a validation measures. Every metric but `signal_per_run` is the mean over the runs of the scenario the infrastructure did not break (`pass_rate` is the share that passed, an unfinished run counting as not passed; `duration` is the wall time in milliseconds).",
            "oneOf": [
              {
                "enum": [
                  "pass_rate",
                  "cost_usd",
                  "duration",
                  "tokens",
                  "function_calls"
                ],
                "type": "string"
              },
              {
                "description": "Occurrences of one of the suggestion's patterns per run, counted by the deterministic detectors over each run's transcript.",
                "enum": [
                  "signal_per_run"
                ],
                "type": "string"
              }
            ]
          },
          "CriterionV1": {
            "additionalProperties": false,
            "properties": {
              "direction": {
                "$ref": "#/definitions/DirectionV1"
              },
              "metric": {
                "$ref": "#/definitions/CriterionMetricV1"
              },
              "min_effect": {
                "format": "double",
                "type": "number"
              },
              "min_runs": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "pattern": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "registered_at": {
                "format": "int64",
                "type": "integer"
              },
              "registered_by": {
                "type": "string"
              },
              "scenario_id": {
                "description": "The scenario whose runs are measured.",
                "type": "string"
              }
            },
            "required": [
              "direction",
              "metric",
              "min_effect",
              "min_runs",
              "registered_at",
              "registered_by",
              "scenario_id"
            ],
            "type": "object"
          },
          "DirectionV1": {
            "enum": [
              "decrease",
              "increase"
            ],
            "type": "string"
          },
          "EvidenceRunV1": {
            "additionalProperties": false,
            "description": "One run of the scenario as the E2E reported it. Runs the infrastructure broke are listed with the E2E's own words, never counted.",
            "properties": {
              "completed": {
                "description": "The E2E reported the run complete with a valid technical outcome.",
                "type": "boolean"
              },
              "completion": {
                "description": "`completion` as the E2E reported it.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "function_calls": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "run_id": {
                "type": "string"
              },
              "signals": {
                "additionalProperties": {
                  "format": "uint32",
                  "minimum": 0,
                  "type": "integer"
                },
                "description": "Occurrences of each of the row's patterns the detectors found in this run's transcript (0 is a measurement); absent when the report held no transcript.",
                "type": [
                  "object",
                  "null"
                ]
              },
              "status": {
                "description": "`passed` or `failed`, the run's task outcome.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "technical": {
                "description": "`technical` as the E2E reported it; anything but `valid` is an infrastructure failure.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "total_tokens": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "wall_time_ms": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              }
            },
            "required": [
              "completed",
              "run_id"
            ],
            "type": "object"
          },
          "EvidenceSideV1": {
            "additionalProperties": false,
            "properties": {
              "execution_id": {
                "type": "string"
              },
              "mean": {
                "description": "Mean of the criterion's metric over those runs; absent without a criterion or without such runs.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "n": {
                "description": "Measurable runs (see `EvidenceRunV1::measurable`) with a value of the criterion's metric (every measurable run while there is no criterion).",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "runs": {
                "description": "Every run of the scenario the execution holds.",
                "items": {
                  "$ref": "#/definitions/EvidenceRunV1"
                },
                "type": "array"
              }
            },
            "required": [
              "execution_id",
              "n",
              "runs"
            ],
            "type": "object"
          },
          "EvidenceV1": {
            "additionalProperties": false,
            "description": "Computed in code from the attached pair's runs, never by a model. It is what the person's verdict is shown next to.",
            "properties": {
              "baseline": {
                "$ref": "#/definitions/EvidenceSideV1"
              },
              "candidate": {
                "$ref": "#/definitions/EvidenceSideV1"
              },
              "computed_at": {
                "format": "int64",
                "type": "integer"
              },
              "computed_outcome": {
                "$ref": "#/definitions/ValidationOutcomeV1"
              },
              "reason": {
                "type": "string"
              },
              "scenario_id": {
                "type": "string"
              }
            },
            "required": [
              "baseline",
              "candidate",
              "computed_at",
              "computed_outcome",
              "reason",
              "scenario_id"
            ],
            "type": "object"
          },
          "FidelityLevelV1": {
            "oneOf": [
              {
                "enum": [
                  "approximate"
                ],
                "type": "string"
              },
              {
                "description": "The provider counts the rebuilt request like the original, up to a fixed per-request overhead measured at the turn's first step.",
                "enum": [
                  "exact"
                ],
                "type": "string"
              }
            ]
          },
          "FidelityV1": {
            "additionalProperties": false,
            "description": "How faithfully the request at the decision point was rebuilt.",
            "properties": {
              "counted_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "estimator": {
                "description": "`provider` when the provider counted, else the estimator's name.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "level": {
                "$ref": "#/definitions/FidelityLevelV1"
              },
              "off_ratio": {
                "description": "The unexplained difference over the recorded tokens.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "reasons": {
                "default": [],
                "description": "Why the rebuild is approximate; empty when exact.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "recorded_tokens": {
                "description": "Input tokens of the original step: input + cache read + cache write.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "required": [
              "level"
            ],
            "type": "object"
          },
          "LifecycleEventV1": {
            "additionalProperties": false,
            "description": "One change of status, with who made it.",
            "properties": {
              "at": {
                "format": "int64",
                "type": "integer"
              },
              "by": {
                "type": "string"
              },
              "note": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "status": {
                "$ref": "#/definitions/LifecycleStatusV1"
              }
            },
            "required": [
              "at",
              "by",
              "status"
            ],
            "type": "object"
          },
          "LifecycleStatusV1": {
            "description": "Where a suggestion stands. Only people move it: the monitor never does.",
            "enum": [
              "new",
              "accepted",
              "in_progress",
              "shipped",
              "rejected",
              "duplicate"
            ],
            "type": "string"
          },
          "LifecycleV1": {
            "additionalProperties": false,
            "properties": {
              "duplicate_of": {
                "description": "What it duplicates: a pull request, or another `<evaluation_id>:<index>` (`duplicate`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "history": {
                "description": "Every change of status, oldest first; empty while `new`.",
                "items": {
                  "$ref": "#/definitions/LifecycleEventV1"
                },
                "type": "array"
              },
              "pr": {
                "description": "The pull request that implements it (`in_progress`, `shipped`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "reason": {
                "description": "Why it was rejected (`rejected`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "status": {
                "$ref": "#/definitions/LifecycleStatusV1"
              },
              "version": {
                "description": "The Harness version that shipped it, a semantic version like `1.8.43`; the recurrence query splits the analyses at it.",
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "history",
              "status"
            ],
            "type": "object"
          },
          "ReplyCallV1": {
            "additionalProperties": false,
            "description": "One function call of a reply.",
            "properties": {
              "description": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "payload": {
                "description": "The payload as JSON text, cut at 600 characters.",
                "type": "string"
              },
              "target": {
                "description": "The function called (through `agent_call` or directly).",
                "type": "string"
              }
            },
            "required": [
              "payload",
              "target"
            ],
            "type": "object"
          },
          "ReplyUsageV1": {
            "additionalProperties": false,
            "properties": {
              "cache_read_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cache_write_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          },
          "ReplyV1": {
            "additionalProperties": false,
            "description": "One reply at the decision point: a sample, or the original step read the same way.",
            "properties": {
              "calls": {
                "items": {
                  "$ref": "#/definitions/ReplyCallV1"
                },
                "type": "array"
              },
              "duration_ms": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "error": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "index": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "signal": {
                "description": "Whether the signal shows in this reply; null when Jev answered unclear, the reply failed or it was not classified yet.",
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "text": {
                "type": "string"
              },
              "thinking": {
                "description": "The opening of the reply's reasoning and of its text, 400 characters each.",
                "type": "string"
              },
              "usage": {
                "allOf": [
                  {
                    "$ref": "#/definitions/ReplyUsageV1"
                  }
                ],
                "default": {}
              }
            },
            "required": [
              "calls",
              "index",
              "text",
              "thinking"
            ],
            "type": "object"
          },
          "ReproducePreviewV1": {
            "additionalProperties": false,
            "description": "What a dry run reports before anything is spent.",
            "properties": {
              "fidelity": {
                "$ref": "#/definitions/FidelityV1"
              },
              "model": {
                "type": "string"
              },
              "original": {
                "$ref": "#/definitions/ReplyV1"
              },
              "provider": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "samples": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "step_cost_usd": {
                "description": "What the original step cost; null when the provider reported nothing.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              }
            },
            "required": [
              "fidelity",
              "model",
              "original",
              "samples"
            ],
            "type": "object"
          },
          "ReproductionChangeKindV1": {
            "oneOf": [
              {
                "description": "The context as the model saw it.",
                "enum": [
                  "none"
                ],
                "type": "string"
              },
              {
                "description": "The suggestion's own `check.change`.",
                "enum": [
                  "proposed"
                ],
                "type": "string"
              },
              {
                "description": "Edits a person wrote.",
                "enum": [
                  "custom"
                ],
                "type": "string"
              }
            ]
          },
          "ReproductionStateV1": {
            "enum": [
              "running",
              "completed",
              "failed"
            ],
            "type": "string"
          },
          "ReproductionV1": {
            "additionalProperties": false,
            "description": "A replay of the decision point: N replies sampled from the rebuilt request, with or without a change. No function ever runs.",
            "properties": {
              "by": {
                "type": "string"
              },
              "change": {
                "default": [],
                "items": {
                  "$ref": "#/definitions/ChangeEditV1"
                },
                "type": "array"
              },
              "change_kind": {
                "$ref": "#/definitions/ReproductionChangeKindV1"
              },
              "check": {
                "allOf": [
                  {
                    "$ref": "#/definitions/SuggestionCheckV1"
                  }
                ],
                "description": "The decision point and signal this replay used."
              },
              "cost_unknown_samples": {
                "default": 0,
                "description": "Samples without a reported cost, failed ones included (the provider may have billed a request that never answered): unknown, not in `cost_usd`, and never estimated.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "cost_usd": {
                "description": "Known cost of the samples; null while no reply reported one. Only a lower bound while `cost_unknown_samples` is above zero. Jev's calls are in tokens below, never priced.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "error": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "fidelity": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/FidelityV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "finished_at": {
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "id": {
                "type": "string"
              },
              "judge_input_tokens": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "judge_output_tokens": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "model": {
                "type": "string"
              },
              "original": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ReplyV1"
                  },
                  {
                    "type": "null"
                  }
                ],
                "description": "The original reply at the decision point, read the same way."
              },
              "phase": {
                "description": "`sampling` or `classifying` while running.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "provider": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "requested": {
                "description": "Replies asked for, extensions included.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "samples": {
                "default": [],
                "items": {
                  "$ref": "#/definitions/ReplyV1"
                },
                "type": "array"
              },
              "started_at": {
                "format": "int64",
                "type": "integer"
              },
              "state": {
                "$ref": "#/definitions/ReproductionStateV1"
              },
              "updated_at": {
                "format": "int64",
                "type": "integer"
              }
            },
            "required": [
              "by",
              "change_kind",
              "check",
              "id",
              "model",
              "requested",
              "started_at",
              "state",
              "updated_at"
            ],
            "type": "object"
          },
          "SignalRuleV1": {
            "oneOf": [
              {
                "description": "The reply asks `engine::functions::info` only for contracts an earlier result in the context already supplied.",
                "enum": [
                  "contract_rediscovery"
                ],
                "type": "string"
              },
              {
                "description": "The reply repeats the last call that failed: same target, equal payload.",
                "enum": [
                  "repeated_error_call"
                ],
                "type": "string"
              }
            ]
          },
          "SignalV1": {
            "additionalProperties": false,
            "description": "How to recognize the behavior in one reply: exactly one of a rule computed in code or a yes/no question Jev answers about the reply.",
            "properties": {
              "question": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "rule": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/SignalRuleV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "type": "object"
          },
          "SuggestionCheckV1": {
            "additionalProperties": false,
            "description": "How a suggestion is checked at the step where its behavior happened.",
            "properties": {
              "change": {
                "default": [],
                "description": "The proposed change as edits of what the model saw before that step; empty when it cannot be expressed as text.",
                "items": {
                  "$ref": "#/definitions/ChangeEditV1"
                },
                "type": "array"
              },
              "decision_point": {
                "description": "The assistant entry where the behavior happened (`…_<step>_assistant`), one of the suggestion's `evidence` entries.",
                "type": "string"
              },
              "signal": {
                "$ref": "#/definitions/SignalV1"
              }
            },
            "required": [
              "decision_point",
              "signal"
            ],
            "type": "object"
          },
          "SuggestionReviewV1": {
            "additionalProperties": false,
            "description": "Everything people decided about one suggestion, in the `eval_suggestion` state scope. The title, component and patterns are copied from the suggestion so the row outlives the analysis's retention.",
            "properties": {
              "criterion": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/CriterionV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "evaluation_id": {
                "type": "string"
              },
              "evidence": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/EvidenceV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "first_run_at": {
                "description": "When the first execution this worker started for the suggestion began, kept when a new start replaces `run`: results of it may have been seen.",
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "harness_component": {
                "type": "string"
              },
              "lifecycle": {
                "$ref": "#/definitions/LifecycleV1"
              },
              "patterns": {
                "description": "`<rule_id>:<target>` of the diagnostics the suggestion's evidence cites: what a validation counts and a release is judged by.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "reproductions": {
                "description": "Replays of the decision point, oldest first: the base and the tested changes.",
                "items": {
                  "$ref": "#/definitions/ReproductionV1"
                },
                "type": "array"
              },
              "run": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ValidationRunV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "scenario_id": {
                "description": "The scenario the suggestion's validation plan names.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "suggestion_index": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "title": {
                "type": "string"
              },
              "verdict": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/VerdictV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "required": [
              "evaluation_id",
              "harness_component",
              "lifecycle",
              "patterns",
              "suggestion_index",
              "title"
            ],
            "type": "object"
          },
          "ValidationOutcomeV1": {
            "oneOf": [
              {
                "description": "The candidate moved the metric by at least `min_effect` in the wanted direction, with `min_runs` completed runs each side.",
                "enum": [
                  "validated_improvement"
                ],
                "type": "string"
              },
              {
                "description": "The metric moved less than `min_effect`.",
                "enum": [
                  "no_improvement"
                ],
                "type": "string"
              },
              {
                "description": "The metric moved by at least `min_effect` the wrong way.",
                "enum": [
                  "regression"
                ],
                "type": "string"
              },
              {
                "description": "Too few completed runs, missing data or no criterion.",
                "enum": [
                  "inconclusive"
                ],
                "type": "string"
              }
            ]
          },
          "ValidationRunStateV1": {
            "oneOf": [
              {
                "enum": [
                  "failed"
                ],
                "type": "string"
              },
              {
                "description": "Recorded before the executions are requested.",
                "enum": [
                  "starting"
                ],
                "type": "string"
              },
              {
                "description": "Both executions were requested; the sweep waits for them.",
                "enum": [
                  "running"
                ],
                "type": "string"
              },
              {
                "description": "Both ended; the pair is about to be attached.",
                "enum": [
                  "finished"
                ],
                "type": "string"
              },
              {
                "description": "The pair is attached to the analysis and its evidence computed.",
                "enum": [
                  "attached"
                ],
                "type": "string"
              }
            ]
          },
          "ValidationRunV1": {
            "additionalProperties": false,
            "description": "The baseline and candidate E2E executions this worker started for a suggestion (`eval::start-validation`).",
            "properties": {
              "baseline_commit": {
                "description": "The full commits the Harness was built from.",
                "type": "string"
              },
              "baseline_execution_id": {
                "description": "Absent until the E2E accepted the execution.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "candidate_commit": {
                "type": "string"
              },
              "candidate_execution_id": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "error": {
                "description": "Why it failed, or why attaching is being retried; starts with a stable code (`e2e_unavailable:`, `e2e_busy:`…).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "finished_at": {
                "description": "When both executions were seen ended; the sweep stops trying to attach the pair some time after.",
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "model": {
                "type": "string"
              },
              "provider": {
                "type": "string"
              },
              "runs": {
                "description": "Runs requested for each side.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "scenario_id": {
                "type": "string"
              },
              "started_at": {
                "format": "int64",
                "type": "integer"
              },
              "state": {
                "$ref": "#/definitions/ValidationRunStateV1"
              }
            },
            "required": [
              "baseline_commit",
              "candidate_commit",
              "model",
              "provider",
              "runs",
              "scenario_id",
              "started_at",
              "state"
            ],
            "type": "object"
          },
          "VerdictV1": {
            "additionalProperties": false,
            "description": "The person's judgment, kept with the criterion it was made against.",
            "properties": {
              "at": {
                "format": "int64",
                "type": "integer"
              },
              "by": {
                "type": "string"
              },
              "controls_checked": {
                "description": "The non-regression controls of the plan that were checked.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "criterion_snapshot": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/CriterionV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "outcome": {
                "$ref": "#/definitions/ValidationOutcomeV1"
              },
              "rationale": {
                "type": "string"
              }
            },
            "required": [
              "at",
              "by",
              "controls_checked",
              "outcome",
              "rationale"
            ],
            "type": "object"
          }
        },
        "properties": {
          "preview": {
            "anyOf": [
              {
                "$ref": "#/definitions/ReproducePreviewV1"
              },
              {
                "type": "null"
              }
            ]
          },
          "reproduction_id": {
            "type": [
              "string",
              "null"
            ]
          },
          "review": {
            "anyOf": [
              {
                "$ref": "#/definitions/SuggestionReviewV1"
              },
              {
                "type": "null"
              }
            ]
          }
        },
        "title": "ReproduceResponseV1",
        "type": "object"
      }
    },
    {
      "description": "Read an analysis with its captured evidence, deterministic diagnostics, Jev triage, suggestions, the monitor's own consumption and linked E2E executions, or null. A completed analysis is not a validated improvement.",
      "metadata": {},
      "name": "eval::result",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "evaluation_id": {
            "type": "string"
          }
        },
        "required": [
          "evaluation_id"
        ],
        "title": "EvaluationIdRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "anyOf": [
          {
            "$ref": "#/definitions/EvalResultResponseV1"
          },
          {
            "type": "null"
          }
        ],
        "definitions": {
          "AnalysisAssetsV1": {
            "additionalProperties": false,
            "properties": {
              "capture": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/TurnCaptureV1"
                  },
                  {
                    "type": "null"
                  }
                ],
                "description": "The observed turn's record, copied when the evidence was captured: what a reproduction rebuilds the request from."
              },
              "evaluation_id": {
                "type": "string"
              },
              "investigation": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/InvestigationV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "snapshot": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/SnapshotV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "triage": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/TriageV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "triage_failure": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/TriageFailureV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "validations": {
                "default": [],
                "items": {
                  "$ref": "#/definitions/ValidationLinkV1"
                },
                "type": "array"
              }
            },
            "required": [
              "evaluation_id"
            ],
            "type": "object"
          },
          "AnalysisCountersV1": {
            "additionalProperties": false,
            "properties": {
              "diagnostics": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "entries": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "rejected_suggestions": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "sessions": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "suggestions": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "validations": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "diagnostics",
              "entries",
              "rejected_suggestions",
              "sessions",
              "suggestions",
              "validations"
            ],
            "type": "object"
          },
          "AnalysisOriginV1": {
            "enum": [
              "automatic",
              "manual"
            ],
            "type": "string"
          },
          "AnalysisRecordV1": {
            "additionalProperties": false,
            "description": "The compact analysis record listed by `eval::list`; evidence and model outputs live separately in the analysis assets.",
            "properties": {
              "analyst": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/AnalystRefV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "code_root": {
                "description": "Frozen at admission with the model: the directory the investigation reads and runs in, absent when code access was off.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "completed_at": {
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "config_revision": {
                "type": "string"
              },
              "counters": {
                "$ref": "#/definitions/AnalysisCountersV1"
              },
              "coverage": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/CoverageLevelV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "created_at": {
                "format": "int64",
                "type": "integer"
              },
              "criteria_version": {
                "type": "string"
              },
              "deadline": {
                "format": "int64",
                "type": "integer"
              },
              "evaluation_id": {
                "type": "string"
              },
              "failure": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/FailureV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "harness_version": {
                "description": "The Harness version the engine reported when a live event admitted the analysis (the turn just ended on it); absent on a manual analysis, which may be of an older session, and when the engine did not say.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "judge_call": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/JudgeCallV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "model": {
                "allOf": [
                  {
                    "$ref": "#/definitions/MonitorModelV1"
                  }
                ],
                "description": "Frozen at admission: later configuration changes never alter it."
              },
              "observation_key": {
                "type": "string"
              },
              "observe_since": {
                "description": "Earlier root turns join the window only if they started after this instant: the monitor was observing then. Turns from before the monitor was enabled are never reported as new.",
                "format": "int64",
                "type": "integer"
              },
              "origin": {
                "$ref": "#/definitions/AnalysisOriginV1"
              },
              "pending_reason": {
                "description": "Why collection is still waiting (descendants running, metrics incomplete).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "routing": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/RoutingV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "rules_version": {
                "type": "string"
              },
              "schema_version": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "session_id": {
                "type": "string"
              },
              "signals": {
                "additionalProperties": {
                  "format": "uint32",
                  "minimum": 0,
                  "type": "integer"
                },
                "description": "Occurrences of each deterministic pattern the collection found, keyed `<rule_id>:<target>` (a target may itself contain colons). Empty both when nothing was found and before collection (`coverage` is set once collection read the evidence).",
                "type": "object"
              },
              "source_title": {
                "description": "The session's title, once collection has read it.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "stages": {
                "description": "When each status began, oldest first.",
                "items": {
                  "$ref": "#/definitions/StageTimeV1"
                },
                "type": "array"
              },
              "status": {
                "$ref": "#/definitions/EvalStatusV1"
              },
              "step": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "supersedes": {
                "description": "The earlier analysis of the same turn this reanalysis was requested over (same `observation_key`); absent on a first analysis.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "turn_id": {
                "type": "string"
              },
              "updated_at": {
                "format": "int64",
                "type": "integer"
              },
              "usage": {
                "allOf": [
                  {
                    "$ref": "#/definitions/MonitorUsageV1"
                  }
                ],
                "description": "The monitor's own consumption, apart from the observed task."
              }
            },
            "required": [
              "config_revision",
              "counters",
              "created_at",
              "criteria_version",
              "deadline",
              "evaluation_id",
              "model",
              "observation_key",
              "observe_since",
              "origin",
              "rules_version",
              "schema_version",
              "session_id",
              "stages",
              "status",
              "step",
              "turn_id",
              "updated_at",
              "usage"
            ],
            "type": "object"
          },
          "AnalystRefV1": {
            "additionalProperties": false,
            "properties": {
              "sent_at": {
                "format": "int64",
                "type": "integer"
              },
              "session_id": {
                "type": "string"
              },
              "turn_id": {
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "sent_at",
              "session_id"
            ],
            "type": "object"
          },
          "Answer": {
            "oneOf": [
              {
                "additionalProperties": false,
                "properties": {
                  "noul": {
                    "format": "double",
                    "type": "number"
                  },
                  "type": {
                    "enum": [
                      "noul"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "noul",
                  "type"
                ],
                "type": "object"
              },
              {
                "additionalProperties": false,
                "properties": {
                  "choice": {
                    "type": "string"
                  },
                  "confidence": {
                    "format": "double",
                    "type": "number"
                  },
                  "probabilities": {
                    "additionalProperties": {
                      "format": "double",
                      "type": "number"
                    },
                    "type": "object"
                  },
                  "type": {
                    "enum": [
                      "choice"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "choice",
                  "confidence",
                  "probabilities",
                  "type"
                ],
                "type": "object"
              },
              {
                "additionalProperties": false,
                "properties": {
                  "confidence": {
                    "format": "double",
                    "type": "number"
                  },
                  "legend": {
                    "additionalProperties": {
                      "$ref": "#/definitions/ScoreLevel"
                    },
                    "type": "object"
                  },
                  "probabilities": {
                    "additionalProperties": {
                      "format": "double",
                      "type": "number"
                    },
                    "type": "object"
                  },
                  "score": {
                    "format": "double",
                    "type": "number"
                  },
                  "type": {
                    "enum": [
                      "score"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "confidence",
                  "legend",
                  "probabilities",
                  "score",
                  "type"
                ],
                "type": "object"
              }
            ]
          },
          "ChangeEditV1": {
            "additionalProperties": false,
            "description": "One edit of what the model saw. `target` is an entry id of the conversation before the decision point, or `system_prompt`. An edit either replaces `find` with `replace` in the target's text or removes the entry.",
            "properties": {
              "find": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "remove": {
                "type": "boolean"
              },
              "replace": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "target": {
                "type": "string"
              }
            },
            "required": [
              "target"
            ],
            "type": "object"
          },
          "CodeRefV1": {
            "additionalProperties": false,
            "description": "Lines of one file of the code directory.",
            "properties": {
              "line_from": {
                "description": "First line, 1-based and inclusive.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "line_to": {
                "description": "Last line, inclusive; at most the file's line count.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "path": {
                "description": "Relative to the code directory, e.g. `harness/src/registry.rs` or `context-manager/src/prune.rs`; never absolute and never with `..`.",
                "type": "string"
              }
            },
            "required": [
              "line_from",
              "line_to",
              "path"
            ],
            "type": "object"
          },
          "ComparabilityCheckV1": {
            "additionalProperties": false,
            "properties": {
              "baseline": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "candidate": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "field": {
                "type": "string"
              },
              "matches": {
                "type": "boolean"
              }
            },
            "required": [
              "field",
              "matches"
            ],
            "type": "object"
          },
          "ComparabilityV1": {
            "additionalProperties": false,
            "description": "The conditions that must match for a fair comparison; only the Harness version is meant to differ.",
            "properties": {
              "checks": {
                "items": {
                  "$ref": "#/definitions/ComparabilityCheckV1"
                },
                "type": "array"
              },
              "comparable": {
                "type": "boolean"
              }
            },
            "required": [
              "checks",
              "comparable"
            ],
            "type": "object"
          },
          "ContextSnapshotV1": {
            "description": "One generation's context accounting. `free = usable - total`, floored at zero: once provider usage lands, `total` is what was billed, which can exceed the `usable` budget the window was fit into — that budget was derived before the generation from an estimate.",
            "properties": {
              "categories": {
                "$ref": "#/definitions/SnapshotCategoriesV1"
              },
              "compacted": {
                "description": "The context carries a conversation summary: compacted on this step, or anchored on an earlier compaction entry (a previous step, the console's `/compact`). The summary's tokens count in `system_prompt`.",
                "type": "boolean"
              },
              "effective_max_output_tokens": {
                "description": "Output allocation `usable` was derived against.",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "estimator": {
                "description": "Which estimator produced the numbers (`heuristic` until the context-manager resolves a real tokenizer). Absent when the context-manager predates the breakdown response.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "free": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "model": {
                "type": "string"
              },
              "prefix_divergences": {
                "description": "Already-sent rows this session's requests rewrote, newest last (at most [`MAX_PREFIX_DIVERGENCES`]), carried from step to step.",
                "items": {
                  "$ref": "#/definitions/PrefixDivergenceV1"
                },
                "type": "array"
              },
              "prompt_sections_fallback": {
                "description": "Why no shared prefix went out: `disabled` (config), `no_stable_prefix` (empty identity prompt) or `prefix_rewritten` (a pre_generate hook replaced the prompt head; its prompt still wins, it just shares nothing).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "prompt_surface_digest": {
                "description": "Identity of the shared prompt prefix this generation went out with (`sha256:` of the stable section) — present only when the request carried `system_sections`. A local identity, never a provider hit; `usage.cache_read` is the reuse signal.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "provider": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "session_cost_usd": {
                "description": "Running cost of the whole session in USD, accumulated across every generation step. `usage.cost_usd` is one step's bill — on providers with steep cache discounts the per-step number swings two orders of magnitude, so a chip showing it alone reads as a bouncing total.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "session_id": {
                "type": "string"
              },
              "step": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "summarized_head_tokens": {
                "description": "Size of the history that summary replaced.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "timestamp": {
                "format": "int64",
                "type": "integer"
              },
              "total": {
                "description": "Final request estimate: categories plus post-assembly growth.",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "turn_id": {
                "type": "string"
              },
              "usable": {
                "description": "The input budget the window was fit into.",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "usage": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/Usage"
                  },
                  {
                    "type": "null"
                  }
                ],
                "description": "Actual provider usage for this generation, stamped after the terminal frame; absent when the provider returned none (or the generation never completed)."
              }
            },
            "required": [
              "categories",
              "compacted",
              "effective_max_output_tokens",
              "free",
              "model",
              "session_id",
              "step",
              "timestamp",
              "total",
              "turn_id",
              "usable"
            ],
            "type": "object"
          },
          "CorrelationV1": {
            "oneOf": [
              {
                "enum": [
                  "unknown"
                ],
                "type": "string"
              },
              {
                "description": "A `registry-changed` notice reached the model between the sources and the repeated call. Temporal correlation, not a proven cause.",
                "enum": [
                  "harness_notice_correlated"
                ],
                "type": "string"
              }
            ]
          },
          "CoverageLevelV1": {
            "oneOf": [
              {
                "description": "Every in-scope transcript was read and the model context holds every diagnostic's evidence.",
                "enum": [
                  "complete"
                ],
                "type": "string"
              },
              {
                "description": "Previews omit or shorten entries; the full transcripts were read.",
                "enum": [
                  "partial"
                ],
                "type": "string"
              },
              {
                "description": "Required evidence did not fit; no conclusion of healthy behavior.",
                "enum": [
                  "insufficient"
                ],
                "type": "string"
              }
            ]
          },
          "CoverageV1": {
            "additionalProperties": false,
            "properties": {
              "context_bytes": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "diagnostics_in_context": {
                "description": "Diagnostics shown to the models; all of them stay in the snapshot.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "entries_read": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "level": {
                "$ref": "#/definitions/CoverageLevelV1"
              },
              "limitations": {
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "sessions_in_scope": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "sessions_out_of_scope": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "context_bytes",
              "diagnostics_in_context",
              "entries_read",
              "level",
              "limitations",
              "sessions_in_scope",
              "sessions_out_of_scope"
            ],
            "type": "object"
          },
          "CriterionMetricV1": {
            "description": "What a validation measures. Every metric but `signal_per_run` is the mean over the runs of the scenario the infrastructure did not break (`pass_rate` is the share that passed, an unfinished run counting as not passed; `duration` is the wall time in milliseconds).",
            "oneOf": [
              {
                "enum": [
                  "pass_rate",
                  "cost_usd",
                  "duration",
                  "tokens",
                  "function_calls"
                ],
                "type": "string"
              },
              {
                "description": "Occurrences of one of the suggestion's patterns per run, counted by the deterministic detectors over each run's transcript.",
                "enum": [
                  "signal_per_run"
                ],
                "type": "string"
              }
            ]
          },
          "CriterionV1": {
            "additionalProperties": false,
            "properties": {
              "direction": {
                "$ref": "#/definitions/DirectionV1"
              },
              "metric": {
                "$ref": "#/definitions/CriterionMetricV1"
              },
              "min_effect": {
                "format": "double",
                "type": "number"
              },
              "min_runs": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "pattern": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "registered_at": {
                "format": "int64",
                "type": "integer"
              },
              "registered_by": {
                "type": "string"
              },
              "scenario_id": {
                "description": "The scenario whose runs are measured.",
                "type": "string"
              }
            },
            "required": [
              "direction",
              "metric",
              "min_effect",
              "min_runs",
              "registered_at",
              "registered_by",
              "scenario_id"
            ],
            "type": "object"
          },
          "DiagnosticV1": {
            "additionalProperties": false,
            "description": "One deterministic observation. Model judgments never remove it.",
            "properties": {
              "correlation": {
                "$ref": "#/definitions/CorrelationV1"
              },
              "evidence": {
                "description": "Calls, results and notices, in transcript order.",
                "items": {
                  "$ref": "#/definitions/EntryRefV1"
                },
                "type": "array"
              },
              "fingerprint": {
                "description": "Rule, session, turn and the call ids of this occurrence.",
                "type": "string"
              },
              "observation": {
                "type": "string"
              },
              "rule_id": {
                "type": "string"
              },
              "rule_version": {
                "type": "string"
              },
              "session_id": {
                "type": "string"
              },
              "target": {
                "type": "string"
              },
              "turn_id": {
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "correlation",
              "evidence",
              "fingerprint",
              "observation",
              "rule_id",
              "rule_version",
              "session_id",
              "target"
            ],
            "type": "object"
          },
          "DirectionV1": {
            "enum": [
              "decrease",
              "increase"
            ],
            "type": "string"
          },
          "E2eAssessmentsV1": {
            "additionalProperties": false,
            "properties": {
              "passed": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "total": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "passed",
              "total"
            ],
            "type": "object"
          },
          "E2eExecutionRefV1": {
            "additionalProperties": false,
            "description": "What `e2e::dashboard::execution-get` reported at lookup time.",
            "properties": {
              "assessments": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/E2eAssessmentsV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "availability": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "conclusion": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "e2e_revision": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "engine_version": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "evidence_error": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "execution_id": {
                "type": "string"
              },
              "harness_version": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "label": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "model": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "provider": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "reports": {
                "items": {
                  "$ref": "#/definitions/E2eReportRefV1"
                },
                "type": "array"
              },
              "reports_available": {
                "description": "True only when the execution reported retained reports.",
                "type": "boolean"
              },
              "scenarios": {
                "items": {
                  "$ref": "#/definitions/E2eScenarioV1"
                },
                "type": "array"
              },
              "started_at": {
                "description": "When the E2E started the execution (ms since the Unix epoch): its results cannot have existed before.",
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "status": {
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "execution_id",
              "reports",
              "reports_available",
              "scenarios"
            ],
            "type": "object"
          },
          "E2eMeasureV1": {
            "additionalProperties": false,
            "properties": {
              "average": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "samples": {
                "description": "Runs that reported this measure; fewer than `run_count` means some runs reported nothing (never counted as zero).",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "samples"
            ],
            "type": "object"
          },
          "E2eReportRefV1": {
            "additionalProperties": false,
            "properties": {
              "available": {
                "type": "boolean"
              },
              "scenario_id": {
                "type": "string"
              },
              "subject_id": {
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "available",
              "scenario_id"
            ],
            "type": "object"
          },
          "E2eScenarioV1": {
            "additionalProperties": false,
            "description": "One scenario's identity and the execution's own averages, as the E2E computed them.",
            "properties": {
              "behavior_sha256": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "completed_runs": {
                "format": "uint32",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "contract_fingerprint": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "cost_usd_per_run": {
                "description": "The cohort's total cost divided by its completed runs; the total also covers failed attempts.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "mean_score": {
                "description": "Mean score of the scored runs (0 to 100).",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "measures": {
                "additionalProperties": {
                  "$ref": "#/definitions/E2eMeasureV1"
                },
                "description": "`function_calls`, `function_call_errors`, `tokens`, `duration_seconds` and `cost_usd`, when reported.",
                "type": "object"
              },
              "median_wall_time_ms": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "p50_function_calls": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "pass_rate": {
                "description": "The scenario's cohort in `plan_execution.measurements.cohorts[]`, when the execution has exactly one for it. Each value below is absent when the E2E did not report it, never zero.\n\nShare of the planned runs that passed (0 to 1).",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "planned_runs": {
                "format": "uint32",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "run_count": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "scenario_id": {
                "type": "string"
              },
              "total_tokens_per_run": {
                "description": "The cohort's total tokens divided by its completed runs.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              }
            },
            "required": [
              "measures",
              "run_count",
              "scenario_id"
            ],
            "type": "object"
          },
          "EntryRefV1": {
            "additionalProperties": false,
            "description": "A transcript entry, addressed the way `session::messages` returns it.",
            "properties": {
              "entry_id": {
                "type": "string"
              },
              "session_id": {
                "type": "string"
              }
            },
            "required": [
              "entry_id",
              "session_id"
            ],
            "type": "object"
          },
          "EvalResultResponseV1": {
            "additionalProperties": false,
            "properties": {
              "assets": {
                "$ref": "#/definitions/AnalysisAssetsV1"
              },
              "record": {
                "$ref": "#/definitions/AnalysisRecordV1"
              },
              "reviews": {
                "default": [],
                "description": "One row per suggestion, in suggestion order: what people decided about it. A suggestion nobody acted on reads as `new`.",
                "items": {
                  "$ref": "#/definitions/SuggestionReviewV1"
                },
                "type": "array"
              }
            },
            "required": [
              "assets",
              "record"
            ],
            "type": "object"
          },
          "EvalStatusV1": {
            "enum": [
              "queued",
              "collecting",
              "judging",
              "investigating",
              "completed",
              "failed",
              "cancelled"
            ],
            "type": "string"
          },
          "EvidenceRunV1": {
            "additionalProperties": false,
            "description": "One run of the scenario as the E2E reported it. Runs the infrastructure broke are listed with the E2E's own words, never counted.",
            "properties": {
              "completed": {
                "description": "The E2E reported the run complete with a valid technical outcome.",
                "type": "boolean"
              },
              "completion": {
                "description": "`completion` as the E2E reported it.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "function_calls": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "run_id": {
                "type": "string"
              },
              "signals": {
                "additionalProperties": {
                  "format": "uint32",
                  "minimum": 0,
                  "type": "integer"
                },
                "description": "Occurrences of each of the row's patterns the detectors found in this run's transcript (0 is a measurement); absent when the report held no transcript.",
                "type": [
                  "object",
                  "null"
                ]
              },
              "status": {
                "description": "`passed` or `failed`, the run's task outcome.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "technical": {
                "description": "`technical` as the E2E reported it; anything but `valid` is an infrastructure failure.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "total_tokens": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "wall_time_ms": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              }
            },
            "required": [
              "completed",
              "run_id"
            ],
            "type": "object"
          },
          "EvidenceSideV1": {
            "additionalProperties": false,
            "properties": {
              "execution_id": {
                "type": "string"
              },
              "mean": {
                "description": "Mean of the criterion's metric over those runs; absent without a criterion or without such runs.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "n": {
                "description": "Measurable runs (see `EvidenceRunV1::measurable`) with a value of the criterion's metric (every measurable run while there is no criterion).",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "runs": {
                "description": "Every run of the scenario the execution holds.",
                "items": {
                  "$ref": "#/definitions/EvidenceRunV1"
                },
                "type": "array"
              }
            },
            "required": [
              "execution_id",
              "n",
              "runs"
            ],
            "type": "object"
          },
          "EvidenceV1": {
            "additionalProperties": false,
            "description": "Computed in code from the attached pair's runs, never by a model. It is what the person's verdict is shown next to.",
            "properties": {
              "baseline": {
                "$ref": "#/definitions/EvidenceSideV1"
              },
              "candidate": {
                "$ref": "#/definitions/EvidenceSideV1"
              },
              "computed_at": {
                "format": "int64",
                "type": "integer"
              },
              "computed_outcome": {
                "$ref": "#/definitions/ValidationOutcomeV1"
              },
              "reason": {
                "type": "string"
              },
              "scenario_id": {
                "type": "string"
              }
            },
            "required": [
              "baseline",
              "candidate",
              "computed_at",
              "computed_outcome",
              "reason",
              "scenario_id"
            ],
            "type": "object"
          },
          "ExcludedProbeV1": {
            "additionalProperties": false,
            "properties": {
              "calls": {
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "code": {
                "type": "string"
              },
              "session_id": {
                "type": "string"
              },
              "target": {
                "type": "string"
              },
              "turn_id": {
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "calls",
              "code",
              "session_id",
              "target"
            ],
            "type": "object"
          },
          "FailureV1": {
            "additionalProperties": false,
            "description": "The stage that failed and why. `stage` is the status the analysis was in.",
            "properties": {
              "code": {
                "type": "string"
              },
              "message": {
                "type": "string"
              },
              "stage": {
                "$ref": "#/definitions/EvalStatusV1"
              }
            },
            "required": [
              "code",
              "message",
              "stage"
            ],
            "type": "object"
          },
          "FidelityLevelV1": {
            "oneOf": [
              {
                "enum": [
                  "approximate"
                ],
                "type": "string"
              },
              {
                "description": "The provider counts the rebuilt request like the original, up to a fixed per-request overhead measured at the turn's first step.",
                "enum": [
                  "exact"
                ],
                "type": "string"
              }
            ]
          },
          "FidelityV1": {
            "additionalProperties": false,
            "description": "How faithfully the request at the decision point was rebuilt.",
            "properties": {
              "counted_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "estimator": {
                "description": "`provider` when the provider counted, else the estimator's name.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "level": {
                "$ref": "#/definitions/FidelityLevelV1"
              },
              "off_ratio": {
                "description": "The unexplained difference over the recorded tokens.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "reasons": {
                "default": [],
                "description": "Why the rebuild is approximate; empty when exact.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "recorded_tokens": {
                "description": "Input tokens of the original step: input + cache read + cache write.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "required": [
              "level"
            ],
            "type": "object"
          },
          "InvestigationV1": {
            "additionalProperties": false,
            "description": "The investigation and its own consumption, kept apart from the observed task's metrics.",
            "properties": {
              "code_root": {
                "description": "The directory the analyst read and ran in; absent when access was off.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "completed_at": {
                "format": "int64",
                "type": "integer"
              },
              "effective_model": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "effective_provider": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "metrics": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/SessionMetricsResponseV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "rejected": {
                "items": {
                  "$ref": "#/definitions/RejectedSuggestionV1"
                },
                "type": "array"
              },
              "requested_model": {
                "type": "string"
              },
              "requested_provider": {
                "type": "string"
              },
              "session_id": {
                "type": "string"
              },
              "signal_assessments": {
                "items": {
                  "$ref": "#/definitions/SignalAssessmentV1"
                },
                "type": "array"
              },
              "suggestions": {
                "items": {
                  "$ref": "#/definitions/SuggestionV1"
                },
                "type": "array"
              },
              "turn_id": {
                "type": "string"
              }
            },
            "required": [
              "completed_at",
              "rejected",
              "requested_model",
              "requested_provider",
              "session_id",
              "signal_assessments",
              "suggestions",
              "turn_id"
            ],
            "type": "object"
          },
          "JudgeCallV1": {
            "additionalProperties": false,
            "description": "A Jev call started by this analysis. Persisted before the call so a restart can tell a lost answer from one never requested.",
            "properties": {
              "deadline": {
                "format": "int64",
                "type": "integer"
              },
              "request_id": {
                "type": "string"
              },
              "started_at": {
                "format": "int64",
                "type": "integer"
              }
            },
            "required": [
              "deadline",
              "request_id",
              "started_at"
            ],
            "type": "object"
          },
          "LifecycleEventV1": {
            "additionalProperties": false,
            "description": "One change of status, with who made it.",
            "properties": {
              "at": {
                "format": "int64",
                "type": "integer"
              },
              "by": {
                "type": "string"
              },
              "note": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "status": {
                "$ref": "#/definitions/LifecycleStatusV1"
              }
            },
            "required": [
              "at",
              "by",
              "status"
            ],
            "type": "object"
          },
          "LifecycleStatusV1": {
            "description": "Where a suggestion stands. Only people move it: the monitor never does.",
            "enum": [
              "new",
              "accepted",
              "in_progress",
              "shipped",
              "rejected",
              "duplicate"
            ],
            "type": "string"
          },
          "LifecycleV1": {
            "additionalProperties": false,
            "properties": {
              "duplicate_of": {
                "description": "What it duplicates: a pull request, or another `<evaluation_id>:<index>` (`duplicate`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "history": {
                "description": "Every change of status, oldest first; empty while `new`.",
                "items": {
                  "$ref": "#/definitions/LifecycleEventV1"
                },
                "type": "array"
              },
              "pr": {
                "description": "The pull request that implements it (`in_progress`, `shipped`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "reason": {
                "description": "Why it was rejected (`rejected`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "status": {
                "$ref": "#/definitions/LifecycleStatusV1"
              },
              "version": {
                "description": "The Harness version that shipped it, a semantic version like `1.8.43`; the recurrence query splits the analyses at it.",
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "history",
              "status"
            ],
            "type": "object"
          },
          "MetricsScopeV1": {
            "oneOf": [
              {
                "description": "Cumulative over the root and every descendant, all turns.",
                "enum": [
                  "session_tree"
                ],
                "type": "string"
              }
            ]
          },
          "ModelUsageV1": {
            "additionalProperties": false,
            "description": "What the steps one model generated used. A conversation can switch models mid-way, and providers differ in what they report (one may send tokens and no cost, another nothing), so the session's counters alone cannot say which model a gap belongs to.",
            "properties": {
              "cache_read_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cache_write_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cost_usd": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "function_calls": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "input_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "model": {
                "type": "string"
              },
              "output_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "provider": {
                "type": "string"
              },
              "reasoning_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "turns": {
                "description": "Model steps (assistant messages) this model generated.",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "cache_read_tokens",
              "cache_write_tokens",
              "cost_usd",
              "function_calls",
              "input_tokens",
              "model",
              "output_tokens",
              "provider",
              "reasoning_tokens",
              "turns"
            ],
            "type": "object"
          },
          "MonitorModelV1": {
            "additionalProperties": false,
            "description": "The Harness model the user chose for investigations. Credentials stay in the provider worker; this request never carries a key.",
            "properties": {
              "model": {
                "type": "string"
              },
              "provider": {
                "type": "string"
              },
              "provider_options": {
                "additionalProperties": true,
                "description": "Provider-native options, namespaced by the selected provider id.",
                "type": [
                  "object",
                  "null"
                ]
              },
              "thinking_level": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ThinkingLevel"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "required": [
              "model",
              "provider"
            ],
            "type": "object"
          },
          "MonitorUsageV1": {
            "additionalProperties": false,
            "description": "Known usage only: absent counters are unknown, never zero.",
            "properties": {
              "judge_calls": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "judge_input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "judge_output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "judge_usage_complete": {
                "description": "False when Jev reported incomplete usage or failed.",
                "type": "boolean"
              },
              "llm_cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "llm_input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "llm_output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "required": [
              "judge_calls",
              "judge_input_tokens",
              "judge_output_tokens",
              "judge_usage_complete"
            ],
            "type": "object"
          },
          "PrefixDivergenceV1": {
            "description": "One generation that rewrote a row an earlier request of the session had already sent: a prefix-matching provider cache (DeepSeek) stops reusing at that row.",
            "properties": {
              "cache_read": {
                "description": "`usage.cache_read` of the rewriting request: what the provider still reused.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "call_id": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "cause": {
                "description": "What rewrote the row, as far as the harness can tell: `prune` (context-manager pruned or reduced function results to fit the budget that step) or `system_prompt` (row 0: a compaction summary or a prompt change).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "index": {
                "description": "First row (0 = system prompt, then one per message) that differs from the session's previous request. Absent when only the provider's wire mapping changed.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "prev_len": {
                "description": "Rows the previous request carried.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "provider_note": {
                "description": "The provider's own wire-level finding, cause included, when it reported one (a final-message warning starting with [`PROVIDER_PREFIX_NOTE`]).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "role": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "step": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "turn_id": {
                "type": "string"
              }
            },
            "required": [
              "step",
              "turn_id"
            ],
            "type": "object"
          },
          "RejectedSuggestionV1": {
            "additionalProperties": false,
            "properties": {
              "index": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "reasons": {
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "title": {
                "type": "string"
              }
            },
            "required": [
              "index",
              "reasons",
              "title"
            ],
            "type": "object"
          },
          "ReplyCallV1": {
            "additionalProperties": false,
            "description": "One function call of a reply.",
            "properties": {
              "description": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "payload": {
                "description": "The payload as JSON text, cut at 600 characters.",
                "type": "string"
              },
              "target": {
                "description": "The function called (through `agent_call` or directly).",
                "type": "string"
              }
            },
            "required": [
              "payload",
              "target"
            ],
            "type": "object"
          },
          "ReplyUsageV1": {
            "additionalProperties": false,
            "properties": {
              "cache_read_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cache_write_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          },
          "ReplyV1": {
            "additionalProperties": false,
            "description": "One reply at the decision point: a sample, or the original step read the same way.",
            "properties": {
              "calls": {
                "items": {
                  "$ref": "#/definitions/ReplyCallV1"
                },
                "type": "array"
              },
              "duration_ms": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "error": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "index": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "signal": {
                "description": "Whether the signal shows in this reply; null when Jev answered unclear, the reply failed or it was not classified yet.",
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "text": {
                "type": "string"
              },
              "thinking": {
                "description": "The opening of the reply's reasoning and of its text, 400 characters each.",
                "type": "string"
              },
              "usage": {
                "allOf": [
                  {
                    "$ref": "#/definitions/ReplyUsageV1"
                  }
                ],
                "default": {}
              }
            },
            "required": [
              "calls",
              "index",
              "text",
              "thinking"
            ],
            "type": "object"
          },
          "ReproductionChangeKindV1": {
            "oneOf": [
              {
                "description": "The context as the model saw it.",
                "enum": [
                  "none"
                ],
                "type": "string"
              },
              {
                "description": "The suggestion's own `check.change`.",
                "enum": [
                  "proposed"
                ],
                "type": "string"
              },
              {
                "description": "Edits a person wrote.",
                "enum": [
                  "custom"
                ],
                "type": "string"
              }
            ]
          },
          "ReproductionStateV1": {
            "enum": [
              "running",
              "completed",
              "failed"
            ],
            "type": "string"
          },
          "ReproductionV1": {
            "additionalProperties": false,
            "description": "A replay of the decision point: N replies sampled from the rebuilt request, with or without a change. No function ever runs.",
            "properties": {
              "by": {
                "type": "string"
              },
              "change": {
                "default": [],
                "items": {
                  "$ref": "#/definitions/ChangeEditV1"
                },
                "type": "array"
              },
              "change_kind": {
                "$ref": "#/definitions/ReproductionChangeKindV1"
              },
              "check": {
                "allOf": [
                  {
                    "$ref": "#/definitions/SuggestionCheckV1"
                  }
                ],
                "description": "The decision point and signal this replay used."
              },
              "cost_unknown_samples": {
                "default": 0,
                "description": "Samples without a reported cost, failed ones included (the provider may have billed a request that never answered): unknown, not in `cost_usd`, and never estimated.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "cost_usd": {
                "description": "Known cost of the samples; null while no reply reported one. Only a lower bound while `cost_unknown_samples` is above zero. Jev's calls are in tokens below, never priced.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "error": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "fidelity": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/FidelityV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "finished_at": {
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "id": {
                "type": "string"
              },
              "judge_input_tokens": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "judge_output_tokens": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "model": {
                "type": "string"
              },
              "original": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ReplyV1"
                  },
                  {
                    "type": "null"
                  }
                ],
                "description": "The original reply at the decision point, read the same way."
              },
              "phase": {
                "description": "`sampling` or `classifying` while running.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "provider": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "requested": {
                "description": "Replies asked for, extensions included.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "samples": {
                "default": [],
                "items": {
                  "$ref": "#/definitions/ReplyV1"
                },
                "type": "array"
              },
              "started_at": {
                "format": "int64",
                "type": "integer"
              },
              "state": {
                "$ref": "#/definitions/ReproductionStateV1"
              },
              "updated_at": {
                "format": "int64",
                "type": "integer"
              }
            },
            "required": [
              "by",
              "change_kind",
              "check",
              "id",
              "model",
              "requested",
              "started_at",
              "state",
              "updated_at"
            ],
            "type": "object"
          },
          "RoutingReasonV1": {
            "oneOf": [
              {
                "enum": [
                  "needs_investigation"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced; kept so stored records still deserialize.",
                "enum": [
                  "diagnostics"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "insufficient_evidence"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "low_confidence"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "coverage_insufficient"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "audit_sample"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "manual_request"
                ],
                "type": "string"
              }
            ]
          },
          "RoutingV1": {
            "additionalProperties": false,
            "properties": {
              "investigate": {
                "type": "boolean"
              },
              "reasons": {
                "items": {
                  "$ref": "#/definitions/RoutingReasonV1"
                },
                "type": "array"
              }
            },
            "required": [
              "investigate",
              "reasons"
            ],
            "type": "object"
          },
          "ScoreLevel": {
            "anyOf": [
              {
                "type": "string"
              },
              {
                "additionalProperties": true,
                "type": "object"
              },
              {
                "items": true,
                "type": "array"
              }
            ],
            "description": "Score levels require a description. Unlike instructions, Choice option descriptions and Noul criteria, the provider rejects a null level (HTTP422)."
          },
          "SessionEvidenceV1": {
            "additionalProperties": false,
            "properties": {
              "depth": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "entries": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "in_scope": {
                "description": "Out-of-scope descendants belong to earlier turns and are not read.",
                "type": "boolean"
              },
              "json_sha256": {
                "description": "SHA-256 of the collected JSON entries (not of the stored bytes).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "omitted_entries": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "parent_session_id": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "parent_turn_id": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "preview": {
                "description": "Masked, shortened entries shown to the models.",
                "items": true,
                "type": "array"
              },
              "reduced_entries": {
                "description": "Preview entries whose strings were truncated or whose secrets were masked.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "session_id": {
                "type": "string"
              }
            },
            "required": [
              "depth",
              "entries",
              "in_scope",
              "omitted_entries",
              "preview",
              "reduced_entries",
              "session_id"
            ],
            "type": "object"
          },
          "SessionMetricsResponseV1": {
            "additionalProperties": false,
            "properties": {
              "by_session": {
                "items": {
                  "$ref": "#/definitions/SessionUsageV1"
                },
                "type": "array"
              },
              "complete": {
                "type": "boolean"
              },
              "root_session_id": {
                "type": "string"
              },
              "totals": {
                "$ref": "#/definitions/SessionUsageTotalsV1"
              },
              "traces": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/SessionTraceMetricsV1"
                  },
                  {
                    "type": "null"
                  }
                ],
                "description": "Trace/span aggregates when the engine's in-memory observability exporter is available. Usage metrics remain available when it is not."
              }
            },
            "required": [
              "by_session",
              "complete",
              "root_session_id",
              "totals"
            ],
            "type": "object"
          },
          "SessionTraceMetricsV1": {
            "additionalProperties": false,
            "properties": {
              "by_session": {
                "items": {
                  "$ref": "#/definitions/SessionTraceUsageV1"
                },
                "type": "array"
              },
              "duration_ms": {
                "description": "Elapsed window from the first observed span to the last observed span.",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "error_span_count": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "span_count": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "trace_count": {
                "description": "Distinct traces across the root session and all descendants.",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "by_session",
              "duration_ms",
              "error_span_count",
              "span_count",
              "trace_count"
            ],
            "type": "object"
          },
          "SessionTraceUsageV1": {
            "additionalProperties": false,
            "properties": {
              "depth": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "duration_ms": {
                "description": "Elapsed window from the session's first observed span to its last.",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "error_span_count": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "parent_session_id": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "session_id": {
                "type": "string"
              },
              "span_count": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "trace_count": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "depth",
              "duration_ms",
              "error_span_count",
              "session_id",
              "span_count",
              "trace_count"
            ],
            "type": "object"
          },
          "SessionUsageTotalsV1": {
            "additionalProperties": false,
            "properties": {
              "by_model": {
                "description": "The same counters per model that generated a step, ordered by provider then model. Absent when nothing generated, and from a harness that predates it.",
                "items": {
                  "$ref": "#/definitions/ModelUsageV1"
                },
                "type": "array"
              },
              "cache_read_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cache_write_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cost_usd": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "function_call_errors": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "function_calls": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "input_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "output_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "reasoning_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "sessions": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "turns": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "cache_read_tokens",
              "cache_write_tokens",
              "cost_usd",
              "function_call_errors",
              "function_calls",
              "input_tokens",
              "output_tokens",
              "reasoning_tokens",
              "sessions",
              "turns"
            ],
            "type": "object"
          },
          "SessionUsageV1": {
            "additionalProperties": false,
            "properties": {
              "by_model": {
                "description": "The same counters per model that generated a step, ordered by provider then model. Absent when nothing generated, and from a harness that predates it.",
                "items": {
                  "$ref": "#/definitions/ModelUsageV1"
                },
                "type": "array"
              },
              "cache_read_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cache_write_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "context": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ContextSnapshotV1"
                  },
                  {
                    "type": "null"
                  }
                ],
                "description": "The session's latest per-generation context snapshot (categories, budget, usage) — absent for sessions that have not generated since snapshots landed."
              },
              "cost_usd": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "depth": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "function_call_errors": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "function_calls": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "input_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "output_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "parent_session_id": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "reasoning_tokens": {
                "description": "Summed over every generation that reported it. A single generation without the counter (a stream the provider cut before usage came back) is left out of the sum. `null` means no generation reported it or more than one did not — never a measured zero, so the key is always present and a reader can tell the two apart.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "session_id": {
                "type": "string"
              },
              "turns": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "cache_read_tokens",
              "cache_write_tokens",
              "cost_usd",
              "depth",
              "function_call_errors",
              "function_calls",
              "input_tokens",
              "output_tokens",
              "reasoning_tokens",
              "session_id",
              "turns"
            ],
            "type": "object"
          },
          "SignalAssessmentV1": {
            "additionalProperties": false,
            "properties": {
              "explanation": {
                "type": "string"
              },
              "fingerprint": {
                "description": "The `fingerprint` of a diagnostic in the evidence.",
                "type": "string"
              },
              "verdict": {
                "$ref": "#/definitions/SignalVerdictV1"
              }
            },
            "required": [
              "explanation",
              "fingerprint",
              "verdict"
            ],
            "type": "object"
          },
          "SignalRuleV1": {
            "oneOf": [
              {
                "description": "The reply asks `engine::functions::info` only for contracts an earlier result in the context already supplied.",
                "enum": [
                  "contract_rediscovery"
                ],
                "type": "string"
              },
              {
                "description": "The reply repeats the last call that failed: same target, equal payload.",
                "enum": [
                  "repeated_error_call"
                ],
                "type": "string"
              }
            ]
          },
          "SignalV1": {
            "additionalProperties": false,
            "description": "How to recognize the behavior in one reply: exactly one of a rule computed in code or a yes/no question Jev answers about the reply.",
            "properties": {
              "question": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "rule": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/SignalRuleV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "type": "object"
          },
          "SignalVerdictV1": {
            "enum": [
              "likely_expected",
              "worth_changing",
              "unclear"
            ],
            "type": "string"
          },
          "SnapshotCategoriesV1": {
            "description": "Where the request's tokens sit. Categories are assembly-time estimates; `hook_guidance` is the measured growth after assembly (pre-generate hook appends and orphan-repair patches), 0 when the request left assembly unchanged.",
            "properties": {
              "hook_guidance": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "messages": {
                "$ref": "#/definitions/SnapshotMessagesV1"
              },
              "overhead": {
                "description": "Provider framing plus response_format / provider_options fields.",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "skills": {
                "default": 0,
                "description": "Selected skill bodies contained in the system prompt.",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "system_prompt": {
                "description": "Final assembled system prompt excluding tokens attributed to `skills`: identity, per-step aids, and any compaction summary section.",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "tools": {
                "description": "Function schemas exposed to the model.",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "messages",
              "overhead",
              "system_prompt",
              "tools"
            ],
            "type": "object"
          },
          "SnapshotMessagesV1": {
            "description": "Estimated tokens of the assembled window's messages, by role.",
            "properties": {
              "assistant": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "custom": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "function_result": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "user": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "assistant",
              "custom",
              "function_result",
              "user"
            ],
            "type": "object"
          },
          "SnapshotV1": {
            "additionalProperties": false,
            "properties": {
              "captured_at": {
                "format": "int64",
                "type": "integer"
              },
              "coverage": {
                "$ref": "#/definitions/CoverageV1"
              },
              "diagnostics": {
                "items": {
                  "$ref": "#/definitions/DiagnosticV1"
                },
                "type": "array"
              },
              "e2e_scenario": {
                "description": "The harness-e2e scenario the observed session ran, when it came from the E2E (`metadata.e2e_scenario`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "excluded_probes": {
                "description": "Protocol probes that failed as expected and were not reported.",
                "items": {
                  "$ref": "#/definitions/ExcludedProbeV1"
                },
                "type": "array"
              },
              "metrics": {
                "$ref": "#/definitions/SessionMetricsResponseV1"
              },
              "metrics_scope": {
                "$ref": "#/definitions/MetricsScopeV1"
              },
              "observed_model": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "observed_provider": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "rules_version": {
                "type": "string"
              },
              "sessions": {
                "items": {
                  "$ref": "#/definitions/SessionEvidenceV1"
                },
                "type": "array"
              },
              "source_result_error": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "source_session_id": {
                "type": "string"
              },
              "source_status": {
                "$ref": "#/definitions/TurnStatus"
              },
              "source_stop_reason": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "source_title": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "source_turn_id": {
                "type": "string"
              },
              "window_turn_ids": {
                "description": "The observed turn plus earlier root turns no analysis covered yet.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              }
            },
            "required": [
              "captured_at",
              "coverage",
              "diagnostics",
              "excluded_probes",
              "metrics",
              "metrics_scope",
              "rules_version",
              "sessions",
              "source_session_id",
              "source_status",
              "source_turn_id",
              "window_turn_ids"
            ],
            "type": "object"
          },
          "StageTimeV1": {
            "additionalProperties": false,
            "properties": {
              "at": {
                "format": "int64",
                "type": "integer"
              },
              "status": {
                "$ref": "#/definitions/EvalStatusV1"
              }
            },
            "required": [
              "at",
              "status"
            ],
            "type": "object"
          },
          "Stats": {
            "additionalProperties": false,
            "description": "Known accepted usage, never a billing estimate. Unreported counters or failure/cancellation make usage_complete false.",
            "properties": {
              "attempts": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "elapsed_ms": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "questions": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "requests": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "usage_complete": {
                "type": "boolean"
              }
            },
            "required": [
              "attempts",
              "elapsed_ms",
              "input_tokens",
              "output_tokens",
              "questions",
              "requests",
              "usage_complete"
            ],
            "type": "object"
          },
          "SuggestionCheckV1": {
            "additionalProperties": false,
            "description": "How a suggestion is checked at the step where its behavior happened.",
            "properties": {
              "change": {
                "default": [],
                "description": "The proposed change as edits of what the model saw before that step; empty when it cannot be expressed as text.",
                "items": {
                  "$ref": "#/definitions/ChangeEditV1"
                },
                "type": "array"
              },
              "decision_point": {
                "description": "The assistant entry where the behavior happened (`…_<step>_assistant`), one of the suggestion's `evidence` entries.",
                "type": "string"
              },
              "signal": {
                "$ref": "#/definitions/SignalV1"
              }
            },
            "required": [
              "decision_point",
              "signal"
            ],
            "type": "object"
          },
          "SuggestionReviewV1": {
            "additionalProperties": false,
            "description": "Everything people decided about one suggestion, in the `eval_suggestion` state scope. The title, component and patterns are copied from the suggestion so the row outlives the analysis's retention.",
            "properties": {
              "criterion": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/CriterionV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "evaluation_id": {
                "type": "string"
              },
              "evidence": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/EvidenceV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "first_run_at": {
                "description": "When the first execution this worker started for the suggestion began, kept when a new start replaces `run`: results of it may have been seen.",
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "harness_component": {
                "type": "string"
              },
              "lifecycle": {
                "$ref": "#/definitions/LifecycleV1"
              },
              "patterns": {
                "description": "`<rule_id>:<target>` of the diagnostics the suggestion's evidence cites: what a validation counts and a release is judged by.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "reproductions": {
                "description": "Replays of the decision point, oldest first: the base and the tested changes.",
                "items": {
                  "$ref": "#/definitions/ReproductionV1"
                },
                "type": "array"
              },
              "run": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ValidationRunV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "scenario_id": {
                "description": "The scenario the suggestion's validation plan names.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "suggestion_index": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "title": {
                "type": "string"
              },
              "verdict": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/VerdictV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "required": [
              "evaluation_id",
              "harness_component",
              "lifecycle",
              "patterns",
              "suggestion_index",
              "title"
            ],
            "type": "object"
          },
          "SuggestionV1": {
            "additionalProperties": false,
            "properties": {
              "check": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/SuggestionCheckV1"
                  },
                  {
                    "type": "null"
                  }
                ],
                "description": "How to reproduce the behavior at the step where it happened, and the proposed change as edits of what the model saw there."
              },
              "code_refs": {
                "default": [],
                "description": "Code the suggestion rests on, as read in the code directory; only with code access. At most 8; each is checked against the directory.",
                "items": {
                  "$ref": "#/definitions/CodeRefV1"
                },
                "maxItems": 8,
                "type": "array"
              },
              "evidence": {
                "items": {
                  "$ref": "#/definitions/EntryRefV1"
                },
                "type": "array"
              },
              "expected_effect": {
                "type": "string"
              },
              "harness_component": {
                "type": "string"
              },
              "hypothesis": {
                "type": "string"
              },
              "limitations": {
                "type": "string"
              },
              "observation": {
                "type": "string"
              },
              "proposed_change": {
                "type": "string"
              },
              "title": {
                "type": "string"
              },
              "validation": {
                "$ref": "#/definitions/ValidationPlanV1"
              }
            },
            "required": [
              "evidence",
              "expected_effect",
              "harness_component",
              "hypothesis",
              "limitations",
              "observation",
              "proposed_change",
              "title",
              "validation"
            ],
            "type": "object"
          },
          "ThinkingLevel": {
            "oneOf": [
              {
                "enum": [
                  "minimal",
                  "low",
                  "medium",
                  "high",
                  "xhigh"
                ],
                "type": "string"
              },
              {
                "description": "No reasoning at all; honoured only where the model advertises `supports_thinking_off` (elsewhere the provider falls back to its own lowest setting, or omits the parameter, and warns).",
                "enum": [
                  "off"
                ],
                "type": "string"
              }
            ]
          },
          "TriageFailureV1": {
            "additionalProperties": false,
            "properties": {
              "code": {
                "type": "string"
              },
              "http_status": {
                "format": "uint16",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "message": {
                "type": "string"
              },
              "request_id": {
                "type": "string"
              },
              "stats": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/Stats"
                  },
                  {
                    "type": "null"
                  }
                ],
                "description": "Known usage of the failed call, when the provider returned it."
              }
            },
            "required": [
              "code",
              "message",
              "request_id"
            ],
            "type": "object"
          },
          "TriageV1": {
            "additionalProperties": false,
            "properties": {
              "answers": {
                "additionalProperties": {
                  "$ref": "#/definitions/Answer"
                },
                "type": "object"
              },
              "completed_at": {
                "format": "int64",
                "type": "integer"
              },
              "criteria_version": {
                "type": "string"
              },
              "model": {
                "description": "The model the provider reports, not the configured default.",
                "type": "string"
              },
              "provider": {
                "type": "string"
              },
              "request_id": {
                "type": "string"
              },
              "stats": {
                "$ref": "#/definitions/Stats"
              }
            },
            "required": [
              "answers",
              "completed_at",
              "criteria_version",
              "model",
              "provider",
              "request_id",
              "stats"
            ],
            "type": "object"
          },
          "TurnCaptureV1": {
            "additionalProperties": false,
            "description": "The observed turn's record as the Harness stored it, which keeps only a session's latest turn.",
            "properties": {
              "hook_guidance_tokens": {
                "default": 0,
                "description": "Tokens `pre_generate` hooks added at the turn's last step: above zero, the system prompt the model saw is not stored anywhere.",
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "options": {
                "description": "The Harness's `TurnOptions`, unmodified."
              },
              "prompt_surface_digest": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "record": {
                "description": "The whole `TurnRecord` as read when the evidence was captured (it repeats `options`): what a later fork of the session needs, since the Harness overwrites it with the next turn. Absent when it would have taken the assets over their size limit (`record_omitted` says so) and on analyses made before it was kept."
              },
              "record_omitted": {
                "description": "Why `record` was not kept.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "turn_id": {
                "type": "string"
              }
            },
            "required": [
              "options",
              "turn_id"
            ],
            "type": "object"
          },
          "TurnStatus": {
            "description": "The coarse, harness-internal turn lifecycle (harness.md § API Reference). Finer-grained than the session's `status`, which the loop derives from it.",
            "enum": [
              "running",
              "awaiting_functions",
              "completed",
              "cancelled",
              "failed"
            ],
            "type": "string"
          },
          "Usage": {
            "properties": {
              "cache_read": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cache_write": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "input": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "output": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "reasoning": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          },
          "ValidationLinkV1": {
            "additionalProperties": false,
            "description": "Baseline and candidate executions linked to one suggestion. A link is a reference, not a verdict: the improvement claim belongs to the E2E comparison and its criteria.",
            "properties": {
              "attached_at": {
                "format": "int64",
                "type": "integer"
              },
              "baseline": {
                "$ref": "#/definitions/E2eExecutionRefV1"
              },
              "candidate": {
                "$ref": "#/definitions/E2eExecutionRefV1"
              },
              "comparability": {
                "$ref": "#/definitions/ComparabilityV1"
              },
              "suggestion_index": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "attached_at",
              "baseline",
              "candidate",
              "comparability",
              "suggestion_index"
            ],
            "type": "object"
          },
          "ValidationOutcomeV1": {
            "oneOf": [
              {
                "description": "The candidate moved the metric by at least `min_effect` in the wanted direction, with `min_runs` completed runs each side.",
                "enum": [
                  "validated_improvement"
                ],
                "type": "string"
              },
              {
                "description": "The metric moved less than `min_effect`.",
                "enum": [
                  "no_improvement"
                ],
                "type": "string"
              },
              {
                "description": "The metric moved by at least `min_effect` the wrong way.",
                "enum": [
                  "regression"
                ],
                "type": "string"
              },
              {
                "description": "Too few completed runs, missing data or no criterion.",
                "enum": [
                  "inconclusive"
                ],
                "type": "string"
              }
            ]
          },
          "ValidationPlanV1": {
            "additionalProperties": false,
            "description": "The E2E plan a suggestion must carry. A reviewer can run it without the LLM that wrote it.",
            "properties": {
              "expectation": {
                "type": "string"
              },
              "invariants": {
                "description": "Task-correctness invariants checked by an independent evaluator.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "non_regression_controls": {
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "primary_metric": {
                "description": "The effort metric compared between baseline and candidate.",
                "type": "string"
              },
              "reproduction": {
                "type": "string"
              },
              "scenario_id": {
                "description": "An existing harness-e2e scenario, or null when a new case is needed.",
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "expectation",
              "invariants",
              "non_regression_controls",
              "primary_metric",
              "reproduction"
            ],
            "type": "object"
          },
          "ValidationRunStateV1": {
            "oneOf": [
              {
                "enum": [
                  "failed"
                ],
                "type": "string"
              },
              {
                "description": "Recorded before the executions are requested.",
                "enum": [
                  "starting"
                ],
                "type": "string"
              },
              {
                "description": "Both executions were requested; the sweep waits for them.",
                "enum": [
                  "running"
                ],
                "type": "string"
              },
              {
                "description": "Both ended; the pair is about to be attached.",
                "enum": [
                  "finished"
                ],
                "type": "string"
              },
              {
                "description": "The pair is attached to the analysis and its evidence computed.",
                "enum": [
                  "attached"
                ],
                "type": "string"
              }
            ]
          },
          "ValidationRunV1": {
            "additionalProperties": false,
            "description": "The baseline and candidate E2E executions this worker started for a suggestion (`eval::start-validation`).",
            "properties": {
              "baseline_commit": {
                "description": "The full commits the Harness was built from.",
                "type": "string"
              },
              "baseline_execution_id": {
                "description": "Absent until the E2E accepted the execution.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "candidate_commit": {
                "type": "string"
              },
              "candidate_execution_id": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "error": {
                "description": "Why it failed, or why attaching is being retried; starts with a stable code (`e2e_unavailable:`, `e2e_busy:`…).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "finished_at": {
                "description": "When both executions were seen ended; the sweep stops trying to attach the pair some time after.",
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "model": {
                "type": "string"
              },
              "provider": {
                "type": "string"
              },
              "runs": {
                "description": "Runs requested for each side.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "scenario_id": {
                "type": "string"
              },
              "started_at": {
                "format": "int64",
                "type": "integer"
              },
              "state": {
                "$ref": "#/definitions/ValidationRunStateV1"
              }
            },
            "required": [
              "baseline_commit",
              "candidate_commit",
              "model",
              "provider",
              "runs",
              "scenario_id",
              "started_at",
              "state"
            ],
            "type": "object"
          },
          "VerdictV1": {
            "additionalProperties": false,
            "description": "The person's judgment, kept with the criterion it was made against.",
            "properties": {
              "at": {
                "format": "int64",
                "type": "integer"
              },
              "by": {
                "type": "string"
              },
              "controls_checked": {
                "description": "The non-regression controls of the plan that were checked.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "criterion_snapshot": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/CriterionV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "outcome": {
                "$ref": "#/definitions/ValidationOutcomeV1"
              },
              "rationale": {
                "type": "string"
              }
            },
            "required": [
              "at",
              "by",
              "controls_checked",
              "outcome",
              "rationale"
            ],
            "type": "object"
          }
        },
        "title": "Nullable_EvalResultResponseV1"
      }
    },
    {
      "description": "Record what a person decided about a suggestion of a terminal analysis: move its lifecycle (new, accepted, in_progress, shipped, rejected, duplicate), register the criterion a validation is judged by, or record a verdict. A verdict of validated_improvement is refused unless the criterion was registered before the first attached pair or run and both sides have min_runs completed runs. The author is `by`, else the eval host's user, else the caller's identity.",
      "metadata": {},
      "name": "eval::review",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "CriterionInputV1": {
            "additionalProperties": false,
            "description": "What a person commits to before the results exist.",
            "properties": {
              "direction": {
                "allOf": [
                  {
                    "$ref": "#/definitions/DirectionV1"
                  }
                ],
                "description": "The change that counts as an effect, wanted when the candidate moves the baseline mean in this direction."
              },
              "metric": {
                "$ref": "#/definitions/CriterionMetricV1"
              },
              "min_effect": {
                "description": "The smallest change of the baseline mean that counts, as an absolute difference in the metric's own unit: signals per run, USD, seconds, calls or tokens; percentage points (0-100) for `pass_rate`. Above 0.",
                "format": "double",
                "type": "number"
              },
              "min_runs": {
                "description": "Completed runs each side needs for a verdict of improvement (1-20).",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "pattern": {
                "description": "`signal_per_run` only: one of the row's `patterns`.",
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "direction",
              "metric",
              "min_effect",
              "min_runs"
            ],
            "type": "object"
          },
          "CriterionMetricV1": {
            "description": "What a validation measures. Every metric but `signal_per_run` is the mean over the runs of the scenario the infrastructure did not break (`pass_rate` is the share that passed, an unfinished run counting as not passed; `duration` is the wall time in milliseconds).",
            "oneOf": [
              {
                "enum": [
                  "pass_rate",
                  "cost_usd",
                  "duration",
                  "tokens",
                  "function_calls"
                ],
                "type": "string"
              },
              {
                "description": "Occurrences of one of the suggestion's patterns per run, counted by the deterministic detectors over each run's transcript.",
                "enum": [
                  "signal_per_run"
                ],
                "type": "string"
              }
            ]
          },
          "DirectionV1": {
            "enum": [
              "decrease",
              "increase"
            ],
            "type": "string"
          },
          "LifecycleStatusV1": {
            "description": "Where a suggestion stands. Only people move it: the monitor never does.",
            "enum": [
              "new",
              "accepted",
              "in_progress",
              "shipped",
              "rejected",
              "duplicate"
            ],
            "type": "string"
          },
          "ReviewActionV1": {
            "oneOf": [
              {
                "description": "Move the suggestion to `status`.",
                "enum": [
                  "set_lifecycle"
                ],
                "type": "string"
              },
              {
                "description": "Register the criterion a validation is judged by, before its results.",
                "enum": [
                  "set_criterion"
                ],
                "type": "string"
              },
              {
                "description": "Record the person's verdict.",
                "enum": [
                  "set_verdict"
                ],
                "type": "string"
              }
            ]
          },
          "ValidationOutcomeV1": {
            "oneOf": [
              {
                "description": "The candidate moved the metric by at least `min_effect` in the wanted direction, with `min_runs` completed runs each side.",
                "enum": [
                  "validated_improvement"
                ],
                "type": "string"
              },
              {
                "description": "The metric moved less than `min_effect`.",
                "enum": [
                  "no_improvement"
                ],
                "type": "string"
              },
              {
                "description": "The metric moved by at least `min_effect` the wrong way.",
                "enum": [
                  "regression"
                ],
                "type": "string"
              },
              {
                "description": "Too few completed runs, missing data or no criterion.",
                "enum": [
                  "inconclusive"
                ],
                "type": "string"
              }
            ]
          }
        },
        "description": "`eval::review`. The fields of the other actions are ignored. The author is the caller's identity when the engine stamps one, else `by`.",
        "properties": {
          "_caller_worker_id": {
            "default": null,
            "description": "Stamped by the engine on every invocation; callers omit it.",
            "type": [
              "string",
              "null"
            ]
          },
          "action": {
            "$ref": "#/definitions/ReviewActionV1"
          },
          "by": {
            "default": null,
            "description": "The author when the caller has no identity.",
            "type": [
              "string",
              "null"
            ]
          },
          "controls_checked": {
            "default": [],
            "items": {
              "type": "string"
            },
            "type": "array"
          },
          "criterion": {
            "anyOf": [
              {
                "$ref": "#/definitions/CriterionInputV1"
              },
              {
                "type": "null"
              }
            ],
            "default": null,
            "description": "`set_criterion`. Refused once a criterion exists and a run started or a pair was attached: a criterion chosen after the results proves nothing."
          },
          "duplicate_of": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          },
          "evaluation_id": {
            "type": "string"
          },
          "note": {
            "default": null,
            "description": "`set_lifecycle`: shown in the history.",
            "type": [
              "string",
              "null"
            ]
          },
          "outcome": {
            "anyOf": [
              {
                "$ref": "#/definitions/ValidationOutcomeV1"
              },
              {
                "type": "null"
              }
            ],
            "default": null,
            "description": "`set_verdict`. `validated_improvement` is refused unless a criterion was registered before the first attached pair or run, and the evidence has `min_runs` completed runs on both sides."
          },
          "pr": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          },
          "rationale": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          },
          "reason": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          },
          "scenario_id": {
            "default": null,
            "description": "`set_criterion`: the scenario to measure; defaults to the one the suggestion's plan names.",
            "type": [
              "string",
              "null"
            ]
          },
          "status": {
            "anyOf": [
              {
                "$ref": "#/definitions/LifecycleStatusV1"
              },
              {
                "type": "null"
              }
            ],
            "default": null,
            "description": "`set_lifecycle`: the status to move to (never back to `new`; a shipped suggestion stays shipped, to complete its `pr` or `version`). `shipped` needs a `pr` (here or already recorded), `rejected` a `reason`, `duplicate` a `duplicate_of`."
          },
          "suggestion_index": {
            "format": "uint",
            "minimum": 0,
            "type": "integer"
          },
          "version": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          }
        },
        "required": [
          "action",
          "evaluation_id",
          "suggestion_index"
        ],
        "title": "ReviewRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "ChangeEditV1": {
            "additionalProperties": false,
            "description": "One edit of what the model saw. `target` is an entry id of the conversation before the decision point, or `system_prompt`. An edit either replaces `find` with `replace` in the target's text or removes the entry.",
            "properties": {
              "find": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "remove": {
                "type": "boolean"
              },
              "replace": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "target": {
                "type": "string"
              }
            },
            "required": [
              "target"
            ],
            "type": "object"
          },
          "CriterionMetricV1": {
            "description": "What a validation measures. Every metric but `signal_per_run` is the mean over the runs of the scenario the infrastructure did not break (`pass_rate` is the share that passed, an unfinished run counting as not passed; `duration` is the wall time in milliseconds).",
            "oneOf": [
              {
                "enum": [
                  "pass_rate",
                  "cost_usd",
                  "duration",
                  "tokens",
                  "function_calls"
                ],
                "type": "string"
              },
              {
                "description": "Occurrences of one of the suggestion's patterns per run, counted by the deterministic detectors over each run's transcript.",
                "enum": [
                  "signal_per_run"
                ],
                "type": "string"
              }
            ]
          },
          "CriterionV1": {
            "additionalProperties": false,
            "properties": {
              "direction": {
                "$ref": "#/definitions/DirectionV1"
              },
              "metric": {
                "$ref": "#/definitions/CriterionMetricV1"
              },
              "min_effect": {
                "format": "double",
                "type": "number"
              },
              "min_runs": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "pattern": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "registered_at": {
                "format": "int64",
                "type": "integer"
              },
              "registered_by": {
                "type": "string"
              },
              "scenario_id": {
                "description": "The scenario whose runs are measured.",
                "type": "string"
              }
            },
            "required": [
              "direction",
              "metric",
              "min_effect",
              "min_runs",
              "registered_at",
              "registered_by",
              "scenario_id"
            ],
            "type": "object"
          },
          "DirectionV1": {
            "enum": [
              "decrease",
              "increase"
            ],
            "type": "string"
          },
          "EvidenceRunV1": {
            "additionalProperties": false,
            "description": "One run of the scenario as the E2E reported it. Runs the infrastructure broke are listed with the E2E's own words, never counted.",
            "properties": {
              "completed": {
                "description": "The E2E reported the run complete with a valid technical outcome.",
                "type": "boolean"
              },
              "completion": {
                "description": "`completion` as the E2E reported it.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "function_calls": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "run_id": {
                "type": "string"
              },
              "signals": {
                "additionalProperties": {
                  "format": "uint32",
                  "minimum": 0,
                  "type": "integer"
                },
                "description": "Occurrences of each of the row's patterns the detectors found in this run's transcript (0 is a measurement); absent when the report held no transcript.",
                "type": [
                  "object",
                  "null"
                ]
              },
              "status": {
                "description": "`passed` or `failed`, the run's task outcome.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "technical": {
                "description": "`technical` as the E2E reported it; anything but `valid` is an infrastructure failure.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "total_tokens": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "wall_time_ms": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              }
            },
            "required": [
              "completed",
              "run_id"
            ],
            "type": "object"
          },
          "EvidenceSideV1": {
            "additionalProperties": false,
            "properties": {
              "execution_id": {
                "type": "string"
              },
              "mean": {
                "description": "Mean of the criterion's metric over those runs; absent without a criterion or without such runs.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "n": {
                "description": "Measurable runs (see `EvidenceRunV1::measurable`) with a value of the criterion's metric (every measurable run while there is no criterion).",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "runs": {
                "description": "Every run of the scenario the execution holds.",
                "items": {
                  "$ref": "#/definitions/EvidenceRunV1"
                },
                "type": "array"
              }
            },
            "required": [
              "execution_id",
              "n",
              "runs"
            ],
            "type": "object"
          },
          "EvidenceV1": {
            "additionalProperties": false,
            "description": "Computed in code from the attached pair's runs, never by a model. It is what the person's verdict is shown next to.",
            "properties": {
              "baseline": {
                "$ref": "#/definitions/EvidenceSideV1"
              },
              "candidate": {
                "$ref": "#/definitions/EvidenceSideV1"
              },
              "computed_at": {
                "format": "int64",
                "type": "integer"
              },
              "computed_outcome": {
                "$ref": "#/definitions/ValidationOutcomeV1"
              },
              "reason": {
                "type": "string"
              },
              "scenario_id": {
                "type": "string"
              }
            },
            "required": [
              "baseline",
              "candidate",
              "computed_at",
              "computed_outcome",
              "reason",
              "scenario_id"
            ],
            "type": "object"
          },
          "FidelityLevelV1": {
            "oneOf": [
              {
                "enum": [
                  "approximate"
                ],
                "type": "string"
              },
              {
                "description": "The provider counts the rebuilt request like the original, up to a fixed per-request overhead measured at the turn's first step.",
                "enum": [
                  "exact"
                ],
                "type": "string"
              }
            ]
          },
          "FidelityV1": {
            "additionalProperties": false,
            "description": "How faithfully the request at the decision point was rebuilt.",
            "properties": {
              "counted_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "estimator": {
                "description": "`provider` when the provider counted, else the estimator's name.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "level": {
                "$ref": "#/definitions/FidelityLevelV1"
              },
              "off_ratio": {
                "description": "The unexplained difference over the recorded tokens.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "reasons": {
                "default": [],
                "description": "Why the rebuild is approximate; empty when exact.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "recorded_tokens": {
                "description": "Input tokens of the original step: input + cache read + cache write.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "required": [
              "level"
            ],
            "type": "object"
          },
          "LifecycleEventV1": {
            "additionalProperties": false,
            "description": "One change of status, with who made it.",
            "properties": {
              "at": {
                "format": "int64",
                "type": "integer"
              },
              "by": {
                "type": "string"
              },
              "note": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "status": {
                "$ref": "#/definitions/LifecycleStatusV1"
              }
            },
            "required": [
              "at",
              "by",
              "status"
            ],
            "type": "object"
          },
          "LifecycleStatusV1": {
            "description": "Where a suggestion stands. Only people move it: the monitor never does.",
            "enum": [
              "new",
              "accepted",
              "in_progress",
              "shipped",
              "rejected",
              "duplicate"
            ],
            "type": "string"
          },
          "LifecycleV1": {
            "additionalProperties": false,
            "properties": {
              "duplicate_of": {
                "description": "What it duplicates: a pull request, or another `<evaluation_id>:<index>` (`duplicate`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "history": {
                "description": "Every change of status, oldest first; empty while `new`.",
                "items": {
                  "$ref": "#/definitions/LifecycleEventV1"
                },
                "type": "array"
              },
              "pr": {
                "description": "The pull request that implements it (`in_progress`, `shipped`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "reason": {
                "description": "Why it was rejected (`rejected`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "status": {
                "$ref": "#/definitions/LifecycleStatusV1"
              },
              "version": {
                "description": "The Harness version that shipped it, a semantic version like `1.8.43`; the recurrence query splits the analyses at it.",
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "history",
              "status"
            ],
            "type": "object"
          },
          "ReplyCallV1": {
            "additionalProperties": false,
            "description": "One function call of a reply.",
            "properties": {
              "description": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "payload": {
                "description": "The payload as JSON text, cut at 600 characters.",
                "type": "string"
              },
              "target": {
                "description": "The function called (through `agent_call` or directly).",
                "type": "string"
              }
            },
            "required": [
              "payload",
              "target"
            ],
            "type": "object"
          },
          "ReplyUsageV1": {
            "additionalProperties": false,
            "properties": {
              "cache_read_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cache_write_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          },
          "ReplyV1": {
            "additionalProperties": false,
            "description": "One reply at the decision point: a sample, or the original step read the same way.",
            "properties": {
              "calls": {
                "items": {
                  "$ref": "#/definitions/ReplyCallV1"
                },
                "type": "array"
              },
              "duration_ms": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "error": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "index": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "signal": {
                "description": "Whether the signal shows in this reply; null when Jev answered unclear, the reply failed or it was not classified yet.",
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "text": {
                "type": "string"
              },
              "thinking": {
                "description": "The opening of the reply's reasoning and of its text, 400 characters each.",
                "type": "string"
              },
              "usage": {
                "allOf": [
                  {
                    "$ref": "#/definitions/ReplyUsageV1"
                  }
                ],
                "default": {}
              }
            },
            "required": [
              "calls",
              "index",
              "text",
              "thinking"
            ],
            "type": "object"
          },
          "ReproductionChangeKindV1": {
            "oneOf": [
              {
                "description": "The context as the model saw it.",
                "enum": [
                  "none"
                ],
                "type": "string"
              },
              {
                "description": "The suggestion's own `check.change`.",
                "enum": [
                  "proposed"
                ],
                "type": "string"
              },
              {
                "description": "Edits a person wrote.",
                "enum": [
                  "custom"
                ],
                "type": "string"
              }
            ]
          },
          "ReproductionStateV1": {
            "enum": [
              "running",
              "completed",
              "failed"
            ],
            "type": "string"
          },
          "ReproductionV1": {
            "additionalProperties": false,
            "description": "A replay of the decision point: N replies sampled from the rebuilt request, with or without a change. No function ever runs.",
            "properties": {
              "by": {
                "type": "string"
              },
              "change": {
                "default": [],
                "items": {
                  "$ref": "#/definitions/ChangeEditV1"
                },
                "type": "array"
              },
              "change_kind": {
                "$ref": "#/definitions/ReproductionChangeKindV1"
              },
              "check": {
                "allOf": [
                  {
                    "$ref": "#/definitions/SuggestionCheckV1"
                  }
                ],
                "description": "The decision point and signal this replay used."
              },
              "cost_unknown_samples": {
                "default": 0,
                "description": "Samples without a reported cost, failed ones included (the provider may have billed a request that never answered): unknown, not in `cost_usd`, and never estimated.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "cost_usd": {
                "description": "Known cost of the samples; null while no reply reported one. Only a lower bound while `cost_unknown_samples` is above zero. Jev's calls are in tokens below, never priced.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "error": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "fidelity": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/FidelityV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "finished_at": {
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "id": {
                "type": "string"
              },
              "judge_input_tokens": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "judge_output_tokens": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "model": {
                "type": "string"
              },
              "original": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ReplyV1"
                  },
                  {
                    "type": "null"
                  }
                ],
                "description": "The original reply at the decision point, read the same way."
              },
              "phase": {
                "description": "`sampling` or `classifying` while running.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "provider": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "requested": {
                "description": "Replies asked for, extensions included.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "samples": {
                "default": [],
                "items": {
                  "$ref": "#/definitions/ReplyV1"
                },
                "type": "array"
              },
              "started_at": {
                "format": "int64",
                "type": "integer"
              },
              "state": {
                "$ref": "#/definitions/ReproductionStateV1"
              },
              "updated_at": {
                "format": "int64",
                "type": "integer"
              }
            },
            "required": [
              "by",
              "change_kind",
              "check",
              "id",
              "model",
              "requested",
              "started_at",
              "state",
              "updated_at"
            ],
            "type": "object"
          },
          "SignalRuleV1": {
            "oneOf": [
              {
                "description": "The reply asks `engine::functions::info` only for contracts an earlier result in the context already supplied.",
                "enum": [
                  "contract_rediscovery"
                ],
                "type": "string"
              },
              {
                "description": "The reply repeats the last call that failed: same target, equal payload.",
                "enum": [
                  "repeated_error_call"
                ],
                "type": "string"
              }
            ]
          },
          "SignalV1": {
            "additionalProperties": false,
            "description": "How to recognize the behavior in one reply: exactly one of a rule computed in code or a yes/no question Jev answers about the reply.",
            "properties": {
              "question": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "rule": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/SignalRuleV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "type": "object"
          },
          "SuggestionCheckV1": {
            "additionalProperties": false,
            "description": "How a suggestion is checked at the step where its behavior happened.",
            "properties": {
              "change": {
                "default": [],
                "description": "The proposed change as edits of what the model saw before that step; empty when it cannot be expressed as text.",
                "items": {
                  "$ref": "#/definitions/ChangeEditV1"
                },
                "type": "array"
              },
              "decision_point": {
                "description": "The assistant entry where the behavior happened (`…_<step>_assistant`), one of the suggestion's `evidence` entries.",
                "type": "string"
              },
              "signal": {
                "$ref": "#/definitions/SignalV1"
              }
            },
            "required": [
              "decision_point",
              "signal"
            ],
            "type": "object"
          },
          "ValidationOutcomeV1": {
            "oneOf": [
              {
                "description": "The candidate moved the metric by at least `min_effect` in the wanted direction, with `min_runs` completed runs each side.",
                "enum": [
                  "validated_improvement"
                ],
                "type": "string"
              },
              {
                "description": "The metric moved less than `min_effect`.",
                "enum": [
                  "no_improvement"
                ],
                "type": "string"
              },
              {
                "description": "The metric moved by at least `min_effect` the wrong way.",
                "enum": [
                  "regression"
                ],
                "type": "string"
              },
              {
                "description": "Too few completed runs, missing data or no criterion.",
                "enum": [
                  "inconclusive"
                ],
                "type": "string"
              }
            ]
          },
          "ValidationRunStateV1": {
            "oneOf": [
              {
                "enum": [
                  "failed"
                ],
                "type": "string"
              },
              {
                "description": "Recorded before the executions are requested.",
                "enum": [
                  "starting"
                ],
                "type": "string"
              },
              {
                "description": "Both executions were requested; the sweep waits for them.",
                "enum": [
                  "running"
                ],
                "type": "string"
              },
              {
                "description": "Both ended; the pair is about to be attached.",
                "enum": [
                  "finished"
                ],
                "type": "string"
              },
              {
                "description": "The pair is attached to the analysis and its evidence computed.",
                "enum": [
                  "attached"
                ],
                "type": "string"
              }
            ]
          },
          "ValidationRunV1": {
            "additionalProperties": false,
            "description": "The baseline and candidate E2E executions this worker started for a suggestion (`eval::start-validation`).",
            "properties": {
              "baseline_commit": {
                "description": "The full commits the Harness was built from.",
                "type": "string"
              },
              "baseline_execution_id": {
                "description": "Absent until the E2E accepted the execution.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "candidate_commit": {
                "type": "string"
              },
              "candidate_execution_id": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "error": {
                "description": "Why it failed, or why attaching is being retried; starts with a stable code (`e2e_unavailable:`, `e2e_busy:`…).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "finished_at": {
                "description": "When both executions were seen ended; the sweep stops trying to attach the pair some time after.",
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "model": {
                "type": "string"
              },
              "provider": {
                "type": "string"
              },
              "runs": {
                "description": "Runs requested for each side.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "scenario_id": {
                "type": "string"
              },
              "started_at": {
                "format": "int64",
                "type": "integer"
              },
              "state": {
                "$ref": "#/definitions/ValidationRunStateV1"
              }
            },
            "required": [
              "baseline_commit",
              "candidate_commit",
              "model",
              "provider",
              "runs",
              "scenario_id",
              "started_at",
              "state"
            ],
            "type": "object"
          },
          "VerdictV1": {
            "additionalProperties": false,
            "description": "The person's judgment, kept with the criterion it was made against.",
            "properties": {
              "at": {
                "format": "int64",
                "type": "integer"
              },
              "by": {
                "type": "string"
              },
              "controls_checked": {
                "description": "The non-regression controls of the plan that were checked.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "criterion_snapshot": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/CriterionV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "outcome": {
                "$ref": "#/definitions/ValidationOutcomeV1"
              },
              "rationale": {
                "type": "string"
              }
            },
            "required": [
              "at",
              "by",
              "controls_checked",
              "outcome",
              "rationale"
            ],
            "type": "object"
          }
        },
        "description": "Everything people decided about one suggestion, in the `eval_suggestion` state scope. The title, component and patterns are copied from the suggestion so the row outlives the analysis's retention.",
        "properties": {
          "criterion": {
            "anyOf": [
              {
                "$ref": "#/definitions/CriterionV1"
              },
              {
                "type": "null"
              }
            ]
          },
          "evaluation_id": {
            "type": "string"
          },
          "evidence": {
            "anyOf": [
              {
                "$ref": "#/definitions/EvidenceV1"
              },
              {
                "type": "null"
              }
            ]
          },
          "first_run_at": {
            "description": "When the first execution this worker started for the suggestion began, kept when a new start replaces `run`: results of it may have been seen.",
            "format": "int64",
            "type": [
              "integer",
              "null"
            ]
          },
          "harness_component": {
            "type": "string"
          },
          "lifecycle": {
            "$ref": "#/definitions/LifecycleV1"
          },
          "patterns": {
            "description": "`<rule_id>:<target>` of the diagnostics the suggestion's evidence cites: what a validation counts and a release is judged by.",
            "items": {
              "type": "string"
            },
            "type": "array"
          },
          "reproductions": {
            "description": "Replays of the decision point, oldest first: the base and the tested changes.",
            "items": {
              "$ref": "#/definitions/ReproductionV1"
            },
            "type": "array"
          },
          "run": {
            "anyOf": [
              {
                "$ref": "#/definitions/ValidationRunV1"
              },
              {
                "type": "null"
              }
            ]
          },
          "scenario_id": {
            "description": "The scenario the suggestion's validation plan names.",
            "type": [
              "string",
              "null"
            ]
          },
          "suggestion_index": {
            "format": "uint",
            "minimum": 0,
            "type": "integer"
          },
          "title": {
            "type": "string"
          },
          "verdict": {
            "anyOf": [
              {
                "$ref": "#/definitions/VerdictV1"
              },
              {
                "type": "null"
              }
            ]
          }
        },
        "required": [
          "evaluation_id",
          "harness_component",
          "lifecycle",
          "patterns",
          "suggestion_index",
          "title"
        ],
        "title": "SuggestionReviewV1",
        "type": "object"
      }
    },
    {
      "description": "List the stored suggestion reviews (lifecycle, criterion, run, evidence, verdict) and, per analysis with suggestions, how many sit at each lifecycle status.",
      "metadata": {},
      "name": "eval::reviews",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "evaluation_id": {
            "default": null,
            "description": "Only this analysis; absent means every analysis.",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "ReviewsRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "ChangeEditV1": {
            "additionalProperties": false,
            "description": "One edit of what the model saw. `target` is an entry id of the conversation before the decision point, or `system_prompt`. An edit either replaces `find` with `replace` in the target's text or removes the entry.",
            "properties": {
              "find": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "remove": {
                "type": "boolean"
              },
              "replace": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "target": {
                "type": "string"
              }
            },
            "required": [
              "target"
            ],
            "type": "object"
          },
          "CriterionMetricV1": {
            "description": "What a validation measures. Every metric but `signal_per_run` is the mean over the runs of the scenario the infrastructure did not break (`pass_rate` is the share that passed, an unfinished run counting as not passed; `duration` is the wall time in milliseconds).",
            "oneOf": [
              {
                "enum": [
                  "pass_rate",
                  "cost_usd",
                  "duration",
                  "tokens",
                  "function_calls"
                ],
                "type": "string"
              },
              {
                "description": "Occurrences of one of the suggestion's patterns per run, counted by the deterministic detectors over each run's transcript.",
                "enum": [
                  "signal_per_run"
                ],
                "type": "string"
              }
            ]
          },
          "CriterionV1": {
            "additionalProperties": false,
            "properties": {
              "direction": {
                "$ref": "#/definitions/DirectionV1"
              },
              "metric": {
                "$ref": "#/definitions/CriterionMetricV1"
              },
              "min_effect": {
                "format": "double",
                "type": "number"
              },
              "min_runs": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "pattern": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "registered_at": {
                "format": "int64",
                "type": "integer"
              },
              "registered_by": {
                "type": "string"
              },
              "scenario_id": {
                "description": "The scenario whose runs are measured.",
                "type": "string"
              }
            },
            "required": [
              "direction",
              "metric",
              "min_effect",
              "min_runs",
              "registered_at",
              "registered_by",
              "scenario_id"
            ],
            "type": "object"
          },
          "DirectionV1": {
            "enum": [
              "decrease",
              "increase"
            ],
            "type": "string"
          },
          "EvidenceRunV1": {
            "additionalProperties": false,
            "description": "One run of the scenario as the E2E reported it. Runs the infrastructure broke are listed with the E2E's own words, never counted.",
            "properties": {
              "completed": {
                "description": "The E2E reported the run complete with a valid technical outcome.",
                "type": "boolean"
              },
              "completion": {
                "description": "`completion` as the E2E reported it.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "function_calls": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "run_id": {
                "type": "string"
              },
              "signals": {
                "additionalProperties": {
                  "format": "uint32",
                  "minimum": 0,
                  "type": "integer"
                },
                "description": "Occurrences of each of the row's patterns the detectors found in this run's transcript (0 is a measurement); absent when the report held no transcript.",
                "type": [
                  "object",
                  "null"
                ]
              },
              "status": {
                "description": "`passed` or `failed`, the run's task outcome.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "technical": {
                "description": "`technical` as the E2E reported it; anything but `valid` is an infrastructure failure.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "total_tokens": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "wall_time_ms": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              }
            },
            "required": [
              "completed",
              "run_id"
            ],
            "type": "object"
          },
          "EvidenceSideV1": {
            "additionalProperties": false,
            "properties": {
              "execution_id": {
                "type": "string"
              },
              "mean": {
                "description": "Mean of the criterion's metric over those runs; absent without a criterion or without such runs.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "n": {
                "description": "Measurable runs (see `EvidenceRunV1::measurable`) with a value of the criterion's metric (every measurable run while there is no criterion).",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "runs": {
                "description": "Every run of the scenario the execution holds.",
                "items": {
                  "$ref": "#/definitions/EvidenceRunV1"
                },
                "type": "array"
              }
            },
            "required": [
              "execution_id",
              "n",
              "runs"
            ],
            "type": "object"
          },
          "EvidenceV1": {
            "additionalProperties": false,
            "description": "Computed in code from the attached pair's runs, never by a model. It is what the person's verdict is shown next to.",
            "properties": {
              "baseline": {
                "$ref": "#/definitions/EvidenceSideV1"
              },
              "candidate": {
                "$ref": "#/definitions/EvidenceSideV1"
              },
              "computed_at": {
                "format": "int64",
                "type": "integer"
              },
              "computed_outcome": {
                "$ref": "#/definitions/ValidationOutcomeV1"
              },
              "reason": {
                "type": "string"
              },
              "scenario_id": {
                "type": "string"
              }
            },
            "required": [
              "baseline",
              "candidate",
              "computed_at",
              "computed_outcome",
              "reason",
              "scenario_id"
            ],
            "type": "object"
          },
          "FidelityLevelV1": {
            "oneOf": [
              {
                "enum": [
                  "approximate"
                ],
                "type": "string"
              },
              {
                "description": "The provider counts the rebuilt request like the original, up to a fixed per-request overhead measured at the turn's first step.",
                "enum": [
                  "exact"
                ],
                "type": "string"
              }
            ]
          },
          "FidelityV1": {
            "additionalProperties": false,
            "description": "How faithfully the request at the decision point was rebuilt.",
            "properties": {
              "counted_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "estimator": {
                "description": "`provider` when the provider counted, else the estimator's name.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "level": {
                "$ref": "#/definitions/FidelityLevelV1"
              },
              "off_ratio": {
                "description": "The unexplained difference over the recorded tokens.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "reasons": {
                "default": [],
                "description": "Why the rebuild is approximate; empty when exact.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "recorded_tokens": {
                "description": "Input tokens of the original step: input + cache read + cache write.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "required": [
              "level"
            ],
            "type": "object"
          },
          "LifecycleEventV1": {
            "additionalProperties": false,
            "description": "One change of status, with who made it.",
            "properties": {
              "at": {
                "format": "int64",
                "type": "integer"
              },
              "by": {
                "type": "string"
              },
              "note": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "status": {
                "$ref": "#/definitions/LifecycleStatusV1"
              }
            },
            "required": [
              "at",
              "by",
              "status"
            ],
            "type": "object"
          },
          "LifecycleStatusV1": {
            "description": "Where a suggestion stands. Only people move it: the monitor never does.",
            "enum": [
              "new",
              "accepted",
              "in_progress",
              "shipped",
              "rejected",
              "duplicate"
            ],
            "type": "string"
          },
          "LifecycleV1": {
            "additionalProperties": false,
            "properties": {
              "duplicate_of": {
                "description": "What it duplicates: a pull request, or another `<evaluation_id>:<index>` (`duplicate`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "history": {
                "description": "Every change of status, oldest first; empty while `new`.",
                "items": {
                  "$ref": "#/definitions/LifecycleEventV1"
                },
                "type": "array"
              },
              "pr": {
                "description": "The pull request that implements it (`in_progress`, `shipped`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "reason": {
                "description": "Why it was rejected (`rejected`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "status": {
                "$ref": "#/definitions/LifecycleStatusV1"
              },
              "version": {
                "description": "The Harness version that shipped it, a semantic version like `1.8.43`; the recurrence query splits the analyses at it.",
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "history",
              "status"
            ],
            "type": "object"
          },
          "ReplyCallV1": {
            "additionalProperties": false,
            "description": "One function call of a reply.",
            "properties": {
              "description": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "payload": {
                "description": "The payload as JSON text, cut at 600 characters.",
                "type": "string"
              },
              "target": {
                "description": "The function called (through `agent_call` or directly).",
                "type": "string"
              }
            },
            "required": [
              "payload",
              "target"
            ],
            "type": "object"
          },
          "ReplyUsageV1": {
            "additionalProperties": false,
            "properties": {
              "cache_read_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cache_write_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          },
          "ReplyV1": {
            "additionalProperties": false,
            "description": "One reply at the decision point: a sample, or the original step read the same way.",
            "properties": {
              "calls": {
                "items": {
                  "$ref": "#/definitions/ReplyCallV1"
                },
                "type": "array"
              },
              "duration_ms": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "error": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "index": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "signal": {
                "description": "Whether the signal shows in this reply; null when Jev answered unclear, the reply failed or it was not classified yet.",
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "text": {
                "type": "string"
              },
              "thinking": {
                "description": "The opening of the reply's reasoning and of its text, 400 characters each.",
                "type": "string"
              },
              "usage": {
                "allOf": [
                  {
                    "$ref": "#/definitions/ReplyUsageV1"
                  }
                ],
                "default": {}
              }
            },
            "required": [
              "calls",
              "index",
              "text",
              "thinking"
            ],
            "type": "object"
          },
          "ReproductionChangeKindV1": {
            "oneOf": [
              {
                "description": "The context as the model saw it.",
                "enum": [
                  "none"
                ],
                "type": "string"
              },
              {
                "description": "The suggestion's own `check.change`.",
                "enum": [
                  "proposed"
                ],
                "type": "string"
              },
              {
                "description": "Edits a person wrote.",
                "enum": [
                  "custom"
                ],
                "type": "string"
              }
            ]
          },
          "ReproductionStateV1": {
            "enum": [
              "running",
              "completed",
              "failed"
            ],
            "type": "string"
          },
          "ReproductionV1": {
            "additionalProperties": false,
            "description": "A replay of the decision point: N replies sampled from the rebuilt request, with or without a change. No function ever runs.",
            "properties": {
              "by": {
                "type": "string"
              },
              "change": {
                "default": [],
                "items": {
                  "$ref": "#/definitions/ChangeEditV1"
                },
                "type": "array"
              },
              "change_kind": {
                "$ref": "#/definitions/ReproductionChangeKindV1"
              },
              "check": {
                "allOf": [
                  {
                    "$ref": "#/definitions/SuggestionCheckV1"
                  }
                ],
                "description": "The decision point and signal this replay used."
              },
              "cost_unknown_samples": {
                "default": 0,
                "description": "Samples without a reported cost, failed ones included (the provider may have billed a request that never answered): unknown, not in `cost_usd`, and never estimated.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "cost_usd": {
                "description": "Known cost of the samples; null while no reply reported one. Only a lower bound while `cost_unknown_samples` is above zero. Jev's calls are in tokens below, never priced.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "error": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "fidelity": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/FidelityV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "finished_at": {
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "id": {
                "type": "string"
              },
              "judge_input_tokens": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "judge_output_tokens": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "model": {
                "type": "string"
              },
              "original": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ReplyV1"
                  },
                  {
                    "type": "null"
                  }
                ],
                "description": "The original reply at the decision point, read the same way."
              },
              "phase": {
                "description": "`sampling` or `classifying` while running.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "provider": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "requested": {
                "description": "Replies asked for, extensions included.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "samples": {
                "default": [],
                "items": {
                  "$ref": "#/definitions/ReplyV1"
                },
                "type": "array"
              },
              "started_at": {
                "format": "int64",
                "type": "integer"
              },
              "state": {
                "$ref": "#/definitions/ReproductionStateV1"
              },
              "updated_at": {
                "format": "int64",
                "type": "integer"
              }
            },
            "required": [
              "by",
              "change_kind",
              "check",
              "id",
              "model",
              "requested",
              "started_at",
              "state",
              "updated_at"
            ],
            "type": "object"
          },
          "ReviewSummaryV1": {
            "additionalProperties": false,
            "description": "Suggestions of one analysis by lifecycle status; a suggestion nobody acted on counts as `new`.",
            "properties": {
              "accepted": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "duplicate": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "evaluation_id": {
                "type": "string"
              },
              "in_progress": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "new": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "rejected": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "shipped": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "suggestions": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "accepted",
              "duplicate",
              "evaluation_id",
              "in_progress",
              "new",
              "rejected",
              "shipped",
              "suggestions"
            ],
            "type": "object"
          },
          "SignalRuleV1": {
            "oneOf": [
              {
                "description": "The reply asks `engine::functions::info` only for contracts an earlier result in the context already supplied.",
                "enum": [
                  "contract_rediscovery"
                ],
                "type": "string"
              },
              {
                "description": "The reply repeats the last call that failed: same target, equal payload.",
                "enum": [
                  "repeated_error_call"
                ],
                "type": "string"
              }
            ]
          },
          "SignalV1": {
            "additionalProperties": false,
            "description": "How to recognize the behavior in one reply: exactly one of a rule computed in code or a yes/no question Jev answers about the reply.",
            "properties": {
              "question": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "rule": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/SignalRuleV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "type": "object"
          },
          "SuggestionCheckV1": {
            "additionalProperties": false,
            "description": "How a suggestion is checked at the step where its behavior happened.",
            "properties": {
              "change": {
                "default": [],
                "description": "The proposed change as edits of what the model saw before that step; empty when it cannot be expressed as text.",
                "items": {
                  "$ref": "#/definitions/ChangeEditV1"
                },
                "type": "array"
              },
              "decision_point": {
                "description": "The assistant entry where the behavior happened (`…_<step>_assistant`), one of the suggestion's `evidence` entries.",
                "type": "string"
              },
              "signal": {
                "$ref": "#/definitions/SignalV1"
              }
            },
            "required": [
              "decision_point",
              "signal"
            ],
            "type": "object"
          },
          "SuggestionReviewV1": {
            "additionalProperties": false,
            "description": "Everything people decided about one suggestion, in the `eval_suggestion` state scope. The title, component and patterns are copied from the suggestion so the row outlives the analysis's retention.",
            "properties": {
              "criterion": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/CriterionV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "evaluation_id": {
                "type": "string"
              },
              "evidence": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/EvidenceV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "first_run_at": {
                "description": "When the first execution this worker started for the suggestion began, kept when a new start replaces `run`: results of it may have been seen.",
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "harness_component": {
                "type": "string"
              },
              "lifecycle": {
                "$ref": "#/definitions/LifecycleV1"
              },
              "patterns": {
                "description": "`<rule_id>:<target>` of the diagnostics the suggestion's evidence cites: what a validation counts and a release is judged by.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "reproductions": {
                "description": "Replays of the decision point, oldest first: the base and the tested changes.",
                "items": {
                  "$ref": "#/definitions/ReproductionV1"
                },
                "type": "array"
              },
              "run": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ValidationRunV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "scenario_id": {
                "description": "The scenario the suggestion's validation plan names.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "suggestion_index": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "title": {
                "type": "string"
              },
              "verdict": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/VerdictV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "required": [
              "evaluation_id",
              "harness_component",
              "lifecycle",
              "patterns",
              "suggestion_index",
              "title"
            ],
            "type": "object"
          },
          "ValidationOutcomeV1": {
            "oneOf": [
              {
                "description": "The candidate moved the metric by at least `min_effect` in the wanted direction, with `min_runs` completed runs each side.",
                "enum": [
                  "validated_improvement"
                ],
                "type": "string"
              },
              {
                "description": "The metric moved less than `min_effect`.",
                "enum": [
                  "no_improvement"
                ],
                "type": "string"
              },
              {
                "description": "The metric moved by at least `min_effect` the wrong way.",
                "enum": [
                  "regression"
                ],
                "type": "string"
              },
              {
                "description": "Too few completed runs, missing data or no criterion.",
                "enum": [
                  "inconclusive"
                ],
                "type": "string"
              }
            ]
          },
          "ValidationRunStateV1": {
            "oneOf": [
              {
                "enum": [
                  "failed"
                ],
                "type": "string"
              },
              {
                "description": "Recorded before the executions are requested.",
                "enum": [
                  "starting"
                ],
                "type": "string"
              },
              {
                "description": "Both executions were requested; the sweep waits for them.",
                "enum": [
                  "running"
                ],
                "type": "string"
              },
              {
                "description": "Both ended; the pair is about to be attached.",
                "enum": [
                  "finished"
                ],
                "type": "string"
              },
              {
                "description": "The pair is attached to the analysis and its evidence computed.",
                "enum": [
                  "attached"
                ],
                "type": "string"
              }
            ]
          },
          "ValidationRunV1": {
            "additionalProperties": false,
            "description": "The baseline and candidate E2E executions this worker started for a suggestion (`eval::start-validation`).",
            "properties": {
              "baseline_commit": {
                "description": "The full commits the Harness was built from.",
                "type": "string"
              },
              "baseline_execution_id": {
                "description": "Absent until the E2E accepted the execution.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "candidate_commit": {
                "type": "string"
              },
              "candidate_execution_id": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "error": {
                "description": "Why it failed, or why attaching is being retried; starts with a stable code (`e2e_unavailable:`, `e2e_busy:`…).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "finished_at": {
                "description": "When both executions were seen ended; the sweep stops trying to attach the pair some time after.",
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "model": {
                "type": "string"
              },
              "provider": {
                "type": "string"
              },
              "runs": {
                "description": "Runs requested for each side.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "scenario_id": {
                "type": "string"
              },
              "started_at": {
                "format": "int64",
                "type": "integer"
              },
              "state": {
                "$ref": "#/definitions/ValidationRunStateV1"
              }
            },
            "required": [
              "baseline_commit",
              "candidate_commit",
              "model",
              "provider",
              "runs",
              "scenario_id",
              "started_at",
              "state"
            ],
            "type": "object"
          },
          "VerdictV1": {
            "additionalProperties": false,
            "description": "The person's judgment, kept with the criterion it was made against.",
            "properties": {
              "at": {
                "format": "int64",
                "type": "integer"
              },
              "by": {
                "type": "string"
              },
              "controls_checked": {
                "description": "The non-regression controls of the plan that were checked.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "criterion_snapshot": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/CriterionV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "outcome": {
                "$ref": "#/definitions/ValidationOutcomeV1"
              },
              "rationale": {
                "type": "string"
              }
            },
            "required": [
              "at",
              "by",
              "controls_checked",
              "outcome",
              "rationale"
            ],
            "type": "object"
          }
        },
        "properties": {
          "reviews": {
            "description": "The stored rows: suggestions somebody acted on or validated.",
            "items": {
              "$ref": "#/definitions/SuggestionReviewV1"
            },
            "type": "array"
          },
          "summaries": {
            "description": "One per analysis that has suggestions, newest first.",
            "items": {
              "$ref": "#/definitions/ReviewSummaryV1"
            },
            "type": "array"
          }
        },
        "required": [
          "reviews",
          "summaries"
        ],
        "title": "ReviewsResponseV1",
        "type": "object"
      }
    },
    {
      "description": "Explicitly start a baseline and a candidate E2E execution for one suggestion. Both commits (candidate_ref, and baseline_ref or the merge base with origin/main) are resolved with git in the configured code_repository and must be on a remote branch; each stack is the E2E's `harness` template with only the Harness pinned to its commit, run in Docker (runs 1-20, default 5). The criterion is registered before anything starts; the sweep attaches the pair when both finish and computes the evidence. Spends model tokens, and never runs unless called. With dry_run: true it only resolves and checks the refs, scenario and runs and answers with the resolved commits (`baseline`, `candidate`, `warnings`); it registers nothing and calls no E2E function.",
      "metadata": {},
      "name": "eval::start-validation",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "CriterionInputV1": {
            "additionalProperties": false,
            "description": "What a person commits to before the results exist.",
            "properties": {
              "direction": {
                "allOf": [
                  {
                    "$ref": "#/definitions/DirectionV1"
                  }
                ],
                "description": "The change that counts as an effect, wanted when the candidate moves the baseline mean in this direction."
              },
              "metric": {
                "$ref": "#/definitions/CriterionMetricV1"
              },
              "min_effect": {
                "description": "The smallest change of the baseline mean that counts, as an absolute difference in the metric's own unit: signals per run, USD, seconds, calls or tokens; percentage points (0-100) for `pass_rate`. Above 0.",
                "format": "double",
                "type": "number"
              },
              "min_runs": {
                "description": "Completed runs each side needs for a verdict of improvement (1-20).",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "pattern": {
                "description": "`signal_per_run` only: one of the row's `patterns`.",
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "direction",
              "metric",
              "min_effect",
              "min_runs"
            ],
            "type": "object"
          },
          "CriterionMetricV1": {
            "description": "What a validation measures. Every metric but `signal_per_run` is the mean over the runs of the scenario the infrastructure did not break (`pass_rate` is the share that passed, an unfinished run counting as not passed; `duration` is the wall time in milliseconds).",
            "oneOf": [
              {
                "enum": [
                  "pass_rate",
                  "cost_usd",
                  "duration",
                  "tokens",
                  "function_calls"
                ],
                "type": "string"
              },
              {
                "description": "Occurrences of one of the suggestion's patterns per run, counted by the deterministic detectors over each run's transcript.",
                "enum": [
                  "signal_per_run"
                ],
                "type": "string"
              }
            ]
          },
          "DirectionV1": {
            "enum": [
              "decrease",
              "increase"
            ],
            "type": "string"
          }
        },
        "description": "`eval::start-validation`: builds a baseline and a candidate stack from the E2E's `harness` template pinned to two pushed commits and starts both executions in Docker. Spends model tokens, so it only ever runs on an explicit request. With `dry_run` it only resolves and checks the refs.",
        "properties": {
          "_caller_worker_id": {
            "default": null,
            "description": "Stamped by the engine on every invocation; callers omit it.",
            "type": [
              "string",
              "null"
            ]
          },
          "baseline_ref": {
            "default": null,
            "description": "Defaults to the merge base of the candidate and `origin/main`.",
            "type": [
              "string",
              "null"
            ]
          },
          "by": {
            "default": null,
            "description": "The author: this name wins over the host's user and the caller's identity.",
            "type": [
              "string",
              "null"
            ]
          },
          "candidate_ref": {
            "description": "A ref of `code_repository` (a branch, a tag or a commit) that is on a remote branch.",
            "type": "string"
          },
          "criterion": {
            "anyOf": [
              {
                "$ref": "#/definitions/CriterionInputV1"
              },
              {
                "type": "null"
              }
            ],
            "default": null,
            "description": "Registered first, before any execution starts. Required unless `dry_run`, which only checks it against `runs` when present."
          },
          "dry_run": {
            "default": false,
            "description": "Resolve and check the refs, the scenario and the runs exactly as a start does, and answer with the commits they resolve to. Registers nothing, calls no E2E function and spends nothing.",
            "type": "boolean"
          },
          "evaluation_id": {
            "type": "string"
          },
          "model": {
            "default": "",
            "description": "Required unless `dry_run`.",
            "type": "string"
          },
          "provider": {
            "default": "",
            "description": "Required unless `dry_run`.",
            "type": "string"
          },
          "runs": {
            "default": null,
            "description": "Runs of the scenario for each side; defaults to 5 (1-20).",
            "format": "uint32",
            "minimum": 0,
            "type": [
              "integer",
              "null"
            ]
          },
          "scenario_id": {
            "description": "The harness-e2e scenario to run.",
            "type": "string"
          },
          "suggestion_index": {
            "format": "uint",
            "minimum": 0,
            "type": "integer"
          }
        },
        "required": [
          "candidate_ref",
          "evaluation_id",
          "scenario_id",
          "suggestion_index"
        ],
        "title": "StartValidationRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "anyOf": [
          {
            "$ref": "#/definitions/SuggestionReviewV1"
          },
          {
            "$ref": "#/definitions/ValidationResolutionV1"
          }
        ],
        "definitions": {
          "ChangeEditV1": {
            "additionalProperties": false,
            "description": "One edit of what the model saw. `target` is an entry id of the conversation before the decision point, or `system_prompt`. An edit either replaces `find` with `replace` in the target's text or removes the entry.",
            "properties": {
              "find": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "remove": {
                "type": "boolean"
              },
              "replace": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "target": {
                "type": "string"
              }
            },
            "required": [
              "target"
            ],
            "type": "object"
          },
          "CriterionMetricV1": {
            "description": "What a validation measures. Every metric but `signal_per_run` is the mean over the runs of the scenario the infrastructure did not break (`pass_rate` is the share that passed, an unfinished run counting as not passed; `duration` is the wall time in milliseconds).",
            "oneOf": [
              {
                "enum": [
                  "pass_rate",
                  "cost_usd",
                  "duration",
                  "tokens",
                  "function_calls"
                ],
                "type": "string"
              },
              {
                "description": "Occurrences of one of the suggestion's patterns per run, counted by the deterministic detectors over each run's transcript.",
                "enum": [
                  "signal_per_run"
                ],
                "type": "string"
              }
            ]
          },
          "CriterionV1": {
            "additionalProperties": false,
            "properties": {
              "direction": {
                "$ref": "#/definitions/DirectionV1"
              },
              "metric": {
                "$ref": "#/definitions/CriterionMetricV1"
              },
              "min_effect": {
                "format": "double",
                "type": "number"
              },
              "min_runs": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "pattern": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "registered_at": {
                "format": "int64",
                "type": "integer"
              },
              "registered_by": {
                "type": "string"
              },
              "scenario_id": {
                "description": "The scenario whose runs are measured.",
                "type": "string"
              }
            },
            "required": [
              "direction",
              "metric",
              "min_effect",
              "min_runs",
              "registered_at",
              "registered_by",
              "scenario_id"
            ],
            "type": "object"
          },
          "DirectionV1": {
            "enum": [
              "decrease",
              "increase"
            ],
            "type": "string"
          },
          "EvidenceRunV1": {
            "additionalProperties": false,
            "description": "One run of the scenario as the E2E reported it. Runs the infrastructure broke are listed with the E2E's own words, never counted.",
            "properties": {
              "completed": {
                "description": "The E2E reported the run complete with a valid technical outcome.",
                "type": "boolean"
              },
              "completion": {
                "description": "`completion` as the E2E reported it.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "function_calls": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "run_id": {
                "type": "string"
              },
              "signals": {
                "additionalProperties": {
                  "format": "uint32",
                  "minimum": 0,
                  "type": "integer"
                },
                "description": "Occurrences of each of the row's patterns the detectors found in this run's transcript (0 is a measurement); absent when the report held no transcript.",
                "type": [
                  "object",
                  "null"
                ]
              },
              "status": {
                "description": "`passed` or `failed`, the run's task outcome.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "technical": {
                "description": "`technical` as the E2E reported it; anything but `valid` is an infrastructure failure.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "total_tokens": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "wall_time_ms": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              }
            },
            "required": [
              "completed",
              "run_id"
            ],
            "type": "object"
          },
          "EvidenceSideV1": {
            "additionalProperties": false,
            "properties": {
              "execution_id": {
                "type": "string"
              },
              "mean": {
                "description": "Mean of the criterion's metric over those runs; absent without a criterion or without such runs.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "n": {
                "description": "Measurable runs (see `EvidenceRunV1::measurable`) with a value of the criterion's metric (every measurable run while there is no criterion).",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "runs": {
                "description": "Every run of the scenario the execution holds.",
                "items": {
                  "$ref": "#/definitions/EvidenceRunV1"
                },
                "type": "array"
              }
            },
            "required": [
              "execution_id",
              "n",
              "runs"
            ],
            "type": "object"
          },
          "EvidenceV1": {
            "additionalProperties": false,
            "description": "Computed in code from the attached pair's runs, never by a model. It is what the person's verdict is shown next to.",
            "properties": {
              "baseline": {
                "$ref": "#/definitions/EvidenceSideV1"
              },
              "candidate": {
                "$ref": "#/definitions/EvidenceSideV1"
              },
              "computed_at": {
                "format": "int64",
                "type": "integer"
              },
              "computed_outcome": {
                "$ref": "#/definitions/ValidationOutcomeV1"
              },
              "reason": {
                "type": "string"
              },
              "scenario_id": {
                "type": "string"
              }
            },
            "required": [
              "baseline",
              "candidate",
              "computed_at",
              "computed_outcome",
              "reason",
              "scenario_id"
            ],
            "type": "object"
          },
          "FidelityLevelV1": {
            "oneOf": [
              {
                "enum": [
                  "approximate"
                ],
                "type": "string"
              },
              {
                "description": "The provider counts the rebuilt request like the original, up to a fixed per-request overhead measured at the turn's first step.",
                "enum": [
                  "exact"
                ],
                "type": "string"
              }
            ]
          },
          "FidelityV1": {
            "additionalProperties": false,
            "description": "How faithfully the request at the decision point was rebuilt.",
            "properties": {
              "counted_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "estimator": {
                "description": "`provider` when the provider counted, else the estimator's name.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "level": {
                "$ref": "#/definitions/FidelityLevelV1"
              },
              "off_ratio": {
                "description": "The unexplained difference over the recorded tokens.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "reasons": {
                "default": [],
                "description": "Why the rebuild is approximate; empty when exact.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "recorded_tokens": {
                "description": "Input tokens of the original step: input + cache read + cache write.",
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "required": [
              "level"
            ],
            "type": "object"
          },
          "LifecycleEventV1": {
            "additionalProperties": false,
            "description": "One change of status, with who made it.",
            "properties": {
              "at": {
                "format": "int64",
                "type": "integer"
              },
              "by": {
                "type": "string"
              },
              "note": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "status": {
                "$ref": "#/definitions/LifecycleStatusV1"
              }
            },
            "required": [
              "at",
              "by",
              "status"
            ],
            "type": "object"
          },
          "LifecycleStatusV1": {
            "description": "Where a suggestion stands. Only people move it: the monitor never does.",
            "enum": [
              "new",
              "accepted",
              "in_progress",
              "shipped",
              "rejected",
              "duplicate"
            ],
            "type": "string"
          },
          "LifecycleV1": {
            "additionalProperties": false,
            "properties": {
              "duplicate_of": {
                "description": "What it duplicates: a pull request, or another `<evaluation_id>:<index>` (`duplicate`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "history": {
                "description": "Every change of status, oldest first; empty while `new`.",
                "items": {
                  "$ref": "#/definitions/LifecycleEventV1"
                },
                "type": "array"
              },
              "pr": {
                "description": "The pull request that implements it (`in_progress`, `shipped`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "reason": {
                "description": "Why it was rejected (`rejected`).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "status": {
                "$ref": "#/definitions/LifecycleStatusV1"
              },
              "version": {
                "description": "The Harness version that shipped it, a semantic version like `1.8.43`; the recurrence query splits the analyses at it.",
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "history",
              "status"
            ],
            "type": "object"
          },
          "ReplyCallV1": {
            "additionalProperties": false,
            "description": "One function call of a reply.",
            "properties": {
              "description": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "payload": {
                "description": "The payload as JSON text, cut at 600 characters.",
                "type": "string"
              },
              "target": {
                "description": "The function called (through `agent_call` or directly).",
                "type": "string"
              }
            },
            "required": [
              "payload",
              "target"
            ],
            "type": "object"
          },
          "ReplyUsageV1": {
            "additionalProperties": false,
            "properties": {
              "cache_read_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cache_write_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          },
          "ReplyV1": {
            "additionalProperties": false,
            "description": "One reply at the decision point: a sample, or the original step read the same way.",
            "properties": {
              "calls": {
                "items": {
                  "$ref": "#/definitions/ReplyCallV1"
                },
                "type": "array"
              },
              "duration_ms": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "error": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "index": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "signal": {
                "description": "Whether the signal shows in this reply; null when Jev answered unclear, the reply failed or it was not classified yet.",
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "text": {
                "type": "string"
              },
              "thinking": {
                "description": "The opening of the reply's reasoning and of its text, 400 characters each.",
                "type": "string"
              },
              "usage": {
                "allOf": [
                  {
                    "$ref": "#/definitions/ReplyUsageV1"
                  }
                ],
                "default": {}
              }
            },
            "required": [
              "calls",
              "index",
              "text",
              "thinking"
            ],
            "type": "object"
          },
          "ReproductionChangeKindV1": {
            "oneOf": [
              {
                "description": "The context as the model saw it.",
                "enum": [
                  "none"
                ],
                "type": "string"
              },
              {
                "description": "The suggestion's own `check.change`.",
                "enum": [
                  "proposed"
                ],
                "type": "string"
              },
              {
                "description": "Edits a person wrote.",
                "enum": [
                  "custom"
                ],
                "type": "string"
              }
            ]
          },
          "ReproductionStateV1": {
            "enum": [
              "running",
              "completed",
              "failed"
            ],
            "type": "string"
          },
          "ReproductionV1": {
            "additionalProperties": false,
            "description": "A replay of the decision point: N replies sampled from the rebuilt request, with or without a change. No function ever runs.",
            "properties": {
              "by": {
                "type": "string"
              },
              "change": {
                "default": [],
                "items": {
                  "$ref": "#/definitions/ChangeEditV1"
                },
                "type": "array"
              },
              "change_kind": {
                "$ref": "#/definitions/ReproductionChangeKindV1"
              },
              "check": {
                "allOf": [
                  {
                    "$ref": "#/definitions/SuggestionCheckV1"
                  }
                ],
                "description": "The decision point and signal this replay used."
              },
              "cost_unknown_samples": {
                "default": 0,
                "description": "Samples without a reported cost, failed ones included (the provider may have billed a request that never answered): unknown, not in `cost_usd`, and never estimated.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "cost_usd": {
                "description": "Known cost of the samples; null while no reply reported one. Only a lower bound while `cost_unknown_samples` is above zero. Jev's calls are in tokens below, never priced.",
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "error": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "fidelity": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/FidelityV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "finished_at": {
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "id": {
                "type": "string"
              },
              "judge_input_tokens": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "judge_output_tokens": {
                "default": 0,
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "model": {
                "type": "string"
              },
              "original": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ReplyV1"
                  },
                  {
                    "type": "null"
                  }
                ],
                "description": "The original reply at the decision point, read the same way."
              },
              "phase": {
                "description": "`sampling` or `classifying` while running.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "provider": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "requested": {
                "description": "Replies asked for, extensions included.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "samples": {
                "default": [],
                "items": {
                  "$ref": "#/definitions/ReplyV1"
                },
                "type": "array"
              },
              "started_at": {
                "format": "int64",
                "type": "integer"
              },
              "state": {
                "$ref": "#/definitions/ReproductionStateV1"
              },
              "updated_at": {
                "format": "int64",
                "type": "integer"
              }
            },
            "required": [
              "by",
              "change_kind",
              "check",
              "id",
              "model",
              "requested",
              "started_at",
              "state",
              "updated_at"
            ],
            "type": "object"
          },
          "ResolvedCommitV1": {
            "additionalProperties": false,
            "description": "One side of a validation, resolved to the commit the E2E would build.",
            "properties": {
              "branch": {
                "description": "A remote branch that holds the commit, as `git branch -r` names it (`origin/feat`); the one named like the ref when several do.",
                "type": "string"
              },
              "commit": {
                "description": "The full commit.",
                "type": "string"
              },
              "short": {
                "description": "The first 12 characters of `commit`.",
                "type": "string"
              }
            },
            "required": [
              "branch",
              "commit",
              "short"
            ],
            "type": "object"
          },
          "SignalRuleV1": {
            "oneOf": [
              {
                "description": "The reply asks `engine::functions::info` only for contracts an earlier result in the context already supplied.",
                "enum": [
                  "contract_rediscovery"
                ],
                "type": "string"
              },
              {
                "description": "The reply repeats the last call that failed: same target, equal payload.",
                "enum": [
                  "repeated_error_call"
                ],
                "type": "string"
              }
            ]
          },
          "SignalV1": {
            "additionalProperties": false,
            "description": "How to recognize the behavior in one reply: exactly one of a rule computed in code or a yes/no question Jev answers about the reply.",
            "properties": {
              "question": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "rule": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/SignalRuleV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "type": "object"
          },
          "SuggestionCheckV1": {
            "additionalProperties": false,
            "description": "How a suggestion is checked at the step where its behavior happened.",
            "properties": {
              "change": {
                "default": [],
                "description": "The proposed change as edits of what the model saw before that step; empty when it cannot be expressed as text.",
                "items": {
                  "$ref": "#/definitions/ChangeEditV1"
                },
                "type": "array"
              },
              "decision_point": {
                "description": "The assistant entry where the behavior happened (`…_<step>_assistant`), one of the suggestion's `evidence` entries.",
                "type": "string"
              },
              "signal": {
                "$ref": "#/definitions/SignalV1"
              }
            },
            "required": [
              "decision_point",
              "signal"
            ],
            "type": "object"
          },
          "SuggestionReviewV1": {
            "additionalProperties": false,
            "description": "Everything people decided about one suggestion, in the `eval_suggestion` state scope. The title, component and patterns are copied from the suggestion so the row outlives the analysis's retention.",
            "properties": {
              "criterion": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/CriterionV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "evaluation_id": {
                "type": "string"
              },
              "evidence": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/EvidenceV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "first_run_at": {
                "description": "When the first execution this worker started for the suggestion began, kept when a new start replaces `run`: results of it may have been seen.",
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "harness_component": {
                "type": "string"
              },
              "lifecycle": {
                "$ref": "#/definitions/LifecycleV1"
              },
              "patterns": {
                "description": "`<rule_id>:<target>` of the diagnostics the suggestion's evidence cites: what a validation counts and a release is judged by.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "reproductions": {
                "description": "Replays of the decision point, oldest first: the base and the tested changes.",
                "items": {
                  "$ref": "#/definitions/ReproductionV1"
                },
                "type": "array"
              },
              "run": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ValidationRunV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "scenario_id": {
                "description": "The scenario the suggestion's validation plan names.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "suggestion_index": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "title": {
                "type": "string"
              },
              "verdict": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/VerdictV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "required": [
              "evaluation_id",
              "harness_component",
              "lifecycle",
              "patterns",
              "suggestion_index",
              "title"
            ],
            "type": "object"
          },
          "ValidationOutcomeV1": {
            "oneOf": [
              {
                "description": "The candidate moved the metric by at least `min_effect` in the wanted direction, with `min_runs` completed runs each side.",
                "enum": [
                  "validated_improvement"
                ],
                "type": "string"
              },
              {
                "description": "The metric moved less than `min_effect`.",
                "enum": [
                  "no_improvement"
                ],
                "type": "string"
              },
              {
                "description": "The metric moved by at least `min_effect` the wrong way.",
                "enum": [
                  "regression"
                ],
                "type": "string"
              },
              {
                "description": "Too few completed runs, missing data or no criterion.",
                "enum": [
                  "inconclusive"
                ],
                "type": "string"
              }
            ]
          },
          "ValidationResolutionV1": {
            "additionalProperties": false,
            "description": "What `eval::start-validation` answers to a `dry_run`: the commits both refs resolve to, every start check passed.",
            "properties": {
              "baseline": {
                "$ref": "#/definitions/ResolvedCommitV1"
              },
              "candidate": {
                "$ref": "#/definitions/ResolvedCommitV1"
              },
              "warnings": {
                "description": "What a start would go ahead with but a person should know.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              }
            },
            "required": [
              "baseline",
              "candidate",
              "warnings"
            ],
            "type": "object"
          },
          "ValidationRunStateV1": {
            "oneOf": [
              {
                "enum": [
                  "failed"
                ],
                "type": "string"
              },
              {
                "description": "Recorded before the executions are requested.",
                "enum": [
                  "starting"
                ],
                "type": "string"
              },
              {
                "description": "Both executions were requested; the sweep waits for them.",
                "enum": [
                  "running"
                ],
                "type": "string"
              },
              {
                "description": "Both ended; the pair is about to be attached.",
                "enum": [
                  "finished"
                ],
                "type": "string"
              },
              {
                "description": "The pair is attached to the analysis and its evidence computed.",
                "enum": [
                  "attached"
                ],
                "type": "string"
              }
            ]
          },
          "ValidationRunV1": {
            "additionalProperties": false,
            "description": "The baseline and candidate E2E executions this worker started for a suggestion (`eval::start-validation`).",
            "properties": {
              "baseline_commit": {
                "description": "The full commits the Harness was built from.",
                "type": "string"
              },
              "baseline_execution_id": {
                "description": "Absent until the E2E accepted the execution.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "candidate_commit": {
                "type": "string"
              },
              "candidate_execution_id": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "error": {
                "description": "Why it failed, or why attaching is being retried; starts with a stable code (`e2e_unavailable:`, `e2e_busy:`…).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "finished_at": {
                "description": "When both executions were seen ended; the sweep stops trying to attach the pair some time after.",
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "model": {
                "type": "string"
              },
              "provider": {
                "type": "string"
              },
              "runs": {
                "description": "Runs requested for each side.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "scenario_id": {
                "type": "string"
              },
              "started_at": {
                "format": "int64",
                "type": "integer"
              },
              "state": {
                "$ref": "#/definitions/ValidationRunStateV1"
              }
            },
            "required": [
              "baseline_commit",
              "candidate_commit",
              "model",
              "provider",
              "runs",
              "scenario_id",
              "started_at",
              "state"
            ],
            "type": "object"
          },
          "VerdictV1": {
            "additionalProperties": false,
            "description": "The person's judgment, kept with the criterion it was made against.",
            "properties": {
              "at": {
                "format": "int64",
                "type": "integer"
              },
              "by": {
                "type": "string"
              },
              "controls_checked": {
                "description": "The non-regression controls of the plan that were checked.",
                "items": {
                  "type": "string"
                },
                "type": "array"
              },
              "criterion_snapshot": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/CriterionV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "outcome": {
                "$ref": "#/definitions/ValidationOutcomeV1"
              },
              "rationale": {
                "type": "string"
              }
            },
            "required": [
              "at",
              "by",
              "controls_checked",
              "outcome",
              "rationale"
            ],
            "type": "object"
          }
        },
        "description": "The suggestion's row once the executions started, or, for a `dry_run`, the resolution.",
        "title": "StartValidationResponseV1"
      }
    },
    {
      "description": "Read one analysis record (status, counters, failure stage) or null.",
      "metadata": {},
      "name": "eval::status",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "evaluation_id": {
            "type": "string"
          }
        },
        "required": [
          "evaluation_id"
        ],
        "title": "EvaluationIdRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "anyOf": [
          {
            "$ref": "#/definitions/AnalysisRecordV1"
          },
          {
            "type": "null"
          }
        ],
        "definitions": {
          "AnalysisCountersV1": {
            "additionalProperties": false,
            "properties": {
              "diagnostics": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "entries": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "rejected_suggestions": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "sessions": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "suggestions": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "validations": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "diagnostics",
              "entries",
              "rejected_suggestions",
              "sessions",
              "suggestions",
              "validations"
            ],
            "type": "object"
          },
          "AnalysisOriginV1": {
            "enum": [
              "automatic",
              "manual"
            ],
            "type": "string"
          },
          "AnalysisRecordV1": {
            "additionalProperties": false,
            "description": "The compact analysis record listed by `eval::list`; evidence and model outputs live separately in the analysis assets.",
            "properties": {
              "analyst": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/AnalystRefV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "code_root": {
                "description": "Frozen at admission with the model: the directory the investigation reads and runs in, absent when code access was off.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "completed_at": {
                "format": "int64",
                "type": [
                  "integer",
                  "null"
                ]
              },
              "config_revision": {
                "type": "string"
              },
              "counters": {
                "$ref": "#/definitions/AnalysisCountersV1"
              },
              "coverage": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/CoverageLevelV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "created_at": {
                "format": "int64",
                "type": "integer"
              },
              "criteria_version": {
                "type": "string"
              },
              "deadline": {
                "format": "int64",
                "type": "integer"
              },
              "evaluation_id": {
                "type": "string"
              },
              "failure": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/FailureV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "harness_version": {
                "description": "The Harness version the engine reported when a live event admitted the analysis (the turn just ended on it); absent on a manual analysis, which may be of an older session, and when the engine did not say.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "judge_call": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/JudgeCallV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "model": {
                "allOf": [
                  {
                    "$ref": "#/definitions/MonitorModelV1"
                  }
                ],
                "description": "Frozen at admission: later configuration changes never alter it."
              },
              "observation_key": {
                "type": "string"
              },
              "observe_since": {
                "description": "Earlier root turns join the window only if they started after this instant: the monitor was observing then. Turns from before the monitor was enabled are never reported as new.",
                "format": "int64",
                "type": "integer"
              },
              "origin": {
                "$ref": "#/definitions/AnalysisOriginV1"
              },
              "pending_reason": {
                "description": "Why collection is still waiting (descendants running, metrics incomplete).",
                "type": [
                  "string",
                  "null"
                ]
              },
              "routing": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/RoutingV1"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "rules_version": {
                "type": "string"
              },
              "schema_version": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "session_id": {
                "type": "string"
              },
              "signals": {
                "additionalProperties": {
                  "format": "uint32",
                  "minimum": 0,
                  "type": "integer"
                },
                "description": "Occurrences of each deterministic pattern the collection found, keyed `<rule_id>:<target>` (a target may itself contain colons). Empty both when nothing was found and before collection (`coverage` is set once collection read the evidence).",
                "type": "object"
              },
              "source_title": {
                "description": "The session's title, once collection has read it.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "stages": {
                "description": "When each status began, oldest first.",
                "items": {
                  "$ref": "#/definitions/StageTimeV1"
                },
                "type": "array"
              },
              "status": {
                "$ref": "#/definitions/EvalStatusV1"
              },
              "step": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "supersedes": {
                "description": "The earlier analysis of the same turn this reanalysis was requested over (same `observation_key`); absent on a first analysis.",
                "type": [
                  "string",
                  "null"
                ]
              },
              "turn_id": {
                "type": "string"
              },
              "updated_at": {
                "format": "int64",
                "type": "integer"
              },
              "usage": {
                "allOf": [
                  {
                    "$ref": "#/definitions/MonitorUsageV1"
                  }
                ],
                "description": "The monitor's own consumption, apart from the observed task."
              }
            },
            "required": [
              "config_revision",
              "counters",
              "created_at",
              "criteria_version",
              "deadline",
              "evaluation_id",
              "model",
              "observation_key",
              "observe_since",
              "origin",
              "rules_version",
              "schema_version",
              "session_id",
              "stages",
              "status",
              "step",
              "turn_id",
              "updated_at",
              "usage"
            ],
            "type": "object"
          },
          "AnalystRefV1": {
            "additionalProperties": false,
            "properties": {
              "sent_at": {
                "format": "int64",
                "type": "integer"
              },
              "session_id": {
                "type": "string"
              },
              "turn_id": {
                "type": [
                  "string",
                  "null"
                ]
              }
            },
            "required": [
              "sent_at",
              "session_id"
            ],
            "type": "object"
          },
          "CoverageLevelV1": {
            "oneOf": [
              {
                "description": "Every in-scope transcript was read and the model context holds every diagnostic's evidence.",
                "enum": [
                  "complete"
                ],
                "type": "string"
              },
              {
                "description": "Previews omit or shorten entries; the full transcripts were read.",
                "enum": [
                  "partial"
                ],
                "type": "string"
              },
              {
                "description": "Required evidence did not fit; no conclusion of healthy behavior.",
                "enum": [
                  "insufficient"
                ],
                "type": "string"
              }
            ]
          },
          "EvalStatusV1": {
            "enum": [
              "queued",
              "collecting",
              "judging",
              "investigating",
              "completed",
              "failed",
              "cancelled"
            ],
            "type": "string"
          },
          "FailureV1": {
            "additionalProperties": false,
            "description": "The stage that failed and why. `stage` is the status the analysis was in.",
            "properties": {
              "code": {
                "type": "string"
              },
              "message": {
                "type": "string"
              },
              "stage": {
                "$ref": "#/definitions/EvalStatusV1"
              }
            },
            "required": [
              "code",
              "message",
              "stage"
            ],
            "type": "object"
          },
          "JudgeCallV1": {
            "additionalProperties": false,
            "description": "A Jev call started by this analysis. Persisted before the call so a restart can tell a lost answer from one never requested.",
            "properties": {
              "deadline": {
                "format": "int64",
                "type": "integer"
              },
              "request_id": {
                "type": "string"
              },
              "started_at": {
                "format": "int64",
                "type": "integer"
              }
            },
            "required": [
              "deadline",
              "request_id",
              "started_at"
            ],
            "type": "object"
          },
          "MonitorModelV1": {
            "additionalProperties": false,
            "description": "The Harness model the user chose for investigations. Credentials stay in the provider worker; this request never carries a key.",
            "properties": {
              "model": {
                "type": "string"
              },
              "provider": {
                "type": "string"
              },
              "provider_options": {
                "additionalProperties": true,
                "description": "Provider-native options, namespaced by the selected provider id.",
                "type": [
                  "object",
                  "null"
                ]
              },
              "thinking_level": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ThinkingLevel"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "required": [
              "model",
              "provider"
            ],
            "type": "object"
          },
          "MonitorUsageV1": {
            "additionalProperties": false,
            "description": "Known usage only: absent counters are unknown, never zero.",
            "properties": {
              "judge_calls": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "judge_input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "judge_output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "judge_usage_complete": {
                "description": "False when Jev reported incomplete usage or failed.",
                "type": "boolean"
              },
              "llm_cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "llm_input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "llm_output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "required": [
              "judge_calls",
              "judge_input_tokens",
              "judge_output_tokens",
              "judge_usage_complete"
            ],
            "type": "object"
          },
          "RoutingReasonV1": {
            "oneOf": [
              {
                "enum": [
                  "needs_investigation"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced; kept so stored records still deserialize.",
                "enum": [
                  "diagnostics"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "insufficient_evidence"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "low_confidence"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "coverage_insufficient"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "audit_sample"
                ],
                "type": "string"
              },
              {
                "description": "Legacy: no longer produced.",
                "enum": [
                  "manual_request"
                ],
                "type": "string"
              }
            ]
          },
          "RoutingV1": {
            "additionalProperties": false,
            "properties": {
              "investigate": {
                "type": "boolean"
              },
              "reasons": {
                "items": {
                  "$ref": "#/definitions/RoutingReasonV1"
                },
                "type": "array"
              }
            },
            "required": [
              "investigate",
              "reasons"
            ],
            "type": "object"
          },
          "StageTimeV1": {
            "additionalProperties": false,
            "properties": {
              "at": {
                "format": "int64",
                "type": "integer"
              },
              "status": {
                "$ref": "#/definitions/EvalStatusV1"
              }
            },
            "required": [
              "at",
              "status"
            ],
            "type": "object"
          },
          "ThinkingLevel": {
            "oneOf": [
              {
                "enum": [
                  "minimal",
                  "low",
                  "medium",
                  "high",
                  "xhigh"
                ],
                "type": "string"
              },
              {
                "description": "No reasoning at all; honoured only where the model advertises `supports_thinking_off` (elsewhere the provider falls back to its own lowest setting, or omits the parameter, and warns).",
                "enum": [
                  "off"
                ],
                "type": "string"
              }
            ]
          }
        },
        "title": "Nullable_AnalysisRecordV1"
      }
    },
    {
      "description": "Internal durable analysis step.",
      "metadata": {
        "internal": true,
        "trace_hidden": true
      },
      "name": "eval::step",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "evaluation_id": {
            "type": "string"
          },
          "step": {
            "format": "uint64",
            "minimum": 0,
            "type": "integer"
          }
        },
        "required": [
          "evaluation_id",
          "step"
        ],
        "title": "StepRequestV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "EvalStatusV1": {
            "enum": [
              "queued",
              "collecting",
              "judging",
              "investigating",
              "completed",
              "failed",
              "cancelled"
            ],
            "type": "string"
          }
        },
        "properties": {
          "skipped": {
            "type": "boolean"
          },
          "status": {
            "$ref": "#/definitions/EvalStatusV1"
          }
        },
        "required": [
          "skipped",
          "status"
        ],
        "title": "StepResponseV1",
        "type": "object"
      }
    },
    {
      "description": "Internal recovery, deadline and retention sweep.",
      "metadata": {
        "internal": true,
        "trace_hidden": true
      },
      "name": "eval::sweep",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "scheduled_at": {
            "default": null,
            "format": "int64",
            "type": [
              "integer",
              "null"
            ]
          },
          "scheduled_time": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "SweepEventV1",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "properties": {
          "expired": {
            "format": "uint64",
            "minimum": 0,
            "type": "integer"
          },
          "reconciled": {
            "format": "uint64",
            "minimum": 0,
            "type": "integer"
          },
          "requeued": {
            "format": "uint64",
            "minimum": 0,
            "type": "integer"
          },
          "retained_deleted": {
            "format": "uint64",
            "minimum": 0,
            "type": "integer"
          }
        },
        "required": [
          "expired",
          "reconciled",
          "requeued",
          "retained_deleted"
        ],
        "title": "SweepResponseV1",
        "type": "object"
      }
    },
    {
      "description": "Serve the eval worker's injected console UI assets (the target of its ade::ui::asset observers; the console calls it through each asset's observer with {path}).",
      "metadata": {
        "internal": true
      },
      "name": "eval::ui-content",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Input of the content function: the console asks for one asset by path.",
        "properties": {
          "path": {
            "description": "The asset path from the observer's args (e.g. `state/page.js`).",
            "type": "string"
          }
        },
        "required": [
          "path"
        ],
        "title": "UiContentInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Output of the content function.",
        "properties": {
          "content": {
            "description": "The asset source, verbatim.",
            "type": "string"
          },
          "content_type": {
            "description": "MIME type the console should serve the asset with.",
            "type": "string"
          }
        },
        "required": [
          "content",
          "content_type"
        ],
        "title": "UiContentResult",
        "type": "object"
      }
    }
  ]
}
```
