# judge-semif

> SemIf provider for the judge hub (judge-semif::evaluate, judge-semif::models::list, judge-semif::cancel) — a frozen open LLM (Qwen3.5-4B GGUF) downloaded from the Hugging Face Hub and run in-process through llama.cpp, answering from the next-token logits of the option letters; picks Metal (macOS), Vulkan (Linux, when a driver is present) or the CPU automatically; no API key or external service.

| field | value |
|-------|-------|
| version | 0.2.0 |
| type | binary |
| license | Apache-2.0 |
| repo | https://github.com/iii-hq/workers |
| supported_targets | x86_64-apple-darwin, aarch64-apple-darwin, x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu |
| author | iii |

## installation

```sh
iii trigger compose::add worker=judge-semif@0.2.0
```

## configuration

```yaml
- context_tokens: 16384
  max_request_bytes: 8388608
  max_timeout_ms: 300000
  model: qwen3.5-4b
  parallel_questions: 8
  threads: 8
```

## dependencies

- `configuration` @ `latest`

## readme

# judge-semif

[SemIf](https://github.com/TheoLeeCJ/SemIf) provider for the [`judge`](../judge/)
hub, running **inside the worker**: no API key, no Python, no llama-server.
SemIf reads runtime-defined decisions from a frozen open LLM: one forward pass,
a softmax over the next-token logits of the option letters `A`–`P`, no decoding.
Its published Qwen3.5-4B baseline agrees with Jev's own references on 0.845 of
TypeSafe's public subset (Jev 0.883). This worker runs the same prompt through
llama.cpp (the [`llama-cpp-2`](https://crates.io/crates/llama-cpp-2) crate) on
the CPU, on Metal (macOS) or on Vulkan (Linux x86_64: AMD, NVIDIA and Intel
GPUs), picked automatically at start.

## Hardware selection

- **macOS**: Metal is linked in; Apple Silicon GPUs run the model.
- **Linux x86_64**: llama.cpp's backends are modules loaded at start from the
  binary's directory (`GGML_BACKEND_DL`). The Vulkan module loads wherever a
  Vulkan loader and driver exist (`libvulkan.so.1`); without them it is
  skipped and the CPU runs the model, so the same package works on GPU
  desktops, servers and containers. The CPU module is picked for the host
  (AVX2, AVX-512, AMX variants). The package ships `libllama`, `libggml`,
  `libggml-base`, the CPU variants and the Vulkan module (≈77 MB) beside the
  binary, which finds them through its `$ORIGIN` runpath.
- **Linux aarch64**: CPU, statically linked.

`gpu_layers: 0` keeps the model on the CPU even when a GPU is present. The
chosen device is logged at start as `selected inference device`.

## Install

```bash
iii trigger compose::add worker=judge-semif
```

The model loads only while SemIf is the judge hub's **default provider**
(`provider: semif` under **Settings → Workers → judge**): the worker follows
the hub's configuration and, when another provider becomes the default,
unregisters its functions and releases the model (VRAM included); it loads
again when SemIf is selected. The first load downloads the pinned GGUF
(`qwen3.5-4b`: `bartowski/Qwen_Qwen3.5-4B-GGUF` Q4_K_M, 3.0 GB) into the
hf-hub cache (`$HF_HOME`, default `~/.cache/huggingface`). The functions
register only once the model answers; until then, and while another provider
is the default, the hub reports `provider_unavailable`, also for calls that
name `"provider": "semif"`. To keep it loaded while another provider is the
default, turn on **Keep every local provider loaded** (`preload_all`) in the
judge settings. A hub build that does not expose
`judge::configuration-id` leaves the model loaded from the start.
Air-gapped installs set `III_SEMIF_GGUF` to a local GGUF file.

Make it the hub's default, then call the hub:

```bash
iii trigger judge::evaluate --timeout-ms 65000 --json '{
  "timeout_ms": 60000, "provider": "semif",
  "evaluations": [{
    "id": "ticket-4411",
    "state": {"subject": "Duplicate charge on invoice #4411",
              "body": "We were billed twice for March. Refund the duplicate today or we cancel."},
    "questions": {
      "department": {"type": "choice", "criteria": {"billing": "invoices, payments, refunds", "technical": "bugs, outages"}},
      "urgency": {"type": "score", "criteria": ["not urgent", "soon", "critical deadline or blocking issue"]},
      "churn_risk": {"type": "noul", "instructions": "Does the user threaten to cancel or leave?"}
    }
  }]
}'
```

## How questions map to SemIf

Each question becomes one SemIf decision: `{"evidence": state, "criterion":
instructions, "options": [...]}` in SemIf's frozen system prompt and Qwen chat
template (thinking disabled), tokenized by the GGUF vocabulary. Token ids match
SemIf's reference tokenizer (`tests/prompt.rs`), and on the CPU the option
probabilities match SemIf's own llama.cpp backend to four decimals.

| Contract | SemIf options | Answer |
|---|---|---|
| `noul` | A = `criteria.true` (or "Yes, the statement holds."), B = `criteria.false` | `noul` = P(A) |
| `choice` | one per criterion, `key: description` | argmax, probabilities, confidence |
| `score` | `level i: <level>` | Σ i·p_i, probabilities, confidence, legend |

`confidence` is TypeSafe's (`judge_contract::confidence`, shared by the local
providers): `(n·p_max − 1) / (n − 1)` for a choice, 1 − the expected distance
from the likeliest level over the levels' mean distance from the middle for a
score. SemIf's probabilities are conditional on the offered options and
uncalibrated. At most
16 options per question. A prompt longer than `context_tokens` answers
`payload_too_large`: SemIf never truncates evidence.

Every question of one evaluation shares its state, and SemIf puts the evidence
first. The worker prefills the prompts' longest common token run once,
snapshots it, restores it into up to `parallel_questions` sequences and decodes
their suffixes together in one batch (SemIf's parallel suffixes). The prefix is
found on tokens, not re-tokenized text: a state ending in `{}` merges with the
following `}` two tokens back, which SemIf's own `_state_prefix` rejects.
`usage.input_tokens` counts the decoded tokens (the shared prefix once).

## Speed (Qwen3.5-4B Q4_K_M, i9-14900K, RX 6900 XT)

Per request, 16 questions about one state; `parallel_questions` 8 unless noted:

| | incident (541-token state) | directory block (4.6k-token state, 64 functions) |
|---|---|---|
| CPU, 8 threads | 3.8 s (6.9 s with parallel 1) | 6.6 s |
| Vulkan | 0.79 s (1.18 s with parallel 1) | 3.4 s |

Before the token-level prefix, the directory block decoded every question in
full: 75k tokens and 35.8 s on Vulkan, against 6k tokens and 3.4 s now.
Parallel suffixes pay off on short states; long states are dominated by the
one-time prefill. On the same card llama.cpp's ROCm backend measured within
10% of Vulkan either way, so the worker ships Vulkan alone. A single short
decision takes 67–140 ms on the GPU and 1.1–1.4 s on the CPU.

GPU, reuse and batching change probabilities by up to 0.02 against fresh CPU
scoring, as SemIf documents for its own fast paths.

## Configuration

**Settings → Workers → judge-semif**:

| Field | Default | Applied |
|---|---|---|
| `model` | `qwen3.5-4b` | next start |
| `threads` | min(8, logical cores) | next start |
| `gpu_layers` | all layers when a GPU backend and device exist | next start (`0` = CPU) |
| `context_tokens` | 16384 | next start |
| `parallel_questions` | 8 | next start (`1` = one question per decode) |
| `max_request_bytes` | 8388608 | new calls |
| `max_timeout_ms` | 300000 | new calls |

## Building

llama.cpp is compiled from source through `crates/llama-runtime`: `cmake`, a C++ compiler and `libclang`
(for bindgen) are required; if libclang lives outside the default search path
set `LIBCLANG_PATH` (and `BINDGEN_EXTRA_CLANG_ARGS=-I<clang>/include` when its
builtin headers are not found). Linux x86_64 builds also need the Vulkan
loader headers, the SPIR-V headers and `glslc` (Ubuntu: `libvulkan-dev
spirv-headers glslc`) to compile the Vulkan module (only the module links
`libvulkan`, the binary does not). The build copies the modules and libraries
beside the binary, so `target/release` has the published layout; the release
catalog ships them as the artifact's `companions`. Windows is not published
yet.

For the full API, read the hub's [reference](../judge/reference.md).

## api reference

```json
{
  "functions": [
    {
      "description": "Returns this worker instance's effective configuration entry ID, without its value.",
      "metadata": {
        "internal": true
      },
      "name": "judge-semif::configuration-id",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Empty user request. Ignore transport metadata such as `_caller_worker_id`, which the engine adds to routed calls; no payload field selects the entry.",
        "title": "ConfigurationIdentityRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Only the entry ID is exposed; configuration values and secrets stay in the service.",
        "properties": {
          "id": {
            "type": "string"
          }
        },
        "required": [
          "id"
        ],
        "title": "ConfigurationIdentityResponse",
        "type": "object"
      }
    },
    {
      "description": "Internal: reload SemIf configuration from authoritative storage.",
      "metadata": {
        "internal": true
      },
      "name": "judge-semif::on-config-change",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Trigger payload for `<worker>::on-config-change`. Advisory only: handlers re-fetch the authoritative value and ignore it, so a direct call can never inject config.",
        "properties": {
          "id": {
            "default": null,
            "description": "Configuration id that changed.",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "OnConfigChangeEvent",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Ack returned by the internal `<worker>::on-config-change` handler.",
        "properties": {
          "ok": {
            "type": "boolean"
          }
        },
        "required": [
          "ok"
        ],
        "title": "OnConfigChangeResponse",
        "type": "object"
      }
    },
    {
      "description": "Serve the judge-semif worker's injected console UI assets (content function for its console:script / console:style triggers).",
      "metadata": {
        "internal": true
      },
      "name": "judge-semif::ui-content",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Input of the content function: the console asks for one asset by path.",
        "properties": {
          "path": {
            "description": "The asset path from the trigger config (e.g. `state/page.js`).",
            "type": "string"
          }
        },
        "required": [
          "path"
        ],
        "title": "UiContentInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Output of the content function.",
        "properties": {
          "content": {
            "description": "The asset source, verbatim.",
            "type": "string"
          },
          "content_type": {
            "description": "MIME type the console should serve the asset with.",
            "type": "string"
          }
        },
        "required": [
          "content",
          "content_type"
        ],
        "title": "UiContentResult",
        "type": "object"
      }
    }
  ],
  "triggers": []
}
```