Skip to content
Sign in

fp

v0.3.2

Lodash-style value transforms (fp::get/pick/take/…) and fp::pipe — worker-side pipelines that move big values function→function without routing them through the model.

license: Apache-2.0Used in 2 kitsfpfunctionallodashpipelinetransform

Install

bash
iii trigger compose::add worker=fp

Run in your project with the iii engine running. Exact versions are immutable and digest-pinned.

fp

Lodash-style value transforms and fp::pipe — worker-side pipelines that move big values function→function over the iii bus without routing them through the model.

Install

iii call compose::add worker=fp

The worker is a deploy: binary Rust worker with no configuration.

Why

An agent that fetches a document and re-types it into the next call's arguments burns its context window and stalls the provider stream mid-call. fp::pipe runs the whole move in one call — each step calls a function and its result lands in the next step's payload — so the value flows worker→worker and the chat only ever sees per-step sizes and a preview.

While connected (and unless turned off) the worker also injects a usage section into the agent system prompt: fp::inject-guidance observes the harness's harness::hook::on-pre-generate with args { on_error: "fail_open", inject_prompt: <the guidance> }. The knob lives in the builtin configuration worker under the fp entry: inject_guidance is ON by default; turn it off in global Settings (or via configuration::set) to save the ~750 tokens per generation (the harness's # Granted functions catalog still advertises the fp::* ids) — it hot-applies, the worker observes the hook or disposes the observer on the spot, no restart. The Console renders the entry with its explicit worker settings form. The guidance stays presence-gated either way (no fp worker, no prompt text). When enabled before the harness is up, the observer stays pending and becomes active when the harness registers the hook function. The worker also observes configuration::on-updated for its own entry (fp::on-config-change).

Functions

function lodash request
fp::pipe — { through: [{function, payload?, into?}], preview_chars? }
fp::batch — { calls: [{id?, function, payload?, namespace?, timeout_ms?}], concurrency?, stop_on_error?, timeout_ms?, max_entry_bytes?, max_calls? } — independent calls, every answer in place (experimental)
fp::get _.get { value, path } — JSON pointer; a miss names the available keys
fp::pick _.pick { value, paths } — subset an object by top-level keys
fp::omit _.omit { value, paths } — drop top-level keys
fp::take _.take { value, n } — first n array elements / string chars
fp::drop _.drop { value, n } — skip the first n
fp::map _.map { value, path } — pluck a pointer from each element; misses → null
fp::filter _.filter { value, matches } — keep elements matching a partial object
fp::split _.split { value, separator }
fp::join _.join { value, separator? } (default ,)
fp::uniq _.uniq { value } — dedupe, first occurrences win
fp::size _.size { value } — array length / string chars / object key count
fp::compact _.compact { value } — remove null elements (0/false/"" are kept)
fp::nth _.nth { value, n } — element at index; negative counts from the end
fp::getOr fp.getOr { value, path, default } — pointer value, or default on a miss
fp::flatten _.flatten { value } — unnest one level
fp::sortBy _.sortBy { value, path } — stable ascending sort by a plucked pointer ("" = the element itself)
fp::reverse _.reverse { value } — reversed copy (immutable, fp-style)
fp::sum _.sum / _.sumBy { value, path? } — total; path plucks the addend from each element. Empty array totals 0
fp::mean _.mean / _.meanBy { value, path? } — arithmetic mean; an empty array errors
fp::min _.min / _.minBy { value, path? } — smallest NUMBER (not the element holding it); an empty array errors
fp::max _.max / _.maxBy { value, path? } — largest NUMBER (not the element holding it); an empty array errors
fp::groupBy _.groupBy { value, path } — { key: [elements] } bucketed by a plucked key ("" = the element itself)
fp::countBy _.countBy { value, path } — { key: count } with the same key rules
fp::when — { value, path?, op, to? } — { passed, value }; as a pipe step a failing comparison stops the pipe (short_circuited: true)
fp::condition — { event, condition_config: { path?, op, to?, negate? } } — { decision: "allow" | "skip", reason }: the fp::when comparison as a guard in a harness observer's conditions (harness::observe)

The reductions are the worker's arithmetic: all-integer inputs fold to an integer (so the result compares cleanly in a fp::when guard), a non-numeric element errors instead of being skipped — a skipped row is a total that silently covers less than the caller counted — and min/max return the number rather than lodash's …By element so a guard can compare it directly.

groupBy/countBy add the per-key axis the reductions lack: countBy gives counts in one step, and groupBy's buckets each feed fp::sum for a per-key total. Group keys must be a string, number, or boolean — lodash coerces a null or object key to "null"/"[object Object]", silently merging distinct groups into one bucket, so those error here instead.

Transforms take their input at value and return { value }. They deviate from lodash where silence would thread garbage through a pipe: a type mismatch is an error (not a silent {}); a map path matching NO element errors naming what was available (pointers pluck stored fields — no computed properties like /length; fp::size counts); compact removes only null (lodash's full-falsey removal would eat legitimate 0/false/""); nth out of bounds and size on a non-collection error instead of returning undefined/0; sortBy requires every element to carry the key with one comparable type. Sporadic map misses still become null, like lodash — fp::compact drops them.

The pipe

fp::pipe { through: [
  { function: "browser::fetch",
    payload:  { url: "https://…", format: "markdown", main_content_only: true } },
  { function: "fp::get",  payload: { path: "/content" } },
  { function: "fp::take", payload: { n: 20000 } },
  { function: "state::set",   payload: { scope: "research", key: "article" } }
]}
  • Step N's result lands in step N+1's payload at into (a JSON pointer, default /value — which is exactly where the transforms and state::set read their input, so most pipes need no into at all).
  • A transform step threads the transformed value itself — the { value } wrapper in the function table is only its direct-call response shape, so there is never a {value} layer to unwrap between steps.
  • The FIRST step receives no threaded value: start with a producing function (a fetch, a state::get) or seed a leading transform via payload.value. An unseeded leading transform fails validation before anything runs.
  • fp::* transform steps run inline in this worker; every other step is one bus call with a 120 s budget. The whole pipe must also fit the caller's own dispatch timeout on fp::pipe.
  • 1–12 steps. The response is receipts only: { steps: [{function, chars}], value_preview } (preview_chars sizes the preview, default 400, capped at 8000 — the receipt must never become the bulk channel it replaces).
  • A failing step stops the pipe; the error carries the completed-step trail.

The batch (experimental)

fp::batch { calls: [
  { id: "info",   function: "engine::workers::info", payload: { name: "todo" } },
  { id: "http",   function: "http::status" },
  { id: "logs",   function: "compose::logs", payload: { container: "todo", tail: 40 } }
]}
  • Independent calls, nothing threaded. They run at once, or concurrency at a time; stop_on_error: true runs them in order and stops after the first error (the rest answer SKIPPED).
  • One result per call, in order: { id, function, status: "ok", result, chars, truncated? } or { id, function, status: "error", code, message }. id defaults to the position. ok is true only when every call answered.
  • Defaults: 10 s per call (timeout_ms, per call or per request), 16 KB per result (max_entry_bytes; a bigger result comes back as the first bytes of its JSON text with truncated: true and the full chars), 12 calls per request (max_calls; more still runs, with a warning that the response may be too large to deliver; hard ceiling 64).
  • Same refusals as pipe steps, plus no nested batch and no pipe as a call. A batch is not a pipe step, and a pipe is not a batch call.
  • Every refusal lists all the problems of the request, not only the first; fix them all in one resend.
  • completed and failed count the outcomes; ok is true only when failed is 0. Same approval surface: the batch runs with this worker's authority.

Boundaries

  • Steps run with THIS worker's authority, not the calling agent's per-step dispatch policy — the fp::pipe call itself is the policy/approval surface (an approver sees the full step list). For that reason the pipe is not agent-callable without approval by default; the pure transforms are. See iii-permissions.yaml.
  • shell::*/coder::* steps run under the harness-forwarded filesystem scope: the harness stamps the trusted fs_scope onto the fp::pipe call itself (harness/src/filesystem_scope.rs) and fp re-stamps it onto each scoped step as the last write before dispatch, overwriting anything authored or threaded at /fs_scope. Without a stamp (no session working directory, or a non-harness caller — cron, worker-to-worker) scoped steps are refused, fail-closed. A path-access rejection inside a pipe just fails the step; only a direct call offers the access-grant ladder.
  • Statically refused as steps: fp::batch and fp::pipe (no nesting), engine::observers::dispose (observer control; agents observe through harness::observe), nested pipes, and every class the agent policy hard-denies — session writes and session::store::*, approval::*, configuration::*/oauth::* (credentials), router::*/provider::* (model spend), harness::*/run::* (turn control), workflow::start (the harness hook stamps the caller's session and folder on a direct call; workflow::status/stop are allowed), stream::* (the deprecated iii-stream surface) and bus internals, *::on-config-change — because steps run with worker authority and must not ride past those denies. Session READS (session::get/list/messages*/get-message/ get-attachment/list-attachments) are allowed, like database::query: the policy only gates them behind approval, and the pipe call is that surface — pulling an attachment out of the session and threading it on is exactly what a pipe is for. state::* is the deliberate exception on the write side: persisting the threaded value is the pipe's purpose. state::claim-namespace and provider-openai-codex::state::* remain operator-only and cannot be pipe steps, even when the pipe itself is approved.

Search the registry

Find workers, kits and executables, or jump to a page.