Skip to main content
Compose runs a group of workers as one project. The daemon reads a worker-compose.yaml, is able to resolve, start, and stop each worker in dependency order, handles engine registration tasks, and supervises running workers.

The daemon

The daemon is itself a worker. It registers under the name compose and exposes the compose::* functions, so every project operation is a standard trigger. Bare iii compose is the command to start the compose worker/daemon. It reads worker-compose.yaml in the working directory and starts only the compose worker. iii compose --up is provided as a convenience. It starts the compose worker, the iii engine, and the workers specified in worker-compose.yaml. It is approximately the equivalent of running iii, iii compose, and iii trigger compose::up as separate commands. iii compose --up is recommended for ease of development while more granular control and independent operation of iii and iii compose daemons is suggested for production.

Starting a project with the daemon

Ctrl^C, SIGINT and SIGTERM all gracefully stop the daemon, every worker run by the daemon, and the iii engine if compose was started with --up. When --up has started the engine, every project and worker stops before the engine process is stopped. compose::stop is the function equivalent of this operation. compose::* functions as documented below are the intended way to manage a running compose daemon.

Process names on Linux

Compose includes its resolved namespace in the process label. The daemon uses iii:c:<namespace>, and an engine started by that daemon uses iii:e:<namespace>. The namespace comes from --namespace, then namespace: in the compose file, then default. For example, iii compose --namespace orders --up produces these labels when it starts an engine:
Use ps -p <PID> -o pid,ppid,comm,args to inspect both fields. The args field retains the full namespace and the original command arguments. Linux limits comm to 15 bytes, so namespaces longer than nine characters use their first two characters, ~, and six hexadecimal hash characters in that field. Use args to read the full namespace when a short label is abbreviated. The daemon sets its command label through an early re-exec with the same PID, before starting the runtime or any children. Existing external engines keep their names. On other operating systems, process names retain their previous behavior.

Compose logs

Compose logs stdout and stderr output from started workers to logs/ in the state directory. The default state directory is <project-dir>/.iii/compose/<namespace>/. When III_COMPOSE_STATE_DIR is set, it is $III_COMPOSE_STATE_DIR/<project-slug>/<namespace>/. The managed engine writes to engine.log in that same state directory. Logs are rotated every 10 MiB. Compose keeps up to 40 MiB of logs. Compose strips terminal control sequences before persisting the engine output. You can use the logs client for a recent snapshot or a live view:
Each line is prefixed with the worker name, and stderr uses a bold prefix on a terminal.

Running it in the background

Compose does not background or daemonize itself. To accomplish this please use standard tooling For example on most terminals (bash, zsh) you can run:
Or on a server with systemd support you can use a unit file. This is a basic example:
Use Type=simple. Type=notify waits for an sd_notify readiness message, which compose does not send.

The compose::* functions

These are the functions that control the compose worker and are the canonical way of interacting with it and making basic changes to the worker-compose.yaml file. file is not required. Left out, it falls back to a worker-compose.yaml in the daemon’s own working directory; without one, the call fails with NO_COMPOSE_FILE. A relative file path is considered relative to the daemon’s directory, not the caller’s working directory. You may pass an absolute path as well.

Starting a project

compose::up starts every worker in the compose file, in dependency order. Workers that are already ready stay as they are.

compose::up failures

A project that fails to start due to a project-related issue ends the command with PROJECT_DID_NOT_START. Partial starts are rolled back in reverse dependency order.

Stopping a project

compose::down stops the project in reverse dependency order.

Adding workers

compose::add worker=state worker=./workers/api declares one or more workers in the compose file and reconciles the project once. On the CLI, repeat worker= for each worker. Each value takes a registry package name (state), a package name with a version (state@0.21.4), or a directory (./workers/api). A JSON payload can mix worker names and container objects in the same list. Each object accepts the container fields from worker-compose.yaml. The container key comes from the last part of the worker name or directory path.
A container object accepts these fields: A package without a version pins to the latest available version, for example 0.23.1. An explicit selector such as state@next stays as next in worker-compose.yaml. Unknown fields and invalid field types are rejected. Compose validates the complete edited file before it writes any changes. For an existing container, omitted fields other than version keep their values. A supplied field replaces that entire field, including maps such as scripts, environment, and config_override. Use {} or [] to clear maps or lists. An omitted start_after keeps existing dependencies; a supplied list replaces them and includes any required package dependencies. Workers whose declarations did not change remain running. Existing workers whose source, version, dependencies, or settings changed restart in place. Newly declared workers start through the normal dependency plan. If the resolved declaration already matches the file, the call makes no file changes and causes no restart.

Package lock

Compose writes worker-compose.lock beside worker-compose.yaml. The compose file keeps the requested selector. The lock keeps the resolved dependency graph, versions, package types, artifact URLs, SHA-256 digests, and default configuration returned by the registry.
Normal starts and restarts use the lock without resolving next again. If the cache is empty, Compose downloads the URL in the lock and verifies its SHA-256 digest. Compose also verifies the extracted cache contents before reuse. A changed cache entry is downloaded again from the locked URL. Use iii compose build --frozen in CI to require the compose file and lock to match. Use iii compose --up --frozen or compose::up frozen=true to apply the same rule during startup. Frozen mode never resolves selectors or changes the lock. It can download a missing artifact only from the URL already in the lock. Commit the lock so development, CI, and deployments use the same package content.

Removing a worker

compose::remove worker=state removes the named worker and every reference to it. The changes are validated before a worker is removed. Compose then stops only that worker and runs an idempotent up. Removal does not resolve the registry graph or remove other workers that were added with this worker. Those remain declared until they are removed explicitly.

Restarting one worker

compose::restart worker=state stops that specified worker and starts it again. All other workers, including dependencies, are left unchanged. compose::restart without a worker argument restarts the entire compose project. It is approximately the equivalent of compose::down followed by compose::up.

Updating workers

compose::update without worker or workers updates every declared package:// worker to its registry’s latest version. It keeps each worker’s registry reference and skips path:// workers.
compose::update worker=state resolves the selector already in the compose file. A tag such as next, a range, and an exact version all remain unchanged in the file. Use worker=state@<selector> to change the selector. For example, use worker=state@latest to move an exact version to the registry’s latest channel. Update resolves the complete dependency graph. It adds new dependencies, updates changed dependencies, and removes stale dependencies that Compose generated and no remaining package root uses. Manually declared workers are not removed. Compose downloads and verifies the new artifact before it changes the lock or stops a worker. A failed resolve or download leaves the prior lock and running workers unchanged. If the resolved artifact and default configuration did not change, Compose updates lock metadata when needed and does not restart the project. If all selected workers already use the requested versions, the operation leaves the file and running processes unchanged. A project with only path:// workers also stays unchanged.
The worker has to be declared already in order to be updated, and it has to be a package://. Use compose::add to add new workers. Workers specified with path:// are not versioned, any updates to these workers will be reflected the next time the worker is restarted.
An update that changes package content or graph topology restarts the whole project once.

Checking status

compose::status reports each declared worker with its state, its pid, an owned flag, its rotating log_path, and last_error when there is one. owned is false for a worker this daemon has knowledge of but does not manage (ie. was not started by the compose daemon).

Worker states

Viewing logs

compose::logs returns recent stdout and stderr for the workers of one project. The response holds one entry per worker with container, entries, cursor, and truncated. Each entry in entries has stream and message. A cursor has generation and offset. truncated is true when the cursor sent is older than the retained archives.

Listing projects

compose::list returns the daemon name, its namespace, its pid, and other project information.

Validating a file

compose::validate validates a compose file and is intended for package develpment work. It takes file. Validation is offline, so package:// workers are reported under deferred_packages and not resolved.

Validation reports

Stopping the daemon

compose::stop stops the compose project, all associated workers, and optionally the engine if started with (--up). The compose daemon will also exit. Before exiting the daemon will return its name, pid, and the projects it is about to stop.
There is no compose::start equivalent to compose::stop. Stopping a compose daemon means it must be restarted from the server it is running on.

Viewing schema

compose::schema takes a function_id argument. With no function_id, it returns every compose::* schema. Pass a function id to return the schema for a given function_id. The pseudo-id worker-compose.yaml returns the file’s JSON Schema as request and a complete small example as response. Each entry holds function_id, description, request, response, default_timeout_ms, and idempotent. The same schemas, descriptions, and metadata are also published through engine::functions::info.

Namespaces

Namespaces are used to allow advanced architectures that require more than one running copy of a given worker, multi-tenancy, some multi-agent workflows, and various isolation schemes between different parts of a iii application. Namespaces are arbitrary and their usage depends largely on the given usecase. They do not prescribe a specific way of constructing your iii application. The two primary points where Namespaces are used are during Worker registration via Compose and during Trigger and Function interactions. All have ways of declaring which namespace to use.

Precedence

namespace is commonly defined in worker-compose.yaml but can be overriden on compose daemon startup with the --namespace flag. Likewise, compose’s own compose::* functions will exist within the same declared namespace.

What a namespace may hold

A valid namespace is made up of the lowercase characters a-z, 0-9, - and _. All other characters are not permitted and will result in an INVALID_NAMESPACE error; including uppercase letters. To prevent naming conflicts there is no coercion of invalid namespaces to valid namespaces.

One daemon to a namespace

Two compose daemons with different namespaces can share an engine. However, a second daemon claiming a namespace that is already served is refused at registration with DAEMON_ALREADY_SERVING.
Only one Compose daemon can serve a namespace on a iii engine. Set namespace: in the compose file or pass --namespace when several daemons must share one iii engine.

worker-compose.yaml

Below is an example of version 1 of a worker compose file. Unknown keys and duplicate keys are errors. Durations can specify a unit such as: 500ms, 30s, 2m.

Top-level fields

Engine fields

Allowed worker keys are configuration, iii-worker-manager, iii-http-functions, iii-stream, and iii-sandbox. Use #instance for another instance of an allowed type, for example iii-worker-manager#rbac. The engine starts iii-engine-functions, iii-telemetry, and iii-observability automatically. Do not declare these workers in the compose file.
Changes to these worker configurations take effect only after the engine restarts.

Worker fields

Each key under containers is the name the worker registers under. path:// directories resolve against the compose file’s directory. A missing directory fails with MISSING_WORKER_DIRECTORY, and a missing env_file fails with MISSING_ENV_FILE during validation. A path worker normally runs as a host process. A non-empty runtime.base_image in its iii.worker.yaml selects a local VM instead. The worker’s scripts.install and start command then run inside that image, and compose keeps the VM state inside the project. An invalid image reference fails the start instead of falling back to the host or to another image.

Worker kinds

Worker packages can be released in multiple different “kinds”. A kind compose cannot run fails with UNSUPPORTED_PACKAGE_KIND. A bundle’s VM is booted by iii-worker, which the installer ships beside iii and which needs glibc on Linux. Compose runs it as a process rather than linking it, so the engine stays portable; a bundle worker on a machine without iii-worker fails saying so, and every other worker kind is unaffected. Bundles need a VM, and windows has none: a bundle worker there fails with BUNDLE_NEEDS_A_VM before anything is downloaded. Run compose under WSL, where the VM has KVM to run on. Every other worker kind runs on windows as it always has. Bundle support can be refused machine-wide with III_BUNDLE_WORKERS_DISABLED=1, which compose honours.

Scripts

Both hooks run with the worker’s environment, working directory, and their own process group. A post_run runs after the worker stops but before the compose daemon exits. The top-level stop_timeout argument is a global timer for a compose daemon to stop. If this time is exceeded all scripts will be exited along with the compose daemon. A path worker’s start command is scripts.run in worker-compose.yaml, or scripts.start in the worker’s own iii.worker.yaml when run is absent. A worker with neither fails with MISSING_START_COMMAND.

Configuration precedence

Lowest to highest: the configuration a package ships, the entry in the configuration worker, then config_override. Maps merge key by key; arrays and scalars replace. A mapping whose name the override changes is replaced whole: the keys beside name belong to the variant it picks. The merged result is written to an owner-only file and its path is passed to the worker as III_CONFIG. A worker that declares config_name does not start when the fetch fails; the error is CONFIG_FETCH_FAILED.

The worker environment

A worker’s environment is defined by the following sources:
  1. A host baseline. On Unix: PATH, HOME, USER, LOGNAME, SHELL, TERM, TMPDIR, TZ, LANG, LC_ALL. Windows adds the variables the platform needs, such as SystemRoot, COMSPEC, and PATHEXT.
  2. The worker’s env_file entries, then its environment map.
  3. The reserved variables, which the daemon owns.
Declaring a reserved variable in environment or an env_file fails with RESERVED_ENV_OVERRIDE.

Expanding values from environment variables

Variables can be expanded with ${VAR}. References expand in any value, not only in environment. For example:
config_override is never expanded, all values are treated as literals.

Build registry packages

build reads and validates the compose file, prepares worker-compose.lock, then downloads every package:// worker into the same cache used by compose::up. The file defaults to ./worker-compose.yaml. The command does not connect to an engine, start a worker, or run lifecycle hooks. Local path:// workers and engine-managed workers need no registry download and are skipped.
The cache is shared under ~/.iii/compose/packages, or under $III_COMPOSE_STATE_DIR/packages when that variable is set. A later compose::up reuses a valid cached artifact if it exists.

Readiness

A worker is considered up when the engine reports a worker of that name in the project’s namespace. Compose polls engine::workers::list every 200 ms until the worker’s startup_timeout runs out. After a worker is ready, the daemon checks it every 250 ms. A worker that exits takes its transitive dependents down with it and is recorded as failed. Automatic restarts are not performed. When the engine connection drops and comes back, every running worker gets its startup_timeout to register again. Dependency shutdown is dependent upon when a shutdown happens:

Where compose keeps state

Compose state is stored in <project-dir>/.iii/compose/<namespace>/. <project-dir> is the directory containing the canonical compose file, including when --file points outside the current working directory or follows a symbolic link. <namespace> is the daemon’s namespace. Two projects can use default with separate engines on different ports. Starting the same project and namespace twice is refused. Two daemons on the same engine must use different namespaces. With the default state layout, two compose files in the same directory must also use different namespaces. For a read-only project directory, set III_COMPOSE_STATE_DIR to a writable directory. Project state is then stored at $III_COMPOSE_STATE_DIR/<project-slug>/<namespace>/. The slug combines the project directory name with a hash of the canonical compose file path, keeping projects with the same directory name separate. With this override, two compose files in the same directory have separate state directories and can use the same namespace on separate engines. The package cache stays shared at ~/.iii/compose/packages, or $III_COMPOSE_STATE_DIR/packages. Add **/.iii/compose/ to the repository’s .gitignore to exclude generated state, including state from nested projects, from version control.
Before upgrading from the shared namespace layout, stop existing Compose daemons with the old version and confirm that their engines and workers have stopped. The new version does not migrate old process records or logs. Previous state remains under ~/.iii/compose/<namespace>/ (or the old III_COMPOSE_STATE_DIR layout); keep any logs or VM data you need before removing it.

Error codes

Compose can output the following error codes:
For why compose is a worker see Understanding iii / Compose. For understanding namespaces see Understanding iii / Namespaces. For the iii compose entry in the command tree, see the CLI reference.