Skip to main content
Every function you register can be observed. By default, each successful call of the function is an event, and the engine runs the target of each observer whose filters match. The function does not need an observe descriptor or any other change. This is auto delivery, the default. See Observing functions. The rest of this page is for the worker that hosts a function (its owner). It shows how the owner can:
With owner delivery, the function’s code delivers each event. An observer’s target runs only when the function’s code calls obs.emit or obs.call for that observer. To send every call to every observer, use auto delivery.
In 0.25 this replaces registerTriggerType: a trigger type becomes a function with an observe descriptor, and a trigger becomes an observer. To move an existing trigger type, see Upgrading from 0.24.x.

Observing functions

In this example, demo::double is a regular function, so it uses auto delivery: each successful call of demo::double is an event. It has no observe descriptor and no hooks. The observer runs demo::on-double after each event, and demo::on-double gets the Observed envelope of the call. Here the source, the target, and the observer are in one worker. They can also be in different workers.
After the call, demo::on-double logs demo::double was called with the input { n: 21 } and the output { n: 42 }.

Publish your own events

Some events do not come from a function call, for example a file change or a message from an outside system. To publish such an event, register a function that stands for it, and call that function each time the event occurs. The function can return its input unchanged. Each call is an event for the observers of the function.

Know when a function is observed

FunctionRef.isObserved() (is_observed() in Python and Rust, worker.IsObserved(id) in Go) returns true when the function has at least 1 active observer. It is fast, so you can call it before you build an event, and skip the event when no observer gets it.
Because the worker calls its own function, the call runs in-process (local dispatch) and reaches the engine only when an observer’s filters match. To run code when an observer is added or removed, give the function onObserverAdded and onObserverRemoved hooks. Auto delivery does not change: each call still goes to each observer.

When to use owner delivery

Advanced function observation with owner delivery

Use owner delivery when different observers of the function must get different events, and filters cannot select those events. For example, each observer of a file watcher gives a path in args and gets the changes to that path only. Also use owner delivery when the function needs the return value of the observer’s target (obs.call).
With delivery: 'owner', the function’s code decides which observers each event is for, from their args, and hands it over. In this case, an observer receives an event only when the function’s code calls obs.emit() or obs.call() on it. obs.emit(event) sends the event and does not wait for the target, and obs.call(payload) runs the target and returns its result to the function. The examples below use this pattern:
  1. onObserverAdded runs once for each new observer, before the observer becomes active. In the examples, it starts what produces events for the observer’s args (a timer, a watcher). To refuse the observer, it throws. After 30 seconds, the observer is rejected with OWNER_TIMEOUT.
  2. The producer calls the function when something happens.
  3. The function’s handler gets the observers the event is for (from observers() or from its own index) and calls obs.emit(event) or obs.call(payload) on each.
onObserverRemoved runs once for each removed observer. In the examples, it stops the producer when no observer needs it. The two ways to hand over an event: Only the hosting worker can call an owner-delivered function. The engine refuses calls from other workers, and from the engine, with FORBIDDEN. The SDK applies the observer’s filters to emit and call. If the filters reject the event, emit returns false, call fails with FILTERS_NOT_MATCHED, and the SDK sends nothing. To check before you send, use obs.matches({ input }). Both go through the engine, which applies the observer’s policy (max_fires, expiry, max_in_flight), adds its metadata, and enforces the loop guard. Both throw NOT_OWNER_DELIVERED on an auto function. emit does not wait. If the function emits faster than the targets can run, the observers’ queues become full. The engine then stops reading the worker’s connection until the queues have space, for a maximum of 250 ms. During this hold:
  • The SDK keeps events in its report buffer. When the buffer is full, it drops events as BUFFER_FULL.
  • All other messages from the worker (its calls and its answers) wait behind the reports.
After each hold, the engine reads the connection for at least 1/4 of the hold time before the next hold. Events that overflow during this time are dropped as OVERFLOW. Thus the engine holds the worker for a maximum of 4/5 of the time. An emitted event travels in the SDK’s report, like a local call: values JSON cannot hold arrive as null (or U+FFFD for a lone surrogate), an event larger than the 8 MiB report buffer is dropped as TOO_LARGE, and a report lost with a dropped connection is not sent again. The event’s caller depends on where the function’s code calls emit. Inside the function’s own handler, the event is that invocation, with its id and its caller. Inside a different handler, that handler is the caller. Outside all handlers, the caller is only the worker. That is why the examples below trigger their own function and emit from its handler. The event’s invocation_id follows the same rule. Inside the function’s own handler, the event has the id of that invocation, and all events emitted during the same invocation have the same id. Anywhere else, each event gets a new id. emit cannot set the id. To pass an id on, put it in the event.

Example: a file watcher that emits

files::watch is a complete owner-delivered source. Each observer gives a file path in its args ({ path }) and gets the changes to that file only. The numbered comments show the steps above. The worker polls each watched file 1 time each second. When many observers watch the same path, the worker still polls that path only 1 time each second. This function uses owner delivery and the two hooks because its observers decide which files to watch. The worker cannot poll every file and then filter the changes. It must learn the path of each observer before it can produce events for that path, and it must stop when no observer wants that path, or its pollers use resources for nothing. The hooks are the only places where the function learns that an observer arrived or left.
A consumer observes it like any other function, with the path as args:
Compare it with files::on-change at the top of this page. With files::on-change, each observer gets all changes and uses filters to select changes. With files::watch, each observer asks for 1 path, and the source watches only the paths that observers ask for. If an observer’s args do not agree with the schema (no path, or a field that the schema does not list), the observer fails with INVALID_ARGS. onObserverAdded does not run.

Example: a command router that calls

The file watcher does not wait for a result. This example needs a result, so it uses obs.call. chat::on-command answers chat commands. Each observer gives 1 command in its args ({ command }). The router calls the target of that observer and returns the target’s result as its own result. In this example, the function’s onObserverAdded hook keeps a map from each command string to its observer. If the map already has the observer’s command string (an exact, case-sensitive match), the hook rejects the observer with COMMAND_TAKEN. This rule and the COMMAND_TAKEN code are part of the example code. The SDK does not compare the args of different observers.
A worker that handles /weather registers its handler and observes the router with its command:
Only the router’s worker can call chat::on-command. When it calls the router, the return value of weather::reply becomes the router’s result:

Errors of obs.call

obs.call fails with the target’s own error, or with one of these codes:

Reject an observer later

An observer that the function accepted can become incorrect later. For example, a configuration reload can remove the resource that the observer watches. obs.reject(code, message) stops delivery to the observer immediately. The engine moves the observer to error with that code, and the SDK then runs onObserverRemoved. If the connection is lost, the SDK sends the rejection again after it connects again.
FunctionRef.observers() (observers() in Python and Rust, worker.Observers(id) in Go) returns the active observers of the function.

Refuse observers

Some functions must not be observed, for example because their input contains secrets. Register them with observe: { allow: false } (ObserveHooks(allow=False) in Python, Observe::auto().allow(false) in Rust, Allow: &no in Go). Every observe on them fails with NOT_OBSERVABLE.

Accept old triggers

A function that replaces a trigger type can keep serving workers that still call registerTrigger({ type, function_id, config }). Declare the type in legacy_trigger_type. The excerpt below turns the old timer trigger type into timer::on-tick, a source shaped like the file watcher with one timer per interval instead of one poller per path. Only the parts that differ for old triggers are shown.
When a registerTrigger arrives for type timer and no worker provides the timer trigger type the old way, the engine changes the trigger into an observer of the function:
  • The observer belongs to the worker that called registerTrigger; its id is the trigger id.
  • It has no args, no filters, and deliver: 'input'. Its target gets the emitted event in the same form as the old trigger handler got the fired payload.
  • The trigger’s metadata becomes the observer’s metadata sidecar.
  • The hook gets obs.legacy = { trigger_type, config }. Validate the config as the old TriggerHandler did, then accept or reject the observer. The old worker gets 1 TriggerRegistrationResult with the result, which includes a rejection.
  • unregisterTrigger (or engine::unregister_trigger) disposes the observer.
The engine looks for the stand-in function where registerTrigger looks for a provider: only in the trigger’s trigger_namespace when it names one, otherwise in the registering worker’s namespace and then in default. The engine uses timer::on-tick for a registerTrigger call only when no worker provides the timer trigger type with registerTriggerType. Thus, during a migration:
  • If a worker still provides timer with registerTriggerType, new timer triggers go to that worker. They do not become observers of timer::on-tick.
  • Triggers that are already observers of timer::on-tick stay observers. They do not move to the old worker.
To send all timer triggers to timer::on-tick, stop calling registerTriggerType("timer") in the same release that adds timer::on-tick.

The observe descriptor

Pass observe when you register the function: The SDK sends only the first 4 fields to the engine. The hooks stay in the worker’s process. The names follow the conventions of each SDK: args_format with delivery: 'auto' fails at registration with INVALID_OBSERVE_DESCRIPTOR. An auto function sends each invocation to each observer, so it does not use observer args.

Hook rules

  • onObserverAdded runs before the SDK acknowledges the observer to the engine. The observer becomes active only after the hook returns. Thus, the hook’s work is complete before the first event. Inside the hook, emit returns false and call fails with OBSERVER_NOT_ACTIVE.
  • Throwing rejects the observer: an ObserveRejection(code, message) sets the code the observer’s ready() fails with; any other error is OBSERVE_REJECTED.
  • The hooks of one function run one at a time, in the engine’s order. Hooks of different functions may run concurrently.
  • A hook still running after 30 seconds counts as failed (OWNER_TIMEOUT, the engine enforces the same observers.owner_ack_timeout_ms), and the next hook may start.
  • onObserverRemoved runs when an accepted observer leaves the function: it is disposed, or it goes to error. For example: dispose(), max_fires or expiry, an observing worker that does not connect again within its grace period, a late obs.reject, or a reconnect that no longer lists the observer.

The Observer object

Hooks get an Observer: the observer as the owner sees it. It does not include the observer’s metadata, policy, or observing worker. The object stays the same for the full life of the observer, so the function’s code can keep it, for example in a map. Language specifics: Python has call (blocking) and call_async; Rust has id(), args::<T>(), args_value(), legacy(), emit(&event) -> Result<bool, Error> and call::<R>(payload).await; Go has Args (a map), DecodeArgs(&v), Legacy, Emit(ctx, event) (bool, error) and Call(ctx, payload) (json.RawMessage, error).

Against an older engine

With an engine older than 0.25, a function with an observe descriptor registers as a plain function. Its hooks do not run, and the SDK logs 1 warning for each function that has hooks or owner delivery. Local dispatch is off, and all calls go through the engine.