observe descriptor or any other change. This is auto
delivery, the default. See Observing functions.
The rest of this page is for the worker that hosts a function (its owner). It shows how the owner
can:
- publish events that do not come from a function call (Publish your own events);
- check for observers, and run code when an observer is added or removed (Know when a function is observed);
- send different events to different observers, based on the
argsof each observer (Advanced function observation with owner delivery); - accept the old
registerTriggercalls of a trigger type the function replaces (Accept old triggers).
registerTriggerType: a trigger type becomes a function with an observe
descriptor, and a trigger becomes an observer. To move an existing trigger type, see
Upgrading from 0.24.x.
Observing functions
In this example,demo::double is a regular function, so it uses auto delivery: each successful
call of demo::double is an event. It has no observe descriptor and no hooks. The observer runs
demo::on-double after each event, and demo::on-double gets the
Observed envelope of the call. Here the source, the
target, and the observer are in one worker. They can also be in different workers.
- Node / TypeScript
- Python
- Rust
- Go
demo::on-double logs demo::double was called with the input { n: 21 } and the
output { n: 42 }.
Publish your own events
Some events do not come from a function call, for example a file change or a message from an outside system. To publish such an event, register a function that stands for it, and call that function each time the event occurs. The function can return its input unchanged. Each call is an event for the observers of the function.- Node / TypeScript
- Python
- Rust
- Go
Know when a function is observed
FunctionRef.isObserved() (is_observed() in Python and Rust, worker.IsObserved(id) in Go)
returns true when the function has at least 1 active observer. It is fast, so you can call it before
you build an event, and skip the event when no observer gets it.
- Node / TypeScript
- Python
- Rust
- Go
onObserverAdded and
onObserverRemoved hooks. Auto delivery does not change: each call still goes to each observer.
- Node / TypeScript
- Python
- Rust
- Go
When to use owner delivery
Advanced function observation with owner delivery
Use owner delivery when different observers of the function must get different events, and
filters cannot select those events. For example, each observer
of a file watcher gives a path in
args and gets the changes to that path only. Also use owner
delivery when the function needs the return value of the observer’s target (obs.call).delivery: 'owner', the function’s code decides which observers each event is for, from their
args, and hands it over. In this case, an observer receives an event only when the function’s code
calls obs.emit() or obs.call() on it. obs.emit(event) sends the event and does not wait for
the target, and obs.call(payload) runs the target and returns its result to the function.
The examples below use this pattern:
onObserverAddedruns once for each new observer, before the observer becomesactive. In the examples, it starts what produces events for the observer’sargs(a timer, a watcher). To refuse the observer, it throws. After 30 seconds, the observer is rejected withOWNER_TIMEOUT.- The producer calls the function when something happens.
- The function’s handler gets the observers the event is for (from
observers()or from its own index) and callsobs.emit(event)orobs.call(payload)on each.
onObserverRemoved runs once for each removed observer. In the examples, it stops the producer when
no observer needs it.
The two ways to hand over an event:
Only the hosting worker can call an owner-delivered function. The engine refuses calls from other
workers, and from the engine, with
FORBIDDEN.
The SDK applies the observer’s filters to emit and call. If the filters reject the event, emit
returns false, call fails with FILTERS_NOT_MATCHED, and the SDK sends nothing. To check before
you send, use obs.matches({ input }). Both go through the engine, which applies the observer’s
policy (max_fires, expiry, max_in_flight), adds its metadata, and enforces the loop guard. Both
throw NOT_OWNER_DELIVERED on an auto function.
emit does not wait. If the function emits faster than the targets can run, the observers’ queues
become full. The engine then stops reading the worker’s connection until the queues have space, for
a maximum of 250 ms. During this hold:
- The SDK keeps events in its report buffer. When the buffer is full, it drops events as
BUFFER_FULL. - All other messages from the worker (its calls and its answers) wait behind the reports.
OVERFLOW. Thus the engine holds the
worker for a maximum of 4/5 of the time.
An emitted event travels in the SDK’s report, like a
local call: values JSON cannot hold arrive as null (or
U+FFFD for a lone surrogate), an event larger than the 8 MiB report buffer is dropped as
TOO_LARGE, and a report lost with a dropped connection is not sent again.
The event’s caller depends on where the function’s code calls emit. Inside the function’s own
handler, the event is that invocation, with its id and its caller. Inside a different handler, that
handler is the caller. Outside all handlers, the caller is only the worker. That is why the examples
below trigger their own function and emit from its handler.
The event’s invocation_id follows the same rule. Inside the function’s own handler, the event has
the id of that invocation, and all events emitted during the same invocation have the same id.
Anywhere else, each event gets a new id. emit cannot set the id. To pass an id on, put it in the
event.
Example: a file watcher that emits
files::watch is a complete owner-delivered source. Each observer gives a file path in its args
({ path }) and gets the changes to that file only. The numbered comments show the steps above. The
worker polls each watched file 1 time each second. When many observers watch the same path, the
worker still polls that path only 1 time each second.
This function uses owner delivery and the two hooks because its observers decide which files to
watch. The worker cannot poll every file and then filter the changes. It must learn the path of each
observer before it can produce events for that path, and it must stop when no observer wants that
path, or its pollers use resources for nothing. The hooks are the only places where the function
learns that an observer arrived or left.
- Node / TypeScript
- Python
- Rust
- Go
files::on-change at the top of this page. With files::on-change, each observer
gets all changes and uses filters to select changes. With files::watch, each observer asks for 1
path, and the source watches only the paths that observers ask for.
If an observer’s args do not agree with the schema (no path, or a field that the schema does not
list), the observer fails with INVALID_ARGS. onObserverAdded does not run.
Example: a command router that calls
The file watcher does not wait for a result. This example needs a result, so it usesobs.call.
chat::on-command answers chat commands. Each observer gives 1 command in its args ({ command }).
The router calls the target of that observer and returns the target’s result as its own result.
In this example, the function’s onObserverAdded hook keeps a map from each command string to its
observer. If the map already has the observer’s command string (an exact, case-sensitive match),
the hook rejects the observer with COMMAND_TAKEN. This rule and the COMMAND_TAKEN code are part
of the example code. The SDK does not compare the args of different observers.
- Node / TypeScript
- Python
- Rust
- Go
/weather registers its handler and observes the router with its command:
chat::on-command. When it calls the router, the return value of
weather::reply becomes the router’s result:
Errors of obs.call
obs.call fails with the target’s own error, or with one of these codes:
Reject an observer later
An observer that the function accepted can become incorrect later. For example, a configuration reload can remove the resource that the observer watches.obs.reject(code, message) stops delivery
to the observer immediately. The engine moves the observer to error with that code, and the SDK
then runs onObserverRemoved. If the connection is lost, the SDK sends the rejection again after it
connects again.
FunctionRef.observers() (observers() in Python and Rust, worker.Observers(id) in Go) returns
the active observers of the function.
Refuse observers
Some functions must not be observed, for example because their input contains secrets. Register them withobserve: { allow: false } (ObserveHooks(allow=False) in Python,
Observe::auto().allow(false) in Rust, Allow: &no in Go). Every observe on them fails with
NOT_OBSERVABLE.
Accept old triggers
A function that replaces a trigger type can keep serving workers that still callregisterTrigger({ type, function_id, config }). Declare the type in legacy_trigger_type. The
excerpt below turns the old timer trigger type into timer::on-tick, a source shaped like the
file watcher with one timer per interval instead of one poller
per path. Only the parts that differ for old triggers are shown.
registerTrigger arrives for type timer and no worker provides the timer trigger type
the old way, the engine changes the trigger into an observer of the function:
- The observer belongs to the worker that called
registerTrigger; its id is the trigger id. - It has no args, no filters, and
deliver: 'input'. Its target gets the emitted event in the same form as the old trigger handler got the fired payload. - The trigger’s metadata becomes the observer’s metadata sidecar.
- The hook gets
obs.legacy = { trigger_type, config }. Validate the config as the oldTriggerHandlerdid, then accept or reject the observer. The old worker gets 1TriggerRegistrationResultwith the result, which includes a rejection. unregisterTrigger(orengine::unregister_trigger) disposes the observer.
registerTrigger looks for a provider: only in the
trigger’s trigger_namespace when it names one, otherwise in the registering worker’s namespace and
then in default.
The engine uses timer::on-tick for a registerTrigger call only when no worker provides the
timer trigger type with registerTriggerType. Thus, during a migration:
- If a worker still provides
timerwithregisterTriggerType, newtimertriggers go to that worker. They do not become observers oftimer::on-tick. - Triggers that are already observers of
timer::on-tickstay observers. They do not move to the old worker.
timer triggers to timer::on-tick, stop calling registerTriggerType("timer") in the
same release that adds timer::on-tick.
The observe descriptor
Pass observe when you register the function:
The SDK sends only the first 4 fields to the engine. The hooks stay in the worker’s process. The
names follow the conventions of each SDK:
args_format with delivery: 'auto' fails at registration with INVALID_OBSERVE_DESCRIPTOR. An
auto function sends each invocation to each observer, so it does not use observer args.
Hook rules
onObserverAddedruns before the SDK acknowledges the observer to the engine. The observer becomesactiveonly after the hook returns. Thus, the hook’s work is complete before the first event. Inside the hook,emitreturnsfalseandcallfails withOBSERVER_NOT_ACTIVE.- Throwing rejects the observer: an
ObserveRejection(code, message)sets the code the observer’sready()fails with; any other error isOBSERVE_REJECTED. - The hooks of one function run one at a time, in the engine’s order. Hooks of different functions may run concurrently.
- A hook still running after 30 seconds counts as failed (
OWNER_TIMEOUT, the engine enforces the sameobservers.owner_ack_timeout_ms), and the next hook may start. onObserverRemovedruns when an accepted observer leaves the function: it is disposed, or it goes toerror. For example:dispose(),max_firesor expiry, an observing worker that does not connect again within its grace period, a lateobs.reject, or a reconnect that no longer lists the observer.
The Observer object
Hooks get an Observer: the observer as the owner sees it. It does not include the observer’s
metadata, policy, or observing worker. The object stays the same for the full life of the observer,
so the function’s code can keep it, for example in a map.
Language specifics: Python has
call (blocking) and call_async; Rust has id(), args::<T>(),
args_value(), legacy(), emit(&event) -> Result<bool, Error> and call::<R>(payload).await; Go
has Args (a map), DecodeArgs(&v), Legacy, Emit(ctx, event) (bool, error) and
Call(ctx, payload) (json.RawMessage, error).
Against an older engine
With an engine older than 0.25, a function with anobserve descriptor registers as a plain
function. Its hooks do not run, and the SDK logs 1 warning for each function that has hooks or owner
delivery. Local dispatch is off, and all calls go through the engine.