> ## Documentation Index
> Fetch the complete documentation index at: https://iii.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Observability and Logging
> How to use structured logging, configure observability, and inspect traces and logs with the iii Console.
## Goal
Make your iii application fully observable: correlate every log entry to the exact trace that produced it, inspect execution timelines to find bottlenecks, and optionally export all telemetry to third-party tools like Grafana, Jaeger, or Datadog.
## The iii Logger
Every iii SDK ships a `Logger` class that emits logs as OpenTelemetry LogRecords. Each log call automatically captures the active **trace ID** and **span ID**, linking the log entry to the distributed trace that produced it.
Language-native logging functions such as `console.log` in Node, `print()` in Python, `tracing::info!` in Rust write to stdout but are not connected to iii traces. This means you cannot find them in the iii Console's trace detail view, and they are invisible to any OTLP-based observability backend.
| Approach | Where it appears | Trace correlation |
| ------------------------------------------- | ---------------------------------- | ----------------------------------------------- |
| `console.log("Order created")` | stdout only | None |
| `print("Order created")` | stdout only | None |
| `tracing::info!("Order created")` | stdout only | None |
| `logger.info("Order created", { orderId })` | stdout, iii Console, OTLP backends | Automatic — linked to the active trace and span |
Avoid using `console.log`, `print()`, or `tracing::info!` for application logs. These bypass the OpenTelemetry pipeline and will not appear in the iii Console or any connected observability tool.
## Trace-correlated logs in iii Console
When you use the iii Logger, every log entry is attached to the trace that was active when the log was emitted. In the iii Console, clicking a trace in the waterfall chart opens a detail drawer. The **Logs** tab shows every log entry from that exact execution — with severity, timestamp, message, and any structured data you attached.
This is the core value of using the iii Logger: you can go from a slow trace in the waterfall chart directly to the logs that explain what happened, without grepping through stdout or cross-referencing timestamps manually.
## Using the Logger
Create a `Logger` instance in your function handler. No configuration is required — trace context is injected automatically.
```typescript theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
import { Logger } from 'iii-sdk'
const logger = new Logger()
logger.info('Worker connected')
```
```python theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
from iii import Logger
logger = Logger()
logger.info("Worker connected")
```
```rust theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
use iii_sdk::Logger;
let logger = Logger::new();
logger.info("Worker connected", None);
```
Pass a second argument with key-value data. Structured data becomes filterable attributes in the iii Console and in any OTLP-compatible backend.
```typescript theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
logger.info('Order processed', { orderId: 'ord_123', amount: 49.99, currency: 'USD' })
logger.warn('Retry attempt', { attempt: 3, maxRetries: 5, endpoint: '/api/charge' })
logger.error('Payment failed', { orderId: 'ord_123', gateway: 'stripe', errorCode: 'card_declined' })
logger.debug('Cache lookup', { key: 'user:42', hit: false })
```
```python theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
logger.info("Order processed", {"order_id": "ord_123", "amount": 49.99, "currency": "USD"})
logger.warn("Retry attempt", {"attempt": 3, "max_retries": 5, "endpoint": "/api/charge"})
logger.error("Payment failed", {"order_id": "ord_123", "gateway": "stripe", "error_code": "card_declined"})
logger.debug("Cache lookup", {"key": "user:42", "hit": False})
```
```rust theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
use serde_json::json;
logger.info("Order processed", Some(json!({ "order_id": "ord_123", "amount": 49.99, "currency": "USD" })));
logger.warn("Retry attempt", Some(json!({ "attempt": 3, "max_retries": 5, "endpoint": "/api/charge" })));
logger.error("Payment failed", Some(json!({ "order_id": "ord_123", "gateway": "stripe", "error_code": "card_declined" })));
logger.debug("Cache lookup", Some(json!({ "key": "user:42", "hit": false })));
```
Prefer key-value objects over string interpolation. Structured fields let you filter, aggregate, and build dashboards — string-interpolated messages do not.
The Logger works anywhere inside a function handler. Trace context is captured from the active invocation automatically.
```typescript theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
import { registerWorker, Logger } from 'iii-sdk'
const iii = registerWorker(process.env.III_URL ?? 'ws://localhost:49134')
iii.registerFunction('orders::create', async (req) => {
const logger = new Logger()
const { customerId, amount } = req.body
const orderId = `order-${Date.now()}`
logger.info('Order created', { orderId, customerId, amount })
// ... business logic ...
logger.info('Order processing complete', { orderId })
return { status_code: 201, body: { orderId } }
})
```
```python theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
import os
from iii import register_worker, Logger
iii = register_worker(os.environ.get("III_URL", "ws://localhost:49134"))
def create_order(req):
logger = Logger()
body = req.get("body", {})
customer_id = body.get("customerId")
amount = body.get("amount")
order_id = f"order-{int(__import__('time').time() * 1000)}"
logger.info("Order created", {"order_id": order_id, "customer_id": customer_id, "amount": amount})
# ... business logic ...
logger.info("Order processing complete", {"order_id": order_id})
return {"status_code": 201, "body": {"orderId": order_id}}
iii.register_function("orders::create", create_order)
```
```rust theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
use iii_sdk::{register_worker, InitOptions, Logger, RegisterFunction};
use serde_json::{json, Value};
let iii = register_worker("ws://127.0.0.1:49134", InitOptions::default());
let reg = RegisterFunction::new_async("orders::create", |req: Value| async move {
let logger = Logger::new();
let customer_id = req["body"]["customerId"].as_str().unwrap_or("");
let amount = req["body"]["amount"].as_f64().unwrap_or(0.0);
let order_id = format!("order-{}", chrono::Utc::now().timestamp_millis());
logger.info("Order created", Some(json!({ "orderId": order_id, "customerId": customer_id, "amount": amount })));
// ... business logic ...
logger.info("Order processing complete", Some(json!({ "orderId": order_id })));
Ok(json!({ "status_code": 201, "body": { "orderId": order_id } }))
});
iii.register_function(reg);
```
### Logger API reference
All three SDKs expose the same four methods:
| Method | Node / TypeScript | Python | Rust |
| ------- | -------------------------- | ------------------------------ | ---------------------------------- |
| Info | `logger.info(msg, data?)` | `logger.info(msg, data=None)` | `logger.info(msg, Option)` |
| Warning | `logger.warn(msg, data?)` | `logger.warn(msg, data=None)` | `logger.warn(msg, Option)` |
| Error | `logger.error(msg, data?)` | `logger.error(msg, data=None)` | `logger.error(msg, Option)` |
| Debug | `logger.debug(msg, data?)` | `logger.debug(msg, data=None)` | `logger.debug(msg, Option)` |
When OpenTelemetry is not initialized (e.g. in unit tests), the Logger falls back to `console.*` (Node), Python `logging` (Python), or `tracing::*` (Rust) — your logs still appear in stdout.
## Configuring observability
The iii engine's Observability worker (`iii-observability`) controls how traces, logs, and metrics are collected and exported. There are two main configurations depending on your environment.
### Local development
For local development, use the `memory` exporter. Traces and logs are stored in the engine's memory and can be inspected through the iii Console. This is the simplest setup and requires no external infrastructure.
```yaml title="iii-config.yaml" theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
workers:
- name: iii-observability
config:
enabled: true
exporter: memory
logs_enabled: true
memory_max_spans: 1000
```
| Field | Purpose | Default |
| --------------------- | ------------------------------------------------- | --------------------------- |
| `exporter` | Where to send traces: `memory`, `otlp`, or `both` | `otlp` |
| `memory_max_spans` | Max spans kept in memory | `1000` |
| `logs_enabled` | Enable structured log storage | `true` (always initialized) |
| `logs_max_count` | Max log entries kept in memory | `1000` |
| `logs_console_output` | Also print logs to the terminal via tracing | `true` |
With `exporter: memory`, all data lives in the engine process. This is ideal for development — no collector, no database, just start the engine and open the console.
### Exporting to third-party tools
For production or when you want to send telemetry to an external system (Grafana, Jaeger, Datadog, or any OTLP-compatible collector), use the `otlp` exporter with an `endpoint`.
```yaml title="iii-config.yaml" theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
workers:
- name: iii-observability
config:
enabled: true
exporter: otlp
endpoint: "http://collector.example.com:4317"
service_name: my-service
service_version: 1.0.0
metrics_enabled: true
logs_enabled: true
logs_exporter: otlp
```
To keep local visibility through iii Console **and** export to a collector simultaneously, use `exporter: both`:
```yaml title="iii-config.yaml" theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
workers:
- name: iii-observability
config:
enabled: true
exporter: both
endpoint: "http://collector.example.com:4317"
service_name: my-service
logs_enabled: true
logs_exporter: both
```
```mermaid theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
graph LR
Engine["iii Engine"] -->|OTLP gRPC| Collector["OTLP Collector"]
Collector --> Grafana["Grafana / Tempo"]
Collector --> Jaeger["Jaeger"]
Collector --> Datadog["Datadog"]
Engine -->|"in-memory (exporter: both)"| Console["iii Console"]
```
The `endpoint` field can also be set via the `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable. See the [Observability worker reference](../workers/iii-observability) for the full list of configuration fields and environment variable overrides.
## Viewing Logs in iii Console
Once you've instrumented your code with the Logger, view trace-correlated logs in the iii Console.
See [Use the Console](./use-console#inspect-a-trace) for step-by-step instructions on navigating traces and viewing span details.
Navigate to **Traces**, select a trace, and open the **Logs** tab on any span to see every log entry from that execution — with severity, timestamp, message, and structured attributes you attached.
The dedicated **Logs** page provides a full log viewer with severity filters, full-text search, and time-range controls. If a log entry has a `trace_id`, click it to jump directly to the corresponding trace.
For the full console feature set, see the [Console reference](../console).
## Result
Your iii application is now fully observable:
* **Structured logs** are correlated to distributed traces automatically — no manual wiring.
* **Local visibility** is available through iii Console with the `memory` exporter — no external infrastructure needed.
* **Third-party export** sends traces, logs, and metrics to any OTLP-compatible backend via the `otlp` or `both` exporter.
* **Bottleneck identification** is possible through waterfall, flame graph, and service breakdown views in the console.
## Next steps
Full iii Console feature reference
Custom spans, metrics, and telemetry utilities
Full configuration reference for traces, logs, metrics, alerts, and sampling
End-to-end multi-step workflow with trace correlation