browser
v0.2.28A browser on the iii bus - shared Chromium with persistent, restorable tabs and incognito tabs. Navigate, act, read the page console, pick elements. Also parses HTML natively without a browser (browser::* - css/xpath/regex, element search, markdown).
Install
iii trigger compose::add worker=browser@0.2.28Run in your project with the iii engine running. Exact versions are immutable and digest-pinned.
Full markdown
/registry/workers/browser.md?version=0.2.28. Paste it into an LLM prompt or pipe it through curl from a worker.Install
iii trigger compose::add worker=browser@0.2.28Dependencies
Readme
browser
A browser on the iii engine bus: one shared
Chromium with tabs, a persistent profile (cookies, logins, storage) under
data/browser, tabs that survive restarts, and incognito tabs that save
nothing. Agents open a tab (a session), read the page as an
accessibility-tree outline, click and type against element refs, and read the
page's own console and network history back as data. The single most
important thing it gives you: "why is my dev server page blank?" becomes
answerable, because the page's console errors are one
browser::console::read away. The console
worker adds the human window: a Chrome-style tab strip over a streaming
viewport (Chromium-pushed screencast frames), an address bar, developer tools
behind the menu, and click-to-pick elements into chat.
It also carries a native Rust scraping surface, browser::*: HTTP
and browser fetching, screenshots, persistent sessions and BFS crawling, plus
CSS/XPath/regex queries, element search and HTML→Markdown that run over any
HTML string with no browser at all. See
Scraping and HTML parsing below.
In the console
An agent reads a page as an accessibility outline (browser::snapshot) while
you watch the live viewport and console feed:
browser::screenshot renders the captured image inline in the chat card:
Pick mode highlights the element under the cursor and drops it into the chat composer as an actionable ref:
Tabs, sleep, and incognito
A session is a tab in the worker's browser. Tabs share one Chromium process
and one profile, so a login in one tab is a login in all of them, exactly as
in a browser. Regular tabs are saved to
- Lifetime. A tab stays open until
browser::sessions::stop, or until the optionalttl_msit was opened with elapses. The console opens tabs with no lifetime. - Sleep. A tab nobody watches (no console viewer, no recording) or calls
for
inactive_after_ms(30 minutes by default) goes to sleep: its page is closed, the tab is kept and still listed (active: false). Any call on it, or selecting it in the console, opens the page again at the url it remembered; back/forward keep working from the tab's own stack. When the last live tab sleeps, Chromium quits and its profile is flushed to disk; the next tab launches it again.max_sessionscaps live tabs: opening one more puts the least recently used unwatched tab to sleep first. - Incognito.
browser::sessions::startwithincognito: trueopens a PRIVATE tab in its own throwaway browser context: no shared cookies or logins, nothing written underdata_dir, no history kept, not restored after a restart, and inactivity closes it for good instead of putting it to sleep. The console shows it in Chrome's dark private-window palette. - Clearing data.
browser::clear-dataclears the site a tab is on (its cookies, its storage, the shared cache) — the ⋮ menu's "Clear cookies and site data".browser::clear-browser-data(Settings → Clear browser data) closes every page, quits Chromium, and deletes the whole profile and downloads; tabs stay and reopen signed out. - Loading like a browser. A page that fails to load (a network error, an
empty HTTP error response such as x.com's 400 to unknown clients) leaves
Chromium's error page in the tab and is reported in
navigate'sok/error, not thrown. Anhttps://url on localhost,*.localhost, or a loopback/private address whose TLS handshake fails (a plain dev server) is retried overhttp://, like an address bar does; public hosts never downgrade. Pages see a plain Chrome user agent, neverHeadlessChrome. - Live view. Every tab renders in its own headless window (a window shows only its active tab, so tabs sharing one would freeze), and the console gets frames at up to 30 fps, latest frame first, whatever the bus latency.
Install
iii trigger compose::add worker=browseriii trigger compose::add resolves the worker and its dependencies, writes
exact declarations to worker-compose.yaml, and reconciles the Compose
project. By default the worker drives a Chromium/Chrome already installed on
the machine; point executable at a specific binary if auto-detection picks
the wrong one.
No Chrome on the machine?
The worker can download its own Chromium. In the ADE, the setup wizard shows a Browser step with a Download Chromium button whenever the browser worker is installed and no Chromium is found (Set up the harness → Browser). From anywhere else:
iii trigger browser::chromium::status # what is found, and where it looked
iii trigger browser::chromium::install # returns { job_id, status } at oncebrowser::chromium::install downloads the Stable
Chrome for Testing
build for this platform (linux64, linux-arm64, mac-arm64, mac-x64, win64; about 200 MB)
and keeps it in ${XDG_CACHE_HOME:-~/.cache}/iii/browser/chrome/
(III_BROWSER_CACHE_DIR replaces ~/.cache/iii/browser), with
current.json naming the version in use. The job runs in the background:
it fetches the version manifest, streams the zip to a .partial file,
unpacks it into a staging folder, runs the binary with --version, then
moves it into place in one rename, so a failed or interrupted install never
leaves a half-installed browser behind. One job runs per worker (a second
call returns the running job) and a lock file in the cache keeps two workers
sharing it apart. Progress arrives on the browser::chromium-install-progress
trigger and in browser::chromium::status (job). On Linux, a machine
missing Chromium's shared libraries fails at the --version step with a
hint listing the Debian packages to install. When a Chromium is already
found it does nothing unless force: true. Agents need approval to call it;
browser::chromium::status is read-only and auto-allowed.
Which binary launches is decided by one resolver, shared by sessions,
browser::doctor, browser::chromium::status and the scraping browser
tiers. First hit wins:
- the config
executable(when set, nothing else is tried); - the
CHROMEenvironment variable; - a system install —
chrome,google-chrome(-stable),chromium(-browser),msedge/microsoft-edgeonPATH, then the usual install locations; - the downloaded copy (
current.json); - the newest Playwright (
ms-playwright/chromium-*), then Puppeteer (~/.cache/puppeteer/chrome/*), Chromium already on the machine.
A Chrome you install yourself later therefore wins over the download without anything being removed.
On a Linux that blocks unprivileged user namespaces for programs without an
AppArmor profile (Ubuntu 23.10 and later), a Chromium from a user cache — the
download, Playwright, Puppeteer — cannot start its sandbox ("No usable
sandbox!"), so the worker launches those with --no-sandbox, as Playwright
does; browser::chromium::status reports sandboxed: false and
browser::doctor lists it as an issue. A distribution Chromium or Google
Chrome ships the profile and keeps its sandbox. When nothing is found, session calls fail with an
error that starts with chromium_missing: and says how to fix it; the ADE
recognises the marker and offers the install.
Engines
Interactive sessions run on one of two engines, both driven over the Chrome
DevTools Protocol, so the browser::* functions are the same code path:
engine |
What it needs | What you get |
|---|---|---|
chromium (default) |
Chrome, Chromium, or Edge installed | Everything: live view, real screenshots, pick mode, styles, downloads, headless: false |
lightpanda |
The Lightpanda binary (lightpanda on PATH, or executable) |
DOM + JavaScript without rendering: navigate, snapshot, act, evaluate/execute, console and network capture, cookies, history, dom::read |
Lightpanda is a headless browser written in Zig (V8 for JavaScript, its own
DOM, libcurl for HTTP) that drops the rendering engine on purpose: a single
binary, sub-100 ms start, around a tenth of Chrome's memory. The worker
spawns lightpanda serve on a loopback port the first time a tab needs a
page and ends it when the last live tab sleeps, the same lifecycle as the
Chromium process; cookies persist through data_dir/lightpanda/cookies.json
(written when the process exits). Element geometry is synthetic but
consistent, so clicking by ref works, and the accessibility tree names
controls from their contents.
What it cannot do, because nothing is laid out or painted: there is no
screencast, so the console's live viewport and the corner preview show a
browser::screenshot instead (Lightpanda renders that as a text-only PNG),
re-taken when the tab navigates or updates rather than on a timer; pick mode (Overlay), browser::styles::read (CSS), clear-data's
per-origin storage wipe, and recording are unavailable; file:// pages are
refused ("UnsupportedProtocol"); headful: true and
browser::sessions::attach are Chromium only. browser::doctor reports the
configured engine and the binary it resolved.
brew install lightpanda-io/browser/lightpanda
# or the nightly binary: https://github.com/lightpanda-io/browser/releases/tag/nightlyThen set engine: lightpanda in the browser configuration (Settings →
browser → Launch).
Quickstart
Start a session, read the page, act on it, then read the console:
use iii_sdk::protocol::TriggerRequest;
use iii_sdk::{register_worker, InitOptions};
use serde_json::json;
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let iii = register_worker("ws://localhost:49134", InitOptions::default());
let started = iii.trigger(TriggerRequest {
function_id: "browser::sessions::start".into(),
payload: json!({ "url": "http://localhost:3000" }),
action: None,
timeout_ms: Some(30_000),
}).await?;
let session_id = started["session_id"].as_str().unwrap();
// The page as text: an a11y outline with [ref=eN] handles.
let snapshot = iii.trigger(TriggerRequest {
function_id: "browser::snapshot".into(),
payload: json!({ "session_id": session_id }),
action: None,
timeout_ms: Some(15_000),
}).await?;
println!("{}", snapshot["tree"].as_str().unwrap());
// What did the page log? Errors only, no dump.
let console = iii.trigger(TriggerRequest {
function_id: "browser::console::read".into(),
payload: json!({ "session_id": session_id, "level": "error" }),
action: None,
timeout_ms: Some(10_000),
}).await?;
println!("{console:#}");
Ok(())
}The rest of the surface: browser::elements (visible controls as an
indexed table with n refs, plus the visible text, from one read),
browser::run (reach a goal in one call: the optional judge worker picks
each click/type/select/scroll from the table, text comes from the caller's
inputs; without a judge it returns judge_unavailable and the table),
browser::act (click/hover/type/select/press/scroll by ref or coordinates,
left/right/middle and double-click; ref targets are scrolled into view and
refused when disabled, hidden or covered; typing into a field by ref replaces
its value), browser::evaluate
(JS expression), browser::screenshot (viewable JPEG), browser::history
(back/forward/reload, surviving sleep and restarts), browser::history::list
(visited pages for a history panel), browser::find-in-page (find bar:
highlight matches, step next/previous), browser::zoom (page zoom
50-200 %), browser::pdf (print the page to a PDF), browser::downloads::list
/ browser::download / browser::download::remove (files the tab
downloaded), browser::clear-data (this site's cookies, storage, and the
cache), browser::clear-browser-data (the whole profile), browser::resize
(live viewport size / device presets), browser::cookies::list / set /
clear (import a cookie file; clear is per site), browser::network::read
(requests + failures), browser::dom::read (DOM tree with refs),
browser::styles::read / browser::styles::write (computed styles + live
inline edits, the design panel backing), and browser::sessions::list /
browser::sessions::stop. Function ids and schemas live in the code and
iii worker info browser.
File transfer functions:
| Function | Purpose |
|---|---|
browser::downloads::list |
List files the session downloaded |
browser::download |
Read one recorded download as base64 |
browser::download::remove |
Delete and forget one recorded download |
browser::upload |
Attach up to eight base64 files to exactly one input[type=file] selected by CSS |
Beyond single actions: browser::execute runs a multi-step async script in
the page — top-level await, log(...), sleep(ms), waitFor(selector), and
a state object that persists across execute calls for the session — so one
call replaces a chain of act/evaluate round-trips. browser::snapshot
accepts diff: true to return only what changed since the previous
snapshot, and reports the document generation its refs belong to (ref
names are unique per snapshot and fail closed when stale, never resolving to
a different element). browser::sessions::start accepts read_only: true
for inspection-only sessions where act/evaluate/execute/styles::write are
rejected. browser::doctor reports the environment — detected Chromium,
where it came from (chromium_source), version, capacity — with an
enable_how string for anything degraded. browser::chromium::status and
browser::chromium::install find and fetch Chromium (see
No Chrome on the machine?).
browser::sessions::attach binds a session to an already-running browser
over CDP (start Chrome with --remote-debugging-port) instead of launching
one, so it reaches the real profile with its logins and extensions. It opens
a fresh tab the session owns, or adopts an existing tab by URL substring and
releases it untouched on stop; browser::tabs::list enumerates a running
browser's tabs. Attach reaches logged-in state, so it is off unless
allow_attach is set in config, and adoption is exclusive per tab.
browser::handoff pauses a session for a step only a human can do (CAPTCHA,
2FA, payment): it mounts an in-page continue banner and blocks the call until
the human clicks it, a browser::handoff::confirm call resolves it, or the
timeout elapses, emitting browser::handoff-requested for the console to
surface. Human acknowledgment is not proof, so the caller verifies the
expected page state after it returns.
browser::recording::start / browser::recording::stop capture a session's
live viewport to a webm or mp4 file by piping the screencast through ffmpeg
(turning screencast on if needed); stop returns the path, duration, and
frame count. While screencast is active a human watching the viewport also
sees a ghost cursor following the agent's clicks and a session-status badge;
both are fixed-position in-page overlays that never touch page content.
browser::doctor reports whether ffmpeg (recording) and attach mode are
available.
Scraping and HTML parsing (browser::*)
The worker also ships a native Rust port of the Scrapling surface: 19 functions covering HTTP and browser fetching, screenshots, persistent sessions, crawling, and — the part that needs no browser at all — parsing HTML you already have.
Start with the parse functions: they work on any HTML string with no browser or network. Adaptive CSS/XPath/extract calls are the exception to statelessness: they persist relocation identities in the configured SQLite database. They pair naturally with the session functions above (navigate, read the page, then parse it), but they don't need one.
iii trigger browser::css --payload '{
"html": "<ul><li><a class=\"product\" href=\"/sku/1\">Widget</a></li><li><a class=\"product\" href=\"/sku/2\">Gadget</a></li></ul>",
"query": "a.product",
"attr": "href",
"first": true
}'
# → { "result": "/sku/1" }first defaults to false, in which case result is an array of every match
instead of just the first.
iii trigger browser::extract --payload '{
"html": "<div class=\"card\"><h3>Widget</h3><span class=\"price\">$19.99</span><a href=\"/sku/1\">buy</a></div>",
"selectors": [
{ "name": "title", "css": "h3" },
{ "name": "price", "css": ".price" },
{ "name": "url", "css": "a", "attr": "href" }
]
}'
# → { "extracted": { "title": "Widget", "price": "$19.99", "url": "/sku/1" } }The 10 parse functions: extract, css, xpath, regex, find,
find-by-text, find-by-regex, find-similar, describe, to-markdown.
Non-adaptive parsing has no operator-tunable defaults. The fixed limit,
find / find-by-text / find-by-regex capping
at 100 items per call (limit clamps to [0, 100]), mirrors the python
worker's hardcoded cap.
Fetching, sessions and crawl
Nine more functions go out to the network. They share one response envelope —
{status, url, headers, cookies, encoding} plus, on request, extracted
(from selectors), content+format (markdown/text) and html — so the
parse layer above is reachable inline, without a second call.
Three fetch tiers, cheapest first; escalate only when the cheaper one fails:
| engine | use when | |
|---|---|---|
fetch |
safe: reqwest/rustls; compat: frozen curl-impersonate | static pages, APIs — no browser, fastest |
dynamic-fetch |
frozen Chrome over raw CDP | the page needs JavaScript to render |
stealthy-fetch |
frozen Chrome with the Patchright command/launch sequence | the site sniffs for automation |
iii trigger browser::fetch --json '{
"url": "https://example.com/",
"selectors": [{ "name": "title", "css": "h1" }],
"format": "text"
}'
# → { "status": 200, "url": "...", "extracted": { "title": "Example Domain" }, ... }All three take a single url or a bulk urls list (bulk returns
{results: [...]}, where a failed URL contributes {url, error} instead of
sinking the batch). dynamic-fetch and stealthy-fetch additionally accept
wait_selector (+ wait_selector_state), network_idle, and wait.
browser::screenshot-url captures a page as image content blocks the console renders
inline — downscaled to 1024px wide and split into at most six 1536px tiles,
with the caption saying so when a page is taller than that.
session-open / session-fetch / session-close / session-list keep state
in a private Scrapling registry. HTTP sessions retain one cookie jar/transport;
dynamic and stealthy sessions retain one browser process and context. All use
UUID4 hex ids and serialize requests FIFO per session. They never appear in
browser::sessions::list, and interactive ids are not accepted. One-shot
browser calls get a fresh process/profile; retries get a fresh page in that
process. Compat mode supports request proxies, remote cdp_url, and
solve_cloudflare on stealthy calls.
crawl walks links breadth-first from start_urls, extracting per page. It
stays on the seed domain by default (www. folded), strips URL fragments when
deduping, and stops at max_pages (20) or max_depth (2). Every page is
emitted on a stream; the RPC response carries only a ≤10-item sample plus the
stream name and group id to read the rest with stream::on.
These functions take a caller-supplied URL, so they are an SSRF surface.
Safe mode rejects caller proxies and checks every connection against private,
loopback, link-local (including cloud metadata), CGNAT, multicast and reserved
ranges. Set browser.scrapling.allow_loopback: true to scrape a local dev
server; every other private range stays blocked. Compat mode intentionally
reproduces the Python wrapper's unrestricted network behavior and should be
enabled only for trusted calls. All nine functions remain at the
needs_approval default in iii-permissions.yaml, unlike the ten parse
functions.
The guarantee differs by tier, and the difference is worth knowing:
fetch(HTTP) — checked before every hop. Redirects are followed by hand precisely so each hop is validated before the request is made, and each connection is pinned to the address that was validated, closing the DNS rebinding window between check and connect.AuthorizationandCookieare dropped on a cross-origin redirect, as curl has done since CVE-2018-1000007.- Browser tiers — checked at the socket boundary. Safe-mode Chrome is forced through an in-process HTTP/CONNECT gate. The gate resolves, checks, and pins every destination before dialing, including redirect destinations; direct bypass, QUIC and WebRTC are disabled.
Two more safe-mode limits worth stating: response bodies are bounded at 32 MiB
whether or not the server declares a content length, and a fetch call is
capped at three times its timeout in total. Compat mode preserves the frozen
worker's unbounded response and retry/redirect quirks.
Compatibility modes and certification
Request/response schemas are golden-pinned to the Python wrapper this
surface replaced. Every call is browser::: the wrapper's
scrapling::screenshot is browser::screenshot-url here, while
browser::screenshot is the interactive session screenshot, and crawl
streams default to browser::crawl.
security_mode: safe is the default. It keeps SSRF checks and resource
ceilings, refuses network options the safe engine cannot enforce, rejects
verify: false, and bounds adaptive storage. security_mode: compat is only
eligible on Tier-1 Linux x86_64/aarch64 builds produced with the certified
curl-impersonate and Chromium artifacts. Other targets reject compat instead
of silently degrading. Eligibility is not a claim that an arbitrary local
build is certified: builds without the frozen artifacts return a capability
error, and callers should keep using safe mode.
The parser/query core, CSS-to-XPath translation, XPath 1.0 evaluation, Python
regex behavior, Markdown conversion, selector generation, and adaptive
relocation are repository-owned compatibility implementations covered by
exact differential fixtures. Adaptive queries persist element identities in
SQLite at adaptive_storage_path; parse functions remain auto-allowed, so
operators should treat that path as durable worker state. Safe mode enforces
adaptive_max_bytes (256 MiB by default) and rolls back a write that would
exceed it. Compat mode keeps the frozen worker's unbounded behavior.
Safe HTTP uses the bounded native engine. Compat HTTP is linked to the frozen curl-impersonate archive; compat browser calls use the certified Chrome build through raw pipe/WebSocket CDP and reproduce the frozen Playwright/Patchright sequences. Persistent browser sessions, proxy rotation, remote CDP, Cloudflare handling and screenshot transforms use that same private runtime. Certified builds fail when pinned artifacts are absent or mismatched; there is no silent fallback from compat to safe.
The standalone scrapling worker was the oracle and the production
fallback during rollout. It has been removed, and with it the Python
differentials that compared the two implementations call by call.
The parse goldens
tests/golden/schemas/browser.*.json and tests/golden/behavior/** are the
frozen record of what the Python implementation answered, captured while both
ran side by side. They are no longer regenerable — the generator ran against
that implementation — so they are now ordinary regression fixtures: a test
failure means this worker's behavior moved, and the fixture is only ever
updated by hand, deliberately, with the change explained.
Configuration
Stored in the configuration worker under the browser key. data_dir is
read at startup; engine, executable, headless, and the viewport apply
the next time the browser process launches (the first live tab after boot,
or after every tab went to sleep). Scrapling settings live in an isolated nested
block: bulk/default policy can be read per call, while the session cap, idle
timeout, and adaptive database path are snapshotted at worker startup.
Restart after changing a startup-snapshotted value.
browser:
engine: chromium # chromium | lightpanda (see Engines above)
executable: '' # empty = auto-detect Chrome/Chromium/Edge (then the browser::chromium::install copy), or `lightpanda` on PATH
data_dir: ./data/browser # profile/ (cookies, logins), downloads/, tabs.json; startup setting
headless: true # false shows a real window locally
max_sessions: 4 # tabs with a page open at once; the LRU unwatched tab sleeps past it
console_buffer: 500 # per-session console ring buffer (entries)
network_buffer: 500 # per-session network ring buffer (entries)
viewport_width: 1280
viewport_height: 800
default_timeout_ms: 30000 # navigation/act/evaluate default
max_timeout_ms: 120000 # ceiling; caller timeout_ms clamped DOWN to this
inactive_after_ms: 1800000 # unused, unwatched tabs sleep after this (incognito closes); 0 disables
screenshot_quality: 60 # JPEG quality 1-100
allowed_schemes: [http, https, file] # `file` lets a local document be rendered; see below
max_snapshot_nodes: 2000 # a11y outline size cap
default_origin_policy: # omitted fields default to allow
access: allow
downloads: allow
uploads: allow
scripting: allow
origin_policies:
'https://app.example.com:8443':
uploads: deny
app.example.com:
scripting: deny
allow_history_access: true
allow_cookie_import: true
allow_attach: false # true = allow sessions::attach into a running browser's real profile
scrapling:
security_mode: safe # safe | compat; compat is Tier-1 certified builds only
chromium_executable: '' # certified Chrome path; empty = discovery
allow_loopback: false # true = permit 127.0.0.1 / ::1 in outbound calls
defaults:
impersonate: chrome
headless: true
network_idle: false
proxy: ''
include_html: false
max_bulk_concurrency: 5
max_sessions: 8
session_idle_timeout_s: 900
adaptive_storage_path: data/scrapling/elements.db # relative to III_COMPOSE_DIR
adaptive_max_bytes: 268435456 # safe only; compat preserves the unbounded wrapper behaviorfile is on the default scheme list so a local document can be opened and
rendered, which is how document::ocr gets pixels out of a scanned PDF. It is
worth knowing what that permits: navigation is not checked against a session's
filesystem scope the way the workers that read files directly are, so anything
that can reach browser::navigate can open any file this process can read.
Narrow the list on a shared machine.
Origin policy keys do not accept wildcards. An exact origin, including its
scheme and non-default port, wins over a bare host; a bare host matches any
scheme or port. Origin keys are URL-normalized before matching, including
lowercased hosts and removal of explicit default ports; bare-host keys match
case-insensitively. URLs with no matching key use default_origin_policy.
Each policy field defaults to allow when omitted.
Sessions started while any origin policy is configured reload the policy on every top-document request, so edits apply to their later navigations. A session started with no origin policy does not enable interception; adding the first policy later applies the navigation gate to new sessions.
The compatibility fields are part of the stable configuration surface. Non-Tier-1 or artifact-free builds retain safe mode and reject compat explicitly instead of approximating it.
The declared production envelope is 4 GiB memory and 2 CPUs. Tier-1 release validation budgets for five concurrent browser processes; that is a release test envelope, not permission to exceed configured session caps.
Custom trigger types
Sibling workers (and the console UI) can subscribe to session activity. All
bindings accept an optional { "session_id": "..." } filter.
| Trigger type | Fires when | Payload to subscribers |
|---|---|---|
browser::session-started |
A tab opened and is ready | { session_id, url, headless, preview, timestamp } — preview: false when the opener passed preview: false to sessions::start (the console's own tab controls do) |
browser::session-stopped |
A tab closed for good | { session_id, reason: "stopped" | "idle" | "expired" | "crashed", timestamp } |
browser::session-updated |
A tab woke (active: true), went to sleep (active: false), or its live page's title changed (active: true) |
{ session_id, active, url, title, timestamp } |
browser::navigated |
The page committed a navigation | { session_id, url, timestamp } |
browser::console-event |
A console/log/exception entry was captured | { session_id, entry } |
browser::picked |
The human picked an element in inspect mode | { session_id, element, timestamp } |
browser::handoff-requested |
A session paused for a human step (CAPTCHA, 2FA, payment) | { session_id, handoff_id, instructions, timestamp } |
browser::frame-event |
Internal: a live screencast frame of a watched tab (console viewport plumbing) | { session_id, frame, width, height, frame_seq, timestamp } |
browser::chromium-install-progress |
A browser::chromium::install job entered a phase or moved ~1 % (at most every 250 ms) |
{ job_id, phase: "resolving" | "downloading" | "extracting" | "verifying" | "done" | "failed", version?, bytes_done, bytes_total?, path?, error?, hint?, timestamp } — not a session event: bind it without a session_id filter |
browser::console-event is high-volume; bind it with a session_id filter
and treat browser::console::read as the durable record. browser::picked
elements carry a ref that browser::act accepts directly, so a human pick
flows straight into agent action.
Element picking
browser::pick::start puts the page in DevTools inspect mode (native hover
highlight); the human's click resolves to tag, attributes, outer HTML, text,
bounds, and recent console errors, emitted as browser::picked. The pick,
hint, screencast, and frame functions are internal: console-UI plumbing, not
agent surface, and they stay out of agent tool lists.
Skills
API reference (JSON)
{
"functions": [
{
"description": "Interact with the page: click (left/right/middle, single or double), hover, type, select a native <select> option, press, scroll, or drag (press at the start point, glide to x2/y2, release). Address elements with a ref from browser::elements (`n4`), browser::snapshot (`e3`) or a pick, or raw viewport coordinates. A ref is scrolled into view first, and a click, type or select on a disabled, hidden or covered element is refused instead of landing on whatever is on top. Typing into an input or textarea by ref replaces its value. Returns once the page had a moment to react.",
"metadata": {},
"name": "browser::act",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"action": {
"description": "`click`, `hover`, `type`, `select`, `press`, `scroll`, or `drag`.",
"type": "string"
},
"button": {
"default": null,
"description": "Mouse button for `click`: `left` (default), `right`, or `middle`.",
"type": [
"string",
"null"
]
},
"click_count": {
"default": null,
"description": "Clicks in the gesture: 2 double-clicks (`click` only, default 1).",
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"delta_y": {
"default": null,
"description": "Scroll distance in pixels; positive scrolls down (`scroll`).",
"format": "double",
"type": [
"number",
"null"
]
},
"key": {
"default": null,
"description": "Key name for `press`: Enter, Tab, Escape, Backspace, Delete, ArrowUp/Down/Left/Right, Home, End, PageUp, PageDown.",
"type": [
"string",
"null"
]
},
"option": {
"default": null,
"description": "Option to choose (`select`, native `<select>` only): its value or visible label.",
"type": [
"string",
"null"
]
},
"ref": {
"default": null,
"description": "Element ref from `browser::elements` (`n4`), `browser::snapshot` (`e3`), `browser::dom::read` (`d7`) or `browser::picked` (`p1`). A click, type or select by ref is refused when the element is disabled, hidden or covered. Refs die on navigation; re-read the page after.",
"type": [
"string",
"null"
]
},
"session_id": {
"type": "string"
},
"text": {
"default": null,
"description": "Text to insert (`type`). With a ref to an `<input>` or `<textarea>` it replaces the current value; otherwise it is inserted at the caret.",
"type": [
"string",
"null"
]
},
"x": {
"default": null,
"description": "Viewport x, when acting by coordinates instead of ref.",
"format": "double",
"type": [
"number",
"null"
]
},
"x2": {
"default": null,
"description": "Drag end x, in viewport pixels (`drag`). The start is `x`/`y` or a `ref`; the end is `x2`/`y2`.",
"format": "double",
"type": [
"number",
"null"
]
},
"y": {
"default": null,
"description": "Viewport y, when acting by coordinates instead of ref.",
"format": "double",
"type": [
"number",
"null"
]
},
"y2": {
"default": null,
"description": "Drag end y, in viewport pixels (`drag`).",
"format": "double",
"type": [
"number",
"null"
]
}
},
"required": [
"action",
"session_id"
],
"title": "ActInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"detail": {
"description": "What was done, for the transcript.",
"type": "string"
},
"ok": {
"type": "boolean"
}
},
"required": [
"detail",
"ok"
],
"title": "ActOutput",
"type": "object"
}
},
{
"description": "Download Chromium (Chrome for Testing, Stable, about 200 MB) into the worker's cache (~/.cache/iii/browser/chrome) for a machine with no Chrome or Chromium. Returns at once with a job_id; progress arrives on the browser::chromium-install-progress trigger and in browser::chromium::status. A Chrome installed on the system still takes precedence. Does nothing when a Chromium is already found, unless force=true.",
"metadata": {},
"name": "browser::chromium::install",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"force": {
"default": false,
"description": "Download even when a Chromium is already found (a system one, or the same version already downloaded).",
"type": "boolean"
}
},
"title": "ChromiumInstallInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"InstallStatus": {
"oneOf": [
{
"description": "A new background job started; follow `job_id`.",
"enum": [
"started"
],
"type": "string"
},
{
"description": "A job was already running; `job_id` is that job.",
"enum": [
"running"
],
"type": "string"
},
{
"description": "Nothing to do: a Chromium is already found (`path`, `version`).",
"enum": [
"already-installed"
],
"type": "string"
}
]
}
},
"properties": {
"job_id": {
"description": "The install job; its progress arrives on the `browser::chromium-install-progress` trigger and in `browser::chromium::status` (`job`). Absent for `already-installed`.",
"type": [
"string",
"null"
]
},
"path": {
"type": [
"string",
"null"
]
},
"status": {
"$ref": "#/definitions/InstallStatus"
},
"version": {
"type": [
"string",
"null"
]
}
},
"required": [
"status"
],
"title": "ChromiumInstallOutput",
"type": "object"
}
},
{
"description": "Whether this machine has a Chromium the worker can launch: the binary, its version, where it came from (config, $CHROME, a system install, the copy browser::chromium::install downloaded, or a Playwright/Puppeteer cache), every place searched, whether a download is available here, and the latest install job. Read-only; never starts a browser.",
"metadata": {},
"name": "browser::chromium::status",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "ChromiumStatusInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"InstallProgress": {
"description": "`browser::chromium-install-progress` — one install job moved: a new phase, or about 1 % more bytes (at most every 250 ms). Also the `job` field of `browser::chromium::status`.",
"properties": {
"bytes_done": {
"description": "Bytes downloaded (downloading) or unpacked (extracting) so far.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"bytes_total": {
"description": "Total for `bytes_done`, when known.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"error": {
"description": "What went wrong (failed).",
"type": [
"string",
"null"
]
},
"hint": {
"description": "How to fix it, when known (failed) — e.g. the system packages a Linux machine is missing.",
"type": [
"string",
"null"
]
},
"job_id": {
"type": "string"
},
"path": {
"description": "The installed binary (done).",
"type": [
"string",
"null"
]
},
"phase": {
"$ref": "#/definitions/Phase"
},
"timestamp": {
"format": "int64",
"type": "integer"
},
"version": {
"description": "Chrome for Testing version being installed, once known.",
"type": [
"string",
"null"
]
}
},
"required": [
"bytes_done",
"job_id",
"phase",
"timestamp"
],
"type": "object"
},
"InstallSupport": {
"description": "Whether `browser::chromium::install` can download a Chromium here.",
"properties": {
"approx_download_mb": {
"description": "Roughly how big the download is.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"dir": {
"description": "Where the download is kept (`III_BROWSER_CACHE_DIR`, else `${XDG_CACHE_HOME:-~/.cache}/iii/browser`, plus `/chrome`).",
"type": "string"
},
"platform": {
"description": "Chrome for Testing platform: `linux64`, `linux-arm64`, `mac-arm64`, `mac-x64`, or `win64`.",
"type": [
"string",
"null"
]
},
"reason": {
"description": "Why there is no download here (unsupported platforms).",
"type": [
"string",
"null"
]
},
"supported": {
"description": "A Chrome for Testing build exists for this platform.",
"type": "boolean"
}
},
"required": [
"approx_download_mb",
"dir",
"supported"
],
"type": "object"
},
"Phase": {
"oneOf": [
{
"description": "Reading the Chrome for Testing manifest.",
"enum": [
"resolving"
],
"type": "string"
},
{
"description": "Streaming the zip; `bytes_done`/`bytes_total` are download bytes.",
"enum": [
"downloading"
],
"type": "string"
},
{
"description": "Unpacking; `bytes_done`/`bytes_total` are unpacked bytes.",
"enum": [
"extracting"
],
"type": "string"
},
{
"description": "Running the new binary with `--version`.",
"enum": [
"verifying"
],
"type": "string"
},
{
"description": "Installed; `path` and `version` are set.",
"enum": [
"done"
],
"type": "string"
},
{
"description": "Stopped; `error` (and maybe `hint`) say why.",
"enum": [
"failed"
],
"type": "string"
}
]
},
"Source": {
"description": "Where the resolved binary came from.",
"oneOf": [
{
"description": "The worker config `executable`.",
"enum": [
"configured"
],
"type": "string"
},
{
"description": "The `CHROME` environment variable.",
"enum": [
"env"
],
"type": "string"
},
{
"description": "A system Chrome/Chromium/Edge.",
"enum": [
"system"
],
"type": "string"
},
{
"description": "The copy `browser::chromium::install` downloaded.",
"enum": [
"managed"
],
"type": "string"
},
{
"description": "A Playwright-installed Chromium.",
"enum": [
"playwright"
],
"type": "string"
},
{
"description": "A Puppeteer-installed Chrome.",
"enum": [
"puppeteer"
],
"type": "string"
}
]
}
},
"properties": {
"engine": {
"description": "The configured interactive engine (`chromium` or `lightpanda`). Scrapling's browser tiers use Chromium either way.",
"type": "string"
},
"found": {
"description": "A Chromium the worker can launch exists.",
"type": "boolean"
},
"install": {
"$ref": "#/definitions/InstallSupport"
},
"job": {
"anyOf": [
{
"$ref": "#/definitions/InstallProgress"
},
{
"type": "null"
}
],
"description": "This worker's latest install job, running or finished."
},
"path": {
"description": "The binary sessions launch.",
"type": [
"string",
"null"
]
},
"sandboxed": {
"description": "False when sessions launch this Chromium with `--no-sandbox` (a cache copy on a Linux that blocks the user-namespace sandbox).",
"type": [
"boolean",
"null"
]
},
"searched": {
"description": "The places looked, in order.",
"items": {
"type": "string"
},
"type": "array"
},
"source": {
"anyOf": [
{
"$ref": "#/definitions/Source"
},
{
"type": "null"
}
],
"description": "Where it came from: `configured`, `env`, `system`, `managed` (the download), `playwright`, or `puppeteer`."
},
"version": {
"description": "`<binary> --version`, first line.",
"type": [
"string",
"null"
]
}
},
"required": [
"engine",
"found",
"install",
"searched"
],
"title": "ChromiumStatusOutput",
"type": "object"
}
},
{
"description": "Read the session's captured console: console.* calls, uncaught exceptions, and browser-level log entries. Filter with pattern/level and page with since_seq instead of dumping everything.",
"metadata": {},
"name": "browser::console::read",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"level": {
"default": null,
"description": "Only entries at this level: `log`, `info`, `warning`, `error`, `debug`, `exception`. `error` also matches `exception`.",
"type": [
"string",
"null"
]
},
"limit": {
"default": null,
"description": "Maximum entries returned, newest kept (default 100).",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"pattern": {
"default": null,
"description": "Regex applied to entry text. Use it: dumping an unfiltered console wastes the caller's context.",
"type": [
"string",
"null"
]
},
"session_id": {
"type": "string"
},
"since_seq": {
"default": null,
"description": "Only entries with `seq` greater than this; resume from the cursor returned as `last_seq`.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"required": [
"session_id"
],
"title": "ConsoleReadInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ConsoleEntry": {
"description": "One captured console/log/exception entry.",
"properties": {
"level": {
"description": "`log`, `info`, `warning`, `error`, `debug`, or `exception`.",
"type": "string"
},
"seq": {
"description": "Monotonic per-session cursor; pass back as `since_seq`.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"source": {
"description": "`url:line` of the emitting frame, when known.",
"type": [
"string",
"null"
]
},
"text": {
"type": "string"
},
"timestamp": {
"format": "int64",
"type": "integer"
}
},
"required": [
"level",
"seq",
"text",
"timestamp"
],
"type": "object"
}
},
"properties": {
"dropped": {
"description": "Entries evicted from the ring buffer since session start.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"entries": {
"items": {
"$ref": "#/definitions/ConsoleEntry"
},
"type": "array"
},
"last_seq": {
"description": "Cursor for the next `since_seq`.",
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"dropped",
"entries",
"last_seq"
],
"title": "ConsoleReadOutput",
"type": "object"
}
},
{
"description": "Diagnose the browser environment: which Chromium the worker would launch, its version, session capacity, and any degraded capability with how to enable it. Never starts a browser.",
"metadata": {},
"name": "browser::doctor",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "DoctorInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"DoctorIssue": {
"description": "One degraded capability plus the way to enable it.",
"properties": {
"enable_how": {
"type": "string"
},
"what": {
"type": "string"
}
},
"required": [
"enable_how",
"what"
],
"type": "object"
},
"Source": {
"description": "Where the resolved binary came from.",
"oneOf": [
{
"description": "The worker config `executable`.",
"enum": [
"configured"
],
"type": "string"
},
{
"description": "The `CHROME` environment variable.",
"enum": [
"env"
],
"type": "string"
},
{
"description": "A system Chrome/Chromium/Edge.",
"enum": [
"system"
],
"type": "string"
},
{
"description": "The copy `browser::chromium::install` downloaded.",
"enum": [
"managed"
],
"type": "string"
},
{
"description": "A Playwright-installed Chromium.",
"enum": [
"playwright"
],
"type": "string"
},
{
"description": "A Puppeteer-installed Chrome.",
"enum": [
"puppeteer"
],
"type": "string"
}
]
}
},
"properties": {
"active_sessions": {
"description": "Tabs with a page open right now.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"allow_cookie_import": {
"type": "boolean"
},
"allow_history_access": {
"type": "boolean"
},
"allowed_schemes": {
"items": {
"type": "string"
},
"type": "array"
},
"attach_enabled": {
"description": "Whether attach mode is enabled (allow_attach).",
"type": "boolean"
},
"browser_running": {
"description": "Whether the shared Chromium process is running.",
"type": "boolean"
},
"chromium_path": {
"description": "The engine binary the worker would launch (a Chromium/Chrome, or the `lightpanda` binary). The field name predates the `engine` setting.",
"type": [
"string",
"null"
]
},
"chromium_source": {
"anyOf": [
{
"$ref": "#/definitions/Source"
},
{
"type": "null"
}
],
"description": "Where the Chromium came from: `configured`, `env`, `system`, `managed` (downloaded by browser::chromium::install), `playwright`, or `puppeteer`. Absent for Lightpanda or when nothing was found."
},
"chromium_version": {
"description": "`<binary> --version`, first line.",
"type": [
"string",
"null"
]
},
"configured_origin_policies": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"data_dir": {
"description": "Where the profile, downloads and tab list live.",
"type": "string"
},
"default_origin_policy_set": {
"type": "boolean"
},
"engine": {
"description": "Configured engine: `chromium` or `lightpanda`.",
"type": "string"
},
"headless_default": {
"type": "boolean"
},
"issues": {
"items": {
"$ref": "#/definitions/DoctorIssue"
},
"type": "array"
},
"max_sessions": {
"description": "Live-tab cap (`max_sessions`).",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"ok": {
"description": "True when sessions can start right now.",
"type": "boolean"
},
"open_tabs": {
"description": "Every tab, live or asleep.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"recording_available": {
"description": "Whether ffmpeg is on PATH, which browser::recording requires.",
"type": "boolean"
}
},
"required": [
"active_sessions",
"allow_cookie_import",
"allow_history_access",
"allowed_schemes",
"attach_enabled",
"browser_running",
"configured_origin_policies",
"data_dir",
"default_origin_policy_set",
"engine",
"headless_default",
"issues",
"max_sessions",
"ok",
"open_tabs",
"recording_available"
],
"title": "DoctorOutput",
"type": "object"
}
},
{
"description": "Read the DOM as a tree of tags with id/class and refs. Structure-oriented complement to browser::snapshot; read deep subtrees by passing a ref.",
"metadata": {},
"name": "browser::dom::read",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"depth": {
"default": null,
"description": "Levels of children to include (default 3).",
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"ref": {
"default": null,
"description": "Subtree root from an earlier ref (`e3`/`p1`) or dom node. Omit for the document root.",
"type": [
"string",
"null"
]
},
"session_id": {
"type": "string"
}
},
"required": [
"session_id"
],
"title": "DomReadInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"DomNode": {
"description": "One DOM node in the outline. `ref` resolves in `browser::act`, `browser::styles::read`, and `browser::styles::write`.",
"properties": {
"child_count": {
"description": "Total children in the document, which may exceed `children` returned at this depth.",
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"children": {
"items": {
"$ref": "#/definitions/DomNode"
},
"type": "array"
},
"classes": {
"type": [
"string",
"null"
]
},
"id": {
"type": [
"string",
"null"
]
},
"ref": {
"type": "string"
},
"tag": {
"description": "Lowercase tag (`div`, `button`) or node name (`#text`).",
"type": "string"
},
"text": {
"description": "Trimmed text content for text nodes.",
"type": [
"string",
"null"
]
}
},
"required": [
"child_count",
"ref",
"tag"
],
"type": "object"
}
},
"properties": {
"root": {
"$ref": "#/definitions/DomNode"
},
"truncated": {
"description": "True when the node cap cut the tree short; read a subtree via `ref`.",
"type": "boolean"
}
},
"required": [
"root",
"truncated"
],
"title": "DomReadOutput",
"type": "object"
}
},
{
"description": "The page's visible, enabled controls in the viewport as an indexed table (ref, role, label, current value, checked/expanded state, what browser::act can do with each, and <select> options), plus the visible text, from one read. Cheaper and flatter than browser::snapshot: use it to drive forms and read what an action changed. Refs (`n4`) stay valid while the element stays in the document.",
"metadata": {},
"name": "browser::elements",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"session_id": {
"type": "string"
}
},
"required": [
"session_id"
],
"title": "ElementsInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Element": {
"description": "One control. `operations` says what `browser::act` can do with it.",
"properties": {
"autocomplete": {
"description": "The field's `autocomplete` hint (`username`, `current-password`, …).",
"type": [
"string",
"null"
]
},
"checked": {
"type": [
"string",
"null"
]
},
"expanded": {
"type": [
"string",
"null"
]
},
"input_type": {
"description": "`<input>` type when not `text` (`email`, `password`, `tel`, …).",
"type": [
"string",
"null"
]
},
"label": {
"description": "Accessible name (aria-labelledby, aria-label, label, text, title, or placeholder).",
"type": "string"
},
"more_options": {
"description": "Options beyond the listed 50; `select` still accepts them by label.",
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"name": {
"description": "The field's `name` (or `id`).",
"type": [
"string",
"null"
]
},
"operations": {
"description": "`click`, `type` and/or `select`.",
"items": {
"type": "string"
},
"type": "array"
},
"options": {
"description": "Enabled `<select>` option labels (first 50), for `act` `select`.",
"items": {
"type": "string"
},
"type": "array"
},
"ref": {
"description": "`n` ref `browser::act` accepts; valid while the element stays in this document.",
"type": "string"
},
"role": {
"type": "string"
},
"selected": {
"type": [
"string",
"null"
]
},
"value": {
"description": "Current value of a field or dropdown; a set password reads `(set)`.",
"type": [
"string",
"null"
]
}
},
"required": [
"label",
"operations",
"ref",
"role"
],
"type": "object"
}
},
"properties": {
"busy": {
"default": false,
"description": "The page is still loading: the document, a region marked `aria-busy`, or (in `browser::run`) requests an action started that had not finished when the run stopped waiting.",
"type": "boolean"
},
"can_scroll_down": {
"type": "boolean"
},
"can_scroll_up": {
"type": "boolean"
},
"elements": {
"description": "Visible, enabled controls inside the viewport, in document order.",
"items": {
"$ref": "#/definitions/Element"
},
"type": "array"
},
"generation": {
"default": 0,
"description": "Document generation; navigation advances it and every `n` ref dies.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"omitted": {
"description": "Controls past the 250-element cap, not listed.",
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"text": {
"description": "Visible text in the viewport, up to 6000 characters.",
"type": "string"
},
"title": {
"type": "string"
},
"url": {
"type": "string"
}
},
"required": [
"can_scroll_down",
"can_scroll_up",
"elements",
"omitted",
"text",
"title",
"url"
],
"title": "ElementsOutput",
"type": "object"
}
},
{
"description": "Evaluate a JavaScript expression in the page and return its completion value. Use for reads the snapshot can't express; prefer browser::act for interactions.",
"metadata": {},
"name": "browser::evaluate",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"expression": {
"description": "JavaScript expression evaluated in the page. The completion value is returned by value; wrap statements in an IIFE.",
"type": "string"
},
"session_id": {
"type": "string"
},
"timeout_ms": {
"default": null,
"description": "Upper bound on evaluation; clamped to `max_timeout_ms`.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"required": [
"expression",
"session_id"
],
"title": "EvaluateInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"error": {
"description": "Exception text when not `ok`.",
"type": [
"string",
"null"
]
},
"ok": {
"type": "boolean"
},
"value": {
"description": "JSON completion value when `ok`."
}
},
"required": [
"ok"
],
"title": "EvaluateOutput",
"type": "object"
}
},
{
"description": "Run a multi-step async JavaScript script in the page: top-level await and return work, with log(...), sleep(ms), waitFor(selector), and a state object that persists across execute calls for the session. One call replaces a chain of act/evaluate round-trips; returns { result, logs, state }.",
"metadata": {},
"name": "browser::execute",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"code": {
"description": "Async JavaScript body run in the page; `state`, `log(...)`, `sleep(ms)` and `waitFor(selector, { timeout })` are in scope. Return plain JSON.",
"type": "string"
},
"session_id": {
"type": "string"
},
"timeout_ms": {
"default": null,
"description": "Upper bound on the run; clamped to `max_timeout_ms`.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"required": [
"code",
"session_id"
],
"title": "ExecuteInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"error": {
"description": "Exception text when not `ok`. A \"context destroyed\" error usually means the script navigated; split the script at the navigation.",
"type": [
"string",
"null"
]
},
"logs": {
"description": "`log(...)` output collected during the run, in order.",
"items": {
"type": "string"
},
"type": "array"
},
"ok": {
"type": "boolean"
},
"result": {
"description": "The script's return value when `ok`."
},
"state": {
"description": "Session state after the run; the next execute call sees this as `state`."
}
},
"required": [
"logs",
"ok",
"state"
],
"title": "ExecuteOutput",
"type": "object"
}
},
{
"description": "Pause a session for a step only a human can do (CAPTCHA, 2FA, payment): show an in-page continue banner and block until the human clicks it, a browser::handoff::confirm call resolves it, or the timeout elapses. Human acknowledgment is not proof — verify the expected page state after it returns.",
"metadata": {},
"name": "browser::handoff",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"instructions": {
"description": "What the human must do before the call continues. Shown in the in-page banner and the handoff-requested event.",
"type": "string"
},
"session_id": {
"type": "string"
},
"timeout_ms": {
"default": null,
"description": "Give up after this many ms and return `via: \"timeout\"` (clamped to `max_timeout_ms`); set generously, a human is slow.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"required": [
"instructions",
"session_id"
],
"title": "HandoffInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"confirmed": {
"description": "True when a human confirmed; false when the wait timed out.",
"type": "boolean"
},
"handoff_id": {
"type": "string"
},
"url": {
"description": "The page URL when the wait ended, so the caller can verify the step actually landed (human acknowledgment is not proof).",
"type": "string"
},
"via": {
"description": "How the confirmation arrived: `in_page`, `confirm_call`, or `timeout`.",
"type": "string"
}
},
"required": [
"confirmed",
"handoff_id",
"url",
"via"
],
"title": "HandoffOutput",
"type": "object"
}
},
{
"description": "Resolve a paused browser::handoff by handoff_id, or the one pending handoff for a session_id. The console calls this when the human confirms outside the page.",
"metadata": {},
"name": "browser::handoff::confirm",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"handoff_id": {
"default": null,
"description": "Confirm a specific handoff by id. Omit to confirm the one pending handoff for `session_id`.",
"type": [
"string",
"null"
]
},
"session_id": {
"default": null,
"type": [
"string",
"null"
]
}
},
"title": "HandoffConfirmInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"handoff_id": {
"type": [
"string",
"null"
]
},
"ok": {
"description": "True when a pending handoff matched and was resolved.",
"type": "boolean"
}
},
"required": [
"ok"
],
"title": "HandoffConfirmOutput",
"type": "object"
}
},
{
"description": "Go back, go forward, or reload the tab's page. History survives the tab sleeping and the worker restarting. Back/forward at the history edge is a no-op with moved=false.",
"metadata": {},
"name": "browser::history",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"action": {
"description": "`back`, `forward`, or `reload`.",
"type": "string"
},
"session_id": {
"type": "string"
}
},
"required": [
"action",
"session_id"
],
"title": "HistoryInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"moved": {
"description": "False when back/forward had no entry to move to.",
"type": "boolean"
},
"ok": {
"type": "boolean"
},
"url": {
"description": "URL after the action. `back`/`forward` at the history edge is a no-op with ok=true.",
"type": "string"
}
},
"required": [
"moved",
"ok",
"url"
],
"title": "HistoryOutput",
"type": "object"
}
},
{
"description": "Navigate a tab to a URL and wait for the page to load. Like a browser, a network failure or an empty HTTP error response leaves Chromium's error page in the tab and is reported in `error` rather than failing the call. Element refs from earlier snapshots are invalidated by navigation.",
"metadata": {},
"name": "browser::navigate",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"session_id": {
"type": "string"
},
"timeout_ms": {
"default": null,
"description": "Upper bound on the navigation wait; clamped to `max_timeout_ms`.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"url": {
"description": "Absolute URL; scheme must be on the configured allowlist.",
"type": "string"
}
},
"required": [
"session_id",
"url"
],
"title": "NavigateInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"error": {
"description": "Chromium's error text when the navigation ended on the browser's own error page: a network failure (`net::ERR_NAME_NOT_RESOLVED`), or an empty HTTP error response (`net::ERR_HTTP_RESPONSE_CODE_FAILURE`, a 4xx/5xx with no body). Like a browser, the tab still shows that page and stays usable.",
"type": [
"string",
"null"
]
},
"ok": {
"description": "False when the tab shows Chromium's error page instead of the site; see `error`.",
"type": "boolean"
},
"timed_out": {
"description": "True when the load event did not fire inside the timeout; the page may still be usable; snapshot to check.",
"type": "boolean"
},
"title": {
"type": [
"string",
"null"
]
},
"url": {
"description": "URL after redirects.",
"type": "string"
}
},
"required": [
"ok",
"timed_out",
"url"
],
"title": "NavigateOutput",
"type": "object"
}
},
{
"description": "Read the session's captured network requests (method, URL, status, failures). failed_only=true is the fast path for 'what broke'.",
"metadata": {},
"name": "browser::network::read",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"failed_only": {
"default": null,
"description": "Only failed requests (network error or status >= 400).",
"type": [
"boolean",
"null"
]
},
"limit": {
"default": null,
"description": "Maximum entries returned, newest kept (default 100).",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"pattern": {
"default": null,
"description": "Regex applied to the request URL.",
"type": [
"string",
"null"
]
},
"session_id": {
"type": "string"
},
"since_seq": {
"default": null,
"description": "Only entries with `seq` greater than this.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"required": [
"session_id"
],
"title": "NetworkReadInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"NetworkEntry": {
"description": "One captured network request.",
"properties": {
"error": {
"type": [
"string",
"null"
]
},
"failed": {
"type": "boolean"
},
"method": {
"type": "string"
},
"mime_type": {
"type": [
"string",
"null"
]
},
"seq": {
"description": "Monotonic per-session cursor; pass back as `since_seq`.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"status": {
"format": "int64",
"type": [
"integer",
"null"
]
},
"timestamp": {
"format": "int64",
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"failed",
"method",
"seq",
"timestamp",
"url"
],
"type": "object"
}
},
"properties": {
"dropped": {
"description": "Entries evicted from the ring buffer since session start.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"entries": {
"items": {
"$ref": "#/definitions/NetworkEntry"
},
"type": "array"
},
"last_seq": {
"description": "Cursor for the next `since_seq`.",
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"dropped",
"entries",
"last_seq"
],
"title": "NetworkReadOutput",
"type": "object"
}
},
{
"description": "Drive the page toward a goal in one call (fill and submit a form, log in, pick filters, step through a wizard): each step reads the visible controls, asks the judge worker which operation (click, type, select, scroll, wait, done, blocked) and which element, acts, and waits for the requests the action started, until the judge says done or blocked, a field needs text you did not give in `inputs`, nothing changes three times, or the step/time budget runs out. State the goal as the end state to reach (\"logged in: the dashboard shows\"), not the clicks. Returns the steps taken and the final page table; read it before reporting success, done is the judge's reading, not proof (a login can end on an error). Without a judge worker it returns status judge_unavailable with the page table: drive with browser::elements and browser::act instead.",
"metadata": {},
"name": "browser::run",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"goal": {
"description": "The end state to reach, as the page would show it (\"logged in: the dashboard shows\", \"the bug is saved and listed\"), not the clicks to make. Name every value the outcome needs.",
"type": "string"
},
"inputs": {
"additionalProperties": {
"type": "string"
},
"default": {},
"description": "Text for fields the goal needs filled: `{\"email\": \"…\", \"password\": \"…\"}`. A key reaches a field by its `n` ref, by its label (case, accents and punctuation ignored: `email` for \"E-mail\"), by what the field is (input type, `name`, `autocomplete`: `password` for a password field labelled \"Senha\"), or as the only key the label contains. Only the keys are shown to the judge.",
"type": "object"
},
"max_steps": {
"default": null,
"description": "Most actions to execute (default 10, at most 60).",
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"session_id": {
"type": "string"
},
"timeout_ms": {
"default": null,
"description": "Whole-run budget in ms (default 30000), clamped to `max_timeout_ms`.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"required": [
"goal",
"session_id"
],
"title": "RunInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Element": {
"description": "One control. `operations` says what `browser::act` can do with it.",
"properties": {
"autocomplete": {
"description": "The field's `autocomplete` hint (`username`, `current-password`, …).",
"type": [
"string",
"null"
]
},
"checked": {
"type": [
"string",
"null"
]
},
"expanded": {
"type": [
"string",
"null"
]
},
"input_type": {
"description": "`<input>` type when not `text` (`email`, `password`, `tel`, …).",
"type": [
"string",
"null"
]
},
"label": {
"description": "Accessible name (aria-labelledby, aria-label, label, text, title, or placeholder).",
"type": "string"
},
"more_options": {
"description": "Options beyond the listed 50; `select` still accepts them by label.",
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"name": {
"description": "The field's `name` (or `id`).",
"type": [
"string",
"null"
]
},
"operations": {
"description": "`click`, `type` and/or `select`.",
"items": {
"type": "string"
},
"type": "array"
},
"options": {
"description": "Enabled `<select>` option labels (first 50), for `act` `select`.",
"items": {
"type": "string"
},
"type": "array"
},
"ref": {
"description": "`n` ref `browser::act` accepts; valid while the element stays in this document.",
"type": "string"
},
"role": {
"type": "string"
},
"selected": {
"type": [
"string",
"null"
]
},
"value": {
"description": "Current value of a field or dropdown; a set password reads `(set)`.",
"type": [
"string",
"null"
]
}
},
"required": [
"label",
"operations",
"ref",
"role"
],
"type": "object"
},
"ElementsOutput": {
"properties": {
"busy": {
"default": false,
"description": "The page is still loading: the document, a region marked `aria-busy`, or (in `browser::run`) requests an action started that had not finished when the run stopped waiting.",
"type": "boolean"
},
"can_scroll_down": {
"type": "boolean"
},
"can_scroll_up": {
"type": "boolean"
},
"elements": {
"description": "Visible, enabled controls inside the viewport, in document order.",
"items": {
"$ref": "#/definitions/Element"
},
"type": "array"
},
"generation": {
"default": 0,
"description": "Document generation; navigation advances it and every `n` ref dies.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"omitted": {
"description": "Controls past the 250-element cap, not listed.",
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"text": {
"description": "Visible text in the viewport, up to 6000 characters.",
"type": "string"
},
"title": {
"type": "string"
},
"url": {
"type": "string"
}
},
"required": [
"can_scroll_down",
"can_scroll_up",
"elements",
"omitted",
"text",
"title",
"url"
],
"type": "object"
},
"NeedsText": {
"properties": {
"input_keys": {
"description": "The `inputs` keys the run was given, none of which matched this field.",
"items": {
"type": "string"
},
"type": "array"
},
"label": {
"type": "string"
},
"ref": {
"type": "string"
}
},
"required": [
"input_keys",
"label",
"ref"
],
"type": "object"
},
"RunStatus": {
"oneOf": [
{
"enum": [
"max_steps",
"deadline"
],
"type": "string"
},
{
"description": "The judge chose DONE and its completion check rated the goal complete on the page. Verify on `page`; it is the judge's reading, not proof.",
"enum": [
"done"
],
"type": "string"
},
{
"description": "The judge found no operation that makes progress.",
"enum": [
"blocked"
],
"type": "string"
},
{
"description": "Three executed steps in a row changed nothing.",
"enum": [
"stalled"
],
"type": "string"
},
{
"description": "A field must be typed and no `inputs` entry matches it: type it with `browser::act` (or rerun with `inputs`), then run again.",
"enum": [
"needs_text"
],
"type": "string"
},
{
"description": "No usable judge (not deployed, no provider, or failing). Drive with `browser::elements` + `browser::act`; `page` is the current table.",
"enum": [
"judge_unavailable"
],
"type": "string"
}
]
},
"RunStep": {
"properties": {
"error": {
"description": "Why the action was refused or failed; the page was re-read and the judge asked again.",
"type": [
"string",
"null"
]
},
"judge_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"label": {
"type": [
"string",
"null"
]
},
"operation": {
"description": "CLICK, TYPE_TEXT, SELECT, SCROLL_UP, SCROLL_DOWN or WAIT.",
"type": "string"
},
"option": {
"description": "The option chosen (SELECT).",
"type": [
"string",
"null"
]
},
"page_changed": {
"type": "boolean"
},
"probability": {
"description": "The judge's probability for the executed choice.",
"format": "double",
"type": "number"
},
"ref": {
"type": [
"string",
"null"
]
}
},
"required": [
"judge_ms",
"operation",
"page_changed",
"probability"
],
"type": "object"
}
},
"properties": {
"elapsed_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"judge_requests": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"needs_text": {
"anyOf": [
{
"$ref": "#/definitions/NeedsText"
},
{
"type": "null"
}
]
},
"page": {
"allOf": [
{
"$ref": "#/definitions/ElementsOutput"
}
],
"description": "The page as the run left it (`browser::elements` shape)."
},
"reason": {
"type": [
"string",
"null"
]
},
"status": {
"$ref": "#/definitions/RunStatus"
},
"steps": {
"description": "Executed actions, in order.",
"items": {
"$ref": "#/definitions/RunStep"
},
"type": "array"
}
},
"required": [
"elapsed_ms",
"judge_requests",
"page",
"status",
"steps"
],
"title": "RunOutput",
"type": "object"
}
},
{
"description": "Internal: start the live viewport feed — frames arrive on the browser::frame-event trigger and browser::frame reads the newest. Console-UI plumbing; agents use browser::screenshot. Not an agent function.",
"metadata": {
"internal": true
},
"name": "browser::screencast::start",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"preview": {
"default": null,
"description": "A corner thumbnail rather than a pane: streams frames but does not count as a viewer for `browser::resize` fit arbitration.",
"type": [
"boolean",
"null"
]
},
"session_id": {
"type": "string"
}
},
"required": [
"session_id"
],
"title": "ScreencastStartInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Neutral acknowledgement returned by the fire-and-forget internal functions (pick start/stop/resolve, screencast start/stop): they emit their result as a trigger event, so the direct return is just `{ ok }`.",
"properties": {
"ok": {
"type": "boolean"
}
},
"required": [
"ok"
],
"title": "AckOutput",
"type": "object"
}
},
{
"description": "Screenshot the session's current page as a JPEG (or a lossless PNG with format=png). Use browser::snapshot for machine-readable structure; screenshot when layout or rendering matters.",
"metadata": {
"display": true
},
"name": "browser::screenshot",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"format": {
"default": null,
"description": "`jpeg` (default, compressed with the configured quality) or `png` (lossless; what pixel comparisons need).",
"type": [
"string",
"null"
]
},
"full_page": {
"default": null,
"description": "Capture the full scrollable page instead of the viewport.",
"type": [
"boolean",
"null"
]
},
"session_id": {
"type": "string"
}
},
"required": [
"session_id"
],
"title": "ScreenshotInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ContentBlock": {
"description": "One block of a viewable response: an image block plus a text line.",
"properties": {
"data": {
"type": [
"string",
"null"
]
},
"mime": {
"type": [
"string",
"null"
]
},
"text": {
"type": [
"string",
"null"
]
},
"type": {
"type": "string"
}
},
"required": [
"type"
],
"type": "object"
},
"ScreenshotDetails": {
"properties": {
"height": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"session_id": {
"type": "string"
},
"url": {
"type": "string"
},
"width": {
"format": "uint32",
"minimum": 0,
"type": "integer"
}
},
"required": [
"height",
"session_id",
"url",
"width"
],
"type": "object"
}
},
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"details": {
"$ref": "#/definitions/ScreenshotDetails"
}
},
"required": [
"content",
"details"
],
"title": "ScreenshotOutput",
"type": "object"
}
},
{
"description": "Attach a session to an already-running browser over CDP. Opens a fresh tab the session owns, or adopts a user tab by URL substring and releases it untouched on stop. Reaches the real profile and its logins; needs Chrome started with --remote-debugging-port and allow_attach in config.",
"metadata": {},
"name": "browser::sessions::attach",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"adopt_url_substring": {
"default": null,
"description": "Adopt the one open tab whose URL contains this substring (released untouched on stop); omit to open a fresh tab the session owns.",
"type": [
"string",
"null"
]
},
"cdp_url": {
"description": "CDP endpoint of the running browser: `http://127.0.0.1:9222` (resolved via `/json/version`) or a `ws://` debugger URL.",
"type": "string"
},
"read_only": {
"default": null,
"description": "Inspection-only session; see browser::sessions::start.",
"type": [
"boolean",
"null"
]
},
"url": {
"default": null,
"description": "URL to open in the fresh tab (ignored when adopting). Omit for about:blank.",
"type": [
"string",
"null"
]
}
},
"required": [
"cdp_url"
],
"title": "AttachInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"adopted": {
"description": "True when the session adopted an existing user tab (released, not closed, on stop); false when it opened a fresh tab it owns.",
"type": "boolean"
},
"read_only": {
"type": "boolean"
},
"session_id": {
"type": "string"
},
"url": {
"type": "string"
}
},
"required": [
"adopted",
"read_only",
"session_id",
"url"
],
"title": "AttachOutput",
"type": "object"
}
},
{
"description": "List every browser tab, live or asleep, with its current URL, title, and activity.",
"metadata": {},
"name": "browser::sessions::list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "ListInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"SessionInfo": {
"properties": {
"active": {
"description": "True while the tab has its page open. A sleeping tab (false) is listed and usable; the next call on it reopens the page at `url`.",
"type": "boolean"
},
"console_entries": {
"description": "Console entries captured since the page opened; 0 while asleep.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"created_ms": {
"format": "int64",
"type": "integer"
},
"headless": {
"type": "boolean"
},
"incognito": {
"description": "Private tab: nothing persisted, closes instead of sleeping.",
"type": "boolean"
},
"last_used_ms": {
"format": "int64",
"type": "integer"
},
"read_only": {
"type": "boolean"
},
"session_id": {
"type": "string"
},
"title": {
"type": [
"string",
"null"
]
},
"ttl_ms": {
"description": "Lifetime the tab was opened with, when any.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"url": {
"type": "string"
}
},
"required": [
"active",
"console_entries",
"created_ms",
"headless",
"incognito",
"last_used_ms",
"read_only",
"session_id",
"url"
],
"type": "object"
}
},
"properties": {
"sessions": {
"items": {
"$ref": "#/definitions/SessionInfo"
},
"type": "array"
}
},
"required": [
"sessions"
],
"title": "ListOutput",
"type": "object"
}
},
{
"description": "Open a browser tab (a session) and return its session_id. Tabs share one browser profile (cookies, logins) and stay open until stopped or until an optional ttl_ms; an unused tab sleeps and wakes on the next call. incognito=true opens a PRIVATE tab: nothing it does is saved, and inactivity closes it for good.",
"metadata": {},
"name": "browser::sessions::start",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"headful": {
"default": null,
"description": "Force a visible window, overriding the configured `headless` default. Applies when this call launches the browser process; a browser that is already running keeps its mode.",
"type": [
"boolean",
"null"
]
},
"incognito": {
"default": null,
"description": "INCOGNITO TAB. Opens the tab in a private browser context: it shares no cookies, logins, or storage with the regular tabs, nothing it does is saved to disk (no cookies, no history, no tab record), it does not come back after a restart, and inactivity closes it for good instead of putting it to sleep. Everything lives in memory for as long as the tab does. Use it for logins you do not want kept, or to see a site signed out.",
"type": [
"boolean",
"null"
]
},
"preview": {
"default": null,
"description": "Whether consoles pop a live preview of the new tab (default true). The console's own tab controls pass `false`: the page that opened the tab already shows it, in every console window. Agents leave it.",
"type": [
"boolean",
"null"
]
},
"read_only": {
"default": null,
"description": "Inspection-only session for its whole lifetime: act, evaluate, execute and styles::write are rejected; navigation and reads work.",
"type": [
"boolean",
"null"
]
},
"ttl_ms": {
"default": null,
"description": "Optional lifetime in milliseconds: the tab closes on its own this long after it opened, even while in use. Omit for a tab that stays until stopped (what the console does).",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"url": {
"default": null,
"description": "URL to open immediately. Omit to start on about:blank.",
"type": [
"string",
"null"
]
}
},
"title": "StartInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"error": {
"description": "Chromium's error text when the requested url did not load and the tab shows the browser's error page instead (a network failure, or an empty HTTP error response such as a 400). Absent when the page came up. The tab is open either way.",
"type": [
"string",
"null"
]
},
"headless": {
"type": "boolean"
},
"incognito": {
"description": "True for a private tab; see `incognito` on the request.",
"type": "boolean"
},
"read_only": {
"type": "boolean"
},
"session_id": {
"description": "Pass this to every other browser function.",
"type": "string"
},
"timed_out": {
"description": "True when the requested url's load event did not fire inside the default navigation timeout; the tab is open and the page may still be usable; snapshot to check.",
"type": "boolean"
},
"url": {
"type": "string"
}
},
"required": [
"headless",
"incognito",
"read_only",
"session_id",
"timed_out",
"url"
],
"title": "StartOutput",
"type": "object"
}
},
{
"description": "Close an interactive browser tab (a session from browser::sessions::start or attach) for good; do this when the task is done, since every tab holds a Chromium page. Idempotent: closing an unknown or already-closed tab succeeds with was_running=false. Scraping sessions from browser::session-open close with browser::session-close.",
"metadata": {},
"name": "browser::sessions::stop",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"session_id": {
"description": "Tab to close. Closing an unknown or already-closed id succeeds.",
"type": "string"
}
},
"required": [
"session_id"
],
"title": "StopInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"ok": {
"type": "boolean"
},
"was_running": {
"description": "False when the session was already gone.",
"type": "boolean"
}
},
"required": [
"ok",
"was_running"
],
"title": "StopOutput",
"type": "object"
}
},
{
"description": "Read the page as an accessibility-tree outline. Lines carry [ref=eN] handles that browser::act accepts; refs stay valid until the next navigation. Prefer this over browser::screenshot; it is cheaper and machine-readable.",
"metadata": {},
"name": "browser::snapshot",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"diff": {
"default": null,
"description": "Return only what changed since the previous snapshot; a full outline when there is no baseline (first snapshot or after a navigation).",
"type": [
"boolean",
"null"
]
},
"session_id": {
"type": "string"
}
},
"required": [
"session_id"
],
"title": "SnapshotInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"SnapshotDiff": {
"description": "Changes since the previous snapshot. Lines are compared without their `[ref=eN]` suffix (ref names are unique per snapshot); `added` lines carry current refs and are directly actionable.",
"properties": {
"added": {
"items": {
"type": "string"
},
"type": "array"
},
"removed": {
"items": {
"type": "string"
},
"type": "array"
},
"unchanged": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"added",
"removed",
"unchanged"
],
"type": "object"
}
},
"properties": {
"diff": {
"anyOf": [
{
"$ref": "#/definitions/SnapshotDiff"
},
{
"type": "null"
}
],
"description": "Present when the caller asked for `diff: true` and a baseline existed. Covers only the emitted nodes of both snapshots; check `truncated` before trusting it as a complete change set."
},
"generation": {
"description": "Document generation the refs belong to; navigation advances it and kills every ref from earlier generations.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"title": {
"type": [
"string",
"null"
]
},
"tree": {
"description": "Indented outline; lines carry `[ref=eN]` handles for `browser::act`. Empty when `diff` is populated.",
"type": "string"
},
"truncated": {
"description": "True when the tree hit `max_snapshot_nodes` and was cut short. Also the signal that a `diff` may be incomplete: the diff is computed over emitted nodes only, so when either snapshot was truncated a node that was emitted before and capped out now can show up in `removed` even though it still exists (and vice versa for `added`).",
"type": "boolean"
},
"url": {
"type": "string"
}
},
"required": [
"generation",
"tree",
"truncated",
"url"
],
"title": "SnapshotOutput",
"type": "object"
}
},
{
"description": "Read an element's computed styles (curated design set by default, or named properties) plus its inline style attribute.",
"metadata": {},
"name": "browser::styles::read",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"properties": {
"default": null,
"description": "Computed property names to return. Omit for a curated design-panel set; pass `[\"*\"]` for every computed property.",
"items": {
"type": "string"
},
"type": [
"array",
"null"
]
},
"ref": {
"description": "Element ref from `browser::snapshot`, `browser::dom::read`, or a pick.",
"type": "string"
},
"session_id": {
"type": "string"
}
},
"required": [
"ref",
"session_id"
],
"title": "StylesReadInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"StyleProperty": {
"properties": {
"name": {
"type": "string"
},
"value": {
"type": "string"
}
},
"required": [
"name",
"value"
],
"type": "object"
}
},
"properties": {
"inline_style": {
"description": "The element's inline `style` attribute, when present.",
"type": [
"string",
"null"
]
},
"properties": {
"items": {
"$ref": "#/definitions/StyleProperty"
},
"type": "array"
},
"ref": {
"type": "string"
}
},
"required": [
"properties",
"ref"
],
"title": "StylesReadOutput",
"type": "object"
}
},
{
"description": "Set one inline CSS property on an element, live in the page. Visual experiment only: the page's source files are untouched, and the edit dies with the next navigation.",
"metadata": {},
"name": "browser::styles::write",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"important": {
"default": null,
"description": "Apply with `!important`.",
"type": [
"boolean",
"null"
]
},
"property": {
"description": "CSS property name (`background-color`).",
"type": "string"
},
"ref": {
"description": "Element ref to edit.",
"type": "string"
},
"session_id": {
"type": "string"
},
"value": {
"description": "CSS value (`#101418`). Empty string removes the inline property.",
"type": "string"
}
},
"required": [
"property",
"ref",
"session_id",
"value"
],
"title": "StylesWriteInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"inline_style": {
"description": "The element's inline `style` attribute after the edit.",
"type": "string"
},
"ok": {
"type": "boolean"
}
},
"required": [
"inline_style",
"ok"
],
"title": "StylesWriteOutput",
"type": "object"
}
},
{
"description": "List the open tabs of a running browser reachable at a CDP endpoint (url, title, and whether a session already adopted each). Read-only; adopt one with browser::sessions::attach.",
"metadata": {},
"name": "browser::tabs::list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"cdp_url": {
"description": "CDP endpoint of the running browser, as in browser::sessions::attach.",
"type": "string"
}
},
"required": [
"cdp_url"
],
"title": "TabsListInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"TabInfo": {
"description": "One open tab reported by `browser::tabs::list`.",
"properties": {
"adopted": {
"description": "True when a session already adopted this tab; it cannot be adopted again until that session stops.",
"type": "boolean"
},
"title": {
"type": [
"string",
"null"
]
},
"url": {
"type": "string"
}
},
"required": [
"adopted",
"url"
],
"type": "object"
}
},
"properties": {
"tabs": {
"items": {
"$ref": "#/definitions/TabInfo"
},
"type": "array"
}
},
"required": [
"tabs"
],
"title": "TabsListOutput",
"type": "object"
}
}
],
"triggers": [
{
"description": "A browser::chromium::install job moved: a new phase (resolving, downloading, extracting, verifying, done, failed) or about 1% more bytes. Not a session event: bind without a session_id filter.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by every `browser::*` trigger binding. The only filter is an optional session-id equality match; unknown fields fail at registration so a misspelled filter key fails loudly instead of silently receiving nothing.",
"properties": {
"session_id": {
"description": "Only deliver events for this browser session.",
"type": [
"string",
"null"
]
}
},
"title": "BindingConfig",
"type": "object"
},
"metadata": {},
"name": "browser::chromium-install-progress",
"return_schema": {}
},
{
"description": "A console/log/exception entry was captured on a session's page.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by every `browser::*` trigger binding. The only filter is an optional session-id equality match; unknown fields fail at registration so a misspelled filter key fails loudly instead of silently receiving nothing.",
"properties": {
"session_id": {
"description": "Only deliver events for this browser session.",
"type": [
"string",
"null"
]
}
},
"title": "BindingConfig",
"type": "object"
},
"metadata": {},
"name": "browser::console-event",
"return_schema": {}
},
{
"description": "A download started, progressed, or finished in a session.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by every `browser::*` trigger binding. The only filter is an optional session-id equality match; unknown fields fail at registration so a misspelled filter key fails loudly instead of silently receiving nothing.",
"properties": {
"session_id": {
"description": "Only deliver events for this browser session.",
"type": [
"string",
"null"
]
}
},
"title": "BindingConfig",
"type": "object"
},
"metadata": {},
"name": "browser::download-changed",
"return_schema": {}
},
{
"description": "Internal: a live screencast frame of a watched tab (console viewport plumbing, high volume).",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by every `browser::*` trigger binding. The only filter is an optional session-id equality match; unknown fields fail at registration so a misspelled filter key fails loudly instead of silently receiving nothing.",
"properties": {
"session_id": {
"description": "Only deliver events for this browser session.",
"type": [
"string",
"null"
]
}
},
"title": "BindingConfig",
"type": "object"
},
"metadata": {},
"name": "browser::frame-event",
"return_schema": {}
},
{
"description": "A session is paused waiting for a human to complete a step (CAPTCHA, 2FA, payment).",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by every `browser::*` trigger binding. The only filter is an optional session-id equality match; unknown fields fail at registration so a misspelled filter key fails loudly instead of silently receiving nothing.",
"properties": {
"session_id": {
"description": "Only deliver events for this browser session.",
"type": [
"string",
"null"
]
}
},
"title": "BindingConfig",
"type": "object"
},
"metadata": {},
"name": "browser::handoff-requested",
"return_schema": {}
},
{
"description": "A paused handoff finished (confirmed in page, by call, or timed out).",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by every `browser::*` trigger binding. The only filter is an optional session-id equality match; unknown fields fail at registration so a misspelled filter key fails loudly instead of silently receiving nothing.",
"properties": {
"session_id": {
"description": "Only deliver events for this browser session.",
"type": [
"string",
"null"
]
}
},
"title": "BindingConfig",
"type": "object"
},
"metadata": {},
"name": "browser::handoff-resolved",
"return_schema": {}
},
{
"description": "The session's page committed a navigation.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by every `browser::*` trigger binding. The only filter is an optional session-id equality match; unknown fields fail at registration so a misspelled filter key fails loudly instead of silently receiving nothing.",
"properties": {
"session_id": {
"description": "Only deliver events for this browser session.",
"type": [
"string",
"null"
]
}
},
"title": "BindingConfig",
"type": "object"
},
"metadata": {},
"name": "browser::navigated",
"return_schema": {}
},
{
"description": "A network request was captured (completed or failed) on a session's page.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by every `browser::*` trigger binding. The only filter is an optional session-id equality match; unknown fields fail at registration so a misspelled filter key fails loudly instead of silently receiving nothing.",
"properties": {
"session_id": {
"description": "Only deliver events for this browser session.",
"type": [
"string",
"null"
]
}
},
"title": "BindingConfig",
"type": "object"
},
"metadata": {},
"name": "browser::network-event",
"return_schema": {}
},
{
"description": "The human picked an element in inspect mode.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by every `browser::*` trigger binding. The only filter is an optional session-id equality match; unknown fields fail at registration so a misspelled filter key fails loudly instead of silently receiving nothing.",
"properties": {
"session_id": {
"description": "Only deliver events for this browser session.",
"type": [
"string",
"null"
]
}
},
"title": "BindingConfig",
"type": "object"
},
"metadata": {},
"name": "browser::picked",
"return_schema": {}
},
{
"description": "A browser tab opened and is ready.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by every `browser::*` trigger binding. The only filter is an optional session-id equality match; unknown fields fail at registration so a misspelled filter key fails loudly instead of silently receiving nothing.",
"properties": {
"session_id": {
"description": "Only deliver events for this browser session.",
"type": [
"string",
"null"
]
}
},
"title": "BindingConfig",
"type": "object"
},
"metadata": {},
"name": "browser::session-started",
"return_schema": {}
},
{
"description": "A browser tab closed for good (stopped, idle, expired, or crashed).",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by every `browser::*` trigger binding. The only filter is an optional session-id equality match; unknown fields fail at registration so a misspelled filter key fails loudly instead of silently receiving nothing.",
"properties": {
"session_id": {
"description": "Only deliver events for this browser session.",
"type": [
"string",
"null"
]
}
},
"title": "BindingConfig",
"type": "object"
},
"metadata": {},
"name": "browser::session-stopped",
"return_schema": {}
},
{
"description": "A browser tab woke up (page open again), went to sleep (page closed, tab kept), or its page's title changed.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by every `browser::*` trigger binding. The only filter is an optional session-id equality match; unknown fields fail at registration so a misspelled filter key fails loudly instead of silently receiving nothing.",
"properties": {
"session_id": {
"description": "Only deliver events for this browser session.",
"type": [
"string",
"null"
]
}
},
"title": "BindingConfig",
"type": "object"
},
"metadata": {},
"name": "browser::session-updated",
"return_schema": {}
}
]
}