agent.run()
DOCUMENTATION / PROJECT
View MarkdownSource on GitHub

Based on AgentRun release · Synced .

AUTOQA.md: AgentRun (Parcha-ai/agentrun)

Baseline for the autoqa skill. It says how to bring up AgentRun Server from a checkout, how to authenticate, and which checks make up the core catalog. Read it first, then plan the diff cases on top of it. It does not replace diff discovery.

Drafted from the first whole-platform QA run (2026-11-01, main ba6a76271). Sources: AGENTS.md, packages/serve/README.md, packages/serve/src/{cli,server,harness,adapters,runs,tools,tool-keys}.ts, apps/console/src/App.tsx.

Unattended scope

When this runs from a pipeline or a periodic QA job, the scope is fixed here and nobody is asked:

  • Periodic whole-platform pass (default when no PR is named): every row of the core catalog below, on every harness that GET /v2/capabilities reports available: true, at low and medium effort. Budget: about $10 of model spend, one build or test job at a time, 60 to 90 minutes of wall time, one headless browser.
  • PR pass: the diff cases, plus health, auth and the rows of the catalog the diff touches.

How to run

Node 24.11 or later (.nvmrc pins 24.21.0); anything older is refused. npm 11 is the documented toolchain, but npm ci also works with npm 10.9 (the npm bundled with Node 22.23.2).

npm ci --ignore-scripts
npm run build   # the console first (apps/console, skipped when its dist is current), then every package

Start it with the launcher, which hands secrets to the server over a pipe: packages/serve/launcher/agentrun-serve --data DIR --port N [--drain-timeout 8s], on its default loopback listen address. node packages/serve/dist/cli.js also works, but then the secrets stay in the environment.

The banner prints AGENTRUN_BASE_URL, then the token (on the first start only), then Open <url>/ and Set your password: <url>/setup. Wait for the Open line. The token is also in DIR/.token (mode 0600). Read it from there and never print it.

Env Why
TMPDIR, --data Use a fresh data dir and a temp dir of its own per run, so no earlier run's state is read.
XDG_CONFIG_HOME The launcher writes agentrun/seal.key there. Point it at a QA dir so your own config directory is never touched.
AGENT_MODEL_BASE_URL, AGENT_MODEL_API_KEY, AGENT_MODEL Gateway for the pi and codex harnesses (OpenAI Responses). Point them at your gateway (its /v1 base URL) and set AGENT_MODEL to a small, cheap model for smokes.
ANTHROPIC_BASE_URL, ANTHROPIC_API_KEY The claude-agent harness only, on Anthropic Messages. Your gateway may serve /v1/messages too.
AGENTRUN_MAX_RUNS Cap concurrent runs (3 is plenty for QA).
PARALLEL_API_KEY, BROWSERBASE_API_KEY + BROWSERBASE_PROJECT_ID, TYPESAFE_API_KEY Optional. They unlock web_fetch via Parallel, the browser tool, and Jev decisions. Without a key, only the checks that need it are UNTESTED: web_fetch's direct fetch and its private-host refusal need none and always run.

Stop the server with SIGTERM to its exact PID. A busy run gets the drain timeout, then resumes at the next start.

Auth recipe

  • API: Authorization: Bearer $(cat DIR/.token). GET /v2/health, GET /v2/capabilities and GET /v2/openapi.json need no credential. Anything else without a token is 401 unauthenticated. A Host on a loopback bind that is not localhost, 127.0.0.1, [::1], the public URL's name or an --allow-host name (any port) is 403.
  • Console: / (/console/, /setup and /sign-in redirect there with 303). On the first visit, set the owner password (12 characters or more; labels New password and Password again). After that, sign in with the Password field. The session is a cookie, and the ?token= link sets none while accounts are on.
  • Forgotten password: agentrun-serve reset-password --data DIR.

Core check catalog

Each row: entry point, then what makes it pass.

  1. Boot + health: banner Open line; GET /v2/health → 200 {"status":"ok","checks":{"store":"ok","disk":"ok"}}.
  2. Capabilities: GET /v2/capabilities → harnesses[] (pi, claude-agent, codex, each with available), models.source, tools[] with available/missing, features, and limits.max_concurrent_runs.
  3. OpenAPI: GET /v2/openapi.json → 3.1 document.
  4. Auth: no token → 401; wrong token → 401 plus an auth_failed log line naming the reason and never the token; bad Host → 403.
  5. Agent create: POST /v2/agents {definition:{id,name,instructions,harness?,model?,tools}} → 201 with warnings: []. Make one agent per harness. claude-agent needs model: "anthropic/<id>", and codex/pi take the host default.
  6. Agent edit: PUT /v2/agents/{id}/definition with If-Match: <home_etag> → a new revision. A stale If-Match → 412. tools:["bash"] → 422. Check GET /v2/agents/{id}/revisions.
  7. Agent archive: archive → hidden from GET /v2/agents, listed under ?archived=true. A new run → 409 agent_not_runnable. Then unarchive.
  8. Runs per harness × effort: POST /v2/agents/{id}/runs {input:{content}, effort:"low"|"medium"}. Poll GET /v2/runs/{id} to a terminal status (cap 300 s). Then check GET /v2/runs/{id}/timeline (final: true), cost.usd, GET /v2/runs/{id}/files and GET /v2/runs/{id}/files/<path>. Use a task that needs two files (two tool calls in one turn): see the open findings.
  9. Spend cap: a priced model and a tiny max_cost_usd. A cap that can't buy the next request's least pauses the run: input_required and waiting, error null, pending.kind: "budget" (with cap_usd, spent_usd, needs_usd, suggested_cap_usd), a spend_cap_reached row on the timeline, the refused request unsent, no files; a higher cap sends it. A cap that buys a smaller request limits it (its output is held to what the cap buys) and the run can complete: e.g. 0.01 on luna completes and 0.0001 pauses. Only a refusal no cap fixes ends failed (spend_cap_reached). See serve's README, "A run paused at its cap".
  10. Cancel: start a multi-step run, POST /v2/runs/{id}/cancel after its first tool_use → cancelled within a few seconds. Then POST /v2/runs/{id}/messages {content} → 409 not_continuable. The body takes content only.
  11. Follow-up: POST /v2/runs/{id}/messages {content} to a completed agentic run → back to completed. The timeline's user_message count goes up by exactly one and the original rows stay first.
  12. Kernel durability, process crash: during a multi-step run, take the holder's pid from the run's row in the agent's store (runs.claim_holder, JSON). Check that its ppid is the server and its cmdline names the run dir, then kill -KILL that exact PID. Pass: the server log says its process went before the run ended; opening it again (1 of 3), the row's claim_epoch goes up by one, the run completes, and the timeline holds exactly one write per file.
  13. Kernel durability, server restart: SIGTERM the server mid-run; once its process is gone, the run's record in store/run.sqlite (table pda_run_status) reads sleeping. Restart on the same data dir. Pass: the run resumes and completes with no duplicate writes. The prompt must say "do not reply or stop until …", or the model may end its turn with a progress note before the signal and the case proves nothing.
  14. web_fetch: a revision naming web_fetch, fetching https://example.com → ok:true with via: direct (or parallel with the key). A loopback URL and the link-local cloud metadata address → refused: … not a public host.
  15. Workflow run: create an agent from packages/dsl/skills/author/examples/review-release-notes.json, run it on its .input.json → completed, with result.output.changes length 2. GET /v2/runs/{id}/graph → totals.
  16. Stats/analytics: GET /v2/stats → analytics.{totals,time,by_harness_effort,by_day,by_source,tools}. ?agent=&from=&to= filters; a malformed from → 400. GET /v2/usage → total_usd, unpriced_runs.
  17. Run archive: POST /v2/runs/{id}/archive → gone from GET /v2/runs, listed under ?archived=true, still readable by id, and stats.archived.runs goes up.
  18. Tool keys: GET /v2/settings/tool-keys → field names with set/source, never a value. Do not PUT real keys.
  19. Session import (read-only): node packages/serve/dist/cli.js import --dry-run --data <QA data dir> → N projects, M sessions; nothing was written (dry run). Pass also requires that the data dir listing is unchanged. On a machine with thousands of sessions it takes minutes.
  20. Console UI (a headless Chromium of your own on loopback, not a browser other work shares): setup, sign-in, Mission Control (#/mission), Agents (#/agents), agent page (#/agents/<id>), editor (#/agents/<id>/edit, every step through History), New agent (#/agents/new, create with the defaults), Playground (#/playground: send a run, then "Open this run" to follow it live), Runs plus a state filter (#/runs?status=cancelled), run detail tabs Report/Timeline/Files and the file viewer, and a workflow run's Graph. Capture console errors and HTTP ≥ 400 per page. The page CSP is default-src 'self', so any violation is a finding.

Subsystems

Each subsystem below names its seam: what it takes from the kernel and what it gives back, as types or routes, by name only. It gives an "alone" recipe that runs it end to end with the others faked, and 3 to 6 checks a QA pass runs when it changed. They are light and continuous: run a subsystem's checks when its code changed, not on every PR. Each lane edits only its own subsection.

Kernel (runs, journal, cap, harness adapters)

Code: packages/serve/src/{runs,app,harness-app,harness,adapters,cap,request-start}.ts, packages/run-adapters, packages/run-adapters-claude-agent, packages/run-adapters-codex, packages/pi-durable-disk, and the timeline of packages/pi-durable-agent. Rows 8 to 13 of the core catalog are its checks on a live model; the checks below are the same ground with no key.

Seam.

Names on main
Serves POST /v2/agents/{id}/runs (input, revision, effort, max_cost_usd), GET /v2/runs, GET /v2/runs/{id}, GET /v2/runs/{id}/timeline (pages or SSE), POST /v2/runs/{id}/{messages,stop,cancel,pause,resume}, GET /v2/runs/{id}/graph, GET /v2/usage, and harnesses[] in GET /v2/capabilities
Run fields others read status, state, result.final_answer, error.code (spend_cap_reached, moderated, run_failed, escalated), cost.usd (null when unknown)
Journal In the run's store, from @parcha/pi-durable-agent/timeline: REQUEST_STARTED (heldUsd, capRefused), REQUEST_ENDED (heldUsd, unsent), REQUEST_COST, REQUEST_REFUSED, REQUEST_COST_PENDING, REQUEST_COST_LATE, REQUEST_CONTINUED. Read through timelineOf and runMetricsOf
Run directory DIR/disk/agents/<agent>/runs/<id>/: serve.json, store/run.sqlite (with the run's record: status, generation, holder.pid), work/, process.log; the run's claim is its row in the agent's store (claim_epoch, claim_holder)
Adapters it calls adapterFor(harness, env, needs: AdapterNeeds) gives an AgentAdapter (check, open) for a SessionSpec, driven by createHarnessRunner. There the cap is the runner's beforeRequest, asked by startModelProxy under a ProviderPolicy; on pi it wraps the models (capModels)
Takes from the tool plane TOOLS (ToolEntry), bound on pi as a ToolBinding; hostTools() gives claude-agent and codex HostTool[], each with its REPLAY (safe or never)
Takes from places The revision's definition.place: runner (the default: a local process under confinePrefix()) or sandbox with a box. A sandbox run on a box this server lacks is 409 agent_not_runnable naming it (boxAvailability); places in capabilities lists the boxes, none available yet. startServe({ box }) takes a BoxHost that sweeps the boxes runs left (the run's box.json)
Gives the console The routes above, and GET /v2/stats analytics, summed from the RunMetrics each run's carryBack stores
Gives the learning loop Not wired into serve yet. The seam is run-adapters' followUpOnce (one more turn, held to reserveUsd), which learn's reflectBudget sizes

Alone. No model key, no tool key, no box and no console. Build as in How to run, without the console lines. Then:

  • pi runs alone through the CLI on the tests' faux model. On faux-output-priced.mjs, one request's worst case is 16,384 output tokens at $100 a million: $1.6384.

    Q=$(mktemp -d)
env -i PATH="$PATH" HOME="$Q" TMPDIR="$Q" node packages/serve/dist/cli.js --data "$Q/data" --port 0 \
  --drain-timeout 2s --faux packages/serve/test/fixtures/faux-output-priced.mjs

    Drive it with the README's quick-start calls; the token is in $Q/data/.token. The faux model answers by the message: What is six times seven? gives The answer is 42., Take your time. holds its answer for 60 s, Answer once <file> exists. gives Now. once the file exists, and a follow-up <message> to Take your time. gives On <message>: done..

  • claude-agent and codex run alone only inside the tests. startServe({ harnessTable }) names a test module of harness table entries (serve's test/fixtures/faux-table.mjs puts every harness but pi on run-adapters' fauxAdapter); the CLI has no flag for it, and no environment variable of the server's own turns it on. claude-run drives the pinned Claude CLI against fakeAnthropic, and codex-run drives agentrun-harness-codex's app-server stand-in. From packages/serve:

    node --test --test-reporter=tap test/harness-run.test.mjs test/cap-unknown.test.mjs \
  test/claude-run.test.mjs test/codex-run.test.mjs

    The checks below name these files without .test.mjs.

Checks. Each reads GET /v2/runs/{id} once the run's timeline says final: true, unless it names another route. The journal is the entries table of the run's store/run.sqlite.

  1. Each harness answers. For every entry of harnesses[] with available: true, a run ends completed with result.final_answer set and cost.usd a number. Each paid request is counted once. Alone: pi through the CLI (What is six times seven?); claude-agent and codex in harness-run, claude-run and codex-run.
  2. The cap refuses before anything is sent. max_cost_usd: 1, below one request's worst case: the run pauses, input_required and waiting, error null, pending.kind: "budget" (spent_usd 0, needs_usd the request's least), result: null, cost.usd: 0. The journal's only request mark is agentrun.request_started with capRefused: true, and a higher cap sends it (PATCH /v2/runs/{id} {max_cost_usd}). Only a refusal no cap fixes (a spend nobody knows, no output ceiling, no price) ends failed, spend_cap_reached. Alone: pi through the CLI, cap-pause and cap-raise; claude-agent and codex in claude-run, codex-run and cap-raise.
  3. A stopped request counts at the worst case the cap held. Take your time. at max_cost_usd: 2; once it reads working, POST /v2/runs/{id}/stop (stopped: true) leaves it input_required. A message is a follow_up, and the cap counts the stopped request at the worst case it held ($1.6384). Its request is limited to what is left of the cap; if what is left can't buy that request's least, the run pauses at it (input_required, pending.kind: "budget", spent_usd 1.6384) with nothing more sent. Either way the run never ends failed for room. At 10 the follow-up completes. cost.usd is null both ways. Read before final, at the first poll that shows the terminal status, it can still be the live figure (0, or 0.0004 after the follow-up). The stopped request's agentrun.request_ended mark carries heldUsd: 1.6384. Alone: pi through the CLI, and cap-unknown. For claude-agent and codex, harness-cap covers a crash mid-request and run-adapters' held-worst-case.test.mjs a request cut short; no test stops one through serve.
  4. Cancel ends the run and nothing more is sent. POST /v2/runs/{id}/cancel on a working run answers 202 with cancel_requested: true; the run ends cancelled, a second cancel says already_terminal: true, and a message is 409 not_continuable. A request cut by the cancel leaves cost.usd null. Alone: runs (pi, on Take your time.) and harness-run (codex, cancelled while a request is out).
  5. A crash or a restart resumes the run. Restart: Answer once <file> exists.; once it reads working, SIGTERM the server's own PID. It exits within --drain-timeout plus 3 s, the run's holder process exits with it, and the run's record in its store reads sleeping. Start it again on the same --data and make the file: the run ends completed with Now.. Crash: kill -KILL the run's holder process (its row's claim_holder); the run opens again and completes, each paid request counted once. Alone: the restart on pi through the CLI, and recovery; the crash in harness-run and claude-run, and in codex-run, which checks that no Codex process is left behind.
  6. A multi-tool turn completes on claude-agent and codex. Two tool calls in one model response end completed, each call shown before its result under the answer that asked for it, none interrupted. Its faux-model test was deleted at the cutover merge (follow-up #2078). On a live model it is part of row 8.

Tool plane

What a run's agent can call, on every harness, and the keys those tools use. Later changes extend this section as they land (native and picked web search and tool settings, monid on every harness, the per-tool permission gate).

Seam.

  • Takes from the kernel:
    • the run's pinned revision (the tools it names);
    • the run's secrets over its pipe (a tool key only to a run that needs it; Parallel's key on the decision descriptor);
    • pi's tool binding (TOOLS[name].bind);
    • the host-tool slot the claude-agent and codex adapters serve (hostTools, served over MCP or app-server dynamic tools);
    • the journal's replay classes (REPLAY: safe may run again after a crash, never is answered interrupted);
    • the run's work/ directory.
  • Gives back:
    • tool registrations per harness, and one journal entry per call;
    • GET /v2/capabilities tools[] (name, harnesses, missing);
    • GET/PUT/DELETE /v2/settings/tool-keys (keys by name, sealed at rest, never returned);
    • files the tools write under work/ (work/evidence/).

Alone. Run the plane with no model and no third party:

  • Tests. startServe({ faux, fauxWeb, fauxParallel, fauxMonid }) in packages/serve/test replaces the model, the resolver and HTTP for fetches, Parallel and monid with test/fixtures/faux-*.mjs. The faux harness (faux-harness.mjs) stands in for claude-agent and codex. node --test packages/serve/test/{web-fetch-direct,web-fetch,host-tools,harness-run,monid,tool-keys,secret-custody}.test.mjs runs the plane end to end that way.
  • On a QA server. --faux MODULE scripts the model's tool calls (see "The faux model" in packages/serve/README.md), so a run calls web_fetch, read, write and edit with no model key. web_fetch then reads real public pages; nothing else leaves the box.

Checks when the tool plane changed.

  1. Every name in GET /v2/settings/tool-keys unlocks is a tool GET /v2/capabilities binds, on the harnesses it lists, and missing names the key each tool still needs.
  2. web_fetch refuses loopback, private, link-local and metadata addresses and ports other than 80/443, at the first hop and after a redirect; a refused URL is never sent to Parallel.
  3. A tool key reaches only a run that names a tool it unlocks, over the pipe, never in any process's environment; a saved key exists on disk only sealed (no plaintext under the data directory).
  4. On claude-agent and codex, the host tools (read, write, edit, web_fetch, and web_search where it is not the harness's own) run and are journaled with their replay class; a write cut off by a crash comes back interrupted, never re-run.
  5. A monid.run is sent once and never retried; a wallet refusal (402) or a workspace control (BLOCKED) comes back by name; an answer over 60,000 characters is filed under work/evidence/monid/.
  6. On pi with no native search, web_search searches through Parallel where the server holds a Parallel key (its fee a Tool cost line), else Exa's keyless endpoint with no key: only {query, numResults} leaves the server, over IPv4; past Exa's free limit it is tried once more, then a tool error in our words (Free web search is rate-limited right now…), never Exa's fix text and never a hang.

Places (boxes, home, files)

Seam. It takes a run's BoxSpec from the kernel (workspace, env, network, mounts) and the run directory the run's files go to. It gives back:

  • Box and BoxProvider from @parcha/pi-transport. Only a provider whose BoxProvider.mounts is true binds mounts (Docker, as bind mounts); every other provider refuses a spec with any as unsupported.
  • @parcha/pi-transport/sync: the safe extractor (extractTar, openExtractor, removeEntries) and the sync layer (seedDir/pullDir, seedAgentHome/pullAgentHome, HOST_OWNED_HOME), each answering with a receipt.
  • Not in the seam: ./launch. Serve does not import it; it is deprecated, to be removed.
  • Internal to pi-transport: bwrapArgs, bwrapFiles and probeBwrap (src/box/bwrap-args.ts).
  • @parcha/pi-transport/testing: registerBoxConformance, and fake providers for each box (fakeBoxes and the provider-shaped fakes).
  • Serve, what a run is told of its files (packages/serve/src/run-manifest.ts): firstMessage adds one line to an agentic run's first message, Files for this run: "inputs/<path>" (<size>), ..., each name quoted, at most MANIFEST_ENTRIES (20), from the run's work/inputs/ as runs.ts sends it to the door, on every harness.
  • Serve: GET /v2/runs/{id}/files and GET /v2/runs/{id}/files/{path}, and the sweep before any run starts (sweepBoxes, settleBox, Quarantine in packages/serve/src/box/sweep.ts).
  • Serve, an agent's place (packages/serve/src/place.ts): definition.place is {mode: "runner"} or {mode: "sandbox", box}, checked by placeIssues; GET /v2/capabilities answers places (placesOf). A sandbox place saves with a warning at definition.place, and its run is 409 agent_not_runnable with details.box.
  • Serve, the box registry (packages/serve/src/boxes.ts): AGENTRUN_BOXES lists the boxes to probe once at start (boxRegistry); not set at all, it is docker (DEFAULT_BOXES), and empty or none is no box; and the start log's event=boxes line says which are available and why the others aren't. Box keys (BOX_KEYS) stay with the server. The registry feeds places (boxAvailability): docker and daytona are available when they are, and every other box says runs in it are not available yet.
  • Serve, a run's place and its box: a run takes its revision's place or the default (resolvePlace), and the run keeps its resolved place (its serve.json, GET /v2/runs/{id}). In the run's process, codingSource(place, box?) (packages/serve/src/coding-tools.ts) is the boxed coding source: on the runner pi's read, write and edit; in a box those and bash, each call on pi-transport's boxEnv over the run's box. pi's tool table and the other harnesses' hostTools both take it, and the tool plane's permission gate wraps it. agentWorkDir(runDir, place) (place.ts) is the run's work/ as its tools see it: <runDir>/work on the runner, /workspace in a box. The box's lifecycle and mounts are packages/serve/src/run-box.ts (runBox, boxMounts, mountSource, destroyRunBoxes, sweepRunBoxes), labelled agentrun.run=<id>; pi-transport's endLeftCommands kills what a crashed run's process left running.
  • Serve, a Daytona place ({mode: "sandbox", box: "daytona"}): the server makes, seeds, pulls and destroys the run's sandbox with its own key (ensureRemoteBox, pullRemoteBox in run-box.ts; remotePlace in place.ts), the network blocked (networkBlockAll). The run's process reaches it only through the box relay (packages/serve/src/box-relay.ts, its address in <data>/.box-relay), which takes a run's call only with that run's own door token, follows no redirect, and sends on with the key only a read of the run's own sandbox and its toolbox's programs and files. work/ is copied in at the sandbox's making and back at each turn's end (a read of the run's files waits for that copy) and before the run's end is recorded; the mark is box-sync.json beside the run. Tests: test/daytona-boxes.test.mjs on test/fixtures/fake-daytona.mjs, a loopback Daytona.

Not on main yet: runs in a box other than Docker and Daytona, a files API for the agent's home, and uploads. Each one extends this section when it lands.

Alone. Everything else is faked: boxes by fake providers, the model by the faux model. Put a docker that exits 1 first on PATH, even where Docker is installed, so nothing reaches a daemon by accident:

mkdir -p "$TMPDIR/qa-bin" && printf '#!/bin/sh\nexit 1\n' > "$TMPDIR/qa-bin/docker" && chmod +x "$TMPDIR/qa-bin/docker"
export PATH="$TMPDIR/qa-bin:$PATH"
npm ci --ignore-scripts

# pi-transport: vitest on source, no build
npm test -w @parcha/pi-transport -- box- fake-boxes   # every box on fakes, Docker's argv without a daemon, bwrap
npm test -w @parcha/pi-transport -- sync-extract sync-run-files               # hostile archives and boxes

# serve: node:test over dist, on the faux model
npm run build
(cd packages/serve && node --test test/place.test.mjs test/boxes.test.mjs test/files.test.mjs \
  test/box-sweep.test.mjs test/box-record.test.mjs test/boxed-tools.test.mjs \
  test/coding-tools.test.mjs test/host-tools.test.mjs test/run-manifest.test.mjs)

boxed-tools.test.mjs runs sandbox runs on a faux box (pi-transport's local box, startServe({ fauxBox })).

The Docker variant runs only where Docker exists, as CI's transport-docker job does: docker pull debian:bookworm-slim && npm run test:live:docker -w @parcha/pi-transport, then serve's AGENTRUN_SERVE_LIVE_DOCKER=1 node --test test/boxed-tools.docker.test.mjs (advisory there). Each .live file the box- filter picks up skips itself without its PI_TRANSPORT_LIVE_* variable, so the commands above start no real box. boxes.test.mjs puts its own fake docker first on PATH and answers Daytona from a loopback fake; never set a box key or a real docker on PATH for a server under test.

Checks when it changed.

  1. Places: place.test.mjs passes. Each wrong shape is a 422 at definition.place and nothing is saved; a sandbox place saves with a warning naming its box, and its run is 409 agent_not_runnable with that box; an edit with no place keeps the agent's.
  2. Box registry and keys: boxes.test.mjs passes. A name in AGENTRUN_BOXES that is no box refuses the start; each probe is bounded; no key's value is in an offer, a reason or the log; a box key set in the server's environment reaches no run's process.
  3. Hostile archives and home pulls: sync-extract, home-vault and sync-run-files pass. Nothing is written outside the destination, no link, device or fifo is created, a setuid mode is capped, a pull never reverts a file the host changed (host_changed), and learning/ is never written back.
  4. Box-made stores: home-untrusted-store passes on Node 22 and on Node 24. A store with a trigger, a view, an ATTACH or a hot journal is refused, and nothing in it runs.
  5. Box arguments and mounts: box-bwrap-args matches its golden vectors and refuses a workspace with a link in it; box-bwrap-probe refuses a setuid or too-old bwrap; box-docker.test refuses a missing source, a relative path, a .. segment, a target of / and a repeated target before docker run; box-local refuses any mount.
  6. Run files and the sweep: files.test.mjs never lists or serves a dotfile or a symlink, and refuses .. with the contract's codes; box-sweep.test.mjs reaps a box whose writer is gone and quarantines one it cannot empty.
  7. Boxed tools: boxed-tools.test.mjs passes. A sandbox run's bash printf hi > out.txt shows in GET /v2/runs/{id}/files on pi, claude-agent and codex; the box's environment is LANG, HOME=/tmp and its PATH, with no key or token; every docker run has --network none, --user the server's uid:gid, one --mount of work/ at /workspace and only NAME=value envs; a run request naming place is a 422 and makes no run; a runner agent naming bash and a sandbox workflow are 422; a link at a run's work/ is refused and nothing is bound; the box is destroyed at completion, cancel and archive, adopted by its label after a crash (the left command killed), and swept at start once its run has ended.
  8. Run files told: run-manifest.test.mjs passes. A run given notes.txt is sent "Summarize the attached file." plus Files for this run: "inputs/notes.txt" (<n> B)., reads it and answers with its text; on claude-agent and codex the first user_message names both files with their sizes; a run given none is sent its content unchanged; a name with ", or an RTL override stays inside its quotes, escaped; a link at work/inputs lists nothing.

Console, import and analytics

Seam. Takes the /v2 routes the console reads: /agents, an agent's /revisions and /runs, /runs and a run's /timeline, /graph and /files, /stats with its analytics block, /usage, /capabilities and /settings/tool-keys. Also the agent store's run_metrics and run_tool_metrics, which /stats sums. Gives back /v2 writes only: create, revise and archive an agent; start a run, message it, stop, cancel or archive it. import gives the store its runs through AgentStore.importRun, each with its run_metrics.

Alone. Each runs with no model key and no network. From a checkout built as in How to run, with fresh directories:

export QA_HOME=$(mktemp -d) QA_DATA=$(mktemp -d)
# A fixture home: synthetic Claude Code, Codex and pi sessions, the importer's own test fixtures.
node --input-type=module -e '
  import { mkdirSync } from "node:fs";
  import { claudeSession, codexSession, piSession } from "./packages/serve/test/fixtures/import-sessions.mjs";
  const home = process.env.QA_HOME, project = `${home}/code/site`;
  mkdirSync(project, { recursive: true });
  claudeSession(home, project, 1); codexSession(home, project, 2); piSession(home, project, 3);'

  1. The console against a seeded store, on the faux model:

    HOME="$QA_HOME" node packages/serve/dist/cli.js import --all --data "$QA_DATA"
XDG_CONFIG_HOME="$QA_HOME/.config" packages/serve/launcher/agentrun-serve --data "$QA_DATA" --port 0 \
  --faux packages/serve/test/fixtures/faux-model.mjs

    Open the console at / on the address the banner prints. The imported project is one agent with its three runs.

  2. The console's own suite, on recorded fixtures: (cd apps/console && npm ci && npm test).

  3. Import alone: HOME="$QA_HOME" node packages/serve/dist/cli.js import --dry-run --all --data "$QA_DATA" prints what it would import (1 project, 3 sessions) and writes nothing.

--all because the fixture's project is in a temp directory, which import skips otherwise.

Checks.

  1. The console loads at / and signs in: setup, then the password. With AGENTRUN_ACCOUNTS=0 the printed link signs the browser in and the token leaves the address.
  2. An agent is created from New agent with its defaults, and a run is started from its page. Once the workflow starter is on main (#1315), a workflow agent is created from it too.
  3. One run on a priced faux model (packages/serve/test/fixtures/faux-priced.mjs): its timeline's usage row, its stored cost.usd and /v2/stats agree, with nothing counted twice.
  4. Mission Control shows the same totals as /v2/stats for a date range (?from=&to=). So does the agent page's Analytics tab, once it is on main (#1354).
  5. An import's dry run counts what the import then stores, and a second import adds nothing.
  6. Settings → Tool keys never shows a value.

Learning loop

Written by the lane that owns this subsystem.

Model and key caveats

  • Run smokes on the cheapest capable model your gateway offers, and a stronger one only when a case needs it. Send all model traffic through your gateway.
  • The claude-agent adapter maps the CLI's Haiku slot to the session model (ANTHROPIC_DEFAULT_HAIKU_MODEL = model), so no Haiku traffic leaves a claude-agent run.
  • Spend-cap worst case is the request's input bound at the dearer of the model's input and cache-write prices, plus its whole output ceiling at its output price. That is a cent or two for one request to a small model at low effort, and can be more than a dollar for one request to a model with a large output ceiling, so a cap under that refuses the first request. A model with no price under a cap is refused outright.
  • A run cut by a crash, a drain or a stop reads cost.usd: null from then on. Mission Control shows those runs as "N runs unknown". That is documented behaviour, not a QA failure.
  • At these task sizes a periodic pass costs cents on small models.

Host caveats

  • Bind to loopback on a free port. The catalog needs no Docker, cloud previews or deploys.
  • Kill only PIDs you started (the server, or a run's holder.pid after checking its ppid and cmdline). Never pkill -f.
  • A fresh checkout or worktree per pass, never one in use for other work. Do not run the full test suites (CI is the suite of record).
  • Keys by name only. The launcher's first start prints the token into the server's stdout, so redact ar_local_… from any log copied into evidence.

Known open findings (2026-11-01; re-check each pass, do not suppress)

  • Fixed by #1363, re-check with kernel check 6: claude-agent and codex runs hung in working when the model made two or more tool calls in one turn (4/4 reproductions). pi was unaffected.
  • Fixed by #1357, re-check with console check 2: the console's New agent with default tools was refused with 422, because pdf_pages and llm (call-only tools) were ticked by default.
  • Fixed by #1356, re-check with tool plane check 1: tool keys listed web_search and the monid.* tools as unlocked, but no such tool was bound. Fixed for web_search by its binding as well: it is bound on every harness now, and the Parallel key lists it again as a tool it unlocks.
  • Not in the server yet: effort packs and output types, native web search, places and boxes, the kernel learning loop, and output/place on the run request.