@reticlehq/server 2.3.0 → 2.5.0

package/docs/agent-cheatsheet.md

@@ -0,0 +1,85 @@
+# Reticle — agent cheat-sheet
+
+One screen to get fluent. Reticle is the **proof layer for AI agents** — no screenshots, no vision model, evidence not prose. Everything below returns structured data. Full guide: [usage.md](usage.md).
+
+## The core loop: look → act → observe → assert
+
+| Verb | Tool | One-liner |
+| --- | --- | --- |
+| **look** | `reticle_snapshot` / `reticle_query` | See the page (semantic tree) / find one specific element. |
+| **act** | `reticle_act` / `reticle_act_and_wait` | Click/fill a `ref` / act + wait for a predicate in one hop. |
+| **observe** | `reticle_observe` / `reticle_wait_for` | Everything the app did after `since` / block until true. |
+| **assert** | `reticle_assert` | Evaluate a predicate → `{ pass, evidence, failureReason? }`. The end of every loop. |
+
+`reticle_act` returns a `since` cursor — pass it to `reticle_observe({ since })` to scope the window. Elements are addressed by stable refs (`e7`) from `snapshot`/`query`; they re-resolve across re-renders.
+
+**`assert`/`wait_for` are auto-scoped to your last act.** By default they only count events buffered _since_ the most recent act, so a stale signal from a previous step can't fake a pass — pass an explicit `since` to override. **Clicks run the code, not pixels:** `reticle_act` click fires the full pointer sequence on the element (no coordinate gesture for the HUD to intercept), reports `occluded:true` when something covers the target, and stays synthetic even with CDP configured (use `args:{ native:true }` for a trusted native click).
+
+**Never sleep — wait deterministically.** Fixed sleeps are the #1 cause of flaky agent tests. Instead:
+
+- `reticle_act_and_wait({ ref, action })` with **no `until`** waits for the page to _settle_ (network + structural DOM idle; ambient count-up/spinner churn is ignored so an animated page still settles) before returning — the one-call replacement for "click then sleep 500ms".
+- Need to wait without acting? `reticle_wait_for({ predicate: { kind: "settled", quietMs } })`.
+- Waiting for a specific outcome? Pass that consequence as the predicate (`{ signal }` / `{ net }`), or `allOf` it with `{ kind: "settled" }` to wait for both the event _and_ the page going quiet.
+
+**Assert a consequence, not just presence.** `{ signal }` / `{ net }` prove the feature actually did something; `{ element }` / `{ text }` only prove something is on screen — which a stale render or a locator healed to the wrong element can fake. A _passing_ presence-only `reticle_assert` returns `advice` nudging you to a consequence; heed it on anything that matters.
+
+## The 4-layer cross-check — never trust a green the state contradicts
+
+A claim is real only when the layers agree. Check more than the UI:
+
+| Layer | Tool(s) | Question it answers |
+| --- | --- | --- |
+| **UI** | `reticle_snapshot` / `reticle_query` | Is it on screen / in the right state? |
+| **signal** | `reticle_capabilities` / `reticle_observe` | Did the app emit the intent it advertised? |
+| **network** | `reticle_network` | Did `POST /x` actually fire and return 200? |
+| **store** | `reticle_state` | Does live framework/store state match? |
+
+> **Rule:** a passing UI assert that the store, network, or signal contradicts is a **false green**.
+
+**Session health is universal.** Every live-session tool result carries a `session` block (`throttled`, `focused`, `lastSeenMs`); when `throttled:true` it also adds a `warning` + `recommendation` (refocus, or `reticle drive`). A throttled/backgrounded tab can silently no-op timers/rAF/pointer gestures — if you see `session.throttled`, distrust a green and refocus first.
+
+> Store reads (`reticle_state`) are the reliable path; the DOM can lie (optimistic UI, stale render).
+
+**Truncation and coverage are declared, never silent.** A big `reticle_state` read carries `truncation` when the transport caps dropped items — its presence means "this is NOT the whole value", so an absence assertion over a truncated read proves nothing; scope with `path`/`depth` instead. A `reticle_assert` verdict carries `coverage` when part of the page was unobservable (cross-origin iframe, closed shadow root) — a green then means "nothing failed in the part I could see", not "the page is correct". Both fields are **omitted when everything was fine**, so their presence is the warning.
+
+**Charts report their own geometry faults.** Any element descriptor containing a broken plot carries `chart: [{ kind, tag, attr, sample }]` — `non-finite-coordinates` (a zero-range scale divided by zero; always a bug), `empty-geometry` (mounted but no data reached it), `degenerate-geometry` (every point identical). No extra call and no flag: query the chart as you would anyway. A healthy chart adds nothing. This matters because a chart is the one widget whose correctness is _geometry_ — the store can be right while the polyline is blank, and neither a state read nor a screenshot catches it. Canvas charts are pixels, not DOM: read their data with `canvasChartData(canvasEl, window)` instead.
+
+**Reads never go silently empty.** A zero-result read returns a `hint`, not a bare `[]`: `reticle_query` → `{ route, presentTestids, knownEmptyState }`; `reticle_network` → `{ totalInWindow, present[] }` (what DID fire); `reticle_console` → `{ totalInWindow, byLevel }` (so "0 errors" ≠ "silent page"); `reticle_state` lists `storeNames` when a store isn't found. Read the hint before assuming "not there." Scope big stores with `reticle_state({ store, path:"a.b.0", depth })` instead of paying for the whole tree; a wrong `path` returns `{ found:false, availableKeys }` so it's self-correcting.
+
+## Core tool set
+
+The tools advertised DIRECTLY — what you'll use 90% of the time. Everything else is one `reticle_run` away (see Token note below).
+
+`reticle_sessions` · `reticle_navigate` · `reticle_snapshot` · `reticle_query` · `reticle_act` · `reticle_act_and_wait` · `reticle_observe` · `reticle_network` · `reticle_console` · `reticle_wait_for` · `reticle_assert` · `reticle_state` · `reticle_inspect` (DOM node → `src/App.tsx:104`) · `reticle_feedback` (tell the maintainers what is missing) · `reticle_session` (hand back: `{action:"yield"}` the moment you stop driving, `{action:"resume"}` after a human pause).
+
+Frequently useful but NOT core, so reach them through `reticle_run({ tool, args })`: `reticle_domain` (learn the app + gaps), `reticle_capabilities` (the app's declared testable surface), `reticle_baseline {action:"diff"}`, `reticle_project` (run history).
+
+**Reach past core when…** you need to record/replay a journey (`reticle_record {action:"start"}/stop`, `reticle_replay`), persist a self-healing golden flow (`reticle_flow_save*` / `reticle_flow_replay` / `reticle_flow_heal`), compile annotations (`reticle_annotate`), explore autonomously (`reticle_explore` lists controls; `reticle_crawl` clicks them all and reports anomalies — **destructive**), reveal a virtualized off-screen row (`reticle_scroll_to` — when `reticle_query` finds nothing because a windowed list hasn't rendered it yet), visual-check (`reticle_screenshot` / `reticle_visual_diff`, pinned with `reticle_viewport` for reproducible baselines), test error/edge states by stubbing the network (`reticle_network_mock` — 500 / offline / delay, driven mode), control time for toasts/debounces/auto-dismiss (`reticle_clock { freeze | advanceMs | reset }`), or work with a human (`reticle_session {action:"end"}` / `reticle_session {action:"resume"}` / `reticle_session {action:"messages"}`, and **`reticle_session {action:"review"}`** to drain + fix the bugs the human flagged from the panel).
+
+## flows vs baselines vs project.json (the persistence layers)
+
+| Artifact | Tool(s) | What it is |
+| --- | --- | --- |
+| **flows** | `reticle_flow_save*` / `reticle_flow_replay` / `reticle_flow_heal` | Replayable **golden journeys**, anchored to testids/signals — drift is legible and self-heals. |
+| **baselines** | `reticle_baseline {action:"save"}` / `reticle_baseline {action:"diff"}` | Structural **"before" snapshots**; `reticle_baseline {action:"diff"}` flags regressions against them. |
+| **project.json** | `reticle_project` | Cross-run **run-history** — "did it behave like last run?" read via `reticle_project`. |
+
+> `reticle_project` / `project.json` are the **run-history layer**. flows answer "does the journey still work?"; baselines answer "did the structure change?"; project.json answers "is this run consistent with prior runs?".
+
+**Visual layer (opt-in).** `reticle_screenshot` saves a PNG baseline to `.reticle/visual/<name>.png`; `reticle_visual_diff` perceptually compares the live page to it (`{ masks }` to ignore volatile regions, `{ maxRatio }` tolerance) → `{ matched, changedPixels, ratio, region, diffPath }`. It answers "does it **look** right" — complementary to the behavioral layers, never a replacement. Both need a **driven browser** (`reticle drive <url>` / `RETICLE_CDP_URL`); without one they return `{ ok:false, reason:"no-visual-provider" }` (the always-on SDK ships no screenshotter).
+
+## Start here
+
+0. Just ran `reticle init` / started the dev server? Poll `reticle_sessions()` until your tab appears — readiness is server-internal now, so the first live call already blocks until the SDK connects.
+1. `reticle_sessions` — find the connected tab (omit `sessionId` if there's only one).
+2. `reticle_domain` — learn the app BEFORE testing: the saved flows, what each asserts, and the **gaps** (declared signals/testids that no flow verifies — untested intent). Tells you what to test and where the real risk is without crawling the whole app. Falls back to `reticle_capabilities` for the raw testable surface (`testids`, `signals`, `stores`, `flows`).
+3. Run the loop: **look → act → observe → assert**, cross-checking the 4 layers on anything that matters.
+
+## Token note
+
+- **Keep observation cheap.** Prefer `reticle_query` / scoped or `interactive` `reticle_snapshot` / `reticle_assert` over dumping the full tree. A full verify loop is ~100 tokens; see [token-efficiency.md](token-efficiency.md) (~73× leaner than full-tree snapshots).
+- **Re-look with `reticle_snapshot({ diff:true })`** after an action — it returns only what changed (`mode:delta`/`unchanged`), ~99% fewer tokens than a full re-snapshot and no stale tree to mis-read. Every snapshot/query result carries `cost:{ bytes, tokens }` — re-scope before reading if it's large.
+- **Cap broad reads.** `reticle_query` takes `limit` (caps descriptors; reports `total`/`truncated`) and `count_only` (just the match count). `reticle_network` / `reticle_console` take `limit` (most-recent-N, reports `droppedOldest`) and carry the same `cost` hint — so a busy page or wide window never floods your context unnoticed.
+- **A saved flow tells you if it's a real test.** `reticle_flow_save` returns `assertions.grade` (`asserted` / `presence-only` / `assertion-free`); if it's not `asserted`, add a consequence (`reticle_annotate` assert-signal/assert-net or a success-state) so it can't pass while broken. On replay, an ambiguous heal (two testids tie) is surfaced, never auto-applied — and an `apply` heal re-replays the rebound flow and **refuses to write** if the success consequence no longer fires (`status:consequence_broken`): it heals the locator, never the intent.
+- **Predicate schema is not bloated.** The recursive predicate DSL used by `reticle_assert` / `reticle_wait_for` / `reticle_act_and_wait` is **factored, not inlined**: when converted to the JSON Schema MCP sends, the predicate body is emitted **once** (~2.7k chars ≈ **~685 tokens** per tool) and recursion is handled by self-`$ref` (`#/properties/predicate`) — no per-recursion duplication. No action needed.
+- **One tool surface.** Reticle advertises the core set directly (sessions/navigate/snapshot/query/act/act_and_wait/observe/network/console/wait_for/assert/state/inspect/feedback/session — the whole detect loop, plus the file-pointer, the feedback channel and the handback) PLUS two meta-tools that keep every other tool one call away: `reticle_tools` (discover — no args lists every tool name + summary; `names:[…]` loads full params on demand) and `reticle_run({ tool, args })` (invoke any tool by name; a top-level `sessionId` is forwarded to the target, so you need not nest it in `args`). So to record/replay/verify a flow, call `reticle_run({ tool:"reticle_flow_verify", sessionId })` (or `reticle_tools` first to see params). There is nothing to pick. `RETICLE_ADVERTISE_ALL_TOOLS=1` (read by the daemon at startup, so restart it) advertises everything with output schemas — a verification switch for suites, not a way to run agents. **Sizes are deliberately not quoted here** — a count in prose goes stale and has three times already. `reticle_tools` reports the live surface, and SKILL.md carries the one gated table.

package/docs/architecture.md

@@ -0,0 +1,113 @@
+# Reticle architecture — how it works and why it's built this way
+
+> For engineers evaluating Reticle, integrating it at scale, or contributing. It explains the moving parts, the data flow, and the design decisions behind them. If you just want to get running, start with [getting-started.md](getting-started.md); come back here when you want to know _why_.
+
+---
+
+## The one-paragraph model
+
+Your app, in dev, embeds a tiny **SDK** that instruments the page (DOM, network, console, routing, framework state) and opens a WebSocket to a local **bridge**. The bridge runs inside the Reticle **server**, which also exposes an **MCP server** — the standard protocol AI agents speak. Your coding agent calls MCP tools (`reticle_query`, `reticle_act`, `reticle_assert`, …); the server turns them into commands over the WebSocket; the SDK executes them in the page and streams back structured events. The agent thus **looks, acts, observes, and asserts** on the real running app — never on a screenshot.
+
+```
+┌─────────────────┐     MCP (stdio/SSE)     ┌──────────────────────────┐     WebSocket      ┌────────────────────┐
+│   AI agent      │ ──────────────────────► │   @reticlehq/server     │ ◄────────────────► │  @reticlehq/browser│
+│ (Claude Code,   │   reticle_query/act/...    │   bridge + MCP + CLI      │   commands/events  │  (SDK in your app)  │
+│  Cursor, ...)   │ ◄────────────────────── │   (Node)                 │                    │   (the DOM)         │
+└─────────────────┘     tool results        └──────────────────────────┘                    └────────────────────┘
+                                                       │
+                                              reads/writes .reticle/  (flows, baselines, runs, contract)
+```
+
+---
+
+## The packages (and the boundaries between them)
+
+Reticle is a pnpm + Turborepo monorepo. The split is not cosmetic — each boundary enforces a rule.
+
+| Package | Runs in | Responsibility | Hard rule |
+| --- | --- | --- | --- |
+| `@reticlehq/core` | both | The **wire contract**: every constant + zod schema crossing any boundary | Depends only on `zod` |
+| `@reticlehq/browser` | the browser | Instrument the page; execute commands; emit events | Never imports Node APIs |
+| `@reticlehq/server` | Node | The bridge, the MCP server, the `reticle` CLI, flow/run storage | Never imports DOM APIs |
+| `@reticlehq/react` | the browser | The SDK **kit** you install in a browser app: re-exports the browser sensor (so one install gives both `reticle` and `install`) and maps a DOM node → React component → source `file:line` | Core works without the source-mapping half |
+| `@reticlehq/babel-plugin`, `@reticlehq/next`, `@reticlehq/vite-plugin` | build time | Stamp `data-reticle-source` for source mapping (and, for Vite, inject `connect()`) | Plain tooling |
+
+> Note: pre-2.0, `@reticlehq/core` was a single umbrella package that re-exported all of the above under subpaths. In 2.0 the umbrella was retired and `@reticlehq/core` became the bottom-of-graph wire contract.
+
+**Why core-as-contract matters:** because the browser and the server are two different runtimes (a DOM and a Node process) that must agree exactly on every message, the temptation is to inline a string like `"net.request"` in both. That's how drift and silent breakage start. Instead, every such string and shape lives once in `@reticlehq/core` as a named constant + a zod schema. The browser and server both import it; neither can invent a message the other doesn't understand. The server zod-parses **every** inbound WebSocket message — malformed input closes the socket rather than flowing into logic.
+
+---
+
+## The data flow, end to end
+
+1. **Connect.** In dev, your app calls `reticle.connect({ session })`. The SDK opens a WebSocket to the bridge (`ws://localhost:4400/reticle` by default) and sends a `HELLO` carrying the session id, protocol version, and (if configured) a pairing token. Each browser tab uses `SESSION_AUTO` — a unique id — so multiple apps/tabs never collide.
+2. **Capture.** The SDK installs observers: a `MutationObserver` for DOM changes, wrappers around `fetch`/`XHR` for network, a console hook, a history hook for routing, and registries the app opts into (`registerStore`, `registerCapabilities`, `reticle.signal`). Events flow into a bounded **ring buffer** — recent history is always available, memory is capped.
+3. **Look / act.** The agent calls an MCP tool. `reticle_query` finds an element by role/text/testid/ component and returns a stable **ref**. `reticle_act` dispatches an action against that ref and returns an **effect** report (did it land, did the DOM mutate, did focus move…). The server sends the command over the WebSocket; the SDK runs it and replies.
+4. **Observe.** After an action, the agent reads what happened — `reticle_network`, `reticle_console`, `reticle_state`, or the reaction digest from `reticle_act_and_wait`. The server pulls the relevant slice of the ring buffer (scoped to a cursor so stale events can't leak in) and returns a compact summary.
+5. **Assert.** `reticle_assert` (and a flow's declared `success`) evaluates a **predicate** over program truth — a network call that returned 200, a store value, a `signal` the app emitted — not just "an element exists." This is the difference between "looks done" and "is done."
+
+---
+
+## The four design decisions that define Reticle
+
+### 1. Assert the consequence, not the appearance
+
+Most agent-browser tools can confirm "an element matching X is present." That's the weakest possible oracle: a wrong or healed-to-wrong element satisfies it, and the regression ships anyway. Reticle grades evidence in tiers:
+
+- **Tier 1 — an app signal** (`reticle.signal('order:placed')`): the strongest, because a wrong element can't fake it. Available when the app emits it (a ~30-second opt-in).
+- **Tier 2 — network + route + state**: a POST returned 200, the URL changed, a store value updated. Strong, and works on most SPAs with no setup.
+- **Tier 3 — DOM/text presence**: the weak fallback Reticle nudges _away_ from.
+
+Reticle is honest about which tier a given assertion used, so "green" carries its own confidence.
+
+### 2. Structured reads, not screenshots
+
+A screenshot is ~1,365 image tokens per look, slow, non-deterministic, and **blind to everything non-visual** — the failed request, the console error, the route that didn't change. Reticle reads the accessibility tree, the network log, the console, and framework state as compact structured data. That's an order of magnitude cheaper _and_ it sees the bugs pixels can't. See [benchmarks.md](benchmarks.md) for the measured comparison.
+
+### 3. Record once, replay deterministically
+
+The same verification runs over and over — every commit, every CI run. Reticle records a **flow** once, then replays it with **no AI model**: it re-resolves each element's durable anchor against the live DOM and re-asserts the declared consequence. A CI gate diffs the verdict exactly, at ~0% flake, for a couple hundred tokens — versus an agent re-driving the whole flow with the model every time. Self- healing rebinds a drifted anchor **only if the consequence still fires**, so it never "heals to the wrong element and ships the regression."
+
+### 4. Dev-only, localhost-only, your app data stays local
+
+The SDK is tree-shaken out of production builds and connects only to a local bridge. The bridge binds to loopback by default; exposing it beyond localhost _requires_ a pairing token (the server refuses to bind a non-loopback host without one). Every environment variable that gates a security control is a single named constant, so a typo can't silently disable auth. Nothing from the app under test — no DOM, network, console, state, or source — ever leaves your machine. The CLI reports anonymous, opt-out usage metrics only (a random id + event names like `invoke`/`session_start`; no code, no PII — see [telemetry](telemetry.md)); opt out with `reticle telemetry disable`, `RETICLE_TELEMETRY=0`, or `DO_NOT_TRACK=1`. The one thing that carries free text is feedback you or your agent deliberately send (`reticle feedback` / `reticle_feedback`) — never collected passively, redacted before sending, and separately disabled with `RETICLE_FEEDBACK=0`.
+
+---
+
+## State on disk: the `.reticle/` workspace
+
+The server persists project state under `.reticle/` in your repo:
+
+- **`flows/`** — recorded, replayable flows (the golden journeys).
+- **`baselines/`** — saved snapshots for diffing.
+- **`runs/`** — verification run artifacts (the evidence trail; writes are atomic and bounded so a crash never leaves a half-written artifact, and the directory is pruned).
+- **the capabilities contract** — the testids/signals/stores/flows the app advertises, frozen under a version so the public artifact can't break silently.
+
+This is plain, reviewable, version-controllable data — not a black box.
+
+---
+
+## Running at scale (multiple apps, multiple projects)
+
+- **Multiple apps / tabs on one bridge:** fine — each connection has a unique `SESSION_AUTO` id; a tool call targets the focused/most-recent session, or you pass an explicit `sessionId`.
+- **Multiple isolated projects:** give each project its own bridge port via `RETICLE_PORT` (set it in the MCP config and dial the same port from the app). A port already in use fails fast with a clear error rather than hanging. See [getting-started.md → Running multiple apps](getting-started.md#running-multiple-apps-at-once).
+- **CI / no MCP:** `reticle verify <url>` replays the saved flows headlessly and exits non-zero on failure — the same verdict artifact the MCP path produces, with no agent in the loop.
+
+---
+
+## Open-core licensing (what's free, what's protected)
+
+- The embeddable **SDK** (`-core`, `-browser`, `-react`) is **Apache-2.0** — safe to ship inside your own app.
+- The **server / CLI** is under the **Functional Source License (FSL-1.1, Apache-2.0 future)** — source-available, converts to Apache-2.0 over time.
+- Enterprise-only features live behind a license gate and are clearly separated.
+
+See [LICENSE](../LICENSE) and each package's own `LICENSE` file. The licensing _mechanism_ is open and inspectable; activation is offline (no phone-home).
+
+---
+
+## Where to go next
+
+- [getting-started.md](getting-started.md) — wire Reticle into your app in a couple of minutes.
+- [benchmarks.md](benchmarks.md) — how we measure, and the honest results vs the alternatives.
+- [usage.md](usage.md) — the full tool reference and advanced modes.
+- [CONTRIBUTING.md](../CONTRIBUTING.md) — the development loop and the rules.

package/docs/benchmarks.md

@@ -0,0 +1,255 @@
+# How we benchmark Reticle (and why you can trust the numbers)
+
+> This page assumes **zero** testing background. By the end you'll understand what software testing is, why AI coding agents made it urgent, how to tell a good verification tool from a bad one, and exactly how Reticle measures up against the main alternatives — including the places Reticle loses. If a term looks like jargon, it's defined the first time it appears.
+>
+> _The headline charts below are rendered from the measured numbers (`assets/readme/`, rendered from the measured `history.jsonl` via `assets/benchmarks/render.mjs`); supporting diagrams live in `bench/artifacts/`._
+
+---
+
+## Part 1 — The problem, from first principles
+
+### What is "testing"?
+
+When you build software, **testing** is the act of checking that it actually does what it's supposed to. You click the button; does the thing happen? You submit the form; did the order get saved?
+
+There are roughly three ways teams do this:
+
+1. **Manual testing** — a human clicks through the app. Accurate, but slow and easy to skip.
+2. **Automated tests** — code that clicks through the app for you. Fast to re-run, but expensive to write and notoriously **flaky** (they break for reasons unrelated to real bugs).
+3. **Nothing** — ship and hope. More common than anyone admits: surveys put ~44% of teams with no automated testing at all.
+
+A **regression** is the specific bug this page cares about: something that _used to work_ and quietly _stopped working_ after a change. Regressions are insidious because the feature looked done — until a later edit broke it and nobody noticed.
+
+### Why AI agents made this urgent
+
+AI coding agents now write a large share of new code. They're good at _producing_ code. They are bad at one specific thing: **knowing whether the code they wrote actually works.** The agent edits files, says _"done ✅"_, and moves on. It never opened the browser. So **you** become its QA department — clicking around to discover that "done" wasn't done. Developers describe being _"gaslit by coding assistants."_ This is the **"done lie."**
+
+The honest fix is to give the agent a way to _check its own work_ — to drive the real running app and confirm the feature actually happened. That's what Reticle does. This page is about proving it does it **well**, not just that it does it.
+
+---
+
+## Part 2 — How would you even measure "good"?
+
+Imagine three verification tools. How do you decide which is best? Two things matter, and they pull against each other:
+
+- **Coverage** — does it actually _catch the bug_? A tool that misses regressions is useless no matter how cheap it is.
+- **Cost** — how much does it make the AI _read_ to do its job? Every observation is fed into the model's limited context as **tokens** (the unit models read/write in). A tool that floods the context with thousands of tokens per look is slow, expensive, and crowds out the actual work.
+
+A tool can cheat either axis: be cheap by looking at almost nothing (and miss bugs), or be thorough by dumping everything (and blow the budget). So the real metric has to reward **both at once**, with coverage as a hard floor.
+
+That gives us our headline number:
+
+> **Verification Efficiency = real regressions caught per 1,000 tokens spent looking** — and it only counts once the tool catches **100% of the bugs** with **zero false alarms** on a known-good control. (Historically abbreviated "VE"; the catch-rate floor was "RCR".)
+
+A **false positive** (or "false alarm") is crying wolf — flagging a bug when nothing is wrong. We require zero, because a tool you can't trust when it's quiet is a tool you'll learn to ignore.
+
+![Verification Efficiency: catches per 1,000 tokens, gated on 100% catch rate](../bench/artifacts/chart-detection.svg)
+
+---
+
+## Part 3 — The competitors, and why these three
+
+We compare Reticle against the two most credible agent-native browser tools, because they're what an AI agent would otherwise reach for:
+
+- **Playwright MCP** — Microsoft's tool that lets an agent drive a browser via the accessibility tree (not screenshots). The closest "serious" alternative.
+- **Chrome DevTools MCP** — Google's tool exposing Chrome's DevTools to an agent.
+- **(Baseline) screenshot agents** — the common "let the model look at a picture" approach.
+
+Why not compare to traditional test frameworks (Playwright the library, Cypress, etc.)? Because those are written and maintained by humans; they don't answer "can an _agent_ verify its own work in the loop." The three above do, so it's an apples-to-apples agent comparison.
+
+Tool versions are pinned in `bench/raw/run-meta.json` so any run is reproducible.
+
+---
+
+## Part 4 — The three measurement passes
+
+Different questions need different rigs. We run three (kept in the raw files as "Layer A/B/C"):
+
+1. **Observation-cost pass** (no AI model involved). For each bug scenario we run each tool's natural recipe and measure the exact size of what it returns — characters, bytes, and a tokenizer count. No model means no randomness: the token cost of _looking_ is measured precisely and repeatably.
+2. **Full-agent-loop pass** (a real model drives). Here an actual AI agent uses each tool end-to-end, and we record the _authoritative_ token usage the model reports. This is the real-world number; it needs an API key, so it's run periodically rather than on every change.
+3. **Replay pass** (no model). Reticle can record a flow once and _replay_ it deterministically with no AI at all. This measures the cost of re-checking a known flow — the thing a test suite does over and over, every commit.
+
+### The scenarios
+
+We inject **10 realistic regressions** into a demo app plus **one no-bug control** (to catch false alarms). The bugs span the failure modes that matter — and deliberately include ones that are _invisible to a screenshot_:
+
+| Scenario | The bug | Why it's here |
+| --- | --- | --- |
+| Hidden API 500 | A request silently returns a server error | A screenshot can't see a failed network call |
+| Wrong status 404 | A request 404s | Same — non-visual |
+| CORS blocked | A request is blocked by the browser | Non-visual |
+| Silent DOM removal | A KPI card vanishes with no error | Tests "did content disappear?" |
+| Route break | Navigation doesn't change the page | Common SPA bug |
+| Missing modal | A dialog fails to open | Interaction regression |
+| Console error, intact UI | The page looks fine but logs an error | Looks-fine-but-broken |
+| Layout shift | The grid jumps / shifts | Only visible in pixels/geometry |
+| Broken form validation | A form accepts bad input | Logic regression |
+| Network timeout | A request hangs forever | A request that never resolves never "logs" |
+| **Control (no bug)** | Nothing is wrong | Anything flagged here is a false positive |
+
+Three of these (failed request, console error, hung request) are **categorically invisible to a screenshot** — no number of pictures will ever catch them. That's a core part of the story.
+
+---
+
+## Part 5 — The honest results, on two very different apps
+
+We ran this two ways on purpose: a **controlled toy app** (where we can inject exact bugs and measure detection) and a **real production app** (where the numbers are messy and honest). The story holds in both — and the real app is where it gets interesting.
+
+![One honest test, two apps — Reticle has the highest Verification Efficiency on the controlled app and the lowest observation cost on the real dashboard](../assets/readme/bench-two-apps.png)
+
+### 5a — The controlled toy app (the demo)
+
+Measured on the observation-cost pass (numbers regenerate from `bench/raw/`):
+
+| Tool | Bugs caught (of 10) | Detection accuracy | Avg tokens per look | Verification Efficiency |
+| --- | --- | --- | --- | --- |
+| **Reticle** | **10 / 10** | **1.00** | **815** | **12.27** |
+| Chrome DevTools MCP | 8 / 10 | 0.82 | 758 | 10.55 |
+| Playwright MCP | 9 / 10 | 0.91 | 1,292 | 6.97 |
+
+- **Reticle is the only tool that caught every regression**, with zero false alarms on the control.
+- DevTools is a hair cheaper per look (758 vs 815 tokens) — **but it's cheaper because it catches less.** On the metric that combines both (Verification Efficiency), Reticle leads at 12.27 vs 10.55.
+- Playwright catches a lot but is ~1.6× more expensive per look, so its efficiency is lowest here.
+
+### 5b — A real production app (the Reticle dashboard)
+
+A toy app is a fair lab, but it's small. The harder, more honest test is a **real, complex app** — the [Reticle](https://reticle.sh) dashboard itself: React 19, authentication, live data, ~15 routes, a node-graph view, virtualized lists. We embedded the SDK (the Vite plugin) and drove the authenticated app with all three tools. Observing it **once** (the primary snapshot + the network log):
+
+![Observing the real Reticle dashboard once — Reticle 1,023 tokens vs DevTools 1,357 vs Playwright 2,193, and Reticle alone asserts success via the app's own signal](../assets/readme/bench-complex-app.png)
+
+| Tool | Snapshot | Network | **Observe total** | Can it assert success? |
+| --- | --- | --- | --- | --- |
+| **Reticle** | 678 | 345 | **1,023** | ✅ via the app's own `auth:logged-in` signal — **46 tok, un-fakeable** |
+| Chrome DevTools MCP | 1,105 | 252 | 1,357 | ❌ DOM/network only — can describe, can't verify intent |
+| Playwright MCP | 1,522 | 671 | 2,193 | ❌ DOM/network only |
+
+On a small page everything is cheap and the gap is modest. On a **big** page the structured read pulls ahead: Reticle is **2.1× leaner than Playwright MCP** and the cheapest overall. And only Reticle can read the app's own program state (`authenticated`, `userId`, `activeProjectId`) and assert the login _actually worked_ from a signal the app emits — the others can only look at the DOM and the network and **guess**.
+
+### 5c — The kicker: a real bug, caught live
+
+Here's what makes the real-app test matter. On the **very first run, before we instrumented anything**, Reticle's network observation flagged two endpoints returning **`500`** — `GET /api/v1/projects` and `/recovery/incidents`, both failing with `column "deleted_at" does not exist` (a database migration that hadn't been applied). **The page rendered fine.** The sidebar loaded, nothing looked broken — a screenshot agent would have called it _"done."_ Reticle saw the broken backend underneath, in one look.
+
+That is the whole thesis, demonstrated on a real app we didn't cherry-pick: **"looks done" ≠ "is done," and the difference is usually non-visual.** (We then fixed the migration; all endpoints 200.)
+
+### 5d — What each tool can actually do
+
+Cost is half the story; capability is the other half. The marks below are what's **first-class and built-in** to each tool (all three are good tools — they compose: drive with theirs, assert with Reticle):
+
+![Capability matrix — Reticle alone asserts via the app signal, reads program state, maps DOM to source, and replays deterministically; Playwright and DevTools win on driving sites you don't own and true pixels](../assets/readme/bench-capability.png)
+
+### Where Reticle loses (stated plainly)
+
+Honesty is the point of this page, so here are the places Reticle does **not** win:
+
+- **Raw cheapness on a single trivial look:** DevTools can be a few percent cheaper per observation when it isn't catching the harder bugs. We trade those tokens for catching more.
+- **True pixels:** a screenshot is the _actual rendered frame_. A bug that only shows up in real paint (a font that failed to load, a GPU/compositing glitch) can be caught by a screenshot diff and missed by Reticle's structural reads — _unless_ Reticle is explicitly asked to do a visual diff. We're honest that structure isn't pixels.
+- **The full-agent-loop pass** is measured for the three tools but a screenshot-agent variant is still future work; we don't claim a head-to-head there we haven't run.
+
+### The part that compounds: re-running
+
+The numbers above are for **one** verification. But a test suite's real job is the **same** check, over and over — every commit, every CI run. Here the picture changes shape:
+
+- Reticle records a flow once, then **replays it with no AI model at all** — re-resolving each element and re-asserting the outcome — for **~175–210 tokens per run**.
+- Playwright MCP and DevTools MCP have no replay: re-checking means an agent **re-drives the whole flow with the model every time**, costing tens of thousands of tokens per run.
+
+That's a **~128–184× cost difference per re-run**, and it grows with how often you run. This is where the "much cheaper than screenshots" claim becomes dramatic rather than incremental.
+
+![Re-run cost: Reticle replays with no model; competitors re-drive every time](../bench/artifacts/chart-avg-tokens.svg)
+
+### The large-page test (where the wedge is biggest)
+
+On a small page, structured reads and screenshots are both cheap, so the gap is modest. The advantage shows on a **big** page. On a deliberately large grid (thousands of DOM nodes):
+
+- A **full page snapshot** costs ~**3,636 tokens**.
+- A **targeted verify loop** (find one button → act → assert the success signal) costs ~**279 tokens** — and stays flat as the page grows to 5,000 rows.
+
+That's a **13× difference**, and against a screenshot agent it's larger still: one screenshot at a normal window size is ~**1,365 image tokens** _per look_, a real loop takes several looks, and the screenshot still can't see the non-visual bugs.
+
+### Many agents at once (the parallel wedge)
+
+A fleet of agents (or a parallel suite) verifying the same app doesn't need a browser each. Reticle keeps **one** headless Chromium and leases each agent an **isolated context** (separate cookies/storage/DOM). Measured on 16 flows: **35.4s** one-at-a-time vs **5.2s** across 8 leased contexts — **6.78× faster**, ~30s saved per batch, 8-way peak concurrency. The alternative — a browser per agent — costs hundreds of MB and seconds of startup each; the pool's edge grows with agent count up to its cap (`multi-agent-throughput`).
+
+---
+
+## Part 6 — Why we believe these numbers (fairness + reproducibility)
+
+- **Every number comes from a committed harness**, not a slide. Re-run it: `pnpm bench`.
+- **A no-bug control** runs in every pass, so a tool can't look good by flagging everything.
+- **Tool versions are pinned**; the raw payloads are saved to `bench/raw/` so anyone can audit them.
+- **The token counter is a tokenizer**, not a guess — and where we use a proxy tokenizer instead of a specific model's, we say so and lean on _relative_ differences, which are robust to the choice.
+- **We publish where we lose** (above). A benchmark that only flatters its author isn't a benchmark.
+
+---
+
+## Part 7 — What this means for you
+
+- **If you're a developer using an AI agent:** Reticle lets the agent confirm its own change actually worked — across the network call, the console, the route, the state — for a few hundred tokens, not a screenshot's thousands. "Done" starts meaning done.
+- **If you run CI / a platform team:** the replay pass is the headline — deterministic re-verification with no model, so a known flow re-checks for ~200 tokens at ~0% flake, every commit.
+- **If you're evaluating tools:** the axis that matters is _bugs caught per token, gated on catching them all_. Cheapness that misses regressions is a false economy.
+
+You are now equipped to read any verification benchmark critically: ask "what's the catch rate, what's the false-positive rate, and what's the cost _per re-run_?" — and be suspicious of any vendor that won't show you where they lose.
+
+---
+
+## Part 8 — At enterprise scale (the honest "if a big company used this")
+
+A fair question from a large engineering org (think a Datadog- or Salesforce-sized team): _does this actually pay off at our scale, or only in a demo?_ Here's the honest accounting, with every assumption stated so you can re-run it with your own numbers. **These are projections from the measured per-run costs above, not a measured enterprise deployment — labeled as such.**
+
+### Where it pays off (and compounds)
+
+The win is **regression re-runs**, because that cost is paid over and over. Take a mid-size surface:
+
+- **50 golden flows**, re-verified on **every PR**, at **200 PRs/day**.
+- Reticle replays each flow **deterministically, with no model**: ~**200 tokens/flow/run** (measured).
+- The agent-driven alternative re-drives each flow with an LLM: ~**30,000 tokens/flow/run** (measured Layer-C comparison).
+
+|  | Per flow / run | 50 flows × 200 PRs/day | Per year (~250 working days) |
+| --- | --- | --- | --- |
+| Reticle replay (no model) | ~200 tok | ~2.0 M tok/day | ~0.5 B tok/yr |
+| Agent re-drive (LLM) | ~30,000 tok | ~300 M tok/day | ~75 B tok/yr |
+
+That's a **~150× difference on the recurring axis**, and it scales linearly with flows × runs — the two numbers an enterprise has _a lot_ of. The deterministic replay also means **~0% verdict flake** (measured over repeated identical runs), which at this scale is the difference between a trusted gate and one engineers learn to ignore (recall: flaky suites get abandoned — that's the documented failure mode Reticle is built to avoid). And because replay needs no model, it has **no per-run API spend and no rate-limit ceiling** — it runs in CI like any other deterministic check.
+
+### Where it does NOT help (stated plainly)
+
+An honest enterprise evaluation has to include the limits:
+
+- **Authoring still costs model tokens.** Recording/annotating a flow the first time uses an agent. The economics only turn positive once a flow is re-run enough times to amortize that — which is exactly the enterprise case (many runs), but a flow you run twice won't pay off.
+- **Tier-1 oracles need instrumentation.** The strongest, un-fakeable assertions come from app signals the team adds (a ~30-second opt-in per success event). Without them you get Tier-2 (network/route/ state) — still strong, still better than "element exists," but not the magic tier. Be realistic about the instrumentation rollout across a large codebase.
+- **It is not a full replacement for an existing E2E platform on day one.** Reticle is verification in the dev/agent loop and in CI; a large org with a mature Playwright/Cypress estate adopts it alongside, starting with the agent loop and the highest-value journeys, not as a big-bang migration.
+- **Structure is not pixels.** As in Part 5: a purely visual regression (a paint/font/compositing bug) needs the opt-in visual diff; the always-on structured reads won't catch it.
+- **Per-project isolation is a setup step.** Multiple apps/teams on one machine each want their own bridge port (`RETICLE_PORT`) — trivial, but it's configuration a platform team should standardize.
+
+### The honest bottom line for a large org
+
+Reticle is **most efficient exactly where big companies hurt most**: a large number of journeys re-verified a large number of times, where an LLM-re-drive approach's cost grows without bound and a flaky human-authored suite gets abandoned. The recurring cost drops by ~two orders of magnitude and becomes deterministic and model-free. It is **least differentiated** for one-off checks, purely visual regressions, and before the team has invested in Tier-1 signals. A sober rollout: start with the agent dev-loop, instrument the top revenue-critical journeys with success signals, wire `reticle verify` into CI for those, and expand from there.
+
+---
+
+### Appendix — the raw artifacts
+
+- `bench/SCORECARD.md` — the one-page standing with the plain-language legend.
+- `bench/METRIC.md` — the exact metric definitions.
+- `bench/METHODOLOGY.md` — full design: controls, scenarios, fairness.
+- `bench/history.jsonl` — every measured run over time.
+- `bench/raw/` — raw payloads and per-scenario results (gitignored locally; regenerated by `pnpm bench`).
+
+---
+
+## SDK overhead — what instrumentation costs the app
+
+An observability layer that slows the app corrupts its own performance verdicts, so Reticle holds itself to a budget: **total instrumentation overhead below 3% of main-thread time**, measured on the _hostile_ fixture — a page that never goes quiet (a ~10 messages/sec feed, a 60fps ticker, a large list), not a polite demo.
+
+The same page is loaded in one browser under three conditions — full SDK, observers with the HUD suppressed, and Reticle absent — and Chrome's own cumulative `TaskDuration` is compared over an identical window (condition order rotated so warm-up can't favour any one of them). Three conditions, not two, so that what instrumenting costs is separable from what the HUD costs.
+
+|                               | main-thread task time | busy                           |
+| ----------------------------- | --------------------- | ------------------------------ |
+| full (observers + HUD)        | 1.669 s               | 20.9%                          |
+| observers only (`?nopresent`) | 1.627 s               | 20.3%                          |
+| Reticle absent (`?no-hud`)    | 1.610 s               | 20.1%                          |
+| **instrumentation**           | **+0.21 pp**          | below the ±1.23 pp noise floor |
+| **presenter HUD**             | **+0.53 pp**          | opt out with `present: false`  |
+
+**Result: instrumentation overhead is smaller than the method can resolve — report it as `< 1.2 percentage points`, inside the 3% budget.** The HUD is measured and reported separately, because it is a developer-visible affordance you can turn off, not a cost of observing.
+
+Getting here required correcting the method and then fixing a real bug. The two-condition version of this bench compared the full SDK against Reticle being absent, so it charged instrumentation for the HUD too, and reported **+5.85 pp — FAIL**. Decomposing it showed almost none of that was instrumentation: a single `backdrop-filter: blur(24px)` on the HUD panel was costing **+4.03 pp** on its own — more than every observer combined — because a backdrop blur re-samples everything behind it whenever that content changes, and this fixture repaints at 60fps. Removing it took total main-thread time down 22%. Full history, including a fix that measured nothing and was reverted rather than kept, is in `bench/overhead/README.md`. Reproduce it yourself with `node bench/overhead/measure.mjs` (see `bench/overhead/README.md`, which records the ruled-out causes). causes).

package/docs/debugging.md

@@ -0,0 +1,102 @@
+# Debugging Reticle itself
+
+> For people working **on** Reticle — and for an agent asked to explain why a flow behaved the way it did. If you are debugging **your app** with Reticle, you want [usage.md](usage.md) instead.
+
+Reticle produces four different signals and they answer four different questions. Reaching for the wrong one is why investigations here have historically started by reading source.
+
+| Question | Signal | Where |
+| --- | --- | --- |
+| What did the **agent** do to the app, and what did the app do back? | the journal | `reticle_observe` (the event window), `reticle_run_export` (a saved run), `~/.reticle/journal/` |
+| What did **Reticle** do internally to answer that call? | the **trace** | daemon log, with `RETICLE_TRACE=1` |
+| What happened at a milestone (daemon start, skew, refusal)? | structured log lines | daemon log |
+| How much is this being used, and does it work? | telemetry | [telemetry.md](telemetry.md) |
+
+## The daemon log
+
+The daemon writes newline-delimited JSON to:
+
+```
+~/.reticle/daemon-<port>.log        # or $RETICLE_STATE_DIR/daemon-<port>.log
+```
+
+`reticle doctor` prints the exact path for your port. Tail it while you work:
+
+```bash
+tail -f ~/.reticle/daemon-4400.log | jq .
+```
+
+## Verbose flow tracing
+
+Off by default, because a trace on every tool call is a cost on the hot path — a verification loop is 50–200 calls. Measured, so the claim is checkable: a disabled span costs **126ns** per site (against ~9ns for a bare call), which is under a microsecond per tool call and not the literal zero it is tempting to write. Turn it on **for the daemon** (the flag is read per call, but the daemon is the process doing the work, so it has to be set where the daemon starts):
+
+```bash
+RETICLE_TRACE=1 reticle serve --port 4400
+```
+
+Every instrumented stage then emits one line **when it ends**, carrying its own duration:
+
+```json
+{"event":"trace","span":"browser.command","ms":412,"depth":1,"callId":"p8123-c7","ok":true,"command":"act"}
+{"event":"trace","span":"tool.handler","ms":430,"depth":0,"callId":"p8123-c7","ok":true,"tool":"reticle_act"}
+```
+
+Read it like this:
+
+- **`callId`** groups every stage of one tool call, and is prefixed with the daemon's pid so two daemons (or one that restarted) can never claim the same id. Several agents can be inside the daemon at once, so lines interleave — the id is the only thing that makes the output a tree instead of a pile.
+- **`depth`** is the nesting level. `0` is the tool handler; anything deeper ran inside it.
+- **`ms`** is that stage's own wall-clock. A parent's `ms` includes its children's.
+- **`ok:false`** carries an `error` field. A stage that threw is still traced — otherwise the trace would show a call that entered a stage and never left, which reads as a hang.
+
+One line per stage, at the end, is deliberate: a start line as well would double the volume to say something the end line already implies.
+
+### Where the time goes
+
+The spans that ship today:
+
+| Span | What it covers |
+| --- | --- |
+| `tool.handler` | the whole tool call, at the one dispatch point both MCP and programmatic callers pass through |
+| `browser.command` | one round-trip to the page |
+| `flow.step` | one step of a replayed flow, with its anchor kind |
+| `crawl.step` | one control the crawl clicked, including its settle budget |
+| `init.plan` / `init.apply` / `init.exec` / `init.exec.retry` / `init.write` | the install, phase by phase |
+
+The first two answer _"is this slow because of us or because of the app under test?"_ —
+
+`browser.command` close to `tool.handler` means the app is taking the time. A large gap between them is Reticle's own overhead, and that is a performance bug of ours.
+
+Filter to one call:
+
+```bash
+grep '"event":"trace"' ~/.reticle/daemon-4400.log | jq -c 'select(.callId=="p8123-c7")'
+```
+
+Or find the slowest stages across a session:
+
+```bash
+grep '"event":"trace"' ~/.reticle/daemon-4400.log | jq -sc 'sort_by(-.ms)[:10] | .[] | {span,ms,tool}'
+```
+
+### The init flow
+
+`reticle init` is synchronous end to end, so it uses `spanSync` — same line, same tree:
+
+```
+init.plan        2ms
+init.exec       133ms  target=package.json command=pnpm
+init.exec.retry 128ms  target=package.json command=pnpm
+init.write        1ms  target=.reticle.json
+init.apply      262ms  steps=5
+```
+
+Two things that trace makes visible and nothing else did. Planning is ~2ms, so init's wall-clock is essentially all subprocess: the package manager, and `claude mcp add` when MCP registration is on. And when the pinned install is refused, the unpinned **retry is a second full package-manager run** — init legitimately takes about twice as long on that path. Every `--local` fixture run takes it, because the version being installed is not published yet.
+
+### Adding a span
+
+```ts
+import { span } from '../trace.js';
+
+const result = await span('predicate.eval', { kind: predicate.kind }, () => evaluate(predicate));
+```
+
+Wrap a stage worth a line in a bottleneck hunt — a session resolve, a browser round-trip, a predicate evaluation. Not every function: the trace earns its keep by being faster to read than the code, and that stops being true somewhere around one line per statement. Nesting needs no plumbing — the call id and depth ride in `AsyncLocalStorage`, so a span added five frames down still lands in the right tree without changing anyone's signature.