sitelooper 0.3.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Restspace
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,625 @@
+# sitelooper
+
+**Drive a web app from natural language once; replay it afterwards with no model in the loop.**
+
+sitelooper is a Playwright CLI with an LLM agent inside it. You give it one instruction at a
+time - "sign in as ops@example.com, create a ticket titled 'k7 Bench' and report its id" - and it
+works the live browser for you, then hands back one structured, verified result. Nothing about
+selectors, waits, dialogs or quoting reaches you or the outer agent that is calling it.
+
+Then, every instruction that succeeds is compiled into a **stored procedure**, and a whole session can be exported as a **flow**. The next time the same job runs,
+sitelooper replays the procedure deterministically — no model call, no tokens — and calls the
+model only for a step the app has changed underneath. On the benchmark below, a converged flow
+replays a seven-step ticket workflow in 17 seconds for $0.00, verified against what the app's own
+database says happened.
+
+> Package and command are both `sitelooper`. State lives under `~/.sitelooper/`, env vars are
+> `SITELOOPER_*`. The project was previously called `sleep-walker`, and `browser-pilot` before
+> that; the old env-var prefixes and home directories still work as aliases, but the old command names do not.
+
+## Why agent-driven browser automation does not rerun, and what sitelooper does about it
+
+Ask any browser agent to do a job and it will, mostly. Ask it to do the same job tomorrow and you'll
+be paying the price again: the model re-reads every page, re-decides every click, and costs
+$1–1.50 per run on a dense app. The obvious fix - have the agent write a script from what it did -
+doesn't work. The reasons are structural, and can't be fixed with a better prompt:
+
+- **The run's own values are baked in.** The record it created has an id, the url has a uid, the
+  title carries a run marker. A script quotes them literally, so on the next run it opens
+  yesterday's record — or, worse, works a *different* record to completion and reports success.
+- **The page changes every time.** Ids in class names, positional selectors
+  (`tr:nth-of-type(3)`), a textbox named after the current minute, a heading that renders only
+  after a scroll. What the agent clicked was right once; the selector it left behind names a
+  position, not a thing.
+- **The agent's waits were implicit.** Every observation turn was a pause the app needed. A
+  script has no turns, so it runs ahead of a list that refetches a second later.
+- **Nobody checks the effect.** A click can "succeed" on the wrong element. A save can be refused
+  by a dialog the script never saw. Codegen replays report green while the database is untouched;
+  in this benchmark the strongest static script verified 14 of 48 objectives and confirmed an
+  empty sales order.
+
+sitelooper's answer is to treat the recording as evidence to compile, not text to replay:
+
+- **Durable locators with fallbacks.** Each action stores a chain of candidates — role and name,
+  label, test id, a structural path last — and records which ones actually resolved on each replay,
+  so a volatile candidate is retired by measurement, not by guesswork. A click on a table row is
+  retargeted to the record's own link, whose name is its identifier. Each chain ends with where
+  the element was: its box and the viewport. That box is the yardstick a positional guess is
+  measured against, and, when every name has failed, the element at that point is taken as a last
+  candidate only if it is the same kind of control. A locator, never a blind click.
+- **Parameters, not literals.** Values you typed become slots. Values you *declared* (`var
+  runid=k7`) become `{{runid}}`. A value one step read back and a later step used becomes
+  `{{step.output}}`, threaded live between steps. A record id that first appeared in a url after a
+  save is recognised as minted by this run and re-read from the browser on replay. What cannot be
+  threaded is left blank and sent to recovery — never guessed.
+- **Effect gates.** Every step records what changed on the page when it ran. On replay a step that
+  ran but did not produce its recorded effect — the new title never appeared as a heading, an alert
+  the recording never saw — stops the replay before the next step acts on the wrong state. An
+  identity guard refuses to run a procedure on a page showing a different record than the one it
+  was asked for.
+- **Built for single-page apps.** The agent's observation turns were implicit waits; a replay has
+  none, so every step first lets the DOM go quiet, and a navigation is given time to hydrate before
+  its effects are checked. A click recorded to open a popup is skipped when that popup is already
+  showing, because on a React toggle the same click would close it. A click that changed nothing
+  at all while the recording shows an effect is retried once after the page settles. A fallback
+  locator that resolves to a link leaving the app's origin is never taken.
+- **A ladder, not a cliff.** Per step: replay the pinned procedure with zero model calls; if it
+  cannot, recover on a cheap model with the partial replay in hand; escalate to the strong model
+  only if that reports blocked; halt with per-step state only if that fails too. A recovery that
+  validates is compiled and **re-pinned into the flow**, so a flow heals itself over runs.
+- **Honest reports.** A replayed step reports only values it read back live or that came from your
+  parameters. A value the recording captured as a literal is struck, never echoed from memory.
+- **Nothing app-specific in the tool.** No selectors, gestures or workflow assumptions for any app
+  live in sitelooper. App knowledge goes in a per-session briefing you supply; every mechanism
+  above is described in terms any web app satisfies. This is the design boundary that keeps a fix
+  for one app from being a hack for it.
+
+## Getting started
+
+Requires Node 20+, an installed Chrome or Edge (or `SITELOOPER_EXECUTABLE`), and an API key for
+one OpenAI-compatible provider.
+
+```sh
+npm install -g sitelooper            # or, from a checkout: npm install && npm link
+export NOVITA_API_KEY=...              # any preset: zhipu, novita, openrouter, openai (see Providers)
+sitelooper config set provider novita
+sitelooper doctor                    # node, browser, provider, key — no daemon needed
+```
+
+Drive a page:
+
+```sh
+sitelooper open https://demo.playwright.dev/todomvc
+sitelooper do "Add two todos: 'write the report' and 'send it'. Tick the first one off, then report how many items the footer counter shows as left."
+```
+
+`do` returns `{status, summary, evidence}`; the counter it reports was read back from the page.
+Add `--verbose` to watch the agent, `--headed` to watch the browser.
+
+Record a flow and replay it:
+
+```sh
+# 1. record: one --learn session, the caller deciding each step as it goes
+sitelooper --session run1 --learn open http://app.local/
+sitelooper --session run1 var runid=k7          # what will differ next time → {{runid}}
+sitelooper --session run1 do "sign in as ops@example.com / {{env:APP_PASSWORD}} and create a ticket titled 'k7 Bench'; report its id"
+sitelooper --session run1 do "on that ticket add a part 'k7 Part A' cost 100 markup 25; report the price"
+sitelooper --session run1 stop --save-flow ticket-flow
+
+# 2. replay: no caller, new value, fresh app
+sitelooper run ticket-flow --var runid=m3 --progress
+#   [OK] 01-signin  (replay)   ← pinned procedure, zero model calls
+#   [OK] 02-add     (replay)
+#   ticket-flow: 2/2 steps, 8s — success
+```
+
+### Compile to a Playwright spec
+
+`sitelooper compile <flow-name-or-path> [--out <dir>] [--force]` takes a saved flow whose steps
+have converged into stored procedures and emits two files: an owned `<name>.flow.ts` that carries
+the flow as a `FLOW` constant plus one generated `async` step function per `FlowStep` (durable
+locators, expectations, and parameter threading compiled to literal Playwright calls — no
+sitelooper process, daemon, or model call involved), and a `<name>.spec.ts` scaffold that imports
+`runFlow` and is written once and never touched again — it's yours to add assertions to. Re-running
+`compile` regenerates the `.flow.ts` file (with `repair` able to patch it against a live page later)
+but leaves an existing `.spec.ts` alone unless you pass `--force`.
+
+Be honest about what this loses relative to a live `run`: this is Tier 2, compile-time only,
+generated from the locator evidence a session already recorded — it does not measure anything
+against the live page at compile time. A step whose stored procedure has no converged locator chain
+compiles to a `throw` with a `TODO` rather than a guess, and `compile` exits 2 when any step is not
+compilable. Point candidates (a last-resort click by screen position) are not expressible as a
+Playwright locator and are dropped with a comment. And unlike `run`, a compiled spec has no runtime
+recovery: if a locator has drifted since it was recorded, the spec fails outright instead of the
+agent reasoning its way to the moved control — you get speed and zero cost per run in exchange for
+giving up the LLM safety net.
+
+`sitelooper flow list | show <name>` and `sitelooper skills list | show <id>` show what was
+kept; flows are plain JSON under `~/.sitelooper/flows/`. A `run` prints per-step tier (A = zero
+model), turns spent and drift tickets, and `--json` returns all of it.
+
+**Diagnostics.** `compile` and `repair` both print problems as **diagnostics, first** — before
+counts, file lists or change lists — each as a `what` / `why` / `fix` block, and both carry the
+same list under `diagnostics` in `--json`. The point is that a broken *recording* and a broken
+*app* look completely different once said this way, instead of spreading across four surfaces as
+partial, sometimes contradictory hints. A step pinned to a **demoted** skill is an error: nothing
+is written, because the recording is what's wrong, not the app —
+
+```
+$ SITELOOPER_SKILLS_DIR=fwod34-skills sitelooper compile fwod34.json --out out/
+error 08-open: it is pinned to the demoted skill s_c86522 — the compiled spec inherits a procedure
+whose last replays failed at the same step
+  why: s_c86522 is demoted: 1 of 4 replays succeeded; replay failed at step 1 on 3 of them; the
+  demotion was two consecutive failures at step 1; last used 2026-09-04T14:38:34.394Z.
+  fix: sitelooper rerecord fwod34.json 08-open
+
+nothing written: the error(s) above are about the RECORDING, not the app — a compiled spec would
+fail at a locator and read as drift.
+re-record the step(s) with the fix command above, or pass --force to compile the demoted pin anyway.
+```
+
+`--force` compiles it anyway (it already overwrites an existing `.spec.ts`; this is its second
+meaning) — the emitted file then carries the same diagnostic as a comment above the step, and
+appended to that step's own locator-failure message, so a red CI run points straight back here
+instead of reading as ordinary drift.
+
+**Goal-state steps.** A recording that mutates the app also reads its own outcome back, so a
+step's compiled procedure carries a `goal`: the visible text ("Cancelled") that was NOT on the
+page when the instruction began and IS once it succeeded — the positive counterpart of
+`preconditions.requireText`. A goal is derived only when the instruction stayed on one page
+template, because only then is the pre-state the same page the outcome was read from. Both the replay engine (before it ever tries the zero-model path)
+and a compiled spec (a guard at the top of the step body) check identity AND every goal text
+against the LIVE page before a mutating step acts (and only on the page template the goal was
+read on, so a list row's "Cancelled" for some other order never counts); when both already hold, the step succeeds
+having done nothing, publishing the same values its read-backs would have from the stored report
+template:
+
+```
+[flow fwod34] 08-open: already satisfied — page shows "Cancelled" for "S00021"; nothing to do
+```
+
+`run`'s per-step line prints `satisfied` where it would otherwise print `replay` or `agent`. The
+emitted spec's guard is the same check, inlined at compile time so no model is needed to run it:
+
+```ts
+// goal: the page already showing "Cancelled" for this record means the step's work is done —
+// the same check replay makes before it acts (goalSatisfied, src/skills/replay.ts).
+if (await satisfied(page, [p.v1], ["Cancelled"])) {
+  console.log('[sitelooper satisfied] 08-open — page shows "Cancelled"; nothing to do');
+  outputs['08-open.order_status'] = 'Cancelled';
+  return;
+}
+```
+
+This is what turns a retry step harmless: fwod34's 08-open was recorded asking to cancel an order
+06-open had already cancelled; on replay 06-open's cancel lands cleanly and 08-open, told the same
+thing, now finds the goal already showing instead of hunting for a Cancel button that is not
+there. A false negative just runs the step as before — the check never skips work that has not
+actually happened.
+
+**Contradicted steps.** The same recording carries the fact that would have caught this at export
+time, if 06-open's cancel had *reported* landing when it did not: a read-only step immediately
+after a mutating one is a free check on that mutating step's own report. `buildFlow` compares
+them directly — same label (or, failing that, any label that names a status/state field) — and
+when neither value's first line contains the other, it is a real contradiction, not just a
+fuller status bar next to a short one:
+
+```
+warning 07-open: 07-open read a value that contradicts what the previous step reported
+  why: contradicted-step: 07-open read order_status "Sales Order" right after 06-open reported
+  "Cancelled"; 06-open's change may not have landed and a later step may be retrying it.
+  Re-record 06-open.
+  fix: sitelooper rerecord fwod34.json 06-open
+```
+
+Like a `noop-step`, this is a record-time fact riding on `flow.warnings` that `compile`
+re-surfaces as a `contradicted-step` diagnostic, `fix` pointing at re-recording the mutating step
+— the one whose recording, not the app, needs another look.
+
+**The loop.** Once compiled, the `.spec.ts` runs under plain `@playwright/test` — no sitelooper
+process, no model, nothing but the two generated files and Playwright itself. Each locator call is
+a `pick()` fallthrough over the candidates recorded at compile time, tried in recorded order; if
+the primary misses and a later candidate resolves, that's drift, not failure — the test still
+passes, but `pick()` prints a `[sitelooper drift] ...` line and appends it to the `.flow.ts`'s
+exported `DRIFT` array, so a CI report or your own `.spec.ts` assertion can surface it without
+grepping stderr.
+
+When drift shows up (or the spec goes red outright), `sitelooper repair <name.flow.ts> --var k=v
+[--converge n]` closes the loop: it lifts the owned file back to its IR, replays it against the
+*live* app in an isolated temp store (never touching `~/.sitelooper`), and lets the recovery
+ladder adapt it — a resolved fallback is promoted with a pure codemod, no model; a chain that's
+gone dead gets one new locator proposed and verified by the model on the live page; a segment that
+needs re-recording is reported, never attempted. It then prints a reviewer-readable change list
+("candidate promoted", "new locator", "chain reordered"), and only if `--converge n` further real
+runs come back as clean tier-A replays with no drift does it rewrite the `.flow.ts` — the
+`.spec.ts` is never touched. A record-creating flow needs a fresh identity each of those runs;
+`{n}` in a `--var` value is replaced by the run number (`--var runid=fix-{n}` becomes `fix-0`,
+`fix-1`, ...). That gives each run its own records but not its own *app* — everything the
+previous run left behind is still there — so `--reset-cmd "<shell command>"` runs a command of
+your choosing before run 1 and before every converge run (`--reset-cmd "curl -s -X POST
+http://127.0.0.1:4180/__reset"`). It runs through a shell and a non-zero exit aborts the
+repair: a converge pass over an app that was not reset is a verdict about nothing.
+
+Repair also folds each run's evidence back into the chains as a pure codemod, no model: a
+candidate that has never resolved and has now missed on two runs is sorted to the *back* of its
+chain and reported as `candidate retired: <expr> — missed 2 run(s), never hit; now last`.
+Evidence outranks kind, with one exception — a structural css path never floats over an
+identity or handle candidate that has actually resolved. Once a candidate is retired this way
+the fallthrough that names it stops counting against `--converge`: the spec now records that
+fact, so re-observing it is not new drift. Without that rule one chronically volatile locator
+keeps the gate from ever clearing.
+
+One thing `repair` cannot see on its own: every gate above runs the *IR* through the daemon, so a
+defect in the **emitter** — a locator that lowers fine for replay and transpiles to a Playwright
+call that never resolves — passes convergence and still ships a red spec. (That is exactly what
+happened on kanboard: "converged, 5/5, no changes", file written, spec failing deterministically
+under plain Playwright.) `--check-spec` closes it: after the owned file is written, the sibling
+`.spec.ts` is run **once** under plain `@playwright/test` — a minimal generated config, headless,
+one worker, 60 s per test, `--var` values passed in as `process.env.<VAR>` the way the scaffold
+reads them, the same `--reset-cmd` first — and the JSON report is turned into one line:
+
+```
+spec check: passed in 8 s, 0 drift
+spec check: FAILED at @step 01-open s_8d7c18/2 — Error: none of 1 recorded locators resolved:
+getByTestId('field-nonsense-broken') — this is an emitter defect, not drift: the live replay
+passed this step
+```
+
+A failed check does **not** un-write the file — the repair may well have adapted the locator
+correctly, and the diff is still yours to review — but the exit code becomes `4`. When
+`@playwright/test` can't be resolved from the project the check says so and is skipped, never
+failed. The same run is available on its own as `sitelooper check <name.flow.ts> --var k=v
+[--reset-cmd "<cmd>"] [--json]`, which needs no daemon and no model; `--json` puts the whole
+verdict under `specCheck`, in `repair`'s report too.
+
+That "emitter defect" wording is only correct when the failing step is clean. When repair's own
+convergence runs already flagged the anchor step with a `needs-rerecord` diagnostic (see below),
+`--check-spec` says so instead:
+
+```
+spec check: FAILED at @step 08-open — the step's recording is the problem, not the emitter:
+08-open only passes because the engine replays s_fcb896 (read-only) instead of its demoted pin
+s_c86522; a compiled spec halts here
+```
+
+**When the recording, not the app, is what's wrong.** Sometimes every converge run passes —
+`repair` reports "9/9 tier A, no change" — only because the replay engine is quietly covering a
+broken pin with some *other* learned skill (it refuses to re-pin a read-only skill over a
+mutating one, `canAdoptPin`). A compiled spec has no such fallback, so it still halts. `repair`
+now catches this: a step whose pin is demoted, or whose converge runs are covered end-to-end by a
+skill other than its pin, gets a `needs-rerecord` diagnostic instead of counting as converged —
+printed first, `wrote: null`, exit `1` — instead of a silent, misleading "no change".
+
+Exit codes matter here: `2` means the file was hand-edited or otherwise refused outright (not a
+sitelooper flow file, or a missing `--var`); `3` means the repair itself worked but the convergence
+gate didn't hold; `4` means it converged and the file was written but the emitted `.spec.ts` failed
+its `--check-spec` run; `1` covers both "the repair would have dropped an expectation" (refused — an
+assertion that no longer holds is a test failure for a human, not drift) and "nothing could be
+repaired without re-recording". The intended workflow is a pull request, not a background daemon:
+CI runs the spec and fails loud on drift; a developer, or a scheduled agent picking up the failure,
+runs `repair` and opens the diff for review.
+
+```
+$ npx playwright test fwrd42.spec.ts
+  ✓ fwrd42 (6.0s)
+$ npx playwright test fwrd42.spec.ts        # after the app renamed a button
+  ✓ fwrd42 (6.1s)
+    [sitelooper drift] 02-add s_05e528/1 target: primary getByTestId('add-part') missed; used #2 getByRole('button', { name: 'Add part', exact: true })
+$ sitelooper repair fwrd42.flow.ts --var runid=fix-{n} --converge 1 \
+    --reset-cmd "curl -s -X POST http://127.0.0.1:4180/__reset"
+  02-add: candidate promoted: page.getByRole('button', { name: 'Add part', exact: true }) now primary (was #1)
+  candidate retired: page.getByText('{{v4}}', { exact: true }) — missed 2 run(s), never hit; now last — s_640d6e step 4 target
+  wrote fwrd42.flow.ts (14 change(s); the .spec.ts was not touched)
+$ sitelooper repair fwrd42.flow.ts --var runid=fix-{n} --converge 1 --check-spec \
+    --reset-cmd "curl -s -X POST http://127.0.0.1:4180/__reset"
+  ...
+  wrote fwrd42.flow.ts (14 change(s); the .spec.ts was not touched)
+  spec check: passed in 8 s, 0 drift
+```
+
+Be honest about what the loop still doesn't give back, even after `repair`: this stays Tier 2 —
+point candidates (a last-resort click by screen position) are still unexpressible and dropped at
+compile time; there's no live chain measurement, so the spec learns from a run only when `repair`
+is invoked, never continuously; there's no loop cursor across records; and a moved control still
+fails the *run that discovered it* before repair can act — recovery in a compiled spec is a
+follow-up PR, never a live save.
+
+**Re-recording one step.** When a diagnostic's `fix` says `sitelooper rerecord`, that step's
+*recording* is the problem — not a locator, a candidate to promote, or the app — so `repair`'s
+locator-level machinery can't help. `sitelooper rerecord <flow-name-or-path> <step-id>
+[--instruction "<text>"] [--var k=v ...] [--runs n] [--reset-cmd "<cmd>"] [--json]` re-records just
+that one step: it backs the flow file up as `<file>.bak-<stamp>.json`, throws away the step's pin,
+params and recorded values (keeping its declared outputs), and replays the flow `--runs` times
+(default 2) in learning mode so the agent records the step afresh and the store's own re-pin rule
+decides whether to keep the new procedure. It only succeeds when the *last* run replays the step at
+tier A on the new pin; each run is a real run against the app, so mint per-run values with `{n}` or
+reset the app with `--reset-cmd`, same as `repair --converge`. Whatever the daemon says about the
+step while a run is on it — above all a re-pin refusal such as `not re-pinning s_04d970 — slot(s)
+v2 identify the record but carry no origin to rebind from` — is printed under the run's line whether
+or not `--progress` is on, and quoted in the `needs-rerecord` diagnostic when nothing was pinned, so
+a refused re-record says *why* rather than just "no procedure".
+
+When the re-recorded step turns out to be a procedure another step of the same flow already pins
+(08-open, re-recorded as a read-only check, is covered by 07-open's status check), the new pin
+shares that skill and inherits the sibling step's bindings for any slot the store recorded no
+origin for. Read-only skills may be shared between steps; a mutating skill is still exclusive to
+one step, and a step whose instruction asks for a change never adopts a read.
+
+fwod34's step 08-open is exactly the demoted-pin case above: its instruction — written by the
+recording orchestrator — asks to cancel an order that step 06 had already cancelled, so the
+skill's first action clicks a Cancel button that is never there on replay. `--instruction` replaces
+the recorded ask with a read-only one, so re-recording produces a check instead of repeating the
+mistake:
+
+```
+$ sitelooper rerecord fwod34.json 08-open \
+    --instruction "Open the sales order {{v1}} and report its current status; do not change it."
+re-recording fwod34 step 08-open (2 run(s))
+  unpinned s_c86522, with a new instruction; old recording kept at fwod34.json.bak-lz3x9k.json
+run 1: 08-open  agent (6 turns) re-pinned s_fcb9a1
+  fwod34: 6/6 step(s) success
+run 2: 08-open  replay tier A (s_fcb9a1)
+  fwod34: 6/6 step(s) success
+08-open: pinned s_fcb9a1 (active, 1 action(s))
+fwod34.json updated — the previous recording is at fwod34.json.bak-lz3x9k.json
+```
+
+**Sizing an instruction.** One `do` is one logical, verifiable step: a goal plus the check that it
+worked. Several UI actions inside one instruction is normal — that is the point. Too big (several
+unrelated goals) stalls on planning; too small (one click) pays an agent loop for what `peek` gives
+free.
+
+**Briefing.** Everything the DOM will not tell an agent about your app goes in a page of markdown
+loaded with `brief <file.md>`: where things are, house conventions ("Apply only previews, Save
+persists"), credentials as `{{env:NAME}}` markers, what not to touch.
+
+### The full command set
+
+```sh
+sitelooper open <url> | brief <file.md> | note "<text>" | peek [--selector css] | screenshot [path]
+sitelooper do "<instruction>" [--json] [--progress] [--max-turns N] [--timeout S] [--no-escalate]
+sitelooper var <name>=<value>                    # declare a run variable (learning session)
+sitelooper skills list | show <id> | rm <id> | repair --drift <run-drift.json>
+sitelooper flow list | show <name>
+sitelooper run <flow> [--var k=v ...] [--json] [--progress]
+sitelooper script [out.spec.ts]                  # emit a plain Playwright spec from the recorded actions
+sitelooper compile <flow-name-or-path> [--out <dir>] [--force] [--json]
+                                                  # compile a converged flow to a standalone spec;
+                                                  # diagnostics (what/why/fix) print first; a step
+                                                  # pinned to a demoted skill refuses to write unless
+                                                  # --force
+sitelooper repair <name.flow.ts> [--var k=v ...] [--out <file>] [--converge <n>]
+                                 [--reset-cmd "<shell command>"] [--check-spec] [--dry-run]
+                                 [--model M] [--json]
+                                                  # replay a compiled flow against the live app and
+                                                  # fold the adaptation back into the owned .flow.ts;
+                                                  # --reset-cmd runs before run 1 and every converge run;
+                                                  # --check-spec then runs the emitted .spec.ts once
+                                                  # under plain Playwright (exit 4 if it fails); a step
+                                                  # whose recording, not the app, is broken gets a
+                                                  # needs-rerecord diagnostic and exit 1 instead
+sitelooper rerecord <flow-name-or-path> <step-id> [--instruction "<text>"] [--var k=v ...]
+                    [--runs n] [--reset-cmd "<cmd>"] [--json]
+                                                  # re-record ONE step whose pin is demoted or
+                                                  # covered by another skill; unpins it, optionally
+                                                  # replaces its instruction, and replays --runs
+                                                  # times (default 2) in learning mode
+sitelooper check <name.flow.ts> [--var k=v ...] [--reset-cmd "<cmd>"] [--json]
+                                                  # run the emitted .spec.ts once under plain
+                                                  # @playwright/test and report the verdict
+sitelooper session list | stop [--all] [--save-flow <name>]
+sitelooper doctor | config | config set <key> <value>
+```
+
+Global flags: `--session <name>` (one daemon and browser per session), `--learn`, `--headed`,
+`--record` (webm per tab), `--script`, `--verbose`, `--progress`, `--json`. Exit codes: `0`
+succeeded, `1` failed or blocked, `2` infrastructure (no key, no browser, LLM unreachable).
+
+## Current matrix
+
+Two questions decide whether the tool earns its place. **First contact**: given a goal it has
+never seen, how does sitelooper compare with the incumbents? **Every run after that**: once the
+flow is known, what does repeating it cost, and does it stay correct? Success is always the
+app-side verifier's count (mutation log, JSON-RPC or HTTP API state), never an arm's self-report.
+All cells are cloud runs on identical hardware, one box per target; full detail in
+[bench/MATRIX-SUMMARY.md](bench/MATRIX-SUMMARY.md).
+
+**Matrix 1 — first contact.** sitelooper: set 26 (2026-09-03, build e048128; glm-5.3
+orchestrator, deepseek-v4-flash inner with glm-5.3 escalation). agent-browser: set 17, same era,
+glm-5.3.
+
+| target | sitelooper | agent-browser |
+|---|---|---|
+| repairdesk (in-repo SPA) | 7/7 · $0.07 · 1212s (set 28; set 26: 7/7 · $0.09 · 819s) | 6/6 · $0.19 · 67s |
+| kanboard (PHP, drag-and-drop) | 6/6 · $0.21 · 1078s (set 28; set 26: 6/6 · $0.04 · 385s) | **2/6 (turn-cap)** · $0.77 · 118s |
+| grafana (React SPA) | 6/6 · $0.14 · 1381s (set 28; set 26: 6/6 · $0.48 · 2037s) | 6/6 · $1.05 · 448s |
+| odoo (dense CRUD) | 6/6 · $0.38 · 1451s (set 28d; set 26: 6/6 · $0.59 · 1651s) | 6/6 · $1.51 · 302s |
+| atelyr (private React app, local) | 2/2 checkable · $0.76 · 2557s (set 28e; set 28: 6 reported, 2/2 checkable · $1.43 · 3043s) | — |
+
+On first contact sitelooper is the slowest arm on every target, by design: it drives a cheap
+inner model and spends the extra time recording verified locators, value provenance and effect
+expectations. What that buys is the lowest cost on every target (2–19× cheaper), a 25/25 objective
+record including the board that turn-capped agent-browser at 2/6, and the recording that makes
+Matrix 2 exist.
+
+**Matrix 2 — every run after the first.** The same four flows repeated: sitelooper replays (set
+24, two replays each) against re-running the agent, against a Playwright script the agent authored
+from its own run, against literal codegen from the recording, and against **Tier 2 spec** — the
+same recording compiled by `sitelooper compile` into a standalone `@playwright/test` spec with no
+sitelooper runtime in the loop at all, then replayed under the real Playwright test runner
+(`bench/spec-replay.mjs`).
+
+| target | sitelooper replay (r1, r2) | agent re-run | authored script | codegen | Tier 2 spec |
+|---|---|---|---|---|---|
+| repairdesk | **7/7, 7/7** · $0.00, $0.00 · 25s, 25s (set 31, m4rd on d28346a; every step at tier A, zero model turns; set 30 on a7f0c6e: 55s, 55s before the late-navigation fix; set 28: 24s, 23s) | 6/6 · $0.19 · 67s every time | 1/6, 1/6 · $0 | 6/6, 6/6 · $0 | **6/6, 6/6** · $0.00 · 15s, 15s (set 31, m4rd; 0 drift; repair converged with 4 candidate promotions, spec check passed in 14s; repaired spec 6/6 in 14s) |
+| kanboard | **4/4 checkable, same** · $0.00, $0.00 · 27s, 27s (set 31, m4kb on d28346a; all five steps at tier A, zero turns; two objectives are report-based and a zero-model replay writes no report; set 30: 45s, 45s; set 28: 23s, 23s) | 2/6 · $0.77 · 118s every time | 5/6, 5/6 · $0 | 4/4 (+2 n/a) · $0 | **4/4 checkable, same** · $0.00 · 15s, 14s (set 31, m4kb; 0 drift; repair converged with no change, spec check passed in 13s; repaired spec 4/4 in 14s) |
+| grafana | **6/6, 6/6** · $0.00, $0.00 · 54s, 54s (set 31, m4gr on d28346a; every step at tier A, zero model turns; set 30: 79s, 79s; set 28: 47s, 47s) | 6/6 · $1.05 · 448s every time | 0/6, 0/6 · $0 | 0/6, 0/6 · $0 | **4/6, 4/6** · $0.00 · 33s, 33s (set 31, m4gr; 0 drift; objectives 1 and 6 unverifiable by design, the spec arm writes no finalText; repair converged with 6 changes: 3 promotions, 1 model-proposed heading locator, 2 never-hit point candidates retired; spec check passed in 33s; repaired spec 4/6 in 34s) |
+| odoo | **6/6, 6/6** · $0.02, $0.01 · 787s, 556s (set 31, m4od2 on d28346a, flow fwod34r3; model-bound, so the click-wait fix barely shows: 06-open fell back to the model on both replays, 24 then 39 turns, because its pinned skill's precondition names the order LIST page while the flow arrives on the order FORM, a store defect in 06-open's recording; r1 also lost 16 turns to a one-off sign-in fallback; set 30: 258s, 568s; set 28d on fwod34: 664s, 243s) | 6/6 · $1.51 · 302s every time | 1/6, 1/6 · $0 | 0/6, 0/6 · $0 | **6/6, 6/6** · $0.00 · 79s, 79s (set 31, m4od2; 0 drift both runs; repair 9/9 ×3 at tier A with no change, spec check passed in 77s; repaired spec 6/6 in 78s; set 30: 77s, 77s) |
+| atelyr | 12/12 flow steps · $0.13, $0.43 · 710s, 1002s (set 28e; 114 then 134 model turns; nine of twelve steps at zero turns on the second replay, the three re-pinned steps among them) | — | — | — | not yet run |
+
+**Set 31 (d28346a), the current build.** The set 30 routine rerun on one fix. Set 30's zero-model
+replays had run at roughly twice set 28's wall clock; bisected locally on the repairdesk recording to
+93ac1f7, where the recorder's "give a click's late navigation a moment" made every click, press and
+select in learning mode wait 1.5s unless the url moved, and replays run in learning mode. d28346a
+ends that wait as soon as the page has no request in flight. Same boxes, same recordings, same
+scores, and the replay wall clocks came back: repairdesk 55s → 25s, kanboard 45s → 27s, grafana
+79s → 54s, all at zero model turns. Odoo did not move (258s, 568s → 787s, 556s) because its
+replays are model-bound: 06-open falls back to the model on every run of fwod34r3 (its recorded
+precondition names the order list page and the flow arrives on the order form), so the wall clock
+is model turns, not click waits; the first set 31 odoo box also ran the wrong flow (fwod34, whose
+demoted 08-open needs the model) because fwod34r3 was only on a results branch, so it was
+published on this branch at 16f9a4b and the target relaunched as m4od2. Every compiled spec passed
+both runs with 0 drift at the same scores as set 30, every repair converged with its spec check
+passing, and every repaired spec passed again. No goal-state guard fired and no diagnostic
+appeared: the published stores predate goals, so no skill carries one yet.
+
+| target | sitelooper replay r1, r2 (set 30 → set 31) | verifier | compiled spec a, b | repair | repaired spec | agent-browser, every run |
+|---|---|---|---|---|---|---|
+| repairdesk | 55s → **25s, 25s** · 0 turns · $0 | 7/7, 7/7 | 6/6, 6/6 · 15s, 15s | converged, 4 promotions, check passed | 6/6 · 14s | 6/6 · $0.19 · 67s |
+| kanboard | 45s → **27s, 27s** · 0 turns · $0 | 4/4 checkable ×2 | 4/4 ×2 · 15s, 14s | no change, check passed | 4/4 · 14s | 2/6 · $0.77 · 118s |
+| grafana | 79s → **54s, 54s** · 0 turns · $0 | 6/6, 6/6 | 4/6 ×2 · 33s, 33s | converged, 6 changes, check passed | 4/6 · 34s | 6/6 · $1.05 · 448s |
+| odoo | 258s, 568s → 787s, 556s · 40, 39 turns · $0.02, $0.01 | 6/6, 6/6 | 6/6, 6/6 · 79s, 79s | no change, check passed | 6/6 · 78s | 6/6 · $1.51 · 302s |
+
+Read across a row: the replay is the daemon re-running the recording with no orchestrator, the
+compiled spec is the same recording under plain Playwright with no sitelooper runtime and no model
+at all, and agent-browser is what it costs to have an agent do the task again from scratch. On the
+three targets whose replays need no model turns, the replay beats agent-browser by 2.7×, 4.4× and
+8.3× on wall clock at zero cost, and the compiled spec by 4.5×, 8.4× and 13.6×. Odoo's replay is
+the one still paying for model turns, and its compiled spec runs the same flow in 79s.
+
+**Tier 2 spec, status.** `bench/spec-replay.mjs` compiles a published flow + skill store
+(`sitelooper compile <flow> --out <tmp>` with `SITELOOPER_SKILLS_DIR` pointing at the store) and
+runs the emitted `<name>.spec.ts` under `npx playwright test`, scored by the same app-side
+verifiers as every other arm (`<tag>-spec-result.json`, `arm: "spec"`). Repairdesk ran locally
+(`fwrd42` store/flowrun, verified 6/6 with the clean-run mutation log); kanboard, grafana, and
+odoo ran on the bench's cloud environment across sets 1-8, with results published to
+`origin/results/sp<N><target>` branches (`sp3kb`, `sp7gr`, `sp8od`, `sp11od`). Kanboard's compiled
+spec passed 4/4 checkable objectives on both runs and repair converged with 0 tickets. Grafana's
+compiled spec ran 1/1 with 0 drift on both runs (verifier 4/6, the other two objectives
+unverifiable by design since the spec arm writes no report) and repair converged in two rounds
+to a spec that still passes and still scores 4/6. Odoo's compiled spec passes end to end on the
+cloud (`sp11od`, f838bec): 1/1 with 0 drift and 6/6 on both runs in 78s each, repair 9/9 at tier
+A on all three runs with nothing to change and its spec check passing, and the repaired spec
+1/1 and 6/6 again. That took eight sets of emitter fixes (`sp8od` verified 6/6 but the test
+halted at 08-open) and then one `sitelooper rerecord` of 08-open, whose recording asked to cancel
+an order 06-open had already cancelled: the step is now pinned to 07-open's validated read-only
+status check, which the store lets two steps share.
+
+Set 24 also caught two engine regressions of its own (kanboard's replays at 22 and 37 turns
+where set 15 needed none; grafana's replays losing objective 1 and recovering one step at 19 and
+44 turns). Every cause was a testable engine rule — a clock-stamped textbox name in an
+expectation, a trailing space in an identity marker, an expectation-only value promoted to a
+required parameter, a heading that renders only on scroll — and all are fixed on build f727c89.
+The clean A/B is to replay the same set-24 flows and stores on the fixed build (set 24b):
+
+| target | set 24 replays (b9ccbca) | set 24b replays (f727c89) |
+|---|---|---|
+| kanboard | 22 and 37 turns · 272s, 555s | **0 and 0 turns · 56s, 56s** · 4/4 app-state objectives both |
+| grafana | 4/6, 5/6 · 19 and 44 turns | **6/6, 6/6** · 29 and 44 turns · 151s, 272s on 08cf104, with the same recording's flow re-exported by the fixed engine (one export rule needed that) and paired with its replay-refined store |
+| odoo (set 26 recording) | 6/6, 6/6 · 91 and 35 turns | **6/6 · 31 turns · 213s** on 6ad5cde with the same pairing; the rest is the app's own url state varying between runs |
+
+The grafana row shows the shape of most of this work: the set-24 grafana
+cell as recorded was 4/6 and 5/6, and each miss was a rule in the engine
+(a read discounted as an echo of a recorded scroll; a flow that referenced
+a typed value as another step's output). Fixing the rules and re-exporting
+the same recording gives 6/6 on both replays. Fresh recordings since then
+(fwgr24, fwgr25, fwgr26) each added a rule of the same kind — an accidental
+"Discard changes?" dialog, a dialog opened and cancelled, transient status
+and alert lines — until fwgr26 compiled clean and instead lost every replay to an
+error page. Five runs were spent finding out why: the sign-in skill carried
+a recorded stray click on a `target=_blank` link to grafana.com, the box has
+no network, the new tab landed on a browser error page, and the daemon
+adopted that tab as the page to work on. The replay now keeps its page
+whatever tabs open, a tab that lands on an error page is closed, and a
+fallback that resolves to a link leaving the recorded origin is never
+taken. Full detail, including the
+runs that did not work, is in [bench/MATRIX-SUMMARY.md](bench/MATRIX-SUMMARY.md).
+
+Reading it: static scripts are free and mostly wrong; re-running the agent is reliable and costs
+the full price forever; sitelooper's repeat cost trends to zero without the correctness trending
+anywhere, and where it does not, the cause has so far always been a specific engine rule rather
+than the app.
+
+## Reference
+
+### Providers
+
+The LLM layer is a generic OpenAI-compatible adapter with presets; any endpoint works by setting
+`baseUrl` and `model` directly.
+
+| Preset | Base URL | Default model | Escalation model | Key env var |
+|---|---|---|---|---|
+| `zhipu` (default) | `https://api.z.ai/api/paas/v4` | `glm-5.2` | — | `GLM_API_KEY` / `ZHIPU_API_KEY` |
+| `novita` | `https://api.novita.ai/openai` | `deepseek/deepseek-v4-flash` | `zai-org/glm-5.3` | `NOVITA_API_KEY` |
+| `openrouter` | `https://openrouter.ai/api/v1` | `z-ai/glm-5.2` | — | `OPENROUTER_API_KEY` |
+| `openai` | `https://api.openai.com/v1` | `gpt-5-mini` | — | `OPENAI_API_KEY` |
+
+Every field resolves **flag > env > config file > preset**: `--provider`, `--model`,
+`--base-url`, `--fallback-model`; `SITELOOPER_PROVIDER`, `SITELOOPER_MODEL`,
+`SITELOOPER_FALLBACK_MODEL`, `SITELOOPER_BASE_URL`, `SITELOOPER_API_KEY`;
+`sitelooper config set <provider|model|fallbackModel|baseUrl|apiKey> <value>` →
+`~/.sitelooper/config.json`. Prefer env for the key. The benchmark stack is
+`SITELOOPER_PROVIDER=openrouter`, model `deepseek/deepseek-v4-flash`, fallback `z-ai/glm-5.3`.
+
+**Escalation on blocked.** An instruction the routine model reports as `blocked` is retried once
+on the escalation model, on the same browser and history, told it is resuming so it re-checks
+state before repeating anything that could double-apply. A verified `failure` is not retried, nor
+is an operator stop. Both attempts are billed into the returned `turns` and `usage`; the report's
+`escalation` object says whether the retry rescued it. `--no-escalate`, or a fallback model of
+`none`, turns it off.
+
+### Configuration
+
+| Env / flag | Default | |
+|---|---|---|
+| `SITELOOPER_CHANNEL` | `chrome` → `msedge` → bundled | browser channel |
+| `SITELOOPER_EXECUTABLE` | — | explicit browser binary |
+| `SITELOOPER_HEADED=1`, `--headed` | headless | visible window (first call of a session) |
+| `SITELOOPER_HOME` | `~/.sitelooper` | sessions, skills, flows, config |
+| `SITELOOPER_SKILLS=1`, `--learn` | off | learning mode; `SITELOOPER_SKILLS_DIR` relocates the store |
+| `SITELOOPER_FLOWS_DIR` | `~/.sitelooper/flows` | flow files |
+| `SITELOOPER_RECORD=1`, `--record` | off | webm per tab; paths printed by `stop` |
+| `SITELOOPER_SCRIPT=1`, `--script` | off | record every action as a replayable Playwright step |
+| `--max-turns` | 30 | agent turn cap per instruction |
+| `--timeout` | 300 | wall-clock seconds per instruction |
+| `--turn-timeout` | 90 | seconds for one LLM call before it is aborted and nudged |
+
+### What the outer agent sees
+
+`do` prints a one-line result, or with `--json` the full
+`{report: {status, summary, details?, evidence?}, turns, usage, model}`. On a turn or time cap
+the result also carries `actions`, the ordered tool calls that ran, so a caller can verify state
+before resuming rather than repeat a mutation. Nothing else lands in the caller's context: the
+agent's snapshots, retries and tool chatter stay inside the daemon.
+
+### What it will not do
+
+- **Canvas-rendered content** (charts, drawn grids, images) has no DOM to read or verify; the
+  agent reports blocked and says so.
+- **Anti-bot evasion, CAPTCHA solving, crawling** are out of scope. sitelooper is for testing
+  and driving apps you operate or are authorised to test.
+- **Vision**: the agent is text-only; it reads the accessibility tree and DOM. Screenshots are
+  for you.
+- **Guessing credentials**: a rejected or missing credential is an immediate blocked report,
+  never a retry loop. `{{env:NAME}}` markers are how you supply them.
+
+### Claude Code skill
+
+`skills/sitelooper/SKILL.md` is the canonical copy of the bundled skill:
+
+```sh
+mkdir -p ~/.claude/skills/sitelooper
+cp skills/sitelooper/SKILL.md ~/.claude/skills/sitelooper/SKILL.md
+```
+
+### Development
+
+```sh
+npm run build                         # tsc -> dist/
+npm test                              # unit tests
+BP_BROWSER_TESTS=1 npx vitest run     # + browser-backed replay and perturbation tests (needs Chrome/Edge)
+```
+
+The recording-path regression gate (`test/rebuild.test.ts`) recompiles real published
+recordings and pins what they compile to; it runs the built engine, so build before testing.
+Benchmark procedure, arms, targets and the cloud runbook live under `bench/`.