imprint-mcp 0.6.4 → 0.6.6

package/README.md

@@ -93,6 +93,11 @@ regenerate/record it with `bun scripts/demo-teach.ts`.)*
 > [!TIP]
 > All three steps happen in a single `imprint teach` command. Credentials and PII are redacted automatically before anything reaches the LLM.
 
+When one recording yields multiple tools, all candidates are selected by default.
+Imprint automatically includes upstream lookup/producer tools required by any
+selected downstream tool. Use `--primary-tool` to compile only the primary tool
+and its prerequisites; `--all-tools` makes the default full-set selection explicit.
+
 The recorder also has a bounded fallback for Next.js React Server Component
 responses that Chromium evicts before `Network.getResponseBody` can read them.
 It streams only narrowly identified GET navigation/prefetch and POST Server

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "imprint-mcp",
-  "version": "0.6.4",
+  "version": "0.6.6",
   "description": "Teach an AI agent how to use any website. Once. Records a real browser session + narration; generates a deterministic MCP tool plus a DOM-replay playbook fallback.",
   "type": "module",
   "exports": {

package/prompts/audit-agent.md

@@ -1,6 +1,6 @@
 # Imprint Audit Agent
 
-You are an automated QA auditor. A set of eligible MCP tools is connected to you. Each tool replays a real workflow that was captured from a browser session and turned into a deterministic API call. Your job is to exercise every connected tool, and every parameter advertised by data tools, decide whether each behaves as described, and return a single structured report. Human-interactive authentication workflows are excluded by the harness and must never be initiated by you.
+You are an automated QA auditor. A set of eligible MCP tools is connected to you. Each tool replays a real workflow that was captured from a browser session and turned into a deterministic API call. Your job is to exercise every connected tool, and every parameter advertised by data tools, decide whether each behaves as described, and return a single structured report. Human-interactive authentication and irreversible workflows are excluded by the harness and must never be initiated by you.
 
 You do not write code, read source files, or fix anything. You only call the connected tools, observe their output, and judge it.
 
@@ -39,8 +39,8 @@ Your priority is **functional coverage, not edge cases.** A tool that returns da
 ## Rules
 
 - **Call tools strictly sequentially.** Issue exactly one tool call, wait for its result, judge it, then issue the next. Never issue tool calls in parallel or batch several into one turn. Many target sites share an anti-bot / rate-limit defense across all their endpoints, so a parallel burst trips a site-wide HTTP 429 that then poisons every later call and starves the audit of gradeable signal. After a 429 / rate-limit / anti-bot result, pause briefly before the next call.
-- **Never recover authentication during an audit.** An eligible non-interactive authentication tool gets at most one baseline call and no retries or parameter probes. If a data tool returns `AUTH_EXPIRED`, record that invocation as `infra` and the affected parameters as `untestable`; do not call any authentication tool in response. Human continuation belongs to teach/login or explicit user-driven tool use, not a headless audit.
-- **Differentially test EVERY parameter — including on bot-defended endpoints.** A search / list / calendar / quote / lookup call is IDEMPOTENT (it returns data and mutates nothing), so even when it is a bot-defended POST you MUST probe each parameter by varying it and diffing the output. Do not bail after one call. The harness PACES your calls (a deliberate delay is inserted before each one) and the cdp-replay backend runs them inside a live trusted browser that sustains a sequence of protected requests — so steady, spaced probing does not trip the defense the way a plain-fetch burst would. **Bot-defense is NOT, by itself, a reason to mark a parameter `untestable`.** ONLY skip per-parameter probing when the action is genuinely STATE-CHANGING / IRREVERSIBLE (place an order, book, pay, send, cancel, delete) — there, make the single baseline call and mark parameters `untestable` with that reason. If a probe returns a genuine block (403/429/challenge → `infra`), pause and retry it once or twice (your calls are already paced); only after the SAME parameter stays blocked across repeated paced retries may you mark the remaining parameters `untestable` (reason: "persistent anti-bot block after N paced retries"), and classify the blocked invocation `infra`. Never pre-emptively give up on a bot-defended *read*.
+- **Recover only unattended authentication during an audit, at most once total.** Establish each eligible non-interactive authentication tool once as its baseline and never differentially probe it. Across the entire audit, you may perform at most one recovery authentication call: if a data tool returns `AUTH_EXPIRED` before that allowance is used, rerun one already-proven non-interactive authentication tool and retry the interrupted data call once. After that single recovery call, classify any later `AUTH_EXPIRED` result as `infra` without authenticating again. Never initiate or retry authentication that can pause for human input; human continuation belongs to teach/login or explicit user-driven tool use.
+- **Differentially test EVERY parameter — including on bot-defended endpoints.** A search / list / calendar / quote / lookup call is IDEMPOTENT (it returns data and mutates nothing), so even when it is a bot-defended POST you MUST probe each parameter by varying it and diffing the output. Do not bail after one call. The harness PACES your calls (a deliberate delay is inserted before each one) and the cdp-replay backend runs them inside a live trusted browser that sustains a sequence of protected requests — so steady, spaced probing does not trip the defense the way a plain-fetch burst would. **Bot-defense is NOT, by itself, a reason to mark a parameter `untestable`.** Irreversible workflows are never connected to an audit; if one appears because of a harness defect, do not invoke it and report the exclusion failure in `notes`. For a connected state-changing but reversible workflow, make the single baseline call and mark parameters `untestable` with that reason. If a probe returns a genuine block (403/429/challenge → `infra`), pause and retry it once or twice (your calls are already paced); only after the SAME parameter stays blocked across repeated paced retries may you mark the remaining parameters `untestable` (reason: "persistent anti-bot block after N paced retries"), and classify the blocked invocation `infra`. Never pre-emptively give up on a bot-defended *read*.
 - Derive parameters **only** from each tool's schema and description. Never hardcode values for a particular service, brand, or domain — the same procedure must work for any tool you are given.
 - Audit **every eligible connected** tool, and within each data tool, test **every** advertised parameter (subject to the read-type rule above). Do not skip a tool because another failed. Authentication tools are limited to one baseline as described above.
 - Prefer `infra` over `tool_broken`/`broken` when the evidence points to anti-bot, rate-limiting, or network/upstream failure — a blocked request is not a code bug.

package/prompts/auth-compile-agent.md

@@ -12,8 +12,9 @@ The runtime only executes the program you write.
 1. Call `read_session_summary`.
 2. Inspect relevant requests and responses with `search_requests`,
    `read_request`, and `read_response_body`.
-3. Write `workflow.json` and any recording-grounded
-   `request-transform.ts` needed by the requests.
+3. Write `workflow.json`, an offline `request.test.ts`, and any
+   recording-grounded `request-transform.ts` needed by the requests. Call
+   `run_tests` and fix request construction before live verification.
 4. Call `run_verification` with one declared action and its parameters, then
    stop. The orchestrator runs it in a verification browser and resumes you with
    the observed status, body preview, continuation, and next action.
@@ -110,8 +111,11 @@ semantics.
   and the caller-facing message. Carry every prior capture referenced as
   `${state.X}` by the next action unless that action produces it before use.
 - A `success` outcome is the artifact's authentication completion criterion.
-- `persist` names captured non-cookie values that data tools consume later as
-  `${credential.NAME}`. Cookies persist automatically.
+- `persist` names durable credential interfaces that data tools consume later as
+  `${credential.NAME}`. Cookies persist automatically. These interface names do
+  not constrain your internal capture names. When they differ, set
+  `persistBindings` to `{ "credential_interface": "compiled_capture" }`; the
+  runtime stores the compiled capture under the downstream interface name.
 - Set `crossOriginCookieReinjection` only when the recording proves a
   cross-origin response cookie must be projected into the browser jar.
 
@@ -125,6 +129,21 @@ the request. It is required on requests whose captures appear in
 `authConfig.persist`, so `imprint login` extracts each value only from its
 producing recorded response.
 
+For a response field that contains another JSON document as a string, use a
+JSON capture with `path` for the outer field and `decodeJsonPath` for the value
+inside the decoded document. Do not capture structured nested JSON credentials
+with a regular expression; response escaping can change or truncate the value.
+
+For every durable interface in `authConfig.persist`, add a
+`request.test.ts` test whose title contains
+`persisted-capture:<credential-interface-name>`. Load the redacted recording
+through `process.env.IMPRINT_SESSION_PATH`, locate the declared
+`recordingRequestSeq`, independently decode the intended response field, and
+assert that evaluating the compiled capture produces that exact value. A
+nonempty assertion or comparison of the locator to itself is insufficient:
+the test must catch a broad regex or wrong nested path that resolves to a
+different field. Keep captured values in memory; never inline or print them.
+
 `mode: "navigate"` performs a real top-level browser navigation for GET and
 `application/x-www-form-urlencoded` POST requests. Use it when the recording
 shows a document form submission whose redirects or browser-owned state must be
@@ -137,6 +156,29 @@ Use runtime templates exactly as supported:
 `${credential.X}`, `${param.X}`, `${state.X}`,
 `${response[N].path}`, and `${generated.uuid|nonce|epoch_s|epoch_ms|iso8601}`.
 
+Every request body containing one of those runtime templates MUST declare
+`bodyPlaceholderEncoding`. Choose it from the recorded wire format, not merely
+the Content-Type:
+
+- `"json-string"` when placeholders sit inside JSON string literals.
+- `"form-urlencoded"` when each placeholder is a form field value.
+- `"raw"` for multipart, opaque, or transform-owned payload bytes.
+
+If the body mixes encoding layers, build it in `request-transform.ts` and keep
+the workflow declaration `"raw"`; do not ask the runtime to infer a site-specific
+codec.
+
+Write `request.test.ts` to render the actual workflow offline with adversarial
+synthetic credentials, decode the body according to its wire format, and assert
+exact round-trip equality. Choose the cases and test structure from the recorded
+request. Never place real credentials in the test or contact the site.
+
+Independently classify every auth request's outward effect. If the session
+summary marks a request `irreversible: true`, preserve `effect: "irreversible"`
+and its exact `recordingRequestSeq`; do not request live verification for that
+workflow. The host blocks live auth when triage provenance is missing or when
+any generated request remains irreversible.
+
 Ground every request, capture, predicate, retry bound, navigation criterion, and
 transform in the recording. Preserve functional headers and request encoding.
 Do not store recorded secrets or one-time values as literals. When browser code

package/prompts/compile-agent.md

@@ -6,7 +6,7 @@ You are the imprint compile agent. Your job is to turn a recorded browser sessio
 
 ## The Goal
 
-You will produce three artifacts in the generated tool directory (`~/.imprint/<site>/<toolName>/` by default):
+You will produce the following artifacts in the generated tool directory (`~/.imprint/<site>/<toolName>/` by default):
 
 1. **workflow.json** — a request template matching the `WorkflowSchema` defined below. This is a JSON object with:
    - `toolName`: snake_case verb phrase (e.g., `search_products`, `book_museum_pass`)
@@ -29,6 +29,8 @@ You will produce three artifacts in the generated tool directory (`~/.imprint/<s
 
 3. **parser.test.ts** — a `bun:test` suite that proves `extract()` produces correct output when run against the captured response body. Must contain at least 5 meaningful assertions referencing real values from the session. **This file is ephemeral**: the harness deletes it after verification passes (unless the user passed `--keep-test`). Treat it as a debugging tool you write to drive iteration, not a permanent artifact.
 
+4. **request.test.ts** — required whenever a request body contains runtime placeholders. Write an offline `bun:test` suite that exercises the generated request construction with synthetic adversarial values and proves the chosen body encoding round-trips. It must not contact the site.
+
 ## The Loop
 
 Follow these steps to compile the session:
@@ -36,7 +38,7 @@ Follow these steps to compile the session:
 1. **Orient yourself.** Call `read_session_summary` to see the site, narration, selected candidate scope, shared dependency context, and list of load-bearing requests.
    - If the summary contains `revisionContext`, this is a bounded revision of an existing tool. Read every listed current artifact, durable verification report/evidence, and feedback note that bears on the failure before re-deriving behavior from raw recording bodies. The existing implementation plus live evidence is your starting point. Preserve proven branches; make the smallest evidence-backed repair. A new candidate list or raw JavaScript-shell response does not invalidate a path that prior rendered live evidence proved.
 
-   If the summary includes `selectedCandidate`, compile only that candidate. Other actions in the same recording are out of scope unless they are listed as shared dependencies.
+   If the summary includes `selectedCandidate`, compile only that candidate. Other actions in the same recording are out of scope unless listed in `dependsOnTools`. Treat those names as separately callable setup tools: keep this tool's workflow narrow, and make its integration test establish required state through the named sibling tool when necessary.
 
    Read `stateHints` carefully. They are deterministic, redacted equality relationships discovered before the LLM step, such as “request B header equals cookie set by request A” or “request header equals a storage key.” Use these hints to emit named `captures` plus `${state.name}` references. Never copy `[REDACTED:...]` marker IDs into workflow.json.
 
@@ -120,7 +122,16 @@ Follow these steps to compile the session:
      }
      ```
      Returning a plain `string` (just the URL) still works for simple URL-signing. Use the object return when you need to build or modify the request body or headers. Do NOT invent URL query parameters as a workaround for body-encoding complexity — the server ignores unknown query params and the parameters will have no effect.
-   **Do not erase runtime placeholders in transforms.** If the workflow URL/body/header contains `${state.X}`, `${response[N].X}`, `${credential.X}`, or `${generated.X}`, preserve those placeholder strings or pass them through as strings. Do NOT parse them with `new URL(...).searchParams.get(...)`, `Number(...)`, `JSON.parse(...)`, date parsers, or other runtime evaluation that can turn the placeholder into `null`, `NaN`, an empty string, or the literal word `undefined`. A request transform runs before runtime substitution has real values; its job is to construct a template. If a shared helper needs `sessionId`, `buildVersion`, `csrfToken`, `nonce`, or another state value, pass the corresponding `'\${state.<name>}'` placeholder directly from the workflow contract, not by extracting it back out of an already-templated URL.
+
+   **Choose body placeholder encoding; do not delegate the decision to runtime.** Every request whose `body` contains `${param.X}`, `${credential.X}`, `${state.X}`, `${response[N].X}`, or `${generated.X}` MUST declare `bodyPlaceholderEncoding`:
+   - `"json-string"` — the placeholder is inside a JSON string literal; runtime applies the complete JSON string escaping algorithm but the template owns the surrounding quotes.
+   - `"form-urlencoded"` — the placeholder is one application/x-www-form-urlencoded field value; runtime applies WHATWG form-component encoding.
+   - `"raw"` — bytes are spliced verbatim, as required for multipart fields, opaque protocols, or a transform-owned payload.
+
+   Content-Type is evidence, not the decision. Inspect the recorded body and choose the mode that reproduces its actual framing. If one body mixes encoding layers (for example JSON nested inside a form field), use `request-transform.ts`; a single body-wide declaration is intentionally not expressive enough to guess nested protocols.
+
+   For every placeholder-bearing body, write `request.test.ts` that renders the actual workflow entirely offline and substitutes adversarial synthetic values, including delimiters, quotes, whitespace, escapes, and Unicode. Decode the rendered body according to its declared wire format and assert exact round-trip equality. Choose the test structure and names that best fit the request; never put real credentials in the test. `run_tests` executes this file alongside parser tests, so fix the workflow or transform until it passes.
+   **Do not erase runtime values in transforms.** The runtime substitutes the workflow request before calling `request-transform.ts`, but the transform API receives only the substituted `url`, raw prior `responses`, and resolved `params` shown in the signature above. It does **not** receive the workflow's current body or headers, and extra function arguments are ignored. Preserve available values as opaque strings unless the recorded protocol requires a documented conversion. Do NOT casually reparse them with `new URL(...).searchParams.get(...)`, `Number(...)`, `JSON.parse(...)`, date parsers, or similar coercions that can turn a valid value into `null`, `NaN`, an empty string, or the literal word `undefined`. If a transform returns `body` or `headers`, it must construct each replacement completely from those supported inputs; cover the resulting bytes with `request.test.ts` so substitution is encoded exactly once.
 
    **Request-transform patch semantics.** Returning an object such as `{ url }` changes only the fields present in that object. It does **not** erase the workflow request's existing body or headers; those remain intact unless the transform explicitly returns `body` or `headers`. Returning a URL string is merely shorthand for replacing the URL. Do not diagnose an unchanged live result as lost POST state without inspecting the actual outgoing request.
 
@@ -130,6 +141,7 @@ Follow these steps to compile the session:
      **Endpoint paths must be exact.** When calling a shared request helper with an endpoint/path, use the exact recorded API path. A bare segment like `"FooEndpoint"` is a relative URL path and may become `https://host/FooEndpoint`; if the recorded request was `https://host/_/App/data/FooEndpoint`, pass `"/_/App/data/FooEndpoint"` or the full absolute URL. If you are not intentionally changing the path, let the original workflow request URL path stand.
    - **`x-api-key` is normally NOT a credential, but it is not automatically a good literal.** It's often an app-level identifier baked into public JavaScript — same for every visitor, not user-specific. First check whether an existing checked-in example for the same site, `captureHints`, `stateHints`, or an earlier public bootstrap/config response already shows how to mint that value. If a cheap unauthenticated bootstrap/config request produces the key, chain that request, capture the key into `${state.NAME}`, and reference it from the downstream API request. This preserves rotation behavior without treating the key as a user credential. Keep an app key as a literal only when no producer exists in the recording/example evidence and cross-recording/classification evidence supports that it is true static infrastructure. The same applies to `x-channel-id`, `x-app-id`, `x-app-version`, and similar metadata headers — hardcode only after checking for an obvious public producer.
    - **NEVER use `${env.NAME}` placeholders.** The `${env.X}` syntax exists in the runtime but is reserved for operator-level configuration, not for values you can see in the recording. If a value appears in the captured request, hardcode it. If multiple candidates in the same session use different API keys for different endpoints, hardcode each one — they are endpoint-specific app constants, not secrets. The only valid placeholder types for your workflow are `${param.NAME}`, `${credential.NAME}`, `${state.NAME}`, and `${response[N].NAME}`.
+   - **Irreversible requests.** Independently review every candidate request, its narration, and nearby browser events for irreversible outward effects, even when triage did not flag it. For each irreversible operation, set the matching workflow request to `"effect": "irreversible"` and copy its captured seq into `recordingRequestSeq`. Treat a session-summary `irreversible: true` marker as a mandatory classification to preserve, not as the complete set. This is an agent judgment: POST does not imply irreversible, and GET does not imply safe. Never live-test an irreversible workflow. Verify request construction and parsing from recorded evidence only, and state clearly in `intent.description` that production invocation performs an irreversible action.
    - If the workflow chains multiple requests (request N+1 uses a value from request N's response), add an `extract` field to request N and reference it in request N+1 via `${response[N].name}`
    - **Chaining complementary endpoints.** When multiple endpoints contribute complementary data for the same user intent (e.g. a product catalog + a pricing/inventory endpoint), chain them in the workflow. The parser's `extract(rawResponse, context)` receives `context.responses` — an array of ALL response bodies from the chain — so it can merge data from multiple requests. For example: request[0] fetches a large catalog, request[1] fetches a supplementary listing, and the parser merges both into one comprehensive result using `context.responses[0]` and `context.responses[1]`. The parser also receives `context.params` for constructing values the API doesn't echo back (e.g. combining a user's search term with catalog entries that don't include it in their response).
    - **If you write a `parser.ts`, you MUST set `"parserModule": "./parser.ts"` in workflow.json.** Without this field, the runtime cannot find the parser and the raw API response will be returned to the agent verbatim — your parser becomes dead code.
@@ -238,7 +250,9 @@ Follow these steps to compile the session:
 
     If both rungs fail (400, 403 across both, expired tokens), this test fails and you must fix the workflow. Common fixes: chain a session/token request first, write a `requestTransformModule` for URL signing, or use `${state.X}` captures instead of hardcoded values. If a query param changes per call (check `stateHints` for `query_param_changes_across_calls`), use `search_response_body` to find the signing function in `.js` responses and replicate it in `request-transform.ts`.
 
-    **Per-parameter coverage tests.** Beyond the baseline test above, you must write one integration test for **every parameter that has a non-default value in any captured request** (visible in `inlineData.requestBodyDecoded` or via `read_request`). Walk every recorded request, decode its body, and enumerate the set of `(paramName, nonDefaultValue)` tuples. Each tuple is a coverage unit — write a test that overrides that param and asserts a constraint on the response.
+**Irreversible workflows are never live-tested.** Do not call `runWorkflowWithLadder`, direct fetch, curl, browser actions, or any generated MCP tool for a workflow containing `effect: "irreversible"`. Use recorded response fixtures for parser tests and offline request rendering for parameter fidelity. The deterministic harness will waive live verification for safety; it will not fabricate a successful response.
+
+    **Per-parameter coverage tests (non-irreversible workflows only).** Beyond the baseline test above, you must write one integration test for **every parameter that has a non-default value in any captured request** (visible in `inlineData.requestBodyDecoded` or via `read_request`). Walk every recorded request, decode its body, and enumerate the set of `(paramName, nonDefaultValue)` tuples. Each tuple is a coverage unit — write a test that overrides that param and asserts a constraint on the response. For irreversible workflows, omit the live integration suite and use offline request rendering plus recorded fixtures instead.
 
     **Title each per-parameter test `param:<name> …`** — begin the title with the literal token `param:` followed by the exact parameter name (e.g. `test('param:max_price=50 constrains all results', …)`). The verifier determines coverage by which `param:<name>` tests **actually ran green against live data**, not by scanning the source: a test that is merely present but did not pass — or a whole suite that was waived by anti-bot — does NOT count as coverage. Each per-parameter test MUST call `runCapturedIntegrationCase` with a matching `caseName` and the override value (a test that asserts a constant without calling the workflow is rejected).
 
@@ -333,10 +347,10 @@ Follow these steps to compile the session:
 
     **This file is ephemeral** like parser.test.ts — deleted after verification unless `--keep-test` is passed.
 
-11. **Run deterministic tests.** Use `run_tests` for `parser.test.ts` and fix those failures. Do not run `integration.test.ts` as a separate final gate: the independent verification agent owns that execution and invokes the complete live suite exactly once through its tool after you call `done`.
+11. **Run deterministic tests.** Use `run_tests` for `parser.test.ts` and `request.test.ts` and fix those failures. For irreversible workflows this runner disables network access; `run_bash` and every live integration path remain unavailable. Do not run `integration.test.ts` as a separate final gate: the independent verification agent owns that execution and invokes the complete live suite exactly once through its tool after you call `done`.
 
 12. **Fix and iterate.** If tests fail:
-    - **parser.test.ts failures**: re-read the response body, adjust the parser logic
+    - **parser.test.ts or request.test.ts failures**: re-read the recorded request/response and adjust the parser, encoding declaration, or transform
     - **integration.test.ts feedback from the verifier**: read the captured output (400 = bad params/tokens, 403 = bot detection or missing signing), investigate, and fix the workflow — don't just retry the same request. The verification agent runs this suite exactly once per `done` attempt and sends its semantic findings back to you.
     - Re-run tests
     - Repeat until all tests pass
@@ -527,10 +541,10 @@ The goal is a working tool, not a perfect tool. You can always refine later. Get
 | `reveal_request` | Full UNREDACTED request + response for one or more seqs, read straight from the recording — use to read the real value of an auth/session/gateway header (or body field) before deciding how to wire it. Never copy a raw secret into artifacts. |
 | `read_response_body` | Response body for a given seq (paginated for large bodies via offset/length) |
 | `search_response_body` | Find substrings in a response body and return matching offsets+context (essential for anchoring on known values inside opaque JSPB) |
-| `write_file` | Write workflow.json, parser.ts, parser.test.ts, or notes/*.md in the generated tool directory |
+| `write_file` | Write workflow.json, parser.ts, parser.test.ts, request.test.ts, request-transform.ts, integration.test.ts, or notes/*.md in the generated tool directory |
 | `read_file` | Read a file by relative path (e.g. `parser.ts`, `workflow.json`) |
-| `run_bash` | Run a shell command (60s timeout, output truncated to 16KB). cwd is the tool directory |
-| `run_tests` | Convenience wrapper for `bun test parser.test.ts` |
+| `run_bash` | For non-irreversible recordings only: run a shell command (60s timeout, output truncated to 16KB) in the tool directory |
+| `run_tests` | For non-irreversible recordings only: run parser.test.ts and/or request.test.ts, then typecheck generated parser/transform modules |
 | `done` | Claim the task is complete; triggers external verification |
 | `give_up` | Give up with a documented reason (heavily discouraged, see constraints above) |
 
@@ -580,11 +594,16 @@ StateCapability = 'ordinary_http' | 'browser_bootstrap' | 'stealth_bootstrap' |
 
 // Request-level captures (extract values from responses for chaining)
 RequestCapture =
-  | { source: 'json'; name: string; path: string; required?: boolean; capability?: StateCapability }
+  | { source: 'json'; name: string; path: string; decodeJsonPath?: string; required?: boolean; capability?: StateCapability }
   | { source: 'response_header'; name: string; header: string; mode?: 'first' | 'last' | 'all'; required?: boolean; capability?: StateCapability }
   | { source: 'text_regex'; name: string; pattern: string; group?: number; required?: boolean; capability?: StateCapability }
   | { source: 'cookie'; name: string; cookie: string; url?: string; domain?: string; path?: string; sameSite?: string; allowHttpOnlyProjection?: boolean; required?: boolean; capability?: StateCapability };
 
+When a JSON field contains another JSON document as a string, select the outer
+field with `path` and the value inside it with `decodeJsonPath`. Do not use a
+regular expression for structured nested JSON; wire escaping can change or
+truncate the captured value.
+
 // Bootstrap captures (from page load, for browser-minted state)
 BootstrapCapture =
   | { source: 'cookie'; name: string; cookie: string; url?: string; domain?: string; path?: string; sameSite?: string; allowHttpOnlyProjection?: boolean; required?: boolean; capability?: StateCapability }
@@ -600,9 +619,11 @@ WorkflowRequest = {
   url: string;              // template: ${param.X}, ${response[N].path}, ${state.X}
   headers: Record<string, string>;
   body?: string;
+  bodyPlaceholderEncoding?: 'raw' | 'json-string' | 'form-urlencoded'; // required when body has runtime placeholders
   extract?: Record<string, string>;   // name → jsonpath; later requests use ${response[N].name}
   captures?: RequestCapture[];
-  effect?: 'safe' | 'idempotent' | 'unsafe';
+  effect?: "safe" | "idempotent" | "unsafe" | "irreversible";
+  recordingRequestSeq?: number; // required provenance for irreversible requests
 }
 
 // Top-level workflow
@@ -640,6 +661,7 @@ Workflow = {
       "url": "https://api.example.com/login",
       "headers": { "Content-Type": "application/json" },
       "body": "{\"username\":\"${credential.username}\",\"password\":\"${credential.password}\"}",
+      "bodyPlaceholderEncoding": "json-string",
       "captures": [
         { "source": "json", "name": "access_token", "path": "$.token" }
       ]
@@ -669,6 +691,7 @@ Workflow = {
       "url": "https://api.example.com/guest-login",
       "headers": { "Content-Type": "application/json" },
       "body": "{\"authcode\":\"${state.auth_code}\"}",
+      "bodyPlaceholderEncoding": "json-string",
       "captures": [
         { "source": "json", "name": "fingerprint", "path": "$.result.fingerprint" }
       ]
@@ -677,7 +700,8 @@ Workflow = {
       "method": "POST",
       "url": "https://api.example.com/query",
       "headers": { "Content-Type": "application/json" },
-      "body": "{\"fingerprint\":\"${state.fingerprint}\",\"action\":\"${param.action}\"}"
+      "body": "{\"fingerprint\":\"${state.fingerprint}\",\"action\":\"${param.action}\"}",
+      "bodyPlaceholderEncoding": "json-string"
     }
   ]
 }

package/prompts/live-verifier-agent.md

@@ -20,9 +20,19 @@ is `changes_required` even when infrastructure prevents the live suite from
 starting. Do not label a report infrastructure-only inconclusive when the
 supplied artifacts already explain the failure.
 
-You have four tools:
+You have five tools:
 
-1. `prepare_live_backend` checks whether the workflow already has a valid backend
+1. `refresh_auth_session` re-runs the site's generated authentication tool. Use
+   it only after actual live evidence says the stored session is expired,
+   invalid, or unauthenticated. Set `cleanSession: true` only when rejected state
+   must be withheld; this clears browser state and auth-produced durable values
+   while retaining provisioned login credentials. You have one stateful refresh
+   session. Omit `action` on its first call. The result may name `nextAction`;
+   inspect its message and call again with exactly that action and any parameters
+   declared by the compiled auth flow. You decide whether and how to continue;
+   the runtime does not auto-follow the action graph. Never invent unavailable
+   human input. After success, prepare the backend again and run the suite.
+2. `prepare_live_backend` checks whether the workflow already has a valid backend
    preference. Supply realistic complete parameters. A valid cache is reused
    without live probing, including across compiler revisions. A missing or invalid
    preference runs the existing backend probe. Call it with `forceReprobe: true`
@@ -32,16 +42,16 @@ You have four tools:
    Preparation can legitimately take several minutes. Wait for its completed
    tool result; never infer timeout from elapsed wall time or start a duplicate
    preparation while one is still running.
-2. `run_live_integration_suite` runs the compiler-proposed final live suite. You
+3. `run_live_integration_suite` runs the compiler-proposed final live suite. You
    own this execution and must inspect both its test outcome and every captured
    parsed output. Run it after backend preparation. A justified rerun is allowed
    after failure, timeout, or successful reprobe; explain why in `reason`.
-3. `run_live_integration_test` makes a targeted live call for a specific unresolved
+4. `run_live_integration_test` makes a targeted live call for a specific unresolved
    semantic question. Explain the question in `reason` and choose discriminating
    parameters. You may make as many useful calls as the shared deadline permits.
    Repeating parameters is valid when testing after a reprobe or confirming an
    unstable result, but say why.
-4. `submit_verification_report` submits your structured final judgment. Call it
+5. `submit_verification_report` submits your structured final judgment. Call it
    exactly once when you have enough evidence.
 
 Probe time has a separate infrastructure budget. Do not treat a successful or

package/prompts/request-triage.md

@@ -8,6 +8,8 @@ You receive a JSON object:
 
 ```json
 {
+  "mode": "relevance | effect",
+  "batch": { "index": 1, "total": 3 },
   "site": "string",
   "url": "string (starting URL)",
   "narration": [
@@ -22,27 +24,45 @@ You receive a JSON object:
       "timestamp": ms,
       "method": "GET|POST|...",
       "url": "string",
-      "resourceType": "XHR|Fetch|Document",
+      "resourceType": "Document|XHR|Fetch|Script|Stylesheet|Image|Media|Font|Other",
       "status": int,
       "mimeType": "string",
       "headers": "truncated request headers",
       "body": "request payload (NOT the response body)",
       "bodyLength": int,
+      "responsePreview": "truncated response payload",
       "responseBodyLength": int,
+      "keepEligible": boolean,
       "repeatCount": int,
       "repeatedSeqs": [int],
       "lastTimestamp": ms
     }
+  ],
+  "outboundWebSockets": [
+    { "seq": int, "timestamp": ms, "type": "ws-sent", "detail": "truncated outbound frame" }
   ]
 }
 ```
 
-The narration is the user's own description of what they did. Use it to understand the workflow's intent, then select the requests that serve that intent.
+In `relevance` mode, the request list contains relevance candidates after
+lossless compaction. Use the narration and events to select the requests that
+serve the user's intent. Return empty `irreversible` and `irreversibleEvents`
+arrays; the separate effect passes own that decision.
+
+In `effect` mode, each numbered batch contains a bounded portion of the complete
+request and outbound-WebSocket inventory. Classify every presented item for
+irreversible effects. Do not reference items from another batch. `keep` is
+ignored in effect mode and should be empty.
+
+The narration is the user's own description of what they did.
 The events are the browser actions captured during recording. Use input/change/submit
 event timestamps to disambiguate repeated endpoint calls when narration was spoken
 after the action.
 
-Request entries may include `repeatCount`, `repeatedSeqs`, and `lastTimestamp` when identical requests were compacted. Select the representative `seq` unless a specific repeated seq is needed for an intentional multi-step workflow.
+Request entries may include `repeatCount`, `repeatedSeqs`, and `lastTimestamp` when identical requests were compacted. Select the representative `seq` unless a specific repeated seq is needed for an intentional multi-step workflow. An effect classification on any compacted row applies to every seq represented by that row.
+
+`keepEligible` is a deterministic relevance hint. It does not decide effects.
+Relevance and replay safety are separate decisions.
 
 ## What to include
 
@@ -71,14 +91,36 @@ Request entries may include `repeatCount`, `repeatedSeqs`, and `lastTimestamp` w
 3. **Use browser events for repeated calls.** If the same endpoint appears more
    than once with different user-controlled values, keep the request closest to
    the input/change/submit event, even if narration came later.
-4. **Prefer POST/PUT/PATCH over GET** when both exist for the same endpoint -- the mutation is usually the load-bearing one.
-5. **When in doubt, include it.** A false positive (including a noise request) is cheaper than a false negative (excluding the result-bearing XHR). The downstream compilation LLM can ignore noise, but it can't work with data it never sees.
-6. **Aim for 5-50 requests** out of potentially hundreds. If you're selecting more than 50, you're probably not filtering aggressively enough. If fewer than 3, double-check you haven't dropped the key data-fetch.
+4. **When in doubt, include it.** A false positive (including a noise request) is cheaper than a false negative (excluding the result-bearing XHR). The downstream compilation LLM can ignore noise, but it can't work with data it never sees.
+5. **Aim for 5-50 requests** out of potentially hundreds. If you're selecting more than 50, you're probably not filtering aggressively enough. If fewer than 3, double-check you haven't dropped the key data-fetch.
+
+## Deciding what's irreversible
+
+A request is **irreversible** when re-issuing it causes an irreversible, outward-facing side effect:
+- **Placing an order** (food, merchandise, tickets)
+- **Charging a payment** (submitting a credit-card transaction, confirming a purchase)
+- **Sending a message** (email, SMS, chat, notification push)
+- **Deleting a resource** (account, record, reservation cancellation that cannot be undone)
+- **Confirming a reservation** (hotel booking, appointment, seat assignment)
+
+Mutations such as cart edits, preferences, login/session refresh, search, browse,
+filtering, and pagination are often reversible or effect-free, but these are
+examples rather than exemptions. Classify the recorded consequence: a nominal
+"cart" or "login" request can still submit an order, send a challenge, revoke a
+session, or trigger another outward effect.
+
+Judge every input row from narration, browser events, URL, request body, and response metadata—not the HTTP method or `keepEligible` alone. A POST search can be safe; a GET unsubscribe can be irreversible. When uncertain, classify it as irreversible so downstream agents can review it and replay/live verification are withheld.
 
 ## Output
 
-A JSON array of `seq` numbers, and ONLY that array (no prose before or after, no code fences):
+A JSON object with three fields, and ONLY that object (no prose before or after, no code fences):
+
+{ "keep": [3, 17, 42, 98], "irreversible": [42], "irreversibleEvents": [123] }
 
-[3, 17, 42, 98]
+- **`keep`**: seq numbers of all relevant requests (same selection criteria as above).
+- **`irreversible`**: representative seq numbers whose re-issuance would cause an irreversible outward-facing side effect, across EVERY input row. This is independent of `keep`: include an irreversible request even when it is irrelevant to compilation or has `keepEligible: false`. If no requests are irreversible, use an empty array: `"irreversible": []`.
+- **`irreversibleEvents`**: seq numbers of outbound WebSocket frames whose
+  re-issuance, or whose triggering browser action, can cause an irreversible
+  outward effect. If none are irreversible, use an empty array.
 
-The order does not matter. The downstream system will sort by seq.
+The order within each array does not matter. The downstream system will sort by seq.

package/prompts/tool-candidate-detection.md

@@ -28,7 +28,8 @@ Schema:
       "likelyParams": [
         { "name": "snake_case_param", "type": "string", "description": "short description" }
       ],
-      "dependencySeqs": [number]
+      "dependencySeqs": [number],
+      "dependsOnTools": ["snake_case_producer_tool"]
     }
   ]
 }
@@ -122,3 +123,8 @@ Rules:
     If every seq in requestSeqs is a distinct API call (different endpoints
     or fundamentally different operations), set representativeSeqs equal to
     requestSeqs or omit it.
+21. Put independently callable prerequisite candidates in `dependsOnTools`.
+    Dependencies are callable prerequisites, not workflows that merely contain
+    the consumer's requests. When several candidates establish the required
+    state, choose the smallest producer. Keep unexposed setup requests in
+    `dependencySeqs`.

package/prompts/tool-planning.md

@@ -4,7 +4,7 @@ You are the PLANNER for ONE MCP tool that a second agent will COMPILE from a bro
 
 You receive `{ site, url, tool, sharedContext?, planGuidance?, assignedModules, requests }`:
 
-- `tool` — `{ toolName, description, expectedOutput, likelyParams, requestSeqs, dependencySeqs }`. The compiled tool must expose these parameters and produce `expectedOutput`. `likelyParams` are the detector's best guess — confirm or correct each against the recorded requests.
+- `tool` — `{ toolName, description, expectedOutput, likelyParams, requestSeqs, dependencySeqs, dependsOnTools }`. The compiled tool must expose these parameters and produce `expectedOutput`. `dependsOnTools` names independently callable setup tools the caller should run first; do not inline or replace them with a broader terminal workflow. `likelyParams` are the detector's best guess — confirm or correct each against the recorded requests.
 - `planGuidance?` — present when a global build plan ran first: `{ parserGuidance, paramChecklist, authRecipe, loadBearingSeqs }` for THIS tool. Treat it as prior guidance and reconcile it with the recorded data; if the recording contradicts it, prefer the recording and say so.
 - `assignedModules[]` — verified shared modules this tool MUST import instead of re-implementing: `{ path, kind, importPath, exportSignatures, purpose }`. A `request-transform` module reproduces the site's request signing/construction; a `parser-helper` extracts data from the response. Reference each by its exact `importPath`.
 - `requests[]` — the recorded requests in scope for this tool: `{ seq, method, url, headers, body, status, mimeType, responsePreview, ... }`. These are the ground truth — decode them, do not guess. `responsePreview` is truncated; note where the full body must be read.