@runcofactor/cli 0.13.0 → 0.15.1

package/dist/skills/debug-run/history/1.3.1.md

@@ -0,0 +1,279 @@
+---
+name: debug-run
+description: Root-cause a Canary workflow run end-to-end. Materializes the run's diagnostic workspace and explores it with Read (multimodal), Bash, Grep, and Glob. Use when the user asks "why did this run fail/hang/take so long", references a runId, pastes a flows/?runId=... URL, or asks "what changed since it last passed".
+allowed-tools: Read, Grep, Glob, Bash(canary workflow debug:*), Bash(canary workflow runs:*), Bash(jq:*), Bash(diff:*), Bash(sed -n:*), Bash(head:*), Bash(tail:*), Bash(wc:*), Bash(grep:*), Bash(find:*), Bash(ls:*), Bash(cat:*)
+---
+
+<!-- AUTOGENERATED from packages/diagnostic-playbook/. Do not edit by hand. -->
+<!-- After editing the partials, mint a new version: -->
+<!--   bun run --filter=@canaryai/cli bump-skill debug-run [patch|minor|major] -->
+
+# Workflow Run Debug Guide
+
+You are debugging a Canary workflow run end-to-end. The same diagnostic data the v2 failure-diagnostic agent uses is available to you here — materialize it once, then explore.
+
+## Materialize the workspace
+
+```bash
+canary workflow debug materialize <runId> --out ./diag-<runId>/
+```
+
+This produces a structured workspace on disk. After it completes, all subsequent navigation uses Read/Bash/Grep against that directory. The workspace is the **single source of truth** — the v2 agent and you read the exact same files.
+
+If you don't have a workflowId, list runs first:
+```bash
+canary workflow runs list <workflowId>
+```
+
+## Your job
+
+You are a senior QA engineer diagnosing why an end-to-end workflow test failed for a QA tester who has to **fix the test**. The reported failure is a **hypothesis**. Your job is to verify it with evidence and produce a tight, scannable report — see the output rubric.
+
+The QA tester reads your `title` first, then `recommendation`. Make those two land. Everything else is supporting detail.
+
+## Failure categories — pick exactly one
+
+- `software_bug` — defect in the application under test (4xx/5xx from app APIs, JS exceptions, broken UI, a behavior delta that is clearly broken)
+- `dependency_issue` — required data, fixture, or external service was missing for the test
+- `agent_confusion` — the AI test agent reasoned incorrectly, picked the wrong element, or got confused
+- `platform_error` — defect in our test runner, harness, sandbox, or recording infrastructure
+- `test_underspecified` — the test instructions themselves are wrong or ambiguous. **Only valid with positive evidence that the authored test changed or is genuinely ambiguous** (e.g. the History section reports AUTHORED TEST CHANGED, or two runs with the same instructions reasonably diverged). **Never** classify `test_underspecified` when the authored test is unchanged across a pass→fail boundary — an unchanged test that used to pass is not a test problem.
+- `app_changed` — the application's behavior, labels, copy, or flow changed versus the last passing run **while the authored test stayed the same**, and you cannot verify from inside the sandbox whether the change was intentional. This is the correct verdict for an unchanged test that breaks because the app moved underneath it (e.g. a button relabeled "Login" → "Acting As"). Name the concrete delta and recommend "X appears to have changed to Y; update the test to match if intentional." Reserve `software_bug` for deltas that are clearly defects (error page, 5xx, broken layout); use `app_changed` when the new behavior looks deliberate or you simply cannot tell. **An `app_changed` verdict must be earned by the green-run comparison** (see compare-to-history): confirm the delta in both runs' evidence, never infer it from the pass→fail boundary alone. If the green baseline passed via cache replay while the failing run improvised a different live verification, that is verification drift (`agent_confusion`), not an app change.
+- `data_collision` — the workflow interacted with the **wrong data**: a leftover org from a prior session, a duplicated row, a record from another run, the wrong tenant, etc. The point of diagnosis is *why the workflow drifted onto wrong data* — stale cached recipe pointing at the wrong DOM node, locator matching multiple elements, variable bound to an earlier session/org, agent fallback choosing the first match. **Do not** investigate the provenance of the colliding data.
+
+Never mark `agent_confusion` or `platform_error` from a failed assertion alone. Look at multiple evidence sources first.
+
+## Output discipline (hard rules)
+
+- **Never** mention internal artifact paths in any user-visible field. Banned substrings in `title`, `failurePoints`, `recommendation`, `evidence.description`, `evidence.detail`, `openQuestions`, `comparisonWithHistory`: `steps/`, `workflow/nodes/`, `logs/`, `thoughts/`, `history/last-green`, `history/runs/`, `browser/frames/`, `the ledger`, `the workspace`, `<stepId>`, `<nodeId>`. Speak in domain terms: "the authored step description", "the network log", "the agent thought at step 4", "the last-passing run on YYYY-MM-DD".
+- **No section headings** inside `failurePoints` or `recommendation`. They are flat bullet lists.
+- **No narrative paragraphs.** If you find yourself writing "Then…", "After that…", "Subsequently…" — collapse to a bullet.
+
+
+## Workspace layout
+
+Your diagnostic workspace is at `{{workspacePath}}`. Everything you need is here. Read in three tiers:
+
+**Tier 1 — always read first:**
+- `summary.md` — run-level overview, step table, error counts, notable events, history hint, and pointers to the right rollup files
+- `reported-failure.md` — the test runner's hypothesis (verify, do not trust)
+
+**Tier 2 — drill-down per step:**
+- `steps/<NNN>-<stepId>.md` — per-step rollup (timing, status, error, thoughts, in-window network/console events, navigation, assertions, artifact paths). Read the failed step's rollup first, then walk backwards.
+
+**Tier 3 — windowed reads only:**
+- `ledger.md` — full chronological narrative. **Do not read whole.** Open `ledger-index.md` first for the step→line-range mapping, then `bash("sed -n '<a>,<b>p' ledger.md")`.
+
+## Other artifacts
+
+- `browser/frames/<NNNN>-<absoluteTsMs>.png` — screenshots per browser action. Read with `read_file`; the model sees them as vision input.
+- `browser/doms/<NNNN>-<absoluteTsMs>.html` — composed self-contained HTML per action.
+- `browser/a11y/<NNNN>-<absoluteTsMs>.yaml` — what the agent's accessibility view looked like.
+- `browser/actions.jsonl` — parsed Playwright actions with timestamps. Use `jq` to filter.
+- `browser/trace.zip` — raw Playwright trace (escape hatch).
+- `workflow/nodes/<nodeId>.json` — full configJson per workflow node. Quote `actionDescription`, `inlineAssertions` from here.
+- `workflow/snapshot.json` — full workflow snapshot the run executed.
+- `thoughts/all.jsonl` — every agent thought with stepId, timestamp, content. Use `jq` to filter.
+- `thoughts/by-step/<stepId>.md` — readable per-step thoughts.
+- `timeline/{steps,actions,navigation,assertions,cache-decisions}.jsonl` — structured event streams.
+- `logs/network.jsonl`, `logs/network-errors.jsonl` — network requests; errors are pre-filtered.
+- `logs/console.jsonl`, `logs/console-errors.jsonl` — console logs; errors+warnings are pre-filtered.
+- `logs/sentry-logs.jsonl`, `logs/sentry-spans.jsonl` — Sentry telemetry. May contain a `partial: true` first line if the Sentry token wasn't available.
+- `variables/inputs.json`, `variables/lineage.jsonl` — run inputs + per-step variable captures.
+- `history/recent-runs.jsonl` — last ~20 runs with status. Useful for picking a green run to compare.
+- `history/runs/<runId>/` — comparison workspaces created by the `materialize_run` tool (in-process sandbox).
+- `history/last-green/` — populated only if you materialize a prior passing run for comparison via the CLI command.
+- `notes/` — your working memory. Write here with `write_file` (only the workspace, no path escapes).
+
+
+## Investigation procedure (Claude Code edition)
+
+Follow this in order. Adapt the v2 agent's procedure to use Claude Code's built-in tools.
+
+1. **Anchor at the reported failure.** `Read summary.md` first — it has the run-level overview, the failed step's ordinal, the file path of the failed step's rollup, and the history hint. Treat the reported failure as a hypothesis, not a fact.
+2. **Verify with evidence — including visual evidence.** `Read steps/<NNN>-<failedStepId>.md` for the failure context. Read the screenshot file referenced there (`browser/frames/<NNNN>-...png`) — `Read` is multimodal, vision describes what's on screen. Look for unexpected modals, banners, blank areas, wrong brand, layout shifts. Cross-reference network errors and console errors in the rollup.
+3. **Walk backwards.** The reported failure is rarely the divergence point. Read the prior step's rollup (`steps/<NNN-1>-...md`), look at its screenshot, and ask: did this step actually accomplish its goal? Repeat going backwards until you find a step where the actual outcome diverged from the intended outcome.
+4. **Compare to history.** If `summary.md`'s history section shows recent passes, the most recent green run's testRunId and materialize command appear there. Run that command, then visually compare the suspect step's screenshot in both runs.
+5. **Present clear evidence.** Your write-up cites specific artifacts (file paths, timestamps, screenshot indices, network entries). When visual comparison is decisive, name both screenshot paths.
+
+## Important
+
+- Never `Read ledger.md` without a specific window — open `ledger-index.md` first to find the line range, then `Bash sed -n '<a>,<b>p' ledger.md`.
+- The workspace is the canonical data layer. If you can't answer a question from the workspace, that's a signal the materializer is missing a domain — file a follow-up rather than reaching for psql/sentry CLI directly.
+
+## Verifying the reported failure
+
+The failure report (in `reported-failure.md` and the top of `summary.md`) is the **runner's hypothesis** about what failed. It is wrong often enough that you must always verify before forming a diagnosis.
+
+Things that look like a step-3 failure but are really upstream:
+- The agent on step 1 picked the wrong "Sign in" link, ended up on the consumer site instead of the staging site, and the failure surfaces as `AuthenticationError` two steps later.
+- A cache-replay on step 2 used a stale selector that matched a hidden element; the step "succeeded" but didn't actually click anything; step 3 fails because the modal never opened.
+- The runner timed out a step but the page actually loaded fine — the timeout is a platform issue, not the app's fault.
+
+Always cross-reference at least three sources before forming a hypothesis:
+1. The screenshot at the failure timestamp (`browser/frames/...`)
+2. The agent's thoughts during the failed step (`thoughts/by-step/<stepId>.md`)
+3. Either the network errors (`logs/network-errors.jsonl`) OR the console errors (`logs/console-errors.jsonl`) within ±2s of the failure timestamp
+
+If the screenshot, thoughts, and logs all paint the same picture, the reported failure is probably accurate. If any one disagrees, walk backwards (see procedure step 3).
+
+
+## Walking backwards
+
+The visible failure is rarely the actual divergence. Common patterns:
+
+- A `click` succeeded 3 steps ago but landed on the *wrong* element — the cache-replay layer hid the mismatch — and the failure surfaces only when the next required field can't be found.
+- A cross-host navigation between two earlier steps invalidated the auth session, and the 401 doesn't appear until the next API call.
+- A variable was captured from the wrong DOM element on step N, and step N+5 now has a meaningless `$var.X` value.
+- The agent went down a wrong branch on a conditional, and the test ran a different path than intended.
+- A prior step showed a validation banner or issue toast, but the workflow continued; the later step fails because the page never becomes ready.
+- A non-blocking assertion or warning already proved a prerequisite was missing; a later action reports the same missing state as a blocking failure.
+- A warning says an expected run, save, import, sync, or publish did not complete; the downstream failure is just the consequence of that missing prerequisite.
+
+Walk-backwards algorithm:
+
+1. `read_file` `steps/<NNN-1>-<prevStepId>.md` for the step immediately before the failure.
+2. `read_file` its screenshot (`browser/frames/<NNNN>-...png`). **Look at it.** Did the step actually accomplish its goal?
+3. Compare to the instruction by `read_file` on `workflow/nodes/<nodeId>.json` — does the screenshot match the intent of `actionDescription` and `inlineAssertions`?
+4. Skim the rollup's "Agent thoughts" section. Was the agent confused? Did it pick the wrong element?
+5. Skim the rollup's "Navigation" section. A cross-host transition (⚠ marker) is a strong signal that auth state was lost between this step and the next.
+6. Check whether the step has warnings, non-blocking failures, cache fallback, validation banners, issue toasts, or visible error badges. Treat these as root-cause candidates even if the step status is `success`.
+7. If everything looks normal here, repeat with `steps/<NNN-2>-...md`, `<NNN-3>`, and so on.
+
+Stop when you find a step where the actual outcome diverged from the intended outcome. That is your root-cause candidate. Cite that step's screenshot and the diverging detail in your `actuallyHappened` narration.
+
+If the reported failure says a later state was not established, your final answer must name the earlier step that should have established it and the concrete evidence showing why it did not.
+
+
+## Visual evidence — when to read frames
+
+A large fraction of diagnoses turn on what the page actually *looked like* at a given moment. Text alone cannot tell you:
+- Whether an unexpected modal/banner is blocking the click target
+- Whether the layout shifted such that the agent's selector resolved to a different element
+- Whether the brand on screen matches what the test expected (cross-host redirects often land on a sibling brand)
+- Whether a stale cached fragment is being shown
+- Whether a button is disabled, missing, or moved
+
+How to use vision:
+
+- Use `read_file` on `browser/frames/<NNNN>-<absoluteTsMs>.png`. The file extension triggers multimodal output — the model receives the image as vision input on the same turn.
+- For run-vs-run comparison, read the frame at the same step in `history/last-green/browser/frames/<NNNN>-...png` and the failing run's frame in adjacent `read_file` calls. Reason about the visual delta directly.
+- When visual evidence is decisive, include a `before_after` entry in your `evidence[]` with the two image paths.
+
+Do not use `bash` to view images — `read_file` is the only tool that returns vision content.
+
+
+## Comparing to a passing run — the green→red methodology
+
+If `summary.md`'s history section names a recent green run, comparison is **required** — it is the highest-yield step for a pass→fail regression. A green→red boundary with an unchanged test eliminates only test-authoring causes. FOUR hypotheses stay live until the comparison discriminates between them: a legitimate app change (`app_changed`), an app regression (`software_bug`), an agent error (`agent_confusion`), and a platform error (`platform_error`). Never default to one of them — earn the verdict with evidence.
+
+Procedure:
+
+1. **Materialize the green baseline.** `summary.md`'s "History" section names the green run and the exact mechanism for this workspace:
+   - In the in-process sandbox: call the `materialize_run` tool with the green run's id. The result names the comparison workspace path (`history/runs/<greenRunId>/`).
+   - In a CLI-materialized workspace: run the printed `canary workflow debug materialize <greenRunId> --out history/last-green/` command via `bash`.
+   Follow whichever instruction `summary.md` prints. `<green>` below means the comparison workspace path.
+2. **Establish execution mode on BOTH sides of the boundary — before any visual comparison.** Read each workspace's `timeline/cache-decisions.jsonl` and `thoughts/all.jsonl`. A cache-replayed run has few or no agent thoughts (it replayed recorded actions); a live-agent run has many. The mode pairing changes which hypotheses are plausible:
+   - **Green replayed, red fell to live agent** → the red run executed a *different procedure* than every green before it. Compare the recorded recipe's verification with what the live agent improvised — a verification-method mismatch points to `agent_confusion` (drift), and the cache miss/uncertainty itself may point to `platform_error`. Do NOT classify `app_changed` from a replay-vs-live pair without an independent, concrete app-behavior delta visible in both runs' frames.
+   - **Both live-agent** → thoughts diff cleanly; interpretation drift vs. a genuine page delta is directly observable.
+   - **Green replayed, red replay-failed** → suspect `platform_error` (replay machinery) or a genuine app change that broke the recorded selectors; the frames decide.
+3. **Visual comparison.** `read_file` the suspect step's screenshot in both runs:
+   - `browser/frames/<NNNN>-...png` — failing
+   - `<green>/browser/frames/<NNNN>-...png` — passing
+   The visual delta is often the smoking gun. Same screens → the app did not change; weigh the agent/platform hypotheses. (If the green workspace has no frames — no trace — fall back to thoughts and node-config diffs.)
+4. **Diff thoughts and node configs:**
+   - `bash diff thoughts/by-step/<stepId>.md <green>/thoughts/by-step/<stepId>.md`
+   - `bash diff workflow/snapshot.json <green>/workflow/snapshot.json | head -100`
+
+Common signals from comparison:
+
+- Identical authored workflow + visual/label delta **confirmed in both runs' frames** = **app changed**. Classify `app_changed` (intent unverifiable) unless the delta is clearly a defect (error page, 5xx, broken layout), in which case `software_bug`.
+- Different workflow snapshot = **test changed** (possibly `test_underspecified` if the new instructions are ambiguous).
+- Same screenshot but different thoughts = **agent interpretation drift** (`agent_confusion`, or `test_underspecified` if the test allowed the prior interpretation).
+- Same everything but new network 5xx = **dependency/backend issue** (likely `software_bug` or `dependency_issue`).
+- Green passed by cache replay while red improvised live = **verification drift**, not an app change — see step 2.
+
+## When comparison is unavailable
+
+If the comparison workspace cannot be produced (the materialize tool/command fails), you have NOT performed the required comparison. Do not pick `app_changed` at high confidence on the regime steer alone — keep the live hypotheses in your writeup, record an explicit openQuestion that the green baseline could not be materialized for comparison, and cap your confidence accordingly.
+
+## The regime gates your verdict
+
+`summary.md`'s History section reports a **Regime** and, when available, an **AUTHORED TEST UNCHANGED / CHANGED** line computed from a content fingerprint of the authored definition. Do not overrotate on a bare "last run green, this run red" — gate the conclusion on the regime:
+
+- **`STABLE_REGRESSION` + AUTHORED TEST UNCHANGED** → the test did not change across the pass→fail boundary, so `test_underspecified` is **off the table** — but that alone does NOT pick a verdict. Run the methodology above to discriminate between `app_changed`, `software_bug`, `agent_confusion`, and `platform_error`. Only classify `app_changed` after confirming a concrete behavior delta against the green workspace; name the delta and record in `openQuestions` that you could not verify whether the change was intentional.
+- **`STABLE_REGRESSION` + AUTHORED TEST CHANGED** → the test was edited; judge whether the new instructions are ambiguous (`test_underspecified`) before blaming the app.
+- **`FLAKY`** → a single green→red does **NOT** justify `app_changed`. The test oscillates; weigh `agent_confusion`, intermittent `dependency_issue`, and `data_collision`. Comparison still helps, but only conclude `app_changed` with a **clear, persistent** behavior delta — never a one-off.
+- **`NEW` / `PERSISTENT_FAIL`** → no stable green baseline; rely on the within-run procedure (verify + walk backwards).
+
+
+## Standing rules
+
+## Standing rule: auth errors after a cross-host transition
+
+If the failure looks like a credential or session issue (401/403, "session expired", login redirect, "please sign in"), **first reconstruct the URL timeline around the failure**.
+
+1. Read the failed step's rollup (`steps/<NNN>-<stepId>.md`) — the "Navigation (in step window)" section lists URL transitions and marks cross-host transitions with `⚠ cross-host`.
+2. Skim the prior step's navigation as well — the cross-host hop often happens between steps.
+3. Compare host names case-insensitively, stripping a leading `www.`. So `staging.example.com` → `www.example.com` is a cross-host change; `www.example.com` ↔ `example.com` is not.
+
+**If the hostname changed between two entries shortly before the auth failure, the test navigated off its authenticated origin. That is the root cause, not stale credentials.** Do not classify as `dependency_issue` (credentials problem) without ruling out cross-host.
+
+Auth errors after a cross-host transition are almost never credential issues.
+
+
+## Standing rule: workflow variables and macros
+
+Workflow steps can use **macros** and **variable references** in their configuration:
+
+- **Macros** (generate values at runtime): `$MACRO_NAME(args)` — e.g. `$EMAIL()`, `$UUID()`, `$RANDOM_STRING(10)`. Used in `inputValues` on action nodes; resolved fresh each run.
+- **Variable references** (reference values from earlier steps): `$var.variable_name`. Variables are created when a step's `inputValues` are resolved. **A variable defined in step N is NOT available within step N itself** — only in step N+1 and later. This is because variables are registered after the step's inputs are resolved.
+
+Common variable-related failures:
+- A step references `$var.X` where X is defined in that same step (will be unresolved → blank value used).
+- A step references `$var.X` where X was defined in a step that was skipped or failed earlier.
+- Misspelled variable names (e.g. `$var.client_name` vs `$var.clientName`).
+- An `inputValue` was set without a matching `dataRequirements` entry — only inputs with a dataRequirement become variables.
+
+When suggesting test instruction changes:
+- Use `read_file` on `workflow/nodes/<nodeId>.json` to see the full configJson before proposing a `configPatch`.
+- Preserve any existing `$var.` references and `$MACRO()` calls — don't break them.
+- For array fields like `inlineAssertions` or `playbookSteps`, the `configPatch` must be a **full replacement array**, not a partial — read the existing array first and include the merged result.
+
+
+## Standing rule: assertion severity and step/run status
+
+Assertion severity drives what a failure means — `fatal` halts the run at the failing step, `fail` marks the step (and ultimately the run) failed but execution continues, `warn` records a warning and the run still passes. Because `fail` and `warn` are non-blocking, later steps run normally after an assertion failure. Assertion records carry a `nonBlocking` flag reflecting how each was treated at run time.
+
+When investigating:
+
+- Open the step's rollup (`steps/<NNN>-<stepId>.md`) and look at the "Assertions" section. Each line shows `passed`/`FAILED`, severity, and a `nonBlocking` marker. Entries marked not-checked (`checked: false`) were configured on the node but never evaluated — the step broke before assertions ran; do not read them as failures.
+- **Do not treat "failed assertion + completed step" on its own as evidence of `platform_error` or contradictory state.** For `warn`/`fail` severities (`nonBlocking` true), that combination is expected and carries no signal about the actual cause.
+- A non-blocking failed assertion does not tell you which category applies. It only rules out one specific false-positive narrative (`platform_error` / bookkeeping). Continue investigating from network, console, thoughts, visual, and history. The actual cause may be any of `software_bug`, `dependency_issue`, `agent_confusion`, `test_underspecified`, or `platform_error`.
+
+When `platform_error` is still a live possibility:
+
+- A failed `fatal`-severity assertion (`nonBlocking: false`) alongside a step recorded as completed successfully. That is a genuine disagreement worth flagging.
+- Infrastructure-level failures (browser session died, worker crash, our own stack traces) independent of any assertion outcome.
+
+Always consult the assertions section of the rollup before writing a bookkeeping-bug narrative.
+
+
+## Reporting
+
+The QA tester reading your report sees, by default, only the `title`, `Key Evidence`, and `Recommendation`. Everything else is twirl-down. Optimise for that.
+
+- `title` — one sentence, concrete and specific. The thesis.
+- `failurePoints` — 2–4 bullets describing the failure; quote the authored expectation inline only when it sharpens the point.
+- `recommendation` — 1–3 imperative bullets matched to the failure category:
+  - `software_bug` → name the app defect; the underlying error is quoted in an evidence item
+  - `dependency_issue` → name the missing data/fixture/credential and where to provision
+  - `agent_confusion` → quote the confused thought, point at visual evidence the agent missed, suggest test-instruction tweaks
+  - `platform_error` → describe the harness failure, suggest filing a platform issue
+  - `test_underspecified` → quote the ambiguous instruction; provide a clearer rewording
+  - `data_collision` → explain *why* the workflow drifted onto wrong data (stale cached recipe, ambiguous locator, stale variable binding); propose a recipe/locator/variable fix only when confidence is ≥80
+- `openQuestions` (optional) — what you couldn't determine. Skip if you have full conviction.
+
+Never reference internal artifact paths (`steps/…`, `workflow/nodes/…`, `logs/…`, `thoughts/…`, `history/…`, `browser/frames/…`, "the ledger", "the workspace") in any field the QA tester sees. Speak in domain terms: "the authored step description", "the network log", "the agent thought at step 4".
+
+Don't hand back "the run failed" or a Sentry error message alone. The whole point is to triangulate to a concrete cause.

package/dist/skills/debug-run/version.json

@@ -1,3 +1,3 @@
 {
-  "current": "1.3.0"
+  "current": "1.3.1"
 }

package/dist/skills/optimize-cache/SKILL.md

@@ -12,7 +12,7 @@ allowed-tools: Read, Grep, Glob, Bash(canary workflow debug:*), Bash(canary work
 
 You are building the deterministic cache + playbook for a workflow from the
 evidence of a real run — the same procedure the server's async cache-optimizer
-agent follows, driven manually through the canary CLI.
+agent follows, driven manually through the `canary` CLI.
 
 ## Materialize the workspace
 
@@ -53,8 +53,15 @@ thoughts, and the run's variable/macro records. Work methodically:
 2. **Pick harvestable nodes.** Only nodes whose step SUCCEEDED are
    harvestable. A node that succeeded in a run that later failed is still
    valid — often the most valuable, since that run won't be green soon.
-   Cache-replayed nodes (their trace actions sit under a `Cache Replay:`
-   group) carry nothing new — skip them.
+   Where a harvestable node's actions live depends on how it executed:
+   - Agent-executed (cache miss/fallback): under its `Step N:` trace group.
+   - Replayed WITH healing (`l2_gap` / `recovery`): under its
+     `Cache Replay N:` group — replayed actions and the healing agent's live
+     actions are recorded together there, and ALL of them are evidence of the
+     proven path. Harvest these exactly like agent steps.
+   `optimizer/segments.json` resolves each segment to its `nodeId`; trust it.
+   A CLEANLY replayed node (its step needed no healing) carries nothing new —
+   skip it.
 3. **Per node, reconstruct the true path.** Cross-reference the node's trace
    actions with the agent's tool calls and thoughts. Identify what actually
    accomplished the task versus exploration.
@@ -78,6 +85,11 @@ gap. Core actions are:
 - **Captures**: values the agent saved for downstream steps (e.g.
   `browser_evaluate` with `saveAs`). These become context extractors on the
   cache row — `declaredOutputs` stay reserved for DOM-captured values.
+  You may author a new capture when the run evidence shows the app generated
+  a value that later cacheable work depends on, even if the live agent did not
+  capture it. The capture expression must read visible DOM or the current URL,
+  return a single string, and save the key used by later `$var.key` /
+  `{{var.key}}` references.
 - **Non-browser effects**: reading email/inbox, running python, reading or
   verifying downloads. These are invisible in the browser trace — source them
   from the agent's tool-call records and thoughts. They become playbook steps
@@ -110,9 +122,28 @@ future runs work with fresh data:
   re-resolves selector text from run state. When a locator can only be
   disambiguated by a run-specific value, bind it to the variable that
   produced the value — never anchor on the literal.
+- **App-generated values the live agent missed** are still recoverable here.
+  If the app creates a value during this run (receipt number, confirmation
+  code, generated entity ID, auto-derived title or name, URL slug) and a
+  later locator, assertion, or download check depends on it, call
+  `declare_dynamic_variable` with the observed value before building
+  locators. Then add a cached `capture` action that saves the same key before
+  the first dependent cached action, and use `$var.key` / `{{var.key}}` where
+  the generated value appears later.
+  Constants chosen by the workflow (names, dates, dropdown options, payment
+  types) do not need this treatment unless they are app-generated.
+  This is a REQUIREMENT, not a suggestion: once you suspect a distinguishing
+  value is app-generated (you saw the app derive it, the workflow never typed
+  it, or it differs from what an earlier step entered), you may NOT anchor on
+  the literal and you may NOT silently gap the step. Find the value on-path
+  (a heading, breadcrumb, row cell, or the URL of a page the step already
+  visits), capture it, and bind the locator to the capture. Gap only when no
+  on-path capture source exists — and say so in the gap reason.
 - **Gaps** are honest: a `gap` compiled entry means "the replay agent performs
   this step live". A gap replays slower but correctly; a bad cached locator
-  replays fast and wrong. Prefer the gap.
+  replays fast and wrong. Prefer the gap — except where the only obstacle is
+  an app-generated value with a visible capture source (see above): there the
+  capture-bound locator is the correct outcome, not a gap.
 - **Volatile identifiers** (seeded entity numbers, ULIDs/UUIDs, timestamps,
   framework-generated class hashes) WILL differ next run even though they
   point at a real element today. Structural prefixes (e.g. a stable
@@ -181,6 +212,12 @@ UNVERIFIED locator must never be committed.
   recipe depended on must survive into the new recipe (as a token, a variable
   binding, or a macro binding). The commit validates this and rejects recipes
   that inline run-specific values.
+- **Declared dynamic variables must be captured.** If you call
+  `declare_dynamic_variable` and later use that key in a locator, assertion,
+  playbook param, or action arg, the committed recipe must include a cached
+  `capture` action whose `args.saveAs` is the same key. Put the capture before
+  the first mutation or verification that requires the value; otherwise replay
+  will fail before it can hydrate the selector.
 - The node hash is recomputed server-side from the node context with your
   playbook applied; the next run computes the same hash and replays your
   recipe. You never supply a hash.

package/dist/{spec-LM5XHOQ5.js → spec-XZ2TKUFZ.js}

@@ -12,7 +12,7 @@ import {
   workflowToSpec,
   writeNodeIds,
   writeWorkflowId
-} from "./chunk-YN763NWQ.js";
+} from "./chunk-26U3DGUM.js";
 import {
   fetchOrExit
 } from "./chunk-3CZU7XCS.js";
@@ -20,15 +20,15 @@ import {
   getArgValue,
   hasFlag,
   resolveAppUrl
-} from "./chunk-PZ4LCN6K.js";
+} from "./chunk-IGHDIWZU.js";
 import {
   isSpecFile,
   readableSpecDirs,
   specDir,
   specExtension,
   splitSpecExtension
-} from "./chunk-JZMJUPZP.js";
-import "./chunk-QGI4UCJX.js";
+} from "./chunk-32P26BZT.js";
+import "./chunk-IWKJQZJQ.js";
 import "./chunk-VKVL7WBN.js";
 
 // src/spec/index.ts
@@ -977,4 +977,4 @@ export {
   stampedSpecPath,
   uniqueName
 };
-//# sourceMappingURL=spec-LM5XHOQ5.js.map
+//# sourceMappingURL=spec-XZ2TKUFZ.js.map