@capagents/forgeqe 1.2.1 → 1.3.0

package/HOW_TO_USE.md

@@ -50,13 +50,14 @@ Everything ForgeQE can do, by area. ForgeQE's agent answers "can you…?" questi
 | Browser | **Run any goal in a real browser**: Carries out a plain-language goal in a real browser one step at a time and verifies the outcome, with screenshots, a report, and the steps it took. The browser comes from forgeqe.yaml: local Chrome or Chromium by default, or BrowserStack or TestMu AI/LambdaTest when configured. Page memory replays steps that already worked on a site, so repeat runs are faster and cheaper. | `browser_run`, `run_details`, `page_memory`, `browser_devices` | [docs](#4-without-the-shell) |
 | Browser | **Saved logins for the application under test**: Signs the browser agent in with saved credentials (including TOTP) and reuses the session; a user can also sign in by hand once in a browser. | `logins` | [docs](#signing-in) |
 | Test cases | **Run manual test cases in browsers**: Reads test cases from Azure DevOps, Jira, Xray, TestRail, Zephyr Scale, qTest, a file, or pasted text, runs each in a real browser with a verdict per step, writes report.html and junit.xml, posts results back, and files or updates bugs for failures. | `find_test_cases`, `run_test_cases` | [docs](#7-test-cases-and-suites) |
-| Test cases | **Check and improve test cases**: Checks test cases before they run (vague or missing expected results, missing test data, placeholders, duplicates) against the story, feature, and epic they test, then rewrites Azure DevOps cases once the user agrees. | `check_test_cases`, `update_test_case` | [docs](#checking-test-cases-before-they-run) |
-| Test cases | **Write test cases from a story**: Writes test cases from a story's acceptance criteria and saves them in Azure DevOps or Jira, linked to the story. | `save_test_cases` | [docs](#where-cases-come-from) |
+| Test cases | **Check and improve test cases**: Checks test cases before they run (vague or missing expected results, missing test data, placeholders, duplicates) against the story, feature, and epic they test, then rewrites Azure DevOps or Jira cases in place once the user agrees. | `check_test_cases`, `update_test_case` | [docs](#checking-test-cases-before-they-run) |
+| Test cases | **Write test cases from a story**: Writes test cases from a story's acceptance criteria and saves them in Azure DevOps or Jira, linked to the story. Reads the cases already linked first, so asking again adds only what is missing and updates cases in place instead of duplicating them; plans end-to-end journeys from the existing cases. | `story_test_cases`, `save_test_cases`, `e2e_journey` | [docs](#where-cases-come-from) |
 | Test cases | **Domain and regulatory test design**: Every agent works as an expert in the product's industry would, automatically: the risks the industry is known for and the obligations of its regulations and standards (for example DORA, PSD2, GDPR, ISO 27001), declared in forgeqe.yaml or worked out from the story and the application. It applies to analysing requirements, writing and checking test cases, exploration, writing, reviewing, and healing test code, API tests, finding tests, and pull request reviews, labelled so coverage can be traced and citing only the control ids in the team's compliance file. forgeqe eval domain measures how well the agent does it. |  | [docs](#16-domain-and-regulations) |
 | Test cases | **Exploratory testing of a story**: Explores a story in real browsers beyond its scripted cases (edge cases, invalid input, boundaries, roles, state), confirms each defect by reproducing it, proposes test cases, and files bugs once the user agrees. | `explore_story` | [docs](#exploratory-testing-from-a-story) |
 | Automated tests | **Turn a passed run into an automated test**: Scribe, the coding agent, writes a test for a passed browser run in the repository's own framework, page objects, and helpers; ForgeQE runs it until it passes and a reviewer approves it. | `write_test`, `run_test` | [docs](#6-tests) |
 | Automated tests | **Heal failing tests**: Runs a repository's tests and updates the ones the application has outgrown against the live app, or reports a real application bug instead. | `heal_tests` | [docs](#fixing-broken-tests) |
 | Automated tests | **Find and run existing tests**: Finds the automated tests that match a feature, tags, ids, a folder, or a sprint, confirms the selection, and runs it once the user agrees, with a verdict per test. | `find_tests`, `run_tests`, `list_sprints` | [docs](#finding-and-running-existing-tests) |
+| Automated tests | **Regression packs for a release**: Builds a regression pack from what goes into a release (an Azure DevOps sprint or work items, a commit range, pull requests, or changes read from any connected tool): the tests those changes can break, each with why, plus risk-area and smoke tests, and the changes no test covers. Runs it locally, on TestMu AI, or in a pipeline, and compares each run with the last. | `build_regression_pack`, `run_regression_pack`, `list_regression_packs` | [docs](#regression-packs-for-a-release) |
 | Automated tests | **Flaky test detection and quarantine**: Finds tests that flip between passing and failing in the repository's run history, with their likely cause, and quarantines them once the user agrees. | `flaky_tests`, `quarantine_tests` | [docs](#flaky-tests) |
 | Pull requests | **Review pull requests**: Reviews an Azure DevOps or GitHub pull request from the local clone for bugs, security, requirements, and tests, and posts the review once the user agrees. | `review_pull_request`, `post_pr_review` | [docs](#reviewing-pull-requests) |
 | Pull requests | **Pull request impact and test selection**: Works out which product areas, pages, requirements, and tests a pull request touches, picks the tests it can break, runs them, and posts the impact or results on the pull request once the user agrees. | `analyze_pr_impact`, `select_pr_tests`, `post_pr_test_results`, `comment_on_pr` | [docs](#what-a-pull-request-impacts-in-the-product) |
@@ -859,6 +860,42 @@ Review my open pull requests
 
 Through `forgeqe mcp`, `post_pr_review` posts only when called again with `confirmed: true`; the MCP client should ask you first.
 
+### Regression packs for a release
+
+A regression pack is the set of your repository's tests that a release's changes can break, saved under a name so you can run it again (here, on TestMu AI, or in a pipeline) and compare runs. Ask in your own words:
+
+```text
+Build a regression pack for sprint 12 and run it on TestMu
+Which tests should we run for the PRs merged this week?
+Build a pack for Jira fix version 2.4 and the commits since release/2.3 in ../shop-app
+```
+
+**Where the changes come from.** Any mix of:
+
+- an Azure DevOps sprint, area, epic or feature, or work item ids: their stories and bugs, the test cases each is tested by, and the files their linked pull requests changed;
+- the commits between two refs of a repository (`from` and `to`): the product's repository, or the test repository's own; work items the commit messages name (`AB#123`, `#123`, `PAY-12`) are read too;
+- pull requests by number or URL, or the ones merged in the last N days (Azure DevOps or GitHub);
+- anything ForgeQE's agent can read through its connected tools (a Jira release, GitHub issues, a Confluence release note): it reads the items and passes them in with their text and links.
+
+**Why each test is in.** A test can be in for more than one reason, and the card and `<name>.md` say each one:
+
+| Reason | Meaning |
+|---|---|
+| linked | it automates a test case that tests a changed work item (`.forgeqe/test-cases.json`, or a case id in the test's attributes) |
+| test code changed | the change is in the test repository, in code the test uses |
+| matches a change | the work item's or pull request's words name what the test does: the elements it uses, its page, its name |
+| risk area | the release touches an area with defects or failures before, and the test works there |
+| smoke | the test is tagged smoke, so it is in every pack |
+
+**Gaps.** Changes no test is in for are listed as *not covered*, each with the closest existing test and what it lacks, so you can extend that test rather than write a new one.
+
+**Running it.** ForgeQE asks before it runs anything.
+
+- *This machine*: ForgeQE builds one command for exactly the pack's tests, the way the repository already runs a test (for .NET, a `--filter` with each test, the settings an earlier job used, TRX results), runs it, and reads each result. A failed test runs once more; one that then passes is flaky.
+- *TestMu AI*: the same command, with the variable `FORGEQE_PW_WS_ENDPOINT` set to TestMu AI's Playwright endpoint for the browser and platform in `testmu.yaml` (the access key goes in the variable, never in the command, and is removed from the output). The tests must connect to it when it is set: for NUnit, override `ConnectOptionsAsync` in the test base class; for Playwright Test, set `connectOptions` from it in `playwright.config`. ForgeQE refuses a TestMu run when the tests do not read the variable, so a run never looks remote while it ran locally.
+
+Each run is kept with the pack: the card says which tests fail now that passed in the last run, and which were fixed. Packs are saved in `.forgeqe/packs/<name>.json` and `<name>.md`; run output and results in `.forgeqe/reports/packs/<name>/<time>/`.
+
 ### Flaky tests
 
 Every `run_tests` and `pr-tests` run keeps each test's result in `.forgeqe/test-history.json` in the repository (commit it to share the history, or keep it between CI runs as a cache). A test that fails runs once more in the same run, with the framework's own option (Playwright `--last-failed`, pytest `--lf`, Jest `--onlyFailures`; otherwise the whole command again when the selection is small): when it then passes it is **flaky**, not failed, and the run does not fail because of it.

package/README.md

@@ -236,7 +236,7 @@ sources:
 
 ### Reports
 
-Every suite writes `report.html`, `junit.xml`, and `suite.json` to `.forgeqe/reports/suites/<time>/`, with each case's run folder beside them; every single run writes its own `report.html` too. The report is one HTML file with the screenshots embedded, so you can open it straight from disk, attach it to a build, or mail it; no server is needed. It has an overview (verdict, totals, step checks, tokens, cost, page memory savings, browser errors, accessibility issues, bugs filed, what changed since the last run, results over earlier runs, the cases that need attention, results by source and tag, the slowest cases, and what was posted where), a test results page (each case's step table with expected and actual results, why it failed, its last passing screenshot beside this run's, browser errors per step, page memory with healed locators, the browser agent's steps with a screenshot each and the clicked element outlined, a timing waterfall, accessibility issues, and the trace and workflow to read in place), a failures page, since last run, trends and flaky tests from the earlier suite runs in the same `.forgeqe/reports/suites/` folder, time and cost (model vs browser time, a parallel worker timeline, cost per case from the model's `price` in `llms.json`), coverage by requirement and group from the test system, accessibility checks for every page visited, and the environment (model, browser, ForgeQE version, CI build, branch, commit). It exports Markdown and JSON and prints cleanly. See `samples/reports/nightly/20260929-020005/report.html` for an example. `report.screenshots: failures` keeps screenshots for failed runs only, `off` drops them; `report.accessibility: false` skips the page checks. On GitHub Actions the suite also writes a job summary.
+Every suite writes `report.html`, `junit.xml`, and `suite.json` to `.forgeqe/reports/suites/<time>/`, with each case's run folder beside them; every single run writes its own `report.html` too. The report is one HTML file with the screenshots embedded, so you can open it straight from disk, attach it to a build, or mail it; no server is needed. It has an overview (verdict, totals, step checks, tokens, cost, page memory savings, browser errors, accessibility issues, bugs filed, what changed since the last run, results over earlier runs, the cases that need attention, results by source and tag, the slowest cases, and what was posted where), a test results page (each case's step table with expected and actual results, why it failed, its last passing screenshot beside this run's, browser errors per step, page memory with healed locators, the browser agent's steps with a screenshot each and the clicked element outlined, a timing waterfall, accessibility issues, and the trace and workflow to read in place), a failures page, since last run, trends and flaky tests from the earlier suite runs in the same `.forgeqe/reports/suites/` folder, time and cost (model vs browser time, a parallel worker timeline, cost per case from the model's `price` in `llms.json`), coverage by requirement and group from the test system, accessibility checks for every page visited, the environment (model, browser, ForgeQE version, CI build, branch, commit), and Tokenomics: every model call by every agent from the usage ledger in `~/.forgeqe/usage` (tokens, cache, cost, time, by agent, job, task, purpose, model, user and machine, never prompt text), the models compared, spend per day with a month forecast, the `budget:` limits from `forgeqe.yaml` (warnings at 80%, 90% and 100%, nothing stopped), and a calculator for what runs would cost at another size, cache rate or model. It exports Markdown and JSON and prints cleanly. See `samples/reports/nightly/20260929-020005/report.html` for an example. `report.screenshots: failures` keeps screenshots for failed runs only, `off` drops them; `report.accessibility: false` skips the page checks. On GitHub Actions the suite also writes a job summary.
 
 ### Static BI
 

package/docs/COST_ARCHITECTURE.md

@@ -0,0 +1,280 @@
+# Cost-aware execution: registry, tiers, and escalation
+
+Status: proposal · 2026-10-15
+
+ForgeQE pays the model for work it has already done. This document sets out how ForgeQE decides when the
+model is needed at all, and how much of it. The approach is built from published, measured techniques.
+The goal for known work: **a rerun of an existing test costs 0–5% of its first run**, and the model is
+paid only for what is new or broken.
+
+## 1. What a request costs today (measured)
+
+From the session logs in a live project (`.forgeqe/logs`, Azure OpenAI gpt-5.4):
+
+| Request | Agent calls | Input tokens | Output tokens | Cached |
+|---|---|---|---|---|
+| "Are we ready to release?" (one release-readiness tool call) | 12 | 577K | 2.6K | 518K |
+| Test-authoring session | many | 1.9M | 8.9K | 1.65M |
+
+These numbers show three things:
+
+1. **Fixed overhead per call.** Every chat-agent call resends about 45K tokens of system prompt, tool
+   definitions and history. Output is under 0.5% of tokens, so the cost is the context, not the work.
+2. **Calls multiply.** The deterministic release check took 12 calls, because the agent asked
+   questions, searched for files, read config and summarised.
+3. **Known work is not recognised:**
+   - Every request goes through the chat agent, including the **rerun** link.
+   - Page memory finds a saved flow only by the exact goal text plus the URL (`memory.py: goal_key`), so a
+     reworded goal misses it.
+   - `replay: verify` replays every step, then starts the browser agent to check the result.
+
+Prompt caching already works (about 90% of input is cached). It lowers the price per token, but not the
+number of tokens or calls, so on its own it cannot reach the target.
+
+## 2. The approaches this design uses
+
+| Technique | Evidence | Where it applies here |
+|---|---|---|
+| **Workflows before agents.** Use fixed code paths for predictable tasks; keep agents for open-ended ones. | Anthropic, [*Building effective agents*](https://www.anthropic.com/research/building-effective-agents) (2024) | Tier 0: rerun, run case, run suite, release check |
+| **Plan caching.** Extract a plan template from a finished run, match new requests by key, and adapt the template with a small model. | Zhang et al., [*Agentic Plan Caching*](https://arxiv.org/abs/2506.14852), NeurIPS 2025: 50.3% lower cost and 27.3% lower latency on average, performance maintained | Tier 1: known flow, new wording or new data |
+| **Workflow memory.** Turn successful trajectories into reusable routines. | Wang et al., [*Agent Workflow Memory*](https://arxiv.org/abs/2409.07429), ICML 2025: +51.1% relative success on WebArena, fewer steps | Flows saved from passing runs, shared across cases |
+| **Model cascade and routing.** Try the cheapest model or step that can answer, and escalate only when a check fails. | Chen et al., [*FrugalGPT*](https://arxiv.org/abs/2305.05176) (up to 98% savings, task-dependent); Ong et al., [*RouteLLM*](https://arxiv.org/abs/2406.18665), ICLR 2025 (over 2x savings at about 95% quality) | The router; result checks on the reviewer model |
+| **Content-addressed memoization.** Key cached work by a fingerprint of its inputs; any change to the inputs invalidates it. | Standard in build systems (Bazel, Gradle remote cache) and in test impact analysis | Registry fingerprints, the tool-result cache |
+| **Record–replay with self-healing locators.** | Already in ForgeQE (`replay.py`, ranked locators) | Tier 0 and Tier 2 execution |
+
+The published figures come from the authors' own benchmarks. ForgeQE has to measure its own (section 7).
+
+## 3. The execution ladder
+
+Every request is resolved to the **lowest tier that can do it**. A tier escalates only on a concrete
+failure signal, never on a guess.
+
+| Tier | When | Model use | Target cost vs. first run |
+|---|---|---|---|
+| **0: Deterministic** | The request resolves to a known entity, and replay and end-state checks pass | None. The reply is filled in from the result. | 0% |
+| **1: Adapt** | A known flow, but with new wording, new test data, or an end-state check that is unsure | One call to the small (reviewer) model with a short prompt (about 2K tokens) | ≤5% |
+| **2: Heal** | Replay stopped at step *k*, or a check failed | The agent starts at step *k* with the steps so far as context, not from the beginning | Proportional to what broke |
+| **3: Explore** | No match in the registry: a new goal or new kind of task | Full agent, as today | 100% |
+
+```
+request ──► Resolver (no model) ──hit──► Tier 0 run ──checks pass──► reply from template
+               │                            │ checks fail / replay stops
+               │ ambiguous                  ▼
+               ▼                         Tier 2 heal (agent from step k)
+         Router (small model, ~2K) ──known──► Tier 1 adapt
+               │ novel
+               ▼
+         Tier 3 full agent ──passes──► registry learns the flow, plan and checks
+```
+
+**Resolver (no model).** Matches IDs (`TC-123`, `SHOP-12`, plan/suite numbers, run folders,
+`last`), file and test names, and aliases from the registry. The **rerun** link and `/rerun` call it
+directly. Plain requests try it before any model is called.
+
+**Router (small model).** Runs only when the resolver is unsure. It sees the request and the top
+registry candidates (ranked by alias and keyword match), not the full tool catalogue. It returns
+`{entity, tier}` or `novel`.
+
+**Safety rules (non-negotiable):**
+- A Tier 0 pass needs evidence: replay complete **and** every recorded end-state check passing. Anything
+  else escalates.
+- **Freshness.** A flow is marked *stale* when the app changes: a new deploy or build id (when
+  configured), a changed page fingerprint seen during replay, or the age limit
+  (`registry.maxAgeDays`). A stale flow runs at Tier 1 once to re-confirm it.
+- **Canary.** Every *N*th Tier 0 run of an entity (default 10) runs at Tier 1, so a check that silently
+  passes is caught.
+- The user can always force a tier: `/rerun --verify`, `/rerun --fresh`.
+
+## 4. The registry: one indexed catalogue
+
+The registry is one SQLite file per project: `.forgeqe/registry/registry.db`.
+- Bun ships `bun:sqlite` and Python ships `sqlite3`, so there are no new dependencies.
+- It is local, and git-ignored by default (its folder carries its own `.gitignore`).
+- Existing JSON stores (page memory, code graph, quality history) stay where they are. The registry
+  **indexes and links** them, it does not replace them.
+
+### 4.1 Taxonomy
+
+Every row has a `kind`, and every kind has a single source of truth:
+
+| Kind | Source of truth | Example |
+|---|---|---|
+| `test_case` | ADO / Jira / Xray / TestRail / cases file | `ado:QI-Claims:TC-4812` |
+| `code_test` | Repo test code (code graph) | `PlaywrightPom.Web.Tests.LoginTests.ValidLogin` |
+| `flow` | Page memory `flows/` | recorded steps for a goal |
+| `page`, `element` | Page memory `pages/` | `/login`, `#username` |
+| `suite`, `plan` | ADO / cases file | `plan 12 / suite 34` |
+| `work_item` | ADO / Jira | story, bug, feature |
+| `pipeline`, `environment`, `login` | Config, ADO, saved sessions | `QI-Claims nightly`, `staging`, `saucedemo-standard` |
+| `run`, `report` | `.forgeqe/reports` | run folders |
+
+### 4.2 Tables
+
+```sql
+-- Everything ForgeQE knows about, categorised. fingerprint = hash of the inputs that define it.
+entities(id TEXT PK, kind TEXT, source TEXT, external_id TEXT, title TEXT,
+         path TEXT, fingerprint TEXT, version TEXT, updated_at TEXT)
+-- Every name a person might use for an entity: ids, titles, file and method names, normalised.
+aliases(alias TEXT, entity_id TEXT, weight REAL, PRIMARY KEY(alias, entity_id))
+-- Links between entities: case↔code_test, case↔flow, flow↔page, test↔page_object, story↔case.
+links(from_id TEXT, rel TEXT, to_id TEXT, PRIMARY KEY(from_id, rel, to_id))
+-- Reusable plans (the plan cache): what to call, with which arguments, how to judge the result.
+plans(id TEXT PK, entity_id TEXT, intent TEXT, tool TEXT, args_template TEXT,
+      checks TEXT, input_fingerprint TEXT, successes INT, failures INT,
+      last_success TEXT, stale INT DEFAULT 0)
+-- End-state checks recorded from a passing run: url, visible text, element, API status.
+checks(plan_id TEXT, kind TEXT, target TEXT, expected TEXT)
+-- The cost ledger: every request, its tier, and what it cost.
+runs(id TEXT PK, entity_id TEXT, plan_id TEXT, tier INT, outcome TEXT,
+     escalated_from INT, tokens_in INT, tokens_out INT, tokens_cached INT,
+     cost REAL, currency TEXT, ms INT, started_at TEXT)
+-- Memoised reads from external systems (ADO, Jira, GitHub), keyed by arguments and source version.
+tool_cache(tool TEXT, args_hash TEXT, source_version TEXT, result_path TEXT,
+           fetched_at TEXT, ttl_s INT, PRIMARY KEY(tool, args_hash, source_version))
+-- Pre-built context briefs per entity, rebuilt only when the fingerprint changes.
+context_packs(entity_id TEXT PK, fingerprint TEXT, brief TEXT, tokens INT, built_at TEXT)
+```
+
+### 4.3 Fingerprints and invalidation
+
+A cached item is valid only while its fingerprint matches its source:
+
+- **test_case:** hash of the normalised steps plus expected results, together with the ADO `rev`.
+  This replaces matching on goal text, so rewording a request no longer misses the flow.
+- **code_test:** git blob sha of the test file and of the page objects it reaches (from the code graph).
+- **flow:** hash of its steps, together with the page fingerprints seen on its last pass.
+- **tool_cache:** the source's own version (work item `rev`, sprint id, ETag) where one exists;
+  otherwise a short TTL. Writes always bypass it.
+
+## 5. Building context (for Tiers 1–3)
+
+The agent's fixed 45K-token context is the main cost driver. Context is built from the registry instead
+of being discovered by the agent:
+
+- **Context packs.** For each entity, a brief of a few hundred tokens: what it is, its linked case, code
+  test, flow, pages and last results. The agent gets the brief rather than spending calls on
+  `read`, `grep` and `code_outline`.
+- **Tool listing by tier:**
+  - Tier 1 gets one tool.
+  - Tier 2 gets the tools of the capability that broke.
+  - Tier 3 gets today's progressive listing (`routing.ts`).
+- **Stable prefix first, variable last**, so prompt caching keeps hitting.
+- **History compaction.** After a request finishes, only its outcome and ids are carried forward, not
+  its tool results.
+
+## 6. Learning: how the registry fills itself
+
+- **Indexing.**
+  - On `forgeqe init`, and incrementally on file change, the code graph and cases sources are indexed
+    into `entities`, `aliases` and `links`.
+  - ADO and Jira items are indexed when they are first fetched.
+- **Promotion.** A Tier 2 or Tier 3 run that passes, and passes the judge when it is on, writes or updates
+  its `plan` and `checks`. A plan needs two passes, or one pass plus user confirmation, before it is
+  allowed to run at Tier 0.
+- **Demotion.** Failures increment `failures`. Two failures in a row mark the plan *stale*, and the next
+  run goes to Tier 1 or 2.
+
+## 7. Measurement and acceptance
+
+The ledger (`runs`) makes cost measurable per tier and per entity. `forgeqe cost` reports the following
+metrics. Each must meet its target before a phase counts as done.
+
+| Metric | Target |
+|---|---|
+| Cost of a rerun of a known passing test, vs. its first run | ≤5% (Tier 0: 0 tokens) |
+| Share of reruns served at Tier 0 or Tier 1 | ≥90% |
+| Tier escalation rate on unchanged apps | ≤5% |
+| **False pass rate** (Tier 0 pass that a Tier 1 canary then fails) | 0 known; every one investigated |
+| Release check ("are we ready", scope known) | 0–1 agent calls |
+
+The evaluation is an offline replay suite under `src/evals`. It holds recorded requests (rerun, reworded
+rerun, changed app, new goal), each with an expected tier and outcome. It runs on every change to the
+resolver, router or registry, and live against the real model before release.
+
+## 8. Compatibility and rollout safety
+
+Nothing here may change what ForgeQE does today until it has been shown to give the same results.
+
+**Built alongside, never in place of**
+- `registry.mode: off | shadow | on` in `forgeqe.yaml`. It defaulted to `off` through phases 1 and 2,
+  and to `on` from 2026-10-15, after the live results in sections 9–10. With `off`, every request takes the
+  agent's path, unchanged.
+- The new code lives in `src/registry/`. Existing modules gain only small hooks at their entry points:
+  the chat request handler, the command parser and config. The chat agent, page memory, replay modes and
+  tools keep their behaviour.
+- Any registry error (missing, locked or corrupt database, an exception in the resolver) falls back to
+  today's path. The registry is a cache: deleting it loses savings, never behaviour.
+- Existing stores (page memory, code graph, quality history, run folders) keep their formats. The registry
+  reads and indexes them and never writes to them.
+- Tier 0 runs through the same `browser_run` tool the agent calls. Existing-test reuse, page memory
+  replay, the configured replay mode, reports and quality checks all apply as they do today.
+
+**Proven before it is switched on**
+- **Tests that lock in today's behaviour.** Written before the hooks, they pin down today's results:
+  a request with the registry `off` goes to the agent, and the **rerun** link goes to the agent.
+  They have to pass unchanged afterwards.
+- **Shadow mode.** The resolver decides and records what it would do (`runs` with `mode = shadow`), while
+  the request still takes today's path. The decisions are compared with real outcomes before `on`.
+- **False pass check.** During rollout, every Tier 0 pass is re-checked at Tier 1. Any disagreement blocks
+  the next phase.
+- **The full suite.** Every existing test passes unchanged in every phase, with no existing test edited
+  to make it pass.
+
+**Switched on gradually**
+- One phase per release. Each one can be turned off on its own.
+- A live check against the real model, Azure DevOps and Teams before each phase is called done.
+- Every reply says which tier answered it. `/rerun` is an explicit command, and the agent remains one
+  sentence away: anything the resolver is unsure of goes to it.
+
+## 9. Phase 1 live result (2026-10-15)
+
+Live run against Azure OpenAI gpt-5.4 and a real headless Chrome: a `/rerun` of a 27-step practice-form run
+whose first run used 242K tokens.
+
+| | Result |
+|---|---|
+| Resolver | Matched the run; no chat-agent calls |
+| Page memory | Replayed 26 of 27 steps without the model |
+| Step 27 | A read-only `evaluate` (reading the fields to check them) is not replayable, so the browser agent took over for 14 model steps |
+| Cost | **178K tokens: 74% of the first run.** The target is not met |
+
+The remaining cost is the browser agent's check after replay. Phase 2 is the step that reaches the target.
+Read-only steps (`evaluate`, extract) and the final check must be answered from recorded end-state checks
+(`run.json` already records `checks` with expected values). The agent comes in only when a check fails.
+
+## 10. Phase 2 live result (2026-10-15)
+
+What phase 2 adds, with `registry.mode: on` only:
+- `evaluate` steps whose script only reads the page are left out of a replay. The test is on the script's
+  source and conservative: any assignment, any page-changing call, any request or deferred code counts
+  as writing. Checked against every `evaluate` in a live project's flows: no write was classed as read-only.
+- A passing run's `record_check` results are kept in its flow, each with the step it was recorded after.
+- A replay finds each check on the page again after that step, with no model. If every check holds, the
+  run passes with no model. The first check that does not hold stops the replay there, and the agent
+  takes over on that page (today's path).
+- The agent still judges the run when: a check is dynamic, the flow has not passed for 30 days, or it is
+  every 10th replay of the flow (the canary).
+
+Live, Azure OpenAI gpt-5.4 and headless Chrome:
+
+| Flow | Run | Tokens | vs. first run |
+|---|---|---|---|
+| Wikipedia search (3 steps, 1 check) | first run | 50.3K | 100% |
+| | rerun: the search suggestion did not show in time, so the agent took 2 steps | 25.1K | 50% |
+| | rerun: replay complete, the check held | **0** | **0%** (6 s) |
+| | rerun with the saved check changed to text the page does not show | 25.1K | the replay refused to pass it, and the agent judged the real page |
+| Practice-form playground (27+ steps) | reruns | 122K, 135K, 82K | 50%, 56%, 34% |
+
+The practice page is built to resist replay: double-click targets, clipboard paste and per-key typing that
+replay's direct input does not reproduce. Its checks correctly never passed a replay that ended in a
+different state. Flows like that stay with the agent until replay can repeat those inputs (a later phase).
+
+## 11. Rollout
+
+| Phase | Scope | Done when |
+|---|---|---|
+| **1** | Registry (entities, aliases, plans, runs). Resolver plus Tier 0 for `/rerun`, the rerun link, and case and run ids. Fingerprint matching for test cases. Reply filled in from the result. | A rerun of a known passing case logs 0 tokens; a reworded request still finds its flow |
+| **2** | Recorded end-state checks replace the post-replay agent check. Canary and freshness rules. | `verify` default costs one small-model call or none; the false pass suite stays green |
+| **3** | Router (small model) plus Tier 1 adapt. Repo code tests run directly (`dotnet test` / `npx playwright test` with filters, results files parsed). | Reworded and new-data reruns at ≤5% |
+| **4** | Tool-result cache, context packs, history compaction. Release check as a fixed workflow. | The release check takes 0–1 agent calls; the agent's average context per call is down by half |
+| **5** | `forgeqe cost` report and the evaluation suite in CI | Metrics in section 7 reported per release |

package/docs/feature-test-prompts.md

@@ -0,0 +1,95 @@
+# ForgeQE feature test prompts
+
+One prompt for each ForgeQE feature, to check the agent end to end. Type each prompt in the ForgeQE terminal (`forgeqe start`), started in the test project's folder (for example `playwright-csharp-pom-framework`).
+
+## Before you start
+
+- Replace the `<…>` placeholders with your own ids (a story, a pull request).
+- ⚠ marks prompts that change something outside ForgeQE: Azure DevOps, Teams, git, or the repository. Run those on test data only.
+- Some prompts need an earlier run. #2, #17, #27, and #28 depend on #1 or #9, so run them after it.
+- Don't use test case 2287 for the API + UI check. As written, its step 2 gets a 403 (`POST /add_to_cart` is not how that site adds to the cart), and it signs in to a personal account. Use #26 instead.
+
+
+
+## Browser and test cases
+
+
+| #   | Feature                      | Prompt                                                                                                                                      | What to check                                                                                     |
+| --- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- |
+| 1   | Browser goals                | `Go to https://www.playwrightautomation.com, open the Practice tab, fill the Text Inputs with realistic values and check they are accepted` | Steps appear live; the run ends with a pass verdict and a report link                             |
+| 2   | Page memory                  | Run #1 again, then: `What does ForgeQE know about the Practice page?`                                                                       | The rerun replays from memory, faster and with fewer tokens; the answer lists locators            |
+| 3   | Saved logins                 | `Sign in to https://www.automationexercise.com/login with the test account and keep the session`                                            | The login is saved; the password never appears in the chat or the logs                            |
+| 4   | Run test cases               | `Run test case 2287 from QI-Claims and don't post results`                                                                                  | Each step has a verdict; report.html and junit.xml are written; nothing is posted to Azure DevOps |
+| 5   | Test case quality            | `Is test case 2287 in QI-Claims good enough to run? Check it against PBI 2286`                                                              | The card says "to review", not "not runnable"; the missing expected results are listed            |
+| 6   | ⚠ Write test cases           | `Write test cases for PBI 2286 in QI-Claims and show them before saving`                                                                    | Each case has expected results and links to PBI 2286; ForgeQE asks before saving                  |
+| 7   | Domain and regulatory design | `Write test cases for PBI 2286 covering PCI DSS obligations for card payments`                                                              | Cases cover card data and masking, and name the regulation                                        |
+| 8   | Exploratory testing          | `Explore PBI 2286 on automationexercise.com and find bugs beyond the test cases`                                                            | A charter, findings with evidence, and suggested bugs                                             |
+
+
+
+
+## Automated tests (Scribe)
+
+
+| #   | Feature                  | Prompt                                                    | What to check                                                                                                     |
+| --- | ------------------------ | --------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
+| 9   | ⚠ Turn a run into a test | `Write a Playwright test for the last passed run`         | A handoff; a C# test that uses the repository's page objects; ForgeQE runs the test; the reviewer gives a verdict |
+| 10  | ⚠ Heal failing tests     | `Heal the failing tests in tests/PlaywrightPom.Web.Tests` | Each failure is healed or reported as an application bug; no test is deleted or skipped                           |
+| 11  | Find and run tests       | `Run the Wikipedia tests in this repo`                    | The right `dotnet test --filter` command; a pass or fail for each test                                            |
+| 12  | Flaky tests              | `Which tests in this repo are flaky?`                     | Tests ranked, each with evidence (retries, mixed results)                                                         |
+
+
+
+
+## Pull requests
+
+
+| #   | Feature                   | Prompt                                                         | What to check                                                 |
+| --- | ------------------------- | -------------------------------------------------------------- | ------------------------------------------------------------- |
+| 13  | Pull request review       | `Review PR <n> in QI-Claims and focus on test coverage`        | Findings with file and line, and a verdict                    |
+| 14  | Impact and test selection | `What does PR <n> impact, and run only the tests it can break` | The test list comes from the code graph; only those tests run |
+
+
+
+
+## API and quality
+
+
+| #   | Feature                       | Prompt                                                                        | What to check                                                          |
+| --- | ----------------------------- | ----------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
+| 15  | API and contract tests        | `Turn the last run's API calls into contract tests and run them`              | Contracts are saved and run without a browser; a JUnit file is written |
+| 16  | Accessibility and performance | `Is the automationexercise.com checkout page accessible, and how fast is it?` | WCAG findings by severity; LCP, CLS, and INP figures                   |
+| 17  | Visual comparison             | Run #1 twice, then: `What changed visually since the last run?`               | Each page is marked same or changed, with diff images                  |
+
+
+
+
+## Release, operations, and systems
+
+
+| #   | Feature                | Prompt                                                                                      | What to check                                                              |
+| --- | ---------------------- | ------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- |
+| 18  | Defect intelligence    | `Which defects in QI-Claims keep coming back?`                                              | Recurring defects, with counts and areas                                   |
+| 19  | Release readiness      | `Are we ready to release QI-Claims? Give requirement coverage`                              | A coverage table and a GO, GO WITH RISKS, or NO-GO verdict                 |
+| 20  | ⚠ Scheduled monitoring | `Check the automationexercise.com login every 15 minutes and alert Teams if it fails`       | The monitor is scheduled and `/monitor` lists it; the alert text is clear  |
+| 21  | ⚠ Notifications        | `Post the last suite's results to the Teams QA channel`                                     | A message arrives in Teams, with a link to the report                      |
+| 22  | Reports                | `Write up the open defects in QI-Claims as a report I can share`                            | report.html and report.md, linked from the card                            |
+| 23  | ⚠ CI pipelines         | `Add an Azure Pipelines workflow that runs tests/cases.yaml on every PR, and make it green` | A pipeline file, the secrets it needs stored, a pushed branch (never main) |
+| 24  | Azure DevOps           | `List the last 3 pipeline runs in QI-Claims and QI-Payments and which failed`               | The agent's plan shows; calls appear in Activity; failure causes are given |
+| 25  | Setup help             | `What can you do here, and what is still not set up?`                                       | The features, each with what it still needs                                |
+
+
+
+
+## Recent additions
+
+
+| #   | Feature                        | Prompt                                                                                                                                                                                                                                           | What to check                                                                                                                                                |
+| --- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| 26  | API and UI steps in one case   | Run a case with these steps: `1. Open https://automationexercise.com/products 2. Call POST https://automationexercise.com/api/searchProduct with form body search_product=top, expect 200 3. Search "top" in the UI, expect "Searched Products"` | `api_request` is used and the server answers 200; workflow.yaml has an `api` step; the card says "API … saved as a reusable API scenario"; the replay passes |
+| 27  | Reuse or extend existing tests | Run #9's flow again with one extra check added                                                                                                                                                                                                   | ForgeQE offers reuse, extend, or new, with coverage evidence; no duplicate test file appears                                                                 |
+| 28  | Reviewer exchange              | Any Scribe job that needs two or more attempts (for example #9 on a new flow)                                                                                                                                                                    | "Reviewer asked for changes" lists each issue with its fix; "Scribe tries again" shows the real error; then the verdict                                      |
+| 29  | Activities panel               | During #24, press **ctrl+o**                                                                                                                                                                                                                     | Activity takes half the width: the plan with spinners, Now and Next, each call shown as it starts; each request has a header row and a result row            |
+| 30  | Refine                         | Type `run 2287 and write automation`, then press **ctrl+r**                                                                                                                                                                                      | The panel scrolls with ↑↓ and PgUp/PgDn; open questions come first; Use, Edit, and Keep stay visible                                                         |
+
+

package/docs/user-guide.html

@@ -378,7 +378,7 @@ forgeqe docs --copy ./docs   # copy the guide, the architecture page, and the re
 <h3 class="band-title">Every feature</h3>
 <div class="area"><a class="area-name" href="#area-browser"><span class="mono">Browser</span></a><ul class="tiles"><li><a class="tile" href="#cap-browser-goals"><b>Run any goal in a real browser</b><span>Go to booking.com and search hotels in Mumbai for next weekend</span></a></li><li><a class="tile" href="#cap-logins"><b>Saved logins for the application under test</b><span>Sign in to the admin site and keep the session</span></a></li></ul></div>
 <div class="area"><a class="area-name" href="#area-test-cases"><span class="mono">Test cases</span></a><ul class="tiles"><li><a class="tile" href="#cap-run-test-cases"><b>Run manual test cases in browsers</b><span>Run SHOP-12 and SHOP-14 from Jira, two browsers at once</span></a></li><li><a class="tile" href="#cap-test-case-quality"><b>Check and improve test cases</b><span>Are the cases in test plan 12 good enough to run?</span></a></li><li><a class="tile" href="#cap-write-test-cases"><b>Write test cases from a story</b><span>Write test cases for story 1234 and save them</span></a></li><li><a class="tile" href="#cap-domain-compliance"><b>Domain and regulatory test design</b><span>Write test cases for story 1234, covering our DORA and PSD2 obligations</span></a></li><li><a class="tile" href="#cap-exploratory-testing"><b>Exploratory testing of a story</b><span>Explore story 1234 and find bugs</span></a></li></ul></div>
-<div class="area"><a class="area-name" href="#area-automated-tests"><span class="mono">Automated tests</span></a><ul class="tiles"><li><a class="tile" href="#cap-write-tests"><b>Turn a passed run into an automated test</b><span>Write a Playwright test for that run in ../app</span></a></li><li><a class="tile" href="#cap-heal-tests"><b>Heal failing tests</b><span>Heal the failing tests in ../app</span></a></li><li><a class="tile" href="#cap-find-run-tests"><b>Find and run existing tests</b><span>Run the checkout tests</span></a></li><li><a class="tile" href="#cap-flaky-tests"><b>Flaky test detection and quarantine</b><span>Which tests are flaky?</span></a></li></ul></div>
+<div class="area"><a class="area-name" href="#area-automated-tests"><span class="mono">Automated tests</span></a><ul class="tiles"><li><a class="tile" href="#cap-write-tests"><b>Turn a passed run into an automated test</b><span>Write a Playwright test for that run in ../app</span></a></li><li><a class="tile" href="#cap-heal-tests"><b>Heal failing tests</b><span>Heal the failing tests in ../app</span></a></li><li><a class="tile" href="#cap-find-run-tests"><b>Find and run existing tests</b><span>Run the checkout tests</span></a></li><li><a class="tile" href="#cap-regression-packs"><b>Regression packs for a release</b><span>Build a regression pack for sprint 12 and run it on TestMu</span></a></li><li><a class="tile" href="#cap-flaky-tests"><b>Flaky test detection and quarantine</b><span>Which tests are flaky?</span></a></li></ul></div>
 <div class="area"><a class="area-name" href="#area-pull-requests"><span class="mono">Pull requests</span></a><ul class="tiles"><li><a class="tile" href="#cap-pr-review"><b>Review pull requests</b><span>Review PR 42 and focus on error handling</span></a></li><li><a class="tile" href="#cap-pr-impact"><b>Pull request impact and test selection</b><span>What does PR 42 impact?</span></a></li></ul></div>
 <div class="area"><a class="area-name" href="#area-api"><span class="mono">API</span></a><ul class="tiles"><li><a class="tile" href="#cap-api-testing"><b>API and contract testing</b><span>Turn the last run's API calls into tests</span></a></li></ul></div>
 <div class="area"><a class="area-name" href="#area-quality"><span class="mono">Quality</span></a><ul class="tiles"><li><a class="tile" href="#cap-accessibility-performance"><b>Accessibility and page performance</b><span>Is the checkout page accessible?</span></a></li><li><a class="tile" href="#cap-visual-comparison"><b>Visual comparison with baselines</b><span>What changed visually since the last run?</span></a></li><li><a class="tile" href="#cap-defect-intelligence"><b>Defect intelligence: trends and prediction</b><span>Which defects keep coming back?</span></a></li></ul></div>
@@ -1127,10 +1127,10 @@ forgeqe suite cases.yaml --no-code --no-write-back --out .forgeqe/reports/smoke
 </article>
 <article class="spec feat" id="cap-test-case-quality">
 <div class="hd"><span class="mono">Test cases</span><h3>Check and improve test cases</h3></div>
-<p class="ds">Checks test cases before they run (vague or missing expected results, missing test data, placeholders, duplicates) against the story, feature, and epic they test, then rewrites Azure DevOps cases once the user agrees.</p>
+<p class="ds">Checks test cases before they run (vague or missing expected results, missing test data, placeholders, duplicates) against the story, feature, and epic they test, then rewrites Azure DevOps or Jira cases in place once the user agrees.</p>
 <div class="facts">
 <div class="fact"><span class="mono">Ask it</span><ul class="asks"><li>Are the cases in test plan 12 good enough to run?</li><li>Fix the expected results of case 4512</li></ul></div>
-<div class="fact"><span class="mono">What it needs</span><ul class="needs"><li><span class="need opt">Optional</span><span>Azure DevOps, to rewrite cases in place: set up under sources: in forgeqe.yaml, or sign in with /auth</span></li></ul></div>
+<div class="fact"><span class="mono">What it needs</span><ul class="needs"><li><span class="need opt">Optional</span><span>Azure DevOps or Jira, to rewrite cases in place: set up under sources: in forgeqe.yaml, or sign in with /auth</span></li></ul></div>
 <div class="fact"><span class="mono">Tools</span><p class="tools"><code>check_test_cases</code> <code>update_test_case</code></p></div>
 </div>
 <div class="detail"><span class="mono">In detail</span>
@@ -1167,11 +1167,11 @@ forgeqe suite cases.yaml --no-code --no-write-back --out .forgeqe/reports/smoke
 </article>
 <article class="spec feat" id="cap-write-test-cases">
 <div class="hd"><span class="mono">Test cases</span><h3>Write test cases from a story</h3></div>
-<p class="ds">Writes test cases from a story's acceptance criteria and saves them in Azure DevOps or Jira, linked to the story.</p>
+<p class="ds">Writes test cases from a story's acceptance criteria and saves them in Azure DevOps or Jira, linked to the story. Reads the cases already linked first, so asking again adds only what is missing and updates cases in place instead of duplicating them; plans end-to-end journeys from the existing cases.</p>
 <div class="facts">
-<div class="fact"><span class="mono">Ask it</span><ul class="asks"><li>Write test cases for story 1234 and save them</li></ul></div>
+<div class="fact"><span class="mono">Ask it</span><ul class="asks"><li>Write test cases for story 1234 and save them</li><li>Build an end-to-end journey from story 1234's cases</li></ul></div>
 <div class="fact"><span class="mono">What it needs</span><ul class="needs"><li><span class="need req">Required</span><span>Azure DevOps or Jira: set up under sources: in forgeqe.yaml, or sign in with /auth</span></li></ul></div>
-<div class="fact"><span class="mono">Tools</span><p class="tools"><code>save_test_cases</code></p></div>
+<div class="fact"><span class="mono">Tools</span><p class="tools"><code>story_test_cases</code> <code>save_test_cases</code> <code>e2e_journey</code></p></div>
 </div>
 <div class="detail"><span class="mono">In detail</span>
 <p class="see">See <a href="#doc-how_to_use-where-cases-come-from">Where cases come from</a>.</p>
@@ -1502,6 +1502,53 @@ Run the tests added since 2026-09-01
 </section>
 </div>
 </article>
+<article class="spec feat" id="cap-regression-packs">
+<div class="hd"><span class="mono">Automated tests</span><h3>Regression packs for a release</h3></div>
+<p class="ds">Builds a regression pack from what goes into a release (an Azure DevOps sprint or work items, a commit range, pull requests, or changes read from any connected tool): the tests those changes can break, each with why, plus risk-area and smoke tests, and the changes no test covers. Runs it locally, on TestMu AI, or in a pipeline, and compares each run with the last.</p>
+<div class="facts">
+<div class="fact"><span class="mono">Ask it</span><ul class="asks"><li>Build a regression pack for sprint 12 and run it on TestMu</li><li>Which tests should we run for the PRs merged this week?</li></ul></div>
+<div class="fact"><span class="mono">What it needs</span><ul class="needs"><li><span class="need input">You give</span><span>a repository with tests</span></li><li><span class="need opt">Optional</span><span>Azure DevOps or Jira, to read a sprint's or release's work items: set up under sources: in forgeqe.yaml, or sign in with /auth</span></li></ul></div>
+<div class="fact"><span class="mono">Tools</span><p class="tools"><code>build_regression_pack</code> <code>run_regression_pack</code> <code>list_regression_packs</code></p></div>
+</div>
+<div class="detail"><span class="mono">In detail</span>
+<section class="doc" id="doc-how_to_use-regression-packs-for-a-release">
+<h4>Regression packs for a release</h4>
+<div class="prose"><p>A regression pack is the set of your repository's tests that a release's changes can break, saved under a name so you can run it again (here, on TestMu AI, or in a pipeline) and compare runs. Ask in your own words:</p>
+<pre><code class="language-text">Build a regression pack for sprint 12 and run it on TestMu
+Which tests should we run for the PRs merged this week?
+Build a pack for Jira fix version 2.4 and the commits since release/2.3 in ../shop-app
+</code></pre>
+<p><strong>Where the changes come from.</strong> Any mix of:</p>
+<ul>
+<li>an Azure DevOps sprint, area, epic or feature, or work item ids: their stories and bugs, the test cases each is tested by, and the files their linked pull requests changed;</li>
+<li>the commits between two refs of a repository (<code>from</code> and <code>to</code>): the product's repository, or the test repository's own; work items the commit messages name (<code>AB#123</code>, <code>#123</code>, <code>PAY-12</code>) are read too;</li>
+<li>pull requests by number or URL, or the ones merged in the last N days (Azure DevOps or GitHub);</li>
+<li>anything ForgeQE's agent can read through its connected tools (a Jira release, GitHub issues, a Confluence release note): it reads the items and passes them in with their text and links.</li>
+</ul>
+<p><strong>Why each test is in.</strong> A test can be in for more than one reason, and the card and <code>&lt;name&gt;.md</code> say each one:</p>
+<div class="tbl"><table>
+<thead>
+<tr><th>Reason</th><th>Meaning</th></tr>
+</thead>
+<tbody>
+<tr><td>linked</td><td>it automates a test case that tests a changed work item (<code>.forgeqe/test-cases.json</code>, or a case id in the test's attributes)</td></tr>
+<tr><td>test code changed</td><td>the change is in the test repository, in code the test uses</td></tr>
+<tr><td>matches a change</td><td>the work item's or pull request's words name what the test does: the elements it uses, its page, its name</td></tr>
+<tr><td>risk area</td><td>the release touches an area with defects or failures before, and the test works there</td></tr>
+<tr><td>smoke</td><td>the test is tagged smoke, so it is in every pack</td></tr>
+</tbody>
+</table></div>
+<p><strong>Gaps.</strong> Changes no test is in for are listed as <em>not covered</em>, each with the closest existing test and what it lacks, so you can extend that test rather than write a new one.</p>
+<p><strong>Running it.</strong> ForgeQE asks before it runs anything.</p>
+<ul>
+<li><em>This machine</em>: ForgeQE builds one command for exactly the pack's tests, the way the repository already runs a test (for .NET, a <code>--filter</code> with each test, the settings an earlier job used, TRX results), runs it, and reads each result. A failed test runs once more; one that then passes is flaky.</li>
+<li><em>TestMu AI</em>: the same command, with the variable <code>FORGEQE_PW_WS_ENDPOINT</code> set to TestMu AI's Playwright endpoint for the browser and platform in <code>testmu.yaml</code> (the access key goes in the variable, never in the command, and is removed from the output). The tests must connect to it when it is set: for NUnit, override <code>ConnectOptionsAsync</code> in the test base class; for Playwright Test, set <code>connectOptions</code> from it in <code>playwright.config</code>. ForgeQE refuses a TestMu run when the tests do not read the variable, so a run never looks remote while it ran locally.</li>
+</ul>
+<p>Each run is kept with the pack: the card says which tests fail now that passed in the last run, and which were fixed. Packs are saved in <code>.forgeqe/packs/&lt;name&gt;.json</code> and <code>&lt;name&gt;.md</code>; run output and results in <code>.forgeqe/reports/packs/&lt;name&gt;/&lt;time&gt;/</code>.</p>
+</div>
+</section>
+</div>
+</article>
 <article class="spec feat" id="cap-flaky-tests">
 <div class="hd"><span class="mono">Automated tests</span><h3>Flaky test detection and quarantine</h3></div>
 <p class="ds">Finds tests that flip between passing and failing in the repository's run history, with their likely cause, and quarantines them once the user agrees.</p>

package/forgeqe.example.yaml

@@ -75,6 +75,13 @@ memory:
   dir: ./.forgeqe/memory # pages, elements, locators, and recorded flows
   replay: verify # verify | trust | off
 
+# Cost-aware execution (docs/COST_ARCHITECTURE.md), on by default. on: a rerun of a passed run (the rerun link,
+# /rerun, "run it again", the same goal word for word) runs as recorded without the agent, and a replay whose
+# recorded checks all hold passes with no model; anything else, or any check that fails, goes to the agent.
+# shadow: only records what it would have done (.forgeqe/registry). off: every request goes to ForgeQE's agent.
+registry:
+  mode: on # on | shadow | off
+
 code:
   # repo: ../my-app          # where passed runs become tests (default: the current folder; --no-code turns it off)
   # test_command: npx playwright test   # the repo's test command, for forgeqe heal; default: the agent finds it
@@ -114,6 +121,14 @@ report:
   screenshots: all            # all | failures | off: step screenshots in report.html
   accessibility: true         # check each page the agent visits (names, labels, image text, contrast, lang, zoom)
 
+# Model spending budgets, in the price currency. They only warn (at 80%, 90% and 100%) and never stop a run.
+# Day and month (UTC) count every ForgeQE process on this machine, from the usage ledger in ~/.forgeqe/usage.
+# budget:
+#   run: 2          # one browser run
+#   day: 25
+#   month: 400
+#   # currency: USD # default: the currency of the model's price
+
 # Offline BI with Evidence: ForgeQE exports CSV datasets and runs Evidence (@evidence-dev/evidence 40.x, needs Node.js)
 # to build a static site, no always-on server. forgeqe bi init scaffolds the Evidence project, forgeqe bi sync refreshes
 # the data, forgeqe bi build does both and builds the site; without Node.js, or with --static, it writes ForgeQE's own

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@capagents/forgeqe",
-  "version": "1.2.1",
+  "version": "1.3.0",
   "description": "ForgeQE: a browser agent CLI that runs goals and test cases in local Chrome or Chromium, or BrowserStack and TestMu AI/LambdaTest when configured, and turns passing runs into tests in your repo",
   "license": "MIT",
   "type": "module",