@capagents/forgeqe 1.1.3 → 1.1.5

package/HOW_TO_USE.md

@@ -47,7 +47,7 @@ Everything ForgeQE can do, by area. ForgeQE's agent answers "can you…?" questi
 <!-- capabilities:start (generated from src/capabilities/registry.ts by bun scripts/gen-capabilities.ts; do not edit) -->
 | Area | Capability | Tools | |
 |------|------------|-------|-|
-| Browser | **Run any goal in a real browser**: Carries out a plain-language goal in a real browser one step at a time and verifies the outcome, with screenshots, a report, and the steps it took. The browser comes from forgeqe.yaml: local Chrome or Chromium by default, or BrowserStack or TestMu AI/LambdaTest when configured. Page memory replays steps that already worked on a site, so repeat runs are faster and cheaper. | `browser_run`, `run_details`, `page_memory` | [docs](#4-without-the-shell) |
+| Browser | **Run any goal in a real browser**: Carries out a plain-language goal in a real browser one step at a time and verifies the outcome, with screenshots, a report, and the steps it took. The browser comes from forgeqe.yaml: local Chrome or Chromium by default, or BrowserStack or TestMu AI/LambdaTest when configured. Page memory replays steps that already worked on a site, so repeat runs are faster and cheaper. | `browser_run`, `run_details`, `page_memory`, `browser_devices` | [docs](#4-without-the-shell) |
 | Browser | **Saved logins for the application under test**: Signs the browser agent in with saved credentials (including TOTP) and reuses the session; a user can also sign in by hand once in a browser. | `logins` | [docs](#signing-in) |
 | Test cases | **Run manual test cases in browsers**: Reads test cases from Azure DevOps, Jira, Xray, TestRail, Zephyr Scale, qTest, a file, or pasted text, runs each in a real browser with a verdict per step, writes report.html and junit.xml, posts results back, and files or updates bugs for failures. | `find_test_cases`, `run_test_cases` | [docs](#7-test-cases-and-suites) |
 | Test cases | **Check and improve test cases**: Checks test cases before they run (vague or missing expected results, missing test data, placeholders, duplicates) against the story, feature, and epic they test, then rewrites Azure DevOps cases once the user agrees. | `check_test_cases`, `update_test_case` | [docs](#checking-test-cases-before-they-run) |
@@ -63,12 +63,13 @@ Everything ForgeQE can do, by area. ForgeQE's agent answers "can you…?" questi
 | API | **API and contract testing**: Builds API contracts (status, schema, key fields) from the calls a browser run saw or from an API description (OpenAPI, Swagger, WSDL, GraphQL, .proto, gRPC reflection), and end-to-end API scenarios from runs, descriptions, test cases, or plain English. Runs them straight against the API without a browser, or writes them as tests in the repository. REST, GraphQL, SOAP, and gRPC. | `api_contracts`, `run_api_contracts`, `api_scenarios`, `run_api_scenarios`, `write_api_tests` | [docs](#api-and-contract-tests) |
 | Quality | **Accessibility and page performance**: Checks every page a run visits against WCAG rules (axe-core) and measures its Core Web Vitals, against budgets, with trends over time per page. | `quality_report`, `quality_trends` | [docs](#accessibility-and-performance-budgets) |
 | Quality | **Visual comparison with baselines**: Compares each page a run captures with its approved baseline and shows what changed; intended changes become the new baseline once the user agrees. | `visual_compare`, `visual_approve` | [docs](#visual-comparison) |
+| Quality | **Defect intelligence: trends and prediction**: Groups failures into defects, keeps their history across suites (new, known, gone, back again), and gives defect trends and where a change is likely to cause defects, with the cases to run first. | `defect_trends`, `predict_defects` | [docs](#bugs-for-failures) |
 | Release | **Requirement coverage and release readiness**: Gives requirement coverage and a GO / GO WITH RISKS / NO-GO verdict for a sprint, area, epic, or stories, from test cases, results, bugs, and flaky tests; can publish it to the wiki. | `release_readiness` | [docs](#requirement-coverage-and-release-readiness) |
 | Operations | **Scheduled monitoring with alerts**: Runs checks on a schedule and alerts Teams or email when one keeps failing, and again when it recovers. | `monitor` | [docs](#9-monitoring) |
 | Operations | **Teams and email notifications**: Tells the team in Microsoft Teams channels or by email when a suite fails or passes, when a release readiness verdict is in, and when the domain evaluation falls below its gate, with the failing cases, their bugs, and links to the report and the CI build. Each channel picks its events; the agent also posts a summary to a channel when asked. | `notify` | [docs](#17-notifications) |
 | Release | **Reports you can share**: Writes up what ForgeQE's agent found, such as release status across projects, open defects, or pipeline health, as a Markdown file and an HTML page in the reports folder, with a summary in the chat and links to open them. | `write_report` | [docs](#reports-from-the-agent) |
 | Operations | **CI pipelines**: Writes a GitHub Actions or Azure Pipelines workflow that runs ForgeQE, stores the secrets it needs, pushes a branch, and makes the pipeline green. | `push_branch`, `set_ci_secret` | [docs](#8-pipelines) |
-| Systems | **Work with Azure DevOps, Jira, and Confluence**: Reads anything in the configured systems (work items, issues, queries, boards, pages), makes changes the user asks for, and builds Azure DevOps dashboards and wiki pages. | `query_api`, `read_confluence`, `create_ado_dashboard`, `write_ado_wiki_page` | [docs](README.md#azure-devops-jira-and-other-systems) |
+| Systems | **Work with Azure DevOps, Jira, GitHub, and Confluence**: Reads anything in the configured systems (work items, issues, queries, boards, pages), makes changes the user asks for, puts run evidence (screenshots, reports) on work items and issues as attachments or comments, and builds Azure DevOps dashboards and wiki pages. | `query_api`, `read_confluence`, `create_ado_dashboard`, `write_ado_wiki_page`, `attach_to_work_item` | [docs](README.md#azure-devops-jira-and-other-systems) |
 | Setup | **Setup, skills, and documentation**: Shows the model profiles and MCP servers, installs or repairs the browser engine, creates a forgeqe.yaml, manages skills, and answers questions about ForgeQE from its documentation and this list. | `forgeqe_setup`, `capabilities` | [docs](#2-scaffold-config) |
 
 Not supported: Native mobile apps (iOS, Android) and desktop applications (instead: web applications in local Chrome or Chromium, or configured remote web grids such as BrowserStack and TestMu AI/LambdaTest); Load, stress, or soak testing with many simultaneous users (instead: per-page Core Web Vitals with budgets); Penetration testing or dynamic security scanning (instead: security findings in pull request reviews).
@@ -612,6 +613,7 @@ code:
   stall_after: 5            # stop the agent's turn when one command it runs has failed this many times
   replay: true              # before the agent starts, run the steps as a Playwright script and hand over what worked
   live_browser: true        # once a step fails, the agent looks at that page in a Playwright browser
+  graph: true               # the code graph: find page objects without reading files, and reuse what already locates an element
   # model: anthropic/claude-sonnet-4-5
   # runtime: /path/to/agent-runtime   # default: the runtime ForgeQE downloads
 ```
@@ -625,10 +627,12 @@ What happens after the run passes:
 
    Scribe first reads the repo (as a read-only agent: it cannot change files or use the live browser yet), while the replay runs when there is one, so the two overlap. What it learns (the framework, the command that runs one test, where page objects and config live, the conventions) is saved with page memory in `scribe-notes.json`, with the tests each job writes there. The next job in the same repo starts from those notes and that list instead of reading the repo again, and extends those tests' page objects when its flow overlaps; notes older than 30 days are read again.
 3. Scribe starts in the repo. It reads `AGENTS.md`, the test framework config, run scripts, page objects, fixtures, and helpers, and follows the imports between them. It writes the test in the repo's language and framework (a C# NUnit repo gets a C# NUnit test) and to fit its code: it extends an existing page object instead of duplicating it and reuses the login flow, test data, and config. A repo with no tests gets Playwright Test in TypeScript. It may change anything the test needs, including dependencies and config.
+
+   Before Scribe starts, ForgeQE reads the repo into a **code graph**, without a model: every class, page object, method, property, and test (TypeScript, JavaScript, C#, Java, Python), the calls between them, and the locators each one builds, in any framework's spelling (Playwright, Selenium, Cypress, WebdriverIO, Testing Library). It is kept in `~/.forgeqe/cache/code-graph`, nothing leaves the machine, and the next job reads again only the files that changed. The graph is tied to page memory, which knows each element by all its locators (its id, role and name, text, test id): so when the run clicked `getByRole('button', { name: 'Login' })` and the repo's `LoginPage.LoginButton` uses `#login-button`, the handoff says under that step "in this repository: LoginPage.LoginButton … used by LoginPage.SignInAsync", and Scribe calls the existing method instead of writing a second locator. Scribe also gets tools that answer from the graph instead of grep and whole-file reads: `code_find` (classes, methods, and tests by name, or by the text a locator uses), `code_outline` (a file's or class's signatures and locators, or the source of one member), `code_element` (which member already locates an element), `code_references` (what calls a member, before it is changed), and `code_impact` (the tests a change reaches, through flows and base classes, whatever the test files are called). `graph: false` turns it off. The handoff also outlines the page objects its steps use and includes the repo's test that uses the most of them, so Scribe takes the test conventions from that test instead of reading other tests, and reads a settings file only to change it.
 4. It writes the test from the handoff and the replay script: steps that worked are ported with their locators, steps that failed are fixed first. Every locator, including those for assertions, comes from the replay script, the page snapshots, page memory, or a check on the page; steps where the browser agent ran JavaScript to look around are left out. It does not walk the flow again or build tests to explore. Once a step fails (in the replay, or in its test), ForgeQE switches on its live browser: a Chrome that Playwright drives, set up like ForgeQE's, where it looks at the failing step's page one tool call per action (open, snapshot, click, type, wait, screenshot, run JavaScript, console errors, failed requests). Until then the live browser tools are off, so no tokens go into looking at pages that work. The live browser needs Node.js (`npx`), and Microsoft's Playwright MCP server is downloaded on first use; with `live_browser: false`, or without `npx`, it looks with a small throwaway script instead. When its browser gets a different page than ForgeQE's did (an error, a challenge, a control that will not open), it compares the two browsers (user agent, `navigator.webdriver`, headless, failed requests and CORS errors, headers the framework adds, console errors) and fixes the difference in the repo's browser configuration. It never judges the site with `curl` or `webfetch`.
 5. Scribe does not run the test itself: it may build or compile, then reports the test file and the command that runs only that test, and ForgeQE runs it. If one command it runs fails `stall_after` times in a row within its work (the same build again and again, say), ForgeQE stops that turn and sends the command and its last output back, asking it to change approach. If the same command stalls again, the job stops as a failure with that output in the notes.
 6. ForgeQE runs that command (with `CI=1`) and checks the exit code, and it watches the processes the command starts for a browser launched by Playwright, Puppeteer, Selenium, or Cypress. On a failure it sends the command, exit code, and output back to Scribe. The output is kept short so the error is never cut off: the page dump Playwright puts into a failed assertion ("Aria snapshot:", often thousands of lines) is taken out and saved as `<run>/test-runs/attempt-N-page.aria.yml`, then the start (where the first error and its stack trace are) and the end (the summary) are sent; the full output is in `<run>/test-runs/attempt-N.log`, and the prompt names both files. A command that exits 0 without starting a browser also goes back: the test must drive the app in a browser. If the repo's browser runs on another machine (Selenium Grid, a cloud service, a container), the agent says where and the check is skipped.
-7. A passing test then goes to a separate reviewer agent, which can read files and `git diff` but not change anything. It checks the change against the goal and the test rules: every part of the goal done through the page and asserted, generic page objects with data in the test, no values or URLs copied from the run, browser settings in the repo's configuration, no leftover files. The changed files come inline in its prompt, so it reads nothing again, and it does not ask to replace locators or texts that came from the replay or the checks, since those matched the live site. It judges only what this job changed: ForgeQE records the working tree before the job, so your uncommitted edits, your untracked files, and ForgeQE's own files (`.forgeqe/`, `forgeqe.yaml`, `llms.json`) are never part of the review. If it finds problems, they go back to Scribe as the next attempt.
+7. A passing test then goes to a separate reviewer agent, which can read files and `git diff` but not change anything. It checks the change against the goal and the test rules: every part of the goal done through the page and asserted, generic page objects with data in the test, no values or URLs copied from the run, browser settings in the repo's configuration, no leftover files. The changed files come inline in its prompt, so it reads nothing again, and it does not ask to replace locators or texts that came from the replay or the checks, since those matched the live site. It judges only what this job changed: ForgeQE records the working tree before the job, so your uncommitted edits, your untracked files, and ForgeQE's own files (`.forgeqe/`, `forgeqe.yaml`, `llms.json`) are never part of the review. It gets the outlines of the classes the changed code uses, the repo's browser settings, and the same `code_*` tools, so it checks reuse and settings without reading those files. The code graph also tells it what the job added that it can check in the code: a new member that locates an element an existing member already locates (to reuse instead), and new XPath or position-based locators. If it finds problems, they go back to Scribe as the next attempt.
 
    Scribe cannot edit ForgeQE's files or run folders, and cannot run `git clean`, `git reset --hard`, `git stash`, `git restore`, or `git checkout -- <path>`. If it still removes a file that existed before the job, or reverts your uncommitted edits, ForgeQE puts the file back after that attempt and tells the agent.
 8. The job is done when ForgeQE's run passes with a browser and the reviewer approves. It stops as a failure after `max_attempts` attempts, when the job passes `code.max_tokens`, when the reviewer rejects a passing test twice, if the review cannot finish, when the test fails the same way three times in a row (the same output apart from numbers such as timings and line numbers; the second time, Scribe is told its change did not change the failure and to look at the failing step before changing code), or straight away on an auth, API, or context-limit error from the model.
@@ -1109,9 +1113,28 @@ sources:
     labels: [forgeqe]      # Jira labels or Azure DevOps tags
     # area_path: Shop\Web   # Azure DevOps
     # on_error: true        # also for cases that could not run
+    # group_by: defect      # defect (default): cases failing the same way share a bug; case: a bug per case
 ```
 
-When results are posted, each case that failed with a functional defect (below) gets a bug: the title is `[ForgeQE] <case id>: <title>`, the description has the result, the steps with verdicts, the preconditions, where the run ended, the build link, and the run folder, and the last screenshot is attached. In Jira the bug is linked to the Jira or Xray test ("relates to"); in Azure DevOps it is "tested by" the test case work item. Each bug carries a label or tag that fingerprints the case (`forgeqe-` and ten hex digits), so the next failure comments "Still failing" on the open bug instead of filing a new one, and a pass adds a note that it passes again. Closed bugs are left alone; a new failure after that files a new bug.
+When results are posted, each case that failed with a functional defect (below) gets a bug: the title is `[ForgeQE] <case id>: <title>`, the description has the result, the steps with verdicts, the preconditions, where the run ended, the build link, and the run folder, and the last screenshot is attached. In Jira the bug is linked to the Jira or Xray test ("relates to"); in Azure DevOps it is "tested by" the test case work item. Each bug carries a label or tag that fingerprints the case (`forgeqe-` and ten hex digits), so the next failure comments "Still failing" on the open bug instead of filing a new one, and a pass adds a note that it passes again. A closed bug is not reopened: a new failure after that files a new bug, titled `[ForgeQE] Regression: …`, that says when the old one was closed and links to it (Jira "relates to", Azure DevOps "Related", a mention on GitHub; a GitHub issue closed as not planned does not count).
+
+**One bug per defect.** Cases that fail the same way share one bug (`group_by: defect`, the default). ForgeQE gives each failing case a signature from the strongest evidence its run recorded: a failed request (a 5xx before a 4xx) by its route and status, then an uncaught page error, then a console error, with ids, numbers and times taken out, so `POST /api/cart/123 → 500` and `/api/cart/456 → 500` match. Errors that the suite's passing runs show too (an analytics call, a missing favicon) are noise and do not count. The first case of a defect files the bug, titled after the defect (`[ForgeQE] shop.example.com/api/cart/:id returns 500`) and labelled with the defect's fingerprint (`forgeqe-d-` and ten hex digits) as well as the case's; every other case failing the same way is added as a comment and gets its own case label on that bug. The next run finds the open bug by either label. A failure with no browser evidence is matched by its page and what the agent saw, within one run only, and its bug is titled after the case as before. `group_by: case` files one bug per failing case. The report's **Defects** page, the terminal summary and the CI job summary list every defect with the cases it breaks.
+
+**Defect history.** Each suite adds its defects to `defect-history.json` next to the suite folders (keep it between CI runs as a cache, like the suite folders, to track a pipeline's defects). A defect is **new** the first time, then **known** ("seen in 3 earlier runs since 2 Oct"). It **went away** when a run did not see it while a case it broke passed; seen again after that, or matching a closed bug in the tracker, it is a **regression**. A defect whose cases keep changing between passing and failing across the last 20 suites (pass, fail, pass) is marked **intermittent**: it still gets a bug, which says so. When a new bug is filed, ForgeQE also searches the tracker once for open bugs people filed about the same route or error (`/api/cart`, `price is undefined`) and lists them on the bug as possibly related; it never merges them. Only defects found by browser evidence are kept in the history: a defect matched by the agent's words gets new words every run. The Defects page shows the history and filters new defects and regressions; the summaries say it next to each defect.
+
+**Reproducing and root cause.** Each defect gets a repro file, `defects/<defect id>.md` in the suite folder, and the same steps go in its bug: how reproducible it is, the case's steps with the agent's verdicts, what the browser did, where it ended, the environment (browser, device, build, commit), the failing request as a `curl` command with its response (the engine takes credentials out of the requests it records), and the run's files (screenshot, replay flow, every step, every API call). The first case of a new product defect runs once more into its `reproduce/` folder: failing the same way makes it **always** reproducible, not failing again makes it **intermittent**; a defect seen in earlier runs is not run again. Then the root cause, always as a hypothesis:
+
+- From the evidence, at no cost: the layer (the server, the front end, access, a route that is not there, the environment) and the component (the route or the script).
+- With `defects.product_repo` set: **suspect commits**, made since the defect's cases last passed, whose changes mention its route or error (`git log -G`), or whose files are named after it.
+- For new defects and regressions, a read-only agent (prompt `defect-rca`) reads the evidence, the suspect commits and the product's code, and writes a hypothesis with its confidence (high only when it read the code that fails and the change that broke it), what it rests on, and checks to confirm it. The hypothesis is added to the bug as a comment and shown on the Defects page. It runs for at most `max_analyses` defects per suite; each is one agent session (about 90K–120K tokens with gpt-5.4 in our runs), shown on its own line under Usage.
+
+```yaml
+defects:
+  product_repo: ../shop      # the application's repository: suspect commits, and the code the analysis reads
+  # reproduce: true          # run the first case of each new defect once more
+  # analyze: true            # a model writes a root-cause hypothesis for new defects and regressions
+  # max_analyses: 3          # most analyses per suite
+```
 
 
 ### Checking test cases before they run
@@ -1234,6 +1257,25 @@ forgeqe visual                       # the latest run's pages against their base
 forgeqe visual approve --page /home  # make this run's /home the baseline (asks first; --yes to skip)
 ```
 
+### Defect trends and prediction
+
+`forgeqe defects` (or "which defects keep coming back?") reads the suites saved in `suite.out_dir` and their `defect-history.json`, and writes `defect-trends.md` next to them: how many defects are still there, gone, or came back after a fix; the time they took to go away; the most seen ones with their bugs; defects by area (a page or an API route, by its path without ids); defects per suite; and how often predictions were right. The report's Defects page shows the same under **Across runs**.
+
+`forgeqe defects predict [--repo DIR] [--base REF] [--head REF]` (or "what can the changes since main break?") says where a change is likely to cause defects. It reads the change between the merge base of `base` (default: the remote's default branch, else `main` or `master`) and `head` in `defects.product_repo` (else the working directory). A change touches an area when it changes a file named after it (`server/cart.ts` for `/api/cart/:id`) or its diff mentions the route. Each touched area gets a score from what the saved suites know:
+
+| Signal | Score |
+|--------|-------|
+| The diff mentions the area's route | +3 |
+| A changed file is named after it | +2 |
+| Each earlier defect there (three at most) | +2 |
+| Defects there came back after a fix | +2 |
+| A defect there is still open | +1 |
+| Its cases failed in 20% (50%) of recent runs | +1 (+2) |
+
+6 or more is high, 3 or more medium. It lists the cases that ended on the high and medium areas as the ones to run first, and the changed product code that no case reaches. It is a heuristic, not a trained model: each prediction is saved in `defect-predictions.json`, and the first suite after it scores it (did its defects fall in the areas rated medium or high; did the high areas get one), so `forgeqe defects` gives its accuracy.
+
+Release readiness uses the defects too: a regression on a case in scope blocks the release (NO-GO), and a new defect with no bug is a risk.
+
 ### Requirement coverage and release readiness
 
 "Are we ready to release sprint 12?", "coverage of epic 300" (or `release_readiness`) builds the coverage matrix for an Azure DevOps scope: a sprint (`current` or a name), an area path, an epic or feature and every story under it, or story ids.

package/README.md

@@ -1,6 +1,6 @@
 # ForgeQE
 
-Browser agent CLI. Say what you want in plain language: a goal to carry out in a browser, test cases to run from Jira, Azure DevOps, Xray or a file, a test to write, or a CI pipeline to set up. ForgeQE's agent works out what that needs and calls ForgeQE's tools. The ForgeQE engine drives Chrome locally, or BrowserStack and TestMu AI/LambdaTest through their Playwright CDP endpoints, and a run that passes becomes a test in your repo's own language and framework, written by **Scribe**, ForgeQE's coding agent. Skills (`SKILL.md` folders, built in or in `.forgeqe/skills`) teach the agents your repo's conventions and the apps you test.
+Browser agent CLI. Say what you want in plain language: a goal to carry out in a browser, test cases to run from Jira, Azure DevOps, Xray or a file, a test to write, or a CI pipeline to set up. ForgeQE's agent works out what that needs and calls ForgeQE's tools. The ForgeQE engine drives Chrome locally, or BrowserStack and TestMu AI/LambdaTest through their Chrome DevTools Protocol (CDP) endpoints, and a run that passes becomes a test in your repo's own language and framework, written by **Scribe**, ForgeQE's coding agent. Skills (`SKILL.md` folders, built in or in `.forgeqe/skills`) teach the agents your repo's conventions and the apps you test.
 
 The engine is the browser service from test-agent-nexus, copied into this package (`py/forgeqe_engine`). It keeps the nexus behaviour: error-recovery system prompt, fast-mode agent tuning, Chrome launch flags, the search/select loop breaker, the 600 s run timeout, compact workflow YAML with semantic locators. There is no dependency on test-agent-nexus, FastAPI, or a database.
 
@@ -68,7 +68,7 @@ ForgeQE sends no telemetry and turns it off in everything it starts:
 
 An unchanged copy keeps following ForgeQE's updates; once you edit it, yours wins.
 
-For remote browsers, `forgeqe.yaml` can now point at provider-specific YAML files under `browser.providers` such as `browserstack` and `testmu`, so the BrowserStack or TestMu AI connection details do not have to live inline under `browser.remote`. `forgeqe init` drops sample files at `.forgeqe/browserstack.yaml` and `.forgeqe/testmu.yaml`. When both are present, the separate file supplies the provider defaults and the inline `browser.remote` values override only the active provider. ForgeQE's agent can also switch providers per request by passing `browser_provider` to `browser_run` or `run_test_cases`, and the CLI can pin one with `--browser-provider`.
+For remote browsers, `forgeqe.yaml` can now point at provider-specific YAML files under `browser.providers` such as `browserstack` and `testmu`, so the BrowserStack or TestMu AI connection details do not have to live inline under `browser.remote`. `forgeqe init` drops sample files at `.forgeqe/browserstack.yaml` and `.forgeqe/testmu.yaml`. When both are present, the separate file supplies the provider defaults and the inline `browser.remote` values override only the active provider. ForgeQE's agent can also switch providers per request by passing `browser_provider` to `browser_run` or `run_test_cases`, and the CLI can pin one with `--browser-provider`. **Devices.** A provider file can list named devices under `devices:`, each the file's shared `remote:` settings with its own browser, version, and platform on top. In the shell, describe the devices you want ("add Chrome on Windows 11 and Chrome 128 on Windows 10 on TestMu") and ForgeQE's agent saves them there with the `browser_devices` tool, keeping the file's comments; then "run SHOP-12 on both" runs the case on every device at the same time, each with its own run folder and row in the report (`SHOP-12@win11-chrome`). From the command line, `forgeqe suite <source> --devices win11-chrome,win10-edge` does the same, and `forgeqe start --devices ...` pins devices for a shell session. Devices run side by side, up to eight at once unless `--parallel` or `suite.parallel` says more; the grid queues sessions past your plan's limit. Each case posts one combined result back to its test system, failed when any device failed, and gets one test, written from its run on the first device. ForgeQE drives remote browsers over CDP, so `remote.browser` must be Chrome or Edge (TestMu AI's CDP endpoint serves Chrome only: an Edge session there never opens), and the default endpoints are the providers' CDP (`/puppeteer`) ones. On a remote grid, the browser stops taking real clicks and keystrokes on a tab once a password form has been submitted (most likely the browser's password manager), so after a sign-in leaves the login page ForgeQE reopens the page it landed on in a fresh tab, with the session intact, and the run log says so.
 
 ```bash
 forgeqe models
@@ -240,7 +240,7 @@ Every suite writes `report.html`, `junit.xml`, and `suite.json` to `.forgeqe/rep
 
 ### Static BI
 
-`forgeqe bi` adds an offline BI layer on top of those reports. It exports curated CSV datasets from ForgeQE's own suite history and release-readiness reports, plus optional Azure DevOps WIQL and Jira JQL datasets from `bi.datasets` in `forgeqe.yaml`, scaffolds an [Evidence](https://github.com/evidence-dev/evidence) project, and runs `evidence build` to produce static HTML. No backend service stays up between runs: the refresh happens during `forgeqe bi sync` or `forgeqe bi build`, and the output is a static site in the configured build directory.
+`forgeqe bi` adds an offline BI layer on top of those reports. It exports curated CSV datasets from ForgeQE's own suite history and release-readiness reports, plus optional Azure DevOps WIQL and Jira JQL datasets from `bi.datasets` in `forgeqe.yaml`, into an [Evidence](https://github.com/evidence-dev/evidence) project that reads them through its CSV source, then runs `evidence sources` and `evidence build` to produce a static site in the configured build directory. ForgeQE uses the open-source Evidence line (`@evidence-dev/evidence` 40.x, pinned in the generated `package.json`), because the newer Evidence Studio CLI has no local build and keeps data in its cloud. Building it needs Node.js 18 or later; serve the folder to view it (`npx serve`), since it cannot load its data when opened from disk. When Node.js is missing or the Evidence build fails, `forgeqe bi build` writes ForgeQE's own self-contained page from the same data instead, which opens straight from disk, and says why; `--static` always does that, and `--evidence` fails instead of falling back. Dataset ids become table names (`forgeqe.<id>`), so they use letters, digits, and `_`. Pages ForgeQE generates carry a "Managed by forgeqe bi" comment in their front matter; a page without it is yours and is never overwritten. Tables with no rows yet stay out of the Evidence project (Evidence cannot build over an empty table), and their sections read "None yet." No backend service stays up between runs.
 
 ```yaml
 bi:
@@ -260,8 +260,9 @@ bi:
 ```
 
 ```bash
-forgeqe bi init
-forgeqe bi build
+forgeqe bi init            # scaffold the Evidence project and install its packages
+forgeqe bi build           # refresh the data and build the site (ForgeQE's own page if Evidence cannot build)
+forgeqe bi build --static  # ForgeQE's own page only, no Node.js needed
 ```
 
 ## Signing in
@@ -290,6 +291,7 @@ ForgeQE's agent connects to each system's MCP server, configured from the same `
 | Azure DevOps Services | Microsoft's [Azure DevOps MCP server](https://github.com/microsoft/azure-devops-mcp) (`npx @azure-devops/mcp`), with all its tools: work items, boards, test plans, pipelines, repos, wiki pages, search, and security alerts | Node.js, and `ADO_PAT` or `az login` |
 | Jira Cloud, Server/Data Center | [mcp-atlassian](https://github.com/sooperset/mcp-atlassian) (`uvx`), with issues, JQL search, comments, and transitions. `mcp: rovo` uses [Atlassian's hosted server](https://github.com/atlassian/atlassian-mcp-server) instead (Cloud, and an admin must allow API tokens) | uv; `JIRA_EMAIL` and `JIRA_API_TOKEN` (Cloud) or a personal access token (Server/DC) |
 | Confluence | mcp-atlassian, shared with Jira (or its own when Jira isn't set up); Atlassian's hosted server covers it on the same site | `confluence: true` on a Jira Cloud site, or `confluence.url` + tokens |
+| GitHub | GitHub's [hosted MCP server](https://github.com/github/github-mcp-server) with the repos, issues, pull requests, and Actions toolsets (`sources.github.toolsets` changes them), on in a repository with a GitHub remote or when `sources.github` is set. GitHub issues are also a test case source (`github:#12`, `github:acme/shop#12,#13`, `github:label=login`, `github:q=login in:title`): results go back as issue comments, and `sources.bugs.system: github` files bugs as issues. GitHub has no API to upload a screenshot to an issue, so a bug names the run folder instead | `GITHUB_TOKEN`, or the GitHub CLI's sign-in (`forgeqe auth login github`) |
 | Xray, TestRail, Zephyr Scale, qTest | none; ForgeQE's API | `XRAY_CLIENT_ID` / `XRAY_CLIENT_SECRET`, `TESTRAIL_EMAIL` / `TESTRAIL_API_KEY`, `ZEPHYR_API_TOKEN`, `QTEST_TOKEN` |
 | Anything else | servers you add under `mcp:` | |
 

package/browserstack.example.yaml

@@ -1,9 +1,9 @@
 provider: browserstack
 remote:
-  browser: playwright-chromium
+  browser: chrome
   browser_version: latest
-  os: osx
-  os_version: sequoia
+  os: OS X
+  os_version: Sequoia
   username_env: BROWSERSTACK_USERNAME
   access_key_env: BROWSERSTACK_ACCESS_KEY
   project: ForgeQE
@@ -12,3 +12,15 @@ remote:
   local: false
   extra:
     browserstack.console: errors
+
+# Named devices: each is the remote settings above with its own on top. Run on them with --devices win11-chrome,mac-edge
+# (or ask ForgeQE's agent: it adds, changes, and removes devices here). Chrome or Edge only: ForgeQE drives the browser over CDP.
+# devices:
+#   win11-chrome:
+#     browser: chrome
+#     os: Windows
+#     os_version: "11"
+#   mac-edge:
+#     browser: edge
+#     os: OS X
+#     os_version: Sequoia

package/docs/user-guide.html

@@ -381,10 +381,10 @@ forgeqe docs --copy ./docs   # copy the guide, the architecture page, and the re
 <div class="area"><a class="area-name" href="#area-automated-tests"><span class="mono">Automated tests</span></a><ul class="tiles"><li><a class="tile" href="#cap-write-tests"><b>Turn a passed run into an automated test</b><span>Write a Playwright test for that run in ../app</span></a></li><li><a class="tile" href="#cap-heal-tests"><b>Heal failing tests</b><span>Heal the failing tests in ../app</span></a></li><li><a class="tile" href="#cap-find-run-tests"><b>Find and run existing tests</b><span>Run the checkout tests</span></a></li><li><a class="tile" href="#cap-flaky-tests"><b>Flaky test detection and quarantine</b><span>Which tests are flaky?</span></a></li></ul></div>
 <div class="area"><a class="area-name" href="#area-pull-requests"><span class="mono">Pull requests</span></a><ul class="tiles"><li><a class="tile" href="#cap-pr-review"><b>Review pull requests</b><span>Review PR 42 and focus on error handling</span></a></li><li><a class="tile" href="#cap-pr-impact"><b>Pull request impact and test selection</b><span>What does PR 42 impact?</span></a></li></ul></div>
 <div class="area"><a class="area-name" href="#area-api"><span class="mono">API</span></a><ul class="tiles"><li><a class="tile" href="#cap-api-testing"><b>API and contract testing</b><span>Turn the last run's API calls into tests</span></a></li></ul></div>
-<div class="area"><a class="area-name" href="#area-quality"><span class="mono">Quality</span></a><ul class="tiles"><li><a class="tile" href="#cap-accessibility-performance"><b>Accessibility and page performance</b><span>Is the checkout page accessible?</span></a></li><li><a class="tile" href="#cap-visual-comparison"><b>Visual comparison with baselines</b><span>What changed visually since the last run?</span></a></li></ul></div>
+<div class="area"><a class="area-name" href="#area-quality"><span class="mono">Quality</span></a><ul class="tiles"><li><a class="tile" href="#cap-accessibility-performance"><b>Accessibility and page performance</b><span>Is the checkout page accessible?</span></a></li><li><a class="tile" href="#cap-visual-comparison"><b>Visual comparison with baselines</b><span>What changed visually since the last run?</span></a></li><li><a class="tile" href="#cap-defect-intelligence"><b>Defect intelligence: trends and prediction</b><span>Which defects keep coming back?</span></a></li></ul></div>
 <div class="area"><a class="area-name" href="#area-release"><span class="mono">Release</span></a><ul class="tiles"><li><a class="tile" href="#cap-release-readiness"><b>Requirement coverage and release readiness</b><span>Are we ready to release?</span></a></li><li><a class="tile" href="#cap-reports"><b>Reports you can share</b><span>Create a release report for all projects</span></a></li></ul></div>
 <div class="area"><a class="area-name" href="#area-operations"><span class="mono">Operations</span></a><ul class="tiles"><li><a class="tile" href="#cap-monitoring"><b>Scheduled monitoring with alerts</b><span>Check the login flow every 15 minutes and alert Teams</span></a></li><li><a class="tile" href="#cap-notifications"><b>Teams and email notifications</b><span>Post the results to the QA channel</span></a></li><li><a class="tile" href="#cap-pipelines"><b>CI pipelines</b><span>Add a GitHub Actions workflow that runs tests/cases.yaml on every PR, and make it green</span></a></li></ul></div>
-<div class="area"><a class="area-name" href="#area-systems"><span class="mono">Systems</span></a><ul class="tiles"><li><a class="tile" href="#cap-systems"><b>Work with Azure DevOps, Jira, and Confluence</b><span>Show the open bugs in this sprint</span></a></li></ul></div>
+<div class="area"><a class="area-name" href="#area-systems"><span class="mono">Systems</span></a><ul class="tiles"><li><a class="tile" href="#cap-systems"><b>Work with Azure DevOps, Jira, GitHub, and Confluence</b><span>Show the open bugs in this sprint</span></a></li></ul></div>
 <div class="area"><a class="area-name" href="#area-setup"><span class="mono">Setup</span></a><ul class="tiles"><li><a class="tile" href="#cap-setup-help"><b>Setup, skills, and documentation</b><span>What can you do?</span></a></li></ul></div>
 <h3 class="band-title">Not supported</h3>
 <ul class="plainlist"><li><b>Native mobile apps (iOS, Android) and desktop applications</b>. Instead: web applications in local Chrome or Chromium, or configured remote web grids such as BrowserStack and TestMu AI/LambdaTest.</li><li><b>Load, stress, or soak testing with many simultaneous users</b>. Instead: per-page Core Web Vitals with budgets.</li><li><b>Penetration testing or dynamic security scanning</b>. Instead: security findings in pull request reviews.</li></ul>
@@ -803,10 +803,10 @@ forgeqe run &quot;Run the cases in smoke.yaml and don't post results&quot; --pla
 <div class="hd"><span class="mono">Browser</span><h3>Run any goal in a real browser</h3></div>
 <p class="ds">Carries out a plain-language goal in a real browser one step at a time and verifies the outcome, with screenshots, a report, and the steps it took. The browser comes from forgeqe.yaml: local Chrome or Chromium by default, or BrowserStack or TestMu AI/LambdaTest when configured. Page memory replays steps that already worked on a site, so repeat runs are faster and cheaper.</p>
 <div class="facts">
-<div class="fact"><span class="mono">Ask it</span><ul class="asks"><li>Go to booking.com and search hotels in Mumbai for next weekend</li><li>What does ForgeQE know about the checkout page?</li></ul></div>
+<div class="fact"><span class="mono">Ask it</span><ul class="asks"><li>Go to booking.com and search hotels in Mumbai for next weekend</li><li>What does ForgeQE know about the checkout page?</li><li>Add Chrome on Windows 11 and Chrome 128 on Windows 10 as TestMu devices, then run SHOP-12 on both</li></ul></div>
 <div class="fact"><span class="mono">What it needs</span><ul class="needs"><li><span class="need req">Required</span><span>A live model profile in llms.json (an API key or Microsoft Entra ID sign-in)</span></li></ul></div>
 <div class="fact"><span class="mono">Limits</span><ul class="plainlist"><li>Web applications only: local Chrome or Chromium, or configured remote web grids such as BrowserStack and TestMu AI/LambdaTest; not native mobile or desktop applications.</li></ul></div>
-<div class="fact"><span class="mono">Tools</span><p class="tools"><code>browser_run</code> <code>run_details</code> <code>page_memory</code></p></div>
+<div class="fact"><span class="mono">Tools</span><p class="tools"><code>browser_run</code> <code>run_details</code> <code>page_memory</code> <code>browser_devices</code></p></div>
 </div>
 <div class="detail"><span class="mono">In detail</span>
 <p class="see">See <a href="#doc-how_to_use-4-without-the-shell">4. Without the shell</a>.</p>
@@ -1074,8 +1074,23 @@ forgeqe suite cases.yaml --no-code --no-write-back --out .forgeqe/reports/smoke
     labels: [forgeqe]      # Jira labels or Azure DevOps tags
     # area_path: Shop\Web   # Azure DevOps
     # on_error: true        # also for cases that could not run
+    # group_by: defect      # defect (default): cases failing the same way share a bug; case: a bug per case
+</code></pre>
+<p>When results are posted, each case that failed with a functional defect (below) gets a bug: the title is <code>[ForgeQE] &lt;case id&gt;: &lt;title&gt;</code>, the description has the result, the steps with verdicts, the preconditions, where the run ended, the build link, and the run folder, and the last screenshot is attached. In Jira the bug is linked to the Jira or Xray test (&quot;relates to&quot;); in Azure DevOps it is &quot;tested by&quot; the test case work item. Each bug carries a label or tag that fingerprints the case (<code>forgeqe-</code> and ten hex digits), so the next failure comments &quot;Still failing&quot; on the open bug instead of filing a new one, and a pass adds a note that it passes again. A closed bug is not reopened: a new failure after that files a new bug, titled <code>[ForgeQE] Regression: …</code>, that says when the old one was closed and links to it (Jira &quot;relates to&quot;, Azure DevOps &quot;Related&quot;, a mention on GitHub; a GitHub issue closed as not planned does not count).</p>
+<p><strong>One bug per defect.</strong> Cases that fail the same way share one bug (<code>group_by: defect</code>, the default). ForgeQE gives each failing case a signature from the strongest evidence its run recorded: a failed request (a 5xx before a 4xx) by its route and status, then an uncaught page error, then a console error, with ids, numbers and times taken out, so <code>POST /api/cart/123 → 500</code> and <code>/api/cart/456 → 500</code> match. Errors that the suite's passing runs show too (an analytics call, a missing favicon) are noise and do not count. The first case of a defect files the bug, titled after the defect (<code>[ForgeQE] shop.example.com/api/cart/:id returns 500</code>) and labelled with the defect's fingerprint (<code>forgeqe-d-</code> and ten hex digits) as well as the case's; every other case failing the same way is added as a comment and gets its own case label on that bug. The next run finds the open bug by either label. A failure with no browser evidence is matched by its page and what the agent saw, within one run only, and its bug is titled after the case as before. <code>group_by: case</code> files one bug per failing case. The report's <strong>Defects</strong> page, the terminal summary and the CI job summary list every defect with the cases it breaks.</p>
+<p><strong>Defect history.</strong> Each suite adds its defects to <code>defect-history.json</code> next to the suite folders (keep it between CI runs as a cache, like the suite folders, to track a pipeline's defects). A defect is <strong>new</strong> the first time, then <strong>known</strong> (&quot;seen in 3 earlier runs since 2 Oct&quot;). It <strong>went away</strong> when a run did not see it while a case it broke passed; seen again after that, or matching a closed bug in the tracker, it is a <strong>regression</strong>. A defect whose cases keep changing between passing and failing across the last 20 suites (pass, fail, pass) is marked <strong>intermittent</strong>: it still gets a bug, which says so. When a new bug is filed, ForgeQE also searches the tracker once for open bugs people filed about the same route or error (<code>/api/cart</code>, <code>price is undefined</code>) and lists them on the bug as possibly related; it never merges them. Only defects found by browser evidence are kept in the history: a defect matched by the agent's words gets new words every run. The Defects page shows the history and filters new defects and regressions; the summaries say it next to each defect.</p>
+<p><strong>Reproducing and root cause.</strong> Each defect gets a repro file, <code>defects/&lt;defect id&gt;.md</code> in the suite folder, and the same steps go in its bug: how reproducible it is, the case's steps with the agent's verdicts, what the browser did, where it ended, the environment (browser, device, build, commit), the failing request as a <code>curl</code> command with its response (the engine takes credentials out of the requests it records), and the run's files (screenshot, replay flow, every step, every API call). The first case of a new product defect runs once more into its <code>reproduce/</code> folder: failing the same way makes it <strong>always</strong> reproducible, not failing again makes it <strong>intermittent</strong>; a defect seen in earlier runs is not run again. Then the root cause, always as a hypothesis:</p>
+<ul>
+<li>From the evidence, at no cost: the layer (the server, the front end, access, a route that is not there, the environment) and the component (the route or the script).</li>
+<li>With <code>defects.product_repo</code> set: <strong>suspect commits</strong>, made since the defect's cases last passed, whose changes mention its route or error (<code>git log -G</code>), or whose files are named after it.</li>
+<li>For new defects and regressions, a read-only agent (prompt <code>defect-rca</code>) reads the evidence, the suspect commits and the product's code, and writes a hypothesis with its confidence (high only when it read the code that fails and the change that broke it), what it rests on, and checks to confirm it. The hypothesis is added to the bug as a comment and shown on the Defects page. It runs for at most <code>max_analyses</code> defects per suite; each is one agent session (about 90K–120K tokens with gpt-5.4 in our runs), shown on its own line under Usage.</li>
+</ul>
+<pre><code class="language-yaml">defects:
+  product_repo: ../shop      # the application's repository: suspect commits, and the code the analysis reads
+  # reproduce: true          # run the first case of each new defect once more
+  # analyze: true            # a model writes a root-cause hypothesis for new defects and regressions
+  # max_analyses: 3          # most analyses per suite
 </code></pre>
-<p>When results are posted, each case that failed with a functional defect (below) gets a bug: the title is <code>[ForgeQE] &lt;case id&gt;: &lt;title&gt;</code>, the description has the result, the steps with verdicts, the preconditions, where the run ended, the build link, and the run folder, and the last screenshot is attached. In Jira the bug is linked to the Jira or Xray test (&quot;relates to&quot;); in Azure DevOps it is &quot;tested by&quot; the test case work item. Each bug carries a label or tag that fingerprints the case (<code>forgeqe-</code> and ten hex digits), so the next failure comments &quot;Still failing&quot; on the open bug instead of filing a new one, and a pass adds a note that it passes again. Closed bugs are left alone; a new failure after that files a new bug.</p>
 </div>
 </section>
 </div>
@@ -1256,6 +1271,7 @@ Acceptance criteria:
   stall_after: 5            # stop the agent's turn when one command it runs has failed this many times
   replay: true              # before the agent starts, run the steps as a Playwright script and hand over what worked
   live_browser: true        # once a step fails, the agent looks at that page in a Playwright browser
+  graph: true               # the code graph: find page objects without reading files, and reuse what already locates an element
   # model: anthropic/claude-sonnet-4-5
   # runtime: /path/to/agent-runtime   # default: the runtime ForgeQE downloads
 </code></pre>
@@ -1271,6 +1287,7 @@ Acceptance criteria:
 </li>
 <li>
 <p>Scribe starts in the repo. It reads <code>AGENTS.md</code>, the test framework config, run scripts, page objects, fixtures, and helpers, and follows the imports between them. It writes the test in the repo's language and framework (a C# NUnit repo gets a C# NUnit test) and to fit its code: it extends an existing page object instead of duplicating it and reuses the login flow, test data, and config. A repo with no tests gets Playwright Test in TypeScript. It may change anything the test needs, including dependencies and config.</p>
+<p>Before Scribe starts, ForgeQE reads the repo into a <strong>code graph</strong>, without a model: every class, page object, method, property, and test (TypeScript, JavaScript, C#, Java, Python), the calls between them, and the locators each one builds, in any framework's spelling (Playwright, Selenium, Cypress, WebdriverIO, Testing Library). It is kept in <code>~/.forgeqe/cache/code-graph</code>, nothing leaves the machine, and the next job reads again only the files that changed. The graph is tied to page memory, which knows each element by all its locators (its id, role and name, text, test id): so when the run clicked <code>getByRole('button', { name: 'Login' })</code> and the repo's <code>LoginPage.LoginButton</code> uses <code>#login-button</code>, the handoff says under that step &quot;in this repository: LoginPage.LoginButton … used by LoginPage.SignInAsync&quot;, and Scribe calls the existing method instead of writing a second locator. Scribe also gets tools that answer from the graph instead of grep and whole-file reads: <code>code_find</code> (classes, methods, and tests by name, or by the text a locator uses), <code>code_outline</code> (a file's or class's signatures and locators, or the source of one member), <code>code_element</code> (which member already locates an element), <code>code_references</code> (what calls a member, before it is changed), and <code>code_impact</code> (the tests a change reaches, through flows and base classes, whatever the test files are called). <code>graph: false</code> turns it off. The handoff also outlines the page objects its steps use and includes the repo's test that uses the most of them, so Scribe takes the test conventions from that test instead of reading other tests, and reads a settings file only to change it.</p>
 </li>
 <li>
 <p>It writes the test from the handoff and the replay script: steps that worked are ported with their locators, steps that failed are fixed first. Every locator, including those for assertions, comes from the replay script, the page snapshots, page memory, or a check on the page; steps where the browser agent ran JavaScript to look around are left out. It does not walk the flow again or build tests to explore. Once a step fails (in the replay, or in its test), ForgeQE switches on its live browser: a Chrome that Playwright drives, set up like ForgeQE's, where it looks at the failing step's page one tool call per action (open, snapshot, click, type, wait, screenshot, run JavaScript, console errors, failed requests). Until then the live browser tools are off, so no tokens go into looking at pages that work. The live browser needs Node.js (<code>npx</code>), and Microsoft's Playwright MCP server is downloaded on first use; with <code>live_browser: false</code>, or without <code>npx</code>, it looks with a small throwaway script instead. When its browser gets a different page than ForgeQE's did (an error, a challenge, a control that will not open), it compares the two browsers (user agent, <code>navigator.webdriver</code>, headless, failed requests and CORS errors, headers the framework adds, console errors) and fixes the difference in the repo's browser configuration. It never judges the site with <code>curl</code> or <code>webfetch</code>.</p>
@@ -1282,7 +1299,7 @@ Acceptance criteria:
 <p>ForgeQE runs that command (with <code>CI=1</code>) and checks the exit code, and it watches the processes the command starts for a browser launched by Playwright, Puppeteer, Selenium, or Cypress. On a failure it sends the command, exit code, and output back to Scribe. The output is kept short so the error is never cut off: the page dump Playwright puts into a failed assertion (&quot;Aria snapshot:&quot;, often thousands of lines) is taken out and saved as <code>&lt;run&gt;/test-runs/attempt-N-page.aria.yml</code>, then the start (where the first error and its stack trace are) and the end (the summary) are sent; the full output is in <code>&lt;run&gt;/test-runs/attempt-N.log</code>, and the prompt names both files. A command that exits 0 without starting a browser also goes back: the test must drive the app in a browser. If the repo's browser runs on another machine (Selenium Grid, a cloud service, a container), the agent says where and the check is skipped.</p>
 </li>
 <li>
-<p>A passing test then goes to a separate reviewer agent, which can read files and <code>git diff</code> but not change anything. It checks the change against the goal and the test rules: every part of the goal done through the page and asserted, generic page objects with data in the test, no values or URLs copied from the run, browser settings in the repo's configuration, no leftover files. The changed files come inline in its prompt, so it reads nothing again, and it does not ask to replace locators or texts that came from the replay or the checks, since those matched the live site. It judges only what this job changed: ForgeQE records the working tree before the job, so your uncommitted edits, your untracked files, and ForgeQE's own files (<code>.forgeqe/</code>, <code>forgeqe.yaml</code>, <code>llms.json</code>) are never part of the review. If it finds problems, they go back to Scribe as the next attempt.</p>
+<p>A passing test then goes to a separate reviewer agent, which can read files and <code>git diff</code> but not change anything. It checks the change against the goal and the test rules: every part of the goal done through the page and asserted, generic page objects with data in the test, no values or URLs copied from the run, browser settings in the repo's configuration, no leftover files. The changed files come inline in its prompt, so it reads nothing again, and it does not ask to replace locators or texts that came from the replay or the checks, since those matched the live site. It judges only what this job changed: ForgeQE records the working tree before the job, so your uncommitted edits, your untracked files, and ForgeQE's own files (<code>.forgeqe/</code>, <code>forgeqe.yaml</code>, <code>llms.json</code>) are never part of the review. It gets the outlines of the classes the changed code uses, the repo's browser settings, and the same <code>code_*</code> tools, so it checks reuse and settings without reading those files. The code graph also tells it what the job added that it can check in the code: a new member that locates an element an existing member already locates (to reuse instead), and new XPath or position-based locators. If it finds problems, they go back to Scribe as the next attempt.</p>
 <p>Scribe cannot edit ForgeQE's files or run folders, and cannot run <code>git clean</code>, <code>git reset --hard</code>, <code>git stash</code>, <code>git restore</code>, or <code>git checkout -- &lt;path&gt;</code>. If it still removes a file that existed before the job, or reverts your uncommitted edits, ForgeQE puts the file back after that attempt and tells the agent.</p>
 </li>
 <li>
@@ -1715,6 +1732,40 @@ forgeqe visual approve --page /home  # make this run's /home the baseline (asks
 </section>
 </div>
 </article>
+<article class="spec feat" id="cap-defect-intelligence">
+<div class="hd"><span class="mono">Quality</span><h3>Defect intelligence: trends and prediction</h3></div>
+<p class="ds">Groups failures into defects, keeps their history across suites (new, known, gone, back again), and gives defect trends and where a change is likely to cause defects, with the cases to run first.</p>
+<div class="facts">
+<div class="fact"><span class="mono">Ask it</span><ul class="asks"><li>Which defects keep coming back?</li><li>What can the changes since main break?</li></ul></div>
+<div class="fact"><span class="mono">What it needs</span><ul class="needs"><li><span class="need input">You give</span><span>suites run into one folder (suite.out_dir)</span></li><li><span class="need opt">Optional</span><span>git on PATH, for predictions from the product's repository</span></li></ul></div>
+<div class="fact"><span class="mono">Limits</span><ul class="plainlist"><li>Prediction is a heuristic from the saved suites and the change, not a trained model; its accuracy shows once suites have scored it.</li></ul></div>
+<div class="fact"><span class="mono">Tools</span><p class="tools"><code>defect_trends</code> <code>predict_defects</code></p></div>
+</div>
+<div class="detail"><span class="mono">In detail</span>
+<p class="see">See <a href="#doc-how_to_use-bugs-for-failures">Bugs for failures</a>.</p>
+<section class="doc" id="doc-how_to_use-defect-trends-and-prediction">
+<h4>Defect trends and prediction</h4>
+<div class="prose"><p><code>forgeqe defects</code> (or &quot;which defects keep coming back?&quot;) reads the suites saved in <code>suite.out_dir</code> and their <code>defect-history.json</code>, and writes <code>defect-trends.md</code> next to them: how many defects are still there, gone, or came back after a fix; the time they took to go away; the most seen ones with their bugs; defects by area (a page or an API route, by its path without ids); defects per suite; and how often predictions were right. The report's Defects page shows the same under <strong>Across runs</strong>.</p>
+<p><code>forgeqe defects predict [--repo DIR] [--base REF] [--head REF]</code> (or &quot;what can the changes since main break?&quot;) says where a change is likely to cause defects. It reads the change between the merge base of <code>base</code> (default: the remote's default branch, else <code>main</code> or <code>master</code>) and <code>head</code> in <code>defects.product_repo</code> (else the working directory). A change touches an area when it changes a file named after it (<code>server/cart.ts</code> for <code>/api/cart/:id</code>) or its diff mentions the route. Each touched area gets a score from what the saved suites know:</p>
+<div class="tbl"><table>
+<thead>
+<tr><th>Signal</th><th>Score</th></tr>
+</thead>
+<tbody>
+<tr><td>The diff mentions the area's route</td><td>+3</td></tr>
+<tr><td>A changed file is named after it</td><td>+2</td></tr>
+<tr><td>Each earlier defect there (three at most)</td><td>+2</td></tr>
+<tr><td>Defects there came back after a fix</td><td>+2</td></tr>
+<tr><td>A defect there is still open</td><td>+1</td></tr>
+<tr><td>Its cases failed in 20% (50%) of recent runs</td><td>+1 (+2)</td></tr>
+</tbody>
+</table></div>
+<p>6 or more is high, 3 or more medium. It lists the cases that ended on the high and medium areas as the ones to run first, and the changed product code that no case reaches. It is a heuristic, not a trained model: each prediction is saved in <code>defect-predictions.json</code>, and the first suite after it scores it (did its defects fall in the areas rated medium or high; did the high areas get one), so <code>forgeqe defects</code> gives its accuracy.</p>
+<p>Release readiness uses the defects too: a regression on a case in scope blocks the release (NO-GO), and a new defect with no bug is a risk.</p>
+</div>
+</section>
+</div>
+</article>
 </section>
 <section class="chapter" id="area-release" aria-labelledby="area-release-h">
 <header class="chead"><span class="mono">10</span><h2 id="area-release-h">Release</h2><p>Requirement coverage and a release verdict</p></header>
@@ -1927,12 +1978,12 @@ forgeqe run &quot;The ForgeQE workflow is failing, fix it&quot;
 <section class="chapter" id="area-systems" aria-labelledby="area-systems-h">
 <header class="chead"><span class="mono">12</span><h2 id="area-systems-h">Systems</h2><p>Azure DevOps, Jira, and Confluence</p></header>
 <article class="spec feat" id="cap-systems">
-<div class="hd"><span class="mono">Systems</span><h3>Work with Azure DevOps, Jira, and Confluence</h3></div>
-<p class="ds">Reads anything in the configured systems (work items, issues, queries, boards, pages), makes changes the user asks for, and builds Azure DevOps dashboards and wiki pages.</p>
+<div class="hd"><span class="mono">Systems</span><h3>Work with Azure DevOps, Jira, GitHub, and Confluence</h3></div>
+<p class="ds">Reads anything in the configured systems (work items, issues, queries, boards, pages), makes changes the user asks for, puts run evidence (screenshots, reports) on work items and issues as attachments or comments, and builds Azure DevOps dashboards and wiki pages.</p>
 <div class="facts">
-<div class="fact"><span class="mono">Ask it</span><ul class="asks"><li>Show the open bugs in this sprint</li><li>Read the checkout requirements from Confluence</li><li>Make a dashboard of this sprint's test results</li></ul></div>
+<div class="fact"><span class="mono">Ask it</span><ul class="asks"><li>Show the open bugs in this sprint</li><li>Read the checkout requirements from Confluence</li><li>Make a dashboard of this sprint's test results</li><li>Add the last screenshot of that run to ADO 256 as a comment</li><li>Run the GitHub issues labelled login as test cases</li></ul></div>
 <div class="fact"><span class="mono">What it needs</span><ul class="needs"><li><span class="need req">Required</span><span>Azure DevOps or Jira or Xray or Confluence: set up under sources: in forgeqe.yaml, or sign in with /auth</span></li></ul></div>
-<div class="fact"><span class="mono">Tools</span><p class="tools"><code>query_api</code> <code>read_confluence</code> <code>create_ado_dashboard</code> <code>write_ado_wiki_page</code></p></div>
+<div class="fact"><span class="mono">Tools</span><p class="tools"><code>query_api</code> <code>read_confluence</code> <code>create_ado_dashboard</code> <code>write_ado_wiki_page</code> <code>attach_to_work_item</code></p></div>
 </div>
 <div class="detail"><span class="mono">In detail</span>
 <section class="doc" id="doc-readme-azure-devops-jira-and-other-systems">
@@ -1946,6 +1997,7 @@ forgeqe run &quot;The ForgeQE workflow is failing, fix it&quot;
 <tr><td>Azure DevOps Services</td><td>Microsoft's <a href="https://github.com/microsoft/azure-devops-mcp" rel="noopener">Azure DevOps MCP server</a> (<code>npx @azure-devops/mcp</code>), with all its tools: work items, boards, test plans, pipelines, repos, wiki pages, search, and security alerts</td><td>Node.js, and <code>ADO_PAT</code> or <code>az login</code></td></tr>
 <tr><td>Jira Cloud, Server/Data Center</td><td><a href="https://github.com/sooperset/mcp-atlassian" rel="noopener">mcp-atlassian</a> (<code>uvx</code>), with issues, JQL search, comments, and transitions. <code>mcp: rovo</code> uses <a href="https://github.com/atlassian/atlassian-mcp-server" rel="noopener">Atlassian's hosted server</a> instead (Cloud, and an admin must allow API tokens)</td><td>uv; <code>JIRA_EMAIL</code> and <code>JIRA_API_TOKEN</code> (Cloud) or a personal access token (Server/DC)</td></tr>
 <tr><td>Confluence</td><td>mcp-atlassian, shared with Jira (or its own when Jira isn't set up); Atlassian's hosted server covers it on the same site</td><td><code>confluence: true</code> on a Jira Cloud site, or <code>confluence.url</code> + tokens</td></tr>
+<tr><td>GitHub</td><td>GitHub's <a href="https://github.com/github/github-mcp-server" rel="noopener">hosted MCP server</a> with the repos, issues, pull requests, and Actions toolsets (<code>sources.github.toolsets</code> changes them), on in a repository with a GitHub remote or when <code>sources.github</code> is set. GitHub issues are also a test case source (<code>github:#12</code>, <code>github:acme/shop#12,#13</code>, <code>github:label=login</code>, <code>github:q=login in:title</code>): results go back as issue comments, and <code>sources.bugs.system: github</code> files bugs as issues. GitHub has no API to upload a screenshot to an issue, so a bug names the run folder instead</td><td><code>GITHUB_TOKEN</code>, or the GitHub CLI's sign-in (<code>forgeqe auth login github</code>)</td></tr>
 <tr><td>Xray, TestRail, Zephyr Scale, qTest</td><td>none; ForgeQE's API</td><td><code>XRAY_CLIENT_ID</code> / <code>XRAY_CLIENT_SECRET</code>, <code>TESTRAIL_EMAIL</code> / <code>TESTRAIL_API_KEY</code>, <code>ZEPHYR_API_TOKEN</code>, <code>QTEST_TOKEN</code></td></tr>
 <tr><td>Anything else</td><td>servers you add under <code>mcp:</code></td><td></td></tr>
 </tbody>

package/forgeqe.example.yaml

@@ -20,10 +20,10 @@ browser:
   overlay: true # visible browser: ForgeQE start page, edge glow, a bar with the current step, your mouse and keyboard locked while the agent works
   # start_url: https://staging.example.com   # optional; omit to let the agent choose the site
   # remote:                    # inline settings for the selected remote provider; override the provider file when both are set
-  #   browser: playwright-chromium   # BrowserStack: playwright-chromium / playwright-firefox / playwright-webkit
+  #   browser: chrome   # BrowserStack: chrome or edge (ForgeQE drives the browser over CDP)
   #   browser_version: latest
-  #   os: osx
-  #   os_version: sequoia
+  #   os: OS X                  # BrowserStack: Windows or OS X
+  #   os_version: Sequoia       # 11, 10, Sequoia, Sonoma, ...
   #   # platform: Windows 11   # TestMu AI prefers platform, or set os/os_version and ForgeQE joins them
   #   username_env: BROWSERSTACK_USERNAME   # TestMu AI default: LT_USERNAME
   #   access_key_env: BROWSERSTACK_ACCESS_KEY   # TestMu AI default: LT_ACCESS_KEY
@@ -75,14 +75,6 @@ memory:
   dir: ./.forgeqe/memory # pages, elements, locators, and recorded flows
   replay: verify # verify | trust | off
 
-# Redis cache for deterministic external data fetches, such as BI datasets. Live browser runs still execute.
-# cache:
-#   redis_url: redis://127.0.0.1:6379
-#   key_prefix: forgeqe
-#   default_ttl_seconds: 3600
-#   bi_ttl_seconds: 900
-#   connect_timeout_ms: 3000
-
 code:
   # repo: ../my-app          # where passed runs become tests (default: the current folder; --no-code turns it off)
   # test_command: npx playwright test   # the repo's test command, for forgeqe heal; default: the agent finds it
@@ -93,6 +85,7 @@ code:
   stall_after: 5             # stop the coding agent when one command it runs has failed this many times
   replay: true               # before the coding agent starts, run the steps as a Playwright script and hand over what worked and where it failed
   live_browser: true         # once a step fails, the coding agent looks at that page in a Playwright browser instead of building tests to look
+  graph: true                # read the repo's code into a graph (classes, members, tests, locators) and link the handoff's elements to the page objects that already locate them
   # model: anthropic/claude-sonnet-4-5   # model for ForgeQE's agent and Scribe; default: the active llms.json profile
   # runtime: /path/to/agent-runtime      # default: the runtime ForgeQE downloads
 
@@ -121,13 +114,15 @@ report:
   screenshots: all            # all | failures | off: step screenshots in report.html
   accessibility: true         # check each page the agent visits (names, labels, image text, contrast, lang, zoom)
 
-# Offline BI with Evidence: ForgeQE exports CSV datasets and runs Evidence to build static HTML, no always-on server.
-# forgeqe bi init scaffolds the Evidence project, forgeqe bi sync refreshes the data, forgeqe bi build does both
-# and runs evidence build. The datasets below are optional: local ForgeQE suite/readiness data exports either way.
+# Offline BI with Evidence: ForgeQE exports CSV datasets and runs Evidence (@evidence-dev/evidence 40.x, needs Node.js)
+# to build a static site, no always-on server. forgeqe bi init scaffolds the Evidence project, forgeqe bi sync refreshes
+# the data, forgeqe bi build does both and builds the site; without Node.js, or with --static, it writes ForgeQE's own
+# page instead. The datasets below are optional: local ForgeQE suite/readiness data exports either way.
+# Dataset ids are table names (forgeqe.<id>): letters, digits, and _.
 # bi:
 #   title: ForgeQE BI
 #   # evidence_dir: ./evidence                    # the managed Evidence project
-#   # data_dir: ./evidence/sources/forgeqe/data  # CSV exports that the Evidence project reads
+#   # data_dir: ./evidence/sources/forgeqe       # CSV exports; copied there when set elsewhere
 #   # build_dir: ./.forgeqe/reports/bi           # static HTML output from evidence build
 #   datasets:
 #     ado_current_sprint:
@@ -183,13 +178,25 @@ sources:
   #   token_env: QTEST_TOKEN                # API bearer token
   #   project_id: 12345
   #   statuses: { passed: PASS, failed: FAIL, error: BLOCKED }   # automation status names in your qTest
-  # bugs:                                   # a bug per failed case; still failing: a comment on the open bug
-  #   system: jira                          # jira | azure_devops
-  #   project: SHOP                         # Jira project key (Azure DevOps uses azure_devops.project)
+  # github:                                 # GitHub issues as test cases (github:#12, github:label=login) and bugs;
+  #   repo: acme/shop                       # also turns on GitHub's MCP server (on anyway in a repo with a GitHub remote)
+  #   token_env: GITHUB_TOKEN               # default; without it, the GitHub CLI's sign-in (forgeqe auth login github)
+  #   toolsets: [repos, issues, pull_requests, actions]   # the MCP server's tool groups; mcp: false turns it off
+  # bugs:                                   # a bug per defect; still failing: a comment on the open bug
+  #   system: jira                          # jira | azure_devops | github
+  #   project: SHOP                         # Jira project key; GitHub owner/repo (Azure DevOps uses azure_devops.project)
   #   type: Bug                             # issue / work item type
   #   labels: [forgeqe]                    # Jira labels or Azure DevOps tags
   #   # area_path: Shop\\Web                # Azure DevOps
   #   # on_error: false                     # also for cases that could not run
+  #   # group_by: defect                    # defect: cases failing the same way share a bug; case: a bug per case
+  #                                         # a defect back after its bug closed gets a new "Regression:" bug linked to it
+
+# defects:                     # defect intelligence: repro files, suspect commits, root-cause hypotheses
+#   product_repo: ../shop      # the application's repository: suspect commits, and the code the analysis reads
+#   reproduce: true            # run the first case of each new defect once more, to see whether it repeats
+#   analyze: true              # a model writes a root-cause hypothesis for new defects and regressions (one session each)
+#   max_analyses: 3            # most analyses per suite
 
 # More MCP servers for ForgeQE's agent. Use {env:NAME} for secrets; never put values here.
 # mcp:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@capagents/forgeqe",
-  "version": "1.1.3",
+  "version": "1.1.5",
   "description": "ForgeQE: a browser agent CLI that runs goals and test cases in local Chrome or Chromium, or BrowserStack and TestMu AI/LambdaTest when configured, and turns passing runs into tests in your repo",
   "license": "MIT",
   "type": "module",
@@ -49,13 +49,13 @@
     "@opencode-ai/sdk": "1.18.33",
     "@opentui/core": "^0.5.12",
     "@opentui/react": "^0.5.12",
+    "@vscode/tree-sitter-wasm": "0.3.1",
     "axe-core": "4.14.0",
     "fast-xml-parser": "^5.11.1",
     "nodemailer": "^10.0.12",
     "pixelmatch": "8.0.0",
     "pngjs": "7.0.0",
     "react": "^19.3.0",
-    "redis": "^4.7.1",
     "yaml": "^2.9.1"
   },
   "devDependencies": {

package/prompts/README.md

@@ -15,6 +15,7 @@ project; leave it as it is to keep getting ForgeQE's improvements when you updat
 | api-tests.md | Writes API and contract tests, and end-to-end API scenarios |
 | api-scenarios.md | Plans API scenarios from test cases or plain English |
 | explore-charters.md | Plans exploratory charters for a story |
+| defect-rca.md | Writes a root-cause hypothesis for a defect from its evidence and the product's code |
 | eval-judge.md | Scores the agent's test cases in `forgeqe eval domain` |
 | refine.md | Refine (ctrl+r) in the shell: turns what you typed into a task specification (Role, Objective, Context, Steps, Rules, Output format, Summary), checked against what ForgeQE knows |
 | conventions.md | How every prompt states its rules: the RFC 2119 key words (included at the top of the agent and structured prompts) |

package/prompts/brain.md

@@ -25,13 +25,15 @@ A message can also be a task specification with the parts Role, Objective, Conte
 
 **Tools** (ForgeQE tools; the client adds the forgeqe_ prefix):
 - browser_run: does one goal in a real browser, and verifies it. Use it for all work on a website. Set purpose to test for a flow of the application under test. Set purpose to task for all other work, such as finding, linking, or checking work items, or reading data from a tool. A task never becomes an automated test.
-- find_test_cases: reads test cases from Azure DevOps, Jira, Xray, TestRail, Zephyr Scale, qTest, files, or text. It does not run them.
+- find_test_cases: reads test cases from Azure DevOps, Jira, Xray, TestRail, Zephyr Scale, qTest, GitHub issues, files, or text. It does not run them.
 - save_test_cases: saves test cases that you wrote from a story into Azure DevOps (Test Case work items) or Jira, linked to the story. It gives a link for each case.
 - check_test_cases: checks test cases before they run. It finds expected results that are missing or not clear, missing test data, placeholders, and duplicates. It runs nothing.
 - update_test_case: writes the steps of an Azure DevOps Test Case again, after the user agrees. It asks the user.
 - release_readiness: gives requirement coverage and a GO / GO WITH RISKS / NO-GO verdict for a sprint, area, epic, or stories. It uses test cases, their results, bugs, and flaky tests. It can publish to the wiki. Examples: "are we ready to release?", "coverage of sprint 12".
 - run_test_cases: runs test cases in real browsers, with a verdict for each step. It writes report.html (steps, screenshots, timings) and junit.xml. It sends results back to the source system. It files or updates bugs for failures when bugs are set up. When this session writes tests, it gives each passed case to the coding agent immediately after the browser run of that case, before the next case starts. It gives the test of each case.
 - read_confluence: reads Confluence pages (requirements, specifications) as text, by id, link, or CQL.
+- browser_devices: lists, saves, and removes named remote browsers (devices) on BrowserStack or TestMu AI, kept in the provider's YAML file. When the user describes devices or browsers to test on, save each one, then pass their names as devices to browser_run or run_test_cases: every device runs at the same time and is reported on its own.
+- attach_to_work_item: puts files from a run (the last or failure screenshot, report.html, junit.xml, trace.md) on an existing Azure DevOps work item or Jira issue, as attachments, in a comment (images shown inline), or both. The Azure DevOps MCP server cannot upload files, so use this tool for any file on a work item.
 - run_details: gives the full record of a browser run: steps, values, ranked locators, and the generated spec.
 - run_test: ForgeQE runs a test command. A test passes only when this tool, write_test, or heal_tests says so.
 - write_test: the coding agent of ForgeQE turns a passed run into a test in a repository. ForgeQE runs the test until it passes.
@@ -39,6 +41,8 @@ A message can also be a task specification with the parts Role, Objective, Conte
 - find_tests: Scribe, the coding agent, finds the automated tests in a repository that match a request. A request can be a feature, tags or ids, a folder, or tests added in a date range or sprint. ForgeQE confirms the selection with the list command of the framework. It runs nothing.
 - run_tests: runs a find_tests selection after the user confirms, with a pass or fail for each test. A test that fails runs one more time. A test that then passes is flaky, not failing. The results of each run go into the test history of the repository.
 - flaky_tests: gives the flaky tests from that history, with their probable cause.
+- defect_trends: gives how the defects of the saved suites went over time: still there, gone, or back after a fix, the time to fix, the most seen, and defects by area. It runs nothing.
+- predict_defects: gives the areas where a change in the product repository can cause defects, with the reasons and the cases to run first. It is a heuristic from the history. It runs nothing.
 - quarantine_tests: puts flaky tests in quarantine, or takes them out, after the user agrees. The failures of a test in quarantine do not fail runs or CI.
 - review_pull_request: Scribe reviews an Azure DevOps or GitHub pull request from the local clone (bugs, security, requirements, tests). The user sees the findings. It posts nothing.
 - post_pr_review: posts a review_pull_request review to the pull request after the user confirms.
@@ -240,6 +244,8 @@ For "run the login tests", "run the tests added this sprint", or "which tests co
 - After a run, you MUST name the flaky tests (passed only when they ran again), the tests in quarantine that failed, and each new flaky test with its probable cause.
 - You SHOULD then offer quarantine_tests, and heal_tests for the failures, with the run command.
 - For "Which tests are flaky?", you MUST use flaky_tests. When no test matches, say what you searched.
+- For "Which defects keep coming back?" or "how are our defects trending?", you MUST use defect_trends.
+- For "What can this change break?" or "where is the risk in this release?", you SHOULD use predict_defects. You MUST say that it is a heuristic, and give its accuracy from defect_trends when it has one.
 
 ### Pull requests
 For "review PR 42", a pull request link, or "review my open PRs":