testaro 78.0.8 → 78.4.0

package/validation/act/isolation-notes.md

@@ -0,0 +1,159 @@
+# Long-lived-browser injection failure — evidence log (2026-08-22)
+
+Motivation beyond this harness: production testaro launches a fresh browser per test act because
+of recurring spurious test-isolation bugs, at a large throughput cost. Anything learned here about
+what long-lived-browser state actually breaks — and what level of recycling (context vs browser)
+resets it — bears on whether that cost can be reduced.
+
+## The incident
+
+Full ACT capture (1,213 testcases × pour + axe, one Chromium browser + one context for the whole
+run, fresh page per testcase × engine, pages closed after use). From testcase 274 — exactly the
+first fixture of rule `7d6734`, immediately after the last `c487ae` fixture — **every** pour row
+for the remainder of the run (940 rows) failed the same way: script element inserted, no
+exception, `pourEngine` global never defined. Axe, whose adapter executes via `page.evaluate`
+instead of a `<script>` tag, kept detecting instances to the final row. No recovery for ~940
+consecutive pages over ~13 minutes.
+
+## Experiments
+
+| # | Setup | Result |
+| --- | --- | --- |
+| 1 | Fresh process, the boundary fixtures (`7d6734`, `afw4f7`) | All pass — fixtures innocent in isolation |
+| 2 | 700 cycles, local benign fixtures, one browser, axe-style page interleaved (data-xpath stamping + axe evaluate), no recycling | 0 failures, flat RSS — page count + interleave alone insufficient |
+| 3 | Full-feed pour re-capture, browser recycled every 100 testcases, no axe interleave | Clean through the entire former cliff region (34/34 on `7d6734`/`307n5z`/`a25f45`) |
+| 4 | Cliff-neighborhood replay (`c487ae`→`a25f45`, 69 testcases), pour + axe interleaved, **no recycling**, fresh process | 69/69 clean — the fixture region + interleave is not a deterministic trigger |
+| 5 | Full-feed pour-only recapture, hardened harness, solo machine, recycling every 100 | **Cliff recurred**: 41 consecutive rows dead starting mid-`1a02b0` (video-transcript fixtures) — in a browser only ~36 pages old. Rules out cumulative-pages-per-browser; the concurrent-load theory weakens too (run was solo). |
+| 6 | Meta-refresh(0) fixtures, local repro (`repro-metarefresh.js`): goto → evaluate (interrupted by the refresh navigation) → `page.close()` | **`page.close()` hangs indefinitely** — never resolves, never rejects — on ~half the rounds, on both a single-redirect page and an a→b→a redirect loop. Playwright 1.62.1, Chromium headless. The context stays usable for *new* pages afterward. Independently, both full captures wedged at exactly the first `bc659a` ("Meta element has no refresh delay") fixture — its passed examples are `content="0; URL=…"` immediate redirects — frozen >35 min in an untimeouted `await page.close()`. |
+| 7 | Replay of the recurrence region (`1a02b0`,`2ee8b8`,`59br37`, 61 testcases), no recycling, fresh process | 61/61 clean — the video region is not a deterministic trigger either. |
+| 8 | Wedged-browser aftermath (from the bc659a incident): `newPage` on the wedged browser | Throws `Protocol error (Target.createTarget): Not supported`, then `Target page, context or browser has been closed` once the process dies — a recognizable signature usable as a replace trigger. A deadline-only trigger let 73 rows (pour) / 886 rows (qualWeb) fail before the next scheduled recycle. |
+
+## Conclusion: probabilistic wedge events, answered by canary-triggered recycling
+
+The injection cliff is **nondeterministic**: it recurred at a different position in a nearly-new
+browser on a quiet machine, and every regional replay of a cliff neighborhood comes back clean.
+Best model: races between fixture-initiated behavior (media load, client-side navigation) and
+harness operations occasionally wedge browser-wide `<script>`-element execution — same family as
+the reproducible close() hang, at roughly 1–2 events per ~1,200 real-page cycles. Chasing a
+deterministic trigger further has diminishing value; the effective countermeasures, all now in
+`capture.js`, are:
+
+1. **Deadline-race every browser-touching await** (nothing can hang the loop);
+2. **Canary assertions** (expected tool global present on text/html pages);
+3. **Replace the browser on any wedge signature**: tripped deadline, `Protocol error
+   (Target.createTarget)`, "has been closed", or a failed canary.
+
+## Upstream status (2026-08-22)
+
+- Playwright: filed as <https://github.com/microsoft/playwright/issues/42366> (likely duplicate
+  of #42068, which the team milestoned v1.63); **fix PR submitted:**
+  <https://github.com/microsoft/playwright/pull/42367> (bounded re-issue of `Target.closeTarget`
+  in `CRBrowser._closePage` + stale-fixme removal; their close-related suites pass 41/41).
+  Note: Playwright's rolled Chromium 152.0.7977.54 narrows the race window sharply through
+  Playwright's call timing (their reload-trigger test passes 6/6 unpatched), but raw CDP still
+  reproduces the contract violation on 152 (~2/10) — so channel builds (Chrome/Edge stable,
+  151-class, 70–80% hit rate) remain the population the guard protects.
+- testaro: `browserClose` hardened with a 10 s settle-deadline per close (try/catch cannot catch
+  non-settlement) — PR <https://github.com/YRA-Tech/testaro/pull/92>. (Discovered en route:
+  jrpool/testaro now redirects to YRA-Tech/testaro — the repo was transferred, so #92 IS the
+  upstream PR.) The same patch is applied to the working tree used by the harness.
+- Chromium: already tracked as <https://issues.chromium.org/issues/536385539> (filed 2026-07-19
+  by a Google engineer; P2/S2; **pending code change 8251379**). Their C++ root cause matches our
+  protocol-level one exactly: `WebContentsImpl::ClosePage` arms the close on the current main
+  RenderFrameHost; a racing navigation commit swaps the RFH; the pending `ClosePage` callback
+  dies with the old RFH and is never re-issued. Our meta-refresh(0) trigger, hit rates, and
+  workaround matrix posted there as comment #2; both issues cross-linked. Practical upshot for
+  testaro: a fix is in flight upstream, but the deadline+canary+context-recycle defenses remain
+  necessary until it ships in a released Chromium — and for the injection cliff, which has no
+  upstream issue yet.
+
+## Discovery #2 — root-caused (2026-08-22 evening)
+
+A `DEBUG=pw:protocol` trace of a hanging round shows the full mechanism: Playwright sends
+`Target.closeTarget`, Chromium replies `{"result":{"success":true}}`, but the target — mid-commit
+in the meta-refresh navigation — never closes; the same session then emits the redirect
+destination's entire load lifecycle after the "successful" close, `Target.targetDestroyed` never
+fires, and `page.close()` waits for it forever. Chromium acknowledges a close it does not
+perform; Playwright trusts the acknowledgment with no timeout.
+
+Matrix (10 rounds/cell): **Chromium-only** (Firefox 153 and WebKit: 0/10 everywhere), and
+**default `close()` only** — `close({runBeforeUnload: true})` 0/10, `context.close()` 0/10.
+After a hang, `context.close()` still settles and destroys the page (**8/8 recoveries**).
+
+The recovery result upgrades the production recommendation: for this wedge class,
+**context-level teardown is a sufficient and reliable recovery** — no browser relaunch needed
+(~10 ms vs ~300 ms). A pooled-browser testaro could run context-per-act with deadline-raced
+closes, escalating to browser replacement only if the context teardown itself trips its
+deadline. (The injection cliff's recovery level remains untested — it is not reproducible on
+demand — so the escalation path should stay.)
+
+## Discovery #2 (original observation): page.close() can hang forever after a meta-refresh navigation
+
+This is a second, *reproducible-at-will* failure mode, distinct from the injection cliff but in
+the same family (navigation racing teardown corrupts page state):
+
+- Signature: `close()` neither resolves nor rejects, so `await page.close().catch(...)` blocks
+  forever — `.catch` guards rejection, not non-settlement. A loop with no deadline around close
+  freezes permanently, silently.
+- Nondeterministic (~50% of rounds locally): it is a race with the refresh-triggered navigation,
+  which also explains why one run passes a fixture another run wedges on.
+- No matching upstream issue found (2026-08-22 search); nearest family member is
+  [playwright#33806](https://github.com/microsoft/playwright/issues/33806) (hangs waiting for
+  pending navigations). **Fileable** with `repro-metarefresh.js` (~40 lines, local pages, no
+  external deps beyond playwright + any large script to inject — the script is incidental).
+- Harness guard shipped in `capture.js`: every browser-touching await is raced against a
+  deadline (`withDeadline`), a tripped deadline presumes a wedged browser and replaces it
+  (`replaceBrowser`, itself deadline-guarded), and captures resume by skipping rows already in
+  the out file. Verified against the live `bc659a` fixtures: one real close-hang tripped the 5 s
+  deadline, the browser was replaced, and all 15 fixtures completed.
+
+### Production translation (testaro)
+
+Pages that navigate during scanning (meta refresh is common on real sites) can hang `close()`
+forever. In a browser-per-act world the leaked hang is bounded by the act's child process exit;
+in any pooled/long-lived design it is fatal without deadlines. Concretely: (1) every
+`page.close()`/`browserClose()` should be deadline-raced, (2) a tripped deadline should recycle
+the browser, not just the page, and (3) the meta-refresh fixture class belongs in any isolation
+test suite as the canonical wedge-inducer.
+
+The overall answer to "could testaro keep a browser across acts?": the observed isolation
+failures here were **rare, probabilistic wedge events with recognizable signatures**, not
+gradual deterministic leakage. That favors a middle path over browser-per-act: pool the browser,
+assert a cheap per-act canary (script-injection round-trip + expected globals), and recycle the
+browser only on a wedge signature or every N acts — the three-countermeasure pattern above,
+already exercised at ~1,200-page scale by this harness. The remaining unknown for production is
+whether *cross-page contamination* (state leaking between scanned pages, distinct from wedges)
+also occurs; this harness's same-tool-different-page design cannot see that class, so a pooled
+design should keep context-per-act (cheap) even if the browser is shared.
+
+## Explained failure classes (distinct from the cliff)
+
+- `contentType application/xml` fixtures: `document.createElement('script')` in an XML document
+  creates a null-namespace element that never executes — expected, benign, now visible via the
+  adapter's diagnostic error (content type, script chars, ready state).
+- "Resulting promise was garbage collected": fixtures that navigate (meta refresh) during the
+  in-page evaluate. All observed on `inapplicable` testcases.
+
+## Current best theory
+
+Not page count, not the axe interleave, not any single fixture so far: either (a) cumulative
+state that only the full 273-testcase real-page history builds up, or (b) a nondeterministic
+in-browser event (renderer/helper-process crash or OOM kill — the host was carrying ~31 GB of
+desktop Chrome RSS during the incident) that wedges `<script>`-element execution while leaving
+CDP `evaluate` execution intact. All fixtures being same-origin (`www.w3.org`) means Chromium packs
+them into shared renderer processes, which is consistent with one wedged process poisoning every
+subsequent same-origin page.
+
+## Production relevance (testaro browser-per-act)
+
+- The failure mode was **silent** — no exception anywhere; only an output-level check (expected
+  global absent) caught it. Any move away from browser-per-act needs a per-act canary assertion
+  of this kind, not just error handling.
+- Recycling every N acts (experiment 3) is so far sufficient as a guard and is the cheap middle
+  ground between browser-per-act and browser-forever; whether *context*-per-act (much cheaper
+  than browser launch) also resets the poisoned state is untested — worth adding
+  `--recycle-context` bisection if experiment 5 reproduces the cliff.
+- If experiment 5 comes back clean, the honest conclusion is: one observed environmental
+  incident in ~5,000 page-cycles, mitigated by periodic recycling — suggestive that testaro's
+  isolation bugs may likewise be rare wedge events rather than deterministic leakage, which would
+  argue for recycle-on-anomaly (canary-triggered) rather than always-fresh browsers.

package/validation/act/playwright-issue-draft.md

@@ -0,0 +1,73 @@
+# Playwright bug report — page.close() never settles after meta-refresh navigation
+
+**FILED 2026-08-22: <https://github.com/microsoft/playwright/issues/42366>**
+Minimal repro: `repro-metarefresh.js` in this directory (self-contained, local HTTP server).
+Chromium side: already tracked as <https://issues.chromium.org/issues/536385539> (pending code
+change 8251379); cross-linked in both directions 2026-08-22 — see `chromium-issue-draft.md`.
+
+---
+
+**Title:** [Bug]: page.close() hangs forever — Chromium's Target.closeTarget returns success:true
+without closing a target that is mid-commit in a client-side (meta refresh) navigation
+
+## System info
+
+- Playwright Version: 1.62.1 (latest at time of report)
+- Operating System: Linux (7.0.0-30-generic)
+- Browser: Chromium (bundled, HeadlessChrome/151.0.7922.34) — **Chromium only**; Firefox and
+  WebKit are unaffected (0/10 each in the same harness)
+- Other info: Node 24.14.1
+
+## Repro
+
+1. Serve a page with `<meta http-equiv="refresh" content="0; URL='/target.html'">` (immediate
+   client-side redirect; the W3C ACT-Rules bc659a "passed" fixtures are real-world examples).
+2. `page.goto(url, {waitUntil: 'load'})` — resolves.
+3. `page.evaluate(...)` — rejects with "Execution context was destroyed" (expected; the refresh
+   navigation fired mid-evaluate).
+4. `await page.close()` — **hangs on ~70–80% of rounds** (8/10, 7/10 across runs): the promise
+   never resolves and never rejects. `.catch()` never fires; a loop awaiting it freezes forever.
+
+## Root cause (from a `DEBUG=pw:protocol` trace of a hanging round)
+
+- Playwright sends `Target.closeTarget {targetId}`.
+- The browser replies `{"result":{"success":true}}`.
+- The target does not close: the meta-refresh navigation was mid-commit, and the same session
+  then emits the full lifecycle of the redirect destination — `Page.frameNavigated`,
+  `DOMContentLoaded`, `load`, paint events, `networkIdle` — all *after* the successful-looking
+  `closeTarget` response.
+- `Target.targetDestroyed` is never emitted; `page.close()` waits for it indefinitely.
+
+So Chromium acknowledges the close without performing it when the close races a navigation
+commit, and Playwright trusts the acknowledgment with no timeout or retry.
+
+The browser-side half is confirmed **without Playwright**: a raw-CDP repro (plain WebSocket, no
+client library) shows `Target.closeTarget` → `success:true` with no `Target.targetDestroyed`
+ever emitted and the target still listed in `Target.getTargets`, ~3/10 rounds. A Chromium bug is
+being filed in parallel (link when available). Playwright still has its own half: `page.close()`
+trusts the acknowledgment indefinitely — a deadline/retry (or routing default close through the
+`runBeforeUnload` path, which is immune) would make clients robust to the browser defect.
+
+## Matrix (10 rounds per cell, same repro)
+
+| | `close()` | `close({runBeforeUnload:true})` | `context.close()` |
+| --- | --- | --- | --- |
+| Chromium | **8/10 hang** | 0/10 | 0/10 |
+| Firefox | 0/10 | 0/10 | 0/10 |
+| WebKit | 0/10 | 0/10 | 0/10 |
+
+After a hang, `context.close()` still settles and destroys the page (**8/8 recoveries**) — the
+context teardown path is unaffected.
+
+## Expected behavior
+
+`page.close()` always settles — resolves once the page is gone, or rejects.
+
+**Workarounds** (verified): `page.close({runBeforeUnload: true})`; or race `close()` against a
+deadline and call `context.close()` on expiry (recovers reliably). Without a guard, a wedged
+browser's subsequent `newPage` can then fail with `Protocol error (Target.createTarget): Not
+supported` followed by "Target page, context or browser has been closed".
+
+Possibly related: #33806 (hangs while waiting for pending navigations) — different in that no
+dialog is involved and the acknowledged-but-not-performed `Target.closeTarget` is visible in the
+protocol trace.

package/validation/act/repro-cdp-raw.js

@@ -0,0 +1,105 @@
+// Playwright-free reproduction attempt: raw CDP over WebSocket (Node's
+// native WebSocket client). Question: does Chromium's Target.closeTarget
+// return {success:true} WITHOUT closing (no Target.targetDestroyed, target
+// still listed) when the close races a meta-refresh navigation commit?
+// If yes → browser-side contract violation, independent of Playwright.
+const http = require('http');
+const {spawn} = require('child_process');
+// Any Chromium binary works; defaults to Playwright's bundled one.
+const {chromium} = require('playwright');
+const CHROME = process.env.CHROME_PATH || chromium.executablePath();
+
+const pages = {
+  '/a.html': `<!DOCTYPE html><html lang="en"><head><title>a</title>
+    <meta http-equiv="refresh" content="0; URL='/target.html'"></head>
+    <body><p>redirecting</p></body></html>`,
+  '/target.html': `<!DOCTYPE html><html lang="en"><head><title>t</title></head>
+    <body><p>arrived</p></body></html>`
+};
+
+(async () => {
+  const server = http.createServer((req, res) => {
+    res.setHeader('content-type', 'text/html');
+    res.end(pages[req.url] || pages['/target.html']);
+  });
+  await new Promise(resolve => server.listen(0, resolve));
+  const base = `http://127.0.0.1:${server.address().port}`;
+
+  // Launch the same Chromium binary Playwright uses, but bare.
+  const chrome = spawn(CHROME, [
+    '--headless=new', '--remote-debugging-port=0', '--no-first-run', '--no-default-browser-check',
+    '--user-data-dir=/tmp/cdp-repro-profile'
+  ]);
+  const wsURL = await new Promise((resolve, reject) => {
+    let stderr = '';
+    chrome.stderr.on('data', chunk => {
+      stderr += chunk;
+      const match = stderr.match(/DevTools listening on (ws:\/\/\S+)/);
+      if (match) resolve(match[1]);
+    });
+    setTimeout(() => reject(new Error('no DevTools ws URL')), 15000);
+  });
+
+  const ws = new WebSocket(wsURL);
+  await new Promise(resolve => ws.onopen = resolve);
+  let nextId = 1;
+  const pending = new Map();
+  const events = [];
+  ws.onmessage = messageEvent => {
+    const message = JSON.parse(messageEvent.data);
+    if (message.id && pending.has(message.id)) {
+      pending.get(message.id)(message);
+      pending.delete(message.id);
+    }
+    else if (message.method) {
+      events.push(message);
+    }
+  };
+  const send = (method, params = {}, sessionId) => new Promise(resolve => {
+    const id = nextId++;
+    pending.set(id, resolve);
+    ws.send(JSON.stringify({id, method, params, ...(sessionId ? {sessionId} : {})}));
+  });
+  const wait = ms => new Promise(resolve => setTimeout(resolve, ms));
+
+  await send('Target.setDiscoverTargets', {discover: true});
+  let contractViolations = 0;
+  const ROUNDS = 10;
+  for (let round = 0; round < ROUNDS; round++) {
+    const {result: {targetId}} = await send('Target.createTarget', {url: 'about:blank'});
+    const {result: {sessionId}} = await send('Target.attachToTarget', {targetId, flatten: true});
+    await send('Page.enable', {}, sessionId);
+    await send('Runtime.enable', {}, sessionId);
+    await send('Page.navigate', {url: `${base}/a.html`}, sessionId);
+    // Leave an evaluate in flight for the refresh navigation to destroy
+    // (mirrors the real-world trigger), then close during the commit window.
+    send('Runtime.evaluate', {
+      expression: 'new Promise(r => setTimeout(r, 100))',
+      awaitPromise: true
+    }, sessionId);
+    await wait(30 + (round % 5) * 15);
+    events.length = 0;
+    const closeReply = await send('Target.closeTarget', {targetId});
+    const success = closeReply.result && closeReply.result.success;
+    await wait(3000);
+    const destroyed = events.some(
+      e => e.method === 'Target.targetDestroyed' && e.params.targetId === targetId
+    );
+    const {result: {targetInfos}} = await send('Target.getTargets');
+    const stillListed = targetInfos.some(t => t.targetId === targetId);
+    const verdict = success && ! destroyed && stillListed ? 'CONTRACT VIOLATION'
+      : success && destroyed ? 'closed properly'
+        : `other (success=${success} destroyed=${destroyed} listed=${stillListed})`;
+    console.log(`round ${round}: closeTarget success=${success}, targetDestroyed=${destroyed}, stillListed=${stillListed} → ${verdict}`);
+    if (verdict === 'CONTRACT VIOLATION') {
+      contractViolations++;
+      // Clean up the orphan so rounds stay independent.
+      await send('Target.closeTarget', {targetId});
+      await wait(500);
+    }
+  }
+  console.log(`${contractViolations}/${ROUNDS} rounds: success:true with target neither destroyed nor removed`);
+  chrome.kill();
+  server.close();
+  process.exit(0);
+})();

package/validation/act/repro-metarefresh.js

@@ -0,0 +1,65 @@
+/*
+  © 2026 Jeff Witt.
+
+  Licensed under the MIT License. See LICENSE file at the project root or
+  https://opensource.org/license/mit/ for details.
+
+  SPDX-License-Identifier: MIT
+*/
+
+/*
+  repro-metarefresh.js
+  Minimal reproduction: page.close() intermittently NEVER SETTLES (neither
+  resolves nor rejects) when the page performed a meta-refresh(0) navigation
+  that interrupted an in-flight evaluate. See playwright-issue-draft.md.
+  Observed on playwright 1.62.1, Chromium headless: ~50% of rounds hang.
+  Run from the repository root: node validation/act/repro-metarefresh.js
+*/
+
+const http = require('http');
+const {chromium} = require('playwright');
+
+const pages = {
+  '/a.html': `<!DOCTYPE html><html lang="en"><head><title>a</title>
+    <meta http-equiv="refresh" content="0; URL='/target.html'"></head>
+    <body><p>redirecting</p></body></html>`,
+  '/target.html': `<!DOCTYPE html><html lang="en"><head><title>t</title></head>
+    <body><p>arrived</p></body></html>`
+};
+
+// Reports whether a promise settles within ms.
+const settles = (promise, ms) => Promise.race([
+  promise.then(() => 'resolved').catch(() => 'rejected'),
+  new Promise(resolve => setTimeout(() => resolve('NEVER SETTLED'), ms))
+]);
+
+(async () => {
+  const server = http.createServer((req, res) => {
+    res.setHeader('content-type', 'text/html');
+    res.end(pages[req.url] || pages['/target.html']);
+  });
+  await new Promise(resolve => server.listen(0, resolve));
+  const base = `http://127.0.0.1:${server.address().port}`;
+  const browser = await chromium.launch();
+  const context = await browser.newContext();
+  let hangs = 0;
+  const ROUNDS = 10;
+  for (let round = 0; round < ROUNDS; round++) {
+    const page = await context.newPage();
+    await page.goto(`${base}/a.html`, {waitUntil: 'load'});
+    // An evaluate for the meta refresh to interrupt (it rejects with
+    // "Execution context was destroyed" — expected and irrelevant).
+    await page.evaluate(
+      () => new Promise(resolve => setTimeout(resolve, 100))
+    ).catch(() => {});
+    const outcome = await settles(page.close(), 10000);
+    console.log(`round ${round}: page.close() ${outcome}`);
+    if (outcome === 'NEVER SETTLED') {
+      hangs++;
+    }
+  }
+  console.log(`${hangs}/${ROUNDS} rounds hung`);
+  await Promise.race([browser.close(), new Promise(r => setTimeout(r, 5000))]);
+  server.close();
+  process.exit(hangs ? 1 : 0);
+})();

package/validation/act/score.js

@@ -0,0 +1,184 @@
+/*
+  © 2026 Jeff Witt.
+
+  Licensed under the MIT License. See LICENSE file at the project root or
+  https://opensource.org/license/mit/ for details.
+
+  SPDX-License-Identifier: MIT
+*/
+
+/*
+  score.js
+  Track-A scoring: computes a per-engine, per-ACT-rule confusion matrix from a
+  capture JSONL (capture.js output).
+
+  Usage:
+    node validation/act/score.js --in results/act-....jsonl [--band asserted|review|both]
+      [--json out.json]
+
+  Scoring policy (criterion-level comparability layer, v1):
+  - An ACT rule's positive criteria are its forConformance WCAG 2.x success
+    criteria from the testcase feed. Rules with none (technique/ARIA-only
+    rules) are reported separately as unscoreable.
+  - An engine FLAGS a testcase when it reports ≥1 finding on any of the rule's
+    criteria. Band `asserted` (default) counts definite failures only (standard
+    instance outcome `failed`); `review` counts engine-flagged uncertainty
+    (outcome `cantTell`); `both` counts either.
+  - failed testcases are the positive class; passed + inapplicable are the
+    negative class. Sensitivity = TP/(TP+FN); specificity = TN/(TN+FP).
+  Known generosities/strictnesses, stated wherever results publish: a finding
+  on the right criterion from an unrelated check still counts (generous); a
+  real finding reported under a different criterion does not (strict).
+*/
+
+// IMPORTS
+
+const fs = require('fs');
+const path = require('path');
+
+// FUNCTIONS
+
+const parseArgs = argv => {
+  const args = {};
+  for (let i = 2; i < argv.length; i += 2) {
+    args[argv[i].replace(/^--/, '')] = argv[i + 1];
+  }
+  return args;
+};
+
+const percent = (numerator, denominator) =>
+  denominator ? `${(100 * numerator / denominator).toFixed(1)}%` : '—';
+
+// OPERATION
+
+const args = parseArgs(process.argv);
+if (! args.in) {
+  console.log('Usage: node validation/act/score.js --in <capture.jsonl> [--band asserted|review|both]');
+  process.exit(1);
+}
+const band = args.band || 'asserted';
+const rows = fs.readFileSync(args.in, 'utf8')
+.split('\n')
+.filter(Boolean)
+.map(line => JSON.parse(line));
+
+// ACT-rule criterion sets from the cached feed.
+const feed = JSON.parse(
+  fs.readFileSync(path.join(__dirname, 'cache', 'testcases.json'), 'utf8')
+);
+const ruleCriteria = {};
+const ruleNames = {};
+feed.testcases.forEach(testcase => {
+  ruleNames[testcase.ruleId] = testcase.ruleName;
+  if (! ruleCriteria[testcase.ruleId]) {
+    const criteria = Object.entries(testcase.ruleAccessibilityRequirements || {})
+    .filter(([key, value]) => /^wcag2\d:/.test(key) && value && value.forConformance)
+    .map(([key]) => key.split(':')[1]);
+    ruleCriteria[testcase.ruleId] = new Set(criteria);
+  }
+});
+
+// Whether a capture row flags its testcase under the scoring policy. Rows
+// with exact ACT-rule maps (actAsserted/actReview — qualWeb) are scored on
+// those; rows scored at the exact level bypass the unscoreable-criteria gate
+// since the mapping needs no WCAG SC bridge.
+const flags = row => {
+  if (row.actAsserted || row.actReview) {
+    const buckets = [];
+    if (band === 'asserted' || band === 'both') {
+      buckets.push(row.actAsserted || {});
+    }
+    if (band === 'review' || band === 'both') {
+      buckets.push(row.actReview || {});
+    }
+    return buckets.some(bucket => !! bucket[row.ruleId]);
+  }
+  const criteria = ruleCriteria[row.ruleId];
+  if (! criteria || ! criteria.size) {
+    return null;
+  }
+  const buckets = [];
+  if (band === 'asserted' || band === 'both') {
+    buckets.push(row.asserted || {});
+  }
+  if (band === 'review' || band === 'both') {
+    buckets.push(row.review || {});
+  }
+  return buckets.some(
+    bucket => Object.entries(bucket).some(([criterion, count]) => count && criteria.has(criterion))
+  );
+};
+
+// Tally per engine × rule.
+const tallies = {};
+const unscoreable = new Set();
+const errored = {};
+rows.forEach(row => {
+  if (row.prevented) {
+    errored[row.engine] = (errored[row.engine] || 0) + 1;
+    return;
+  }
+  const flagged = flags(row);
+  if (flagged === null) {
+    unscoreable.add(row.ruleId);
+    return;
+  }
+  tallies[row.engine] ??= {};
+  const tally = tallies[row.engine][row.ruleId] ??= {TP: 0, FP: 0, TN: 0, FN: 0};
+  if (row.expected === 'failed') {
+    tally[flagged ? 'TP' : 'FN']++;
+  }
+  else {
+    tally[flagged ? 'FP' : 'TN']++;
+  }
+});
+
+// Report.
+const report = {band, engines: {}};
+Object.entries(tallies).forEach(([engine, ruleTallies]) => {
+  console.log(`\n## ${engine} (band: ${band})\n`);
+  console.log('| ACT rule | Name | TP | FN | FP | TN | Sens | Spec |');
+  console.log('| --- | --- | --- | --- | --- | --- | --- | --- |');
+  const totals = {TP: 0, FP: 0, TN: 0, FN: 0};
+  const perRule = {};
+  Object.entries(ruleTallies)
+  .sort(([, a], [, b]) => (b.TP + b.FN) - (a.TP + a.FN))
+  .forEach(([ruleId, tally]) => {
+    ['TP', 'FP', 'TN', 'FN'].forEach(key => {
+      totals[key] += tally[key];
+    });
+    perRule[ruleId] = {
+      ...tally,
+      sensitivity: tally.TP + tally.FN ? tally.TP / (tally.TP + tally.FN) : null,
+      specificity: tally.TN + tally.FP ? tally.TN / (tally.TN + tally.FP) : null
+    };
+    // Rows where the engine saw nothing at all and nothing was expected are
+    // uninformative for display; keep them in totals and JSON regardless.
+    if (tally.TP + tally.FN === 0 && tally.FP === 0) {
+      return;
+    }
+    console.log(
+      `| ${ruleId} | ${ruleNames[ruleId].slice(0, 45)} | ${tally.TP} | ${tally.FN} | ${tally.FP} `
+      + `| ${tally.TN} | ${percent(tally.TP, tally.TP + tally.FN)} `
+      + `| ${percent(tally.TN, tally.TN + tally.FP)} |`
+    );
+  });
+  console.log(
+    `| **all** | | ${totals.TP} | ${totals.FN} | ${totals.FP} | ${totals.TN} `
+    + `| ${percent(totals.TP, totals.TP + totals.FN)} `
+    + `| ${percent(totals.TN, totals.TN + totals.FP)} |`
+  );
+  report.engines[engine] = {totals, perRule, errored: errored[engine] || 0};
+});
+if (unscoreable.size) {
+  console.log(
+    `\nUnscoreable (no forConformance WCAG 2.x SC): ${[...unscoreable].join(', ')}`
+  );
+}
+Object.entries(errored).forEach(([engine, count]) => {
+  console.log(`Errored/prevented rows for ${engine}: ${count}`);
+});
+if (args.json) {
+  fs.writeFileSync(args.json, JSON.stringify(report, null, 2));
+  console.log(`\nJSON → ${args.json}`);
+}

package/validation/act/stage3a-stress-report.md

@@ -0,0 +1,69 @@
+# Stage-3a stress report — pour on 300 real pages (2026-08-22)
+
+Capture: `results/stage3a-2026-08-22.jsonl` — 300 pages (250 seeded-random US CrUX tail + 50
+top-1000 head; `yra-100k/data/stage3a-sample-300.txt`, seed 42) × pour, axe, ed11y, qualWeb, via
+the hardened URL-mode harness (per-instance element paths captured for Stage 3b). The run
+completed unattended: 4 canary-triggered browser replacements, zero stalls.
+
+## Gate verdict: PASS
+
+| Engine | Raw ran-rate | Effective ran-rate¹ | p50 | p95 | Instances |
+| --- | --- | --- | --- | --- | --- |
+| pour | 96.3% | **99.0%** | 2.0 s | 7.7 s | 50,123 |
+| axe | 97.0% | 99.7% | 1.7 s | 6.5 s | 32,071 |
+| ed11y | 96.7% | 99.3% | 1.0 s | 4.3 s | 6,029 |
+| qualWeb | 94.7% | 97.3% | 4.1 s | 13.4 s | 113,656 |
+
+¹ Excluding the 8 pages unreachable for **every** engine (dead/hostile sites — page problems,
+not engine problems). Pour's gate was ≥95%: passed on either denominator. Latency is well inside
+the worker budget. No trigger-happy rule: nothing asserts on >90% of pages (max:
+`target-size-enhanced`, 86% — an AAA rule where near-ubiquity is plausible; flagged for tic
+quality review, not a defect).
+
+## Prevention taxonomy (engine-specific residue is small)
+
+Navigation failures/timeouts dominate (6–7 per engine — the dead-site class). Residue:
+`navigated-during-eval` 1–3/engine (meta-refresh-class pages; results legitimately unobtainable);
+reporter timeouts on 2 extremely heavy pages (pour 1, ed11y 3); qualWeb "No DOM" ×7 (its own
+HTML ingestion fails on some pages — an incumbent robustness datum, not a candidate problem);
+one axe adapter TypeError on one page (page-specific, pre-existing adapter code). Zero CSP
+preventions — the nonce passthrough works in the wild.
+
+## Volume anatomy — the kitchen-sink effect
+
+Pour's headline 50k instances is dominated by AAA and best-practice rules (~30k: contrast-
+enhanced 13.1k, target-size-enhanced 9.2k, region 8.1k), a direct consequence of the adapter's
+run-every-rule setting (matching axe's). The A/AA-comparable picture is much closer, and on the
+single biggest A/AA category the two engines nearly agree:
+
+- **color-contrast (1.4.3): pour 8,829 instances / 74% of pages vs axe 8,795 / 76%** — near-
+  identical at scale.
+- link-name: pour 536/30% vs axe 660/31%. heading-order: 153/30% vs 269/32%.
+- Divergences to watch in Stage 3b: `region` (pour 8.1k vs axe 2.7k — counting granularity) and
+  axe's `hidden-content` (11.3k, 87% of pages — its own kitchen-sink review rule).
+
+## Assertion-band inversion (the vendor claim, reproduced)
+
+Of pour's instances, **72% are asserted violations** (severity 2–3) and 28% review; axe inverts:
+**34% violations, 66% review/incomplete**. This independently reproduces pour.dev's "twice the
+failing elements with fewer check-by-eye verdicts" claim — and it makes Stage-4 triage of pour's
+*asserted* band the critical validation: pour stakes much more on definite assertions than axe
+does.
+
+## ACT-identified FP classes at real-world scale
+
+- **Contrast** (both rules): 21.9k instances = 44% of pour's volume — the FP mechanisms found on
+  ACT fixtures (symbol-only text, letter-as-icon glyphs) live inside this mass. Highest-priority
+  triage sample for Stage 4.
+- **Language applicability** (`valid-lang-parts`): fired on **zero** real pages — the ACT FP
+  class is real but rare in the wild. `html-lang` asserts on 31 pages (10.7%) — consistent with
+  WebAIM's ~13% missing-language prevalence, so likely mostly true positives.
+- `button-name` 111 instances / 32 pages, `link-name` 536 / 88 pages — moderate volumes; the
+  known name-computation gap (descendant `aria-labelledby`) warrants a targeted triage slice.
+
+## Next (Stage 3b)
+
+Per-instance element paths are already captured for all four engines. Run
+`propose-tic-mappings`-style co-occurrence between pour rule IDs and the mapped incumbents'
+issues over this same JSONL; disposition all 70 pour rules that fired (plus the ~16 that
+didn't); then Stage-4 triage sampling with the contrast and link/button-name slices prioritized.