@aarwitz/tapp 0.16.2 → 0.16.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +11 -2
- package/README.md +18 -3
- package/bin/tapp.js +23 -3
- package/mcp-server/src/ci-report.js +2 -3
- package/mcp-server/src/html-report.js +3 -4
- package/mcp-server/src/index.js +25 -14
- package/mcp-server/src/report.js +35 -4
- package/mcp-server/src/web-explorer.js +319 -50
- package/package.json +1 -1
package/AGENTS.md
CHANGED
|
@@ -11,7 +11,7 @@ the app already on the simulator); it also accepts a repo dir, a `path/to/App.ap
|
|
|
11
11
|
bundle id, or (qa only) an http(s) URL. You never need to know a bundle id up front.
|
|
12
12
|
|
|
13
13
|
```bash
|
|
14
|
-
npx -y @aarwitz/tapp qa [target] # autonomous QA →
|
|
14
|
+
npx -y @aarwitz/tapp qa [target] # autonomous QA → scoped release verdict + findings (≈ tapp_run_qa)
|
|
15
15
|
npx -y @aarwitz/tapp open [target] # launch + screen summary + screenshot saved to a file (≈ tapp_open_app)
|
|
16
16
|
npx -y @aarwitz/tapp tree [target] # accessibility tree, --json for every element (≈ tapp_ui_tree)
|
|
17
17
|
npx -y @aarwitz/tapp shot # screenshot the booted sim → file path (≈ tapp_screenshot)
|
|
@@ -21,6 +21,11 @@ npx -y @aarwitz/tapp qa app.apk --platform android --app-id com.acme.app
|
|
|
21
21
|
npx -y @aarwitz/tapp flow run .tapp/flows/smoke.yml # committed, keyless E2E replay
|
|
22
22
|
```
|
|
23
23
|
|
|
24
|
+
For focused web evidence, `open` and `tree` accept one semantic interaction plus an async content
|
|
25
|
+
wait: `tapp open https://example.com --tap "Not now" --wait-for "Dashboard"`. Tapp waits for the
|
|
26
|
+
page to stabilize before capturing it and warns honestly if the bounded wait ends while it is still
|
|
27
|
+
loading or changing.
|
|
28
|
+
|
|
24
29
|
**Seeing the screen, per client:** if you can read image files into your context (Claude
|
|
25
30
|
Code's Read tool, Codex's view-image), open the saved screenshot path the CLI prints —
|
|
26
31
|
that IS the screen. If you cannot (Cursor, VS Code Copilot), connect the MCP server
|
|
@@ -39,7 +44,7 @@ installs, returns the bundle id) → `tapp_run_qa {appBundleId}`.
|
|
|
39
44
|
|---|---|---|
|
|
40
45
|
| "Show me / screenshot a screen" | `tapp_open_app` (launch + screenshot + tree, ~15s) | `tapp_run_qa` (a full multi-minute QA exploration) |
|
|
41
46
|
| "Tap through / drive / fill a form / log in" | `tapp_session_start` → `session_act` loop | repeated `open_app` calls (cold relaunch each time) |
|
|
42
|
-
| "Is my app broken? Is it ship-ready? Find bugs" | `tapp_run_qa` — `appBundleId` for iOS, `androidAppId` for Android, `url` for owned web apps | a manual session (QA exploration is autonomous) |
|
|
47
|
+
| "Is my app broken? Is it ship-ready? Find bugs" | `tapp_run_qa` — `appBundleId` for iOS, `androidAppId` for Android, `url` for owned web apps; web `ready` means the disclosed automated checks passed, not that copy/privacy/brand claims were reviewed | a manual session (QA exploration is autonomous) |
|
|
43
48
|
| "Make this flow a repeatable test" | drive it in a session, then `tapp_flow_save`; replay with `tapp_flow_run` | re-driving it by hand every time |
|
|
44
49
|
| "What's on screen right now?" | `tapp_screenshot` / `tapp_ui_tree` | relaunching the app |
|
|
45
50
|
|
|
@@ -89,6 +94,10 @@ Returns `{verdict, confidence, headline, screensExplored, actionsPerformed, find
|
|
|
89
94
|
the user** for them rather than re-running blind.
|
|
90
95
|
- Diff two runs: pass the previous run's `findings` as `baselineFindings` → you get a
|
|
91
96
|
`regression` block (`new` / `persisting` / `resolved`, plus a CI `gate` signal).
|
|
97
|
+
- On web, report the exact verdict but preserve its scope: Tapp deterministically checks technical
|
|
98
|
+
behavior such as failed requests, missing assets, placeholder links, and inert controls. It does
|
|
99
|
+
not validate marketing claims against APIs, API field privacy, brand consistency, or subjective
|
|
100
|
+
marketplace credibility unless an explicit reviewed test/contract covers them.
|
|
92
101
|
|
|
93
102
|
## Flows (deterministic E2E tests)
|
|
94
103
|
|
package/README.md
CHANGED
|
@@ -26,7 +26,7 @@ Three platforms, one judgment layer:
|
|
|
26
26
|
does not link a Tapp SDK.
|
|
27
27
|
- **Web (beta)** — built *on* Playwright. Your agent already has browser hands; tapp adds the
|
|
28
28
|
autonomous exploration, the deterministic detectors (uncaught exceptions, failed requests,
|
|
29
|
-
dead buttons, broken links, error pages), and the same verdict.
|
|
29
|
+
dead buttons, broken links, placeholder `href="#"` links, error pages), and the same verdict.
|
|
30
30
|
|
|
31
31
|
```
|
|
32
32
|
you: "Add a logout button to the settings screen"
|
|
@@ -99,6 +99,14 @@ npx -y @aarwitz/tapp build [dir] # just build + install (scheme auto-detecte
|
|
|
99
99
|
Web (beta): `npx -y @aarwitz/tapp qa http://localhost:3000` *(one-time setup:
|
|
100
100
|
`npm i -g playwright && npx playwright install chromium`)*
|
|
101
101
|
|
|
102
|
+
Focused web inspection waits briefly for loading states to settle. If a consent or location modal
|
|
103
|
+
blocks the screen, dismiss it and wait for the content you care about in the same package-only call:
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
npx -y @aarwitz/tapp open https://example.com --tap "Not now" --wait-for "Dashboard"
|
|
107
|
+
npx -y @aarwitz/tapp tree https://example.com --tap "Not now" --wait-for "Dashboard" --json
|
|
108
|
+
```
|
|
109
|
+
|
|
102
110
|
Android:
|
|
103
111
|
|
|
104
112
|
```bash
|
|
@@ -201,14 +209,21 @@ ranks runs, it doesn't promise odds.
|
|
|
201
209
|
`tapp_run_qa` explores like a user — accessibility surfaces on iOS/Android and a real browser on web —
|
|
202
210
|
and detects crashes, failed sign-ins, dead buttons, stuck loading screens, error surfaces,
|
|
203
211
|
navigation loops, and dead ends (plus, on web: uncaught JS exceptions, failed/5xx requests,
|
|
204
|
-
broken links and assets
|
|
212
|
+
broken links and assets, and visible placeholder links with no destination). The verdict is
|
|
213
|
+
**deterministic** (no LLM in the run loop) and **honest**:
|
|
205
214
|
|
|
206
215
|
- `blocked` — a release-blocking issue was found.
|
|
207
216
|
- `caution` — issues to review, or the run couldn't see enough.
|
|
208
|
-
- `ready` — genuinely explored with no blockers. **A shallow run
|
|
217
|
+
- `ready` — genuinely explored with no detected blockers in the checks that ran. **A shallow run
|
|
218
|
+
is never `ready`** — if the
|
|
209
219
|
app crashed on launch or a login wall blocked exploration, you get `inconclusive: true`,
|
|
210
220
|
not a false pass. Absence of findings is not a pass.
|
|
211
221
|
|
|
222
|
+
Web beta presents a `ready` result as **AUTOMATED CHECKS PASSED**, not “ship-ready.” Its report
|
|
223
|
+
explicitly excludes content/claim accuracy, privacy and API data minimization, brand/SEO
|
|
224
|
+
consistency, and subjective visual credibility. Those require reviewed contracts, privacy review,
|
|
225
|
+
or human/vision judgment; a green technical crawl must not imply they were validated.
|
|
226
|
+
|
|
212
227
|
Apps behind a login? Pass `testEmail`/`testPassword` (typed into the login form automatically),
|
|
213
228
|
`appLaunchArgs` (e.g. `["--uitesting"]` if your app supports a bypass), or explicit `loginSteps`
|
|
214
229
|
for custom login UIs.
|
package/bin/tapp.js
CHANGED
|
@@ -424,6 +424,7 @@ switch (command) {
|
|
|
424
424
|
testEmail: flags.email,
|
|
425
425
|
testPassword: flags.password,
|
|
426
426
|
baselineFindings,
|
|
427
|
+
surface: "cli",
|
|
427
428
|
onProgress,
|
|
428
429
|
})
|
|
429
430
|
: platform === "android"
|
|
@@ -435,6 +436,7 @@ switch (command) {
|
|
|
435
436
|
testPassword: flags.password,
|
|
436
437
|
baselineFindings,
|
|
437
438
|
clearData: flags["keep-data"] !== true,
|
|
439
|
+
surface: "cli",
|
|
438
440
|
onProgress,
|
|
439
441
|
})
|
|
440
442
|
: await engine.runQaIos({
|
|
@@ -442,6 +444,7 @@ switch (command) {
|
|
|
442
444
|
maxActions: flags.actions,
|
|
443
445
|
timeout: flags.timeout,
|
|
444
446
|
args: { testEmail: flags.email, testPassword: flags.password, baselineFindings },
|
|
447
|
+
surface: "cli",
|
|
445
448
|
onProgress,
|
|
446
449
|
});
|
|
447
450
|
process.stderr.write("\n");
|
|
@@ -469,13 +472,21 @@ switch (command) {
|
|
|
469
472
|
}
|
|
470
473
|
try {
|
|
471
474
|
const { inspectWebPage } = await import(path.join(packageRoot, "mcp-server", "src", "web-explorer.js"));
|
|
472
|
-
const snap = await inspectWebPage({
|
|
475
|
+
const snap = await inspectWebPage({
|
|
476
|
+
url,
|
|
477
|
+
timeoutMs: Number(flags.timeout) * 1000 || 15_000,
|
|
478
|
+
tapText: typeof flags.tap === "string" ? flags.tap : "",
|
|
479
|
+
waitForText: typeof flags["wait-for"] === "string" ? flags["wait-for"] : "",
|
|
480
|
+
});
|
|
473
481
|
const out = typeof flags.out === "string" ? path.resolve(flags.out) : path.join(tappHome, "shots", `web-${Date.now()}.png`);
|
|
474
482
|
fs.mkdirSync(path.dirname(out), { recursive: true });
|
|
475
483
|
fs.writeFileSync(out, snap.image);
|
|
476
484
|
console.log(`🌐 Opened \`${snap.url}\`\n`);
|
|
485
|
+
if (typeof flags.tap === "string") console.log(`👆 Tapped \`${flags.tap}\`\n`);
|
|
486
|
+
if (typeof flags["wait-for"] === "string") console.log(`⏳ Found \`${flags["wait-for"]}\`\n`);
|
|
477
487
|
console.log(engine.formatScreen(snap.screenTitle, snap.elements));
|
|
478
488
|
console.log(`\n📸 Screenshot: ${out}`);
|
|
489
|
+
if (!snap.settled) console.error("⚠️ Page still showed a loading or changing state when the bounded wait ended.");
|
|
479
490
|
} catch (error) {
|
|
480
491
|
console.error(`❌ ${error.message || String(error)}`);
|
|
481
492
|
process.exit(1);
|
|
@@ -533,9 +544,16 @@ switch (command) {
|
|
|
533
544
|
}
|
|
534
545
|
try {
|
|
535
546
|
const { inspectWebPage } = await import(path.join(packageRoot, "mcp-server", "src", "web-explorer.js"));
|
|
536
|
-
const snap = await inspectWebPage({
|
|
537
|
-
|
|
547
|
+
const snap = await inspectWebPage({
|
|
548
|
+
url,
|
|
549
|
+
timeoutMs: Number(flags.timeout) * 1000 || 15_000,
|
|
550
|
+
screenshot: false,
|
|
551
|
+
tapText: typeof flags.tap === "string" ? flags.tap : "",
|
|
552
|
+
waitForText: typeof flags["wait-for"] === "string" ? flags["wait-for"] : "",
|
|
553
|
+
});
|
|
554
|
+
if (flags.json) console.log(JSON.stringify({ platform: "web", url: snap.url, screenTitle: snap.screenTitle, settled: snap.settled, elements: snap.elements }, null, 2));
|
|
538
555
|
else console.log(engine.formatScreen(snap.screenTitle, snap.elements));
|
|
556
|
+
if (!snap.settled) console.error("⚠️ Page still showed a loading or changing state when the bounded wait ended.");
|
|
539
557
|
} catch (error) {
|
|
540
558
|
console.error(`❌ ${error.message || String(error)}`);
|
|
541
559
|
process.exit(1);
|
|
@@ -1358,9 +1376,11 @@ switch (command) {
|
|
|
1358
1376
|
|
|
1359
1377
|
Zero-config verbs (agents and humans can just run these — no server, no setup):
|
|
1360
1378
|
tapp open [target] Launch the app → screen summary + screenshot saved to a file
|
|
1379
|
+
(web: --tap TEXT · --wait-for TEXT · --out FILE)
|
|
1361
1380
|
tapp qa [target] Autonomous QA → verdict + findings + evidence
|
|
1362
1381
|
(--platform ios|android|web · --app-id ID · --apk FILE · --actions N)
|
|
1363
1382
|
tapp tree [target] Accessibility tree of the current screen (--json for every element)
|
|
1383
|
+
(web: --tap TEXT · --wait-for TEXT)
|
|
1364
1384
|
tapp flow run FILE Replay a committed deterministic Flow (no AI/API key)
|
|
1365
1385
|
tapp flow validate FILE Validate a Flow without launching a target
|
|
1366
1386
|
tapp task validate FILE Validate a reusable deterministic Task (+ optional UI Map grounding)
|
|
@@ -26,7 +26,7 @@
|
|
|
26
26
|
// any fail on any finding at all, or any flow failure. Strictest.
|
|
27
27
|
import fs from "fs";
|
|
28
28
|
import path from "node:path";
|
|
29
|
-
import { buildQaReport, computeRegression, computeContentCollapse, computeReachabilityLoss } from "./report.js";
|
|
29
|
+
import { buildQaReport, computeRegression, computeContentCollapse, computeReachabilityLoss, verdictBadge } from "./report.js";
|
|
30
30
|
import { writeHtmlReport } from "./html-report.js";
|
|
31
31
|
import { buildUiMapFromMarkers, writeUiMap } from "./ui-map.js";
|
|
32
32
|
import { proposeSelectorMaintenance, validateWebMaintenanceProposal } from "./maintenance-proposal.js";
|
|
@@ -337,12 +337,11 @@ function enrichPrPlan(plan, contracts, currentUiMap = null, markersPath = "", pr
|
|
|
337
337
|
return { ...plan, selected, explorationTargets, maintenanceCandidates, execution: counts };
|
|
338
338
|
}
|
|
339
339
|
|
|
340
|
-
const VERDICT_BADGE = { ready: "🟢 SHIP-READY", caution: "🟡 CAUTION", blocked: "🔴 BLOCKED" };
|
|
341
340
|
const SEV_ICON = { critical: "🟥", high: "🟧", medium: "🟨", low: "🟩" };
|
|
342
341
|
|
|
343
342
|
function renderMarkdown(report, regression, flows, scenarios, contracts, prPlan, gate) {
|
|
344
343
|
const lines = [];
|
|
345
|
-
lines.push(`## tapp release check — ${
|
|
344
|
+
lines.push(`## tapp release check — ${verdictBadge(report)}`);
|
|
346
345
|
lines.push("");
|
|
347
346
|
lines.push(report.headline);
|
|
348
347
|
lines.push("");
|
|
@@ -8,9 +8,8 @@
|
|
|
8
8
|
|
|
9
9
|
import fs from "fs";
|
|
10
10
|
import path from "path";
|
|
11
|
-
import { buildQaReport } from "./report.js";
|
|
11
|
+
import { buildQaReport, verdictBadge } from "./report.js";
|
|
12
12
|
|
|
13
|
-
const BADGE = { ready: "🟢 SHIP-READY", caution: "🟡 CAUTION", blocked: "🔴 BLOCKED" };
|
|
14
13
|
const SEV_COLOR = { critical: "#cf222e", high: "#bc4c00", medium: "#9a6700", low: "#57606a" };
|
|
15
14
|
|
|
16
15
|
const esc = (s) => String(s ?? "").replace(/[&<>"]/g, (c) => ({ "&": "&", "<": "<", ">": ">", '"': """ }[c]));
|
|
@@ -102,7 +101,7 @@ export function writeHtmlReport(captureDir, { report, label = "" } = {}) {
|
|
|
102
101
|
</style>
|
|
103
102
|
</head>
|
|
104
103
|
<body>
|
|
105
|
-
<h1>${
|
|
104
|
+
<h1>${esc(verdictBadge(r))} <span class="dim">· release score ${r.releaseScore ?? r.confidence}/100</span></h1>
|
|
106
105
|
<div class="meta">${esc(label)} · ${r.screensExplored} screens · ${r.actionsPerformed} actions · ${r.findingCounts.total} finding(s)</div>
|
|
107
106
|
<div class="headline">${esc(r.headline)}</div>
|
|
108
107
|
<h2>Findings</h2>
|
|
@@ -114,7 +113,7 @@ ${findingsHtml}
|
|
|
114
113
|
${shotsHtml || "<p class='dim'>No screenshots captured.</p>"}
|
|
115
114
|
</div>
|
|
116
115
|
${videoHtml}
|
|
117
|
-
<footer>Generated by <a href="https://github.com/aarwitz/tapp">Tapp</a> —
|
|
116
|
+
<footer>Generated by <a href="https://github.com/aarwitz/tapp">Tapp</a> — deterministic automated checks with explicit coverage limits. Review unchecked product risks before release.</footer>
|
|
118
117
|
</body>
|
|
119
118
|
</html>
|
|
120
119
|
`;
|
package/mcp-server/src/index.js
CHANGED
|
@@ -12,7 +12,7 @@ import {
|
|
|
12
12
|
ListToolsRequestSchema,
|
|
13
13
|
} from "@modelcontextprotocol/sdk/types.js";
|
|
14
14
|
|
|
15
|
-
import { parseOcqaMarkers, buildQaReport, computeRegression } from "./report.js";
|
|
15
|
+
import { parseOcqaMarkers, buildQaReport, computeRegression, verdictBadge } from "./report.js";
|
|
16
16
|
import { existingProjectArtifactPath, projectArtifactDirectory } from "./project-paths.js";
|
|
17
17
|
|
|
18
18
|
const __filename = fileURLToPath(import.meta.url);
|
|
@@ -1289,7 +1289,6 @@ function errorResult(message, details = {}) {
|
|
|
1289
1289
|
// programmatic use. This is what makes Tapp feel like a modern dev harness
|
|
1290
1290
|
// ("Explored 14 screens · 3 issues · ship: caution") rather than a wall of JSON.
|
|
1291
1291
|
const SEV = { critical: "🔴", high: "🟠", medium: "🟡", low: "⚪️" };
|
|
1292
|
-
const VERDICT_BADGE = { ready: "🟢 SHIP-READY", caution: "🟡 CAUTION", blocked: "🔴 BLOCKED" };
|
|
1293
1292
|
|
|
1294
1293
|
/** Result with a human-readable text block first and structured data attached for the agent. */
|
|
1295
1294
|
function richResult(text, structured) {
|
|
@@ -1304,9 +1303,24 @@ function fmtDuration(ms) {
|
|
|
1304
1303
|
}
|
|
1305
1304
|
|
|
1306
1305
|
/** Format a QA report as a scannable release readout with next-step suggestions. */
|
|
1307
|
-
function
|
|
1306
|
+
export function qaNextSteps(report, surface = "mcp") {
|
|
1307
|
+
if (surface === "cli") {
|
|
1308
|
+
const next = [];
|
|
1309
|
+
if (report?.findings?.length) next.push("inspect the evidence with `tapp report latest`");
|
|
1310
|
+
next.push("re-run with `--baseline <report.json>` to gate a fix");
|
|
1311
|
+
next.push("replay a committed journey with `tapp flow run <file>`");
|
|
1312
|
+
return next;
|
|
1313
|
+
}
|
|
1314
|
+
const next = [];
|
|
1315
|
+
if (report?.findings?.length) next.push("open a flagged screen with `tapp_open_app`");
|
|
1316
|
+
next.push("re-run with `baselineFindings` to gate a fix");
|
|
1317
|
+
next.push("drive it step-by-step via `tapp_session_start`");
|
|
1318
|
+
return next;
|
|
1319
|
+
}
|
|
1320
|
+
|
|
1321
|
+
function formatQaReport(report, { regression, inputHint, timedOut, bundleId, aiConfigured, reportHtml, recording, uiMap, surface = "mcp" } = {}) {
|
|
1308
1322
|
const c = report.findingCounts || {};
|
|
1309
|
-
const badge =
|
|
1323
|
+
const badge = verdictBadge(report);
|
|
1310
1324
|
const sevBits = ["critical", "high", "medium", "low"]
|
|
1311
1325
|
.map((k) => (c[k] ? `${SEV[k]} ${c[k]} ${k}` : null))
|
|
1312
1326
|
.filter(Boolean)
|
|
@@ -1359,10 +1373,7 @@ function formatQaReport(report, { regression, inputHint, timedOut, bundleId, aiC
|
|
|
1359
1373
|
}
|
|
1360
1374
|
}
|
|
1361
1375
|
}
|
|
1362
|
-
const next =
|
|
1363
|
-
if (report.findings && report.findings.length) next.push("open a flagged screen with `tapp_open_app`");
|
|
1364
|
-
next.push("re-run with `baselineFindings` to gate a fix");
|
|
1365
|
-
next.push("drive it step-by-step via `tapp_session_start`");
|
|
1376
|
+
const next = qaNextSteps(report, surface);
|
|
1366
1377
|
L.push("");
|
|
1367
1378
|
L.push(`**Next** — ${next.join(" · ")}`);
|
|
1368
1379
|
// The gate hook belongs at the moment the user thinks "I want this on every PR" —
|
|
@@ -1428,7 +1439,7 @@ export function formatScreen(screenTitle, elements) {
|
|
|
1428
1439
|
// `tapp` CLI verbs in bin/tapp.js — same pattern as report.js. Keep orchestration HERE so
|
|
1429
1440
|
// the surfaces can't drift.)
|
|
1430
1441
|
|
|
1431
|
-
export async function runQaWeb({ url, maxActions, timeout, testEmail, testPassword, baselineFindings, seedRoutes = [], seedTargets = [], onProgress = () => {} }) {
|
|
1442
|
+
export async function runQaWeb({ url, maxActions, timeout, testEmail, testPassword, baselineFindings, seedRoutes = [], seedTargets = [], surface = "mcp", onProgress = () => {} }) {
|
|
1432
1443
|
const actions = Math.max(1, Math.min(1000, asInteger(maxActions, 60)));
|
|
1433
1444
|
const timeoutSec = Math.max(30, Math.min(3600, asInteger(timeout, 600)));
|
|
1434
1445
|
const id = "web-" + new Date().toISOString().replace(/[-:T]/g, "").slice(0, 14).replace(/^(\d{8})/, "$1-");
|
|
@@ -1465,11 +1476,11 @@ export async function runQaWeb({ url, maxActions, timeout, testEmail, testPasswo
|
|
|
1465
1476
|
reportHtml = writeHtmlReport(outDir, { report, label: url.trim() });
|
|
1466
1477
|
} catch { /* evidence page is best-effort */ }
|
|
1467
1478
|
const structured = { ...report, regression, platform: "web", uiMap, reportHtml, exploration: { seedRoutes: webResult.seedRoutes || [], targets: webResult.seedTargets || [] }, capture: { id, path: outDir, relativePath: path.relative(repoRoot, outDir) } };
|
|
1468
|
-
const text = formatQaReport(report, { regression, bundleId: url.trim(), aiConfigured: !!backend, reportHtml, uiMap: uiMap.error ? null : uiMap });
|
|
1479
|
+
const text = formatQaReport(report, { regression, bundleId: url.trim(), aiConfigured: !!backend, reportHtml, uiMap: uiMap.error ? null : uiMap, surface });
|
|
1469
1480
|
return { structured, text };
|
|
1470
1481
|
}
|
|
1471
1482
|
|
|
1472
|
-
export async function runQaAndroid({ appId, apkPath, serial, maxActions, timeout, testEmail, testPassword, baselineFindings, clearData = true, seedTargets = [], onProgress = () => {} }) {
|
|
1483
|
+
export async function runQaAndroid({ appId, apkPath, serial, maxActions, timeout, testEmail, testPassword, baselineFindings, clearData = true, seedTargets = [], surface = "mcp", onProgress = () => {} }) {
|
|
1473
1484
|
const actions = Math.max(1, Math.min(1000, asInteger(maxActions, 60)));
|
|
1474
1485
|
const timeoutSec = Math.max(30, Math.min(3600, asInteger(timeout, 600)));
|
|
1475
1486
|
const id = "android-" + new Date().toISOString().replace(/[-:T]/g, "").slice(0, 14).replace(/^(\d{8})/, "$1-");
|
|
@@ -1508,11 +1519,11 @@ export async function runQaAndroid({ appId, apkPath, serial, maxActions, timeout
|
|
|
1508
1519
|
reportHtml = writeHtmlReport(outDir, { report, label: appId.trim() });
|
|
1509
1520
|
} catch {}
|
|
1510
1521
|
const structured = { ...report, regression, platform: "android", uiMap, reportHtml, exploration: { targets: androidResult.seedTargets || [] }, capture: { id, path: outDir, relativePath: path.relative(repoRoot, outDir) } };
|
|
1511
|
-
const text = formatQaReport(report, { regression, bundleId: appId.trim(), aiConfigured: !!backend, reportHtml, uiMap: uiMap.error ? null : uiMap });
|
|
1522
|
+
const text = formatQaReport(report, { regression, bundleId: appId.trim(), aiConfigured: !!backend, reportHtml, uiMap: uiMap.error ? null : uiMap, surface });
|
|
1512
1523
|
return { structured, text };
|
|
1513
1524
|
}
|
|
1514
1525
|
|
|
1515
|
-
export async function runQaIos({ bundleId, maxActions, timeout, args = {}, onProgress = () => {} }) {
|
|
1526
|
+
export async function runQaIos({ bundleId, maxActions, timeout, args = {}, surface = "mcp", onProgress = () => {} }) {
|
|
1516
1527
|
const captureScript = path.join(scriptsDir, "quick-capture.sh");
|
|
1517
1528
|
if (!fs.existsSync(captureScript)) return { error: "Capture script not found", details: { captureScript } };
|
|
1518
1529
|
|
|
@@ -1576,7 +1587,7 @@ export async function runQaIos({ bundleId, maxActions, timeout, args = {}, onPro
|
|
|
1576
1587
|
timedOut,
|
|
1577
1588
|
autoBooted: sim.autoBooted || false,
|
|
1578
1589
|
};
|
|
1579
|
-
const text = formatQaReport(report, { regression, inputHint, timedOut, bundleId, aiConfigured: !!backend, reportHtml, recording, uiMap: uiMap.error ? null : uiMap });
|
|
1590
|
+
const text = formatQaReport(report, { regression, inputHint, timedOut, bundleId, aiConfigured: !!backend, reportHtml, recording, uiMap: uiMap.error ? null : uiMap, surface });
|
|
1580
1591
|
return { structured, text };
|
|
1581
1592
|
}
|
|
1582
1593
|
|
package/mcp-server/src/report.js
CHANGED
|
@@ -86,6 +86,7 @@ export const ISSUE_CATEGORY = {
|
|
|
86
86
|
submit_failed: "unresponsive_element",
|
|
87
87
|
error_surface: "network_error_surface",
|
|
88
88
|
unresponsive_element: "unresponsive_element",
|
|
89
|
+
placeholder_link: "broken_link",
|
|
89
90
|
dead_end: "navigation_dead_end",
|
|
90
91
|
navigation_loop: "repeated_loop",
|
|
91
92
|
navigation_trap: "navigation_dead_end",
|
|
@@ -100,6 +101,11 @@ export function severityRank(s) {
|
|
|
100
101
|
return { critical: 0, high: 1, medium: 2, low: 3 }[s] ?? 4;
|
|
101
102
|
}
|
|
102
103
|
|
|
104
|
+
export function verdictBadge(report) {
|
|
105
|
+
if (report?.platform === "web" && report?.verdict === "ready") return "🟢 AUTOMATED CHECKS PASSED";
|
|
106
|
+
return { ready: "🟢 SHIP-READY", caution: "🟡 CAUTION", blocked: "🔴 BLOCKED" }[report?.verdict] || report?.verdict;
|
|
107
|
+
}
|
|
108
|
+
|
|
103
109
|
// Turn a capture's OCQA markers into the same ship/no-ship report Tapp produces:
|
|
104
110
|
// deduped findings + a trustworthy verdict with a coverage floor (mirrors OrchestratorService).
|
|
105
111
|
export function buildQaReport(markersFilePath, { platform = "ios" } = {}) {
|
|
@@ -152,11 +158,31 @@ export function buildQaReport(markersFilePath, { platform = "ios" } = {}) {
|
|
|
152
158
|
|
|
153
159
|
const inputFieldsEncountered = Array.from(inputsByScreen.entries()).map(([screen, fields]) => ({ screen, fields }));
|
|
154
160
|
|
|
161
|
+
// Web resource failures belong to the resource, not every route that referenced it.
|
|
162
|
+
// Chromium can also surface one 404 through both response and requestfailed listeners;
|
|
163
|
+
// keep the concrete missing-asset finding and discard that transport-level duplicate.
|
|
164
|
+
const normalizedIssues = rawIssues.map((issue) => {
|
|
165
|
+
if (platform !== "web") return issue;
|
|
166
|
+
if (issue.type === "placeholder_link" && issue.target) return { ...issue, screen: null };
|
|
167
|
+
if (!["missing_asset", "network_error"].includes(issue.type)) return issue;
|
|
168
|
+
const title = String(issue.title || "");
|
|
169
|
+
const match = issue.type === "missing_asset"
|
|
170
|
+
? title.match(/^404 asset:\s+(\S+)/i)
|
|
171
|
+
: title.match(/^Request failed:\s+(\S+)/i);
|
|
172
|
+
const resource = String(issue.target || match?.[1] || "").replace(/[?#].*$/, "");
|
|
173
|
+
return resource ? { ...issue, screen: null, target: resource } : issue;
|
|
174
|
+
});
|
|
175
|
+
const missingResources = new Set(normalizedIssues
|
|
176
|
+
.filter((issue) => issue.type === "missing_asset" && issue.target)
|
|
177
|
+
.map((issue) => issue.target));
|
|
178
|
+
const reportIssues = normalizedIssues.filter((issue) =>
|
|
179
|
+
!(issue.type === "network_error" && issue.target && missingResources.has(issue.target) && /^Request failed:/i.test(String(issue.title || ""))));
|
|
180
|
+
|
|
155
181
|
// Dedup by stable signature (type|screen|target) so repeated detections count once —
|
|
156
182
|
// but DIFFERENT controls failing on the same screen each count.
|
|
157
183
|
const seen = new Set();
|
|
158
184
|
const findings = [];
|
|
159
|
-
for (const i of
|
|
185
|
+
for (const i of reportIssues) {
|
|
160
186
|
const key = `${i.type}|${i.screen}|${i.target ?? ""}`;
|
|
161
187
|
if (seen.has(key)) continue;
|
|
162
188
|
seen.add(key);
|
|
@@ -187,7 +213,9 @@ export function buildQaReport(markersFilePath, { platform = "ios" } = {}) {
|
|
|
187
213
|
const headline = inconclusive
|
|
188
214
|
? `Inconclusive — only ${screensExplored} screen(s) / ${actionsPerformed} action(s) explored. The app may have crashed on launch, be stuck behind a sign-in wall, or otherwise prevent exploration. Absence of issues is NOT a pass.`
|
|
189
215
|
: verdict === "ready"
|
|
190
|
-
?
|
|
216
|
+
? platform === "web"
|
|
217
|
+
? "Automated web checks passed — no release-blocking technical issues found in the exercised surfaces. This is not a content, privacy, brand, or business-claim review."
|
|
218
|
+
: "Ship-ready — no release-blocking issues found."
|
|
191
219
|
: verdict === "caution"
|
|
192
220
|
? `Proceed with caution — ${findings.length} issue(s) to review.`
|
|
193
221
|
: `Not ready — ${findings.length} issue(s): ${crit} critical, ${high} high, ${med} medium, ${low} low.`;
|
|
@@ -202,11 +230,14 @@ export function buildQaReport(markersFilePath, { platform = "ios" } = {}) {
|
|
|
202
230
|
if (platform === "web") {
|
|
203
231
|
checkedFor = [
|
|
204
232
|
"page errors (uncaught exceptions)", "failed/5xx requests", "broken links (404)",
|
|
205
|
-
"dead buttons", "error text on pages", "load timeouts",
|
|
233
|
+
"placeholder links with no destination", "dead buttons", "error text on pages", "load timeouts",
|
|
206
234
|
];
|
|
207
235
|
notChecked = [
|
|
208
236
|
"app-specific business logic (cover with Flows: record or generate, then assert)",
|
|
209
|
-
"
|
|
237
|
+
"content and claim accuracy (including copy versus API data)",
|
|
238
|
+
"privacy or API data minimization",
|
|
239
|
+
"brand and SEO consistency",
|
|
240
|
+
"visual credibility or asset quality (vision review; needs an API key)",
|
|
210
241
|
"only the first few visible buttons per page are probed (web beta)",
|
|
211
242
|
"content & reachability regressions require a baseline",
|
|
212
243
|
];
|
|
@@ -20,11 +20,210 @@ import path from "path";
|
|
|
20
20
|
import { createRequire } from "module";
|
|
21
21
|
import { execFileSync } from "child_process";
|
|
22
22
|
|
|
23
|
-
const SETTLE_MS = 500;
|
|
24
23
|
const CLICK_SETTLE_MS = 700;
|
|
25
24
|
const NAV_TIMEOUT_MS = 15_000;
|
|
26
25
|
const BUTTONS_PER_PAGE = 4;
|
|
27
26
|
const ERROR_TEXT_RE = /\b(something went wrong|internal server error|an error occurred|failed to load|unhandled exception)\b/i;
|
|
27
|
+
const STANDALONE_ERROR_TEXT_RE = /^(something went wrong|internal server error|an error occurred|failed to load|unhandled exception)(?:[.!:]|\s|$)/i;
|
|
28
|
+
|
|
29
|
+
export function webErrorSurfaceText({ alertText = "", candidateTexts = [] } = {}) {
|
|
30
|
+
const alert = String(alertText || "").trim();
|
|
31
|
+
if (alert && ERROR_TEXT_RE.test(alert)) return alert;
|
|
32
|
+
return (candidateTexts || [])
|
|
33
|
+
.map((text) => String(text || "").trim())
|
|
34
|
+
.find((text) => STANDALONE_ERROR_TEXT_RE.test(text)) || "";
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
export function webControlLabel({ text = "", value = "", ariaLabel = "", title = "", id = "" } = {}) {
|
|
38
|
+
return [text, value, ariaLabel, title, id]
|
|
39
|
+
.map((part) => String(part || "").trim())
|
|
40
|
+
.find(Boolean) || "button";
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
export function webPlaceholderLinkFindings(links = []) {
|
|
44
|
+
const findings = [];
|
|
45
|
+
const seen = new Set();
|
|
46
|
+
for (const link of links || []) {
|
|
47
|
+
const rawHref = String(link?.rawHref || "").trim().toLowerCase();
|
|
48
|
+
const placeholder = rawHref === "#" || /^javascript:(?:void\(0\);?|;?)$/.test(rawHref);
|
|
49
|
+
if (!placeholder || link?.handlerHint) continue;
|
|
50
|
+
const label = String(link?.label || "").replace(/\s+/g, " ").trim().slice(0, 100);
|
|
51
|
+
const fingerprint = String(link?.fingerprint || "link").replace(/\s+/g, " ").trim().slice(0, 100) || "link";
|
|
52
|
+
const target = label || `unlabeled:${fingerprint}`;
|
|
53
|
+
if (seen.has(target)) continue;
|
|
54
|
+
seen.add(target);
|
|
55
|
+
findings.push({
|
|
56
|
+
type: "placeholder_link",
|
|
57
|
+
severity: label ? "medium" : "low",
|
|
58
|
+
title: label
|
|
59
|
+
? `Link "${label}" has no destination (${rawHref === "#" ? 'href="#"' : `href="${rawHref}"`})`
|
|
60
|
+
: `Unlabeled link has no destination (${rawHref === "#" ? 'href="#"' : `href="${rawHref}"`})`,
|
|
61
|
+
target,
|
|
62
|
+
});
|
|
63
|
+
}
|
|
64
|
+
return findings;
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
export function webControlHadEffect({ wired = false, before = {}, after = {} } = {}) {
|
|
68
|
+
if (wired) return true;
|
|
69
|
+
return ["url", "title", "heading", "dialogs", "local"]
|
|
70
|
+
.some((key) => String(before?.[key] ?? "") !== String(after?.[key] ?? ""));
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
export function shouldReportWebRequestFailure(errorText = "") {
|
|
74
|
+
// Chromium emits ERR_ABORTED when Tapp deliberately leaves a page while images/video are
|
|
75
|
+
// still loading. That is navigation lifecycle noise, not evidence that the resource is broken.
|
|
76
|
+
return !/\bnet::ERR_ABORTED\b/i.test(String(errorText));
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
export function webPageAppearsBlank({ textLen = 0, controlCount = 0, visualContentCount = 0 } = {}) {
|
|
80
|
+
return Number(textLen) === 0 && Number(controlCount) === 0 && Number(visualContentCount) === 0;
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
async function installWebListenerTracking(context) {
|
|
84
|
+
await context.addInitScript(() => {
|
|
85
|
+
const key = Symbol.for("tapp.clickListeners");
|
|
86
|
+
const add = EventTarget.prototype.addEventListener;
|
|
87
|
+
const remove = EventTarget.prototype.removeEventListener;
|
|
88
|
+
EventTarget.prototype.addEventListener = function tappTrackedAdd(type, listener, options) {
|
|
89
|
+
if (type === "click" && this instanceof Element && listener) {
|
|
90
|
+
if (!this[key]) Object.defineProperty(this, key, { value: new Set(), configurable: true });
|
|
91
|
+
this[key].add(listener);
|
|
92
|
+
}
|
|
93
|
+
return add.call(this, type, listener, options);
|
|
94
|
+
};
|
|
95
|
+
EventTarget.prototype.removeEventListener = function tappTrackedRemove(type, listener, options) {
|
|
96
|
+
if (type === "click" && this instanceof Element && this[key]) this[key].delete(listener);
|
|
97
|
+
return remove.call(this, type, listener, options);
|
|
98
|
+
};
|
|
99
|
+
});
|
|
100
|
+
}
|
|
101
|
+
|
|
102
|
+
async function captureWebControlState(page, locator) {
|
|
103
|
+
const global = await page.evaluate(() => {
|
|
104
|
+
const visible = (element) => {
|
|
105
|
+
const style = window.getComputedStyle(element);
|
|
106
|
+
return style.visibility !== "hidden" && style.display !== "none" && element.getClientRects().length > 0;
|
|
107
|
+
};
|
|
108
|
+
const dialogs = [...document.querySelectorAll("dialog[open], [role=dialog], [aria-modal=true]")]
|
|
109
|
+
.filter(visible)
|
|
110
|
+
.map((element) => (element.getAttribute("aria-label") || element.textContent || "dialog").replace(/\s+/g, " ").trim().slice(0, 160))
|
|
111
|
+
.sort();
|
|
112
|
+
return {
|
|
113
|
+
title: document.title,
|
|
114
|
+
heading: document.querySelector("h1")?.textContent?.replace(/\s+/g, " ").trim() || "",
|
|
115
|
+
dialogs: JSON.stringify(dialogs),
|
|
116
|
+
};
|
|
117
|
+
}).catch(() => ({ title: "", heading: "", dialogs: "" }));
|
|
118
|
+
|
|
119
|
+
const local = await locator.evaluate((element) => {
|
|
120
|
+
const key = Symbol.for("tapp.clickListeners");
|
|
121
|
+
const visible = (candidate) => {
|
|
122
|
+
const style = window.getComputedStyle(candidate);
|
|
123
|
+
return style.visibility !== "hidden" && style.display !== "none" && candidate.getClientRects().length > 0;
|
|
124
|
+
};
|
|
125
|
+
let wired = false;
|
|
126
|
+
for (let candidate = element; candidate && candidate !== document.body; candidate = candidate.parentElement) {
|
|
127
|
+
if ((candidate[key] && candidate[key].size > 0) || typeof candidate.onclick === "function" || candidate.hasAttribute("onclick")) {
|
|
128
|
+
wired = true;
|
|
129
|
+
break;
|
|
130
|
+
}
|
|
131
|
+
}
|
|
132
|
+
if (!wired && element.matches("button[type=submit], input[type=submit]") && element.closest("form")) wired = true;
|
|
133
|
+
const describe = (root) => ({
|
|
134
|
+
tag: root.tagName,
|
|
135
|
+
className: typeof root.className === "string" ? root.className : "",
|
|
136
|
+
hidden: root.hidden,
|
|
137
|
+
open: root.hasAttribute("open"),
|
|
138
|
+
ariaExpanded: root.getAttribute("aria-expanded"),
|
|
139
|
+
ariaPressed: root.getAttribute("aria-pressed"),
|
|
140
|
+
ariaSelected: root.getAttribute("aria-selected"),
|
|
141
|
+
text: (root.textContent || "").replace(/\s+/g, " ").trim().slice(0, 300),
|
|
142
|
+
controls: [...root.querySelectorAll("button, a[href], input, textarea, select, [role=button]")]
|
|
143
|
+
.slice(0, 40)
|
|
144
|
+
.map((control) => ({
|
|
145
|
+
tag: control.tagName,
|
|
146
|
+
visible: visible(control),
|
|
147
|
+
disabled: !!control.disabled || control.getAttribute("aria-disabled") === "true",
|
|
148
|
+
checked: "checked" in control ? !!control.checked : null,
|
|
149
|
+
expanded: control.getAttribute("aria-expanded"),
|
|
150
|
+
pressed: control.getAttribute("aria-pressed"),
|
|
151
|
+
selected: control.getAttribute("aria-selected"),
|
|
152
|
+
label: (control.getAttribute("aria-label") || control.textContent || control.getAttribute("value") || "").replace(/\s+/g, " ").trim().slice(0, 80),
|
|
153
|
+
})),
|
|
154
|
+
});
|
|
155
|
+
const region = element.parentElement || element;
|
|
156
|
+
const controlledId = element.getAttribute("aria-controls");
|
|
157
|
+
const controlled = controlledId ? document.getElementById(controlledId) : null;
|
|
158
|
+
return { wired, signature: JSON.stringify([describe(region), controlled ? describe(controlled) : null]) };
|
|
159
|
+
}).catch(() => ({ wired: false, signature: "detached" }));
|
|
160
|
+
|
|
161
|
+
return { url: page.url(), ...global, local: local.signature, wired: local.wired };
|
|
162
|
+
}
|
|
163
|
+
|
|
164
|
+
// Wait for a page to stop presenting an explicit loading state and for its semantic
|
|
165
|
+
// surface to remain unchanged across a couple of samples. This is intentionally bounded:
|
|
166
|
+
// live counters and animation-heavy pages still return evidence, marked unsettled.
|
|
167
|
+
export async function waitForWebStability(page, { timeoutMs = 5_000, intervalMs = 250, stableSamples = 3 } = {}) {
|
|
168
|
+
const boundedTimeout = Math.max(250, Math.min(15_000, Number(timeoutMs) || 5_000));
|
|
169
|
+
const boundedInterval = Math.max(100, Math.min(1_000, Number(intervalMs) || 250));
|
|
170
|
+
const requiredSamples = Math.max(1, Math.min(5, Number(stableSamples) || 2));
|
|
171
|
+
const started = Date.now();
|
|
172
|
+
let previousSignature = "";
|
|
173
|
+
let matchingSamples = 0;
|
|
174
|
+
let latest = { busy: false, signature: "" };
|
|
175
|
+
|
|
176
|
+
while (Date.now() - started < boundedTimeout) {
|
|
177
|
+
latest = await page.evaluate(() => {
|
|
178
|
+
const visible = (element) => {
|
|
179
|
+
const style = window.getComputedStyle(element);
|
|
180
|
+
return style.visibility !== "hidden" && style.display !== "none" && element.getClientRects().length > 0;
|
|
181
|
+
};
|
|
182
|
+
const busySelector = "[aria-busy=true], [role=progressbar], .loading, .spinner, [class*=loading i], [class*=spinner i]";
|
|
183
|
+
const busyElement = [...document.querySelectorAll(busySelector)].some(visible);
|
|
184
|
+
const busyText = [...document.querySelectorAll("h1, h2, h3, p, [role=status]")]
|
|
185
|
+
.filter(visible)
|
|
186
|
+
.map((element) => (element.textContent || "").trim())
|
|
187
|
+
.some((text) => /^(loading|fetching|please wait|preparing|connecting)(?:[.…!]*|\s.*)$/i.test(text));
|
|
188
|
+
const bodyText = (document.body?.innerText || "").replace(/\s+/g, " ").trim().slice(0, 2_000);
|
|
189
|
+
const signature = JSON.stringify([
|
|
190
|
+
location.href,
|
|
191
|
+
document.querySelector("h1")?.textContent?.trim() || "",
|
|
192
|
+
document.title,
|
|
193
|
+
document.querySelectorAll("button, a[href], input, textarea, select, [role=button]").length,
|
|
194
|
+
bodyText,
|
|
195
|
+
]);
|
|
196
|
+
return { busy: busyElement || busyText, signature };
|
|
197
|
+
}).catch(() => latest);
|
|
198
|
+
|
|
199
|
+
if (!latest.busy && latest.signature === previousSignature) matchingSamples += 1;
|
|
200
|
+
else matchingSamples = !latest.busy ? 1 : 0;
|
|
201
|
+
previousSignature = latest.signature;
|
|
202
|
+
if (!latest.busy && matchingSamples >= requiredSamples) {
|
|
203
|
+
return { settled: true, busy: false, elapsedMs: Date.now() - started };
|
|
204
|
+
}
|
|
205
|
+
await page.waitForTimeout(boundedInterval);
|
|
206
|
+
}
|
|
207
|
+
return { settled: false, busy: !!latest.busy, elapsedMs: Date.now() - started };
|
|
208
|
+
}
|
|
209
|
+
|
|
210
|
+
async function tapWebText(page, text, timeoutMs) {
|
|
211
|
+
const requested = String(text || "").trim();
|
|
212
|
+
if (!requested) return false;
|
|
213
|
+
const candidates = [
|
|
214
|
+
page.getByRole("button", { name: requested, exact: true }).first(),
|
|
215
|
+
page.getByRole("link", { name: requested, exact: true }).first(),
|
|
216
|
+
page.getByText(requested, { exact: true }).first(),
|
|
217
|
+
];
|
|
218
|
+
for (const candidate of candidates) {
|
|
219
|
+
if (!(await candidate.isVisible().catch(() => false))) continue;
|
|
220
|
+
try {
|
|
221
|
+
await candidate.click({ timeout: Math.min(timeoutMs, 5_000) });
|
|
222
|
+
return true;
|
|
223
|
+
} catch {}
|
|
224
|
+
}
|
|
225
|
+
throw new Error(`Could not tap visible text “${requested}”`);
|
|
226
|
+
}
|
|
28
227
|
|
|
29
228
|
// npx installs Tapp into its own cache, so a plain import("playwright") only resolves
|
|
30
229
|
// for repo-dev checkouts. Probe, in order: our own node_modules; the user's project
|
|
@@ -100,7 +299,7 @@ export function webBrowserLaunchOptions(environment = process.env) {
|
|
|
100
299
|
// Focused one-screen inspection for the agent-facing `tapp open <url>` and `tapp tree <url>`
|
|
101
300
|
// commands. This deliberately does no exploration or judgment; it opens exactly one page,
|
|
102
301
|
// captures the visible semantic controls, and optionally takes one screenshot.
|
|
103
|
-
export async function inspectWebPage({ url, timeoutMs = NAV_TIMEOUT_MS, screenshot = true }) {
|
|
302
|
+
export async function inspectWebPage({ url, timeoutMs = NAV_TIMEOUT_MS, screenshot = true, tapText = "", waitForText = "" }) {
|
|
104
303
|
let target;
|
|
105
304
|
try { target = new URL(url); }
|
|
106
305
|
catch { throw new Error("Web inspection needs a valid http(s) URL"); }
|
|
@@ -115,7 +314,20 @@ export async function inspectWebPage({ url, timeoutMs = NAV_TIMEOUT_MS, screensh
|
|
|
115
314
|
page.setDefaultTimeout(boundedTimeout);
|
|
116
315
|
const response = await page.goto(target.href, { waitUntil: "domcontentloaded", timeout: boundedTimeout });
|
|
117
316
|
if (response && response.status() >= 400) throw new Error(`Could not open ${target.href}: HTTP ${response.status()}`);
|
|
118
|
-
await page.
|
|
317
|
+
let stability = await waitForWebStability(page, { timeoutMs: Math.min(5_000, boundedTimeout) });
|
|
318
|
+
if (tapText) {
|
|
319
|
+
await tapWebText(page, tapText, boundedTimeout);
|
|
320
|
+
stability = await waitForWebStability(page, { timeoutMs: Math.min(5_000, boundedTimeout) });
|
|
321
|
+
}
|
|
322
|
+
if (waitForText) {
|
|
323
|
+
const requested = String(waitForText).trim();
|
|
324
|
+
try {
|
|
325
|
+
await page.getByText(requested, { exact: false }).first().waitFor({ state: "visible", timeout: boundedTimeout });
|
|
326
|
+
} catch {
|
|
327
|
+
throw new Error(`Timed out waiting for visible text “${requested}”`);
|
|
328
|
+
}
|
|
329
|
+
stability = await waitForWebStability(page, { timeoutMs: Math.min(5_000, boundedTimeout) });
|
|
330
|
+
}
|
|
119
331
|
const observed = await page.evaluate(() => {
|
|
120
332
|
const visible = (element) => element.offsetParent !== null;
|
|
121
333
|
const controls = [...document.querySelectorAll("button, a[href], input, textarea, select, [role=button], [role=tab], [role=checkbox], [role=switch]")]
|
|
@@ -150,6 +362,8 @@ export async function inspectWebPage({ url, timeoutMs = NAV_TIMEOUT_MS, screensh
|
|
|
150
362
|
screenTitle: webScreenTitle(observed, target.pathname || target.href),
|
|
151
363
|
elements: observed.controls,
|
|
152
364
|
image,
|
|
365
|
+
settled: stability.settled,
|
|
366
|
+
busy: stability.busy,
|
|
153
367
|
};
|
|
154
368
|
} finally {
|
|
155
369
|
await browser.close().catch(() => {});
|
|
@@ -221,7 +435,9 @@ export async function exploreWeb({ url, maxActions = 40, timeoutSec = 300, outDi
|
|
|
221
435
|
|
|
222
436
|
const { chromium } = await loadPlaywright();
|
|
223
437
|
const browser = await chromium.launch(webBrowserLaunchOptions());
|
|
224
|
-
const
|
|
438
|
+
const context = await browser.newContext({ viewport: { width: 1280, height: 900 } });
|
|
439
|
+
await installWebListenerTracking(context);
|
|
440
|
+
const page = await context.newPage();
|
|
225
441
|
page.setDefaultTimeout(NAV_TIMEOUT_MS);
|
|
226
442
|
|
|
227
443
|
const deadline = Date.now() + timeoutSec * 1000;
|
|
@@ -231,14 +447,9 @@ export async function exploreWeb({ url, maxActions = 40, timeoutSec = 300, outDi
|
|
|
231
447
|
emit("ISSUE", { type, severity, title, screen, ...(target ? { target } : {}) });
|
|
232
448
|
};
|
|
233
449
|
|
|
234
|
-
// Request counter: cheap "did that click cause network activity" signal for the
|
|
235
|
-
// dead-button check (a button that fires a request is not dead).
|
|
236
|
-
let requestCount = 0;
|
|
237
|
-
|
|
238
450
|
// Async defect listeners: attribute to whatever screen is current when they fire.
|
|
239
451
|
let currentScreen = start.pathname;
|
|
240
452
|
let lastActionTarget = "";
|
|
241
|
-
page.on("request", () => { requestCount += 1; });
|
|
242
453
|
page.on("pageerror", (err) => issue("js_exception", "high", `Uncaught JS exception: ${String(err.message || err).slice(0, 120)}`, currentScreen));
|
|
243
454
|
page.on("response", (res) => {
|
|
244
455
|
try {
|
|
@@ -246,7 +457,7 @@ export async function exploreWeb({ url, maxActions = 40, timeoutSec = 300, outDi
|
|
|
246
457
|
if (u.origin !== start.origin) return;
|
|
247
458
|
if (res.status() >= 500) issue("network_error", "high", `${res.status()} from ${u.pathname.slice(0, 80)}`, currentScreen);
|
|
248
459
|
else if (res.status() === 404 && res.request().resourceType() !== "document") {
|
|
249
|
-
issue("missing_asset", "medium", `404 asset: ${u.pathname.slice(0, 80)}`, currentScreen);
|
|
460
|
+
issue("missing_asset", "medium", `404 asset: ${u.pathname.slice(0, 80)}`, currentScreen, u.pathname);
|
|
250
461
|
}
|
|
251
462
|
} catch {}
|
|
252
463
|
});
|
|
@@ -254,7 +465,9 @@ export async function exploreWeb({ url, maxActions = 40, timeoutSec = 300, outDi
|
|
|
254
465
|
try {
|
|
255
466
|
const u = new URL(req.url());
|
|
256
467
|
if (u.origin !== start.origin) return;
|
|
257
|
-
|
|
468
|
+
const errorText = req.failure()?.errorText || "?";
|
|
469
|
+
if (!shouldReportWebRequestFailure(errorText)) return;
|
|
470
|
+
issue("network_error", "medium", `Request failed: ${u.pathname.slice(0, 80)} (${errorText})`, currentScreen, u.pathname);
|
|
258
471
|
} catch {}
|
|
259
472
|
});
|
|
260
473
|
|
|
@@ -269,7 +482,8 @@ export async function exploreWeb({ url, maxActions = 40, timeoutSec = 300, outDi
|
|
|
269
482
|
: { target: start.pathname + start.search, visitKey: `pr-path:${target.id}`, action: `PR target ${target.node.name}`, fromScreen: null, prTarget: true, targetId: target.id, pathTarget: target }),
|
|
270
483
|
...normalizedSeeds.filter((target) => !targetRoutes.has(target)).map((target) => ({ target, action: `PR target ${target}`, fromScreen: null, prTarget: true })),
|
|
271
484
|
];
|
|
272
|
-
const screenshotFor = new
|
|
485
|
+
const screenshotFor = new Map();
|
|
486
|
+
const placeholderLinksSeen = new Set();
|
|
273
487
|
let actions = 0;
|
|
274
488
|
let screenCount = 0;
|
|
275
489
|
let lastScreen = null;
|
|
@@ -320,8 +534,45 @@ export async function exploreWeb({ url, maxActions = 40, timeoutSec = 300, outDi
|
|
|
320
534
|
title: document.title.trim(),
|
|
321
535
|
controlCount: document.querySelectorAll("a[href], button, [role=button], input, select, textarea").length,
|
|
322
536
|
textLen: (document.body?.innerText || "").trim().length,
|
|
323
|
-
|
|
537
|
+
visualContentCount: [...document.querySelectorAll("img, picture, video, canvas, svg, iframe, object, embed")]
|
|
538
|
+
.filter((el) => {
|
|
539
|
+
const style = window.getComputedStyle(el);
|
|
540
|
+
return style.visibility !== "hidden" && style.display !== "none" && el.getClientRects().length > 0;
|
|
541
|
+
}).length,
|
|
542
|
+
alertText: [...document.querySelectorAll("[role=alert], [aria-live=assertive]")]
|
|
324
543
|
.map((el) => el.textContent.trim()).filter(Boolean).join(" ").slice(0, 120),
|
|
544
|
+
errorCandidateTexts: [...document.querySelectorAll("h1, h2, h3, p, [data-error], [data-testid*=error i]")]
|
|
545
|
+
.filter((el) => el.offsetParent !== null)
|
|
546
|
+
.map((el) => (el.textContent || "").trim().slice(0, 240))
|
|
547
|
+
.filter(Boolean)
|
|
548
|
+
.slice(0, 40),
|
|
549
|
+
busy: [...document.querySelectorAll("[aria-busy=true], [role=progressbar], .loading, .spinner, [class*=loading i], [class*=spinner i]")]
|
|
550
|
+
.some((el) => {
|
|
551
|
+
const style = window.getComputedStyle(el);
|
|
552
|
+
return style.visibility !== "hidden" && style.display !== "none" && el.getClientRects().length > 0;
|
|
553
|
+
}) || [...document.querySelectorAll("h1, h2, h3, p, [role=status]")]
|
|
554
|
+
.filter((el) => el.offsetParent !== null)
|
|
555
|
+
.map((el) => (el.textContent || "").trim())
|
|
556
|
+
.some((text) => /^(loading|fetching|please wait|preparing|connecting)(?:[.…!]*|\s.*)$/i.test(text)),
|
|
557
|
+
placeholderLinks: [...document.querySelectorAll("a[href]")]
|
|
558
|
+
.filter((el) => el.offsetParent !== null)
|
|
559
|
+
.map((el, index) => {
|
|
560
|
+
const listenerKey = Symbol.for("tapp.clickListeners");
|
|
561
|
+
const dataHandler = [...el.attributes]
|
|
562
|
+
.some((attribute) => /^data-(action|toggle|target|modal|waitlist)(?:-|$)/i.test(attribute.name));
|
|
563
|
+
const svgPath = el.querySelector("svg path")?.getAttribute("d") || "";
|
|
564
|
+
return {
|
|
565
|
+
rawHref: el.getAttribute("href") || "",
|
|
566
|
+
label: (el.getAttribute("aria-label") || el.getAttribute("title") || el.innerText || el.textContent || "").replace(/\s+/g, " ").trim().slice(0, 100),
|
|
567
|
+
handlerHint: el.getAttribute("role") === "button"
|
|
568
|
+
|| el.hasAttribute("onclick")
|
|
569
|
+
|| typeof el.onclick === "function"
|
|
570
|
+
|| !!el[listenerKey]?.size
|
|
571
|
+
|| el.hasAttribute("aria-controls")
|
|
572
|
+
|| dataHandler,
|
|
573
|
+
fingerprint: el.id || el.getAttribute("data-testid") || svgPath.slice(0, 80) || `link-${index + 1}`,
|
|
574
|
+
};
|
|
575
|
+
}),
|
|
325
576
|
inputs,
|
|
326
577
|
controls,
|
|
327
578
|
};
|
|
@@ -332,7 +583,7 @@ export async function exploreWeb({ url, maxActions = 40, timeoutSec = 300, outDi
|
|
|
332
583
|
const screen = webScreenTitle(info, key);
|
|
333
584
|
const evidenceKey = `${key}::${screen}`;
|
|
334
585
|
currentScreen = screen;
|
|
335
|
-
emit("STATE", { screen, url: key, elements: info.controlCount, role: webScreenRole(screen, info.inputs), controls: info.controls, inputs: info.inputs, settled:
|
|
586
|
+
emit("STATE", { screen, url: key, elements: info.controlCount, role: webScreenRole(screen, info.inputs), controls: info.controls, inputs: info.inputs, settled: !info.busy });
|
|
336
587
|
const completedNavigation = pendingNavigation;
|
|
337
588
|
const transitionFrom = webTransitionOrigin(completedNavigation, lastScreen);
|
|
338
589
|
const transitionAction = completedNavigation?.action || lastActionTarget;
|
|
@@ -342,15 +593,28 @@ export async function exploreWeb({ url, maxActions = 40, timeoutSec = 300, outDi
|
|
|
342
593
|
lastScreen = screen;
|
|
343
594
|
if (completedNavigation?.prTarget) emit("PR_TARGET", { ...(completedNavigation.targetId ? { targetId: completedNavigation.targetId } : {}), route: completedNavigation.target, status: "observed", screen });
|
|
344
595
|
|
|
345
|
-
|
|
346
|
-
screenshotFor.
|
|
596
|
+
const busyRouteEntry = !info.busy
|
|
597
|
+
? [...screenshotFor.entries()].find(([, value]) => value.route === key && value.busy)
|
|
598
|
+
: null;
|
|
599
|
+
const existingKey = screenshotFor.has(evidenceKey) ? evidenceKey : busyRouteEntry?.[0];
|
|
600
|
+
const existingScreenshot = existingKey ? screenshotFor.get(existingKey) : null;
|
|
601
|
+
const shouldCapture = !existingScreenshot || (existingScreenshot.busy && !info.busy);
|
|
602
|
+
if (shouldCapture) {
|
|
603
|
+
const screenshotPath = existingScreenshot?.path || path.join(outDir, `state_${screenshotFor.size + 1}_${slug(screen)}.png`);
|
|
604
|
+
if (existingKey && existingKey !== evidenceKey) screenshotFor.delete(existingKey);
|
|
605
|
+
screenshotFor.set(evidenceKey, { path: screenshotPath, busy: info.busy, route: key });
|
|
347
606
|
screenCount = screenshotFor.size;
|
|
348
|
-
await page.screenshot({ path:
|
|
607
|
+
await page.screenshot({ path: screenshotPath }).catch(() => {});
|
|
349
608
|
// Deterministic per-page detectors run once per distinct screen.
|
|
350
|
-
if (info
|
|
351
|
-
else
|
|
352
|
-
|
|
353
|
-
issue("error_surface", "high",
|
|
609
|
+
if (webPageAppearsBlank(info)) issue("blank_screen", "high", "Page rendered no visible content", screen);
|
|
610
|
+
else {
|
|
611
|
+
const errorText = webErrorSurfaceText({ alertText: info.alertText, candidateTexts: info.errorCandidateTexts });
|
|
612
|
+
if (errorText) issue("error_surface", "high", `Error shown: ${errorText.slice(0, 80)}`, screen);
|
|
613
|
+
}
|
|
614
|
+
for (const finding of webPlaceholderLinkFindings(info.placeholderLinks)) {
|
|
615
|
+
if (placeholderLinksSeen.has(finding.target)) continue;
|
|
616
|
+
placeholderLinksSeen.add(finding.target);
|
|
617
|
+
issue(finding.type, finding.severity, finding.title, screen, finding.target);
|
|
354
618
|
}
|
|
355
619
|
}
|
|
356
620
|
return { key, screen, info };
|
|
@@ -370,7 +634,7 @@ export async function exploreWeb({ url, maxActions = 40, timeoutSec = 300, outDi
|
|
|
370
634
|
emit("ACTION", { type: "login", target: "Sign in", screen, narrative: "Filled and submitted the sign-in form with the provided test credentials" });
|
|
371
635
|
actions += 1;
|
|
372
636
|
await submitWebLogin(page).catch(() => false);
|
|
373
|
-
await page.
|
|
637
|
+
await waitForWebStability(page, { timeoutMs: Math.min(5_000, CLICK_SETTLE_MS * 6) });
|
|
374
638
|
// Still on the login form after a submit = the sign-in failed — full stop. (A quiet
|
|
375
639
|
// credential rejection often shows NO other symptom, so this must not be coupled to
|
|
376
640
|
// whether some other detector happened to fire during the attempt.)
|
|
@@ -417,7 +681,6 @@ export async function exploreWeb({ url, maxActions = 40, timeoutSec = 300, outDi
|
|
|
417
681
|
// we just left; observe() refines this to the page title once it settles.
|
|
418
682
|
currentScreen = target;
|
|
419
683
|
const nav = await page.goto(start.origin + target, { waitUntil: "domcontentloaded" }).catch((err) => ({ navError: String(err.message || err) }));
|
|
420
|
-
await page.waitForTimeout(SETTLE_MS);
|
|
421
684
|
if (nav && nav.navError) {
|
|
422
685
|
if (entry.prTarget) emit("PR_TARGET", { ...(entry.targetId ? { targetId: entry.targetId } : {}), ...(entry.pathTarget ? {} : { route: target }), status: "failed", error: nav.navError.slice(0, 160) });
|
|
423
686
|
pendingNavigation = null;
|
|
@@ -425,6 +688,7 @@ export async function exploreWeb({ url, maxActions = 40, timeoutSec = 300, outDi
|
|
|
425
688
|
progress();
|
|
426
689
|
continue;
|
|
427
690
|
}
|
|
691
|
+
await waitForWebStability(page);
|
|
428
692
|
if (nav && typeof nav.status === "function" && nav.status() === 404) {
|
|
429
693
|
issue("broken_link", "medium", `Broken link: ${target} → 404`, target);
|
|
430
694
|
}
|
|
@@ -456,15 +720,17 @@ export async function exploreWeb({ url, maxActions = 40, timeoutSec = 300, outDi
|
|
|
456
720
|
else if (selector.kind === "label") locator = page.getByText(selector.value, { exact: true }).first();
|
|
457
721
|
else continue;
|
|
458
722
|
if (await locator.isVisible().catch(() => false)) {
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
723
|
+
try {
|
|
724
|
+
await locator.click({ timeout: Math.min(step.wait?.timeoutMs || NAV_TIMEOUT_MS, NAV_TIMEOUT_MS) });
|
|
725
|
+
acted = true;
|
|
726
|
+
break;
|
|
727
|
+
} catch {}
|
|
462
728
|
}
|
|
463
729
|
}
|
|
464
730
|
}
|
|
465
731
|
if (!acted) { pathError = `Observed control was not found: ${action.target}`; break; }
|
|
466
732
|
pendingNavigation = { action: action.target, fromScreen: beforeScreen };
|
|
467
|
-
await page
|
|
733
|
+
await waitForWebStability(page);
|
|
468
734
|
ob = await observe() || ob;
|
|
469
735
|
progress();
|
|
470
736
|
}
|
|
@@ -502,37 +768,40 @@ export async function exploreWeb({ url, maxActions = 40, timeoutSec = 300, outDi
|
|
|
502
768
|
const n = Math.min(await buttons.count().catch(() => 0), BUTTONS_PER_PAGE);
|
|
503
769
|
for (let i = 0; i < n && actions < maxActions && Date.now() < deadline; i++) {
|
|
504
770
|
const b = buttons.nth(i);
|
|
505
|
-
const label = (
|
|
771
|
+
const label = webControlLabel({
|
|
772
|
+
text: await b.textContent().catch(() => ""),
|
|
773
|
+
value: await b.getAttribute("value").catch(() => ""),
|
|
774
|
+
ariaLabel: await b.getAttribute("aria-label").catch(() => ""),
|
|
775
|
+
title: await b.getAttribute("title").catch(() => ""),
|
|
776
|
+
id: await b.getAttribute("id").catch(() => ""),
|
|
777
|
+
}).slice(0, 40);
|
|
506
778
|
if (/log ?out|sign ?out|delete|remove/i.test(label)) continue; // don't destroy test state
|
|
507
|
-
|
|
508
|
-
//
|
|
509
|
-
//
|
|
510
|
-
|
|
511
|
-
await page
|
|
512
|
-
.evaluate(() => {
|
|
513
|
-
window.__tappMut = 0;
|
|
514
|
-
if (window.__tappMo) window.__tappMo.disconnect();
|
|
515
|
-
window.__tappMo = new MutationObserver((muts) => { window.__tappMut += muts.length; });
|
|
516
|
-
window.__tappMo.observe(document.body, { childList: true, subtree: true, attributes: true, characterData: true });
|
|
517
|
-
})
|
|
518
|
-
.catch(() => {});
|
|
519
|
-
const dialogsBefore = await page.locator("dialog[open], [role=dialog], [aria-modal=true]").count().catch(() => 0);
|
|
520
|
-
const reqBefore = requestCount;
|
|
779
|
+
// Capture only durable, user-visible semantics around this control. A global
|
|
780
|
+
// MutationObserver is intentionally avoided: carousels, chat launchers, and live
|
|
781
|
+
// counters can mutate while an unrelated dead button is clicked, creating verdict jitter.
|
|
782
|
+
const beforeState = await captureWebControlState(page, b);
|
|
521
783
|
actions += 1;
|
|
522
784
|
lastActionTarget = label;
|
|
523
785
|
emit("ACTION", { type: "tap", target: label, screen: webActionScreen(ob), narrative: `Tapped "${label}"` });
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
786
|
+
let clickSucceeded = false;
|
|
787
|
+
try {
|
|
788
|
+
await b.click({ timeout: 3000 });
|
|
789
|
+
clickSucceeded = true;
|
|
790
|
+
} catch {}
|
|
791
|
+
if (!clickSucceeded) {
|
|
792
|
+
progress();
|
|
793
|
+
continue;
|
|
794
|
+
}
|
|
795
|
+
await waitForWebStability(page);
|
|
796
|
+
if (page.url() !== beforeState.url) {
|
|
527
797
|
await observe();
|
|
528
798
|
await page.goBack({ waitUntil: "domcontentloaded" }).catch(() => {});
|
|
529
|
-
await page
|
|
799
|
+
await waitForWebStability(page);
|
|
530
800
|
} else {
|
|
531
|
-
const
|
|
532
|
-
const
|
|
533
|
-
const hadEffect = mutations > 0 || dialogsAfter !== dialogsBefore || requestCount > reqBefore;
|
|
801
|
+
const afterState = await captureWebControlState(page, b);
|
|
802
|
+
const hadEffect = webControlHadEffect({ wired: beforeState.wired, before: beforeState, after: afterState });
|
|
534
803
|
if (!hadEffect) {
|
|
535
|
-
issue("unresponsive_element", "medium", `Button "${label}"
|
|
804
|
+
issue("unresponsive_element", "medium", `Button "${label}" has no wiring or observable effect`, ob.screen, label);
|
|
536
805
|
} else {
|
|
537
806
|
// Same-URL SPA transitions are real screens too; URL-only observation
|
|
538
807
|
// under-counted coverage and made healthy applications inconclusive.
|
package/package.json
CHANGED