@appliqation/visual-regression 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Appliqation Pty Ltd
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md ADDED
@@ -0,0 +1,87 @@
1
+ # Appliqation Visual-Regression
2
+
3
+ **Checks one route for a real visual regression by diffing it against its own live production counterpart — no stored baseline files, no manual baseline-approval workflow.**
4
+
5
+ Point it at a route and a test case — it navigates to that route on production and a target environment, masks any known dynamic regions, full-page screenshots both, pixel-diffs them for real, and has a model judge the result from the actual evidence: a genuine regression, an expected divergence, or a route that simply doesn't exist on production yet (declines rather than substituting any other baseline).
6
+
7
+ ## Why this exists
8
+
9
+ A Playwright `getByRole` assertion can pass while the page looks broken — an overlapping modal, a button pushed off-screen by a CSS regression, invisible text from a contrast bug. Nothing else in this agent family checks appearance, only behaviour. Traditional visual-regression tooling solves this with a stored baseline image that a human has to keep re-approving every time a legitimate UI change ships — in practice, that maintenance burden is what kills most setups. This agent sidesteps it entirely by using **production itself as the baseline, fetched live at comparison time**. Production always represents current truth by definition; there's nothing to store or re-approve.
10
+
11
+ ## The one rule that matters more than anything else here
12
+
13
+ **If the route doesn't exist on production, that's not a failure — it's not applicable.** This agent never falls back to comparing against a design file or mock (a live render vs. a Figma file is a different, unreliable problem — design fidelity, not regression detection). No production equivalent means no comparison, reported plainly, nothing forced.
14
+
15
+ The mechanical work — navigating both environments, masking, screenshotting, pixel-diffing — is entirely code-owned, never something the model claims: it happens inside one atomic `capture_and_diff` call, and the real diff statistics it returns are what the model is required to cite, never a number it asserts on its own. The model's only real job is judgment, reported back through a structured `submit_verdict` call rather than free-text prose — the same discipline `appq:autotest-validator` already uses for its own verdicts.
16
+
17
+ ## Quick start
18
+
19
+ ```bash
20
+ npm install -g @appliqation/visual-regression
21
+ npx playwright install chromium
22
+ ```
23
+
24
+ Create a `.env` file (in whatever directory you'll run it from) with:
25
+
26
+ ```
27
+ APPQ_API_KEY=your-appliqation-api-key # read-only is enough
28
+ ANTHROPIC_API_KEY=your-anthropic-key # or OPENAI_API_KEY — pick one
29
+ ```
30
+
31
+ ```bash
32
+ appliqation-visual-regression check \
33
+ --test-case-uuid 1350-2732cd99-81d6-44ce-a053-1aa2e2efc42c \
34
+ --route /subscribe \
35
+ --baseline-environment Prod \
36
+ --target-environment Stage \
37
+ --mask ".reader-count" \
38
+ --mask "[data-testid=timestamp]"
39
+ ```
40
+
41
+ Add `--json`/`--ci` for a structured summary. The exit code is 0 for `expected-divergence`/`not-applicable`, 1 for `regression`/`inconclusive` (fail-closed on ambiguity) — the JSON summary's `verdict` field is what actually distinguishes the outcomes, not the exit code alone.
42
+
43
+ ## What this agent does not do (on purpose)
44
+
45
+ - **No route enumeration or inference.** `--route` is always explicit — there's no structured route data on a scenario/test case to derive it from (confirmed: routes only ever exist inside free-text step descriptions). A real caller with a just-completed run derives it from real observed navigation data (`get_execution_evidence`), never by guessing at step text.
46
+ - **No auto-detection of dynamic regions.** `--mask` is caller-supplied CSS selectors only. The model already has to reason about "is this difference data-driven or a real break" regardless, so masking is an optimization, not a prerequisite.
47
+ - **No write capability.** This agent never calls an Appliqation write tool and never files a defect — it reports a verdict, nothing else. What happens to a confirmed regression (or a secondary observation) is entirely the caller's decision.
48
+ - **No ID/slug-based dynamic-content routes.** `/blog/123` on staging has no reliable way to be matched to its "equivalent" content on production (`/blog/345`) without this agent guessing at content equivalence — a single `--route` assumes path identity means content identity. Static, stable routes only.
49
+ - **No multi-step workflow replay.** This agent navigates directly to a URL — it doesn't replay a login flow or rebuild cart state. Auth-gated pages are covered via `--storage-state`; deeper application state built up through a workflow is not.
50
+
51
+ ## Primary finding vs. secondary observations
52
+
53
+ A full-page diff can surface something real that's unrelated to what the check was actually for. The verdict carries one **primary finding** (what drove the classification) and a separate list of **secondary observations** — other real differences noticed elsewhere on the page. Secondary observations are always reported, never silently dropped, and never affect the verdict or exit code.
54
+
55
+ ## Configuration
56
+
57
+ Copy `.env.example` to `.env`. Requires `APPQ_API_KEY` (read-only access is sufficient — this agent never calls an appq write tool) and one of `ANTHROPIC_API_KEY`/`OPENAI_API_KEY`.
58
+
59
+ ## Running this safely
60
+
61
+ This agent has a real browser and navigates to whatever URLs `--baseline-environment`/`--target-environment` resolve to — it has no filesystem write access and no shell surface at all (unlike `heal-selector`/`scriptgen`, it never patches anything).
62
+
63
+ **Run this inside a container with an egress allowlist**, same as every agent in this family. This process only ever legitimately needs to reach:
64
+
65
+ - your LLM provider (`api.anthropic.com` or `api.openai.com`)
66
+ - your configured `APPQ_ORIGIN` (`appq.appliqation.io` by default)
67
+ - the two sites under test — whatever `--baseline-environment`/`--target-environment` resolve to
68
+
69
+ Anything else this process tries to reach is unexpected and worth investigating.
70
+
71
+ ## Development
72
+
73
+ ```bash
74
+ git clone https://github.com/appliqation/visual-regression.git
75
+ cd visual-regression
76
+ npm install
77
+ cp .env.example .env # fill in APPQ_API_KEY (read-only) and an LLM key
78
+ npm run dev -- check --test-case-uuid <uuid> --route </path> --baseline-environment <name> --target-environment <name>
79
+ npm run typecheck
80
+ npm test
81
+ ```
82
+
83
+ See `CLAUDE.md` for a map of this repo if you're working in it with an AI coding assistant.
84
+
85
+ ## License
86
+
87
+ MIT — see [LICENSE](./LICENSE).
@@ -0,0 +1,37 @@
1
+ // Extracted out of cli/index.ts so this is testable without triggering that
2
+ // file's top-level program.parseAsync(process.argv) side effect — same
3
+ // reasoning as every sibling agent's audit module.
4
+ import { safeRecord, safeClose } from '@appliqation/agent-core';
5
+ import { exitCodeFor } from './output.js';
6
+ export async function recordVisualRegressionRun(args) {
7
+ const { sink, startedAt, endedAt, model, usage, route, baselineUrl, targetUrl, testCaseUuid, result } = args;
8
+ const summary = result
9
+ ? {
10
+ route,
11
+ baselineUrl,
12
+ targetUrl,
13
+ diffRan: result.diffRan,
14
+ diffPixelCount: result.diffPixelCount,
15
+ diffPercentage: result.diffPercentage,
16
+ verdict: (result.verdict ?? 'inconclusive'),
17
+ primaryFinding: result.primaryFinding,
18
+ secondaryFindings: result.secondaryFindings,
19
+ report: result.report,
20
+ }
21
+ : undefined;
22
+ await safeRecord(sink, {
23
+ agent: 'appliqation-visual-regression',
24
+ subcommand: 'check',
25
+ startedAt,
26
+ endedAt,
27
+ durationMillis: endedAt - startedAt,
28
+ model,
29
+ usage,
30
+ turns: result?.turns,
31
+ budgetExceeded: result?.budgetExceeded,
32
+ exitCode: summary ? exitCodeFor(summary) : 1,
33
+ outcome: summary ? { ...summary, testCaseUuid } : { route, baselineUrl, targetUrl, testCaseUuid, error: true },
34
+ });
35
+ await safeClose(sink);
36
+ }
37
+ //# sourceMappingURL=audit.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"audit.js","sourceRoot":"","sources":["../../src/cli/audit.ts"],"names":[],"mappings":"AAAA,4EAA4E;AAC5E,uEAAuE;AACvE,mDAAmD;AAEnD,OAAO,EAAE,UAAU,EAAE,SAAS,EAAoC,MAAM,yBAAyB,CAAC;AAElG,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAiB1C,MAAM,CAAC,KAAK,UAAU,yBAAyB,CAAC,IAAmC;IACjF,MAAM,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,KAAK,EAAE,WAAW,EAAE,SAAS,EAAE,YAAY,EAAE,MAAM,EAAE,GAAG,IAAI,CAAC;IAC7G,MAAM,OAAO,GAAwC,MAAM;QACzD,CAAC,CAAC;YACE,KAAK;YACL,WAAW;YACX,SAAS;YACT,OAAO,EAAE,MAAM,CAAC,OAAO;YACvB,cAAc,EAAE,MAAM,CAAC,cAAc;YACrC,cAAc,EAAE,MAAM,CAAC,cAAc;YACrC,OAAO,EAAE,CAAC,MAAM,CAAC,OAAO,IAAI,cAAc,CAAY;YACtD,cAAc,EAAE,MAAM,CAAC,cAAc;YACrC,iBAAiB,EAAE,MAAM,CAAC,iBAAiB;YAC3C,MAAM,EAAE,MAAM,CAAC,MAAM;SACtB;QACH,CAAC,CAAC,SAAS,CAAC;IAEd,MAAM,UAAU,CAAC,IAAI,EAAE;QACrB,KAAK,EAAE,+BAA+B;QACtC,UAAU,EAAE,OAAO;QACnB,SAAS;QACT,OAAO;QACP,cAAc,EAAE,OAAO,GAAG,SAAS;QACnC,KAAK;QACL,KAAK;QACL,KAAK,EAAE,MAAM,EAAE,KAAK;QACpB,cAAc,EAAE,MAAM,EAAE,cAAc;QACtC,QAAQ,EAAE,OAAO,CAAC,CAAC,CAAC,WAAW,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC;QAC5C,OAAO,EAAE,OAAO,CAAC,CAAC,CAAC,EAAE,GAAG,OAAO,EAAE,YAAY,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,WAAW,EAAE,SAAS,EAAE,YAAY,EAAE,KAAK,EAAE,IAAI,EAAE;KAC/G,CAAC,CAAC;IACH,MAAM,SAAS,CAAC,IAAI,CAAC,CAAC;AACxB,CAAC"}
@@ -0,0 +1,159 @@
1
+ #!/usr/bin/env node
2
+ // `check`: compare one route between production and a target environment for
3
+ // a real visual regression. See src/orchestrator/visualRegression.ts for the
4
+ // mechanism and src/policy/visualRegressionPrompt.ts for the actual
5
+ // methodology (no appq-served prompt exists for this yet — the policy is
6
+ // local, bundled here).
7
+ //
8
+ // Generic, caller-agnostic inputs — not designed around any one caller.
9
+ // project_id is always derived from --test-case-uuid, never accepted as a
10
+ // separate input, same reasoning as every sibling agent. --route stays a
11
+ // single, caller-supplied string — no route enumeration/inference (no
12
+ // structured route data exists on scenarios/test cases to derive it from).
13
+ import { Command } from 'commander';
14
+ import { createMcpClient, createAnthropicAdapter, createOpenAiAdapter, createUsageAccumulator, resolveScenarioId, fetchScenarioInfo, resolveUrl, } from '@appliqation/agent-core';
15
+ import { config, resolveProvider, resolveModel } from '../config/env.js';
16
+ import { checkVisualRegression } from '../orchestrator/visualRegression.js';
17
+ import { recordVisualRegressionRun } from './audit.js';
18
+ import { printJsonSummary, printHumanSummary, exitCodeFor } from './output.js';
19
+ const client = createMcpClient({ origin: config.appqOrigin, apiKey: config.appqApiKey() });
20
+ function buildAdapter() {
21
+ const provider = resolveProvider();
22
+ const model = resolveModel();
23
+ return provider === 'anthropic'
24
+ ? createAnthropicAdapter(config.anthropicApiKey, model, config.anthropicMaxTokens)
25
+ : createOpenAiAdapter(config.openaiApiKey, model, config.openaiMaxOutputTokens);
26
+ }
27
+ function logEvent(prefix) {
28
+ return (e) => {
29
+ if (e.type === 'assistant') {
30
+ const text = (e.detail ?? '').trim();
31
+ if (text)
32
+ console.error(`${prefix}[thinking] ${text}`);
33
+ }
34
+ else if (e.type === 'tool') {
35
+ const d = e.detail;
36
+ console.error(`${prefix}[tool] ${d.name} -> ${d.result.slice(0, 300)}`);
37
+ }
38
+ else if (e.type === 'log') {
39
+ console.error(`${prefix}[log] ${e.detail}`);
40
+ }
41
+ else if (e.type === 'usage') {
42
+ const u = e.detail;
43
+ const cacheNote = u.cacheReadTokens
44
+ ? ` (${u.cacheReadTokens} from cache)`
45
+ : u.cacheWriteTokens
46
+ ? ` (${u.cacheWriteTokens} written to cache)`
47
+ : '';
48
+ console.error(`${prefix}[usage] in=${u.inputTokens} out=${u.outputTokens}${cacheNote}`);
49
+ }
50
+ };
51
+ }
52
+ function collect(value, previous) {
53
+ return [...previous, value];
54
+ }
55
+ /** Joins a base environment URL with a route path — an absolute route (leading "/") replaces the base's own path, same as any URL resolution. */
56
+ function joinRoute(baseUrl, route) {
57
+ return new URL(route, baseUrl).toString();
58
+ }
59
+ const program = new Command();
60
+ program
61
+ .name('appliqation-visual-regression')
62
+ .description('Check one route for a real visual regression by diffing it against its own live production ' +
63
+ 'counterpart — no stored baseline files, no manual baseline-approval workflow. Declines rather than ' +
64
+ 'compares when the route does not exist on production. See README.md for the full story.');
65
+ program
66
+ .command('check')
67
+ .description('Navigate to the same route on a baseline (production) and target environment, mask configured ' +
68
+ 'dynamic regions, full-page screenshot both, pixel-diff them for real, and have the model judge ' +
69
+ "regression vs. expected divergence vs. inconclusive — never the model's own claim about the pixels, " +
70
+ 'always the real diff it was actually shown.')
71
+ .requiredOption('--test-case-uuid <uuid>', 'test case this route belongs to — derives project_id, supplies expected_result context')
72
+ .requiredOption('--route </path>', 'the route to check, e.g. /subscribe — same route is checked on both environments')
73
+ .requiredOption('--baseline-environment <name>', 'environment name treated as the source of truth (commonly, but not necessarily, "Prod")')
74
+ .requiredOption('--target-environment <name>', 'environment name being checked against the baseline')
75
+ .option('--mask <selector>', 'CSS selector to mask before capturing (repeatable) — reduces noise on known dynamic regions', collect, [])
76
+ .option('--storage-state <path>', 'Playwright storageState file, for auth-gated routes')
77
+ .option('--max-turns <n>', 'override BUDGET_MAX_TURNS for this run')
78
+ .option('--json', 'print a single structured JSON summary on stdout instead of a human-readable report')
79
+ .option('--ci', 'shorthand for --json; exit code already reflects the real verdict either way')
80
+ .action(async (opts) => {
81
+ const json = (opts.json ?? false) || (opts.ci ?? false);
82
+ const adapter = buildAdapter();
83
+ const scenarioId = resolveScenarioId({ testCaseUuid: opts.testCaseUuid });
84
+ const { projectId } = await fetchScenarioInfo(client, scenarioId);
85
+ const [baselineBaseUrl, targetBaseUrl] = await Promise.all([
86
+ resolveUrl(client, opts.baselineEnvironment, projectId),
87
+ resolveUrl(client, opts.targetEnvironment, projectId),
88
+ ]);
89
+ const baselineUrl = joinRoute(baselineBaseUrl, opts.route);
90
+ const targetUrl = joinRoute(targetBaseUrl, opts.route);
91
+ const budget = { ...config.budget, ...(opts.maxTurns ? { maxTurns: Number(opts.maxTurns) } : {}) };
92
+ const startedAt = Date.now();
93
+ const usage = createUsageAccumulator();
94
+ const baseLog = logEvent('');
95
+ let result;
96
+ try {
97
+ result = await checkVisualRegression({
98
+ client,
99
+ adapter,
100
+ testCaseUuid: opts.testCaseUuid,
101
+ route: opts.route,
102
+ baselineUrl,
103
+ targetUrl,
104
+ maskSelectors: opts.mask,
105
+ storageStatePath: opts.storageState,
106
+ budget,
107
+ onEvent: (e) => {
108
+ baseLog(e);
109
+ if (e.type === 'usage')
110
+ usage.onUsage(e.detail);
111
+ },
112
+ });
113
+ }
114
+ finally {
115
+ // Audit write happens whether the run succeeded or threw — see
116
+ // @appliqation/agent-core's audit/sink.ts: safeRecord() (used inside
117
+ // recordVisualRegressionRun) never lets a failed/unreachable audit
118
+ // sink affect this process's real outcome.
119
+ await recordVisualRegressionRun({
120
+ sink: config.auditSink,
121
+ startedAt,
122
+ endedAt: Date.now(),
123
+ model: resolveModel(),
124
+ usage: usage.totals(),
125
+ route: opts.route,
126
+ baselineUrl,
127
+ targetUrl,
128
+ testCaseUuid: opts.testCaseUuid,
129
+ result,
130
+ });
131
+ }
132
+ if (!json) {
133
+ console.log('\n=== Report ===\n');
134
+ console.log(result.report);
135
+ console.error(`\n(${result.turns} turns, budget exceeded: ${result.budgetExceeded})`);
136
+ }
137
+ const summary = {
138
+ route: opts.route,
139
+ baselineUrl,
140
+ targetUrl,
141
+ diffRan: result.diffRan,
142
+ diffPixelCount: result.diffPixelCount,
143
+ diffPercentage: result.diffPercentage,
144
+ verdict: (result.verdict ?? 'inconclusive'),
145
+ primaryFinding: result.primaryFinding,
146
+ secondaryFindings: result.secondaryFindings,
147
+ report: result.report,
148
+ };
149
+ if (json)
150
+ printJsonSummary(summary);
151
+ else
152
+ printHumanSummary(summary);
153
+ process.exitCode = exitCodeFor(summary);
154
+ });
155
+ program.parseAsync(process.argv).catch((err) => {
156
+ console.error(`Error: ${err instanceof Error ? err.message : String(err)}`);
157
+ process.exitCode = 1;
158
+ });
159
+ //# sourceMappingURL=index.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/cli/index.ts"],"names":[],"mappings":";AACA,6EAA6E;AAC7E,6EAA6E;AAC7E,oEAAoE;AACpE,yEAAyE;AACzE,wBAAwB;AACxB,EAAE;AACF,wEAAwE;AACxE,0EAA0E;AAC1E,yEAAyE;AACzE,sEAAsE;AACtE,2EAA2E;AAE3E,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AACpC,OAAO,EACL,eAAe,EACf,sBAAsB,EACtB,mBAAmB,EACnB,sBAAsB,EACtB,iBAAiB,EACjB,iBAAiB,EACjB,UAAU,GAEX,MAAM,yBAAyB,CAAC;AACjC,OAAO,EAAE,MAAM,EAAE,eAAe,EAAE,YAAY,EAAE,MAAM,kBAAkB,CAAC;AACzE,OAAO,EAAE,qBAAqB,EAAE,MAAM,qCAAqC,CAAC;AAE5E,OAAO,EAAE,yBAAyB,EAAE,MAAM,YAAY,CAAC;AACvD,OAAO,EAAE,gBAAgB,EAAE,iBAAiB,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAG/E,MAAM,MAAM,GAAG,eAAe,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,UAAU,EAAE,MAAM,EAAE,MAAM,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC;AAE3F,SAAS,YAAY;IACnB,MAAM,QAAQ,GAAG,eAAe,EAAE,CAAC;IACnC,MAAM,KAAK,GAAG,YAAY,EAAE,CAAC;IAC7B,OAAO,QAAQ,KAAK,WAAW;QAC7B,CAAC,CAAC,sBAAsB,CAAC,MAAM,CAAC,eAAgB,EAAE,KAAK,EAAE,MAAM,CAAC,kBAAkB,CAAC;QACnF,CAAC,CAAC,mBAAmB,CAAC,MAAM,CAAC,YAAa,EAAE,KAAK,EAAE,MAAM,CAAC,qBAAqB,CAAC,CAAC;AACrF,CAAC;AAED,SAAS,QAAQ,CAAC,MAAc;IAC9B,OAAO,CAAC,CAAqC,EAAE,EAAE;QAC/C,IAAI,CAAC,CAAC,IAAI,KAAK,WAAW,EAAE,CAAC;YAC3B,MAAM,IAAI,GAAG,CAAE,CAAC,CAAC,MAAiB,IAAI,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC;YACjD,IAAI,IAAI;gBAAE,OAAO,CAAC,KAAK,CAAC,GAAG,MAAM,cAAc,IAAI,EAAE,CAAC,CAAC;QACzD,CAAC;aAAM,IAAI,CAAC,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;YAC7B,MAAM,CAAC,GAAG,CAAC,CAAC,MAA0C,CAAC;YACvD,OAAO,CAAC,KAAK,CAAC,GAAG,MAAM,UAAU,CAAC,CAAC,IAAI,OAAO,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,CAAC,CAAC;QAC1E,CAAC;aAAM,IAAI,CAAC,CAAC,IAAI,KAAK,KAAK,EAAE,CAAC;YAC5B,OAAO,CAAC,KAAK,CAAC,GAAG,MAAM,SAAS,CAAC,CAAC,MAAM,EAAE,CAAC,CAAC;QAC9C,CAAC;aAAM,IAAI,CAAC,CAAC,IAAI,KAAK,OAAO,EAAE,CAAC;YAC9B,MAAM,CAAC,GAAG,CAAC,CAAC,MAA4G,CAAC;YACzH,MAAM,SAAS,GAAG,CAAC,CAAC,eAAe;gBACjC,CAAC,CAAC,KAAK,CAAC,CAAC,eAAe,cAAc;gBACtC,CAAC,CAAC,CAAC,CAAC,gBAAgB;oBAClB,CAAC,CAAC,KAAK,CAAC,CAAC,gBAAgB,oBAAoB;oBAC7C,CAAC,CAAC,EAAE,CAAC;YACT,OAAO,CAAC,KAAK,CAAC,GAAG,MAAM,cAAc,CAAC,CAAC,WAAW,QAAQ,CAAC,CAAC,YAAY,GAAG,SAAS,EAAE,CAAC,CAAC;QAC1F,CAAC;IACH,CAAC,CAAC;AACJ,CAAC;AAED,SAAS,OAAO,CAAC,KAAa,EAAE,QAAkB;IAChD,OAAO,CAAC,GAAG,QAAQ,EAAE,KAAK,CAAC,CAAC;AAC9B,CAAC;AAED,iJAAiJ;AACjJ,SAAS,SAAS,CAAC,OAAe,EAAE,KAAa;IAC/C,OAAO,IAAI,GAAG,CAAC,KAAK,EAAE,OAAO,CAAC,CAAC,QAAQ,EAAE,CAAC;AAC5C,CAAC;AAED,MAAM,OAAO,GAAG,IAAI,OAAO,EAAE,CAAC;AAC9B,OAAO;KACJ,IAAI,CAAC,+BAA+B,CAAC;KACrC,WAAW,CACV,6FAA6F;IAC3F,qGAAqG;IACrG,yFAAyF,CAC5F,CAAC;AAEJ,OAAO;KACJ,OAAO,CAAC,OAAO,CAAC;KAChB,WAAW,CACV,gGAAgG;IAC9F,iGAAiG;IACjG,sGAAsG;IACtG,6CAA6C,CAChD;KACA,cAAc,CAAC,yBAAyB,EAAE,wFAAwF,CAAC;KACnI,cAAc,CAAC,iBAAiB,EAAE,kFAAkF,CAAC;KACrH,cAAc,CAAC,+BAA+B,EAAE,yFAAyF,CAAC;KAC1I,cAAc,CAAC,6BAA6B,EAAE,qDAAqD,CAAC;KACpG,MAAM,CAAC,mBAAmB,EAAE,6FAA6F,EAAE,OAAO,EAAE,EAAE,CAAC;KACvI,MAAM,CAAC,wBAAwB,EAAE,qDAAqD,CAAC;KACvF,MAAM,CAAC,iBAAiB,EAAE,wCAAwC,CAAC;KACnE,MAAM,CAAC,QAAQ,EAAE,qFAAqF,CAAC;KACvG,MAAM,CAAC,MAAM,EAAE,8EAA8E,CAAC;KAC9F,MAAM,CACL,KAAK,EAAE,IAUN,EAAE,EAAE;IACH,MAAM,IAAI,GAAG,CAAC,IAAI,CAAC,IAAI,IAAI,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,KAAK,CAAC,CAAC;IACxD,MAAM,OAAO,GAAG,YAAY,EAAE,CAAC;IAE/B,MAAM,UAAU,GAAG,iBAAiB,CAAC,EAAE,YAAY,EAAE,IAAI,CAAC,YAAY,EAAE,CAAC,CAAC;IAC1E,MAAM,EAAE,SAAS,EAAE,GAAG,MAAM,iBAAiB,CAAC,MAAM,EAAE,UAAU,CAAC,CAAC;IAClE,MAAM,CAAC,eAAe,EAAE,aAAa,CAAC,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC;QACzD,UAAU,CAAC,MAAM,EAAE,IAAI,CAAC,mBAAmB,EAAE,SAAS,CAAC;QACvD,UAAU,CAAC,MAAM,EAAE,IAAI,CAAC,iBAAiB,EAAE,SAAS,CAAC;KACtD,CAAC,CAAC;IACH,MAAM,WAAW,GAAG,SAAS,CAAC,eAAe,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC;IAC3D,MAAM,SAAS,GAAG,SAAS,CAAC,aAAa,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC;IAEvD,MAAM,MAAM,GAAG,EAAE,GAAG,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,MAAM,CAAC,IAAI,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC;IAEnG,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;IAC7B,MAAM,KAAK,GAAG,sBAAsB,EAAE,CAAC;IACvC,MAAM,OAAO,GAAG,QAAQ,CAAC,EAAE,CAAC,CAAC;IAC7B,IAAI,MAA0C,CAAC;IAC/C,IAAI,CAAC;QACH,MAAM,GAAG,MAAM,qBAAqB,CAAC;YACnC,MAAM;YACN,OAAO;YACP,YAAY,EAAE,IAAI,CAAC,YAAY;YAC/B,KAAK,EAAE,IAAI,CAAC,KAAK;YACjB,WAAW;YACX,SAAS;YACT,aAAa,EAAE,IAAI,CAAC,IAAI;YACxB,gBAAgB,EAAE,IAAI,CAAC,YAAY;YACnC,MAAM;YACN,OAAO,EAAE,CAAC,CAAC,EAAE,EAAE;gBACb,OAAO,CAAC,CAAC,CAAC,CAAC;gBACX,IAAI,CAAC,CAAC,IAAI,KAAK,OAAO;oBAAE,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAA4G,CAAC,CAAC;YACxJ,CAAC;SACF,CAAC,CAAC;IACL,CAAC;YAAS,CAAC;QACT,+DAA+D;QAC/D,qEAAqE;QACrE,mEAAmE;QACnE,2CAA2C;QAC3C,MAAM,yBAAyB,CAAC;YAC9B,IAAI,EAAE,MAAM,CAAC,SAAS;YACtB,SAAS;YACT,OAAO,EAAE,IAAI,CAAC,GAAG,EAAE;YACnB,KAAK,EAAE,YAAY,EAAE;YACrB,KAAK,EAAE,KAAK,CAAC,MAAM,EAAE;YACrB,KAAK,EAAE,IAAI,CAAC,KAAK;YACjB,WAAW;YACX,SAAS;YACT,YAAY,EAAE,IAAI,CAAC,YAAY;YAC/B,MAAM;SACP,CAAC,CAAC;IACL,CAAC;IAED,IAAI,CAAC,IAAI,EAAE,CAAC;QACV,OAAO,CAAC,GAAG,CAAC,oBAAoB,CAAC,CAAC;QAClC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;QAC3B,OAAO,CAAC,KAAK,CAAC,MAAM,MAAM,CAAC,KAAK,4BAA4B,MAAM,CAAC,cAAc,GAAG,CAAC,CAAC;IACxF,CAAC;IAED,MAAM,OAAO,GAA4B;QACvC,KAAK,EAAE,IAAI,CAAC,KAAK;QACjB,WAAW;QACX,SAAS;QACT,OAAO,EAAE,MAAM,CAAC,OAAO;QACvB,cAAc,EAAE,MAAM,CAAC,cAAc;QACrC,cAAc,EAAE,MAAM,CAAC,cAAc;QACrC,OAAO,EAAE,CAAC,MAAM,CAAC,OAAO,IAAI,cAAc,CAAY;QACtD,cAAc,EAAE,MAAM,CAAC,cAAc;QACrC,iBAAiB,EAAE,MAAM,CAAC,iBAAiB;QAC3C,MAAM,EAAE,MAAM,CAAC,MAAM;KACtB,CAAC;IACF,IAAI,IAAI;QAAE,gBAAgB,CAAC,OAAO,CAAC,CAAC;;QAC/B,iBAAiB,CAAC,OAAO,CAAC,CAAC;IAChC,OAAO,CAAC,QAAQ,GAAG,WAAW,CAAC,OAAO,CAAC,CAAC;AAC1C,CAAC,CACF,CAAC;AAEJ,OAAO,CAAC,UAAU,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,KAAK,CAAC,CAAC,GAAY,EAAE,EAAE;IACtD,OAAO,CAAC,KAAK,CAAC,UAAU,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;IAC5E,OAAO,CAAC,QAAQ,GAAG,CAAC,CAAC;AACvB,CAAC,CAAC,CAAC"}
@@ -0,0 +1,32 @@
1
+ // --json/--ci's renderer, matching the family's shape (heal-selector/
2
+ // scriptgen/defect-fix output.ts). exitCodeFor() is driven only by
3
+ // `verdict` (the primary finding) — `secondaryFindings` never affects it,
4
+ // same reasoning the policy prompt gives the model: secondary observations
5
+ // are reported, never merged into the primary judgment.
6
+ export function printJsonSummary(summary) {
7
+ console.log(JSON.stringify(summary, null, 2));
8
+ }
9
+ export function printHumanSummary(summary) {
10
+ console.log(`\n=== ${summary.route} ===\n`);
11
+ console.log(` Baseline: ${summary.baselineUrl}`);
12
+ console.log(` Target: ${summary.targetUrl}`);
13
+ if (!summary.diffRan) {
14
+ console.log('\n capture_and_diff never actually ran — no verdict was computed.');
15
+ return;
16
+ }
17
+ if (summary.diffPercentage !== null) {
18
+ console.log(` Diff: ${summary.diffPixelCount} pixels (${summary.diffPercentage.toFixed(2)}%)`);
19
+ }
20
+ console.log(`\n Verdict: ${summary.verdict.toUpperCase()}`);
21
+ console.log(` Primary finding: ${summary.primaryFinding || '(none stated)'}`);
22
+ if (summary.secondaryFindings.length > 0) {
23
+ console.log(' Secondary observations (out of scope for this check):');
24
+ for (const s of summary.secondaryFindings)
25
+ console.log(` - ${s}`);
26
+ }
27
+ }
28
+ /** 0 for expected-divergence/not-applicable — nothing wrong, or nothing to check. 1 for regression/inconclusive — fail-closed on ambiguity, matching family convention. */
29
+ export function exitCodeFor(summary) {
30
+ return summary.verdict === 'expected-divergence' || summary.verdict === 'not-applicable' ? 0 : 1;
31
+ }
32
+ //# sourceMappingURL=output.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"output.js","sourceRoot":"","sources":["../../src/cli/output.ts"],"names":[],"mappings":"AAAA,sEAAsE;AACtE,mEAAmE;AACnE,0EAA0E;AAC1E,2EAA2E;AAC3E,wDAAwD;AAiBxD,MAAM,UAAU,gBAAgB,CAAC,OAAgC;IAC/D,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;AAChD,CAAC;AAED,MAAM,UAAU,iBAAiB,CAAC,OAAgC;IAChE,OAAO,CAAC,GAAG,CAAC,SAAS,OAAO,CAAC,KAAK,QAAQ,CAAC,CAAC;IAC5C,OAAO,CAAC,GAAG,CAAC,eAAe,OAAO,CAAC,WAAW,EAAE,CAAC,CAAC;IAClD,OAAO,CAAC,GAAG,CAAC,eAAe,OAAO,CAAC,SAAS,EAAE,CAAC,CAAC;IAChD,IAAI,CAAC,OAAO,CAAC,OAAO,EAAE,CAAC;QACrB,OAAO,CAAC,GAAG,CAAC,oEAAoE,CAAC,CAAC;QAClF,OAAO;IACT,CAAC;IACD,IAAI,OAAO,CAAC,cAAc,KAAK,IAAI,EAAE,CAAC;QACpC,OAAO,CAAC,GAAG,CAAC,WAAW,OAAO,CAAC,cAAc,YAAY,OAAO,CAAC,cAAc,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;IAClG,CAAC;IACD,OAAO,CAAC,GAAG,CAAC,gBAAgB,OAAO,CAAC,OAAO,CAAC,WAAW,EAAE,EAAE,CAAC,CAAC;IAC7D,OAAO,CAAC,GAAG,CAAC,sBAAsB,OAAO,CAAC,cAAc,IAAI,eAAe,EAAE,CAAC,CAAC;IAC/E,IAAI,OAAO,CAAC,iBAAiB,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACzC,OAAO,CAAC,GAAG,CAAC,yDAAyD,CAAC,CAAC;QACvE,KAAK,MAAM,CAAC,IAAI,OAAO,CAAC,iBAAiB;YAAE,OAAO,CAAC,GAAG,CAAC,SAAS,CAAC,EAAE,CAAC,CAAC;IACvE,CAAC;AACH,CAAC;AAED,2KAA2K;AAC3K,MAAM,UAAU,WAAW,CAAC,OAAgC;IAC1D,OAAO,OAAO,CAAC,OAAO,KAAK,qBAAqB,IAAI,OAAO,CAAC,OAAO,KAAK,gBAAgB,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AACnG,CAAC"}
@@ -0,0 +1,55 @@
1
+ import 'dotenv/config';
2
+ import { DEFAULT_ANTHROPIC_MODEL, DEFAULT_OPENAI_MODEL } from '@appliqation/agent-core/providers';
3
+ import { required, optional } from '@appliqation/agent-core/config';
4
+ import { resolveAuditSink } from '@appliqation/agent-core/audit';
5
+ export const config = {
6
+ appqOrigin: optional('APPQ_ORIGIN') ?? 'https://appq.appliqation.io',
7
+ appqApiKey: () => required('APPQ_API_KEY'),
8
+ anthropicApiKey: optional('ANTHROPIC_API_KEY'),
9
+ openaiApiKey: optional('OPENAI_API_KEY'),
10
+ anthropicModel: optional('ANTHROPIC_MODEL'),
11
+ openaiModel: optional('OPENAI_MODEL'),
12
+ anthropicMaxTokens: Number(optional('ANTHROPIC_MAX_TOKENS') ?? 8192),
13
+ openaiMaxOutputTokens: Number(optional('OPENAI_MAX_OUTPUT_TOKENS') ?? 8192),
14
+ // One capture-then-judge pass — the mechanical work (two navigations, one
15
+ // diff) is fixed cost; the model's only real work is a single judgment
16
+ // call, so this budget is deliberately tight compared to heal-selector's.
17
+ budget: {
18
+ maxCalls: Number(optional('BUDGET_MAX_CALLS') ?? 15),
19
+ // Two real navigations (baseline + target) happen inside the one
20
+ // capture_and_diff tool call, not as separate browser_navigate calls —
21
+ // this cap is a backstop, not something the model spends turn-by-turn.
22
+ maxPages: Number(optional('BUDGET_MAX_PAGES') ?? 10),
23
+ maxMillis: Number(optional('BUDGET_MAX_MILLIS') ?? 5 * 60 * 1000),
24
+ maxTurns: Number(optional('BUDGET_MAX_TURNS') ?? 10),
25
+ // A broad backstop against runaway spend, not a tuned budget. Includes
26
+ // cache tokens — and matters more here than most siblings, since every
27
+ // turn after capture_and_diff carries three real images in context.
28
+ maxTotalTokens: Number(optional('BUDGET_MAX_TOTAL_TOKENS') ?? 1_000_000),
29
+ },
30
+ // Playwright browser tools' evidence ring-buffer cap — see
31
+ // @appliqation/agent-core's evidence/capture.ts. Unused in practice: this
32
+ // agent never exposes BROWSER_TOOL_DEFS to the model, capture_and_diff
33
+ // drives its own browser directly. Kept for config-shape consistency with
34
+ // every sibling agent.
35
+ ringBufferCap: Number(optional('RING_BUFFER_CAP') ?? 10),
36
+ // Observability, entirely opt-in — see @appliqation/agent-core's audit/sink.ts.
37
+ auditSink: resolveAuditSink({
38
+ auditMongoUri: optional('AUDIT_MONGO_URI'),
39
+ auditMongoDb: optional('AUDIT_MONGO_DB'),
40
+ auditMongoCollection: optional('AUDIT_MONGO_COLLECTION'),
41
+ auditJsonlPath: optional('AUDIT_JSONL_PATH'),
42
+ }),
43
+ };
44
+ export function resolveProvider() {
45
+ if (config.anthropicApiKey)
46
+ return 'anthropic';
47
+ if (config.openaiApiKey)
48
+ return 'openai';
49
+ throw new Error('Set ANTHROPIC_API_KEY or OPENAI_API_KEY');
50
+ }
51
+ export function resolveModel() {
52
+ const provider = resolveProvider();
53
+ return provider === 'anthropic' ? (config.anthropicModel ?? DEFAULT_ANTHROPIC_MODEL) : (config.openaiModel ?? DEFAULT_OPENAI_MODEL);
54
+ }
55
+ //# sourceMappingURL=env.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"env.js","sourceRoot":"","sources":["../../src/config/env.ts"],"names":[],"mappings":"AAAA,OAAO,eAAe,CAAC;AACvB,OAAO,EAAE,uBAAuB,EAAE,oBAAoB,EAAE,MAAM,mCAAmC,CAAC;AAClG,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,MAAM,gCAAgC,CAAC;AACpE,OAAO,EAAE,gBAAgB,EAAE,MAAM,+BAA+B,CAAC;AAEjE,MAAM,CAAC,MAAM,MAAM,GAAG;IACpB,UAAU,EAAE,QAAQ,CAAC,aAAa,CAAC,IAAI,6BAA6B;IACpE,UAAU,EAAE,GAAG,EAAE,CAAC,QAAQ,CAAC,cAAc,CAAC;IAC1C,eAAe,EAAE,QAAQ,CAAC,mBAAmB,CAAC;IAC9C,YAAY,EAAE,QAAQ,CAAC,gBAAgB,CAAC;IACxC,cAAc,EAAE,QAAQ,CAAC,iBAAiB,CAAC;IAC3C,WAAW,EAAE,QAAQ,CAAC,cAAc,CAAC;IACrC,kBAAkB,EAAE,MAAM,CAAC,QAAQ,CAAC,sBAAsB,CAAC,IAAI,IAAI,CAAC;IACpE,qBAAqB,EAAE,MAAM,CAAC,QAAQ,CAAC,0BAA0B,CAAC,IAAI,IAAI,CAAC;IAC3E,0EAA0E;IAC1E,uEAAuE;IACvE,0EAA0E;IAC1E,MAAM,EAAE;QACN,QAAQ,EAAE,MAAM,CAAC,QAAQ,CAAC,kBAAkB,CAAC,IAAI,EAAE,CAAC;QACpD,iEAAiE;QACjE,uEAAuE;QACvE,uEAAuE;QACvE,QAAQ,EAAE,MAAM,CAAC,QAAQ,CAAC,kBAAkB,CAAC,IAAI,EAAE,CAAC;QACpD,SAAS,EAAE,MAAM,CAAC,QAAQ,CAAC,mBAAmB,CAAC,IAAI,CAAC,GAAG,EAAE,GAAG,IAAI,CAAC;QACjE,QAAQ,EAAE,MAAM,CAAC,QAAQ,CAAC,kBAAkB,CAAC,IAAI,EAAE,CAAC;QACpD,uEAAuE;QACvE,uEAAuE;QACvE,oEAAoE;QACpE,cAAc,EAAE,MAAM,CAAC,QAAQ,CAAC,yBAAyB,CAAC,IAAI,SAAS,CAAC;KACzE;IACD,2DAA2D;IAC3D,0EAA0E;IAC1E,uEAAuE;IACvE,0EAA0E;IAC1E,uBAAuB;IACvB,aAAa,EAAE,MAAM,CAAC,QAAQ,CAAC,iBAAiB,CAAC,IAAI,EAAE,CAAC;IAExD,gFAAgF;IAChF,SAAS,EAAE,gBAAgB,CAAC;QAC1B,aAAa,EAAE,QAAQ,CAAC,iBAAiB,CAAC;QAC1C,YAAY,EAAE,QAAQ,CAAC,gBAAgB,CAAC;QACxC,oBAAoB,EAAE,QAAQ,CAAC,wBAAwB,CAAC;QACxD,cAAc,EAAE,QAAQ,CAAC,kBAAkB,CAAC;KAC7C,CAAC;CACH,CAAC;AAEF,MAAM,UAAU,eAAe;IAC7B,IAAI,MAAM,CAAC,eAAe;QAAE,OAAO,WAAW,CAAC;IAC/C,IAAI,MAAM,CAAC,YAAY;QAAE,OAAO,QAAQ,CAAC;IACzC,MAAM,IAAI,KAAK,CAAC,yCAAyC,CAAC,CAAC;AAC7D,CAAC;AAED,MAAM,UAAU,YAAY;IAC1B,MAAM,QAAQ,GAAG,eAAe,EAAE,CAAC;IACnC,OAAO,QAAQ,KAAK,WAAW,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,cAAc,IAAI,uBAAuB,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,WAAW,IAAI,oBAAoB,CAAC,CAAC;AACtI,CAAC"}
@@ -0,0 +1,79 @@
1
+ // Drives the visual-regression methodology (src/policy/visualRegressionPrompt.ts
2
+ // — no appq-served prompt exists for this, so runLoop() is used directly,
3
+ // same pattern appliqation-heal-selector/appliqation-autopilot use for their
4
+ // own local policies). Deliberately NOT the browser+coding-tools composition
5
+ // heal-selector uses — this agent never exposes BROWSER_TOOL_DEFS to the
6
+ // model at all; the entire mechanical pipeline is one atomic, code-owned
7
+ // tool (capture_and_diff). The model's only real work is judgment, captured
8
+ // through a second tool (submit_verdict) as structured data, never parsed
9
+ // out of free-text report prose.
10
+ import { runLoop, fetchAppqToolDefs, createGatedAppqDispatcher, } from '@appliqation/agent-core';
11
+ import { READONLY_CONTEXT_TOOLS } from '../tools/safety.js';
12
+ import { CAPTURE_TOOL_DEFS, CaptureAndDiffTool } from '../tools/captureAndDiff.js';
13
+ import { SUBMIT_VERDICT_TOOL_DEF, VerdictCapture } from '../tools/verdictTool.js';
14
+ import { buildVisualRegressionPrompt } from '../policy/visualRegressionPrompt.js';
15
+ function seedMessage(opts) {
16
+ const lines = [
17
+ `Route being checked: ${opts.route}`,
18
+ `Baseline (production) URL: ${opts.baselineUrl}`,
19
+ `Target URL: ${opts.targetUrl}`,
20
+ `Masked selectors: ${opts.maskSelectors.length > 0 ? opts.maskSelectors.join(', ') : '(none configured)'}`,
21
+ ];
22
+ if (opts.testCaseUuid) {
23
+ lines.push(`Test case UUID (for context via get_scenario): ${opts.testCaseUuid}`);
24
+ }
25
+ else {
26
+ lines.push('No test case given — no expected_result context available; judge from the screenshots alone.');
27
+ }
28
+ lines.push('Begin now — call capture_and_diff.');
29
+ return lines.join('\n');
30
+ }
31
+ export async function checkVisualRegression(opts) {
32
+ const appqToolDefs = opts.testCaseUuid ? await fetchAppqToolDefs(opts.client, READONLY_CONTEXT_TOOLS) : [];
33
+ const gatedAppq = createGatedAppqDispatcher(opts.client, READONLY_CONTEXT_TOOLS);
34
+ const capture = new CaptureAndDiffTool({
35
+ baselineUrl: opts.baselineUrl,
36
+ targetUrl: opts.targetUrl,
37
+ maskSelectors: opts.maskSelectors,
38
+ storageStatePath: opts.storageStatePath,
39
+ });
40
+ const verdictCapture = new VerdictCapture();
41
+ let diffRan = false;
42
+ const diffState = { data: null };
43
+ const dispatch = async (name, args) => {
44
+ if (name === 'capture_and_diff') {
45
+ const result = await capture.dispatch(name, args);
46
+ diffRan = true;
47
+ const data = result.data;
48
+ if (data && typeof data.diffPixelCount === 'number' && typeof data.diffPercentage === 'number') {
49
+ diffState.data = { diffPixelCount: data.diffPixelCount, diffPercentage: data.diffPercentage };
50
+ }
51
+ return result;
52
+ }
53
+ if (name === 'submit_verdict')
54
+ return verdictCapture.dispatch(name, args);
55
+ return gatedAppq(name, args);
56
+ };
57
+ const loopResult = await runLoop({
58
+ adapter: opts.adapter,
59
+ system: buildVisualRegressionPrompt(),
60
+ seedMessage: seedMessage(opts),
61
+ tools: [...appqToolDefs, ...CAPTURE_TOOL_DEFS, SUBMIT_VERDICT_TOOL_DEF],
62
+ dispatch,
63
+ budget: opts.budget,
64
+ onEvent: opts.onEvent,
65
+ });
66
+ const verdict = verdictCapture.get();
67
+ return {
68
+ report: loopResult.report,
69
+ turns: loopResult.turns,
70
+ budgetExceeded: loopResult.budgetExceeded,
71
+ diffRan,
72
+ diffPixelCount: diffState.data?.diffPixelCount ?? null,
73
+ diffPercentage: diffState.data?.diffPercentage ?? null,
74
+ verdict: verdict?.verdict,
75
+ primaryFinding: verdict?.primaryFinding ?? '',
76
+ secondaryFindings: verdict?.secondaryFindings ?? [],
77
+ };
78
+ }
79
+ //# sourceMappingURL=visualRegression.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"visualRegression.js","sourceRoot":"","sources":["../../src/orchestrator/visualRegression.ts"],"names":[],"mappings":"AAAA,iFAAiF;AACjF,0EAA0E;AAC1E,6EAA6E;AAC7E,6EAA6E;AAC7E,yEAAyE;AACzE,yEAAyE;AACzE,4EAA4E;AAC5E,0EAA0E;AAC1E,iCAAiC;AAEjC,OAAO,EACL,OAAO,EACP,iBAAiB,EACjB,yBAAyB,GAM1B,MAAM,yBAAyB,CAAC;AACjC,OAAO,EAAE,sBAAsB,EAAE,MAAM,oBAAoB,CAAC;AAC5D,OAAO,EAAE,iBAAiB,EAAE,kBAAkB,EAAE,MAAM,4BAA4B,CAAC;AACnF,OAAO,EAAE,uBAAuB,EAAE,cAAc,EAAgB,MAAM,yBAAyB,CAAC;AAChG,OAAO,EAAE,2BAA2B,EAAE,MAAM,qCAAqC,CAAC;AAkClF,SAAS,WAAW,CAAC,IAA6B;IAChD,MAAM,KAAK,GAAG;QACZ,wBAAwB,IAAI,CAAC,KAAK,EAAE;QACpC,8BAA8B,IAAI,CAAC,WAAW,EAAE;QAChD,eAAe,IAAI,CAAC,SAAS,EAAE;QAC/B,qBAAqB,IAAI,CAAC,aAAa,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,mBAAmB,EAAE;KAC3G,CAAC;IACF,IAAI,IAAI,CAAC,YAAY,EAAE,CAAC;QACtB,KAAK,CAAC,IAAI,CAAC,kDAAkD,IAAI,CAAC,YAAY,EAAE,CAAC,CAAC;IACpF,CAAC;SAAM,CAAC;QACN,KAAK,CAAC,IAAI,CAAC,8FAA8F,CAAC,CAAC;IAC7G,CAAC;IACD,KAAK,CAAC,IAAI,CAAC,oCAAoC,CAAC,CAAC;IACjD,OAAO,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC1B,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,qBAAqB,CAAC,IAA6B;IACvE,MAAM,YAAY,GAAG,IAAI,CAAC,YAAY,CAAC,CAAC,CAAC,MAAM,iBAAiB,CAAC,IAAI,CAAC,MAAM,EAAE,sBAAsB,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;IAC3G,MAAM,SAAS,GAAG,yBAAyB,CAAC,IAAI,CAAC,MAAM,EAAE,sBAAsB,CAAC,CAAC;IAEjF,MAAM,OAAO,GAAG,IAAI,kBAAkB,CAAC;QACrC,WAAW,EAAE,IAAI,CAAC,WAAW;QAC7B,SAAS,EAAE,IAAI,CAAC,SAAS;QACzB,aAAa,EAAE,IAAI,CAAC,aAAa;QACjC,gBAAgB,EAAE,IAAI,CAAC,gBAAgB;KACxC,CAAC,CAAC;IACH,MAAM,cAAc,GAAG,IAAI,cAAc,EAAE,CAAC;IAE5C,IAAI,OAAO,GAAG,KAAK,CAAC;IACpB,MAAM,SAAS,GAAwE,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;IAEtG,MAAM,QAAQ,GAAmB,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,EAAE;QACpD,IAAI,IAAI,KAAK,kBAAkB,EAAE,CAAC;YAChC,MAAM,MAAM,GAAe,MAAM,OAAO,CAAC,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;YAC9D,OAAO,GAAG,IAAI,CAAC;YACf,MAAM,IAAI,GAAG,MAAM,CAAC,IAAwE,CAAC;YAC7F,IAAI,IAAI,IAAI,OAAO,IAAI,CAAC,cAAc,KAAK,QAAQ,IAAI,OAAO,IAAI,CAAC,cAAc,KAAK,QAAQ,EAAE,CAAC;gBAC/F,SAAS,CAAC,IAAI,GAAG,EAAE,cAAc,EAAE,IAAI,CAAC,cAAc,EAAE,cAAc,EAAE,IAAI,CAAC,cAAc,EAAE,CAAC;YAChG,CAAC;YACD,OAAO,MAAM,CAAC;QAChB,CAAC;QACD,IAAI,IAAI,KAAK,gBAAgB;YAAE,OAAO,cAAc,CAAC,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;QAC1E,OAAO,SAAS,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;IAC/B,CAAC,CAAC;IAEF,MAAM,UAAU,GAAG,MAAM,OAAO,CAAC;QAC/B,OAAO,EAAE,IAAI,CAAC,OAAO;QACrB,MAAM,EAAE,2BAA2B,EAAE;QACrC,WAAW,EAAE,WAAW,CAAC,IAAI,CAAC;QAC9B,KAAK,EAAE,CAAC,GAAG,YAAY,EAAE,GAAG,iBAAiB,EAAE,uBAAuB,CAAC;QACvE,QAAQ;QACR,MAAM,EAAE,IAAI,CAAC,MAAM;QACnB,OAAO,EAAE,IAAI,CAAC,OAAO;KACtB,CAAC,CAAC;IAEH,MAAM,OAAO,GAAG,cAAc,CAAC,GAAG,EAAE,CAAC;IAErC,OAAO;QACL,MAAM,EAAE,UAAU,CAAC,MAAM;QACzB,KAAK,EAAE,UAAU,CAAC,KAAK;QACvB,cAAc,EAAE,UAAU,CAAC,cAAc;QACzC,OAAO;QACP,cAAc,EAAE,SAAS,CAAC,IAAI,EAAE,cAAc,IAAI,IAAI;QACtD,cAAc,EAAE,SAAS,CAAC,IAAI,EAAE,cAAc,IAAI,IAAI;QACtD,OAAO,EAAE,OAAO,EAAE,OAAO;QACzB,cAAc,EAAE,OAAO,EAAE,cAAc,IAAI,EAAE;QAC7C,iBAAiB,EAAE,OAAO,EAAE,iBAAiB,IAAI,EAAE;KACpD,CAAC;AACJ,CAAC"}
@@ -0,0 +1,75 @@
1
+ // The actual decision-making methodology. There is no appq-served prompt for
2
+ // this — genuinely new methodology, no server-side equivalent exists yet —
3
+ // so this is a local, bundled prompt, driven through @appliqation/agent-core's
4
+ // runLoop() directly, same pattern appliqation-heal-selector and
5
+ // appliqation-autopilot already use for their own local policies.
6
+ export function buildVisualRegressionPrompt() {
7
+ return `You are a narrow specialist: you check ONE route for a real visual regression by comparing it \
8
+ against its own live production counterpart. You never compare against a design file or mock — production \
9
+ IS the source of truth here, fetched live, not a stored baseline. You never touch application code, never \
10
+ file a defect, never take any write action of any kind. Your entire output is a verdict and a report.
11
+
12
+ ## Phase 0 — Prerequisites
13
+
14
+ You have two tool surfaces: read-only Appliqation context (\`get_scenario\`, if a test case was given — \
15
+ its \`expected_result\`/step text is context for what this page is supposed to show, never a substitute \
16
+ for looking at the real screenshots) and exactly one action tool, \`capture_and_diff\`, which takes no \
17
+ arguments — the baseline URL, target URL, and any mask selectors were already fixed before this run \
18
+ started. Call it exactly once.
19
+
20
+ ## Phase 1 — Load context (only if a test case was given)
21
+
22
+ If a test case UUID is present in the seed message, call \`get_scenario\` to load its \`expected_result\`/ \
23
+ step text. This tells you what this page is supposed to show, so you can weigh a real difference against \
24
+ actual intent rather than reasoning about pixels in a vacuum. If no test case was given, skip straight to \
25
+ Phase 2 — this agent works fine with just the two URLs.
26
+
27
+ ## Phase 2 — Capture and diff
28
+
29
+ Call \`capture_and_diff\`. Its result is authoritative: real screenshots, a real diff overlay image, and \
30
+ real, code-computed diff statistics (pixel count, percentage). If it comes back with \
31
+ \`VERDICT: not-applicable (predetermined...)\`, do not override that — that determination was made by the \
32
+ tool itself (the route returned HTTP 404 on the baseline environment), not something for you to second-guess \
33
+ or re-diff. Report \`not-applicable\` and stop; skip Phase 3 entirely.
34
+
35
+ ## Phase 3 — Judge
36
+
37
+ Otherwise, look at all three images (baseline, target, diff overlay) plus the real diff percentage, and \
38
+ classify:
39
+
40
+ - **\`regression\`** — a real, visible break: a missing or overlapping element, a button pushed off-screen \
41
+ or hidden, unreadable/invisible text, a broken layout. This is what a real user would notice as "this looks \
42
+ broken," not just "these pixels differ."
43
+ - **\`expected-divergence\`** — the pixels really do differ, but for a legitimate reason: different \
44
+ data (an article title, a count, a timestamp — especially anything that looks like it should have been \
45
+ masked but wasn't configured to be), a new feature present on the target that isn't live on production yet, \
46
+ a staging banner, an environment label. Different is not automatically broken.
47
+ - **\`inconclusive\`** — you genuinely cannot tell from the evidence. Fail closed here rather than guessing \
48
+ either direction — same discipline \`appq:autotest-validator\`'s own reconciliation uses for ambiguous \
49
+ evidence.
50
+ - **\`not-applicable\`** — only ever set by Phase 2's short-circuit, never something you decide yourself in \
51
+ this phase.
52
+
53
+ **Primary finding vs. secondary observations.** A full-page diff can surface something real that has \
54
+ nothing to do with what this check was actually for (if a test case's \`expected_result\` pointed at one \
55
+ part of the page, or the check is just "does this route look right" broadly). Your **primary finding** is \
56
+ whichever difference is most connected to the page's own stated purpose (or, with no test case given, the \
57
+ single most significant difference you see) — this is what drives your verdict. Anything else real you \
58
+ notice elsewhere on the page goes in **secondary observations** — reported, never silently dropped, but \
59
+ never merged into or affecting the primary verdict. You take no action on either; reporting clearly is the \
60
+ whole job.
61
+
62
+ ## Phase 4 — Report
63
+
64
+ State your verdict plainly, then:
65
+ - Cite the real diff percentage from \`capture_and_diff\`'s own output — never restate a different number, \
66
+ never round away precision that changes the picture.
67
+ - Describe *specifically* what changed and where (e.g. "the 'Subscribe' button in the hero, present on \
68
+ production, is not rendered on target" — not "the hero section looks different").
69
+ - List secondary observations separately and clearly labeled as outside the scope of this check.
70
+ - If the verdict is \`inconclusive\`, say exactly what evidence was missing or ambiguous, not just "unclear."
71
+
72
+ Never blur outcomes together — a human (or another agent) reading your report should immediately know \
73
+ whether this route is fine, broken, or genuinely undecidable, and why.`;
74
+ }
75
+ //# sourceMappingURL=visualRegressionPrompt.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"visualRegressionPrompt.js","sourceRoot":"","sources":["../../src/policy/visualRegressionPrompt.ts"],"names":[],"mappings":"AAAA,6EAA6E;AAC7E,2EAA2E;AAC3E,+EAA+E;AAC/E,iEAAiE;AACjE,kEAAkE;AAElE,MAAM,UAAU,2BAA2B;IACzC,OAAO;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;uEAkE8D,CAAC;AACxE,CAAC"}
@@ -0,0 +1,140 @@
1
+ // The one real piece of new mechanism in this repo. Deliberately NOT driven
2
+ // by the model turn-by-turn (BROWSER_TOOL_DEFS is never offered) — the
3
+ // entire mechanical pipeline (navigate baseline, navigate target, mask,
4
+ // screenshot, pixel-diff) is one atomic, code-owned tool call. The model's
5
+ // only job, in the turn after this returns, is judgment: given the two real
6
+ // screenshots + a diff overlay + real diff stats, classify the result. This
7
+ // keeps the objective measurement itself (pixel counts, dimensions, the
8
+ // not-applicable determination) something the model can never quietly
9
+ // override with its own claim — same "never trust the model's own report"
10
+ // discipline every sibling agent applies to its own domain.
11
+ import { chromium } from 'playwright';
12
+ import { PNG } from 'pngjs';
13
+ import pixelmatch from 'pixelmatch';
14
+ export const CAPTURE_TOOL_DEFS = [
15
+ {
16
+ name: 'capture_and_diff',
17
+ description: 'Navigate to the same route on the baseline (production) and target environment, mask any ' +
18
+ 'configured dynamic regions, take full-page screenshots of both, and pixel-diff them. Takes no ' +
19
+ 'arguments — baseline URL, target URL, and mask selectors were already given when this run started. ' +
20
+ 'Call this exactly once. Returns the two real screenshots, a diff overlay image, and real diff ' +
21
+ 'statistics — or, if the route does not exist on the baseline environment, a pre-determined ' +
22
+ 'not-applicable result with no diff attempted.',
23
+ inputSchema: { type: 'object', properties: {} },
24
+ },
25
+ ];
26
+ function decodePng(buffer) {
27
+ const png = PNG.sync.read(buffer);
28
+ return { width: png.width, height: png.height, data: png.data };
29
+ }
30
+ /**
31
+ * pixelmatch requires both inputs to be the exact same dimensions. Real
32
+ * pages on two environments can legitimately differ in full-page height
33
+ * (different content length, not a bug) — rather than fail, pad both onto a
34
+ * shared canvas sized to the larger of the two, filling new area with a
35
+ * fixed, obviously-synthetic color. The padded region then genuinely shows
36
+ * up as 100% different in the diff, which is correct: a real dimension
37
+ * mismatch is real information for the model to reason about, not something
38
+ * to silently crop away.
39
+ */
40
+ function padToMatch(a, b) {
41
+ const width = Math.max(a.width, b.width);
42
+ const height = Math.max(a.height, b.height);
43
+ if (width === a.width && height === a.height && width === b.width && height === b.height) {
44
+ return { a, b };
45
+ }
46
+ const pad = (img) => {
47
+ const out = new PNG({ width, height });
48
+ out.data.fill(0);
49
+ for (let y = 0; y < img.height; y++) {
50
+ img.data.copy(out.data, (y * width) * 4, y * img.width * 4, y * img.width * 4 + img.width * 4);
51
+ }
52
+ return { width, height, data: out.data };
53
+ };
54
+ return { a: pad(a), b: pad(b) };
55
+ }
56
+ function toLlmImage(buffer, label) {
57
+ return { data: buffer.toString('base64'), mimeType: 'image/png', label };
58
+ }
59
+ async function captureFullPage(page, url, maskSelectors) {
60
+ const response = await page.goto(url, { waitUntil: 'networkidle' }).catch(() => null);
61
+ const httpStatus = response?.status() ?? null;
62
+ if (httpStatus === 404) {
63
+ return { status: 'not-found', httpStatus, png: null };
64
+ }
65
+ // Best-effort: let CSS transitions/animations settle before capture, same
66
+ // reasoning every screenshot-diff tool needs — a mid-transition frame is
67
+ // noise, not a real difference.
68
+ await page.addStyleTag({ content: '*, *::before, *::after { animation-duration: 0s !important; animation-delay: 0s !important; transition-duration: 0s !important; }' }).catch(() => { });
69
+ const mask = maskSelectors.map((selector) => page.locator(selector));
70
+ const png = await page.screenshot({ fullPage: true, type: 'png', mask });
71
+ return { status: 'ok', httpStatus, png };
72
+ }
73
+ export class CaptureAndDiffTool {
74
+ opts;
75
+ constructor(opts) {
76
+ this.opts = opts;
77
+ }
78
+ async dispatch(name, _args) {
79
+ if (name !== 'capture_and_diff') {
80
+ return { ok: false, text: `Unknown tool: ${name}` };
81
+ }
82
+ const browser = await chromium.launch();
83
+ try {
84
+ const context = await browser.newContext(this.opts.storageStatePath ? { storageState: this.opts.storageStatePath } : {});
85
+ const page = await context.newPage();
86
+ const baseline = await captureFullPage(page, this.opts.baselineUrl, this.opts.maskSelectors);
87
+ if (baseline.status === 'not-found') {
88
+ return {
89
+ ok: true,
90
+ text: `VERDICT: not-applicable (predetermined by capture_and_diff, do not override this)\n` +
91
+ `Reason: the baseline (production) URL ${this.opts.baselineUrl} returned HTTP 404 — this route ` +
92
+ `does not exist on the baseline environment. No comparison is possible; the target environment ` +
93
+ `was never navigated to.`,
94
+ };
95
+ }
96
+ const target = await captureFullPage(page, this.opts.targetUrl, this.opts.maskSelectors);
97
+ if (target.status === 'not-found') {
98
+ return {
99
+ ok: true,
100
+ text: `VERDICT: not-applicable (predetermined by capture_and_diff, do not override this)\n` +
101
+ `Reason: the baseline exists (HTTP ${baseline.httpStatus}) but the target URL ${this.opts.targetUrl} ` +
102
+ `returned HTTP 404. No comparison is possible.`,
103
+ };
104
+ }
105
+ const decodedBaseline = decodePng(baseline.png);
106
+ const decodedTarget = decodePng(target.png);
107
+ const { a: paddedBaseline, b: paddedTarget } = padToMatch(decodedBaseline, decodedTarget);
108
+ const diffPng = new PNG({ width: paddedBaseline.width, height: paddedBaseline.height });
109
+ const diffPixelCount = pixelmatch(paddedBaseline.data, paddedTarget.data, diffPng.data, paddedBaseline.width, paddedBaseline.height, { threshold: 0.1 });
110
+ const totalPixels = paddedBaseline.width * paddedBaseline.height;
111
+ const diffPercentage = totalPixels > 0 ? (diffPixelCount / totalPixels) * 100 : 0;
112
+ const diffOverlayBuffer = PNG.sync.write(diffPng);
113
+ const dimensionNote = decodedBaseline.width !== decodedTarget.width || decodedBaseline.height !== decodedTarget.height
114
+ ? ` Note: baseline is ${decodedBaseline.width}x${decodedBaseline.height}, target is ` +
115
+ `${decodedTarget.width}x${decodedTarget.height} — different full-page dimensions, padded onto a ` +
116
+ `shared ${paddedBaseline.width}x${paddedBaseline.height} canvas before diffing. The padded region ` +
117
+ `shows up as fully different in the overlay; that may just reflect different content length, not a ` +
118
+ `layout break — reason about it accordingly.`
119
+ : '';
120
+ return {
121
+ ok: true,
122
+ text: `Baseline (production): ${this.opts.baselineUrl} (HTTP ${baseline.httpStatus})\n` +
123
+ `Target: ${this.opts.targetUrl} (HTTP ${target.httpStatus})\n` +
124
+ `Masked selectors: ${this.opts.maskSelectors.length > 0 ? this.opts.maskSelectors.join(', ') : '(none)'}\n` +
125
+ `Real diff: ${diffPixelCount} pixels differ out of ${totalPixels} (${diffPercentage.toFixed(2)}%).${dimensionNote}\n` +
126
+ `These numbers are real and code-computed — never restate a different figure.`,
127
+ data: { diffPixelCount, diffPercentage, baselineHttpStatus: baseline.httpStatus, targetHttpStatus: target.httpStatus },
128
+ images: [
129
+ toLlmImage(baseline.png, `Baseline (production) — ${this.opts.baselineUrl}`),
130
+ toLlmImage(target.png, `Target — ${this.opts.targetUrl}`),
131
+ toLlmImage(diffOverlayBuffer, 'Diff overlay (magenta/highlighted = differs)'),
132
+ ],
133
+ };
134
+ }
135
+ finally {
136
+ await browser.close();
137
+ }
138
+ }
139
+ }
140
+ //# sourceMappingURL=captureAndDiff.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"captureAndDiff.js","sourceRoot":"","sources":["../../src/tools/captureAndDiff.ts"],"names":[],"mappings":"AAAA,4EAA4E;AAC5E,uEAAuE;AACvE,wEAAwE;AACxE,2EAA2E;AAC3E,4EAA4E;AAC5E,4EAA4E;AAC5E,wEAAwE;AACxE,sEAAsE;AACtE,0EAA0E;AAC1E,4DAA4D;AAE5D,OAAO,EAAE,QAAQ,EAA2B,MAAM,YAAY,CAAC;AAC/D,OAAO,EAAE,GAAG,EAAE,MAAM,OAAO,CAAC;AAC5B,OAAO,UAAU,MAAM,YAAY,CAAC;AAGpC,MAAM,CAAC,MAAM,iBAAiB,GAAiB;IAC7C;QACE,IAAI,EAAE,kBAAkB;QACxB,WAAW,EACT,2FAA2F;YAC3F,gGAAgG;YAChG,qGAAqG;YACrG,gGAAgG;YAChG,6FAA6F;YAC7F,+CAA+C;QACjD,WAAW,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,UAAU,EAAE,EAAE,EAAE;KAChD;CACF,CAAC;AAeF,SAAS,SAAS,CAAC,MAAc;IAC/B,MAAM,GAAG,GAAG,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IAClC,OAAO,EAAE,KAAK,EAAE,GAAG,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,CAAC,MAAM,EAAE,IAAI,EAAE,GAAG,CAAC,IAAI,EAAE,CAAC;AAClE,CAAC;AAED;;;;;;;;;GASG;AACH,SAAS,UAAU,CAAC,CAAa,EAAE,CAAa;IAC9C,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,KAAK,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC;IACzC,MAAM,MAAM,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC;IAC5C,IAAI,KAAK,KAAK,CAAC,CAAC,KAAK,IAAI,MAAM,KAAK,CAAC,CAAC,MAAM,IAAI,KAAK,KAAK,CAAC,CAAC,KAAK,IAAI,MAAM,KAAK,CAAC,CAAC,MAAM,EAAE,CAAC;QACzF,OAAO,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC;IAClB,CAAC;IACD,MAAM,GAAG,GAAG,CAAC,GAAe,EAAc,EAAE;QAC1C,MAAM,GAAG,GAAG,IAAI,GAAG,CAAC,EAAE,KAAK,EAAE,MAAM,EAAE,CAAC,CAAC;QACvC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QACjB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,GAAG,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YACpC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,GAAG,KAAK,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,GAAG,CAAC,KAAK,GAAG,CAAC,EAAE,CAAC,GAAG,GAAG,CAAC,KAAK,GAAG,CAAC,GAAG,GAAG,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC;QACjG,CAAC;QACD,OAAO,EAAE,KAAK,EAAE,MAAM,EAAE,IAAI,EAAE,GAAG,CAAC,IAAI,EAAE,CAAC;IAC3C,CAAC,CAAC;IACF,OAAO,EAAE,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC;AAClC,CAAC;AAED,SAAS,UAAU,CAAC,MAAc,EAAE,KAAa;IAC/C,OAAO,EAAE,IAAI,EAAE,MAAM,CAAC,QAAQ,CAAC,QAAQ,CAAC,EAAE,QAAQ,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;AAC3E,CAAC;AAED,KAAK,UAAU,eAAe,CAAC,IAAU,EAAE,GAAW,EAAE,aAAuB;IAC7E,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,EAAE,SAAS,EAAE,aAAa,EAAE,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,CAAC;IACtF,MAAM,UAAU,GAAG,QAAQ,EAAE,MAAM,EAAE,IAAI,IAAI,CAAC;IAC9C,IAAI,UAAU,KAAK,GAAG,EAAE,CAAC;QACvB,OAAO,EAAE,MAAM,EAAE,WAAW,EAAE,UAAU,EAAE,GAAG,EAAE,IAAI,EAAE,CAAC;IACxD,CAAC;IACD,0EAA0E;IAC1E,yEAAyE;IACzE,gCAAgC;IAChC,MAAM,IAAI,CAAC,WAAW,CAAC,EAAE,OAAO,EAAE,mIAAmI,EAAE,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;IACzL,MAAM,IAAI,GAAc,aAAa,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,CAAC,CAAC;IAChF,MAAM,GAAG,GAAG,MAAM,IAAI,CAAC,UAAU,CAAC,EAAE,QAAQ,EAAE,IAAI,EAAE,IAAI,EAAE,KAAK,EAAE,IAAI,EAAE,CAAC,CAAC;IACzE,OAAO,EAAE,MAAM,EAAE,IAAI,EAAE,UAAU,EAAE,GAAG,EAAE,CAAC;AAC3C,CAAC;AAED,MAAM,OAAO,kBAAkB;IACT;IAApB,YAAoB,IAA2B;QAA3B,SAAI,GAAJ,IAAI,CAAuB;IAAG,CAAC;IAEnD,KAAK,CAAC,QAAQ,CAAC,IAAY,EAAE,KAA8B;QACzD,IAAI,IAAI,KAAK,kBAAkB,EAAE,CAAC;YAChC,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,IAAI,EAAE,iBAAiB,IAAI,EAAE,EAAE,CAAC;QACtD,CAAC;QAED,MAAM,OAAO,GAAG,MAAM,QAAQ,CAAC,MAAM,EAAE,CAAC;QACxC,IAAI,CAAC;YACH,MAAM,OAAO,GAAG,MAAM,OAAO,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,gBAAgB,CAAC,CAAC,CAAC,EAAE,YAAY,EAAE,IAAI,CAAC,IAAI,CAAC,gBAAgB,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;YACzH,MAAM,IAAI,GAAG,MAAM,OAAO,CAAC,OAAO,EAAE,CAAC;YAErC,MAAM,QAAQ,GAAG,MAAM,eAAe,CAAC,IAAI,EAAE,IAAI,CAAC,IAAI,CAAC,WAAW,EAAE,IAAI,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC;YAC7F,IAAI,QAAQ,CAAC,MAAM,KAAK,WAAW,EAAE,CAAC;gBACpC,OAAO;oBACL,EAAE,EAAE,IAAI;oBACR,IAAI,EACF,qFAAqF;wBACrF,yCAAyC,IAAI,CAAC,IAAI,CAAC,WAAW,kCAAkC;wBAChG,gGAAgG;wBAChG,yBAAyB;iBAC5B,CAAC;YACJ,CAAC;YAED,MAAM,MAAM,GAAG,MAAM,eAAe,CAAC,IAAI,EAAE,IAAI,CAAC,IAAI,CAAC,SAAS,EAAE,IAAI,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC;YACzF,IAAI,MAAM,CAAC,MAAM,KAAK,WAAW,EAAE,CAAC;gBAClC,OAAO;oBACL,EAAE,EAAE,IAAI;oBACR,IAAI,EACF,qFAAqF;wBACrF,qCAAqC,QAAQ,CAAC,UAAU,wBAAwB,IAAI,CAAC,IAAI,CAAC,SAAS,GAAG;wBACtG,+CAA+C;iBAClD,CAAC;YACJ,CAAC;YAED,MAAM,eAAe,GAAG,SAAS,CAAC,QAAQ,CAAC,GAAI,CAAC,CAAC;YACjD,MAAM,aAAa,GAAG,SAAS,CAAC,MAAM,CAAC,GAAI,CAAC,CAAC;YAC7C,MAAM,EAAE,CAAC,EAAE,cAAc,EAAE,CAAC,EAAE,YAAY,EAAE,GAAG,UAAU,CAAC,eAAe,EAAE,aAAa,CAAC,CAAC;YAE1F,MAAM,OAAO,GAAG,IAAI,GAAG,CAAC,EAAE,KAAK,EAAE,cAAc,CAAC,KAAK,EAAE,MAAM,EAAE,cAAc,CAAC,MAAM,EAAE,CAAC,CAAC;YACxF,MAAM,cAAc,GAAG,UAAU,CAAC,cAAc,CAAC,IAAI,EAAE,YAAY,CAAC,IAAI,EAAE,OAAO,CAAC,IAAI,EAAE,cAAc,CAAC,KAAK,EAAE,cAAc,CAAC,MAAM,EAAE,EAAE,SAAS,EAAE,GAAG,EAAE,CAAC,CAAC;YACzJ,MAAM,WAAW,GAAG,cAAc,CAAC,KAAK,GAAG,cAAc,CAAC,MAAM,CAAC;YACjE,MAAM,cAAc,GAAG,WAAW,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,cAAc,GAAG,WAAW,CAAC,GAAG,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC;YAClF,MAAM,iBAAiB,GAAG,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC;YAElD,MAAM,aAAa,GACjB,eAAe,CAAC,KAAK,KAAK,aAAa,CAAC,KAAK,IAAI,eAAe,CAAC,MAAM,KAAK,aAAa,CAAC,MAAM;gBAC9F,CAAC,CAAC,sBAAsB,eAAe,CAAC,KAAK,IAAI,eAAe,CAAC,MAAM,cAAc;oBACnF,GAAG,aAAa,CAAC,KAAK,IAAI,aAAa,CAAC,MAAM,mDAAmD;oBACjG,UAAU,cAAc,CAAC,KAAK,IAAI,cAAc,CAAC,MAAM,4CAA4C;oBACnG,oGAAoG;oBACpG,6CAA6C;gBAC/C,CAAC,CAAC,EAAE,CAAC;YAET,OAAO;gBACL,EAAE,EAAE,IAAI;gBACR,IAAI,EACF,0BAA0B,IAAI,CAAC,IAAI,CAAC,WAAW,UAAU,QAAQ,CAAC,UAAU,KAAK;oBACjF,WAAW,IAAI,CAAC,IAAI,CAAC,SAAS,UAAU,MAAM,CAAC,UAAU,KAAK;oBAC9D,qBAAqB,IAAI,CAAC,IAAI,CAAC,aAAa,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,QAAQ,IAAI;oBAC3G,cAAc,cAAc,yBAAyB,WAAW,KAAK,cAAc,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,aAAa,IAAI;oBACrH,8EAA8E;gBAChF,IAAI,EAAE,EAAE,cAAc,EAAE,cAAc,EAAE,kBAAkB,EAAE,QAAQ,CAAC,UAAU,EAAE,gBAAgB,EAAE,MAAM,CAAC,UAAU,EAAE;gBACtH,MAAM,EAAE;oBACN,UAAU,CAAC,QAAQ,CAAC,GAAI,EAAE,2BAA2B,IAAI,CAAC,IAAI,CAAC,WAAW,EAAE,CAAC;oBAC7E,UAAU,CAAC,MAAM,CAAC,GAAI,EAAE,YAAY,IAAI,CAAC,IAAI,CAAC,SAAS,EAAE,CAAC;oBAC1D,UAAU,CAAC,iBAAiB,EAAE,8CAA8C,CAAC;iBAC9E;aACF,CAAC;QACJ,CAAC;gBAAS,CAAC;YACT,MAAM,OAAO,CAAC,KAAK,EAAE,CAAC;QACxB,CAAC;IACH,CAAC;CACF"}
@@ -0,0 +1,13 @@
1
+ // This agent's own domain knowledge of which appq tools it may touch — the
2
+ // enforcement mechanism (assertToolAllowed / the gated dispatcher) lives in
3
+ // @appliqation/agent-core, shared with every sibling agent; only the
4
+ // allowlist content is local. Zero write tools — genuinely absent, not
5
+ // gated behind a flag. This agent never calls an appq write tool; it only
6
+ // ever reports a verdict, never files a defect or persists anything.
7
+ export const READONLY_CONTEXT_TOOLS = new Set([
8
+ // The test case's own expected_result/steps, when a --test-case-uuid was
9
+ // given — the ground truth a diff's "is this expected" judgment gets
10
+ // checked against. Optional context, not required to proceed.
11
+ 'get_scenario',
12
+ ]);
13
+ //# sourceMappingURL=safety.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"safety.js","sourceRoot":"","sources":["../../src/tools/safety.ts"],"names":[],"mappings":"AAAA,2EAA2E;AAC3E,4EAA4E;AAC5E,qEAAqE;AACrE,uEAAuE;AACvE,0EAA0E;AAC1E,qEAAqE;AAErE,MAAM,CAAC,MAAM,sBAAsB,GAAG,IAAI,GAAG,CAAC;IAC5C,yEAAyE;IACzE,qEAAqE;IACrE,8DAA8D;IAC9D,cAAc;CACf,CAAC,CAAC"}
@@ -0,0 +1,54 @@
1
+ // The verdict itself is inherently a judgment call — unlike heal-selector's
2
+ // declined/verified (both derivable from real code signals: file writes, a
3
+ // real command exit code), there's no objective code-computed equivalent
4
+ // for "is this a regression." Rather than parse it out of free-text report
5
+ // prose (fragile), the model reports it through a real tool call's
6
+ // structured arguments — same discipline `appq:autotest-validator` already
7
+ // uses (writing its verdict via `update_run_results`'s real enum argument,
8
+ // not asserting it in prose). This tool never causes a side effect; it's
9
+ // purely how the model's judgment reaches this process as structured data.
10
+ export const SUBMIT_VERDICT_TOOL_DEF = {
11
+ name: 'submit_verdict',
12
+ description: 'Report your final judgment. Call this exactly once, after capture_and_diff (and get_scenario, if a ' +
13
+ 'test case was given) — never before you have the real diff evidence. This is how your verdict reaches ' +
14
+ 'the caller; a verdict only stated in your closing prose is not captured.',
15
+ inputSchema: {
16
+ type: 'object',
17
+ properties: {
18
+ verdict: {
19
+ type: 'string',
20
+ enum: ['regression', 'expected-divergence', 'not-applicable', 'inconclusive'],
21
+ description: 'regression = a real visible break. expected-divergence = pixels differ but for a legitimate ' +
22
+ 'reason (data, a new unreleased feature, a staging banner). inconclusive = you genuinely cannot ' +
23
+ 'tell — fail closed. not-applicable = only ever set because capture_and_diff itself pre-determined ' +
24
+ 'it (route missing on baseline) — never decide this yourself in any other case.',
25
+ },
26
+ primary_finding: {
27
+ type: 'string',
28
+ description: 'The one finding most connected to what this check was for — what drove the verdict above. Be specific about what and where.',
29
+ },
30
+ secondary_findings: {
31
+ type: 'array',
32
+ items: { type: 'string' },
33
+ description: 'Other real differences noticed elsewhere on the page, unrelated to the primary finding. Empty array if none. Never affects the verdict above.',
34
+ },
35
+ },
36
+ required: ['verdict', 'primary_finding'],
37
+ },
38
+ };
39
+ export class VerdictCapture {
40
+ captured;
41
+ dispatch(name, args) {
42
+ if (name !== 'submit_verdict')
43
+ return { ok: false, text: `Unknown tool: ${name}` };
44
+ const verdict = args.verdict;
45
+ const primaryFinding = String(args.primary_finding ?? '');
46
+ const secondaryFindings = Array.isArray(args.secondary_findings) ? args.secondary_findings.map(String) : [];
47
+ this.captured = { verdict, primaryFinding, secondaryFindings };
48
+ return { ok: true, text: 'Verdict recorded.' };
49
+ }
50
+ get() {
51
+ return this.captured;
52
+ }
53
+ }
54
+ //# sourceMappingURL=verdictTool.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"verdictTool.js","sourceRoot":"","sources":["../../src/tools/verdictTool.ts"],"names":[],"mappings":"AAAA,4EAA4E;AAC5E,2EAA2E;AAC3E,yEAAyE;AACzE,2EAA2E;AAC3E,mEAAmE;AACnE,2EAA2E;AAC3E,2EAA2E;AAC3E,yEAAyE;AACzE,2EAA2E;AAM3E,MAAM,CAAC,MAAM,uBAAuB,GAAe;IACjD,IAAI,EAAE,gBAAgB;IACtB,WAAW,EACT,qGAAqG;QACrG,wGAAwG;QACxG,0EAA0E;IAC5E,WAAW,EAAE;QACX,IAAI,EAAE,QAAQ;QACd,UAAU,EAAE;YACV,OAAO,EAAE;gBACP,IAAI,EAAE,QAAQ;gBACd,IAAI,EAAE,CAAC,YAAY,EAAE,qBAAqB,EAAE,gBAAgB,EAAE,cAAc,CAAC;gBAC7E,WAAW,EACT,8FAA8F;oBAC9F,iGAAiG;oBACjG,oGAAoG;oBACpG,gFAAgF;aACnF;YACD,eAAe,EAAE;gBACf,IAAI,EAAE,QAAQ;gBACd,WAAW,EAAE,6HAA6H;aAC3I;YACD,kBAAkB,EAAE;gBAClB,IAAI,EAAE,OAAO;gBACb,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACzB,WAAW,EAAE,+IAA+I;aAC7J;SACF;QACD,QAAQ,EAAE,CAAC,SAAS,EAAE,iBAAiB,CAAC;KACzC;CACF,CAAC;AAQF,MAAM,OAAO,cAAc;IACjB,QAAQ,CAA8B;IAE9C,QAAQ,CAAC,IAAY,EAAE,IAA6B;QAClD,IAAI,IAAI,KAAK,gBAAgB;YAAE,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,IAAI,EAAE,iBAAiB,IAAI,EAAE,EAAE,CAAC;QACnF,MAAM,OAAO,GAAG,IAAI,CAAC,OAAkB,CAAC;QACxC,MAAM,cAAc,GAAG,MAAM,CAAC,IAAI,CAAC,eAAe,IAAI,EAAE,CAAC,CAAC;QAC1D,MAAM,iBAAiB,GAAG,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,kBAAkB,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,kBAAkB,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;QAC5G,IAAI,CAAC,QAAQ,GAAG,EAAE,OAAO,EAAE,cAAc,EAAE,iBAAiB,EAAE,CAAC;QAC/D,OAAO,EAAE,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,mBAAmB,EAAE,CAAC;IACjD,CAAC;IAED,GAAG;QACD,OAAO,IAAI,CAAC,QAAQ,CAAC;IACvB,CAAC;CACF"}
package/package.json ADDED
@@ -0,0 +1,52 @@
1
+ {
2
+ "name": "@appliqation/visual-regression",
3
+ "version": "0.1.0",
4
+ "description": "Standalone agent that checks a page for real visual regressions by diffing it against its own live production counterpart — no stored baseline files, no manual baseline-approval workflow. Declines rather than compares when no production equivalent exists.",
5
+ "license": "MIT",
6
+ "repository": {
7
+ "type": "git",
8
+ "url": "git+https://github.com/appliqation/visual-regression.git"
9
+ },
10
+ "homepage": "https://github.com/appliqation/visual-regression#readme",
11
+ "bugs": {
12
+ "url": "https://github.com/appliqation/visual-regression/issues"
13
+ },
14
+ "type": "module",
15
+ "bin": {
16
+ "appliqation-visual-regression": "./dist/cli/index.js"
17
+ },
18
+ "engines": {
19
+ "node": ">=20"
20
+ },
21
+ "files": [
22
+ "dist/",
23
+ "README.md",
24
+ "LICENSE"
25
+ ],
26
+ "scripts": {
27
+ "build": "tsc -p tsconfig.build.json",
28
+ "dev": "tsx src/cli/index.ts",
29
+ "typecheck": "tsc -p tsconfig.json --noEmit",
30
+ "lint": "eslint src --ext .ts",
31
+ "test": "vitest run",
32
+ "test:watch": "vitest"
33
+ },
34
+ "dependencies": {
35
+ "@appliqation/agent-core": "^0.1.5",
36
+ "commander": "^13.1.0",
37
+ "dotenv": "^16.4.7",
38
+ "pixelmatch": "^7.2.0",
39
+ "playwright": "^1.51.0",
40
+ "pngjs": "^7.0.0"
41
+ },
42
+ "devDependencies": {
43
+ "@eslint/js": "^9.39.5",
44
+ "@types/node": "^22.13.10",
45
+ "@types/pngjs": "^6.0.5",
46
+ "eslint": "^9.39.5",
47
+ "tsx": "^4.19.3",
48
+ "typescript": "^5.8.2",
49
+ "typescript-eslint": "^8.67.0",
50
+ "vitest": "^3.2.7"
51
+ }
52
+ }