styleproof 4.2.0 → 4.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +64 -0
- package/README.md +2 -1
- package/bin/styleproof-capture.mjs +15 -1
- package/bin/styleproof-diff.mjs +79 -28
- package/bin/styleproof-map.mjs +3 -1
- package/dist/crawl-surfaces.js +8 -0
- package/dist/diff.d.ts +1 -0
- package/dist/diff.js +13 -6
- package/dist/report.js +1 -1
- package/dist/runner.d.ts +12 -0
- package/dist/runner.js +29 -5
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,70 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [4.4.0] - 2026-07-13
|
|
11
|
+
|
|
12
|
+
### Changed
|
|
13
|
+
|
|
14
|
+
- **Spec-driven captures now run in parallel across Playwright workers.** The
|
|
15
|
+
capture describe declares `parallel` mode itself, so surface×width tests fan
|
|
16
|
+
out over the consumer's `workers` even when the project pins
|
|
17
|
+
`fullyParallel: false` for its behaviour suite. Every generated test was
|
|
18
|
+
already independent (own page, own map/HAR files, own self-check; ledger and
|
|
19
|
+
manifest tests mkdir and tolerate any order), so the maps are byte-identical —
|
|
20
|
+
a real consumer's 150-capture run dropped from 24.5 to 6.0 minutes at 4
|
|
21
|
+
workers on the same box. A spec file whose OWN sibling tests read the captured
|
|
22
|
+
maps in file order can opt out with `parallel: false`. Found while
|
|
23
|
+
dogfooding against a large single-route consumer app.
|
|
24
|
+
|
|
25
|
+
### Fixed
|
|
26
|
+
|
|
27
|
+
- **An absolute `STYLEMAP_DIR` (or `--dir`) is respected as-is.** It was joined
|
|
28
|
+
under `baseDir`, so `STYLEMAP_DIR=/abs/path styleproof-map` stranded the maps
|
|
29
|
+
at `.styleproof/maps/abs/path` where no consumer looks. Relative dirs nest
|
|
30
|
+
under `baseDir` unchanged. Also found dogfooding against a consumer app.
|
|
31
|
+
|
|
32
|
+
## [4.3.0] - 2026-07-13
|
|
33
|
+
|
|
34
|
+
### Changed
|
|
35
|
+
|
|
36
|
+
- **A removed surface now blocks (exit 1) instead of riding the exit-3 "only new
|
|
37
|
+
surfaces" path.** A surface captured only on the base side is a deleted route
|
|
38
|
+
or a dropped width — a reviewable change, printed as `✗ REMOVED surface` —
|
|
39
|
+
never an onboarding case the approve-all box waves through under a "new
|
|
40
|
+
surfaces" banner. Exit 3 now means only NEW (head-side-only) surfaces.
|
|
41
|
+
- **Stale acknowledgements gate everywhere.** A stale `allowRemoved` entry in
|
|
42
|
+
`styleproof.inventory.json` now blocks like a stale data-residue
|
|
43
|
+
acknowledgement (and the Action's inventory hard-gate counts both), so a
|
|
44
|
+
ledger entry left behind cannot pre-acknowledge the next removal of that key.
|
|
45
|
+
|
|
46
|
+
### Fixed
|
|
47
|
+
|
|
48
|
+
- **Crawls never capture an off-origin redirect target.** A same-origin link
|
|
49
|
+
that 302s to another origin (SSO, `/out?url=…`) previously entered the map as
|
|
50
|
+
a surface — third-party, nondeterministic content. The CLI sweep now skips the
|
|
51
|
+
page loudly and continues; a spec-driven capture fails naming the surface; an
|
|
52
|
+
entry URL that redirects off-origin is a hard error.
|
|
53
|
+
- **The gate now surfaces what it did NOT compare.** The diff prints (and
|
|
54
|
+
`--json` carries) the count of auto-detected volatile subtrees excluded from
|
|
55
|
+
the comparison, and warns when the forced-state layer was skipped on BOTH
|
|
56
|
+
sides — previously a head-side volatile region or a twice-skipped state layer
|
|
57
|
+
could hide a real change with zero trace in the output.
|
|
58
|
+
- **A corrupt coverage ledger is a hard error (exit 2), not a silent disarm.**
|
|
59
|
+
Unparseable `styleproof-coverage.json` used to read as "no registry", quietly
|
|
60
|
+
turning the coverage, determinism, and residue gates into warnings at once.
|
|
61
|
+
The residue footer also no longer misattributes a ledger-less bundle to the
|
|
62
|
+
`dataResidue: "warn"` opt-out.
|
|
63
|
+
- **The Action fails closed on unexpected diff exit codes.** A node crash, OOM
|
|
64
|
+
kill, or missing file (127/137/143/…) previously fell through to
|
|
65
|
+
`changed=false` — a green "No visual changes" status for a run that never
|
|
66
|
+
compared anything.
|
|
67
|
+
- **Approval can no longer wave through certification failures.** The Action
|
|
68
|
+
gained a post-approval hard gate (like the inventory gate) for incomplete
|
|
69
|
+
coverage, unproven determinism, and armed data-residue failures — none of
|
|
70
|
+
which are restyles the approve-all checkbox should clear.
|
|
71
|
+
- Stale `exclude` entries are printed with the coverage verdict, and "coverage
|
|
72
|
+
complete" now says "captured or explicitly excluded".
|
|
73
|
+
|
|
10
74
|
## [4.2.0] - 2026-07-13
|
|
11
75
|
|
|
12
76
|
### Changed
|
package/README.md
CHANGED
|
@@ -986,6 +986,7 @@ It's **asynchronous by design**: approval is a checkbox tick handled by a separa
|
|
|
986
986
|
| `dataResidue` | `'gate'` | Name data-boundary (`replayUrl`) requests that **fail** during capture (network error / 4xx/5xx — the fallback branch got captured). Always warned + recorded; `'gate'` (the default) also blocks the diff on an unacknowledged one, `'warn'` is the opt-out that records + warns without gating. See [Data residue](#data-residue-a-failed-data-request-is-named-not-swallowed). |
|
|
987
987
|
| `freezeClock` | `true` | Pin `Date.now()`/`new Date()` so time-derived styling can't drift; timers keep running so settling still works. |
|
|
988
988
|
| `clockTime` | `2025-01-01T00:00:00Z` | The frozen instant. |
|
|
989
|
+
| `parallel` | `true` | Run the generated capture tests across Playwright workers, even when the project config pins `fullyParallel: false` — every capture test is independent, so a multi-surface spec speeds up ~workers×. Set `false` only for a spec file whose own sibling tests read the captured maps in file order. |
|
|
989
990
|
| `selfCheck` | on while recording | Capture each surface twice and fail on any difference — proves the capture is deterministic. Off on the replay run; `STYLEPROOF_SELFCHECK=1` forces both. |
|
|
990
991
|
| `screenshots` | `true` | Save full-page screenshots for the report's before/after crops. |
|
|
991
992
|
| `baseDir` | `__stylemaps__` | Output root directory. |
|
|
@@ -1013,7 +1014,7 @@ Non-visual and framework-injected elements (`<meta>`/`<title>`/`<script>`/`<styl
|
|
|
1013
1014
|
|
|
1014
1015
|
- `styleproof-init` — scaffold the gate: the capture spec (inventory guard on; Next.js repos get route discovery + the coverage guard, others a crawl-by-default spec), a dedicated `playwright.styleproof.config.ts` (production-build `webServer`, parallel capture), `.gitignore` cache entries, the cache-first report workflow, the approval workflow, and the pre-push publish hook. One command. Generated commands follow the repo's lockfile (`bun.lock`/`bun.lockb`, `pnpm-lock.yaml`, `yarn.lock`, or npm by default), respect pnpm/Corepack version pins, and detect Vite/Next production preview commands instead of assuming every repo has `start`.
|
|
1015
1016
|
- `styleproof-map` — capture the current commit's computed-style map through Playwright. By default it writes `.styleproof/maps/current`, keeps screenshots for reports, writes a manifest, and uploads to `styleproof-maps` outside CI when the working tree was clean and a git remote exists. Pass `--crawl-base-url` plus repeated `--crawl-route` to run `styleproof-variants` before capture, `--no-upload`, `--restore --sha <commit>`, `--spec`, `--dir`, `--base-dir`, or `--no-screenshots` for custom flows.
|
|
1016
|
-
- `styleproof-diff` — the certify gate. With no args, it restores cached maps for the current commit and inferred base (`GITHUB_BASE_REF`, `branch.<name>.gh-merge-base`, `gh pr view`, then main/master fallbacks); `styleproof-diff main` / `styleproof-diff master` pins the base; `styleproof-diff <beforeDir> <afterDir>` keeps the manual two-directory form for CI fallback captures. Exits `0` certified (identical); `1` on a reviewable diff — computed-style/DOM/state differences, and equally an unacknowledged inventory removal, an unacknowledged failing data endpoint under an armed `dataResidue: 'gate'`, an incomplete coverage registry, or an unproven-determinism capture; `2` on a usage/capture error (including a **manifest-less side** — since **v4**, a two-directory compare where a side ships maps but no `styleproof-manifest.json` is refused loudly, naming the bare side(s), because the same-environment guard can't be enforced without one; re-capture with current StyleProof; **and** a **missing map** — a bundle that claims to exist yet holds zero captures, i.e. a `styleproof-manifest.json` present with no maps, on either side, or a head capture that produced nothing; refused loudly rather than mislabelled as all-new — **and** the no-args case where the cached base map can't be restored at all: no map-store remote, no cached bundle, nothing to compare. A "nothing was compared" outcome always exits `2`, never a soft `0` that would read as certified; the error names the two ways forward — run in CI where the base is restorable, or use the two-directory form); `3` when only new surfaces are present (no baseline for _those_ surfaces to diff against — new surfaces against an existing baseline, or a base dir with no maps at all (and hence no manifest), meaning no baseline was ever captured: the first-adoption review path; approval policy decides whether to gate). A clean run prints `0 changed surfaces across N captured surface(s)`, and `--json` includes `compared`. The human output **groups the same way the report does**: surfaces that changed identically collapse into one finding (with the per-surface count on its header), longhands fold into shorthands, and size/position-derived longhands fold behind a `(+N derived longhands)` count — so one real change reads as one entry, not dozens of raw lines. A change that rode the shared frame every view draws (a persistent nav/header/footer) is promoted to a "🧱 Global chrome change" callout up top. `--json` stays the complete, unchanged machine contract — every surface and every raw longhand — regardless of the human grouping.
|
|
1017
|
+
- `styleproof-diff` — the certify gate. With no args, it restores cached maps for the current commit and inferred base (`GITHUB_BASE_REF`, `branch.<name>.gh-merge-base`, `gh pr view`, then main/master fallbacks); `styleproof-diff main` / `styleproof-diff master` pins the base; `styleproof-diff <beforeDir> <afterDir>` keeps the manual two-directory form for CI fallback captures. Exits `0` certified (identical); `1` on a reviewable diff — computed-style/DOM/state differences, and equally an unacknowledged inventory removal, an unacknowledged failing data endpoint under an armed `dataResidue: 'gate'`, an incomplete coverage registry, or an unproven-determinism capture; `2` on a usage/capture error (including a **manifest-less side** — since **v4**, a two-directory compare where a side ships maps but no `styleproof-manifest.json` is refused loudly, naming the bare side(s), because the same-environment guard can't be enforced without one; re-capture with current StyleProof; **and** a **missing map** — a bundle that claims to exist yet holds zero captures, i.e. a `styleproof-manifest.json` present with no maps, on either side, or a head capture that produced nothing; refused loudly rather than mislabelled as all-new — **and** the no-args case where the cached base map can't be restored at all: no map-store remote, no cached bundle, nothing to compare. A "nothing was compared" outcome always exits `2`, never a soft `0` that would read as certified; the error names the two ways forward — run in CI where the base is restorable, or use the two-directory form); `3` when only new surfaces are present — surfaces captured only on the **head** side (a surface present only on the **base** side is a **removed** surface, a reviewable change: exit `1`) — (no baseline for _those_ surfaces to diff against — new surfaces against an existing baseline, or a base dir with no maps at all (and hence no manifest), meaning no baseline was ever captured: the first-adoption review path; approval policy decides whether to gate). A clean run prints `0 changed surfaces across N captured surface(s)`, and `--json` includes `compared`. The human output **groups the same way the report does**: surfaces that changed identically collapse into one finding (with the per-surface count on its header), longhands fold into shorthands, and size/position-derived longhands fold behind a `(+N derived longhands)` count — so one real change reads as one entry, not dozens of raw lines. A change that rode the shared frame every view draws (a persistent nav/header/footer) is promoted to a "🧱 Global chrome change" callout up top. `--json` stays the complete, unchanged machine contract — every surface and every raw longhand — regardless of the human grouping.
|
|
1017
1018
|
- `styleproof-report` — render the diff to a Markdown report with before/after crops. With no args, it reports cached maps for the current commit against the inferred base; `styleproof-report main` / `styleproof-report master` pins the base; `styleproof-report <beforeDir> <afterDir> --out <dir>` keeps the manual two-directory form. Add `--include-content` for the opt-in, advisory content section (see above).
|
|
1018
1019
|
- `styleproof-capture` — one-shot capture of any URL (no spec): `styleproof-capture <url> --key <name> --out <dir>`, with `--widths` (omit to auto-detect `@media` bands), `--wait <selector>`, `--ignore <selector>`, `--no-screenshots`, and the crawler flags (`--crawl`, `--setup <file>`, `--require-full-coverage` → exit 4 on residue, `--until-covered`, `--workers <n>`, `--no-data-states`) described in [Match a design pixel-for-pixel](#match-a-design-pixel-for-pixel).
|
|
1019
1020
|
- `styleproof-variants` — crawl a running app for one-step state variants and write `styleproof.variants.generated.json`. Pass `--base-url`, repeat `--route`, and use `--strict` when unresolved skipped/live candidates should fail automation.
|
|
@@ -190,6 +190,19 @@ function enqueueLinkedPages(links, sweep) {
|
|
|
190
190
|
}
|
|
191
191
|
}
|
|
192
192
|
|
|
193
|
+
// Crawl one page of the sweep. The entry page failing is a broken run (rethrow);
|
|
194
|
+
// a LINKED page failing (e.g. an off-origin redirect) returns null after warning,
|
|
195
|
+
// and the sweep continues.
|
|
196
|
+
async function crawlPage(browser, page, url, prefix, statesLeft) {
|
|
197
|
+
try {
|
|
198
|
+
return await crawlAndCapture(page, pageCrawlOptions(browser, url, prefix, statesLeft));
|
|
199
|
+
} catch (e) {
|
|
200
|
+
if (prefix === '') throw e;
|
|
201
|
+
console.log(`⚠ ${url}: ${e instanceof Error ? e.message : String(e)} — page skipped`);
|
|
202
|
+
return null;
|
|
203
|
+
}
|
|
204
|
+
}
|
|
205
|
+
|
|
193
206
|
async function runCrawl() {
|
|
194
207
|
const browser = await chromium.launch();
|
|
195
208
|
try {
|
|
@@ -209,7 +222,8 @@ async function runCrawl() {
|
|
|
209
222
|
|
|
210
223
|
while (sweep.queue.length > 0 && statesLeft > 0) {
|
|
211
224
|
const { url, prefix } = sweep.queue.shift();
|
|
212
|
-
const report = await
|
|
225
|
+
const report = await crawlPage(browser, page, url, prefix, statesLeft);
|
|
226
|
+
if (!report) continue; // linked page skipped loudly (e.g. off-origin redirect)
|
|
213
227
|
reports.push(report);
|
|
214
228
|
statesLeft -= report.surfaces.length;
|
|
215
229
|
if (opts.followLinks) enqueueLinkedPages(await harvestPageLinks(page, url), sweep);
|
package/bin/styleproof-diff.mjs
CHANGED
|
@@ -20,7 +20,8 @@
|
|
|
20
20
|
*
|
|
21
21
|
* Custom properties (--*) are ignored: they are inputs, not outcomes (see
|
|
22
22
|
* README). Exit code 0 = identical, 1 = reviewable differences, 2 = usage/capture
|
|
23
|
-
* error, 3 = only
|
|
23
|
+
* error, 3 = only NEW surfaces (present only on the head side, no baseline to diff
|
|
24
|
+
* against). A REMOVED surface (present only on the base side) is a change: exit 1.
|
|
24
25
|
*/
|
|
25
26
|
import fs from 'node:fs';
|
|
26
27
|
import path from 'node:path';
|
|
@@ -109,12 +110,15 @@ function printInventoryAudit(audit) {
|
|
|
109
110
|
);
|
|
110
111
|
}
|
|
111
112
|
for (const it of delta.added) console.log(` + added: ${it.key} ("${it.label}")`);
|
|
112
|
-
for (const k of staleAllowances)
|
|
113
|
+
for (const k of staleAllowances)
|
|
114
|
+
console.log(` ✗ stale allowRemoved (key is not actually removed): ${k} — prune it from styleproof.inventory.json`);
|
|
113
115
|
if (unexplained.length)
|
|
114
116
|
console.log(
|
|
115
117
|
` → ${unexplained.length} unacknowledged removal(s): restore the affordance, or record the decision in styleproof.inventory.json {"<key>":"<why>"}.`,
|
|
116
118
|
);
|
|
117
|
-
|
|
119
|
+
// A stale allowance BLOCKS like a stale residue acknowledgement: left in place it
|
|
120
|
+
// pre-acknowledges the NEXT removal of that key, so the ledger must not rot.
|
|
121
|
+
return unexplained.length + staleAllowances.length;
|
|
118
122
|
}
|
|
119
123
|
|
|
120
124
|
// ── data-residue guard (gate by default) ─────────────────────────────────────────
|
|
@@ -139,11 +143,11 @@ function loadAcknowledgedResidue() {
|
|
|
139
143
|
// Audit the HEAD bundle's residue against the ack ledger, carrying whether the head
|
|
140
144
|
// ledger armed the gate. Returns null when no captured map carried residue AND the
|
|
141
145
|
// gate wasn't armed — so a clean healthy run prints/gates nothing (byte-identical).
|
|
142
|
-
function readResidueAudit(dirB, armed) {
|
|
146
|
+
function readResidueAudit(dirB, armed, hasLedger) {
|
|
143
147
|
const headResidue = readResidue(dirB);
|
|
144
148
|
if (!armed && !headResidue.some((m) => m.dataResidue?.length)) return null;
|
|
145
149
|
const acknowledged = loadAcknowledgedResidue();
|
|
146
|
-
return { acknowledged, ...auditRunResidue(headResidue, acknowledged, armed) };
|
|
150
|
+
return { acknowledged, hasLedger, ...auditRunResidue(headResidue, acknowledged, armed) };
|
|
147
151
|
}
|
|
148
152
|
|
|
149
153
|
// Print the Data-residue section; return the count of UNACKNOWLEDGED failing endpoints
|
|
@@ -155,17 +159,20 @@ function residueLine(r, ackReason, armed) {
|
|
|
155
159
|
return ` ${armed ? '✗ ' : '⚠ '}${r.surface} · ${r.endpoint} (${r.reason})${armed ? ', unacknowledged' : ''}`;
|
|
156
160
|
}
|
|
157
161
|
|
|
158
|
-
/** The action footer: the gate (default) names the remedy; warn mode is the opt-out.
|
|
159
|
-
|
|
162
|
+
/** The action footer: the gate (default) names the remedy; warn mode is the opt-out.
|
|
163
|
+
* A bundle with NO ledger at all is named as such — not misattributed to the opt-out. */
|
|
164
|
+
function residueFooter(armed, unacknowledgedCount, hasLedger) {
|
|
160
165
|
if (!unacknowledgedCount) return null;
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
166
|
+
if (armed)
|
|
167
|
+
return ` → ${unacknowledgedCount} unacknowledged failing endpoint(s): fixture each (page.route / liveStates), acknowledge intentional ones in styleproof.data-residue.json {"<key>":"<why>"}, or opt down with \`dataResidue: "warn"\` in the capture spec.`;
|
|
168
|
+
if (!hasLedger)
|
|
169
|
+
return ' → recorded and warned — the head bundle carries no coverage ledger (ad-hoc or pre-3.10 capture), so the residue gate cannot arm. A spec-driven capture records the ledger and gates by default.';
|
|
170
|
+
return ' → recorded and warned (dataResidue: "warn" — the opt-out). Remove it to restore the default gate that BLOCKS on these.';
|
|
164
171
|
}
|
|
165
172
|
|
|
166
173
|
function printResidueAudit(audit) {
|
|
167
174
|
if (!audit) return 0;
|
|
168
|
-
const { residue, unacknowledged, staleAcknowledgements, armed } = audit;
|
|
175
|
+
const { residue, unacknowledged, staleAcknowledgements, armed, hasLedger } = audit;
|
|
169
176
|
if (!residue.length && !staleAcknowledgements.length) {
|
|
170
177
|
console.log('\n🩹 Data residue: no failing data-boundary request during capture');
|
|
171
178
|
return 0;
|
|
@@ -174,7 +181,7 @@ function printResidueAudit(audit) {
|
|
|
174
181
|
for (const r of residue) console.log(residueLine(r, audit.acknowledged[r.key], armed));
|
|
175
182
|
for (const k of staleAcknowledgements)
|
|
176
183
|
console.log(` ⚠ stale acknowledgement (endpoint no longer failing/present): ${k}`);
|
|
177
|
-
const footer = residueFooter(armed, unacknowledged.length);
|
|
184
|
+
const footer = residueFooter(armed, unacknowledged.length, hasLedger);
|
|
178
185
|
if (footer) console.log(footer);
|
|
179
186
|
// Only an ARMED gate blocks; warn-mode surfaces without gating. A stale acknowledgement
|
|
180
187
|
// always blocks when armed, so the ledger can't rot (mirrors the `exclude` guard).
|
|
@@ -204,14 +211,22 @@ function readLedger(dir) {
|
|
|
204
211
|
try {
|
|
205
212
|
return JSON.parse(fs.readFileSync(p, 'utf8'));
|
|
206
213
|
} catch {
|
|
207
|
-
|
|
214
|
+
// A ledger that EXISTS but cannot be parsed is tampering or truncation, and
|
|
215
|
+
// reading it as "no registry" would silently disarm the coverage,
|
|
216
|
+
// determinism, AND residue gates at once. Fail loud instead.
|
|
217
|
+
console.error(
|
|
218
|
+
`${COMMAND}: corrupt coverage ledger: ${p} — recapture the bundle; refusing to compare with disarmed gates.`,
|
|
219
|
+
);
|
|
220
|
+
process.exit(2);
|
|
208
221
|
}
|
|
209
222
|
}
|
|
210
223
|
|
|
211
224
|
// Print the completeness verdict; return true if it BLOCKS (a registered surface is missing).
|
|
212
225
|
function printCoverageVerdict(v) {
|
|
213
226
|
if (v.basis === 'complete') {
|
|
214
|
-
console.log(`\n✓ coverage complete — all ${v.registrySize} registered surface(s) captured`);
|
|
227
|
+
console.log(`\n✓ coverage complete — all ${v.registrySize} registered surface(s) captured or explicitly excluded`);
|
|
228
|
+
for (const k of v.staleExclusions ?? [])
|
|
229
|
+
console.log(` ⚠ stale exclude (not in the registry): ${k} — prune it from the spec`);
|
|
215
230
|
return false;
|
|
216
231
|
}
|
|
217
232
|
if (v.basis === 'unasserted') {
|
|
@@ -271,7 +286,8 @@ options:
|
|
|
271
286
|
-h, --help show this help
|
|
272
287
|
|
|
273
288
|
exit: 0 identical (certified), 1 differences found, 2 usage/capture error,
|
|
274
|
-
3 only
|
|
289
|
+
3 only NEW surfaces (present only on the head side, no baseline to diff
|
|
290
|
+
against); a REMOVED surface (present only on the base side) exits 1
|
|
275
291
|
`;
|
|
276
292
|
|
|
277
293
|
const argv = process.argv.slice(2);
|
|
@@ -359,7 +375,7 @@ try {
|
|
|
359
375
|
determinismVerdict = auditDeterminism(readLedger(dirA), headLedger);
|
|
360
376
|
// Data-residue: the head bundle's failing data endpoints, gated only if its ledger
|
|
361
377
|
// armed `dataResidue: 'gate'`. Same "read while the dirs exist" rule as the ledgers.
|
|
362
|
-
residueAudit = readResidueAudit(dirB, headLedger?.dataResidue === 'gate');
|
|
378
|
+
residueAudit = readResidueAudit(dirB, headLedger?.dataResidue === 'gate', headLedger != null);
|
|
363
379
|
// Element-path sets per surface, for the shared-chrome tier — same "read while
|
|
364
380
|
// the dirs exist" rule as the ledgers above.
|
|
365
381
|
surfacePaths = surfaceElementPaths(dirA, dirB);
|
|
@@ -369,7 +385,7 @@ try {
|
|
|
369
385
|
} finally {
|
|
370
386
|
cleanupCachedCaptureDirs(cacheCapture);
|
|
371
387
|
}
|
|
372
|
-
const { surfaces, counts, compared } = result;
|
|
388
|
+
const { surfaces, counts, compared, volatile, statesUncertified } = result;
|
|
373
389
|
|
|
374
390
|
// ── grouped human output ─────────────────────────────────────────────────────
|
|
375
391
|
// Reuse the report's dedup so one real change doesn't print once per surface with
|
|
@@ -410,11 +426,17 @@ function elementLines(findings) {
|
|
|
410
426
|
return lines;
|
|
411
427
|
}
|
|
412
428
|
|
|
413
|
-
//
|
|
429
|
+
// One-sided surfaces keep their own line. The two directions are NOT the same
|
|
430
|
+
// verdict: only-in-after is a NEW surface (no baseline, review before
|
|
431
|
+
// baselining); only-in-before is a REMOVED surface — a route or width that
|
|
432
|
+
// existed and is gone, which is a change, never an onboarding case.
|
|
414
433
|
for (const sd of surfaces) {
|
|
415
434
|
if (!sd.missing) continue;
|
|
416
|
-
|
|
417
|
-
|
|
435
|
+
console.log(
|
|
436
|
+
sd.missing === 'before'
|
|
437
|
+
? `\n${sd.surface}: new surface — captured only in the after set, no baseline to compare`
|
|
438
|
+
: `\n${sd.surface}: ✗ REMOVED surface — captured only in the before set; the head no longer renders it`,
|
|
439
|
+
);
|
|
418
440
|
}
|
|
419
441
|
|
|
420
442
|
// Group the changed surfaces the way the report does, so an identical change
|
|
@@ -472,6 +494,13 @@ if (jsonOut) {
|
|
|
472
494
|
counts,
|
|
473
495
|
surfaces,
|
|
474
496
|
compared,
|
|
497
|
+
// Subtrees excluded from every layer of the comparison because a side
|
|
498
|
+
// auto-detected them as volatile (still mutating at capture settle).
|
|
499
|
+
// Changes inside them are NOT certified by this diff.
|
|
500
|
+
volatileExcluded: volatile,
|
|
501
|
+
// Surfaces whose forced-state layer was skipped on BOTH sides — the
|
|
502
|
+
// :hover/:focus/:active layer compared {} vs {} and certifies nothing.
|
|
503
|
+
statesUncertified,
|
|
475
504
|
coverage: coverageVerdict,
|
|
476
505
|
determinism: determinismVerdict,
|
|
477
506
|
// The inventory verdict, machine-readable — parallel to coverage/determinism and
|
|
@@ -514,26 +543,48 @@ if (jsonOut) {
|
|
|
514
543
|
}
|
|
515
544
|
|
|
516
545
|
const total = counts.dom + counts.style + counts.state;
|
|
517
|
-
const newSurfaces = surfaces.filter((s) => s.missing).length;
|
|
546
|
+
const newSurfaces = surfaces.filter((s) => s.missing === 'before').length;
|
|
547
|
+
const removedSurfaces = surfaces.filter((s) => s.missing === 'after').length;
|
|
518
548
|
// One SurfaceDiff per distinct surface across both sides (incl. missing-on-one-side).
|
|
519
549
|
const surfaceCount = surfaces.length;
|
|
550
|
+
if (volatile > 0)
|
|
551
|
+
console.log(
|
|
552
|
+
`\n⚠ ${volatile} auto-detected volatile subtree(s) excluded from the comparison (still mutating at capture\n` +
|
|
553
|
+
' settle) — changes inside them are NOT certified. Fixture the region, or `ignore` it deliberately.',
|
|
554
|
+
);
|
|
555
|
+
if (statesUncertified > 0)
|
|
556
|
+
console.log(
|
|
557
|
+
`\n⚠ forced-state layer uncertified on ${statesUncertified} surface(s): BOTH captures skipped it, so\n` +
|
|
558
|
+
' :hover/:focus/:active differences there were never compared.',
|
|
559
|
+
);
|
|
520
560
|
const newNote = newSurfaces ? ` (+${newSurfaces} new surface(s) with no baseline)` : '';
|
|
521
|
-
const
|
|
561
|
+
const removedNote = removedSurfaces ? ` + ${removedSurfaces} REMOVED surface(s)` : '';
|
|
562
|
+
const invNote = invRemovals ? ` + ${invRemovals} inventory gate failure(s) (unacknowledged or stale)` : '';
|
|
522
563
|
// residueFails counts unacknowledged failing endpoints AND stale acknowledgements (both gate).
|
|
523
564
|
const resNote = residueFails ? ` + ${residueFails} data-residue gate failure(s) (unacknowledged or stale)` : '';
|
|
524
565
|
const covNote = coverageFails ? ` + ${coverageVerdict.uncovered.length} uncaptured registered surface(s)` : '';
|
|
525
566
|
const detNote = determinismFails ? ' + determinism unproven' : '';
|
|
526
|
-
const clean =
|
|
567
|
+
const clean =
|
|
568
|
+
total === 0 &&
|
|
569
|
+
removedSurfaces === 0 &&
|
|
570
|
+
invRemovals === 0 &&
|
|
571
|
+
residueFails === 0 &&
|
|
572
|
+
!coverageFails &&
|
|
573
|
+
!determinismFails;
|
|
527
574
|
console.log(
|
|
528
575
|
clean
|
|
529
576
|
? newSurfaces === 0
|
|
530
577
|
? `\n✓ 0 changed surfaces across ${compared} captured surface(s): every computed style, pseudo-element, and hover/focus/active state matches`
|
|
531
578
|
: `\nℹ ${newSurfaces} new surface(s) captured with no baseline to compare — review before baselining`
|
|
532
|
-
: `\n✗ ${counts.dom} DOM change(s), ${counts.style} computed-style difference(s), ${counts.state} state-delta difference(s) across ${surfaceCount} surfaces${newNote}${invNote}${resNote}${covNote}${detNote}`,
|
|
579
|
+
: `\n✗ ${counts.dom} DOM change(s), ${counts.style} computed-style difference(s), ${counts.state} state-delta difference(s) across ${surfaceCount} surfaces${newNote}${removedNote}${invNote}${resNote}${covNote}${detNote}`,
|
|
533
580
|
);
|
|
534
|
-
// 0 = identical, 1 = reviewable differences (incl.
|
|
535
|
-
// unacknowledged
|
|
536
|
-
//
|
|
581
|
+
// 0 = identical, 1 = reviewable differences (incl. a REMOVED surface, inventory/residue gate
|
|
582
|
+
// failures — unacknowledged or stale — an incomplete coverage registry, or an
|
|
583
|
+
// unproven-determinism capture), 3 = ONLY new surfaces (no baseline). 2 = usage.
|
|
537
584
|
process.exit(
|
|
538
|
-
total > 0 ||
|
|
585
|
+
total > 0 || removedSurfaces > 0 || invRemovals > 0 || residueFails > 0 || coverageFails || determinismFails
|
|
586
|
+
? 1
|
|
587
|
+
: newSurfaces > 0
|
|
588
|
+
? 3
|
|
589
|
+
: 0,
|
|
539
590
|
);
|
package/bin/styleproof-map.mjs
CHANGED
|
@@ -253,7 +253,9 @@ function runVariantCrawl(env) {
|
|
|
253
253
|
if (status !== 0) process.exit(status);
|
|
254
254
|
}
|
|
255
255
|
|
|
256
|
-
|
|
256
|
+
// An ABSOLUTE STYLEMAP_DIR/--dir is respected as-is; a relative one nests under
|
|
257
|
+
// baseDir (.styleproof/maps by default) — mirrors the runner's resolveOutputDir.
|
|
258
|
+
const targetDir = path.isAbsolute(dir) ? dir : path.join(baseDir, dir);
|
|
257
259
|
// Sample the tree state the capture is ABOUT to render, so the manifest can bind the
|
|
258
260
|
// map to it. A capture runs for minutes; if the source is edited or HEAD moves in that
|
|
259
261
|
// window, the map renders one state but would otherwise be stamped clean@post-HEAD and
|
package/dist/crawl-surfaces.js
CHANGED
|
@@ -408,6 +408,14 @@ async function scrollReveal(page) {
|
|
|
408
408
|
async function gotoFresh(page, opts) {
|
|
409
409
|
await page.setViewportSize({ width: opts.widths[0] ?? 1280, height: opts.height });
|
|
410
410
|
await page.goto(opts.url, { waitUntil: 'load' });
|
|
411
|
+
// A same-origin URL can still 302 off-origin (SSO, /out?url=…). External
|
|
412
|
+
// content is nondeterministic and never belongs in a map — abort the crawl of
|
|
413
|
+
// this page rather than capture a third-party render as if it were the app.
|
|
414
|
+
const wanted = URL.canParse(opts.url) ? new URL(opts.url).origin : null; // relative url → origin unknowable
|
|
415
|
+
const landed = new URL(page.url()).origin;
|
|
416
|
+
if (wanted && landed !== wanted) {
|
|
417
|
+
throw new Error(`styleproof crawl: ${opts.url} redirected off-origin to ${landed} — external content is never captured`);
|
|
418
|
+
}
|
|
411
419
|
// Tolerant wait: the generic settle below is the real readiness signal; an
|
|
412
420
|
// optional waitSelector just accelerates it and must not fail the crawl.
|
|
413
421
|
const ready = opts.waitSelector ? page.locator(opts.waitSelector).first() : null;
|
package/dist/diff.d.ts
CHANGED
|
@@ -87,6 +87,7 @@ export declare function diffStyleMapDirs(dirA: string, dirB: string): {
|
|
|
87
87
|
surfaces: SurfaceDiff[];
|
|
88
88
|
counts: DiffCounts;
|
|
89
89
|
volatile: number;
|
|
90
|
+
statesUncertified: number;
|
|
90
91
|
compared: number;
|
|
91
92
|
};
|
|
92
93
|
export declare function diffContentMaps(a: StyleMap, b: StyleMap): ContentChange[];
|
package/dist/diff.js
CHANGED
|
@@ -281,7 +281,7 @@ export function diffStyleMapDirs(dirA, dirB) {
|
|
|
281
281
|
throw new MissingHeadMapError();
|
|
282
282
|
const surfaces = [];
|
|
283
283
|
const counts = { dom: 0, style: 0, state: 0 };
|
|
284
|
-
|
|
284
|
+
const uncompared = { volatile: 0, statesUncertified: 0 };
|
|
285
285
|
for (const surface of names) {
|
|
286
286
|
if (!indexA[surface] || !indexB[surface]) {
|
|
287
287
|
// A surface present on only one side has no baseline to diff against — it's
|
|
@@ -291,15 +291,22 @@ export function diffStyleMapDirs(dirA, dirB) {
|
|
|
291
291
|
surfaces.push({ surface, missing: indexA[surface] ? 'after' : 'before', findings: [] });
|
|
292
292
|
continue;
|
|
293
293
|
}
|
|
294
|
-
const
|
|
295
|
-
const mapB = loadStyleMap(indexB[surface]);
|
|
296
|
-
volatile += new Set([...(mapA.volatile ?? []), ...(mapB.volatile ?? [])]).size;
|
|
297
|
-
const findings = diffStyleMaps(mapA, mapB);
|
|
294
|
+
const findings = diffSurfacePair(indexA[surface], indexB[surface], uncompared);
|
|
298
295
|
tallyCounts(findings, counts);
|
|
299
296
|
if (findings.length)
|
|
300
297
|
surfaces.push({ surface, findings });
|
|
301
298
|
}
|
|
302
|
-
return { surfaces, counts,
|
|
299
|
+
return { surfaces, counts, ...uncompared, compared: names.length };
|
|
300
|
+
}
|
|
301
|
+
/** Diff one paired surface, tallying what was NOT compared (volatile subtrees;
|
|
302
|
+
* a forced-state layer skipped on BOTH sides — {} vs {} certifies nothing). */
|
|
303
|
+
function diffSurfacePair(fileA, fileB, uncompared) {
|
|
304
|
+
const mapA = loadStyleMap(fileA);
|
|
305
|
+
const mapB = loadStyleMap(fileB);
|
|
306
|
+
uncompared.volatile += new Set([...(mapA.volatile ?? []), ...(mapB.volatile ?? [])]).size;
|
|
307
|
+
if (mapA.statesSkipped && mapB.statesSkipped)
|
|
308
|
+
uncompared.statesUncertified++;
|
|
309
|
+
return diffStyleMaps(mapA, mapB);
|
|
303
310
|
}
|
|
304
311
|
/**
|
|
305
312
|
* Diff the OPT-IN content layer: elements whose own rendered text changed.
|
package/dist/report.js
CHANGED
|
@@ -1000,7 +1000,7 @@ function reportHeadline(args) {
|
|
|
1000
1000
|
const candidates = liveCandidateLabels.length
|
|
1001
1001
|
? ` Auto-detected live-state candidate(s): ${liveCandidateLabels.slice(0, 5).join('; ')}.`
|
|
1002
1002
|
: '';
|
|
1003
|
-
md.push('', `_${volatileCount} live region(s) auto-excluded as nondeterministic (a stream, ticker, or late-loading content) —
|
|
1003
|
+
md.push('', `_${volatileCount} live region(s) auto-excluded as nondeterministic (a stream, ticker, or late-loading content) — changes inside them are NOT certified by this check.${candidates}_`);
|
|
1004
1004
|
}
|
|
1005
1005
|
if (contentCount > 0 && (changeGroups.length > 0 || missing.length > 0)) {
|
|
1006
1006
|
md.push('', `📝 _${contentCount} advisory content change(s) below — they don't affect the check._`);
|
package/dist/runner.d.ts
CHANGED
|
@@ -150,6 +150,14 @@ export type DefineOptions = {
|
|
|
150
150
|
* `selfCheck` explicitly to override.
|
|
151
151
|
*/
|
|
152
152
|
selfCheck?: boolean;
|
|
153
|
+
/**
|
|
154
|
+
* Run the generated capture tests in PARALLEL across Playwright workers
|
|
155
|
+
* (default true). Every capture test is independent, so parallel is safe and
|
|
156
|
+
* ~workers× faster on a multi-surface spec — even when the project config
|
|
157
|
+
* pins `fullyParallel: false`. Set false ONLY for a spec file whose OTHER
|
|
158
|
+
* tests read the captured maps in file order (an in-file assertion suite).
|
|
159
|
+
*/
|
|
160
|
+
parallel?: boolean;
|
|
153
161
|
/**
|
|
154
162
|
* Opt-in content layer (default OFF). Record each element's own rendered text
|
|
155
163
|
* so the report's optional content section can surface copy changes (run
|
|
@@ -239,6 +247,10 @@ export declare function passLiveStreams(page: Page, url: string): Promise<void>;
|
|
|
239
247
|
* forces it on either way.
|
|
240
248
|
*/
|
|
241
249
|
export declare function defaultSelfCheck(replayFrom: string | undefined, env?: string | undefined): boolean;
|
|
250
|
+
/** Resolve a capture output dir: an ABSOLUTE `dir` is respected as-is (a user's
|
|
251
|
+
* `STYLEMAP_DIR=/abs/path` must not be buried under `baseDir`); a relative one
|
|
252
|
+
* nests under `baseDir` as before. */
|
|
253
|
+
export declare function resolveOutputDir(baseDir: string, dir: string): string;
|
|
242
254
|
/**
|
|
243
255
|
* Output base dir: explicit `baseDir` wins, then `STYLEPROOF_BASEDIR`, then the
|
|
244
256
|
* default. Lets CLIs and CI redirect capture into cache/fallback dirs without
|
package/dist/runner.js
CHANGED
|
@@ -304,7 +304,7 @@ async function pinInputs(page, harName, s) {
|
|
|
304
304
|
}
|
|
305
305
|
}
|
|
306
306
|
else {
|
|
307
|
-
await page.routeFromHAR(path.join(s.baseDir, s.dir, harName), {
|
|
307
|
+
await page.routeFromHAR(path.join(resolveOutputDir(s.baseDir, s.dir), harName), {
|
|
308
308
|
url: s.replayUrl,
|
|
309
309
|
update: true,
|
|
310
310
|
updateContent: 'embed', // single portable file, no sidecar resources
|
|
@@ -443,7 +443,7 @@ async function capturePopupCandidate(page, surface, width, height, s, options, c
|
|
|
443
443
|
return;
|
|
444
444
|
}
|
|
445
445
|
}
|
|
446
|
-
const stem = path.join(s.baseDir, s.dir, `${surface.key}-${popupId}@${width}`);
|
|
446
|
+
const stem = path.join(resolveOutputDir(s.baseDir, s.dir), `${surface.key}-${popupId}@${width}`);
|
|
447
447
|
saveStyleMap(`${stem}.json.gz`, map);
|
|
448
448
|
if (s.screenshots)
|
|
449
449
|
await page.screenshot({ path: `${stem}.png`, fullPage: true, animations: 'disabled' });
|
|
@@ -495,7 +495,7 @@ async function captureSurface(page, surface, width, s) {
|
|
|
495
495
|
// Attach data-residue AFTER the self-check re-run so both runs' failures are folded
|
|
496
496
|
// (deduped in the watcher). Warn always; the recorded residue is what the gate reads.
|
|
497
497
|
attachDataResidue(map, residue.residue());
|
|
498
|
-
const stem = path.join(s.baseDir, s.dir, `${surface.key}@${width}`);
|
|
498
|
+
const stem = path.join(resolveOutputDir(s.baseDir, s.dir), `${surface.key}@${width}`);
|
|
499
499
|
saveStyleMap(`${stem}.json.gz`, map);
|
|
500
500
|
if (s.screenshots) {
|
|
501
501
|
// captureStyleMap froze animations/transitions, so this is the same settled
|
|
@@ -533,6 +533,12 @@ function attachDataResidue(map, residue) {
|
|
|
533
533
|
export function defaultSelfCheck(replayFrom, env = process.env.STYLEPROOF_SELFCHECK) {
|
|
534
534
|
return env === '1' || !replayFrom;
|
|
535
535
|
}
|
|
536
|
+
/** Resolve a capture output dir: an ABSOLUTE `dir` is respected as-is (a user's
|
|
537
|
+
* `STYLEMAP_DIR=/abs/path` must not be buried under `baseDir`); a relative one
|
|
538
|
+
* nests under `baseDir` as before. */
|
|
539
|
+
export function resolveOutputDir(baseDir, dir) {
|
|
540
|
+
return path.isAbsolute(dir) ? dir : path.join(baseDir, dir);
|
|
541
|
+
}
|
|
536
542
|
/**
|
|
537
543
|
* Output base dir: explicit `baseDir` wins, then `STYLEPROOF_BASEDIR`, then the
|
|
538
544
|
* default. Lets CLIs and CI redirect capture into cache/fallback dirs without
|
|
@@ -602,7 +608,7 @@ function resolveSettings(c) {
|
|
|
602
608
|
*/
|
|
603
609
|
function writeCoverageLedgerTest(settings, dir, expected, exclude, captureSurfaces) {
|
|
604
610
|
test('styleproof coverage ledger', () => {
|
|
605
|
-
const outDir =
|
|
611
|
+
const outDir = resolveOutputDir(settings.baseDir, dir);
|
|
606
612
|
fs.mkdirSync(outDir, { recursive: true });
|
|
607
613
|
// Determinism basis: self-check ON proves it (a drift would have failed the capture);
|
|
608
614
|
// else a replay run is deterministic by construction; else it's unproven.
|
|
@@ -648,7 +654,7 @@ function writeCoverageLedgerTest(settings, dir, expected, exclude, captureSurfac
|
|
|
648
654
|
function writeBrowserBuildTest(settings, dir) {
|
|
649
655
|
test('styleproof browser build', ({ page }) => {
|
|
650
656
|
const version = page.context().browser()?.version();
|
|
651
|
-
const outDir =
|
|
657
|
+
const outDir = resolveOutputDir(settings.baseDir, dir);
|
|
652
658
|
writeBrowserBuildSidecar(outDir, version);
|
|
653
659
|
writeCaptureManifest({ dir: outDir, screenshots: settings.screenshots });
|
|
654
660
|
});
|
|
@@ -681,6 +687,16 @@ export function defineStyleMapCapture(options) {
|
|
|
681
687
|
return;
|
|
682
688
|
const settings = resolveSettings(options);
|
|
683
689
|
test.describe('styleproof capture', () => {
|
|
690
|
+
// Every generated test is independent — its own page, its own map/HAR files,
|
|
691
|
+
// its own self-check; the ledger/manifest tests mkdir and tolerate any order.
|
|
692
|
+
// Declare the block PARALLEL so captures fan out across the consumer's
|
|
693
|
+
// Playwright workers even when the project pins `fullyParallel: false` for
|
|
694
|
+
// its behaviour suite (150 serial surface×width captures is a ~25-minute CI
|
|
695
|
+
// step; 4 workers make it ~4x faster with byte-identical maps).
|
|
696
|
+
// `parallel: false` keeps file order for specs whose own sibling tests read
|
|
697
|
+
// the captured maps.
|
|
698
|
+
if (options.parallel !== false)
|
|
699
|
+
test.describe.configure({ mode: 'parallel' });
|
|
684
700
|
writeCoverageLedgerTest(settings, dir, expected ?? null, exclude, captureSurfaces);
|
|
685
701
|
writeBrowserBuildTest(settings, dir);
|
|
686
702
|
for (const surface of captureSurfaces) {
|
|
@@ -815,10 +831,18 @@ export function defineCrawlCapture(options) {
|
|
|
815
831
|
: null;
|
|
816
832
|
if (gap)
|
|
817
833
|
throw new Error(gap);
|
|
834
|
+
// The nav's hrefs are same-origin by selection, but a link can still 302
|
|
835
|
+
// off-origin (SSO, /out?url=…). External content is nondeterministic and
|
|
836
|
+
// never belongs in a map, so the landing origin is verified per surface.
|
|
837
|
+
const entryOrigin = new URL(page.url()).origin;
|
|
818
838
|
const captureSurfaces = links.flatMap((link) => expandSurfaceVariants({
|
|
819
839
|
key: link.key,
|
|
820
840
|
go: async (p) => {
|
|
821
841
|
await p.goto(link.url, { waitUntil: 'load' });
|
|
842
|
+
const landed = new URL(p.url()).origin;
|
|
843
|
+
if (landed !== entryOrigin) {
|
|
844
|
+
throw new Error(`styleproof crawl: ${link.url} redirected off-origin to ${landed} — external content is never captured`);
|
|
845
|
+
}
|
|
822
846
|
if (settle)
|
|
823
847
|
await settle(p);
|
|
824
848
|
},
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "styleproof",
|
|
3
|
-
"version": "4.
|
|
3
|
+
"version": "4.4.0",
|
|
4
4
|
"description": "Catch every CSS change before it ships — review PRs and certify refactors by the browser's computed styles, not pixels. Works with any styling system.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"playwright",
|