@decocms/parity 0.35.8 → 0.35.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +18 -19
- package/CHANGELOG.md +36 -0
- package/dist/cli.js +23178 -23021
- package/package.json +1 -1
package/AGENTS.md
CHANGED
|
@@ -50,9 +50,11 @@ Do **not** fake single-site with `parity run --prod X --cand X` — it runs ever
|
|
|
50
50
|
|
|
51
51
|
```ts
|
|
52
52
|
{
|
|
53
|
-
verdict: { status: "pass"|"warn"|"fail", score: 0..100, scoreVersion:
|
|
54
|
-
// score
|
|
55
|
-
//
|
|
53
|
+
verdict: { status: "pass"|"warn"|"fail", score: 0..100, scoreVersion: 3, pagesAnalyzed, critical, high, medium, low },
|
|
54
|
+
// score = 100·e^(-penaltyDensity/35), penaltyDensity = Σ severity weights ÷ pagesAnalyzed.
|
|
55
|
+
// v3: capped at 79 only by a live critical finding or a check that failed with no finding (it threw).
|
|
56
|
+
// A FAIL from high findings keeps status "fail" but the score stays the formula — "91 · FAIL" is valid;
|
|
57
|
+
// read status for the gate, score for progress. It's a progress meter: it RISES as issues get fixed.
|
|
56
58
|
previousRun?: { id, timestamp, score, scoreDelta }, // trend vs last comparable run (same hosts + scoreVersion)
|
|
57
59
|
topIssues: Issue[], // LLM-ranked, deduplicated; usually 5-10 items
|
|
58
60
|
issues: Issue[], // all raw issues from every check
|
|
@@ -60,9 +62,9 @@ Do **not** fake single-site with `parity run --prod X --cand X` — it runs ever
|
|
|
60
62
|
visualDiff: {
|
|
61
63
|
pagesChecked, pagesWithDiffs, pagesPassed, pagesFailed,
|
|
62
64
|
parityOk: boolean, // ⭐ binary signal — see below
|
|
63
|
-
pagesFromCache: number, //
|
|
64
|
-
llmCallsUsed: number,
|
|
65
|
-
results: VisualDiffPage[] // per page: prod/cand
|
|
65
|
+
pagesFromCache: number, // always 0 now (legacy: LLM verdict cache)
|
|
66
|
+
llmCallsUsed: number, // always 0 now (legacy: LLM vision calls)
|
|
67
|
+
results: VisualDiffPage[] // per page: prod/cand/diff images, sections, findings[] (element + property + px), reason
|
|
66
68
|
},
|
|
67
69
|
flowCaptures: FlowCapture[] // per-flow page captures with vitals + network
|
|
68
70
|
}
|
|
@@ -91,7 +93,9 @@ if (run.verdict.critical === 0 && run.verdict.high === 0) {
|
|
|
91
93
|
}
|
|
92
94
|
```
|
|
93
95
|
|
|
94
|
-
`visualDiff.parityOk` is `true` iff every page in `results` has `verdict === "pass"
|
|
96
|
+
`visualDiff.parityOk` is `true` iff at least one page was compared and every page in `results` has `verdict === "pass"` (pixel-identical after masking prod's own motion). `"failed"` means the page was **not compared** (different heights, fonts loading, too much motion) — never treat it as a pass. Trust this flag for "is parity OK?" automation.
|
|
97
|
+
|
|
98
|
+
The visual module runs on the `parity heatmap` engine — deterministic, no LLM. Each `findings[]` entry names the element, the property and both values (`rect.offsetY: 572px → 590px`), so fix the property it names rather than eyeballing screenshots; re-verify one page with `parity heatmap --prod <url> --cand <url> --viewports <vp>`.
|
|
95
99
|
|
|
96
100
|
## Driving fixes from the output
|
|
97
101
|
|
|
@@ -99,7 +103,7 @@ The most useful flow for an AI agent is:
|
|
|
99
103
|
|
|
100
104
|
1. Run `parity run --preset full` → wait → read `report.json`
|
|
101
105
|
2. For each `topIssue` with severity ≥ high, open the file(s) referenced in `suggestedFix`
|
|
102
|
-
3. For visual diffs: read `visualDiff.results[].sectionsOnlyInProd` —
|
|
106
|
+
3. For visual diffs: read `visualDiff.results[].findings` (element + property + px) and `.sectionsOnlyInProd` — the latter are Deco sections present in prod's DOM but missing in cand. Most likely cause: not registered in `src/setup.ts → registerSections()`.
|
|
103
107
|
4. For console errors: read `issues[].evidence[]` for HAR paths; the failed request is usually the root cause.
|
|
104
108
|
5. Re-run `parity journey` (cheaper) after each fix to confirm.
|
|
105
109
|
|
|
@@ -110,20 +114,19 @@ The most useful flow for an AI agent is:
|
|
|
110
114
|
If neither `ANTHROPIC_API_KEY` nor `OPENROUTER_API_KEY` is set:
|
|
111
115
|
|
|
112
116
|
- `topIssues` will fall back to deterministic severity-sorted merge (still useful)
|
|
113
|
-
-
|
|
114
|
-
- `sectionsOnlyInProd` is still populated (uses DOM, not vision)
|
|
117
|
+
- the visual module is unaffected — it never used an LLM (heatmap engine)
|
|
115
118
|
- Selector discovery is skipped; defaults from `.parityrc.json` or platform heuristics kick in
|
|
116
119
|
|
|
117
120
|
The CLI never fails just because the LLM is unavailable. Issues will still be reported, just less polished.
|
|
118
121
|
|
|
119
122
|
## Cost-conscious usage
|
|
120
123
|
|
|
121
|
-
|
|
124
|
+
The visual module costs no LLM calls. In tight loops:
|
|
122
125
|
|
|
123
126
|
- Use `--preset smoke` first (no LLM, ~30s) to validate the URLs respond
|
|
124
127
|
- Use `--no-visual-diff` if you only care about functional checks
|
|
125
128
|
- Use `parity journey` (no aggregation step) for the tightest CI loop
|
|
126
|
-
-
|
|
129
|
+
- Use `parity heatmap` on the one page you just fixed instead of a whole run
|
|
127
130
|
|
|
128
131
|
## Picking which pages to verify
|
|
129
132
|
|
|
@@ -143,15 +146,10 @@ These flags override sitemap sampling entirely. Every listed path gets captured
|
|
|
143
146
|
|
|
144
147
|
## Cache flags
|
|
145
148
|
|
|
146
|
-
-
|
|
147
|
-
- `--no-cache` — ignore existing cache entries; force fresh LLM judgment on every page.
|
|
148
|
-
- `--clear-cache` — wipe the cache file before the run starts (use after a prompt change or when verdicts feel stale).
|
|
149
|
-
|
|
150
|
-
The cache key is `sha256(prod_screenshot_bytes + cand_screenshot_bytes + LLM_PROMPT_VERSION)`. Any byte change in either screenshot invalidates the entry naturally. The `LLM_PROMPT_VERSION` constant (in `src/llm/visual-semantic-diff.ts`) bumps the entire cache atomically when the prompt or tool schema changes — no need to wipe by hand.
|
|
149
|
+
`--no-cache` / `--clear-cache` are accepted and do nothing: they controlled the LLM visual verdict cache, and the heatmap verdict is deterministic, so there is nothing left to cache.
|
|
151
150
|
|
|
152
151
|
Hard caps already in place:
|
|
153
152
|
|
|
154
|
-
- 12 max Claude Vision calls per `run` (visual diff)
|
|
155
153
|
- 8 max selector recovery calls per run
|
|
156
154
|
- 120s timeout per Vision call, 60s per text call
|
|
157
155
|
|
|
@@ -160,7 +158,8 @@ Hard caps already in place:
|
|
|
160
158
|
- **First run is slower** — Playwright downloads Chromium (~150MB). After that, ~30s for smoke, ~5min for full.
|
|
161
159
|
- **Live prod × cand side by side lives in `parity cockpit --prod <url> --cand <url>`**, not the HTML report; its proxy strips X-Frame-Options so any site renders.
|
|
162
160
|
- **Cookies are isolated per run** — no shared session between prod and cand. If a flow needs auth, you'll need to handle that in `.parityrc.json` or extend the flow.
|
|
163
|
-
- **Visual diff catches missing sections**
|
|
161
|
+
- **Visual diff catches missing sections** via DOM (`data-section`); if the migrated site doesn't emit that attribute, the heatmap still shows the region but the section name is lost.
|
|
162
|
+
- **A page whose height differs between prod and cand is `failed` (not compared)** — the pixel diff would only cover the intersection. Fix the height first (the first box-tree delta usually names why), or compare the section with `parity heatmap --selector`.
|
|
164
163
|
- **`parity run` doesn't apply schema changes from old runs** — if you change `src/types/schema.ts`, old `report.json` files won't parse. Delete `parity-output/` or re-run.
|
|
165
164
|
|
|
166
165
|
## Files agents should and should not write
|
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,42 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/).
|
|
7
7
|
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
### Changed
|
|
11
|
+
|
|
12
|
+
* **`parity run`'s visual module runs on the `parity heatmap` engine — no LLM.**
|
|
13
|
+
The old pass sent screenshot pairs to a vision model and filed what it said.
|
|
14
|
+
On real runs it invented findings — a stat card on neither side, prod and cand
|
|
15
|
+
swapped, a "section empty" that was a lazy load mid-flight — and a different set
|
|
16
|
+
each run, so the score was not reproducible. Now every visual issue is a
|
|
17
|
+
measured region: zero-tolerance pixel diff with prod's own motion masked out,
|
|
18
|
+
named by the box-tree delta under it (`[offset] button "Grab Yours Now!" sits
|
|
19
|
+
18px further down than prod (rect.offsetY: 572px → 590px)`). Same pages in,
|
|
20
|
+
same findings out. Severity per region: `horizontal-overflow` high,
|
|
21
|
+
`image-source` low, otherwise high ≥ 50,000 differing px, medium ≥ 2,500,
|
|
22
|
+
low below; a uniform horizontal shift of several items of one list (a
|
|
23
|
+
carousel track at a different position) is low. A page that cannot be compared (different heights, fonts loading,
|
|
24
|
+
too much motion) is `failed` → an inconclusive issue, never a pass; a pass
|
|
25
|
+
that compared nothing is flagged as such, and `parityOk` needs ≥ 1 page.
|
|
26
|
+
* **Visual Diff tab** shows prod · cand · diff and the element/property/px list,
|
|
27
|
+
with a `parity heatmap` command per page. Its visual prompt is the heatmap's
|
|
28
|
+
findings in `parity heatmap`'s report shape. Visual Diff and Prompt LLM tabs
|
|
29
|
+
are no longer hidden without an LLM.
|
|
30
|
+
* `report.json` is additive: `visualDiff.results[]` gains `engine`, `findings`,
|
|
31
|
+
`diffPixels`, `noiseFloorPixels`, `maskedCoverage`, `reason`; old reports with
|
|
32
|
+
LLM `differences` still render.
|
|
33
|
+
* The visual pass captures at `deviceScaleFactor: 1` (what `parity heatmap`
|
|
34
|
+
compares) and no longer turns itself off when no LLM provider is configured.
|
|
35
|
+
* `parity heatmap` attributes the cand-side region of a moved element (where cand
|
|
36
|
+
put it) instead of reporting it `unattributed`.
|
|
37
|
+
|
|
38
|
+
### Removed
|
|
39
|
+
|
|
40
|
+
* The vision-model visual diff (`src/llm/visual-semantic-diff.ts`) and its
|
|
41
|
+
cross-run verdict cache. `--no-cache` / `--clear-cache` are kept as no-ops so
|
|
42
|
+
existing scripts still parse.
|
|
43
|
+
|
|
8
44
|
## [0.28.1] — 2026-08-24
|
|
9
45
|
|
|
10
46
|
### Fixed
|