@decocms/parity 0.35.8 → 0.35.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/AGENTS.md +18 -19
  2. package/CHANGELOG.md +36 -0
  3. package/dist/cli.js +23178 -23021
  4. package/package.json +1 -1
package/AGENTS.md CHANGED
@@ -50,9 +50,11 @@ Do **not** fake single-site with `parity run --prod X --cand X` — it runs ever
50
50
 
51
51
  ```ts
52
52
  {
53
- verdict: { status: "pass"|"warn"|"fail", score: 0..100, scoreVersion: 2, pagesAnalyzed, critical, high, medium, low },
54
- // score v2 = 100·e^(-penaltyDensity/35), penaltyDensity = Σ severity weights ÷ pagesAnalyzed.
55
- // Any FAIL verdict (critical issue or failed check) caps it at 79. It's a progress meter: it RISES as issues get fixed.
53
+ verdict: { status: "pass"|"warn"|"fail", score: 0..100, scoreVersion: 3, pagesAnalyzed, critical, high, medium, low },
54
+ // score = 100·e^(-penaltyDensity/35), penaltyDensity = Σ severity weights ÷ pagesAnalyzed.
55
+ // v3: capped at 79 only by a live critical finding or a check that failed with no finding (it threw).
56
+ // A FAIL from high findings keeps status "fail" but the score stays the formula — "91 · FAIL" is valid;
57
+ // read status for the gate, score for progress. It's a progress meter: it RISES as issues get fixed.
56
58
  previousRun?: { id, timestamp, score, scoreDelta }, // trend vs last comparable run (same hosts + scoreVersion)
57
59
  topIssues: Issue[], // LLM-ranked, deduplicated; usually 5-10 items
58
60
  issues: Issue[], // all raw issues from every check
@@ -60,9 +62,9 @@ Do **not** fake single-site with `parity run --prod X --cand X` — it runs ever
60
62
  visualDiff: {
61
63
  pagesChecked, pagesWithDiffs, pagesPassed, pagesFailed,
62
64
  parityOk: boolean, // ⭐ binary signal — see below
63
- pagesFromCache: number, // how many verdicts came from cross-run cache
64
- llmCallsUsed: number,
65
- results: VisualDiffPage[] // per page: prod/cand screenshots, sections, LLM diffs, cachedAt?
65
+ pagesFromCache: number, // always 0 now (legacy: LLM verdict cache)
66
+ llmCallsUsed: number, // always 0 now (legacy: LLM vision calls)
67
+ results: VisualDiffPage[] // per page: prod/cand/diff images, sections, findings[] (element + property + px), reason
66
68
  },
67
69
  flowCaptures: FlowCapture[] // per-flow page captures with vitals + network
68
70
  }
@@ -91,7 +93,9 @@ if (run.verdict.critical === 0 && run.verdict.high === 0) {
91
93
  }
92
94
  ```
93
95
 
94
- `visualDiff.parityOk` is `true` iff every page in `results` has `verdict === "pass"`. The check itself accounts for the budget-cap edge case: pages where the LLM was skipped AND pctDiff was meaningfully high are surfaced as `"diffs"` rather than silently falling through to `"pass"`. Trust this flag for "is parity OK?" automation.
96
+ `visualDiff.parityOk` is `true` iff at least one page was compared and every page in `results` has `verdict === "pass"` (pixel-identical after masking prod's own motion). `"failed"` means the page was **not compared** (different heights, fonts loading, too much motion) — never treat it as a pass. Trust this flag for "is parity OK?" automation.
97
+
98
+ The visual module runs on the `parity heatmap` engine — deterministic, no LLM. Each `findings[]` entry names the element, the property and both values (`rect.offsetY: 572px → 590px`), so fix the property it names rather than eyeballing screenshots; re-verify one page with `parity heatmap --prod <url> --cand <url> --viewports <vp>`.
95
99
 
96
100
  ## Driving fixes from the output
97
101
 
@@ -99,7 +103,7 @@ The most useful flow for an AI agent is:
99
103
 
100
104
  1. Run `parity run --preset full` → wait → read `report.json`
101
105
  2. For each `topIssue` with severity ≥ high, open the file(s) referenced in `suggestedFix`
102
- 3. For visual diffs: read `visualDiff.results[].sectionsOnlyInProd` — these are Deco sections present in prod's DOM but missing in cand. Most likely cause: not registered in `src/setup.ts → registerSections()`.
106
+ 3. For visual diffs: read `visualDiff.results[].findings` (element + property + px) and `.sectionsOnlyInProd` — the latter are Deco sections present in prod's DOM but missing in cand. Most likely cause: not registered in `src/setup.ts → registerSections()`.
103
107
  4. For console errors: read `issues[].evidence[]` for HAR paths; the failed request is usually the root cause.
104
108
  5. Re-run `parity journey` (cheaper) after each fix to confirm.
105
109
 
@@ -110,20 +114,19 @@ The most useful flow for an AI agent is:
110
114
  If neither `ANTHROPIC_API_KEY` nor `OPENROUTER_API_KEY` is set:
111
115
 
112
116
  - `topIssues` will fall back to deterministic severity-sorted merge (still useful)
113
- - `visualDiff.results[].differences` will be empty (only pixelmatch heatmap survives)
114
- - `sectionsOnlyInProd` is still populated (uses DOM, not vision)
117
+ - the visual module is unaffected — it never used an LLM (heatmap engine)
115
118
  - Selector discovery is skipped; defaults from `.parityrc.json` or platform heuristics kick in
116
119
 
117
120
  The CLI never fails just because the LLM is unavailable. Issues will still be reported, just less polished.
118
121
 
119
122
  ## Cost-conscious usage
120
123
 
121
- A `--preset full` run with visual diff ≈ 6-24 Claude calls × ~$0.01-0.02 each. Cheap, but in tight loops:
124
+ The visual module costs no LLM calls. In tight loops:
122
125
 
123
126
  - Use `--preset smoke` first (no LLM, ~30s) to validate the URLs respond
124
127
  - Use `--no-visual-diff` if you only care about functional checks
125
128
  - Use `parity journey` (no aggregation step) for the tightest CI loop
126
- - **The cross-run cache** (`parity-output/cache/verdicts.json`) automatically skips the LLM call on pages whose screenshots haven't changed since the last verdict. Re-runs after a partial fix typically only spend LLM calls on the pages that actually changed.
129
+ - Use `parity heatmap` on the one page you just fixed instead of a whole run
127
130
 
128
131
  ## Picking which pages to verify
129
132
 
@@ -143,15 +146,10 @@ These flags override sitemap sampling entirely. Every listed path gets captured
143
146
 
144
147
  ## Cache flags
145
148
 
146
- - Default: cache is **on**. Persists at `<output>/cache/verdicts.json` between runs.
147
- - `--no-cache` — ignore existing cache entries; force fresh LLM judgment on every page.
148
- - `--clear-cache` — wipe the cache file before the run starts (use after a prompt change or when verdicts feel stale).
149
-
150
- The cache key is `sha256(prod_screenshot_bytes + cand_screenshot_bytes + LLM_PROMPT_VERSION)`. Any byte change in either screenshot invalidates the entry naturally. The `LLM_PROMPT_VERSION` constant (in `src/llm/visual-semantic-diff.ts`) bumps the entire cache atomically when the prompt or tool schema changes — no need to wipe by hand.
149
+ `--no-cache` / `--clear-cache` are accepted and do nothing: they controlled the LLM visual verdict cache, and the heatmap verdict is deterministic, so there is nothing left to cache.
151
150
 
152
151
  Hard caps already in place:
153
152
 
154
- - 12 max Claude Vision calls per `run` (visual diff)
155
153
  - 8 max selector recovery calls per run
156
154
  - 120s timeout per Vision call, 60s per text call
157
155
 
@@ -160,7 +158,8 @@ Hard caps already in place:
160
158
  - **First run is slower** — Playwright downloads Chromium (~150MB). After that, ~30s for smoke, ~5min for full.
161
159
  - **Live prod × cand side by side lives in `parity cockpit --prod <url> --cand <url>`**, not the HTML report; its proxy strips X-Frame-Options so any site renders.
162
160
  - **Cookies are isolated per run** — no shared session between prod and cand. If a flow needs auth, you'll need to handle that in `.parityrc.json` or extend the flow.
163
- - **Visual diff catches missing sections** mostly via DOM (`data-section`); if the migrated site doesn't emit that attribute, the LLM still sees the screenshots but loses the section-name context.
161
+ - **Visual diff catches missing sections** via DOM (`data-section`); if the migrated site doesn't emit that attribute, the heatmap still shows the region but the section name is lost.
162
+ - **A page whose height differs between prod and cand is `failed` (not compared)** — the pixel diff would only cover the intersection. Fix the height first (the first box-tree delta usually names why), or compare the section with `parity heatmap --selector`.
164
163
  - **`parity run` doesn't apply schema changes from old runs** — if you change `src/types/schema.ts`, old `report.json` files won't parse. Delete `parity-output/` or re-run.
165
164
 
166
165
  ## Files agents should and should not write
package/CHANGELOG.md CHANGED
@@ -5,6 +5,42 @@ All notable changes to this project will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/).
7
7
 
8
+ ## [Unreleased]
9
+
10
+ ### Changed
11
+
12
+ * **`parity run`'s visual module runs on the `parity heatmap` engine — no LLM.**
13
+ The old pass sent screenshot pairs to a vision model and filed what it said.
14
+ On real runs it invented findings — a stat card on neither side, prod and cand
15
+ swapped, a "section empty" that was a lazy load mid-flight — and a different set
16
+ each run, so the score was not reproducible. Now every visual issue is a
17
+ measured region: zero-tolerance pixel diff with prod's own motion masked out,
18
+ named by the box-tree delta under it (`[offset] button "Grab Yours Now!" sits
19
+ 18px further down than prod (rect.offsetY: 572px → 590px)`). Same pages in,
20
+ same findings out. Severity per region: `horizontal-overflow` high,
21
+ `image-source` low, otherwise high ≥ 50,000 differing px, medium ≥ 2,500,
22
+ low below; a uniform horizontal shift of several items of one list (a
23
+ carousel track at a different position) is low. A page that cannot be compared (different heights, fonts loading,
24
+ too much motion) is `failed` → an inconclusive issue, never a pass; a pass
25
+ that compared nothing is flagged as such, and `parityOk` needs ≥ 1 page.
26
+ * **Visual Diff tab** shows prod · cand · diff and the element/property/px list,
27
+ with a `parity heatmap` command per page. Its visual prompt is the heatmap's
28
+ findings in `parity heatmap`'s report shape. Visual Diff and Prompt LLM tabs
29
+ are no longer hidden without an LLM.
30
+ * `report.json` is additive: `visualDiff.results[]` gains `engine`, `findings`,
31
+ `diffPixels`, `noiseFloorPixels`, `maskedCoverage`, `reason`; old reports with
32
+ LLM `differences` still render.
33
+ * The visual pass captures at `deviceScaleFactor: 1` (what `parity heatmap`
34
+ compares) and no longer turns itself off when no LLM provider is configured.
35
+ * `parity heatmap` attributes the cand-side region of a moved element (where cand
36
+ put it) instead of reporting it `unattributed`.
37
+
38
+ ### Removed
39
+
40
+ * The vision-model visual diff (`src/llm/visual-semantic-diff.ts`) and its
41
+ cross-run verdict cache. `--no-cache` / `--clear-cache` are kept as no-ops so
42
+ existing scripts still parse.
43
+
8
44
  ## [0.28.1] — 2026-08-24
9
45
 
10
46
  ### Fixed