ui_guardrails 1.1.0 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +21 -3
- data/doc/RESEARCH-visual-diff.md +226 -0
- data/lib/guardrails/a11y_audit.rb +26 -3
- data/lib/guardrails/a11y_deep.rb +27 -2
- data/lib/guardrails/audit.rb +31 -3
- data/lib/guardrails/class_itis.rb +30 -3
- data/lib/guardrails/cross_codebase_patterns.rb +30 -3
- data/lib/guardrails/partial_similarity.rb +30 -3
- data/lib/guardrails/report/finding.rb +63 -0
- data/lib/guardrails/report/html/template.html.erb +237 -0
- data/lib/guardrails/report/html.rb +104 -0
- data/lib/guardrails/report/run.rb +224 -0
- data/lib/guardrails/stimulus_audit.rb +88 -12
- data/lib/guardrails/tui/editor.rb +71 -0
- data/lib/guardrails/tui/keys.rb +101 -0
- data/lib/guardrails/tui/screen.rb +361 -0
- data/lib/guardrails/tui/state.rb +356 -0
- data/lib/guardrails/tui.rb +154 -0
- data/lib/guardrails/version.rb +1 -1
- data/lib/guardrails/view_component_audit.rb +57 -7
- data/lib/guardrails/visual_diff.rb +36 -6
- data/lib/tasks/guardrails.rake +41 -192
- metadata +16 -9
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: a78df8763f35470a40ab6c8c8dcc6daf1351d35e733772a33afde5d1b7af1a54
|
|
4
|
+
data.tar.gz: a1f8aa3366f9a350673c55ff4200716e6c37db2f777592b914ada3236f67682e
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: f16927ab59ea1e9e89fe51203d3a0ca0f14331956fdcce4ca1e957642579129ba143651d066d9bb763f85fb4b2f3e5afd93a657a9266550b8435e0159478c659
|
|
7
|
+
data.tar.gz: 7f4dfebecd72b184a84258ca90f48f978c603eb01739d3217f8cad8b24d0ab7991f2d8a79dc53834e7e67002b0da74219cf68e3aee0ca60c344febb1e3e722e0
|
data/README.md
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
|
-
# Guardrails
|
|
1
|
+
# UI-Guardrails
|
|
2
2
|
|
|
3
3
|
A Rails toolset that prevents UI drift in AI-assisted applications. Static audits over your views, components, stylesheets, JS controllers, and tokens — surfacing the kinds of inconsistencies that compound silently as an AI assistant ships code faster than design-system discipline can keep up.
|
|
4
4
|
|
|
5
5
|
Built and maintained by [Meticulous](https://meticulous.com).
|
|
6
6
|
|
|
7
|
-
**Current release:** 1.
|
|
7
|
+
**Current release:** 1.3.0 — V0 + V1 + V2 complete, published on [RubyGems.org as `ui_guardrails`](https://rubygems.org/gems/ui_guardrails). The Ruby module stays `Guardrails` (so `require "guardrails"` is unchanged) — only the gem package name on rubygems carries the `ui_` prefix, to clear RubyGems' similarity rule against the unrelated [`guard-rails`](https://rubygems.org/gems/guard-rails) gem. See [`doc/ROADMAP.md`](doc/ROADMAP.md) for status, [`CHANGELOG.md`](CHANGELOG.md) for the full naming rationale.
|
|
8
8
|
|
|
9
9
|
---
|
|
10
10
|
|
|
@@ -92,6 +92,7 @@ APPLY=1 bundle exec rake guardrails:audit
|
|
|
92
92
|
|---|---|
|
|
93
93
|
| `guardrails:init` | Stack detection, writes `guardrails.yml`, scaffolds prefers-color-scheme / prefers-contrast media queries. Refuses to overwrite an existing config — `FORCE=1` overrides. |
|
|
94
94
|
| `guardrails:audit` | Runs every detector — view drift, stimulus, partial similarity, view-components, a11y, cross-codebase patterns, class-itis. Exits 1 on violations. |
|
|
95
|
+
| `guardrails:tui` | The same audit, browsable: a severity roll-up you drill into (category → finding → source), regroup by file, filter, and jump from into your editor or the HTML report. See [Browsing findings](#browsing-findings). |
|
|
95
96
|
| `guardrails:icons` | Generates an SVG sprite from `app/assets/images/icons/`, flags inline `<svg>` in views, reports unused icons. |
|
|
96
97
|
| `guardrails:tokens` | Parses your color and type-scale tokens (CSS vars / SCSS vars / Tailwind v3 config / Tailwind v4 `@theme`), reports hex literals in stylesheets that should reference a token. |
|
|
97
98
|
| `guardrails:a11y:deep` | Reads axe-core JSON output and folds it into the unified report. Doesn't run axe itself (no Capybara / headless Chrome runtime deps) — point it at axe output your existing tooling produces. |
|
|
@@ -195,7 +196,22 @@ Every task prints a human-readable text report by default and exits 1 when viola
|
|
|
195
196
|
FORMAT=json bundle exec rake guardrails:audit > findings.json
|
|
196
197
|
```
|
|
197
198
|
|
|
198
|
-
The JSON payload has a `summary:` block with finding counts per category plus per-detector arrays — see
|
|
199
|
+
The JSON payload has a `summary:` block with finding counts per category plus per-detector arrays — see `Guardrails::Report::Run#to_h` for the exact shape.
|
|
200
|
+
|
|
201
|
+
### Browsing findings
|
|
202
|
+
|
|
203
|
+
On a real codebase the text report is long (one of our test apps produces 981 findings). Two ways to navigate it instead of scrolling it:
|
|
204
|
+
|
|
205
|
+
```bash
|
|
206
|
+
bundle exec rake guardrails:tui # interactive, in the terminal
|
|
207
|
+
FORMAT=html bundle exec rake guardrails:audit # one self-contained HTML file
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
**`guardrails:tui`** opens on the same severity roll-up the text report leads with. `⏎` drills in — category → findings → a single finding with its suggestion, every location, and the source around it. `tab` regroups the whole thing by file (the files with the most findings first), `/` filters across category, title, suggestion, and path, `o` opens the highlighted location in your editor at the right line, `w` writes and opens the HTML report, `r` re-runs the audit, `?` lists the rest. It takes the same env vars as `guardrails:audit` except `APPLY` and `SUGGEST` — browsing never rewrites files. No new dependencies: it's built on Ruby's bundled `io/console`.
|
|
211
|
+
|
|
212
|
+
The editor is `$GUARDRAILS_EDITOR`, then `$VISUAL`, then `$EDITOR`. VS Code-family, Zed, Sublime, TextMate, JetBrains, Helix, and vim/nvim/nano/emacs all open at the finding's line; anything else gets the file path. With no editor set, it falls back to `open` / `xdg-open`.
|
|
213
|
+
|
|
214
|
+
**`FORMAT=html`** writes `tmp/guardrails/audit.html` (override with `OUTPUT=path`): the roll-up, every finding with its suggestion, locations, and snippet, a live filter, and severity toggles. Everything is inline — no server, no assets, no network — so it works opened from disk or uploaded as a CI artifact. File locations become editor links once you pick your editor from the menu in the header (remembered per browser). Note the file embeds absolute paths from the machine that generated it; that's what makes the editor links work.
|
|
199
215
|
|
|
200
216
|
### Common env vars
|
|
201
217
|
|
|
@@ -204,6 +220,8 @@ The JSON payload has a `summary:` block with finding counts per category plus pe
|
|
|
204
220
|
| `SUGGEST=1` | Write the markdown checklist alongside the text report. |
|
|
205
221
|
| `APPLY=1` | Auto-fix raw_color + tailwind_arbitrary where tokens match. |
|
|
206
222
|
| `FORMAT=json` | Emit one JSON document to stdout (all other audit output is suppressed). |
|
|
223
|
+
| `FORMAT=html` / `OUTPUT=path` | Write the self-contained HTML report (default `tmp/guardrails/audit.html`). |
|
|
224
|
+
| `GUARDRAILS_EDITOR=cmd` | Editor `guardrails:tui` opens locations in. Falls back to `$VISUAL`, then `$EDITOR`. |
|
|
207
225
|
| `FORCE=1` | Bypass `init`'s refuse-to-overwrite default. |
|
|
208
226
|
| `AXE_JSON=path` | Fold axe-core findings into the unified report. |
|
|
209
227
|
| `VISUAL_DIFF=1` | Fold visual-diff findings into `guardrails:audit`. Embedded installs can flip this on permanently via `Guardrails.configure { \|c\| c.visual_diff.enabled = true }`. |
|
|
@@ -0,0 +1,226 @@
|
|
|
1
|
+
# Visual-diff integration — research memo
|
|
2
|
+
|
|
3
|
+
> Status: research, not implementation. Closes the V2 visual-diff item's "evaluate a stable visual-regression target" prerequisite before any code lands.
|
|
4
|
+
>
|
|
5
|
+
> Last updated: 2026-05-10
|
|
6
|
+
|
|
7
|
+
## Executive summary
|
|
8
|
+
|
|
9
|
+
**Recommendation: ship a `Guardrails::VisualDiff` parser that consumes screenshot-diff tool output, the same way `Guardrails::A11yDeep` consumes axe-core JSON.** Don't bundle Chromium / Playwright / Capybara as runtime deps — that was the constraint that originally parked this item, and it hasn't changed.
|
|
10
|
+
|
|
11
|
+
The cleanest external producers to support, in priority order:
|
|
12
|
+
|
|
13
|
+
1. **`snap_diff-capybara`** (formerly `capybara-screenshot-diff`) — the Rails-native default. Active gem (1.12.0 released April 2026), commits baselines to git, no cloud dep. Highest fit-for-purpose.
|
|
14
|
+
2. **BackstopJS** — fully OSS, clean JSON output. Worth a second adapter for non-Rails-monolith projects (the Avo dogfood pattern).
|
|
15
|
+
3. **Percy / Chromatic** — SaaS, vendor-locked. Defer until a user actually asks; webhook ingestion is the integration shape if they do.
|
|
16
|
+
|
|
17
|
+
The talk narrative writes itself: **static analysis catches the source, visual diff catches what the rendered DOM does next.** Pair this with `ClassItis` — AI-generated 8-class soup looks plausible in the source but pixel-shifts the baseline once a single utility is wrong.
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## The constraint that shaped A11yDeep applies here too
|
|
22
|
+
|
|
23
|
+
When deferring deep a11y in the original roadmap, the call was:
|
|
24
|
+
|
|
25
|
+
> axe-core wrapper with `--deep` mode NOT shipped — would require Capybara + headless Chrome runtime deps.
|
|
26
|
+
|
|
27
|
+
Then in 0.6.0 we shipped `A11yDeep` as a **parser** for axe JSON output: zero new runtime deps, user runs axe however they already do, Guardrails provides the merge + report + exit-code contract. The same calculus applies to visual diff. Bundling Playwright/Chromium would balloon `bundle install` for users who don't run system tests; bundling Selenium would tie us to Ruby browser-driver decay. The right move is to consume what existing tools emit.
|
|
28
|
+
|
|
29
|
+
**Acceptance criteria for any candidate tool**, in priority order:
|
|
30
|
+
|
|
31
|
+
1. **Emits a machine-readable diff summary** — JSON, JUnit XML, or a parseable HTML/filesystem layout. We need per-scenario: name, baseline path, current path, diff path (if any), mismatch % or pass/fail, optionally the URL/component selector.
|
|
32
|
+
2. **Runs without Guardrails being involved.** The user's existing test toolchain runs the tool; we ingest the output. No shelling out to `npx`, no requiring the tool be present at audit time.
|
|
33
|
+
3. **OSS-friendly licensing** *or* a stable public API for SaaS tools.
|
|
34
|
+
4. **Active maintenance.** Sub-12-month release cadence and >50 GitHub stars as a rough proxy.
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## Per-tool brief
|
|
39
|
+
|
|
40
|
+
### snap_diff-capybara (Rails-native) — **PRIMARY RECOMMENDATION**
|
|
41
|
+
|
|
42
|
+
- **What it is:** Ruby gem (formerly `capybara-screenshot-diff`, now maintained at `github.com/snap-diff/snap_diff-capybara` by `donv` and `jetthoughts`). Latest 1.12.0, April 2026. MIT-licensed. ~770k total downloads.
|
|
43
|
+
- **How it works:** `screenshot "name"` inside Capybara system tests. Baselines live at `doc/screenshots/` and are **committed to git** — UI changes get reviewed in PR like code. No external auth, no cloud.
|
|
44
|
+
- **Diff artifacts:**
|
|
45
|
+
- `*.png` baseline
|
|
46
|
+
- `*.diff.png` (changed pixels highlighted red)
|
|
47
|
+
- `*.heatmap.diff.png` (pixel-variance density)
|
|
48
|
+
- **`snap_diff_report.html`** — interactive multi-view dashboard
|
|
49
|
+
- **CI flow:** Tests fail on baseline mismatch. There's a reusable GitHub Actions workflow that uploads the HTML report and posts PR comments. Ships its own baseline-update flow.
|
|
50
|
+
- **Runtime:** Ruby 3.2+, any Capybara-compatible driver (Selenium / Playwright / Cuprite). libvips 8.9+ for fast comparisons, falls back to ChunkyPNG.
|
|
51
|
+
- **For Guardrails to consume:** the HTML report has a fixed structure, but the gem doesn't (yet) emit a top-level JSON summary. **Either:**
|
|
52
|
+
- **(a)** ask upstream for a `snap_diff_report.json` companion to the HTML — small ask, this is the maintained gem
|
|
53
|
+
- **(b)** walk `doc/screenshots/` ourselves: pair `name.png` with `name.diff.png`, treat presence of the diff file as a failing finding
|
|
54
|
+
- **Fit:** ★★★★★. Same ecosystem (Ruby/Rails/Capybara), no new deps for users already running system tests, baselines-in-git matches the Guardrails ethos of "the source is the source of truth."
|
|
55
|
+
|
|
56
|
+
### BackstopJS
|
|
57
|
+
|
|
58
|
+
- **What it is:** Node CLI, MIT-licensed, actively maintained, the dominant OSS visual-regression tool outside the Storybook ecosystem.
|
|
59
|
+
- **How it works:** `backstop init`, `backstop test`, `backstop approve`. Configured via `backstop.json` (scenarios = URL × viewport × selector). Headless Chrome via Puppeteer or Firefox/WebKit via Playwright. All artifacts local under `backstop_data/`.
|
|
60
|
+
- **Output:** HTML interactive report, **JSON report**, JUnit XML, CLI text. Exit 0/1.
|
|
61
|
+
- **For Guardrails to consume:** The JSON report already exists and is the canonical machine-readable output. Per-scenario: name, label, mismatch percentage, pass/fail, paths to reference + test + diff images.
|
|
62
|
+
- **Fit:** ★★★★☆. Native machine-readable output, no SaaS dep, but Node-only — Ruby/Rails shops would have a parallel toolchain. Justifiable for the "Rails frontend that already runs JS tooling" subset (any non-trivial Rails app).
|
|
63
|
+
|
|
64
|
+
### Playwright `toHaveScreenshot` / `toMatchSnapshot`
|
|
65
|
+
|
|
66
|
+
- **What it is:** Built-in Playwright assertion. The team that maintains Playwright also maintains Chromatic (Microsoft → Storybook acquisition), so it's becoming the de facto cross-tool snapshot mechanism.
|
|
67
|
+
- **How it works:** `await expect(page).toHaveScreenshot()`. Stores baselines in `__screenshots__/`. Updates via `--update-snapshots`.
|
|
68
|
+
- **Output:** JUnit / JSON via Playwright's reporter API; HTML report; per-test pass/fail in CI.
|
|
69
|
+
- **For Guardrails to consume:** Playwright JSON reporter emits a structured test-result document; each failure includes `attachments` with the diff image path. Parseable, but the schema is "Playwright test results" not "visual diff" — we'd be extracting the visual subset.
|
|
70
|
+
- **Fit:** ★★★☆☆. Future-proof and increasingly canonical, but the integration is heavier (have to filter Playwright results for screenshot-specific failures). Worth supporting in v2 of the adapter.
|
|
71
|
+
|
|
72
|
+
### Percy (BrowserStack)
|
|
73
|
+
|
|
74
|
+
- **What it is:** SaaS, all builds go through Percy's cloud. Originally a standalone product, BrowserStack-acquired in 2020.
|
|
75
|
+
- **Ruby story:** `percy-capybara` gem exists. Latest release v5.0.0 — July 2021 — i.e. **maintenance-stale**. 4+ years without a meaningful release; the active surface is the JS SDKs.
|
|
76
|
+
- **Output:** Dashboard URL + per-build webhook. No local artifacts beyond the snapshot upload.
|
|
77
|
+
- **Pricing:** Pricing page is gated behind BrowserStack's product navigation; no clear OSS-free tier surfaced via the public site. Assume paid-only for non-trial use.
|
|
78
|
+
- **For Guardrails to consume:** `build-finished` webhook payload is the integration shape — but that requires a receiver, which Guardrails-the-gem isn't. Could ship a parser for the webhook JSON that users POST to their CI.
|
|
79
|
+
- **Fit:** ★★☆☆☆. Stale Ruby SDK, vendor-locked, no clear OSS pricing. Defer until a user asks specifically.
|
|
80
|
+
|
|
81
|
+
### Chromatic
|
|
82
|
+
|
|
83
|
+
- **What it is:** SaaS, made by the Storybook team. The standard visual-regression for Storybook-based component libraries; expanded to Playwright + Cypress in 2024–25.
|
|
84
|
+
- **Ruby story:** None. JS-ecosystem only. No Lookbook integration despite Lookbook being the Rails-shaped Storybook equivalent.
|
|
85
|
+
- **Output:** Cloud dashboard, web API for build status, no local artifacts.
|
|
86
|
+
- **Pricing:** Free tier (5,000 snapshots/month, Chrome-only) — generous. Paid starts at $179/mo. "Building in the open?" OSS plan available on application.
|
|
87
|
+
- **Fit:** ★★☆☆☆ for Rails apps without Storybook. ★★★★☆ for ViewComponent shops who use Storybook for non-Rails components alongside. Pair-of-tools story is workable but indirect; Lookbook integration would be the missing link, and that's a Lookbook upstream ask, not a Guardrails one.
|
|
88
|
+
|
|
89
|
+
### Honorable mentions
|
|
90
|
+
|
|
91
|
+
- **Loki** — Storybook-only, narrow audience for Rails-flavored Guardrails. Skip.
|
|
92
|
+
- **`reg-suit`** — Cloud or self-hosted, language-agnostic, JSON report. Reasonable BackstopJS-alternative; not common enough in Rails shops to justify a dedicated adapter at v1.
|
|
93
|
+
|
|
94
|
+
---
|
|
95
|
+
|
|
96
|
+
## Fit table
|
|
97
|
+
|
|
98
|
+
| Tool | OSS | Rails-native | Machine-readable output | Cloud dep | Active | Fit score |
|
|
99
|
+
|---|---|---|---|---|---|---|
|
|
100
|
+
| snap_diff-capybara | ✅ MIT | ✅ Ruby gem | 🟡 HTML report (no JSON yet) | ❌ baselines in git | ✅ active | ★★★★★ |
|
|
101
|
+
| BackstopJS | ✅ MIT | ❌ Node | ✅ JSON report | ❌ local FS | ✅ active | ★★★★☆ |
|
|
102
|
+
| Playwright snapshots | ✅ Apache | ❌ Node | ✅ JSON via reporter | ❌ local FS | ✅ active | ★★★☆☆ |
|
|
103
|
+
| Percy | 🔒 SaaS | 🟡 stale gem (2021) | ✅ webhook | ✅ required | 🟡 webhooks active, SDK stale | ★★☆☆☆ |
|
|
104
|
+
| Chromatic | 🔒 SaaS | ❌ JS-only | ✅ API | ✅ required | ✅ active | ★★☆☆☆ |
|
|
105
|
+
|
|
106
|
+
---
|
|
107
|
+
|
|
108
|
+
## Proposed integration shape (mirrors A11yDeep)
|
|
109
|
+
|
|
110
|
+
```
|
|
111
|
+
lib/guardrails/visual_diff.rb — parser + report
|
|
112
|
+
lib/guardrails/visual_diff/snap_diff.rb — adapter for snap_diff-capybara
|
|
113
|
+
lib/guardrails/visual_diff/backstop.rb — adapter for BackstopJS JSON
|
|
114
|
+
spec/guardrails/visual_diff_spec.rb
|
|
115
|
+
spec/guardrails/visual_diff/snap_diff_spec.rb
|
|
116
|
+
spec/guardrails/visual_diff/backstop_spec.rb
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
Normalized `Finding` shape (across both adapters):
|
|
120
|
+
|
|
121
|
+
```ruby
|
|
122
|
+
Finding = Struct.new(
|
|
123
|
+
:scenario, # "homepage", "checkout_cart"
|
|
124
|
+
:viewport, # "desktop"; nil if not applicable
|
|
125
|
+
:mismatch_ratio, # Float 0.0..1.0 (or nil for snap_diff which is pass/fail)
|
|
126
|
+
:baseline_path, # relative path on disk
|
|
127
|
+
:current_path, # relative path on disk
|
|
128
|
+
:diff_path, # relative path to .diff.png, nil if pass
|
|
129
|
+
:url, # optional, BackstopJS scenarios have URLs
|
|
130
|
+
:selector, # optional
|
|
131
|
+
keyword_init: true
|
|
132
|
+
)
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
Rake task (mirrors `guardrails:a11y:deep`):
|
|
136
|
+
|
|
137
|
+
```bash
|
|
138
|
+
# Either auto-detect from `doc/screenshots/` (snap_diff convention) ...
|
|
139
|
+
bundle exec rake guardrails:visual:deep
|
|
140
|
+
|
|
141
|
+
# ... or point at an explicit producer:
|
|
142
|
+
SNAP_DIFF_DIR=doc/screenshots bundle exec rake guardrails:visual:deep
|
|
143
|
+
BACKSTOP_JSON=backstop_data/html_report/jsonReport.json bundle exec rake guardrails:visual:deep
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
Folded into `guardrails:audit` via `VISUAL_DIFF=...` env var, same shape as `AXE_JSON`.
|
|
147
|
+
|
|
148
|
+
JSON output extends the existing audit payload:
|
|
149
|
+
|
|
150
|
+
```json
|
|
151
|
+
{
|
|
152
|
+
"summary": { "...": "...", "visual_diff": 3 },
|
|
153
|
+
"visual_diff": [
|
|
154
|
+
{ "scenario": "checkout/cart", "mismatch_ratio": 0.082,
|
|
155
|
+
"baseline_path": "doc/screenshots/checkout_cart.png",
|
|
156
|
+
"diff_path": "doc/screenshots/checkout_cart.diff.png" }
|
|
157
|
+
]
|
|
158
|
+
}
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
Exit-code contract: a finding's `mismatch_ratio > threshold` (configurable, default `0.0` — any diff fails) bumps the audit to exit 1. For snap_diff (pass/fail), any failure is a fail.
|
|
162
|
+
|
|
163
|
+
---
|
|
164
|
+
|
|
165
|
+
## What this buys the gem (and the talk)
|
|
166
|
+
|
|
167
|
+
1. **Closes V2.** Roadmap is fully shipped on the three core items.
|
|
168
|
+
2. **Talk narrative is now four layers deep:**
|
|
169
|
+
- *Static AST drift* (V0): the source-level mistakes.
|
|
170
|
+
- *Structural drift* (V2 cross-codebase + class-itis): the patterns the source shouldn't have.
|
|
171
|
+
- *Runtime a11y drift* (V1.6 deep a11y): what the static checks can't see.
|
|
172
|
+
- *Visual drift* (V2 visual-diff): what the rendered DOM does next.
|
|
173
|
+
|
|
174
|
+
AI assistants fail at each layer differently. Each Guardrails detector maps to a specific failure mode.
|
|
175
|
+
|
|
176
|
+
3. **Same install/run story as everything else.** No new install footprint. Users keep their preferred screenshot tool; Guardrails provides the unified report.
|
|
177
|
+
|
|
178
|
+
---
|
|
179
|
+
|
|
180
|
+
## Open decisions for the morning
|
|
181
|
+
|
|
182
|
+
1. **Primary adapter: snap_diff or BackstopJS?**
|
|
183
|
+
- My read: snap_diff-capybara, since (a) Rails-native, (b) we'd be working with the maintainers of a small focused gem who'd likely accept a JSON-output PR upstream, and (c) baselines-in-git matches Guardrails' "source is canonical" ethos.
|
|
184
|
+
- The risk: snap_diff doesn't currently emit JSON, so v0.8.0 might need an upstream contribution. We could ship the filesystem-walking adapter first (look at `doc/screenshots/*.diff.png`) and contribute the JSON emitter separately.
|
|
185
|
+
|
|
186
|
+
2. **Scope: ship one adapter or two?**
|
|
187
|
+
- One (snap_diff) keeps the PR small and proves the pattern.
|
|
188
|
+
- Two (snap_diff + BackstopJS) covers the Avo / non-Rails-monolith case from day one.
|
|
189
|
+
- I'd ship one and tag, then add BackstopJS in a follow-up if there's signal.
|
|
190
|
+
|
|
191
|
+
3. **Threshold default: `0.0` (any diff fails) or something more lenient?**
|
|
192
|
+
- A11yDeep defaulted to "any non-nil impact fails". The visual-diff equivalent is "any pixel mismatch fails" — strict, but visual diff is opt-in (you set up baselines deliberately), so strict is the right default.
|
|
193
|
+
- Per-call override via `VISUAL_DIFF_THRESHOLD=0.01` env or `visual_diff.threshold` in `guardrails.yml`.
|
|
194
|
+
|
|
195
|
+
4. **Where does this surface in the Lookbook panel?**
|
|
196
|
+
- `ComponentReport#for` could add `visual_diff: [...]` showing per-component baseline mismatches. Pulls together: "this component drifted statically (raw_color) AND visually (3% mismatch on the desktop viewport)."
|
|
197
|
+
- Probably out of scope for v0.8.0 — first land the audit/CI integration, layer Lookbook in v0.9.0.
|
|
198
|
+
|
|
199
|
+
5. **Is this 0.8.0 or 1.0.0?**
|
|
200
|
+
- Argument for 0.8.0: matches the pattern of every other increment.
|
|
201
|
+
- Argument for 1.0.0: with this shipped, all three V0–V2 column-headers are done. We could draw the "ready for serious use" line here and tag a 1.0.0.
|
|
202
|
+
- My read: 0.8.0 for the visual-diff feature, then a separate 1.0.0 release after dogfooding the full V2 surface against Patchvault/Talos/Forem/Avo. The 0.2.x→0.7.x cadence already moved through dogfood-driven patches; another round before 1.0.0 fits.
|
|
203
|
+
|
|
204
|
+
---
|
|
205
|
+
|
|
206
|
+
## Out of scope
|
|
207
|
+
|
|
208
|
+
- **Running the screenshot tool ourselves.** Not now, possibly not ever — the gem's ethos is parse-only.
|
|
209
|
+
- **Hosted baseline storage.** Baselines stay in the user's repo (snap_diff convention) or wherever their existing tool puts them.
|
|
210
|
+
- **Image-diff algorithm work.** Pixel comparison is a solved problem with multiple good libraries; we don't add value by reinventing it.
|
|
211
|
+
- **Real-browser rendering.** Same as the headless Chrome reasoning in A11yDeep.
|
|
212
|
+
|
|
213
|
+
---
|
|
214
|
+
|
|
215
|
+
## What's needed to start
|
|
216
|
+
|
|
217
|
+
If we go with snap_diff-capybara as the primary adapter:
|
|
218
|
+
|
|
219
|
+
1. Decide if we file an upstream PR to add `snap_diff_report.json` emission, or build the filesystem-walking adapter first.
|
|
220
|
+
2. Add `snap_diff-capybara` to `examples/demo/Gemfile` and seed a couple of system tests so the integration spec has something to verify against.
|
|
221
|
+
3. Roughly 3 slices, mirroring the A11yDeep PR:
|
|
222
|
+
- Slice 1: `VisualDiff` parser + `Finding` struct + snap_diff adapter + spec
|
|
223
|
+
- Slice 2: rake task wiring (standalone + `guardrails:audit` fold-in) + JSON output extension
|
|
224
|
+
- Slice 3: docs (new `doc/VISUAL-DIFF.md` mirroring `doc/A11Y.md`) + ROADMAP + CHANGELOG → 0.8.0
|
|
225
|
+
|
|
226
|
+
Estimated scope: a single PR, comparable to 0.6.0's `A11yDeep` (~17 specs, ~400 LOC). Could be shipped same day as kick-off if no upstream blocker.
|
|
@@ -4,6 +4,7 @@ require "pathname"
|
|
|
4
4
|
require "set"
|
|
5
5
|
require_relative "erb_parser"
|
|
6
6
|
require_relative "report/style"
|
|
7
|
+
require_relative "report/finding"
|
|
7
8
|
|
|
8
9
|
module Guardrails
|
|
9
10
|
# Static a11y checks that don't require a browser — element-level rules
|
|
@@ -43,8 +44,32 @@ module Guardrails
|
|
|
43
44
|
findings
|
|
44
45
|
end
|
|
45
46
|
|
|
47
|
+
# Detector-agnostic view of `findings` (see Report::Finding).
|
|
48
|
+
def categories(findings)
|
|
49
|
+
return [] if findings.empty?
|
|
50
|
+
|
|
51
|
+
[Report::Category.new(
|
|
52
|
+
name: "a11y (static)", severity: :error, framing: FRAMING.join(" "),
|
|
53
|
+
findings: findings.map { |f|
|
|
54
|
+
Report::Finding.new(
|
|
55
|
+
category: "a11y (static)", severity: :error,
|
|
56
|
+
title: "#{f.rule}: #{f.snippet.to_s[0, 60]}",
|
|
57
|
+
suggestion: SUGGESTION_FOR_RULE[f.rule.to_s],
|
|
58
|
+
locations: [Report::Location.new(file: f.file, line: f.line, column: f.column)],
|
|
59
|
+
snippet: f.snippet
|
|
60
|
+
)
|
|
61
|
+
}
|
|
62
|
+
)]
|
|
63
|
+
end
|
|
64
|
+
|
|
46
65
|
private
|
|
47
66
|
|
|
67
|
+
FRAMING = [
|
|
68
|
+
"Element-level a11y rules answerable from view source — missing alt text,",
|
|
69
|
+
"unnamed buttons, unlabeled inputs, link without name. Full WCAG coverage",
|
|
70
|
+
"needs runtime checks; layer axe-core via AXE_JSON= for that."
|
|
71
|
+
].freeze
|
|
72
|
+
|
|
48
73
|
def view_files
|
|
49
74
|
SCAN_PATTERNS
|
|
50
75
|
.flat_map { |pattern| Dir.glob(@root.join(pattern)) }
|
|
@@ -249,9 +274,7 @@ module Guardrails
|
|
|
249
274
|
noun = findings.length == 1 ? "issue" : "issues"
|
|
250
275
|
@output.puts ""
|
|
251
276
|
@output.puts @style.section_heading(:error, "a11y (#{findings.length} static #{noun})")
|
|
252
|
-
@output.puts "
|
|
253
|
-
@output.puts " unnamed buttons, unlabeled inputs, link without name. Full WCAG coverage"
|
|
254
|
-
@output.puts " needs runtime checks; layer axe-core via AXE_JSON= for that."
|
|
277
|
+
FRAMING.each { |line| @output.puts " #{line}" }
|
|
255
278
|
|
|
256
279
|
findings.each do |f|
|
|
257
280
|
@output.puts ""
|
data/lib/guardrails/a11y_deep.rb
CHANGED
|
@@ -4,6 +4,7 @@ require "json"
|
|
|
4
4
|
require "pathname"
|
|
5
5
|
require "set"
|
|
6
6
|
require_relative "report/style"
|
|
7
|
+
require_relative "report/finding"
|
|
7
8
|
|
|
8
9
|
module Guardrails
|
|
9
10
|
# Consumes axe-core JSON output and folds the findings into Guardrails'
|
|
@@ -64,8 +65,33 @@ module Guardrails
|
|
|
64
65
|
findings.any? { |f| @failing_impacts.include?(f.impact.to_s) }
|
|
65
66
|
end
|
|
66
67
|
|
|
68
|
+
# Detector-agnostic view of `findings` (see Report::Finding). These
|
|
69
|
+
# come from a live page, not a source file, so there are no
|
|
70
|
+
# locations — url and selector ride along as details. Severity is
|
|
71
|
+
# per finding, mapped from axe's impact.
|
|
72
|
+
def categories(findings)
|
|
73
|
+
return [] if findings.empty?
|
|
74
|
+
|
|
75
|
+
[Report::Category.new(
|
|
76
|
+
name: "a11y (deep)", severity: :error, framing: FRAMING.join(" "),
|
|
77
|
+
findings: findings.map { |f|
|
|
78
|
+
Report::Finding.new(
|
|
79
|
+
category: "a11y (deep)", severity: impact_to_severity(f.impact),
|
|
80
|
+
title: "[#{f.impact || 'unknown'}] #{f.rule}: #{f.description}",
|
|
81
|
+
suggestion: f.help_url && "see #{f.help_url}",
|
|
82
|
+
details: [["url", f.url], ["selector", f.selector], ["help", f.help_url]].reject { |_, v| v.nil? }
|
|
83
|
+
)
|
|
84
|
+
}
|
|
85
|
+
)]
|
|
86
|
+
end
|
|
87
|
+
|
|
67
88
|
private
|
|
68
89
|
|
|
90
|
+
FRAMING = [
|
|
91
|
+
"Runtime accessibility issues axe-core caught against your live pages.",
|
|
92
|
+
"Each links to dequeuniversity.com for the canonical remediation."
|
|
93
|
+
].freeze
|
|
94
|
+
|
|
69
95
|
def parse_input
|
|
70
96
|
raw = @input.is_a?(Hash) || @input.is_a?(Array) ? @input : JSON.parse(File.read(@input.to_s, encoding: Encoding::UTF_8))
|
|
71
97
|
parse(raw)
|
|
@@ -110,8 +136,7 @@ module Guardrails
|
|
|
110
136
|
:error,
|
|
111
137
|
"a11y deep (#{findings.length} #{noun} from axe-core)"
|
|
112
138
|
)
|
|
113
|
-
@output.puts "
|
|
114
|
-
@output.puts " Each links to dequeuniversity.com for the canonical remediation."
|
|
139
|
+
FRAMING.each { |line| @output.puts " #{line}" }
|
|
115
140
|
|
|
116
141
|
grouped.each do |url, page_findings|
|
|
117
142
|
@output.puts ""
|
data/lib/guardrails/audit.rb
CHANGED
|
@@ -6,6 +6,7 @@ require "stringio"
|
|
|
6
6
|
require "yaml"
|
|
7
7
|
require_relative "erb_parser"
|
|
8
8
|
require_relative "report/style"
|
|
9
|
+
require_relative "report/finding"
|
|
9
10
|
|
|
10
11
|
module Guardrails
|
|
11
12
|
class Audit
|
|
@@ -88,6 +89,32 @@ module Guardrails
|
|
|
88
89
|
remaining
|
|
89
90
|
end
|
|
90
91
|
|
|
92
|
+
# Detector-agnostic view of `violations` (see Report::Finding) for
|
|
93
|
+
# front-ends that list and link findings rather than print them.
|
|
94
|
+
# One Category per violation type, in the text report's order.
|
|
95
|
+
def categories(violations)
|
|
96
|
+
by_type = violations.group_by(&:type)
|
|
97
|
+
(TYPE_ORDER + (by_type.keys - TYPE_ORDER)).filter_map do |type|
|
|
98
|
+
list = by_type[type]
|
|
99
|
+
next if list.nil? || list.empty?
|
|
100
|
+
|
|
101
|
+
severity = SEVERITY_FOR_TYPE.fetch(type, :warning)
|
|
102
|
+
auto_fix = AUTO_FIXABLE_TYPES.include?(type)
|
|
103
|
+
Report::Category.new(
|
|
104
|
+
name: type.to_s, severity: severity, framing: FRAMING_FOR_TYPE[type], auto_fix: auto_fix,
|
|
105
|
+
findings: list.map { |v|
|
|
106
|
+
Report::Finding.new(
|
|
107
|
+
category: type.to_s, severity: severity, auto_fix: auto_fix,
|
|
108
|
+
title: "#{type}: #{format_value(v)}",
|
|
109
|
+
suggestion: suggestion_for_violation(v),
|
|
110
|
+
locations: [Report::Location.new(file: v.file, line: v.line, column: v.column)],
|
|
111
|
+
snippet: v.snippet
|
|
112
|
+
)
|
|
113
|
+
}
|
|
114
|
+
)
|
|
115
|
+
end
|
|
116
|
+
end
|
|
117
|
+
|
|
91
118
|
private
|
|
92
119
|
|
|
93
120
|
def load_audit_config
|
|
@@ -459,10 +486,8 @@ module Guardrails
|
|
|
459
486
|
# (a suggestion-shaped warning) and a11y (errors but grouped
|
|
460
487
|
# separately because A11yAudit owns them — the audit rake task
|
|
461
488
|
# threads them in via this same method).
|
|
462
|
-
type_order = %i[inline_style raw_color tailwind_arbitrary helper_recommended
|
|
463
|
-
image_alt button_name link_name input_label]
|
|
464
489
|
by_type = violations.group_by(&:type)
|
|
465
|
-
ordered =
|
|
490
|
+
ordered = TYPE_ORDER + (by_type.keys - TYPE_ORDER)
|
|
466
491
|
|
|
467
492
|
ordered.each do |type|
|
|
468
493
|
list = by_type[type] || []
|
|
@@ -472,6 +497,9 @@ module Guardrails
|
|
|
472
497
|
end
|
|
473
498
|
end
|
|
474
499
|
|
|
500
|
+
TYPE_ORDER = %i[inline_style raw_color tailwind_arbitrary helper_recommended
|
|
501
|
+
image_alt button_name link_name input_label].freeze
|
|
502
|
+
|
|
475
503
|
SEVERITY_FOR_TYPE = {
|
|
476
504
|
inline_style: :warning,
|
|
477
505
|
raw_color: :error,
|
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
require "pathname"
|
|
4
4
|
require_relative "erb_parser"
|
|
5
5
|
require_relative "report/style"
|
|
6
|
+
require_relative "report/finding"
|
|
6
7
|
|
|
7
8
|
module Guardrails
|
|
8
9
|
# Finds repeating "class soup" — the same long class list applied to
|
|
@@ -98,8 +99,36 @@ module Guardrails
|
|
|
98
99
|
.sort_by { |c| [-c.count, -c.class_count] }
|
|
99
100
|
end
|
|
100
101
|
|
|
102
|
+
# Detector-agnostic view of `clusters` (see Report::Finding). One
|
|
103
|
+
# finding per cluster, carrying every occurrence and the full
|
|
104
|
+
# class list (the text report truncates both for terminal width).
|
|
105
|
+
def categories(clusters)
|
|
106
|
+
return [] if clusters.empty?
|
|
107
|
+
|
|
108
|
+
[Report::Category.new(
|
|
109
|
+
name: "class-itis", severity: :suggestion, framing: framing_lines.join(" "),
|
|
110
|
+
findings: clusters.map { |cluster|
|
|
111
|
+
Report::Finding.new(
|
|
112
|
+
category: "class-itis", severity: :suggestion,
|
|
113
|
+
title: "<#{cluster.tag}> with #{cluster.class_count} classes, #{cluster.count} occurrences",
|
|
114
|
+
suggestion: suggestion_for(cluster),
|
|
115
|
+
locations: cluster.occurrences.map { |occ| Report::Location.new(file: occ.file, line: occ.line, column: occ.column) },
|
|
116
|
+
details: [["class", cluster.classes.join(" ")]]
|
|
117
|
+
)
|
|
118
|
+
}
|
|
119
|
+
)]
|
|
120
|
+
end
|
|
121
|
+
|
|
101
122
|
private
|
|
102
123
|
|
|
124
|
+
def framing_lines
|
|
125
|
+
[
|
|
126
|
+
"The same multi-class list applied to the same tag in many places —",
|
|
127
|
+
"classic AI-paste pattern. Consider extracting a shared component or",
|
|
128
|
+
"an @apply rule. Threshold: >= #{@min_classes} classes, >= #{@min_occurrences} occurrences."
|
|
129
|
+
]
|
|
130
|
+
end
|
|
131
|
+
|
|
103
132
|
def view_files
|
|
104
133
|
paths = VIEW_PATTERNS.flat_map { |g| Dir.glob(@root.join(g)) }.map { |p| Pathname(p) }.uniq
|
|
105
134
|
paths.reject { |p| ignored?(p) }
|
|
@@ -174,9 +203,7 @@ module Guardrails
|
|
|
174
203
|
:suggestion,
|
|
175
204
|
"class-itis (#{clusters.length} #{noun}, #{total_occurrences} occurrences)"
|
|
176
205
|
)
|
|
177
|
-
@output.puts "
|
|
178
|
-
@output.puts " classic AI-paste pattern. Consider extracting a shared component or"
|
|
179
|
-
@output.puts " an @apply rule. Threshold: >= #{@min_classes} classes, >= #{@min_occurrences} occurrences."
|
|
206
|
+
framing_lines.each { |line| @output.puts " #{line}" }
|
|
180
207
|
|
|
181
208
|
clusters.each do |cluster|
|
|
182
209
|
@output.puts ""
|
|
@@ -5,6 +5,7 @@ require "digest"
|
|
|
5
5
|
require "set"
|
|
6
6
|
require_relative "erb_parser"
|
|
7
7
|
require_relative "report/style"
|
|
8
|
+
require_relative "report/finding"
|
|
8
9
|
|
|
9
10
|
module Guardrails
|
|
10
11
|
# Finds recurring structural patterns across the codebase — element
|
|
@@ -98,8 +99,36 @@ module Guardrails
|
|
|
98
99
|
dedupe_nested(patterns)
|
|
99
100
|
end
|
|
100
101
|
|
|
102
|
+
# Detector-agnostic view of `patterns` (see Report::Finding). One
|
|
103
|
+
# finding per shape, carrying every occurrence — the text report
|
|
104
|
+
# caps the list for terminal width; this doesn't.
|
|
105
|
+
def categories(patterns)
|
|
106
|
+
return [] if patterns.empty?
|
|
107
|
+
|
|
108
|
+
[Report::Category.new(
|
|
109
|
+
name: "cross-codebase patterns", severity: :suggestion, framing: framing_lines.join(" "),
|
|
110
|
+
findings: patterns.map { |pattern|
|
|
111
|
+
Report::Finding.new(
|
|
112
|
+
category: "cross-codebase patterns", severity: :suggestion,
|
|
113
|
+
title: "shape: #{truncate_shape(pattern.shape)} (#{pattern.size} elements, #{pattern.count} occurrences)",
|
|
114
|
+
suggestion: suggestion_for(pattern),
|
|
115
|
+
locations: pattern.occurrences.map { |occ| Report::Location.new(file: occ.file, line: occ.line, column: occ.column) },
|
|
116
|
+
details: [["shape", pattern.shape]]
|
|
117
|
+
)
|
|
118
|
+
}
|
|
119
|
+
)]
|
|
120
|
+
end
|
|
121
|
+
|
|
101
122
|
private
|
|
102
123
|
|
|
124
|
+
def framing_lines
|
|
125
|
+
[
|
|
126
|
+
"These element subtrees repeat #{@min_occurrences}+ times across your views and",
|
|
127
|
+
"components. Each is a candidate for extracting into a shared partial or",
|
|
128
|
+
"ViewComponent. Threshold: >= #{@min_size} elements, >= #{@min_occurrences} occurrences."
|
|
129
|
+
]
|
|
130
|
+
end
|
|
131
|
+
|
|
103
132
|
# Drop redundant inner shapes. When a table repeats N times, three
|
|
104
133
|
# patterns end up with identical counts:
|
|
105
134
|
#
|
|
@@ -222,9 +251,7 @@ module Guardrails
|
|
|
222
251
|
:suggestion,
|
|
223
252
|
"cross-codebase patterns (#{patterns.length} #{noun}, #{total_occurrences} occurrences)"
|
|
224
253
|
)
|
|
225
|
-
@output.puts "
|
|
226
|
-
@output.puts " components. Each is a candidate for extracting into a shared partial or"
|
|
227
|
-
@output.puts " ViewComponent. Threshold: >= #{@min_size} elements, >= #{@min_occurrences} occurrences."
|
|
254
|
+
framing_lines.each { |line| @output.puts " #{line}" }
|
|
228
255
|
|
|
229
256
|
patterns.each do |pattern|
|
|
230
257
|
@output.puts ""
|
|
@@ -4,6 +4,7 @@ require "pathname"
|
|
|
4
4
|
require "set"
|
|
5
5
|
require_relative "erb_parser"
|
|
6
6
|
require_relative "report/style"
|
|
7
|
+
require_relative "report/finding"
|
|
7
8
|
|
|
8
9
|
module Guardrails
|
|
9
10
|
class PartialSimilarity
|
|
@@ -176,8 +177,36 @@ module Guardrails
|
|
|
176
177
|
groups.sort_by { |g| -g[:files].size }
|
|
177
178
|
end
|
|
178
179
|
|
|
180
|
+
# Detector-agnostic view of `findings` (see Report::Finding). One
|
|
181
|
+
# finding per pair — the unit the summary counts — rather than per
|
|
182
|
+
# connected group the way the text report condenses them.
|
|
183
|
+
def categories(findings)
|
|
184
|
+
return [] if findings.empty?
|
|
185
|
+
|
|
186
|
+
[Report::Category.new(
|
|
187
|
+
name: "similar partials", severity: :suggestion, framing: framing_lines.join(" "),
|
|
188
|
+
findings: findings.map { |f|
|
|
189
|
+
Report::Finding.new(
|
|
190
|
+
category: "similar partials", severity: :suggestion,
|
|
191
|
+
title: "#{format('%.2f', f.score)} similar: #{f.file_a} ↔ #{f.file_b}",
|
|
192
|
+
suggestion: suggestion_for_pair({ score_max: f.score }),
|
|
193
|
+
locations: [Report::Location.new(file: f.file_a), Report::Location.new(file: f.file_b)],
|
|
194
|
+
details: [["tags", "#{f.tag_count_a} / #{f.tag_count_b}"]]
|
|
195
|
+
)
|
|
196
|
+
}
|
|
197
|
+
)]
|
|
198
|
+
end
|
|
199
|
+
|
|
179
200
|
private
|
|
180
201
|
|
|
202
|
+
def framing_lines
|
|
203
|
+
[
|
|
204
|
+
"Templates with >= #{@threshold} structural similarity. Likely duplicates;",
|
|
205
|
+
"consider extracting the common shape into a partial or parameterizing",
|
|
206
|
+
"one with locals to subsume the others."
|
|
207
|
+
]
|
|
208
|
+
end
|
|
209
|
+
|
|
181
210
|
def collect_partials
|
|
182
211
|
PARTIAL_PATTERNS
|
|
183
212
|
.flat_map { |pattern| Dir.glob(@root.join(pattern)) }
|
|
@@ -212,9 +241,7 @@ module Guardrails
|
|
|
212
241
|
:suggestion,
|
|
213
242
|
"similar partials (#{groups.length} #{group_noun}, #{findings.length} pairs, #{total_files} files)"
|
|
214
243
|
)
|
|
215
|
-
@output.puts "
|
|
216
|
-
@output.puts " consider extracting the common shape into a partial or parameterizing"
|
|
217
|
-
@output.puts " one with locals to subsume the others."
|
|
244
|
+
framing_lines.each { |line| @output.puts " #{line}" }
|
|
218
245
|
|
|
219
246
|
groups.each do |group|
|
|
220
247
|
@output.puts ""
|