@setsquare/review-sdk 0.0.1 → 0.1.1

package/NOTICE

@@ -1,5 +1,5 @@
 Setsquare
-Copyright (c) 2026 Setsquare
+Copyright (c) 2026 WORKTREE AMIGOS LTD (company no. 17430189)
 
 This product includes software developed by the Setsquare project
 (https://setsquare.dev).
@@ -39,7 +39,8 @@ the same thing. The accurate phrase is "Apache-2.0 licensed".
 Trademarks
 --------------------------------------------------------------------------------
 
-"Setsquare" and the Setsquare logo are trademarks of the Setsquare project. No
+"Setsquare" and the Setsquare logo are unregistered trademarks of WORKTREE
+AMIGOS LTD. No trademark registration is sought or held (decision D1). No
 trademark licence is granted by the Apache License — section 6 reserves this
 expressly — and none is granted by this notice. You may state accurately that
 your software uses or is compatible with Setsquare; you may not use the name or

package/README.md

@@ -62,6 +62,14 @@ export default withSetsquare(defineConfig({/* your config, untouched */}));
 
 Then add `.setsquare/` to your `.gitignore`.
 
+**One install brings the `setsquare` command with it**, which is why there is no
+second install line above. `@setsquare/review-cli` arrives as a dependency and
+owns every command's implementation; the SDK owns the *name*. If you look, you
+will find both packages declaring a `setsquare` binary, and that is deliberate
+rather than a collision — pnpm links binaries only for **direct** dependencies,
+so an SDK-only install would otherwise leave you with no `setsquare` at all.
+Both resolve to the same code.
+
 ## What you get
 
 Every settled page in your suite is scanned against **WCAG 2.2 AA**, and the run
@@ -85,6 +93,141 @@ and an evidence file at `.setsquare/runs/<timestamp>/run.json` — the full find
 their WCAG criteria, measured contrast ratios, the exact ruleset used, and
 performance timings.
 
+### Checks that need the page driven, not read
+
+Some of what runs is not axe. A DOM-snapshot analyser can only answer questions
+about the document as it stands; several WCAG criteria are about what the page
+*does*. These run because we are already inside your test, with your page:
+
+| Criterion | What it does |
+|---|---|
+| **1.4.8** Visual Presentation, AAA | Measures line spacing, the measure in characters, and justification, on blocks of text. |
+| **1.4.12** Text Spacing | Applies the SC's four spacing metrics and measures what clips. |
+| **2.1.2** No Keyboard Trap | Tabs both ways and looks for a cycle focus never leaves. |
+| **2.1.4** Character Key Shortcuts, opt-in | Watches which key handlers a page registers, and on which characters. |
+| **2.4.7** Focus Visible | Diffs the computed style before and after focus — including `::before`/`::after`. |
+| **2.4.11** Focus Not Obscured (Minimum) | Focuses each element and checks nothing covers it entirely. |
+| **2.4.12** Focus Not Obscured (Enhanced), AAA | The same walk, stricter: no part of it covered. |
+| **2.4.13** Focus Appearance, AAA | Measures the indicator's thickness and its contrast, and states both. |
+| **3.1.5** Reading Level, AAA | A Flesch–Kincaid grade per page. |
+| **3.2.1** On Focus | Focuses each control and watches for a change of context nobody asked for. |
+| **3.2.2** On Input, opt-in | Changes each `<select>`, checkbox and radio and watches the same. |
+| **3.2.3** Consistent Navigation | Compares navigation order **across surfaces**, after the run. |
+| **3.2.4** Consistent Identification | Compares the names given to the same target across surfaces. |
+| **3.2.6** Consistent Help | Compares where help mechanisms appear across surfaces. |
+| **2.4.5** Multiple Ways | Looks across surfaces for navigation, a search control and a sitemap. |
+| **3.3.1 / 4.1.3** Error Identification, Status Messages | Reads a page your test put into an error state. |
+
+**If you use Tailwind and 1.4.8 fires across most of your prose, it is one line
+of theme configuration rather than anything you wrote.** Tailwind's type scale
+ships line heights below the 1.5 the SC asks for — `text-sm` is `20px` on a
+`14px` font, which is 1.43 — and line height **inherits**, so one utility on one
+container hands it to every paragraph inside, whatever size those paragraphs
+are. One utility, dozens of findings. The fix is in `@theme`, once:
+
+```css
+@theme inline {
+  --text-xs--line-height: 1.55;
+  --text-sm--line-height: 1.55;
+  --text-base--line-height: 1.55;
+  --text-lg--line-height: 1.55;
+  --text-xl--line-height: 1.55;
+}
+```
+
+That raises the **default** and leaves `leading-tight` and friends working, so
+headings and single-line labels keep the leading they are meant to have — which
+matters, because 1.4.8 is about *blocks* of text and WCAG defines a block as
+more than one sentence. 1.55 rather than 1.5 so a clamped font size cannot land
+a block a hair under the bar. Worth checking before editing anything, because a
+run of these usually has one cause rather than one per block.
+
+The three walks that drive the page — focus, keyboard and reflow — run at
+**teardown only**, and that is a real limitation rather than an implementation
+detail. Focusing an element fires your page's own focus, focusin and blur
+handlers; the keyboard walk presses Tab and Escape, which fires your key
+handlers too; the reflow check resizes the viewport, which re-runs your media
+queries. A handler that opened a menu or started a request has already run by
+the time anything is restored.
+
+Scroll position, the previously focused element and the viewport are all put
+back; side effects cannot be. So a scan of a named state
+(`{ state: 'menu-open' }`) carries no evidence from these checks, while
+whatever your test left on screen does.
+
+And twenty criteria are reported as **no automated rule can decide this** —
+separately from the criteria nobody has built a rule for yet, because those are
+different facts and an auditor's first question about an untested criterion is
+which of the two it is. Each one carries the reason and what would have to
+change for it to move, so the list is a position rather than a wall. Where
+**presence** is checkable and **adequacy** is not — a caption track exists, but
+whether it matches the audio is a judgement — the entry says so at that
+granularity rather than writing the criterion off whole.
+
+That list is the concrete floor under "automated evidence is not conformance".
+A run that tells you twenty criteria no tool will ever check for you has told
+you something more useful than a green tick.
+
+Alongside those, eight **heuristics** point at criteria no rule can decide —
+meaningful sequence (1.3.2), sensory instructions (1.3.3), weak headings
+(2.4.6), unheaded runs (2.4.10), abbreviations (3.1.4), location cues (2.4.8),
+missing caption tracks (1.2.1/1.2.2), and more than one route to a page
+(2.4.5). They follow one discipline: **never a violation, always needs a
+person, fire only above a margin, and say what would resolve it.** A heuristic
+that reports a failure is worse than no heuristic, because someone who
+disproves one stops trusting all of them.
+
+They are on by default, being read-only and unable to fail a build, and each is
+switchable on its own — `heuristics: { sensoryCharacteristics: false }` — so
+disagreeing with one costs you one rather than all eight. The three that read
+English words stay silent on a page in another language rather than guessing.
+
+The last three below are the ones no per-page tool can have. They are statements
+about a **set** of pages, computed after the whole run from what each scan
+already carried — so they cost no extra page work, and they exist because the
+run holds every surface at once.
+
+They are also the ones with the most dangerous failure mode, so: **a run with
+fewer than two surfaces reports them as untested, never as passing.** A
+one-page run cannot evidence consistency, and a clean pass on a run that had
+nothing to compare would be the most flattering possible lie. The summary says
+"these criteria compare pages against each other — this run had one surface"
+rather than folding them in with the criteria nothing can check.
+
+Only links in navigation, header and footer landmarks are compared, because
+3.2.4 is about repeated components: naming an inline prose link differently in
+two sentences is ordinary writing, not an inconsistency.
+
+**2.1.4 is off unless you turn it on** (`keyShortcuts: true`), and for a
+different reason again. There is no way to enumerate a page's event listeners
+after the fact, so the only route is to wrap `addEventListener` before your own
+scripts run, for the whole life of the page. Everything else here happens after
+a page has loaded and can be undone. Even proven inert — the wrapper records
+and delegates with your original arguments, so `removeEventListener` still
+matches, and the fixture suite asserts exactly that — it is a larger presence in
+your application than a scanner has any business having by default.
+
+It only ever reports **needs a person**: 2.1.4 is satisfied if the shortcut can
+be turned off, remapped, or is only active when a component has focus, and all
+three are affordances no check can see.
+
+**3.2.2 is off unless you turn it on** (`onInput: true`), and the default is the
+decision rather than caution: a change event is where an application saves, and
+writing to your data is a worse failure than any finding the rule could produce
+— worse than being a flake, because a flake is visible and a silent write is
+not. Turn it on where the suite runs against disposable data, which is most
+suites. Even then, only `<select>`, checkboxes and radios are touched and their
+values are put back; nothing is ever typed into a text field, no form is ever
+submitted, and no navigation is ever allowed to commit.
+
+Where geometry or stacking makes the answer a judgement — a translucent overlay
+over the focused element, an indicator drawn with a box-shadow, a focus ring
+against a gradient — these report **needs a person**, never a pass and never a
+fail. So does an element whose focus indicator we could not observe: many sites
+style `:focus-visible` only, and a programmatic focus does not always match it,
+so "nothing changed" is reported as something to check rather than as a
+failure.
+
 ## Every screenshot on one page
 
 Playwright takes the screenshots; reading them means opening its report and
@@ -151,6 +294,106 @@ test('the editor reports validation errors accessibly', async ({ page }) => {
 
 An explicit scan always outranks an inferred one for the same context.
 
+## Scanning at more than one viewport
+
+A rule only ever sees the viewport it ran in. A navigation that collapses into a
+menu below a breakpoint is a *different document*, not a narrower one, so a scan
+at 1280 has no opinion about the same page at 390.
+
+Setsquare cannot widen that for you — it scans the pages your suite drives, at
+the sizes your suite drives them, and it takes no screenshots at all. A project
+per width does, and it needs no changes to your tests:
+
+```ts
+export default withSetsquare(
+  defineConfig({
+    projects: [
+      { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
+      {
+        name: 'tablet',
+        testMatch: /a11y\.spec\.ts/,
+        use: { ...devices['Desktop Chrome'], viewport: { width: 768, height: 1024 } },
+      },
+      {
+        name: 'mobile',
+        testMatch: /a11y\.spec\.ts/,
+        use: { ...devices['Desktop Chrome'], viewport: { width: 390, height: 844 } },
+      },
+    ],
+  }),
+);
+```
+
+Spreading `Desktop Chrome` and overriding only `viewport` holds the engine
+constant, so the width is the only variable — `devices['iPhone 15']` is WebKit
+and changes both at once. `testMatch` keeps the extra projects to the specs whose
+layout is the subject; a spec that sets its own viewport, or that checks request
+origins or resolves links, returns the same answer at every width. And 320 needs
+no project of its own: `setsquare-reflow` measures WCAG 1.4.10 there on every
+surface already, by resizing at teardown and putting the viewport back.
+
+The viewport is part of a scan's identity, so evidence from several widths
+survives instead of collapsing into one row, and the report, the contact sheet
+and the per-surface tables group by band — laptop 1024px and wider, tablet 768px
+to 1023px, mobile narrower than 768px — with the exact size beside every name.
+Those names describe a width and never a device: a 640px viewport is in the
+mobile band whether it is a phone or a desktop browser with text at 200%, and
+every artifact that prints a band name prints that caveat with it.
+
+With one viewport in the run, Setsquare says nothing about narrow widths. Not
+"no narrow-viewport failures found" — that would be a claim about your phone
+layout from a run that never looked at it.
+
+## Telling the report what your own tests already cover
+
+Some success criteria no automated engine evaluates. Reflow at 320px, focus not
+being obscured by a sticky header, a keyboard trap, whether the navigation is
+consistent between pages — these are checked by driving the page, and a suite
+that has grown up around a real product is usually checking several of them
+already. Until you say so, the conformance report has no way to know: your spec
+arrives as a pass or a fail with no criterion attached to it.
+
+So say so:
+
+```ts
+import { setsquare } from '@setsquare/review-sdk';
+
+test('no horizontal scrolling at 320px', async ({ page }) => {
+  setsquare.covers('1.4.10');
+
+  await page.setViewportSize({ width: 320, height: 800 });
+  await page.goto('/pricing');
+  expect(await hasHorizontalOverflow(page)).toBe(false);
+});
+```
+
+That criterion now appears in the report as **covered by your own test**, with
+your test named next to it.
+
+**It is recorded as your claim, not as our finding, and the report says so
+everywhere it appears.** We do not check that a test declaring `1.4.10` tests
+reflow — there is no way to — so a declaration is never merged into the state
+that means every automated rule passed. It is its own state, worded
+differently, with its own glyph in the grid. That is the point: the declaration
+is exactly as strong as the test behind it, and the report never pretends
+otherwise.
+
+Three details worth knowing:
+
+- **A declaring test that fails** is reported as *your own test for this
+  criterion did not pass*, and it stops the level reading as clean. It does not
+  become a detected failure: we did not observe it, and a red test can be red
+  for a reason that has nothing to do with the criterion.
+- **A declaring test that is skipped** is reported as such, rather than folded
+  into "no automated check exists" — a check that exists and did not run is a
+  different problem.
+- **An id that is not a WCAG 2.2 success criterion throws**, immediately, in
+  your test. This is the one thing in this package that does: everywhere else a
+  problem warns and carries on, because our evidence-gathering has no standing
+  to fail your build. A typo here is different — the alternative is dropping
+  your declaration silently, which would leave you believing you have coverage
+  you do not have.
+
 ## Configuration
 
 Everything is optional. An empty options object produces a useful run.
@@ -164,17 +407,58 @@ export default withSetsquare(defineConfig({/* … */}), {
   surfaces: { '/jobs/:id/edit': 'job-editor' },
   mask: ['.candidate-name'],
   outputDir: '.setsquare',
+  quietMs: 500, // how still counts as settled
+  settleTimeoutMs: 5000, // how long we wait for it
   promo: false,
 });
 ```
 
+### When a page gets scanned, and the two knobs that decide it
+
+Scans fire automatically when a page **settles**: `document.readyState` is
+`complete`, and neither the URL nor the DOM has changed for `quietMs`. Any
+mutation restarts that window, which is what makes the scanner wait for late
+content instead of auditing a skeleton.
+
+- **`quietMs`** (default `500`) — how long a page must be still. Shorter is not
+  better: measured against a route that renders a conformant shell and injects
+  its real content a second later, 250ms lands *inside* that gap and reports a
+  clean page. 500ms does not.
+- **`settleTimeoutMs`** (default `5000`) — how long we wait for quiet before
+  scanning anyway. **It is a trigger, not a give-up.** A page that never goes
+  still — a poller, a carousel, a progress animation — is extremely common, and
+  declining to scan it would produce exactly the "no findings" that reads as "no
+  problems".
+
+**What happens to a page that settles late.** If it is still on screen when the
+test ends, it is scanned — at the ceiling, and again at teardown if it changed
+in between, so the evidence describes the finished page. What cannot be
+recovered is a surface the test **navigated away from** while it was still
+rendering: it is gone before anything can look at it, and the run reports it as
+visited-but-never-scanned rather than quietly dropping it. Raising
+`settleTimeoutMs` does not help there — the limit is the test's pace, not ours.
+
+**The report tells you which problem you have.** A run with scans cut short
+names the longest wait and the number of DOM mutations still arriving. A page a
+few mutations from quiet will settle if you give it longer; a page with
+thousands never will, and no value of `settleTimeoutMs` changes that.
+
 **Why AA by default.** It is what conformance means in practice, and what
 EN 301 549 and procurement ask for. AAA is a deliberate commitment — WCAG itself
 does not recommend it as a general policy — so opting in is the honest default.
 
-**Themes** are read from `data-theme` on the live page when your suite toggles
-them in-test, so a suite whose project names carry no theme still gets one context
-per theme without configuration.
+**`personas`, `themes` and `surfaces` map your names to ours, and the direction
+is the same in all three: the key is what is in your repository, the value is
+the label you want in the report.** So `personas: { 'a11y-admin': 'recruiter' }`
+means the Playwright **project** named `a11y-admin` is reported as the persona
+`recruiter`; `themes: { 'dark-suite': 'dark' }` reads a project name and reports
+a theme; and `surfaces` maps a URL pattern to a surface name. Keys are Playwright
+project names in the first two and URL patterns in the third — nothing here maps
+the other way round.
+
+**Themes** are also read from `data-theme` on the live page when your suite
+toggles them in-test, so a suite whose project names carry no theme still gets
+one context per theme without configuring anything.
 
 ## One logical run across several Playwright invocations
 

package/dist/autoscan.d.ts

@@ -1,6 +1,28 @@
-import type { BrowserContext, TestInfo } from '@playwright/test';
+import type { BrowserContext, Page, TestInfo } from '@playwright/test';
+import type { AxeFinding, RuleInapplicable, RulePass } from '@setsquare/schema';
 import { type ResolvedSetsquareOptions } from './options.js';
 export declare function installAutoScan(context: BrowserContext, options: ResolvedSetsquareOptions, testInfo: TestInfo, warn: (message: string) => void): Promise<void>;
+/**
+ * Setsquare's own driven checks: the ones that need the page acted on rather
+ * than read.
+ *
+ * Extracted so the teardown scan and the checks-only teardown record run
+ * exactly the same set. Two call sites with two hand-maintained lists is how a
+ * check ends up silently missing from half the runs.
+ */
+export declare function runDrivenChecks(page: Page, options: ResolvedSetsquareOptions, 
+/**
+ * Had this page met the quiet definition when the checks ran?
+ *
+ * Defaults to true, so an explicit `setsquare.scan()` suppresses nothing: the
+ * customer chose the moment, and second-guessing their choice would be us
+ * deciding their page was not ready when they said it was.
+ */
+pageWentQuiet?: boolean): Promise<{
+    findings: AxeFinding[];
+    passes: RulePass[];
+    inapplicable: RuleInapplicable[];
+}>;
 /**
  * Scan whatever the test left on screen, at teardown.
  *

package/dist/autoscan.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"autoscan.d.ts","sourceRoot":"","sources":["../src/autoscan.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,cAAc,EAAQ,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAavE,OAAO,EAAqB,KAAK,wBAAwB,EAAE,MAAM,cAAc,CAAC;AA+DhF,wBAAgB,eAAe,CAC7B,OAAO,EAAE,cAAc,EACvB,OAAO,EAAE,wBAAwB,EACjC,QAAQ,EAAE,QAAQ,EAClB,IAAI,EAAE,CAAC,OAAO,EAAE,MAAM,KAAK,IAAI,GAC9B,OAAO,CAAC,IAAI,CAAC,CAqEf;AA6LD;;;;;;;;;;;;;;;;;;;GAmBG;AACH,wBAAsB,aAAa,CACjC,OAAO,EAAE,cAAc,EACvB,OAAO,EAAE,wBAAwB,EACjC,QAAQ,EAAE,QAAQ,EAClB,IAAI,EAAE,CAAC,OAAO,EAAE,MAAM,KAAK,IAAI,GAC9B,OAAO,CAAC,IAAI,CAAC,CA0Df"}
+{"version":3,"file":"autoscan.d.ts","sourceRoot":"","sources":["../src/autoscan.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,cAAc,EAAE,IAAI,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AACvE,OAAO,KAAK,EAAE,UAAU,EAAE,gBAAgB,EAAE,QAAQ,EAAE,MAAM,mBAAmB,CAAC;AAoBhF,OAAO,EAAqB,KAAK,wBAAwB,EAAE,MAAM,cAAc,CAAC;AAmGhF,wBAAgB,eAAe,CAC7B,OAAO,EAAE,cAAc,EACvB,OAAO,EAAE,wBAAwB,EACjC,QAAQ,EAAE,QAAQ,EAClB,IAAI,EAAE,CAAC,OAAO,EAAE,MAAM,KAAK,IAAI,GAC9B,OAAO,CAAC,IAAI,CAAC,CAqFf;AAsMD;;;;;;;GAOG;AACH,wBAAsB,eAAe,CACnC,IAAI,EAAE,IAAI,EACV,OAAO,EAAE,wBAAwB;AACjC;;;;;;GAMG;AACH,aAAa,UAAO,GACnB,OAAO,CAAC;IAAE,QAAQ,EAAE,UAAU,EAAE,CAAC;IAAC,MAAM,EAAE,QAAQ,EAAE,CAAC;IAAC,YAAY,EAAE,gBAAgB,EAAE,CAAA;CAAE,CAAC,CAuC3F;AAuGD;;;;;;;;;;;;;;;;;;;GAmBG;AACH,wBAAsB,aAAa,CACjC,OAAO,EAAE,cAAc,EACvB,OAAO,EAAE,wBAAwB,EACjC,QAAQ,EAAE,QAAQ,EAClB,IAAI,EAAE,CAAC,OAAO,EAAE,MAAM,KAAK,IAAI,GAC9B,OAAO,CAAC,IAAI,CAAC,CAiGf"}