reflection-check 0.0.1

package/docs/target-ir-and-adapters.md

@@ -0,0 +1,111 @@
+# Target IR and adapters
+
+Reflection keeps external input formats separate from the contract runners by compiling them into a small Target IR: a normalized inventory of things Reflection can validate.
+
+```text
+Reflection config      optional adapters
+       │                    │
+       └──────────┬─────────┘
+                  ▼
+              Target IR
+                  ▼
+          contract runners / reports
+```
+
+## Why Target IR exists
+
+Target IR is the seam between _where validation targets come from_ and _how Reflection validates them_. The current Reflection config is one source. Future integrations can compile their own manifests into the same shape without making browser, visual, component, or design runners depend on that integration.
+
+This gives us three useful constraints:
+
+- **Neutral core:** runner code should work from Reflection concepts, not external product names.
+- **Optional adapters:** adapters can be present or absent without changing normal `reflection run` behavior.
+- **Agent-readable inventory:** agents can inspect a single list of targets and understand the validation surface.
+
+## Current target families
+
+The IR currently supports four families:
+
+| Family | Source contract | Typical run modes | Purpose |
+| --- | --- | --- | --- |
+| `browser-route` | browser routes | `smoke`, `full` | Render a route and evaluate DOM/layout/console expectations. |
+| `route-visual` | browser visual smoke cases | `smoke`, `full` | Compare route screenshots against read-only baselines. |
+| `component-visual` | Storybook component cases | `visual`, `full` | Compare component story screenshots against read-only baselines. |
+| `design-command` | design command checks | `design`, `full` | Run project-owned design contract commands. |
+
+All targets include:
+
+- `id` — stable target id.
+- `family` — one of the target families above.
+- `source` — where the target came from, for example `reflection-config` or `adapter`.
+- `runModes` — modes where the target is relevant.
+- `blocking` — whether a failing target should block the run.
+
+## Reflection config compiler
+
+`compileReflectionTargets(config)` compiles the typed Reflection config into Target IR.
+
+Example:
+
+```ts
+const ir = compileReflectionTargets(config);
+
+console.log(ir.targets.map((target) => `${target.family}:${target.id}`));
+// [
+//   'browser-route:login',
+//   'route-visual:login-mobile',
+//   'component-visual:button-primary',
+//   'design-command:tokens'
+// ]
+```
+
+The compiler preserves visual metadata that matters to later review, including zero-valued thresholds. Do not use truthiness checks when copying optional numeric metadata; use explicit `!== undefined` checks so values like `0` survive.
+
+## Adapter contract
+
+Adapters should compile their input into the same target shape and mark targets with `source: 'adapter'`.
+
+Adapters must not make core runners aware of the source format. The runner should receive normalized route/story/visual/command information and should not branch on adapter names.
+
+A good adapter is:
+
+- **optional** — normal Reflection config works without it;
+- **validated** — malformed adapter input fails before runner execution;
+- **neutral** — no external product names leak into core commands or reports unless the user explicitly names their own target ids;
+- **lossless enough for review** — route paths, viewports, expectations, baselines, thresholds, and blocking semantics are preserved in IR.
+
+## Route manifest adapter proof
+
+Phase 6.2 adds a fixture JSON route-manifest adapter in `src/adapters/route-manifest.ts`. Its job is deliberately narrow: prove that an external manifest can become browser-route Target IR without changing the browser runner.
+
+The manifest shape is:
+
+```json
+{
+  "project": "example-app",
+  "baseUrl": "http://127.0.0.1:5173",
+  "routes": [
+    {
+      "id": "login",
+      "path": "/login",
+      "viewports": ["desktop", "mobile"],
+      "expects": [{ "role": "heading", "name": "Welcome" }],
+      "blocking": true
+    }
+  ]
+}
+```
+
+That adapter emits `browser-route` targets using the generic Target IR. It does not introduce a new runner or a new user-facing external product concept.
+
+Use the pure parser for already-loaded JSON:
+
+```ts
+const ir = parseRouteManifestTargets(routeManifestJson);
+```
+
+Use the loader when the manifest lives on disk:
+
+```ts
+const ir = await loadRouteManifestTargets('routes.json');
+```

package/docs/validation-process.md

@@ -0,0 +1,172 @@
+# Reflection validation process
+
+Reflection is meant to be a validation protocol that humans, agents, and CI can run without guessing what changed or whether visual evidence is safe to accept.
+
+The command split is intentional:
+
+```bash
+reflection doctor  # lightweight CLI/setup check; project contracts run through --config
+reflection run     # produce evidence and report.json; pass --config for project contracts
+reflection review  # summarize the latest evidence for humans/agents
+reflection update  # accept intentional visual changes; never automatic
+```
+
+## Local validation loop
+
+Use this loop before claiming frontend work is complete:
+
+```bash
+reflection doctor
+reflection run --config reflection.config.ts --mode smoke
+reflection review --json
+```
+
+If Reflection is being run from this repository during development, build first and use the local CLI:
+
+```bash
+pnpm build
+node dist/cli.js doctor
+node dist/cli.js run --config examples/basic-react/reflection.config.ts --mode smoke
+node dist/cli.js review --json
+```
+
+Interpret the review result as the contract:
+
+- `status: "pass"` means the validation evidence passed.
+- `status: "pass-with-review"` means blocking checks passed, but review items need a human-readable summary and artifact links.
+- `status: "fail"` means blocking checks failed; fix those before calling the work complete.
+
+Always include the `reportPath`, blocking failures, review items, and artifact paths when reporting results to a human.
+
+## Agent instructions
+
+Agents should treat Reflection as an evidence gate, not as a self-healing tool.
+
+Required agent behavior:
+
+1. Run `reflection doctor` before the validation flow when setup may be uncertain. It is currently a lightweight setup check; the configured project contract is exercised by `reflection run --config ...`.
+2. Run `reflection run --config reflection.config.ts --mode smoke` to generate current evidence.
+3. Run `reflection review --json` to get the machine-readable summary.
+4. Fix blocking failures before finishing the task.
+5. Summarize review items with artifact paths instead of hiding them.
+6. Never run non-dry `reflection update` without explicit human approval.
+7. Never run `reflection update` in CI.
+
+Agents may propose intentional baseline changes with a dry run:
+
+```bash
+reflection update --route <routeId> --from-run latest --dry-run
+reflection update --case <routeVisualCaseId> --from-run latest --dry-run
+```
+
+`reflection update` currently promotes route-level `visualSmoke` baselines. Component visual baselines still require manual review/copy until component baseline promotion is implemented.
+
+Only after explicit human approval may an agent run the matching non-dry update:
+
+```bash
+reflection update --route <routeId> --from-run latest
+# or
+reflection update --case <routeVisualCaseId> --from-run latest
+```
+
+After any non-dry update, inspect the resulting git diff and report exactly which baseline files changed.
+
+## CI validation loop
+
+CI should generate and publish evidence, but must never update baselines.
+
+Recommended CI command shape:
+
+```bash
+reflection doctor
+reflection run --ci --config reflection.config.ts --mode smoke
+reflection review --report-dir artifacts/reflection --json
+```
+
+`reflection run --ci` writes to `artifacts/reflection` by default. `reflection review` defaults to `.reflection`, so CI review commands must pass the CI report root explicitly.
+
+For this repository's built CLI:
+
+```bash
+pnpm install --frozen-lockfile
+pnpm build
+node dist/cli.js doctor
+node dist/cli.js run --ci --config examples/basic-react/reflection.config.ts --mode smoke
+node dist/cli.js review --report-dir artifacts/reflection --json
+```
+
+Upload Reflection run artifacts even on failure so humans and agents can inspect what happened:
+
+```text
+artifacts/reflection/**
+```
+
+For local non-CI runs, the default report root remains `.reflection`. If a project config or command uses `--report-dir`, upload and review that explicit report root instead.
+
+## Baseline update policy
+
+Reflection does not silently heal visual diffs.
+
+- Normal `reflection run` must not create or update baselines.
+- `reflection update --dry-run` is safe for agents to use when proposing a change.
+- Non-dry `reflection update` is a human-approved mutation step.
+- CI must treat any `reflection update` attempt as invalid.
+- Prefer targeted updates (`--route` or `--case`) over `--all`.
+- `--all` must be explicit and should be reserved for deliberate broad rebaselines.
+
+## Adding Reflection to repository agent files
+
+Reflection keeps its canonical operating instructions in this file. Repository-level agent files should carry only a short pointer section so agents discover the validation process without duplicating the whole protocol.
+
+Use this file-selection logic when adding Reflection to a repository:
+
+1. Look for existing agent instruction files:
+   - `AGENTS.md`
+   - `CLAUDE.md`
+   - `.github/copilot-instructions.md`
+   - `copilot-instructions.md`
+2. If one or more of those files already exist, add the Reflection validation section to the existing file or files and do not create another agent instruction file.
+3. If none of those files exist, create `AGENTS.md` and add the Reflection validation section there.
+4. Keep the canonical process in `docs/validation-process.md`; the agent instruction file should point here instead of copying this full guide.
+
+Suggested section:
+
+````markdown
+## Reflection validation
+
+Use Reflection as the UI evidence gate before claiming frontend work is complete.
+
+Run:
+
+```bash
+reflection doctor
+reflection run --config reflection.config.ts --mode smoke
+reflection review --json
+```
+
+Rules:
+
+- Treat blocking failures as task blockers.
+- Summarize review items with artifact paths for the human.
+- Use `reflection update --route <routeId> --from-run latest --dry-run` only to propose intentional visual changes.
+- Do not run non-dry `reflection update` unless the human explicitly approves it.
+- Never run `reflection update` in CI.
+
+Full protocol: `docs/validation-process.md`.
+````
+
+Do not duplicate the full protocol into repository-level agent files. Keep the detailed process here and add only the pointer section above to the selected existing agent file, or to a new `AGENTS.md` when no supported agent instruction file exists.
+
+## Minimum completion evidence
+
+When reporting a completed Reflection validation run, include:
+
+- command(s) run
+- pass/fail status
+- report path
+- blocking failures, if any
+- review items, if any
+- artifact paths relevant to changed or failing UI
+- whether any baseline update was dry-run only or human-approved non-dry
+
+A good final agent message should make the evidence easy to inspect without requiring the human to re-run commands immediately.

package/docs/visual-contract.md

@@ -0,0 +1,174 @@
+# Visual contract
+
+The visual contract compares current screenshots against approved baseline images. It is deliberately review-first: visual diffs are review-only by default, and baseline updates are explicit human-approved mutations.
+
+Reflection currently supports two visual surfaces:
+
+- route-level visual smoke cases from browser route screenshots;
+- Storybook component visual cases.
+
+## Evidence screenshots vs baselines
+
+Keep these concepts separate:
+
+| Concept | Location | Created by | Meaning |
+| --- | --- | --- | --- |
+| Evidence screenshot | `.reflection/runs/<run-id>/**` or configured report root | `reflection run` | What the current run rendered. Safe to regenerate. |
+| Baseline image | Configured `baselineRoot` + `baseline` | `reflection update` after review, or manual fixture setup | Approved reference image. Should be source-controlled when intentional. |
+| Diff image | `.reflection/runs/<run-id>/visual/**/diff.png` | `reflection run` during comparison | Highlight of pixels that differ between evidence and baseline. |
+
+Normal runs never mutate baselines.
+
+## Review-only default
+
+A visual mismatch or missing baseline is review-only unless the case opts into strict/blocking behavior.
+
+Use review-only defaults while adopting Reflection:
+
+```ts
+visualSmoke: [
+  {
+    id: 'home-mobile',
+    route: 'home',
+    viewport: 'mobile',
+    baselineRoot: 'tests/fixtures/baselines',
+    baseline: 'browser/home/mobile.chromium-linux.light.png'
+  }
+]
+```
+
+Promote a visual case to blocking only when the baseline is stable and the team wants CI/task completion to fail on visual drift:
+
+```ts
+{
+  id: 'home-mobile',
+  route: 'home',
+  viewport: 'mobile',
+  baseline: 'browser/home/mobile.chromium-linux.light.png',
+  strict: true
+}
+```
+
+`strict: true` or `blocking: true` makes a failing visual comparison blocking. Otherwise it produces `pass-with-review` so the artifact can be inspected without hiding the change.
+
+## Route-level visual smoke
+
+Route visual smoke cases reuse browser route screenshots. A case must point at a configured route id and viewport:
+
+```ts
+browser: {
+  routes: [
+    {
+      id: 'login',
+      path: '/login',
+      viewports: ['mobile'],
+      expects: [{ screenshot: 'final' }]
+    }
+  ],
+  visualSmoke: [
+    {
+      id: 'login-mobile',
+      route: 'login',
+      viewport: 'mobile',
+      baselineRoot: 'tests/fixtures/baselines',
+      baseline: 'browser/login/mobile.chromium-linux.light.png',
+      threshold: { maxDiffPixelRatio: 0.01 }
+    }
+  ]
+}
+```
+
+If the matching browser route result is missing, the visual check becomes a review item by default or a blocking failure when strict/blocking is enabled.
+
+## Component visual baselines
+
+Component visual cases use Storybook. Reflection resolves the configured `storyId` through Storybook `/index.json`, opens the iframe URL, captures a screenshot, and compares it with the configured baseline.
+
+```ts
+component: {
+  storybook: {
+    command: 'pnpm storybook --host 127.0.0.1 --port 6006',
+    readyUrl: 'http://127.0.0.1:6006',
+    reuseExisting: true,
+    timeoutMs: 60_000
+  },
+  cases: [
+    {
+      id: 'button-primary',
+      storyId: 'atoms-button--primary',
+      viewport: 'component',
+      baselineRoot: 'tests/fixtures/baselines',
+      baseline: 'components/button/primary.chromium-linux.light.png',
+      threshold: { maxDiffPixelRatio: 0.005 }
+    }
+  ]
+}
+```
+
+### Pseudo states
+
+Prefer story-controlled states. A story named `Button/Hover` or `Button/Focused` is usually more deterministic than moving the mouse in the browser.
+
+When the browser must force a state, Reflection requires effective animation stabilization:
+
+```ts
+{
+  id: 'button-hover',
+  storyId: 'atoms-button--primary',
+  baseline: 'components/button/hover.chromium-linux.light.png',
+  stateNote: 'Browser-forced hover until a story-controlled hover variant exists.',
+  browserState: {
+    kind: 'hover',
+    selector: 'button',
+    animationStabilization: {
+      disableAnimations: true
+    }
+  }
+}
+```
+
+Reports include `statePolicy` metadata:
+
+- `story-controlled` when no `browserState` is configured;
+- `browser-forced-with-stabilization` when Reflection applies hover/focus in the browser.
+
+## Thresholds
+
+Visual thresholds support:
+
+```ts
+threshold: {
+  maxDiffPixels: 25,
+  maxDiffPixelRatio: 0.01
+}
+```
+
+`maxDiffPixelRatio: 0` is valid and means exact pixel matching. Reflection preserves zero-valued thresholds intentionally.
+
+## Updating baselines
+
+Always inspect artifacts first:
+
+```bash
+reflection review --json
+```
+
+Then dry-run a targeted update:
+
+```bash
+reflection update --route login --from-run latest --dry-run
+reflection update --case login-mobile --from-run latest --dry-run
+```
+
+Only after explicit human approval:
+
+```bash
+reflection update --route login --from-run latest
+reflection update --case login-mobile --from-run latest
+```
+
+`reflection update` currently promotes route-level `visualSmoke` baselines. For component visual cases, inspect the run artifacts and copy an approved `actual.png` to the configured component baseline path manually until targeted component baseline promotion is implemented.
+
+After a non-dry update, inspect the git diff and report exactly which baseline files changed.
+
+CI must never update baselines. `reflection update --ci` is refused.

package/package.json

@@ -0,0 +1,62 @@
+{
+  "name": "reflection-check",
+  "version": "0.0.1",
+  "description": "CLI for evidence-backed rendered UI validation.",
+  "license": "MIT",
+  "type": "module",
+  "main": "./dist/index.js",
+  "types": "./dist/index.d.ts",
+  "exports": {
+    ".": {
+      "import": "./dist/index.js",
+      "types": "./dist/index.d.ts"
+    }
+  },
+  "bin": {
+    "reflection": "dist/cli.js",
+    "reflection-check": "dist/cli.js"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "files": [
+    "dist",
+    "docs",
+    "LICENSE",
+    "README.md"
+  ],
+  "keywords": [
+    "ui-validation",
+    "playwright",
+    "visual-regression",
+    "design-systems",
+    "developer-tools"
+  ],
+  "scripts": {
+    "build": "tsc -p tsconfig.json",
+    "check": "pnpm typecheck && pnpm test && pnpm build",
+    "prepack": "pnpm build",
+    "smoke:package": "node scripts/smoke-package-install.mjs",
+    "typecheck": "tsc -p tsconfig.json --noEmit",
+    "test": "vitest run",
+    "reflection": "pnpm build && node dist/cli.js"
+  },
+  "dependencies": {
+    "commander": "^14.0.3",
+    "jiti": "^2.7.0",
+    "pixelmatch": "^7.2.0",
+    "pngjs": "^7.0.0",
+    "playwright": "^1.60.0",
+    "zod": "^4.4.3"
+  },
+  "devDependencies": {
+    "@types/node": "^25.7.0",
+    "@types/pngjs": "^6.0.5",
+    "typescript": "^5.6.3",
+    "vitest": "^2.1.9"
+  },
+  "engines": {
+    "node": ">=22"
+  },
+  "packageManager": "pnpm@10.12.1"
+}