reflection-check 0.0.1

package/docs/artifacts-and-gc.md

@@ -0,0 +1,125 @@
+# Artifacts and garbage collection
+
+Reflection writes every run as a reviewable artifact bundle. The bundle is designed for humans, agents, and CI systems to inspect after success or failure.
+
+## Default artifact roots
+
+| Context | Default root |
+| --- | --- |
+| Local run | `.reflection` |
+| CI run with `--ci` | `artifacts/reflection` |
+| Explicit override | `--report-dir <path>` |
+
+A run is written under:
+
+```text
+<report-root>/runs/<run-id>/
+```
+
+The latest run pointer is:
+
+```text
+<report-root>/runs/latest
+```
+
+## Run bundle layout
+
+Typical files:
+
+```text
+.reflection/runs/<run-id>/
+  report.md
+  report.json
+  manifest.json
+  browser/
+    <route-id>/
+      <viewport>/
+        actual.png
+        metadata.json
+  visual/
+    <case-id>/
+      expected.png
+      actual.png
+      diff.png
+  server/
+    app.log
+    storybook.log
+```
+
+Not every run contains every folder. For example, a browser-only run may have screenshots but no component visual artifacts.
+
+## `report.json`
+
+`report.json` is the stable machine-readable summary for agents and CI. It includes:
+
+- `runId`, `project`, mode, CI flag, and environment metadata;
+- overall status: `pass`, `pass-with-review`, `fail`, or `error`;
+- structured checks with `id`, `suite`, `target`, `status`, `severity`, artifacts, metadata, and suggested next steps;
+- top-level artifact references.
+
+Use `reflection review --json` rather than hand-parsing reports when an agent needs a compact summary of latest evidence.
+
+## `report.md`
+
+`report.md` is the human-readable report. Link this in PRs or task summaries when the recipient wants to inspect the run quickly.
+
+## `manifest.json`
+
+`manifest.json` records the report files currently tracked for run retention plus whether the run is pinned.
+
+Garbage collection uses this manifest to decide whether a run directory is eligible for deletion. Runs with missing or malformed manifests are skipped rather than deleted. Runtime evidence such as browser screenshots, visual artifacts, and logs still belongs to the run directory even though the current manifest only enumerates report files.
+
+## Evidence artifacts
+
+Evidence artifacts are run-scoped and safe to regenerate:
+
+- screenshots from browser and component checks;
+- visual `expected`, `actual`, and `diff` images copied or generated for the run;
+- server logs;
+- reports and metadata.
+
+Evidence artifacts are not approved baselines. Approved baselines live outside run directories in the configured baseline root.
+
+## Garbage collection
+
+Use GC to clean old run artifacts without touching baselines:
+
+```bash
+reflection gc --dry-run
+reflection gc --delete
+```
+
+With a custom report root:
+
+```bash
+reflection gc --report-dir artifacts/reflection --dry-run
+```
+
+Safety behavior:
+
+- GC only operates under `<report-root>/runs`.
+- `latest` is never treated as a run directory.
+- Symlinked `runs` directories are refused.
+- Symlinked run directories are skipped.
+- Runs with missing, invalid, mismatched, or pinned manifests are skipped.
+- Baseline roots are not part of GC.
+
+Use `--dry-run` first in local development and CI diagnostics. Use `--delete` only when the listed eligible run directories are safe to remove.
+
+## What to commit
+
+Usually commit:
+
+- `reflection.config.ts`;
+- baseline images after explicit approval;
+- docs or agent pointer sections;
+- workflow files that run Reflection.
+
+Usually do not commit:
+
+- `.reflection/runs/**`;
+- `artifacts/reflection/runs/**`;
+- server logs;
+- transient screenshots and visual diffs.
+
+CI should upload run artifacts even when validation fails so humans and agents can inspect evidence.

package/docs/browser-contract.md

@@ -0,0 +1,98 @@
+# Browser contract
+
+The browser contract validates rendered routes in a real browser. It answers: can the app render the expected route, at the expected viewport, without obvious layout or console regressions?
+
+Browser route checks run in `smoke` and `full` modes.
+
+## What a route check does
+
+For each configured route and viewport, Reflection:
+
+1. Starts or reuses the configured app server.
+2. Opens `baseUrl + route.path` in Playwright.
+3. Applies configured screenshot masking selectors.
+4. Evaluates route expectations.
+5. Captures screenshot evidence when requested.
+6. Emits a structured `CheckResult` into `report.json`.
+
+A route check is browser evidence, not a baseline approval. A screenshot captured by `{ screenshot: 'final' }` is evidence for the current run. It becomes a visual comparison input only when referenced by a `visualSmoke` case.
+
+## Config example
+
+```ts
+browser: {
+  enabled: true,
+  blocking: true,
+  baseUrl: 'http://127.0.0.1:5173',
+  server: {
+    command: 'pnpm dev --host 127.0.0.1',
+    readyUrl: 'http://127.0.0.1:5173',
+    reuseExisting: true,
+    timeoutMs: 60_000
+  },
+  maskSelectors: ['[data-reflection-mask]'],
+  routes: [
+    {
+      id: 'login',
+      path: '/login',
+      viewports: ['desktop', 'mobile'],
+      expects: [
+        { role: 'heading', name: 'Login' },
+        { label: 'Email' },
+        { label: 'Password' },
+        { role: 'button', name: 'Sign in' },
+        { noText: 'Register' },
+        { noHorizontalOverflow: true },
+        { noConsoleErrors: true },
+        { screenshot: 'final' }
+      ]
+    }
+  ]
+}
+```
+
+## Expectations
+
+| Expectation | Use for |
+| --- | --- |
+| `urlIncludes` | Route redirects or canonical URL fragments. |
+| `urlEquals` | Exact final URL checks. |
+| `role` + optional `name` | Accessible UI assertions; preferred for user-visible structure. |
+| `label` | Form field labels. |
+| `text` | Required visible text. |
+| `noText` | Text that must not appear, such as old copy or errors. |
+| `selector` | A CSS selector that must exist. |
+| `elementVisible` | A CSS selector that must be visible. |
+| `elementNotVisible` | A CSS selector that must not be visible. |
+| `noHorizontalOverflow` | Mobile/layout guard against body-level horizontal overflow. |
+| `noConsoleErrors` | Fails on browser console errors captured during the route visit. |
+| `screenshot` | Captures current route screenshot evidence. |
+
+Prefer accessible expectations (`role`, `label`, visible text) before selectors. Selectors are useful for stable app-specific contracts, but they should not become a substitute for testing what a user can perceive.
+
+## Viewports
+
+Routes can list one or more named viewports:
+
+```ts
+viewports: ['desktop', 'mobile']
+```
+
+Reflection stores viewport metadata in each route check. Route-level visual smoke cases match against this metadata, so the `route` and `viewport` fields in a `visualSmoke` case must correspond to an actual browser route result.
+
+## Console and layout failures
+
+`noConsoleErrors` and `noHorizontalOverflow` are intentionally simple high-signal checks:
+
+- `noConsoleErrors` catches runtime exceptions and error logs that can otherwise hide behind a passing screenshot.
+- `noHorizontalOverflow` catches common responsive regressions before visual review.
+
+These should usually be included on smoke routes.
+
+## Screenshots are evidence
+
+Browser screenshots are run artifacts under `.reflection/runs/<run-id>/browser/<route-id>/<viewport>/actual.png`, with route metadata beside them. They answer "what did this run render?"
+
+They are not approved baselines. Approved baselines are separate files configured by `visualSmoke` or component visual cases. Normal `reflection run` does not create or mutate baselines.
+
+See `docs/visual-contract.md` for visual comparison behavior.

package/docs/ci.md

@@ -0,0 +1,44 @@
+# Reflection in CI
+
+Reflection is designed to be safe in CI: runs create evidence artifacts, but they do not mutate visual baselines.
+
+## Recommended command
+
+```bash
+reflection run --ci --config reflection.config.ts --mode smoke
+```
+
+When `--ci` is enabled and `--report-dir` is not provided, Reflection writes artifacts to:
+
+```text
+artifacts/reflection
+```
+
+The generated report is written under:
+
+```text
+artifacts/reflection/runs/<run-id>/report.md
+artifacts/reflection/runs/<run-id>/report.json
+artifacts/reflection/runs/latest
+```
+
+## CI defaults
+
+- `reflection run --ci` records the environment profile as `ci`.
+- Workers default to `1` for stable browser and visual evidence.
+- Videos and traces remain off by default unless a future explicit policy enables them.
+- CI never updates baselines. Baseline promotion must use `reflection update` outside CI with an explicit human review step; `reflection update --ci` is refused.
+
+## Exit codes
+
+| Exit code | Meaning |
+| --- | --- |
+| `0` | Success, including `pass` and `pass-with-review` reports. Review items can be uploaded as artifacts without failing CI. |
+| `1` | Blocking validation failure. A blocking check failed. |
+| `2` | Tool/configuration error. Reflection could not complete because setup, config, dependency, or runtime execution failed. |
+| `64` | Invalid CLI usage, such as an unsupported run mode or invalid option value. |
+| `69` | Missing dependency. Reserved for dependency checks such as browser/runtime availability. |
+
+## GitHub Actions example
+
+See `.github/workflows/reflection.yml` for a repo-owned example that installs dependencies, builds Reflection, runs `reflection run --ci`, and uploads `artifacts/reflection` for review.

package/docs/configuration.md

@@ -0,0 +1,210 @@
+# Reflection configuration
+
+Reflection reads a project config from the path passed to `reflection run --config <path>`. Use `reflection.config.ts` as the conventional root filename. The config is validated before runners execute, so malformed contracts fail as setup errors instead of producing ambiguous reports.
+
+The loader supports TypeScript config files (`.ts`, `.mts`, `.cts`) through `jiti` and JavaScript ESM config files through dynamic import.
+
+## Minimal shape
+
+```ts
+import { defineReflection } from 'reflection-check';
+
+export default defineReflection({
+  project: 'my-app',
+  contracts: {
+    browser: {
+      baseUrl: 'http://127.0.0.1:5173',
+      routes: []
+    }
+  }
+});
+```
+
+During Reflection repository development, examples import `defineReflection` from source instead:
+
+```ts
+import { defineReflection } from '../../src/core/define-reflection';
+```
+
+## Top-level fields
+
+| Field | Required | Notes |
+| --- | --- | --- |
+| `project` | yes | Stable project name written into reports. |
+| `contracts.browser` | no | Browser route and route-level visual smoke checks. |
+| `contracts.design` | no | Project-owned command checks. |
+| `contracts.component` | no | Storybook-backed component visual checks. |
+
+Run mode is currently selected by the CLI with `reflection run --mode <mode>` and defaults to `smoke` when omitted. The `--ci` flag changes CI/report-root behavior but does not select a separate config-defined run mode yet.
+
+## Run modes
+
+| Mode | Runs |
+| --- | --- |
+| `smoke` | Browser route checks and route-level visual smoke cases. |
+| `design` | Design command checks. |
+| `visual` | Storybook component visual checks. |
+| `full` | Browser, design, and component contracts. |
+
+## Browser contract
+
+```ts
+browser: {
+  enabled: true,
+  blocking: true,
+  baseUrl: 'http://127.0.0.1:5173',
+  server: {
+    command: 'pnpm dev --host 127.0.0.1',
+    readyUrl: 'http://127.0.0.1:5173',
+    reuseExisting: true,
+    timeoutMs: 60_000
+  },
+  maskSelectors: ['[data-reflection-mask]'],
+  routes: [
+    {
+      id: 'home',
+      name: 'Home route',
+      path: '/',
+      viewports: ['desktop', 'mobile'],
+      expects: [
+        { role: 'heading', name: 'Home' },
+        { noHorizontalOverflow: true },
+        { noConsoleErrors: true },
+        { screenshot: 'final' }
+      ]
+    }
+  ],
+  visualSmoke: [
+    {
+      id: 'home-mobile',
+      route: 'home',
+      viewport: 'mobile',
+      baselineRoot: 'tests/fixtures/baselines',
+      baseline: 'browser/home/mobile.chromium-linux.light.png',
+      threshold: { maxDiffPixelRatio: 0.01 },
+      strict: false
+    }
+  ]
+}
+```
+
+Browser fields:
+
+| Field | Notes |
+| --- | --- |
+| `enabled` | Defaults to `true`; set `false` to skip. |
+| `blocking` | Defaults to `true`; route assertion failures are blocking unless configured otherwise by current runner behavior. |
+| `baseUrl` | Absolute URL used to visit route paths. |
+| `server` | Optional managed server. If omitted, Reflection assumes `baseUrl` is already reachable. |
+| `maskSelectors` | Selectors masked in browser screenshots. |
+| `routes` | Route assertions executed in `smoke` and `full` modes. |
+| `visualSmoke` | Route screenshot baseline comparisons driven from successful browser screenshots. |
+
+Supported browser expectations:
+
+```ts
+{ urlIncludes: '/dashboard' }
+{ urlEquals: 'http://127.0.0.1:5173/dashboard' }
+{ role: 'heading', name: 'Dashboard' }
+{ label: 'Email' }
+{ text: 'Welcome back' }
+{ noText: 'Stack trace' }
+{ selector: '[data-ready="true"]' }
+{ elementVisible: '[data-toast]' }
+{ elementNotVisible: '[data-loading]' }
+{ noHorizontalOverflow: true }
+{ noConsoleErrors: true }
+{ screenshot: 'final' }
+```
+
+## Route-level visual smoke cases
+
+Route visual smoke cases compare the screenshot captured by a browser route check against a read-only baseline.
+
+```ts
+visualSmoke: [
+  {
+    id: 'login-mobile',
+    route: 'login',
+    viewport: 'mobile',
+    baselineRoot: 'tests/fixtures/baselines',
+    baseline: 'browser/login/mobile.chromium-linux.light.png',
+    threshold: {
+      maxDiffPixels: 25,
+      maxDiffPixelRatio: 0.01
+    },
+    blocking: false,
+    strict: false
+  }
+]
+```
+
+`blocking: true` or `strict: true` promotes a visual diff to a blocking failure. By default, visual diffs and missing baselines are review-only. The current `reflection update` command promotes route-level `visualSmoke` baselines only; component visual baselines should be copied or reviewed manually until component update support lands.
+
+## Design command contract
+
+```ts
+design: {
+  enabled: true,
+  commands: [
+    {
+      id: 'tokens',
+      command: 'pnpm design:check',
+      cwd: '.',
+      blocking: true
+    }
+  ]
+}
+```
+
+Each design command runs as a project-owned process and writes stdout/stderr evidence into the run artifacts. Use this for token checks, lint-like design-system checks, or other deterministic commands that should participate in the same report.
+
+## Component visual contract
+
+```ts
+component: {
+  enabled: true,
+  storybook: {
+    command: 'pnpm storybook --host 127.0.0.1 --port 6006',
+    readyUrl: 'http://127.0.0.1:6006',
+    reuseExisting: true,
+    timeoutMs: 60_000
+  },
+  cases: [
+    {
+      id: 'primary-button',
+      storyId: 'atoms-button--primary',
+      viewport: 'component',
+      baselineRoot: 'tests/fixtures/baselines',
+      baseline: 'components/button/primary.chromium-linux.light.png',
+      threshold: { maxDiffPixelRatio: 0.005 },
+      stateNote: 'Preferred: story-controlled primary state.'
+    },
+    {
+      id: 'primary-button-hover',
+      storyId: 'atoms-button--primary',
+      baselineRoot: 'tests/fixtures/baselines',
+      baseline: 'components/button/primary-hover.chromium-linux.light.png',
+      browserState: {
+        kind: 'hover',
+        selector: 'button',
+        animationStabilization: { disableAnimations: true }
+      }
+    }
+  ]
+}
+```
+
+Component visual cases resolve Storybook `/index.json`, open the story iframe, capture an actual screenshot, and compare it against the configured baseline.
+
+Pseudo-state policy:
+
+- Prefer story-controlled variants for hover, focus, active, selected, open, and disabled states.
+- If a browser-forced state is necessary, configure `browserState` with `kind: 'hover' | 'focus'`, a selector, and effective animation stabilization.
+- Reflection records `statePolicy` metadata in reports so reviewers can tell whether the state came from the story or was forced in the browser.
+
+## Artifact roots
+
+Local runs default to `.reflection`. CI runs default to `artifacts/reflection` when `--ci` is passed. Either can be overridden with `--report-dir`.
+
+Baselines are separate from run artifacts. They live wherever `baselineRoot` plus `baseline` points, and normal runs never update them.

package/docs/getting-started.md

@@ -0,0 +1,166 @@
+# Getting started with Reflection
+
+Reflection is a local-first CLI for evidence-backed rendered UI validation. A configured run can start a dev server, visit routes, assert browser expectations, capture screenshots, compare visual baselines, and write reviewable reports.
+
+Use this guide when adding Reflection to a new repository manually.
+
+## 1. Install
+
+Install the package as a development dependency:
+
+```bash
+pnpm add -D reflection-check
+```
+
+Then verify the CLI is available:
+
+```bash
+pnpm exec reflection doctor
+```
+
+The package install exposes both `reflection` and `reflection-check` binaries. The docs use `reflection` as the primary command.
+
+For local tarball testing before a registry publish, create and install a packed artifact instead.
+
+From the Reflection repository:
+
+```bash
+pnpm install --frozen-lockfile
+pnpm pack
+```
+
+From the consuming repository:
+
+```bash
+pnpm add -D /absolute/path/to/reflection/reflection-check-0.0.1.tgz
+pnpm exec reflection doctor
+```
+
+## 2. Add a minimal config
+
+Create `reflection.config.ts` at the repository root:
+
+```ts
+import { defineReflection } from 'reflection-check';
+
+export default defineReflection({
+  project: 'my-app',
+  contracts: {
+    browser: {
+      enabled: true,
+      blocking: true,
+      baseUrl: 'http://127.0.0.1:5173',
+      server: {
+        command: 'pnpm dev --host 127.0.0.1',
+        readyUrl: 'http://127.0.0.1:5173',
+        reuseExisting: true,
+        timeoutMs: 60_000
+      },
+      routes: [
+        {
+          id: 'home',
+          path: '/',
+          viewports: ['desktop', 'mobile'],
+          expects: [
+            { role: 'heading', name: 'Home' },
+            { noHorizontalOverflow: true },
+            { noConsoleErrors: true },
+            { screenshot: 'final' }
+          ]
+        }
+      ]
+    }
+  }
+});
+```
+
+During Reflection repository development, the in-repo fixture imports the helper from source instead of the published package.
+
+## 3. Run the local validation loop
+
+```bash
+reflection doctor
+reflection run --config reflection.config.ts --mode smoke
+reflection review --latest
+```
+
+During Reflection development, use the built CLI:
+
+```bash
+pnpm build
+node dist/cli.js doctor
+node dist/cli.js run --config examples/basic-react/reflection.config.ts --mode smoke
+node dist/cli.js review --latest
+```
+
+`reflection run` writes reports and artifacts under `.reflection/runs/<run-id>/` by default.
+
+`reflection doctor` is currently a lightweight setup check. The configured project contract is exercised by `reflection run --config reflection.config.ts ...`.
+
+Important files:
+
+```text
+.reflection/runs/latest
+.reflection/runs/<run-id>/report.md
+.reflection/runs/<run-id>/report.json
+.reflection/runs/<run-id>/manifest.json
+.reflection/runs/<run-id>/browser/**
+.reflection/runs/<run-id>/visual/**
+.reflection/runs/<run-id>/server/**
+```
+
+## 4. Understand the result
+
+Reflection has three useful completion states:
+
+- `pass` — blocking checks passed and there are no review-only visual items.
+- `pass-with-review` — blocking checks passed, but there are review items such as non-strict visual diffs or missing baselines.
+- `fail` / `error` — blocking validation failed or the tool could not complete.
+
+Agents and CI should treat `fail` and `error` as task blockers. `pass-with-review` is intentionally not the same as failure: it asks a human or agent to inspect artifacts and decide whether visible changes are expected.
+
+## 5. Add a visual baseline only after review
+
+The first visual run may produce review items if no baseline exists yet. Inspect the actual screenshot first. If it is intentional, propose an update dry-run:
+
+```bash
+reflection update --config reflection.config.ts --route home --from-run latest --dry-run
+```
+
+Only after explicit human approval should a non-dry update run:
+
+```bash
+reflection update --config reflection.config.ts --route home --from-run latest
+```
+
+Normal `reflection run` never updates baselines.
+
+## 6. Add a short agent pointer
+
+Do not copy every Reflection rule into `AGENTS.md` or another agent file. Add a short pointer to the canonical process instead:
+
+````md
+## Reflection validation
+
+Use Reflection as the UI evidence gate before claiming frontend work is complete.
+
+Run:
+
+```bash
+reflection doctor
+reflection run --config reflection.config.ts --mode smoke
+reflection review --json
+```
+
+Rules:
+
+- Treat blocking failures as task blockers.
+- Summarize review items with artifact paths.
+- Use `reflection update --dry-run` only to propose intentional visual changes.
+- Do not run non-dry `reflection update` unless the human explicitly approves it.
+- Never run `reflection update` in CI.
+
+Full protocol: `docs/validation-process.md`.
+````
+
+See `docs/agent-workflows.md` for the agent-facing workflow and `docs/configuration.md` for the full config shape currently supported.