@pygmalionjs/pygmalion 0.8.1 → 0.9.0

package/docs/coverage-contracts.md

@@ -1,444 +0,0 @@
-# Coverage contracts
-
-A host that ships a storyboard wants two questions answered before a change
-merges:
-
-1. **Representation.** Is every source route the application can render shown
-   somewhere the editor can open — a route preview or a registered component
-   fixture — or excluded for a stated reason?
-2. **Classification.** Is every user scenario verified exactly once, and by
-   which declaration?
-
-They are different questions with different answers. A declared preview URL
-identifies where a screen should render; these static checks do not execute
-that screen or verify a user interaction. A scenario claim identifies the
-declaration responsible for a scenario; it says nothing about routes nobody
-declared. The package answers each with its own functions, and a green answer
-to one never stands in for the other.
-
-Everything here runs in plain Node: no DOM, no Vite server, no editor mount,
-no network. The functions are pure; they take declarations and return
-diagnostics with structured identities. `scripts/smoke-consumer/smoke.ts` is a
-complete consumer that imports them from the packed package and runs the
-positive and negative cases below.
-
-## Who owns what
-
-| The host owns | The package validates |
-| --- | --- |
-| Source discovery and router-specific normalization: which files are routes, what their canonical pattern is, what their stable id is | That every discovered route is represented, by which screen, and how |
-| Screen cases, fixtures, and the mapping from a fixture to the routes it stands for | That every mapping names a route discovery reports and a component the registry has |
-| Verdicts for routes no screen represents: covered by a named declaration, or excluded for a reason | That every verdict names a surface that exists and a declaration that exists, exactly once |
-| The scenario inventory and the claims that classify it | That every scenario is classified exactly once against declarations that exist |
-| Product scope, migration baselines, and what the CI gate fails on | Counts by outcome, with exclusions and todo items apart from coverage |
-
-The package never reads a router, a file system, or a running application in
-this path. When a guarantee needs product knowledge, it belongs in the host;
-the section [What stays in the host](#what-stays-in-the-host) lists those.
-
-## Identify a route by its pattern, not by a preview URL
-
-Discovery hands the audit `DiscoveredRoute` values:
-
-```ts
-interface DiscoveredRoute {
-  /** The canonical pattern, in whatever syntax the host normalized to. */
-  path?: string;
-  /** The host's stable identity for the route — usually its source path. */
-  id?: string;
-}
-```
-
-`path` is the route as the router declares it, not an address someone visited:
-`/items/:id`, or `/items/[id]` if the host keeps the framework's syntax. `id`
-is the join key back to the source. A candidate for an unrepresented route is
-identified as `route:<normalized path>` and carries the `id` as `sourceId`, so
-a diagnostic can be matched to a route name or a source file without parsing
-its message.
-
-The package matches a discovered pattern against a screen's concrete `route`
-in one way: segment for segment, where a `:param` or `*` segment accepts any
-nonempty concrete segment. There is no prefix matching and no other parameter
-syntax. A host whose router writes `[id]` either normalizes to `:id` before
-calling the audit, or maps the route explicitly as the next section shows.
-Router-specific discovery and normalization stay in the host on purpose; the
-package does not carry a parser for any framework.
-
-## Represent a route: preview, mapping, or fixture
-
-A declared screen says what it represents through three fields, all of which a
-`DesignScreenCase` already has:
-
-```ts
-interface DeclaredScreenRoute {
-  id: string;
-  /** The concrete canonical address the screen boots. */
-  route?: string;
-  /** The registered component the screen renders instead of navigating. */
-  componentName?: string;
-  /** Source routes the screen explicitly represents, by discovered id or path. */
-  coverageRoutes?: readonly string[];
-}
-```
-
-`auditRouteCoverage` compares the two lists:
-
-```ts
-import { auditRouteCoverage } from '@pygmalionjs/pygmalion';
-
-const audit = auditRouteCoverage({
-  routes,                     // DiscoveredRoute[] from the host's discovery
-  declaredScreens,            // DesignScreenCase[] or result.routeMappings
-  registeredComponents,       // the component registry, or its names
-});
-audit.candidates;   // SurfaceCandidate[] — routes nothing represents
-audit.represented;  // RouteRepresentation[] — every (route, screen) pair that holds
-audit.diagnostics;  // RouteCoverageDiagnostic[] — mappings that name something missing
-```
-
-A screen represents a route in one of three ways, recorded as `via` on each
-`RouteRepresentation`:
-
-| `via` | How | Example |
-| --- | --- | --- |
-| `preview` | The screen's concrete `route` satisfies the discovered pattern | `route: '/records/main'` represents `/records/:section` |
-| `mapping` | The screen lists the route in `coverageRoutes` and boots a route of its own | `route: '/items/fixture-1', coverageRoutes: ['/items/[id]']` |
-| `fixture` | The screen lists the route in `coverageRoutes` and renders a registered component | `componentName: 'GuardedFixture', coverageRoutes: ['/guarded']` |
-
-An explicit entry matches exactly: it equals the discovered route's `id`, or
-it normalizes to the discovered route's path. No parameter is substituted and
-no prefix is expanded. That is what keeps a fixture from covering more than the
-host said, and a concrete preview from covering a pattern by resemblance.
-
-The two opt-in checks are on by supplying their input. With
-`registeredComponents`, a screen whose `componentName` is not registered
-represents nothing, and the audit reports it. Without it, a fixture mapping is
-taken at its word, which is the 0.7 behaviour plus explicit mappings.
-
-### Diagnostics
-
-Invalid mappings are reported apart from missing representation. The
-candidates list is "what owes a verdict"; the diagnostics list is "what the
-host wrote that no longer holds". A stale mapping produces both: the
-diagnostic on the screen, and the route back among the candidates, because a
-mapping that names something missing must never keep covering.
-
-| `code` | Meaning | Fields |
-| --- | --- | --- |
-| `unknown-route` | A `coverageRoutes` entry matches no discovered route; it was removed or renamed | `screenId`, `route`, `componentName?` |
-| `unregistered-component` | The screen's `componentName` is not among `registeredComponents` | `screenId`, `componentName` |
-| `empty-mapping` | A `coverageRoutes` entry is blank | `screenId`, `route` |
-| `missing-render-source` | A mapping has no nonempty preview route or component fixture | `screenId`, `route` |
-
-Starting in 0.8.1, `coverageRoutes` alone never represents a route. A mapped
-screen must also supply a nonempty preview `route` or a component fixture.
-Empty or whitespace-only preview routes do not implicitly represent `/`.
-Pass `registeredComponents` to check that every named fixture is registered.
-
-A mapping without a render source produces `missing-render-source` and adds
-no representation. If no other valid screen represents that source route, it
-stays among the candidates. If another screen does represent it, the invalid
-mapping still produces a diagnostic: a gate must inspect both `diagnostics`
-and the candidates' verdicts, even when `candidates` is empty.
-
-`enumerateRouteCandidates(input)` returns `audit.candidates` alone, for a
-caller that only needs the holes.
-
-### Examples
-
-A static route with a preview:
-
-```ts
-auditRouteCoverage({
-  routes: [{ id: 'app/settings/page', path: '/settings' }],
-  declaredScreens: [{ id: 'screen:settings', route: '/settings' }],
-});
-// candidates: []   represented: [{ screenId: 'screen:settings', via: 'preview', … }]
-```
-
-A dynamic route, in the framework's own syntax, represented by the concrete
-preview that instantiates it:
-
-```ts
-auditRouteCoverage({
-  routes: [{ id: 'app/items/[id]/page', path: '/items/[id]' }],
-  declaredScreens: [
-    {
-      id: 'screen:item-detail',
-      route: '/items/fixture-1',
-      coverageRoutes: ['app/items/[id]/page'],
-    },
-  ],
-});
-// candidates: []   represented: [{ via: 'mapping', sourceId: 'app/items/[id]/page', … }]
-```
-
-Without the `coverageRoutes` entry the same input leaves `route:/items/[id]`
-among the candidates: a concrete address does not satisfy a bracket pattern,
-because the package does not parse that syntax. The route stays missing until
-the host supplies the mapping, a registered fixture, or a normalized pattern
-(`path: '/items/:id'`, which the concrete preview then satisfies).
-
-A guarded route, represented by a registered component fixture and no
-navigation URL:
-
-```ts
-auditRouteCoverage({
-  routes: [{ id: 'app/guarded/page', path: '/guarded' }],
-  declaredScreens: [
-    { id: 'screen:guarded', componentName: 'GuardedFixture', coverageRoutes: ['/guarded'] },
-  ],
-  registeredComponents: registry,
-});
-// candidates: []   represented: [{ via: 'fixture', componentName: 'GuardedFixture', … }]
-```
-
-Remove `GuardedFixture` from the registry and the same call reports
-`unregistered-component` on `screen:guarded` and `route:/guarded` among the
-candidates.
-
-A multi-screen journey — every step is its own source route and its own
-screen; the journey itself is declared to the storyboard as a `screenFlows`
-path, which is a separate, host-owned declaration:
-
-```ts
-auditRouteCoverage({
-  routes: [
-    { id: 'app/checkout/page', path: '/checkout' },
-    { id: 'app/checkout/review/page', path: '/checkout/review' },
-    { id: 'app/checkout/done/page', path: '/checkout/done' },
-  ],
-  declaredScreens: [
-    { id: 'screen:checkout', route: '/checkout' },
-    { id: 'screen:checkout-review', route: '/checkout/review' },
-    { id: 'screen:checkout-done', componentName: 'CheckoutDoneFixture', coverageRoutes: ['/checkout/done'] },
-  ],
-  registeredComponents: registry,
-});
-```
-
-A step reached from another step without a navigation (a confirmation dialog
-on the review screen) is a screen state or an interactive-state axis of the
-review screen, not a route; the [screen state contract](./screen-state-contract.md)
-decides which.
-
-### Audit before the collection strips the mapping
-
-`createDesignScreenCollection` removes `coverageRoutes` from the pages it
-returns, because the editor does not need it. The result keeps the route facts
-beside them as `routeMappings`, one `DeclaredScreenRoute` per case, so the
-audit runs on exactly the declarations the editor will show:
-
-```ts
-const result = createDesignScreenCollection({ cases, scenarios, behaviorScenarios });
-const audit = auditRouteCoverage({
-  routes,
-  declaredScreens: result.routeMappings,
-  registeredComponents: registry,
-});
-```
-
-The collection throws when scenario classification fails, so a gate that wants
-the route audit even then passes the original `cases` instead; they satisfy
-`DeclaredScreenRoute` as written.
-
-## Answer the remaining routes: verdicts
-
-A route nothing represents still exists; the host says what happens to it. A
-`SurfaceClassification` names the candidate and gives one of two verdicts:
-
-```ts
-const classifications: SurfaceClassification[] = [
-  { id: 'route:/labs', verdict: 'covered', by: 'screen:labs' },
-  { id: 'route:/legacy', verdict: 'excluded', reason: 'redirects to /settings on every entry' },
-];
-```
-
-`validateSurfaceDemands(candidates, classifications, options?)` checks them.
-Every failure is an error; a warning would let a surface go unreviewed while
-the build stayed green.
-
-| `code` | Meaning |
-| --- | --- |
-| `unclassified` | A candidate has no verdict |
-| `stale-verdict` | A verdict names a surface no enumerator reports |
-| `duplicate-verdict` | Two verdicts answer one surface |
-| `duplicate-candidate` | An enumerator reported one surface twice |
-| `exclusion-reason` | An exclusion's reason is shorter than `MIN_SURFACE_REASON_LENGTH` |
-| `empty-target` | A covered verdict names nothing |
-| `unknown-target` | A covered verdict names a declaration that is not among `knownTargets` |
-
-`unknown-target` is the check a deleted screen used to slip past: a verdict
-saying `by: 'screen:settings'` kept covering `/settings` after the screen was
-gone, because the target only had to be nonempty. Pass the declarations and it
-becomes an error:
-
-```ts
-import { declaredCoverageTargets, validateSurfaceDemands } from '@pygmalionjs/pygmalion';
-
-const knownTargets = declaredCoverageTargets({
-  screens: cases,              // named by their own id, e.g. 'screen:settings'
-  components: registry,        // named 'component:<name>'
-  targets: axes.map((axis) => `axis:${axis.id}`),  // anything else, verbatim
-});
-const diagnostics = validateSurfaceDemands(audit.candidates, classifications, { knownTargets });
-```
-
-A screen is named by its own id — the same string a scenario claim's
-`screenId` uses. A host with a different vocabulary for targets builds the set
-itself and hands it to `knownTargets` directly.
-
-## Classify scenarios exactly once
-
-`createDesignScreenCollection` takes the scenario inventory and the cases that
-demonstrate it and throws before the editor opens when any scenario is
-classified zero times or twice, or when a case names a scenario the inventory
-lacks. Its `coverage` counts screens and behaviors apart and lists what is
-missing. The invariant is exact-once classification of the scenarios the host
-supplied; the function does not discover scenarios, and it does not learn
-about routes.
-
-`validateScenarioCoverage(claims, context)` is the same invariant for the
-claims ledger, where a scenario may be verified by a frame, a screen-state
-sibling, a component prop, a viewport preset, a catalog sheet, an interactive
-axis, a card slot, or a list length, or admitted as `behavior` or `todo` with
-a reason. Every diagnostic carries a `code`, the `scenarioId`, the claim's
-`kind`, its `screenId` when it named one, and the `target` the context lacks:
-
-| `code` | Meaning |
-| --- | --- |
-| `unknown-scenario` | The claim names a scenario the inventory lacks |
-| `unclaimed` | A scenario has no claim |
-| `duplicate-claim` | A scenario has more than one claim |
-| `insufficient-reason` | A `behavior` or `todo` reason is below the floor |
-| `unknown-screen` | The screen or catalog sheet is not declared |
-| `not-a-state-group` | A `screen-state` claim names a screen with no sibling states |
-| `unknown-component`, `unknown-prop`, `unknown-value` | The registry lacks the component, the prop, or the value |
-| `unknown-preset` | The viewport preset is not declared |
-| `unknown-axis`, `unknown-option` | The interactive axis or option, or the card option, is not declared |
-| `unknown-card`, `unknown-slot` | The card or slot is not declared |
-| `unknown-list`, `count-out-of-range` | The list is not declared, or the count is outside its range |
-| `base-option` | Warning: the claimed option is the axis base, which reproduces the capture |
-
-### Case and step identity
-
-Exact-once classification depends on the granularity of scenario ids. The
-rule: one scenario id per state a reviewer should be able to select again,
-classified by exactly one declaration.
-
-- A journey of three screens is three scenario ids (or one id per step), each
-  classified by its own case. One scenario id spanning three screens cannot be
-  classified exactly once.
-- A screen-state variant is a case of its own in the same `scenario` group,
-  distinguished by `state`. It carries its own scenario id, or the group's
-  canonical case carries the set in `scenarioIds`.
-- A case demonstrates every id in its `scenarioIds`; the canonical
-  `scenarioId` is only where a reviewer is sent first.
-- A viewport variant of a case is not a new scenario. Expand with
-  `expandDesignScreenCasesViewports`, which derives variant ids from the case
-  id and keeps the scenario ids with the authored case.
-- An interaction step inside a recipe is how a case is reached, not a
-  scenario. A step a reviewer must be able to select on its own is a
-  screen-state axis option or a case.
-- A route candidate id (`route:/checkout`) and a scenario id (`US-checkout`)
-  are different namespaces. A covered verdict names the screen that
-  represents the route; a claim names the screen that verifies the scenario.
-  The same screen may do both.
-
-## A preview is not a scenario
-
-A route the audit reports as represented has a screen that renders it. That
-is all it says. Whether the screen demonstrates a user scenario is the claims
-ledger's question, answered by a claim naming that screen; whether an
-interaction is covered is a screen-state axis or an interactive-state claim,
-never inferred from a URL. A gate that reports "all routes represented" has
-not reported scenario coverage.
-
-## Counts a report shows
-
-Two summaries turn diagnostics into numbers without re-deriving them:
-
-```ts
-summarizeSurfaceDemands(candidates, classifications, { knownTargets });
-// { total, covered, excluded, invalid, unclassified, excludedIds, invalidIds, unclassifiedIds }
-
-summarizeScenarioCoverage(claims, context);
-// { total, covered, behavior, todo, invalid, unclaimed, behaviorIds, todoIds, invalidIds, unclaimedIds, unknownIds }
-```
-
-`covered` counts only what validated clean. An exclusion is in `excluded`,
-never in `covered`. A `todo` is in `todo`. A verdict or claim with an error is
-in `invalid`. A surface or scenario nobody answered is in `unclassified` or
-`unclaimed`. Nothing moves between buckets, so a report that shows all of
-them shows the whole picture, and a report that shows only `covered` is
-visibly incomplete.
-
-## Join diagnostics to source
-
-| Diagnostic | Identity fields |
-| --- | --- |
-| `SurfaceDemandDiagnostic` | `code`, `kind` (`route`, `control`, `request`), `candidateId`, `sourceId?`, `frameId?`, `target?` |
-| `RouteCoverageDiagnostic` | `code`, `screenId`, `route?`, `componentName?` |
-| `ScenarioCoverageDiagnostic` | `code`, `scenarioId`, `kind?`, `screenId?`, `target?` |
-| `RouteRepresentation` | `candidateId`, `path`, `sourceId?`, `screenId`, `via`, `componentName?` |
-
-`message` is for people. A gate joins on the fields: `sourceId` back to the
-route's source path, `screenId` to the case and its `sourcePath`,
-`scenarioId` to the scenario's `sourcePath`, `target` to the declaration that
-is missing.
-
-## What stays in the host
-
-These guarantees need product knowledge, so the package does not provide
-them; a host adopting the gate implements them itself:
-
-- **Discovery and normalization.** Which files are routes, what their canonical
-  pattern is, and a stable `id` for each. The package matches patterns in one
-  syntax and parses no framework's.
-- **Scope.** Which discovered routes are in the product at all. A route that is
-  out of scope gets an `excluded` verdict with a reason; the package does not
-  decide eligibility.
-- **Fixture fidelity.** That a registered component fixture renders what the
-  route renders. The package checks that the component is registered, not
-  what it draws.
-- **The inventory.** Scenario ids and their granularity, per the rules above.
-- **Baselines.** A migration window is a host-held set of candidate ids that
-  are allowed to stay unclassified for now, subtracted from the diagnostics
-  before failing the build. The package does not downgrade severities.
-- **CI policy.** Which diagnostics fail the build, on which Node version. The
-  package declares `engines.node`; the gate should run on the host's
-  supported version and on the declared minimum.
-
-## A headless gate, end to end
-
-```ts
-import {
-  auditRouteCoverage,
-  createDesignScreenCollection,
-  declaredCoverageTargets,
-  summarizeScenarioCoverage,
-  summarizeSurfaceDemands,
-  validateScenarioCoverage,
-  validateSurfaceDemands,
-} from '@pygmalionjs/pygmalion';
-
-const routes = discoverRoutes();          // host: DiscoveredRoute[]
-const result = createDesignScreenCollection({ cases, scenarios, behaviorScenarios });
-
-const audit = auditRouteCoverage({
-  routes,
-  declaredScreens: result.routeMappings,
-  registeredComponents: registry,
-});
-const surfaces = validateSurfaceDemands(audit.candidates, classifications, {
-  knownTargets: declaredCoverageTargets({ screens: cases, components: registry }),
-});
-const ledger = validateScenarioCoverage(claims, context);
-
-const failures = [...audit.diagnostics, ...surfaces, ...ledger].filter(
-  (diagnostic) => diagnostic.severity === 'error' && !baseline.has(keyOf(diagnostic)),
-);
-console.table(summarizeSurfaceDemands(audit.candidates, classifications));
-console.table(summarizeScenarioCoverage(claims, context));
-process.exit(failures.length === 0 ? 0 : 1);
-```