@anselmdk/feature-spec-md 0.3.1 → 0.5.0-rc.1

package/README.md

@@ -1,8 +1,8 @@
 # feature-spec-md
 
-Markdown specs for AI-assisted, testable spec driven development.
+Markdown specs for AI-assisted, testable spec-driven development.
 
-The concept is deliberately small:
+`feature-spec-md` helps you describe a product in Markdown, ask an AI to turn those specs into executable tests, and then prove which parts of the specification are covered by those tests. It is intentionally small: your specs stay as plain Markdown, your tests stay in your normal test runner, and the tool connects the two through stable spec IDs.
 
 ```txt
 model + features + stack + design
@@ -10,9 +10,34 @@ model + features + stack + design
 -> validation, coverage, screenshots, and reports
 ```
 
-The specs are meant to be written with an AI before implementation. The tests are meant to be written with an AI from those specs. `feature-spec-md` checks that the Markdown stays structured and that executable tests still cover the model items, rules, and scenarios the specs define.
+## What the tool does
 
-## What You Write
+The shortest version is: `feature-spec-md` turns Markdown specifications into a testable contract.
+
+It gives you:
+
+- **Spec document formats** for domain models, product features, technical stack notes, and UI/design direction.
+- **Stable IDs** for model items, rules, and scenarios, so tests can reference exactly what they implement.
+- **Validation** that checks frontmatter, headings, IDs, cross-document references, and test coverage expectations.
+- **Coverage reporting** that shows which model items, rules, and scenarios are implemented by tests.
+- **Scenario evidence policy** so specs can declare whether behavior should be tested by unit, integration, Playwright, manual, or no executable tests.
+- **HTML reports** that are useful for reviews, CI artifacts, or published build reports.
+- **Screenshot evidence** for scenarios when using the Playwright helper.
+- **A library API** for projects that want to parse specs, check coverage, collect screenshots, or render reports from their own tooling.
+
+The specs are meant to be written with an AI before implementation. The tests are meant to be written with an AI from those specs. `feature-spec-md` then checks that the Markdown stays structured and that executable tests still cover the model items, rules, and scenarios the specs define.
+
+## Demo project
+
+A complete demo app is available in [`anselmdk/feature-spec-md-demo`](https://github.com/anselmdk/feature-spec-md-demo). It shows a small support-ticket desk built from model, feature, stack, and design specs, with unit tests, Playwright tests, generated coverage, and a published report.
+
+Demo reports, including scenario screenshots, are available here:
+
+<https://feature-spec-md.anselm.dk/demo/>
+
+Use the demo repository when you want to see the expected project shape, script names, screenshot manifest flow, and report output in a real app.
+
+## What you write
 
 Use four ordinary Markdown document types:
 
@@ -23,9 +48,42 @@ Use four ordinary Markdown document types:
 *.design.md   product, UI, and interaction direction
 ```
 
-Each document has frontmatter, a short `## Purpose`, and stable IDs. Tests reference those IDs in titles, tags, annotations, comments, or metadata.
+Each document has frontmatter, a short `## Purpose`, and stable IDs.
+
+### Model specs
+
+Model specs define the shared vocabulary that features can refer to. Use them for domain concepts, states, entities, and important business terms.
+
+```md
+### ACCOUNT-M001: Registered person
+
+A person who has completed registration and can request sign-in links.
+```
+
+### Feature specs
+
+Feature specs describe user-facing behavior with rules and scenarios. Rules and scenarios get stable IDs that tests can reference.
 
 ```md
+---
+id: ACCOUNT-ACCESS
+title: Account access
+test: playwright
+screenshots: required
+---
+
+# Account access
+
+## Purpose
+
+Allow registered people to access their account securely.
+
+## Rules
+
+- ACCOUNT-ACCESS-R001: Sign-in links MUST expire.
+
+## Scenarios
+
 ### ACCOUNT-ACCESS-S001: Registered person signs in
 
 Given a registered person is on the sign-in page
@@ -33,13 +91,46 @@ When they request and open a valid sign-in link
 Then they are signed in
 ```
 
+Feature specs can declare the expected test and evidence policy:
+
+```txt
+test: unit | integration | playwright | manual | skip
+screenshots: required | optional | skip
+```
+
+If `screenshots` is omitted, Playwright scenarios default to `required`; non-Playwright scenarios default to `skip`. Scenario-level overrides can be written directly below a scenario heading:
+
+```md
+### ACCOUNT-ACCESS-S002: Link expiry is calculated
+Test: unit
+Screenshots: skip
+
+Given a sign-in link was created 31 minutes ago
+When expiry is calculated
+Then the link is expired
+```
+
+See [SPEC_FORMAT.md](SPEC_FORMAT.md) and [docs/evidence-policy.md](docs/evidence-policy.md) for the exact format.
+
+### Stack specs
+
+Stack specs document technical decisions that shape the implementation: framework, storage, test runner, deployment constraints, external services, or architecture decisions.
+
+### Design specs
+
+Design specs capture product, UI, and interaction direction: layout priorities, states, accessibility expectations, empty states, or copy tone.
+
+## How tests connect to specs
+
+Tests reference spec IDs in titles, tags, annotations, comments, or metadata. The tool scans test files and matches those references back to the Markdown documents.
+
 ```ts
 test("ACCOUNT-ACCESS-S001 registered person signs in", async ({ page }) => {
   // Covers ACCOUNT-ACCESS-R001 and ACCOUNT-M001.
 });
 ```
 
-See [SPEC_FORMAT.md](SPEC_FORMAT.md) for the exact document format.
+This keeps the test runner independent from `feature-spec-md`. You can use Playwright, Vitest, Node test, or another runner as long as the source files contain the relevant IDs.
 
 ## Install
 
@@ -59,19 +150,22 @@ npx feature-spec-md init --kind design --dir specs
 ## Workflow
 
 1. Ask an AI to draft or update `*.model.md`, `*.feature.md`, `*.stack.md`, and `*.design.md` files.
-2. Run `npx feature-spec-md check` until the spec set is valid.
-3. Ask an AI to write executable tests from the specs, preserving the relevant `-M001`, `-R001`, and `-S001` IDs in the test source.
-4. Run `npx feature-spec-md coverage` to see which scenarios, rules, and model items have tests.
-5. Run `npx feature-spec-md report` to generate an HTML implementation report for review or CI artifacts.
+2. Declare each feature/scenario test evidence policy where the default is not right.
+3. Run `npx feature-spec-md check` until the spec set is valid.
+4. Ask an AI to write executable tests from the specs, preserving the relevant `-M001`, `-R001`, and `-S001` IDs in the test source.
+5. Run `npx feature-spec-md coverage` to see which scenarios, rules, and model items have tests.
+6. Run `npx feature-spec-md report` to generate an HTML implementation report for review or CI artifacts.
+7. Use `npx feature-spec-md github-report` in GitHub Actions when the report should be linked from the job summary or published.
 
-The longer flow, including AI prompts and CI setup, is in [docs/spec-driven-flow.md](docs/spec-driven-flow.md).
+The longer flow, including AI prompts and CI setup, is in [docs/spec-driven-flow.md](docs/spec-driven-flow.md). The demo repository also shows the flow in practice: <https://github.com/anselmdk/feature-spec-md-demo>.
 
-## CLI
+## CLI tools
 
 ```bash
 npx feature-spec-md check
 npx feature-spec-md coverage --fail-on-missing
 npx feature-spec-md report --out test-results/feature-spec-report/index.html
+npx feature-spec-md github-report --report-dir test-results/spec-report
 ```
 
 By default the CLI scans:
@@ -92,20 +186,57 @@ npx feature-spec-md check \
   --tests "e2e/**/*.spec.ts"
 ```
 
-The `check` command validates spec structure, references between documents, and test coverage. Scenario coverage is required by default when tests are scanned. Use `--require-scenario-coverage=false` while drafting. Use `--require-rule-coverage` and `--require-model-coverage` when rules and model items must also fail validation if they have no test references.
+### `check`
+
+Validates the spec set and, by default, requires scenario coverage when tests are scanned.
+
+```bash
+npx feature-spec-md check
+```
+
+`check` validates spec structure, references between documents, and test coverage. Use `--require-scenario-coverage=false` while drafting. Use `--require-rule-coverage` and `--require-model-coverage` when rules and model items must also fail validation if they have no test references.
+
+### `coverage`
+
+Prints a terminal implementation report showing covered and missing model items, rules, and scenarios.
+
+```bash
+npx feature-spec-md coverage --fail-on-missing
+```
+
+Use `--fail-on-missing` when missing model item, rule, or scenario coverage should fail CI.
+
+### `report`
 
-The `coverage` command prints a terminal implementation report. Use `--fail-on-missing` when missing model item, rule, or scenario coverage should fail CI.
+Writes an HTML report for local review, CI artifacts, or publishing.
 
-The `report` command writes an HTML report. It can include screenshot evidence from Playwright or another test runner by passing one or more screenshot manifest files:
+```bash
+npx feature-spec-md report \
+  --tests "tests/**/*.ts" \
+  --out test-results/spec-report/index.html
+```
+
+The report can include screenshot evidence from Playwright or another test runner by passing one or more screenshot manifest files:
 
 ```bash
 npx feature-spec-md report \
   --screenshots "test-results/spec-report/screenshots-*.json"
 ```
 
-## GitHub Actions report publishing
+When CI should fail for missing declared screenshot evidence, add `--enforce-evidence`:
 
-`feature-spec-md github-report` writes the GitHub Actions job summary and can either prepare outputs for a GitHub artifact upload or publish the generated report to FTP.
+```bash
+npx feature-spec-md report \
+  --screenshots "test-results/spec-report/screenshots-*.json" \
+  --enforce-evidence \
+  --out test-results/spec-report/index.html
+```
+
+This gate only fails for scenarios whose resolved screenshot policy is `required`.
+
+### `github-report`
+
+Writes a GitHub Actions job summary and prepares the generated report for either artifact upload or FTP publishing.
 
 ```bash
 npx feature-spec-md github-report \
@@ -113,17 +244,17 @@ npx feature-spec-md github-report \
   --publish ftp
 ```
 
-For FTP publishing, configure repository secrets in GitHub under **Settings → Secrets and variables → Actions → Repository secrets**. For this repository, the direct settings URL is:
+## Playwright screenshot evidence
 
-```txt
-https://github.com/anselmdk/feature-spec-md/settings/secrets/actions
-```
+The package exports a Playwright helper from `@anselmdk/feature-spec-md/playwright`. It maps scenario step text back to the spec line, wraps the implementation in a Playwright `test.step`, captures a screenshot after the step, attaches it to the test, and writes a screenshot manifest such as `test-results/spec-report/screenshots-0.json`.
 
-GitHub's documentation for repository secrets is here:
+That manifest can then be passed to `feature-spec-md report` with `--screenshots "test-results/spec-report/screenshots-*.json"` so the HTML report can show scenario evidence next to the relevant spec step. With `--enforce-evidence`, missing screenshots fail only for scenarios declared as `screenshots: required`.
 
-```txt
-https://docs.github.com/actions/security-guides/using-secrets-in-github-actions#creating-secrets-for-a-repository
-```
+## GitHub Actions report publishing
+
+`feature-spec-md github-report` writes the GitHub Actions job summary and can either prepare outputs for a GitHub artifact upload or publish the generated report to FTP.
+
+For FTP publishing, configure repository secrets in GitHub under **Settings → Secrets and variables → Actions → Repository secrets**.
 
 Required secrets or environment variables:
 
@@ -155,20 +286,6 @@ Example GitHub Actions step:
     FEATURE_SPEC_REPORT_BASE_URL: ${{ secrets.FEATURE_SPEC_REPORT_BASE_URL }}
 ```
 
-With `FEATURE_SPEC_REPORT_BASE_URL=http://feature-spec-md.anselm.dk/` and GitHub Actions build number `42`, the report is uploaded to a build-numbered directory and linked as:
-
-```txt
-http://feature-spec-md.anselm.dk/42/
-```
-
-The build index is created or updated at:
-
-```txt
-http://feature-spec-md.anselm.dk/
-```
-
-FTP reports are uploaded into a build-numbered directory using `GITHUB_RUN_NUMBER`, and an `index.html` file is created or updated at the public base URL so all uploaded builds can be browsed. When FTP publishing is used, the command emits `upload-github-artifact=false` and links the GitHub Actions job summary directly to the hosted report URL.
-
 ## Library API
 
 Most integrations can use the top-level document API:
@@ -176,14 +293,28 @@ Most integrations can use the top-level document API:
 ```ts
 import {
   checkSpecDocuments,
+  collectSpecScreenshots,
   parseSpecDocument,
   renderHtmlReport,
+  validateScenarioScreenshots,
   validateSpecDocument,
 } from "@anselmdk/feature-spec-md";
 ```
 
+Useful exports include:
+
+- `parseSpecDocument` and kind-specific parsers for reading Markdown specs.
+- `validateSpecDocument` and `validateSpecGraph` for checking one document or a connected spec set.
+- `checkSpecDocuments` for loading specs and tests, validating documents, and computing coverage in one call.
+- `buildSpecCoverageSummary` and `collectSpecTestReferences` for custom coverage workflows.
+- `collectSpecScreenshots` and `validateScenarioScreenshots` for loading and enforcing screenshot evidence.
+- `renderHtmlReport` for generating the same report UI from your own integration.
+- `writeTextFile` for small report-writing integrations.
+
 Feature-only helpers such as `parseFeatureSpec` and `checkFeatureSpecs` remain available for compatibility.
 
+The package also exports `@anselmdk/feature-spec-md/specDocuments` and `@anselmdk/feature-spec-md/playwright` for more focused imports.
+
 ## Development
 
 ```bash

package/SPEC_FORMAT.md

@@ -20,15 +20,6 @@ A spec set can contain four document types:
 - `*.stack.md` defines technical platform choices and their rationale.
 - `*.design.md` defines product, UI, layout, visual, and interaction direction.
 
-The short version:
-
-```txt
-model tells tests what things mean
-features tell tests what behavior must work
-stack tells implementation what tools and constraints to use
-design tells implementation what experience to create
-```
-
 ## Frontmatter
 
 Every file starts with YAML-like frontmatter:
@@ -53,6 +44,8 @@ Optional fields:
 - `owner`
 - `model`: a single referenced model id, for feature and design files
 - `models`: comma-separated referenced model ids, for feature and design files
+- `test`: default scenario test type for feature files
+- `screenshots`: default screenshot evidence policy for feature files
 
 IDs use uppercase words separated by hyphens. IDs are the contract between specs, tests, reports, and implementation work.
 
@@ -128,115 +121,81 @@ When they create a card with the title "Write release notes"
 Then the card "Write release notes" is visible in the To do column
 ```
 
-Allowed step keywords:
-
-- `Given`
-- `When`
-- `Then`
-- `And`
-- `But`
+Allowed step keywords are `Given`, `When`, `Then`, `And`, and `But`.
 
 Rules state durable product truths. Scenarios show concrete examples that executable tests can implement.
 
-## Stack Files
-
-Stack files use the `.stack.md` suffix.
+### Scenario Test And Evidence Policy
 
-A stack file defines technical platform choices and the reasoning behind them.
+Feature specs can declare how scenarios are expected to be tested and whether screenshot evidence is required.
 
-Required sections:
+Feature-level frontmatter sets defaults for all scenarios in a feature:
 
 ```md
-# Kanban tech stack
-
-## Purpose
-
-## Stack
+---
+id: KANBAN-CARD-AUTHORING
+title: Card authoring
+test: playwright
+screenshots: required
+---
 ```
 
-Optional sections:
+Scenario-level overrides go directly below a scenario heading:
 
 ```md
-## Context
+### KANBAN-CARD-AUTHORING-S002: Card title is normalized
+Test: unit
+Screenshots: skip
 
-## Rationale
-
-## Consequences
+Given the raw card title contains leading whitespace
+When the title is normalized
+Then the stored title has no leading whitespace
 ```
 
-Example:
+Supported `test` values are `unit`, `integration`, `playwright`, `manual`, and `skip`.
 
-```md
----
-id: KANBAN-STACK
-title: Kanban tech stack
-status: draft
----
+Supported `screenshots` values are `required`, `optional`, and `skip`. `screenshots: none` is accepted as an alias for `screenshots: skip`.
 
-# Kanban tech stack
+Defaults:
 
-## Purpose
+- If `test` is omitted, scenarios default to `unit`.
+- If `screenshots` is omitted and `test` resolves to `playwright`, screenshots default to `required`.
+- If `screenshots` is omitted and `test` resolves to `unit`, `integration`, `manual`, or `skip`, screenshots default to `skip`.
 
-Define the initial technical stack for implementing the Kanban board.
+See [Scenario Test And Evidence Policy](docs/evidence-policy.md) for details and CI examples.
 
-## Stack
-
-| Area     | Choice     |
-| -------- | ---------- |
-| Frontend | React      |
-| Language | TypeScript |
-| Testing  | Playwright |
-```
-
-## Design Files
-
-Design files use the `.design.md` suffix.
+## Stack Files
 
-A design file defines product, UI, layout, visual, and interaction direction.
+Stack files use the `.stack.md` suffix and define technical platform choices and the reasoning behind them.
 
 Required sections:
 
 ```md
-# Kanban board design
+# Kanban tech stack
 
 ## Purpose
 
-## Design
+## Stack
 ```
 
-Optional sections:
-
-```md
-## Principles
-
-## Layout
+Optional sections are `## Context`, `## Rationale`, and `## Consequences`.
 
-## Interaction
+## Design Files
 
-## Visual style
-```
+Design files use the `.design.md` suffix and define product, UI, layout, visual, and interaction direction.
 
-Example:
+Required sections:
 
 ```md
----
-id: KANBAN-DESIGN
-title: Kanban board design
-status: draft
-model: KANBAN
----
-
 # Kanban board design
 
 ## Purpose
 
-Define the visual and interaction design direction for the Kanban board.
-
 ## Design
-
-The board should feel lightweight, immediate, and calm.
 ```
 
+Optional sections are `## Principles`, `## Layout`, `## Interaction`, and `## Visual style`.
+
 ## Splitting Guidance
 
 Split model files by coherent domain vocabulary, ownership, or lifecycle.
@@ -251,7 +210,7 @@ Keep each document small enough for an AI to read, revise, and use as test-writi
 
 ## Test Coverage Convention
 
-Spec files do not contain test mappings.
+Spec files declare expected test and evidence policy, but they do not map scenarios to concrete test files.
 
 Tests reference model item, rule, and scenario IDs in test titles, tags, annotations, comments, or metadata.
 
@@ -261,6 +220,7 @@ Generated tooling can answer:
 - Which scenarios have tests?
 - Which rules have executable coverage?
 - Which tests reference deleted or unknown spec IDs?
+- Which scenarios are expected to have screenshot evidence?
 - Which visible flows have screenshots, traces, or other evidence?
 
 Example: