saffron-ai 0.3.1 → 0.4.1

package/README.md

@@ -4,40 +4,43 @@
 
 Free to use · [npm: saffron-ai](https://www.npmjs.com/package/saffron-ai) · [site](https://saffron-ai.lovable.app)
 
-Write tests as plain `.feature` files. On the first run, an AI agent (Claude Agent SDK + Playwright MCP) executes each scenario in a real browser and records what it did into a **cache** — a human-reviewable JSON action list committed to git. Every run after that replays the cache with plain Playwright: **zero AI calls, zero tokens, ~100ms per scenario**.
+Write tests as plain `.feature` files. On the first run, an AI agent (Claude Agent SDK + Playwright MCP) executes each scenario in a real browser and records what it did into a **cache**. A human-reviewable JSON action list committed to git. Every run after that replays the cache with plain Playwright: **zero AI calls, zero tokens, ~100ms per scenario**.
 
-When the UI changes and a cached step fails at runtime, the agent takes over mid-execution with full scenario context, adapts, and finishes the run — then files a **cache proposal** you accept or reject, like a snapshot test update. Reports state exactly what was adapted and what it cost.
+When the UI changes and a cached step fails at runtime, the agent takes over mid-execution with full scenario context, adapts, and finishes the run, then files a **cache proposal** you accept or reject, like a snapshot test update. Reports state exactly what was adapted and what it cost.
 
 ## The rules that keep it honest
 
-- **Assertions are sacred.** By default no `Then` is ever healed — the AI may help *reach* an assertion, never make it pass; enforced mechanically, not by prompt. Projects may opt into `adaptable-mid` for mid-scenario checkpoints, but the **final assertion block** of a scenario is strict under every policy, forever.
+- **Assertions are sacred.** By default no `Then` is ever healed: the AI may help *reach* an assertion, never make it pass; enforced mechanically, not by prompt. Projects may opt into `adaptable-mid` for mid-scenario checkpoints, but the **final assertion block** of a scenario is strict under every policy, forever.
 - **Three result states.** Green = cached pass. Yellow = passed with AI adaptation (pending your review). Red = failed.
 - **Diagnose before adapting.** On a step failure the agent first decides: UI drift (heal) or application defect (fail with a diagnosis).
-- **Caches are git artifacts.** Proposals show diffs — including a suggested `.feature` edit when the written steps no longer match reality — and nothing is committed without you.
+- **Caches are git artifacts.** Proposals show diffs, including a suggested `.feature` edit when the written steps no longer match reality, and nothing is committed without you.
 - **Verified proposals.** Every recording/heal is proof-replayed zero-AI before it's filed (with one bounded refinement pass on failure), so proposals arrive stamped `verified ✓` or honestly `UNVERIFIED ✗`.
-- **Honest cost reporting.** Reports break out prompt-cache reads/writes — the real bill of agent sessions — next to the in+out token count, per scenario and in totals.
-- **Trend memory.** Every run appends to `.saffron/history.jsonl`; reports show deltas vs the previous run and 20-run sparklines, and Saffron flags **chronic scenarios** (healing repeatedly — re-record instead of paying again) plus recurring failure themes.
+- **Honest cost reporting.** Reports break out prompt-cache reads/writes (the real bill of agent sessions) next to the in+out token count, per scenario and in totals.
+- **Trend memory.** Every run appends to `.saffron/history.jsonl`; reports show deltas vs the previous run and 20-run sparklines, and Saffron flags **chronic scenarios** (healing repeatedly. Re-record instead of paying again) plus recurring failure themes.
 
 ## Step reuse: new scenarios get cheaper as your suite grows
 
-Like a maturing Cucumber suite, most of a new feature file is steps you already have. Saffron derives a **step index** from committed caches and, when recording a new scenario, replays every seeded island of known steps zero-AI — the agent records only the genuinely novel steps, wherever they sit. Exact step text is the identity (Cucumber-style); quoted values may differ (`"admin"` seeds from `"bob"`); data-table steps seed on matching keys. Measured on a live site: a 6-step scenario with one novel step recorded for **$0.15 / 6 AI calls**, vs $2.07 for a full unseeded recording of comparable length.
+Like a maturing Cucumber suite, most of a new feature file is steps you already have. Saffron derives a **step index** from committed caches and, when recording a new scenario, replays every seeded island of known steps zero-AI. The agent records only the genuinely novel steps, wherever they sit. Exact step text is the identity (Cucumber-style); quoted values may differ (`"admin"` seeds from `"bob"`); data-table steps seed on matching keys. Measured on a live site: a 6-step scenario with one novel step recorded for **$0.15 / 6 AI calls**, vs $2.07 for a full unseeded recording of comparable length.
 
-**Step sets** (shipped): `.saffron` files — a superset dialect of Gherkin — add the `StepSet:` keyword for named, reusable step sequences invoked with `StepSet <name>` inside any scenario. Sets expand at parse time, so their steps cache and seed like ordinary steps; editing a set makes every invoking scenario honestly stale (re-recorded mostly seeded), and heal edits route to the set definition — one fix, every caller follows. Sets are **project-wide**: keep application-wide flows in a sets-only library file (convention: `features/shared.steps.saffron`) and invoke them from any feature.
+**Step sets** (shipped): `.saffron` files, a superset dialect of Gherkin, add the `StepSet:` keyword for named, reusable step sequences invoked with `StepSet <name>` inside any scenario. Sets expand at parse time, so their steps cache and seed like ordinary steps; editing a set makes every invoking scenario honestly stale (re-recorded mostly seeded), and heal edits route to the set definition, one fix, every caller follows. Sets are **project-wide**: keep application-wide flows in a sets-only library file (convention: `features/shared.steps.saffron`) and invoke them from any feature.
 
-**IDE integration & authoring** (shipped): `saffron steps` lists the project vocabulary with recorded/divergent/unrecorded badges (`--snippets` for native VS Code completion), `saffron mcp` serves it to AI assistants, `saffron lsp` brings completion/navigation/diagnostics to **JetBrains** (incl. Community editions via LSP4IJ) and Neovim, `saffron author` drafts feature files from prose in your own vocabulary, and duplicate wordings that record identical actions get **behavior-proven rename proposals**. The **Saffron VS Code extension** lives in the sibling repo `saffron-vscode`.
+**IDE integration & authoring** (shipped): `saffron steps` lists the project vocabulary with recorded/divergent/unrecorded badges (`--snippets` for native VS Code completion), `saffron mcp` serves it to AI assistants, `saffron lsp` brings completion/navigation/diagnostics to **JetBrains** (incl. Community editions via LSP4IJ) and Neovim, `saffron author` drafts feature files from prose in your own vocabulary, and duplicate wordings that record identical actions get **behavior-proven rename proposals**. The **Saffron VS Code extension** is on the [Marketplace](https://marketplace.visualstudio.com/items?itemName=ChathurangaJayasinghe.saffron-vscode) (`code --install-extension ChathurangaJayasinghe.saffron-vscode`). JetBrains users get the same in one click from the **Saffron JetBrains plugin** (sibling repo `saffron-jetbrains`: `.saffron` file type, bundled grammar, LSP4IJ wiring; Marketplace listing pending).
 
-Data tables and doc strings are first-class: 2-column key/value tables parameterize the recording (`<table:username>`), multi-row record tables parameterize per cell (`<table:1:firstName>`), and `"""` doc strings record as `<docstring>` — so editing *values* or *content* replays at zero tokens, while structural changes (keys, headers, row counts) honestly re-record. Unambiguous params are scenario-wide, so a later assertion on a note's text follows content edits too. **Secrets** stay out of everything: `{env:VAR}` resolves from the environment (or a git-ignored `.env`) at replay, recordings and reports are masked back to the token, and missing variables fail fast by name.
+Data tables and doc strings are first-class: 2-column key/value tables parameterize the recording (`<table:username>`), multi-row record tables parameterize per cell (`<table:1:firstName>`), and `"""` doc strings record as `<docstring>`, so editing *values* or *content* replays at zero tokens, while structural changes (keys, headers, row counts) honestly re-record. Unambiguous params are scenario-wide, so a later assertion on a note's text follows content edits too. **Secrets** stay out of everything: `{env:VAR}` resolves from the environment (or a git-ignored `.env`) at replay, recordings and reports are masked back to the token, and missing variables fail fast by name.
 
 ## Install
 
 ```bash
 npm install --save-dev saffron-ai
 npx playwright install chromium
+npx saffron init        # config + the Saffron agent skill for your AI assistants
 npx saffron run
 ```
 
+**Built for the AI era:** the package ships an [Agent Skill](https://agentskills.io) that teaches Claude Code, Codex, Cursor, Copilot and any skill-aware agent to write Saffron tests efficiently. Reuse recorded wordings, keep assertions sacred, secrets as `{env:VAR}`, so the tests your agents write replay at zero tokens instead of paying for near-duplicate recordings.
+
 CI tip: `saffron run --strict` treats yellow (passed-with-adaptation) as a
-failure until its cache proposal is reviewed — cached-green-only builds.
+failure until its cache proposal is reviewed: cached-green-only builds.
 
 ## Quick start (this repo)
 
@@ -49,7 +52,7 @@ npx playwright install chromium
 # start the demo app
 node examples/demo-app/server.mjs 4173 &
 
-# replay the example suite from its committed caches — zero AI
+# replay the example suite from its committed caches, zero AI
 node dist/cli/index.js -p examples run
 
 # delete a cache and watch the agent re-record it (needs Claude Code auth
@@ -63,13 +66,14 @@ node dist/cli/index.js -p examples accept --all
 
 | Command | What it does |
 |---|---|
-| `saffron run [paths] [--headed] [--filter @tag] [--no-agent] [--strict] [--browser b] [--workers n] [--heal-model m] [--no-verify] [--no-reuse] [--model m] [--storage-state f]` | Run features. Cached replays are deterministic; misses/failures escalate to the agent (unless `--no-agent`). Replay cross-browser with `--browser firefox\|webkit`, parallelize with `--workers N`, heal on a cheaper model with `--heal-model`. Exit 1 on red (and on yellow with `--strict`). |
-| `saffron accept [file \| --all] [--with-feature-edit] [--propagate]` | Promote cache proposals to committed caches; `--with-feature-edit` also rewrites the adapted steps in the `.feature` file (and keeps the cache in sync); `--propagate` applies the heal's locator fixes to every other cache using the same locator — one heal repairs N scenarios before they ever fail. No args: list pending proposals. |
+| `saffron run [paths] [--headed] [--filter @tags] [--no-agent] [--strict] [--browser b] [--workers n] [--heal-model m] [--no-verify] [--no-reuse] [--model m] [--storage-state f]` | Run features. Cached replays are deterministic; misses/failures escalate to the agent (unless `--no-agent`). Replay cross-browser with `--browser firefox\|webkit`, parallelize with `--workers N`, heal on a cheaper model with `--heal-model`. Exit 1 on red (and on yellow with `--strict`). |
+| `saffron accept [file \| --all] [--with-feature-edit] [--propagate]` | Promote cache proposals to committed caches; `--with-feature-edit` also rewrites the adapted steps in the `.feature` file (and keeps the cache in sync); `--propagate` applies the heal's locator fixes to every other cache using the same locator. One heal repairs N scenarios before they ever fail. No args: list pending proposals. |
 | `saffron reject [file \| --all]` | Discard proposals; the agent will try again next run. |
 | `saffron steps [search] [--json] [--snippets]` | List/search the step vocabulary (files + caches) with recorded/divergent/unrecorded badges and usage; `--snippets` writes `.vscode/saffron.code-snippets` for native VS Code completion. |
-| `saffron mcp` | Serve the vocabulary to AI assistants over stdio MCP (`search_steps`, `list_step_sets`) — e.g. `claude mcp add saffron -- npx saffron mcp`. |
+| `saffron mcp` | Serve the vocabulary to AI assistants over stdio MCP (`search_steps`, `list_step_sets`), e.g. `claude mcp add saffron -- npx saffron mcp`. |
 | `saffron author <prose-file> [-o out]` | Draft a `.saffron` feature file from plain-paragraph requirements, reusing the project's step vocabulary (AI; reports how many lines are seedable). |
-| `saffron lsp` | Run the Saffron language server over stdio — JetBrains (LSP4IJ/Ultimate), Neovim, any LSP editor: badge completion, StepSet go-to-definition, hover, diagnostics. |
+| `saffron lsp` | Run the Saffron language server over stdio for JetBrains (LSP4IJ/Ultimate), Neovim, any LSP editor: badge completion, StepSet go-to-definition, hover, diagnostics. Highlighting for JetBrains comes from the bundled TextMate grammar (`textmate/saffron` in the package); run `npx saffron lsp` in a terminal for setup steps. |
+| `saffron init [--agents list]` | Make the project agent-ready: installs the bundled **Saffron agent skill** into `.claude/skills` and `.agents/skills` (Copilot/Cursor via `--agents`), registers the `saffron mcp` server in `.mcp.json` / `.cursor/mcp.json` / `.vscode/mcp.json`, adds a managed block to `AGENTS.md`/`CLAUDE.md`, scaffolds config. |
 | `saffron report` | Open the latest HTML report. |
 
 Configuration lives in `saffron.config.json` at your project root:
@@ -104,7 +108,7 @@ cached replays *and* the agent start authenticated. Generate one with
 The full execution-model diagram is on the site: https://saffron-ai.lovable.app/docs
 
 - Caches store per-step action lists with semantic targets (`role` + accessible name, `nameRegex` for volatile labels, CSS fallbacks) and `<param>` placeholders, so one cache serves every Examples row of a Scenario Outline.
-- Dynamic-content vocabulary for real-world apps: `captureText` stores displayed values (prices, counters) under a name; `expectDiffers` compares against captured values; `expectMatches`/`expectAttribute` assert patterns instead of literals; `{date±N}` templates keep date-picker locators valid across days; the replayer follows links that open new tabs; native dialogs, file uploads, drag-and-drop, and iframe-scoped elements replay on all three engines; network-aware steps wait on or assert backend API responses (`waitForResponse` / sacred `expectResponse`) with URL/method/status/body matchers — "wait for the order API to return 201" instead of a sleep.
+- Dynamic-content vocabulary for real-world apps: `captureText` stores displayed values (prices, counters) under a name; `expectDiffers` compares against captured values; `expectMatches`/`expectAttribute` assert patterns instead of literals; `{date±N}` templates keep date-picker locators valid across days; the replayer follows links that open new tabs; native dialogs, file uploads, drag-and-drop, and iframe-scoped elements replay on all three engines; network-aware steps wait on or assert backend API responses (`waitForResponse` / sacred `expectResponse`) with URL/method/status/body matchers, "wait for the order API to return 201" instead of a sleep.
 - The agent and the replayer share one browser over CDP, so healing continues from the exact page state where replay failed.
 - Every recorded action carries a page fingerprint (hash of the normalized aria snapshot) for token-free drift detection.
 
@@ -114,14 +118,14 @@ The full execution-model diagram is on the site: https://saffron-ai.lovable.app/
 .saffron/
   cache/         committed replay caches (commit these)
   proposals/     pending AI-generated cache updates (review these)
-  history.jsonl  one line per run — trends (commit recommended)
+  history.jsonl  one line per run: trends (commit recommended)
   reports/       latest.html / latest.json (gitignore these)
 features/        your .feature files
 ```
 
 ## Roadmap
 
-See the roadmap on **[the site](https://saffron-ai.lovable.app/#roadmap)** — the single source of truth for
+See the roadmap on **[the site](https://saffron-ai.lovable.app/#roadmap)**, the single source of truth for
 milestones, statuses, and decision gates. Headlines:
 action-vocabulary additions as real-world failures surface them, the
 Level-2 shared step library (gated on Level-1 divergence data), LLM-less
@@ -131,10 +135,10 @@ far is in the roadmap's "Where we are" table.
 
 ## Documentation
 
-- [Website](https://saffron-ai.lovable.app) — what Saffron is, the honesty rules, economics, quickstart
-- [Documentation](https://saffron-ai.lovable.app/docs) — the cache lifecycle, step sets & the `.saffron` dialect, the step vocabulary, and the guarantees — with diagrams
+- [Website](https://saffron-ai.lovable.app): what Saffron is, the honesty rules, economics, quickstart
+- [Documentation](https://saffron-ai.lovable.app/docs): the cache lifecycle, step sets & the `.saffron` dialect, the step vocabulary, and the guarantees, with diagrams
 - [Issues & questions](https://github.com/s-chathuranga-j/saffron-ai/issues)
 - **Docs site**: `npm run docs` builds a self-contained page at
 ## License
 
-Free to use — including commercially — under the Saffron Free Use License (see LICENSE). The source opens up later; a CLA will accompany that step.
+Free to use, including commercially, under the Saffron Free Use License (see LICENSE). The source opens up later; a CLA will accompany that step.