saffron-ai 0.7.1 → 0.8.0

package/CHANGELOG.md

@@ -4,6 +4,122 @@ All notable changes to Saffron, in one place: the `saffron-ai` runner (npm),
 the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
 JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates.
 
+## 0.8.0
+
+Released 2026-09-21.
+
+Test data moves out of the feature file: a `data/` folder, `{data:...}`
+references, list checks, `Examples` from a file, values unique per run, and
+data per environment.
+
+Runner
+
+- **Test data outside the feature file: `{data:file.key}`.** Put values in
+  JSON files under `data/` and reference them from a step, a table cell or
+  a doc string: `When I sign in as {data:users.admin.email}`. Same family
+  as `{env:VAR}`: the recording stores the token and replay reads the file,
+  so changing a value changes the next replay at zero tokens. A recording
+  that still spells out the old value goes stale instead of replaying
+  yesterday's data, and a missing file or key stops the run by name before
+  a browser opens. New config key `dataDir` (default `data`). Not for
+  secrets: data files are committed.
+- **Check a whole list in one step.** `Then the status filter should list
+  every {data:enums.OrderStatus}` records ONE assertion (`expectEach`) that
+  names the list and its container. Replay reads the list from the file, so
+  a value added to the data is checked on the next run at zero tokens, and
+  a missing one fails by name with how many were found. Values match as
+  whole text ("Paid" is not "Unpaid"), extra entries such as "All" are
+  allowed, and a closed dropdown works.
+- **`Examples` from a file**, in `.saffron` files: `Examples: {data:roles}`
+  reads `data/roles.csv` (header row = placeholders) or a JSON list of
+  records. One recording, one zero-token replay per row; a row added to the
+  file records nothing. A placeholder without a column, a missing file and a
+  `.feature` file using it are each parse errors that say what to do.
+- **`{unique:name}`**: a value that is new on every run and the same within
+  one, for scenarios that create things. The agent types the generated
+  value while recording and Saffron turns it back into the token; a run that
+  is not green lists what each token stood for. `{unique:name:digits}` for
+  numeric fields.
+- **Data per environment**: `--env staging` (or `SAFFRON_ENV`, or `env` in
+  the config) lays `data/users.staging.json` over `data/users.json`, key by
+  key. The reference in the step stays the same on every target.
+- `saffron status --json` gains `data`: the folder, the environment, the
+  files, and references that do not resolve. `saffron lsp` completes
+  `{data:` paths, shows values on hover, jumps to the key and flags
+  unresolved references. The bundled grammar highlights `{data:...}` and
+  `{unique:...}`. `saffron init --examples` installs `data/catalog.json` and
+  a scenario that uses it, and the agent skill, its syntax reference and the
+  AGENTS.md block teach the new syntax.
+
+- **`saffron prune` covers failure screenshots.** A deleted scenario also
+  leaves its screenshot folder behind under `.saffron/artifacts/`. Prune
+  lists those folders with the caches and proposals and removes them with
+  `--yes`; a live scenario's folder is kept, per Examples row. `saffron
+  status --json` reports them as orphans of kind `artifact`, and both IDE
+  panels (VS Code 0.2.6, JetBrains 0.2.3) show them.
+
+## 0.7.2
+
+Released 2026-09-21.
+
+Runner
+
+- **Failure screenshots.** When a step fails, Saffron saves a screenshot of
+  the tab it failed in and shows it in the HTML report, embedded so the
+  report stays one file. A healed scenario keeps the picture of the moment
+  its cached step stopped working, for whoever reviews the proposal; a
+  scenario still red after the agent ran is photographed again at the end.
+  `latest.json` carries the paths under `evidence`. New config key
+  `screenshot`: `viewport` (default), `full-page` or `off`. Files live in
+  `.saffron/artifacts/`, latest run only, and `saffron init` git-ignores
+  it: **add `.saffron/artifacts/` to your `.gitignore` if your project was
+  initialised earlier.** A screenshot shows whatever your page shows,
+  including a secret it displays; `{env:VAR}` masking cannot reach pixels.
+
+- **A feature file that fails to parse says which file, and why.** Invalid
+  Gherkin reached the terminal as a raw stack trace with positions and no
+  file name. It is now one line per problem, `file:line:column`. A standard
+  `.feature` file that uses `StepSet` gets the actual answer instead of a
+  parser dump: rename it to `.saffron`, nothing else needs to change.
+
+- `saffron status --json` gains `lastRun.attention`: every scenario of the
+  last run that was not green, red first, with the failed step, the first
+  line of the error and the screenshots that still exist on disk.
+
+Editors (VS Code extension 0.2.6, JetBrains plugin 0.2.3)
+
+- **`{data:...}` in the editor.** VS Code completes data files and keys with
+  their values, shows the current value (or a list's values, or an Examples
+  file's columns) on hover, jumps to the key, and underlines a reference
+  that does not resolve, reading the same files and environment overlay the
+  runner does. JetBrains gets the same through `saffron lsp`. Both highlight
+  `{data:...}` and `{unique:...}` like `{env:VAR}`, including on an
+  `Examples:` line.
+- **Last run, with the screenshot one click away.** Both IDEs list what the
+  last run left to look at: failed scenarios, then healed ones, with the
+  step that failed. VS Code opens the screenshot in its image preview on
+  click; JetBrains opens it in the image viewer on double-click. Each also
+  jumps to the scenario. With a runner that predates failure screenshots,
+  both say so instead of reporting that everything passed.
+
+Docs
+
+- **A page for "`.saffron` and `.feature` files"**: what each is, a
+  side-by-side table, which to write, and what renaming a file means for its
+  recordings. The rest of the docs say "feature file" for both, where they
+  used to say `.feature` alone.
+
+Editors (JetBrains plugin 0.2.2)
+
+- **LSP4IJ is an optional dependency.** A first upload of 0.2.2 was rejected
+  by the Marketplace verifier on 2026.3 EAP because LSP4IJ had no release
+  for that build yet, and a mandatory dependency that cannot be resolved
+  blocks the whole plugin. Highlighting, run configurations and the tool
+  window now work without it; completion, go-to-definition, hover and
+  diagnostics switch on when LSP4IJ is installed, and a one-time
+  notification says so when it is missing. The presence check uses no
+  plugin-manager API, so it stays clean as that API turns internal.
+
 ## 0.7.1
 
 Released 2026-09-20.
@@ -588,7 +704,7 @@ Released 2026-07-10.
 First public release, free for any use under the Saffron Free Use License.
 
 - Gherkin-native runner: an AI agent records each scenario once; later runs
-  replay with plain Playwright at zero tokens; runtime healing files
+  are replayed by Saffron in a real browser at zero tokens; runtime healing files
   reviewable proposals; assertions are never healed.
 - Verified proposals (zero-AI proof replay), honest cost reporting, run
   history and trends, chronic-scenario detection.

package/README.md

@@ -2,9 +2,9 @@
 
 **Gherkin-native test runner with zero-token cached replay and runtime AI healing.**
 
-Free to use · [npm: saffron-ai](https://www.npmjs.com/package/saffron-ai) · [site](https://saffron-ai.lovable.app)
+Free to use · [npm: saffron-ai](https://www.npmjs.com/package/saffron-ai) · [site](https://saffron-ai.io)
 
-Write tests as plain `.feature` files. On the first run, an AI agent (Claude Agent SDK + Playwright MCP) executes each scenario in a real browser and records what it did into a **cache**. A human-reviewable JSON action list committed to git. Every run after that replays the cache with plain Playwright: **zero AI calls, zero tokens, ~100ms per scenario**.
+Write tests as Gherkin feature files: `.saffron`, Saffron's superset of Gherkin with step sets, or standard `.feature`. On the first run, an AI agent (Claude Agent SDK + Playwright MCP) executes each scenario in a real browser and records what it did into a **cache**. A human-reviewable JSON action list committed to git. Every run after that, Saffron replays the cache in a real browser: **zero AI calls, zero tokens**, at the speed of the browser and your app. The cache is Saffron's own JSON, not a generated script; replay runs on Playwright's browser engine, and Playwright MCP is only how the agent operates the browser while it records or heals.
 
 When the UI changes and a cached step fails at runtime, the agent takes over mid-execution with full scenario context, adapts, and finishes the run, then files a **cache proposal** you accept or reject, like a snapshot test update. Reports state exactly what was adapted and what it cost.
 
@@ -13,7 +13,7 @@ When the UI changes and a cached step fails at runtime, the agent takes over mid
 - **Assertions are sacred.** By default no `Then` is ever healed: the AI may help *reach* an assertion, never make it pass; enforced mechanically, not by prompt. Projects may opt into `adaptable-mid` for mid-scenario checkpoints, but the **final assertion block** of a scenario is strict under every policy, forever.
 - **Three result states.** Green = cached pass. Yellow = passed with AI adaptation (pending your review). Red = failed.
 - **Diagnose before adapting.** On a step failure the agent first decides: UI drift (heal) or application defect (fail with a diagnosis).
-- **Caches are git artifacts.** Proposals show diffs, including a suggested `.feature` edit when the written steps no longer match reality, and nothing is committed without you.
+- **Caches are git artifacts.** Proposals show diffs, including a suggested feature-file edit when the written steps no longer match reality, and nothing is committed without you.
 - **Verified proposals.** Every recording/heal is proof-replayed zero-AI before it's filed (with one bounded refinement pass on failure), so proposals arrive stamped `verified ✓` or honestly `UNVERIFIED ✗`.
 - **Honest cost reporting.** Reports break out prompt-cache reads/writes (the real bill of agent sessions) next to the in+out token count, per scenario and in totals, and name the models used. On an API key the dollars are what was billed; on a Claude subscription the headline is the 5-hour plan window before and after the run, with the dollars as the API-equivalent.
 - **Trend memory.** Every run appends to `.saffron/history.jsonl`; reports show deltas vs the previous run and 20-run sparklines, and Saffron flags **chronic scenarios** (healing repeatedly. Re-record instead of paying again) plus recurring failure themes.
@@ -89,9 +89,11 @@ Configuration lives in `saffron.config.json` at your project root:
 {
   "baseURL": "http://localhost:4173",
   "features": "features",
+  "dataDir": "data",
   "actionTimeoutMs": 5000,
   "pollIntervalMs": 100,
   "retries": 1,
+  "screenshot": "viewport",
   "model": "claude-sonnet-5",
   "maxTurns": 100,
   "storageState": ".auth/state.json",
@@ -113,7 +115,7 @@ cached replays *and* the agent start authenticated. Generate one with
 
 ## How it works
 
-The full execution-model diagram is on the site: https://saffron-ai.lovable.app/docs
+The full execution-model diagram is on the site: https://saffron-ai.io/docs
 
 - Caches store per-step action lists with semantic targets (`role` + accessible name, `nameRegex` for volatile labels, CSS fallbacks) and `<param>` placeholders, so one cache serves every Examples row of a Scenario Outline.
 - Dynamic-content vocabulary for real-world apps: `captureText` stores displayed values (prices, counters) under a name; `expectDiffers` compares against captured values; `expectMatches`/`expectAttribute` assert patterns instead of literals; `{date±N}` templates keep date-picker locators valid across days; the replayer follows links that open new tabs, and multi-tab scenarios are plain prose ("switch back to the first tab", "close the preview tab", "open a new tab at …") recorded as tab actions; native dialogs, file uploads, drag-and-drop, and iframe-scoped elements replay on all three engines; network-aware steps wait on or assert backend API responses (`waitForResponse` / sacred `expectResponse`) with URL/method/status/body matchers, "wait for the order API to return 201" instead of a sleep.
@@ -128,12 +130,13 @@ The full execution-model diagram is on the site: https://saffron-ai.lovable.app/
   proposals/     pending AI-generated cache updates (review these)
   history.jsonl  one line per run: trends (commit recommended)
   reports/       latest.html / latest.json (gitignore these)
-features/        your .feature files
+  artifacts/     failure screenshots from the latest run (gitignore these)
+features/        your .saffron and .feature files
 ```
 
 ## Roadmap
 
-See the roadmap on **[the site](https://saffron-ai.lovable.app/#roadmap)**, the single source of truth for
+See the roadmap on **[the site](https://saffron-ai.io/#roadmap)**, the single source of truth for
 milestones, statuses, and decision gates. Headlines:
 action-vocabulary additions as real-world failures surface them, the
 Level-2 shared step library (gated on Level-1 divergence data), LLM-less
@@ -143,10 +146,10 @@ far is in the roadmap's "Where we are" table.
 
 ## Documentation
 
-- [Website](https://saffron-ai.lovable.app): what Saffron is, the honesty rules, economics, quickstart
-- [Documentation](https://saffron-ai.lovable.app/docs): the cache lifecycle, step sets & the `.saffron` dialect, the step vocabulary, and the guarantees, with diagrams
-- [Changelog](CHANGELOG.md), also at https://saffron-ai.lovable.app/docs/changelog
-- [Issues & questions](https://github.com/s-chathuranga-j/saffron-ai/issues)
+- [Website](https://saffron-ai.io): what Saffron is, the honesty rules, economics, quickstart
+- [Documentation](https://saffron-ai.io/docs): the cache lifecycle, step sets & the `.saffron` dialect, the step vocabulary, and the guarantees, with diagrams
+- [Changelog](CHANGELOG.md), also at https://saffron-ai.io/docs/changelog
+- [Issues & questions](https://github.com/s-chathuranga-j/saffron-community/issues)
 - **Docs site**: `npm run docs` builds a self-contained page at
 ## License