saffron-ai 0.7.0 → 0.7.2

package/CHANGELOG.md

@@ -4,6 +4,132 @@ All notable changes to Saffron, in one place: the `saffron-ai` runner (npm),
 the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
 JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates.
 
+## 0.7.2
+
+Released 2026-09-21.
+
+Runner
+
+- **Failure screenshots.** When a step fails, Saffron saves a screenshot of
+  the tab it failed in and shows it in the HTML report, embedded so the
+  report stays one file. A healed scenario keeps the picture of the moment
+  its cached step stopped working, for whoever reviews the proposal; a
+  scenario still red after the agent ran is photographed again at the end.
+  `latest.json` carries the paths under `evidence`. New config key
+  `screenshot`: `viewport` (default), `full-page` or `off`. Files live in
+  `.saffron/artifacts/`, latest run only, and `saffron init` git-ignores
+  it: **add `.saffron/artifacts/` to your `.gitignore` if your project was
+  initialised earlier.** A screenshot shows whatever your page shows,
+  including a secret it displays; `{env:VAR}` masking cannot reach pixels.
+
+- **A feature file that fails to parse says which file, and why.** Invalid
+  Gherkin reached the terminal as a raw stack trace with positions and no
+  file name. It is now one line per problem, `file:line:column`. A standard
+  `.feature` file that uses `StepSet` gets the actual answer instead of a
+  parser dump: rename it to `.saffron`, nothing else needs to change.
+
+- `saffron status --json` gains `lastRun.attention`: every scenario of the
+  last run that was not green, red first, with the failed step, the first
+  line of the error and the screenshots that still exist on disk.
+
+Editors (VS Code extension 0.2.6, JetBrains plugin 0.2.3)
+
+- **Last run, with the screenshot one click away.** Both IDEs list what the
+  last run left to look at: failed scenarios, then healed ones, with the
+  step that failed. VS Code opens the screenshot in its image preview on
+  click; JetBrains opens it in the image viewer on double-click. Each also
+  jumps to the scenario. With a runner that predates failure screenshots,
+  both say so instead of reporting that everything passed.
+
+Docs
+
+- **A page for "`.saffron` and `.feature` files"**: what each is, a
+  side-by-side table, which to write, and what renaming a file means for its
+  recordings. The rest of the docs say "feature file" for both, where they
+  used to say `.feature` alone.
+
+Editors (JetBrains plugin 0.2.2)
+
+- **LSP4IJ is an optional dependency.** A first upload of 0.2.2 was rejected
+  by the Marketplace verifier on 2026.3 EAP because LSP4IJ had no release
+  for that build yet, and a mandatory dependency that cannot be resolved
+  blocks the whole plugin. Highlighting, run configurations and the tool
+  window now work without it; completion, go-to-definition, hover and
+  diagnostics switch on when LSP4IJ is installed, and a one-time
+  notification says so when it is missing. The presence check uses no
+  plugin-manager API, so it stays clean as that API turns internal.
+
+## 0.7.1
+
+Released 2026-09-20.
+
+Runner
+
+- **`expectUrl` checks the active page, not any open tab.** A URL sitting in
+  a leftover tab could satisfy the assertion for the wrong reason. *A
+  recording made before tab openers were marked, with a slow popup and
+  `retries: 0`, can now fail where it used to pass: re-record it, or leave
+  the default retry, which re-follows the newest tab.*
+- **A changed table value only replays if the recording really uses it.**
+  Matching key sets used to be enough, so a recording that still carried the
+  old value as a literal, or never referenced the cell, stayed "valid" and
+  typed stale data. The old value is looked for only where a value can live
+  (typed, matched or named text, with tokens removed), so an empty cell or a
+  short value like "on" does not force a needless re-record.
+- **Propagation never rewrites an assertion**, on either side: assertion
+  steps and assertion actions are skipped when a fix is extracted and when
+  it is applied, the step wording must match, and a locator inside a frame
+  is a different locator. When one heal sends the same locator to more than
+  one place, nothing propagates and `saffron accept` says so.
+- **`saffron prune` and `saffron status` respect features outside the
+  configured directory.** A scenario run as `saffron run e2e/login.feature`
+  had its recordings listed as orphans. Feature files named by existing
+  recordings are now inspected too, and one that fails to parse blocks the
+  prune like any other.
+- **Step reuse no longer brings back a stale table value.** Changing `alice`
+  to `bob` correctly invalidated the scenario's cache, and step reuse then
+  seeded the same actions straight back, matching on table keys alone: a
+  verified proposal at zero AI calls whose table said `bob` while its
+  actions still typed and asserted `alice`. Seeds now pass the same rule as
+  cache validation, and a step with no table of its own is refused when it
+  carries another step's old value as a literal. The old value is looked
+  for in locators built from data as well (a test id like `user-alice`,
+  fallback selectors, a frame, a response URL pattern), not only in typed
+  and named text: a fill can be parameterized while the assertion beside it
+  still targets the old user.
+- **A feature edit cannot land on the wrong step.** Edits address steps by
+  index, so a step inserted while a proposal waited shifted every index
+  after it, and an edit meant for "I click login" rewrote the new step.
+  `saffron accept --with-feature-edit` now compares the scenario with the
+  steps the proposal was recorded against and refuses before anything is
+  consumed: the file, the cache and the proposal stay as they were.
+- **A skipped feature edit no longer reaches the cache.** When one edit
+  applied, the accepted cache was given the new text of every edit,
+  including a Background edit that was correctly skipped, so the feature and
+  its cache disagreed at once. Only edits that landed are synced. Two edits
+  that reach the same StepSet line through two invocations are applied once
+  when they agree, and both refused when they do not, naming the two
+  wordings: the line can only say one thing.
+- `saffron status` parses each feature file once. The vocabulary and the
+  orphan scan each parsed the whole project again; both now reuse what
+  status already read. IDE panels call status on every refresh.
+
+Editors (VS Code extension 0.2.5)
+
+- **Completion badges and duplicate-step diagnostics follow cache creation
+  and deletion.** Only changes to existing JSON files were watched, so
+  accepting a first recording or pruning one left the vocabulary stale until
+  something else triggered a rebuild. Recordings are watched for all three
+  events (and only recordings, not every JSON file), and the watcher is
+  disposed with the extension.
+- **Vocabulary is per project in a multi-root workspace.** Step sets,
+  wording and recorded badges were built once for the whole workspace, so
+  project B's `Login` step set could answer go-to-definition in project A,
+  and a step recorded in A showed as recorded in B.
+- **The inline diff button on a proposal row works.** An inline action
+  passes the tree item, not the file path the row click passes; both forms
+  are accepted, and the project guard applies to both.
+
 ## 0.7.0
 
 Released 2026-09-19.
@@ -517,7 +643,7 @@ Released 2026-07-10.
 First public release, free for any use under the Saffron Free Use License.
 
 - Gherkin-native runner: an AI agent records each scenario once; later runs
-  replay with plain Playwright at zero tokens; runtime healing files
+  are replayed by Saffron in a real browser at zero tokens; runtime healing files
   reviewable proposals; assertions are never healed.
 - Verified proposals (zero-AI proof replay), honest cost reporting, run
   history and trends, chronic-scenario detection.

package/README.md

@@ -2,9 +2,9 @@
 
 **Gherkin-native test runner with zero-token cached replay and runtime AI healing.**
 
-Free to use · [npm: saffron-ai](https://www.npmjs.com/package/saffron-ai) · [site](https://saffron-ai.lovable.app)
+Free to use · [npm: saffron-ai](https://www.npmjs.com/package/saffron-ai) · [site](https://saffron-ai.io)
 
-Write tests as plain `.feature` files. On the first run, an AI agent (Claude Agent SDK + Playwright MCP) executes each scenario in a real browser and records what it did into a **cache**. A human-reviewable JSON action list committed to git. Every run after that replays the cache with plain Playwright: **zero AI calls, zero tokens, ~100ms per scenario**.
+Write tests as Gherkin feature files: `.saffron`, Saffron's superset of Gherkin with step sets, or standard `.feature`. On the first run, an AI agent (Claude Agent SDK + Playwright MCP) executes each scenario in a real browser and records what it did into a **cache**. A human-reviewable JSON action list committed to git. Every run after that, Saffron replays the cache in a real browser: **zero AI calls, zero tokens**, at the speed of the browser and your app. The cache is Saffron's own JSON, not a generated script; replay runs on Playwright's browser engine, and Playwright MCP is only how the agent operates the browser while it records or heals.
 
 When the UI changes and a cached step fails at runtime, the agent takes over mid-execution with full scenario context, adapts, and finishes the run, then files a **cache proposal** you accept or reject, like a snapshot test update. Reports state exactly what was adapted and what it cost.
 
@@ -13,7 +13,7 @@ When the UI changes and a cached step fails at runtime, the agent takes over mid
 - **Assertions are sacred.** By default no `Then` is ever healed: the AI may help *reach* an assertion, never make it pass; enforced mechanically, not by prompt. Projects may opt into `adaptable-mid` for mid-scenario checkpoints, but the **final assertion block** of a scenario is strict under every policy, forever.
 - **Three result states.** Green = cached pass. Yellow = passed with AI adaptation (pending your review). Red = failed.
 - **Diagnose before adapting.** On a step failure the agent first decides: UI drift (heal) or application defect (fail with a diagnosis).
-- **Caches are git artifacts.** Proposals show diffs, including a suggested `.feature` edit when the written steps no longer match reality, and nothing is committed without you.
+- **Caches are git artifacts.** Proposals show diffs, including a suggested feature-file edit when the written steps no longer match reality, and nothing is committed without you.
 - **Verified proposals.** Every recording/heal is proof-replayed zero-AI before it's filed (with one bounded refinement pass on failure), so proposals arrive stamped `verified ✓` or honestly `UNVERIFIED ✗`.
 - **Honest cost reporting.** Reports break out prompt-cache reads/writes (the real bill of agent sessions) next to the in+out token count, per scenario and in totals, and name the models used. On an API key the dollars are what was billed; on a Claude subscription the headline is the 5-hour plan window before and after the run, with the dollars as the API-equivalent.
 - **Trend memory.** Every run appends to `.saffron/history.jsonl`; reports show deltas vs the previous run and 20-run sparklines, and Saffron flags **chronic scenarios** (healing repeatedly. Re-record instead of paying again) plus recurring failure themes.
@@ -92,6 +92,7 @@ Configuration lives in `saffron.config.json` at your project root:
   "actionTimeoutMs": 5000,
   "pollIntervalMs": 100,
   "retries": 1,
+  "screenshot": "viewport",
   "model": "claude-sonnet-5",
   "maxTurns": 100,
   "storageState": ".auth/state.json",
@@ -113,7 +114,7 @@ cached replays *and* the agent start authenticated. Generate one with
 
 ## How it works
 
-The full execution-model diagram is on the site: https://saffron-ai.lovable.app/docs
+The full execution-model diagram is on the site: https://saffron-ai.io/docs
 
 - Caches store per-step action lists with semantic targets (`role` + accessible name, `nameRegex` for volatile labels, CSS fallbacks) and `<param>` placeholders, so one cache serves every Examples row of a Scenario Outline.
 - Dynamic-content vocabulary for real-world apps: `captureText` stores displayed values (prices, counters) under a name; `expectDiffers` compares against captured values; `expectMatches`/`expectAttribute` assert patterns instead of literals; `{date±N}` templates keep date-picker locators valid across days; the replayer follows links that open new tabs, and multi-tab scenarios are plain prose ("switch back to the first tab", "close the preview tab", "open a new tab at …") recorded as tab actions; native dialogs, file uploads, drag-and-drop, and iframe-scoped elements replay on all three engines; network-aware steps wait on or assert backend API responses (`waitForResponse` / sacred `expectResponse`) with URL/method/status/body matchers, "wait for the order API to return 201" instead of a sleep.
@@ -128,12 +129,13 @@ The full execution-model diagram is on the site: https://saffron-ai.lovable.app/
   proposals/     pending AI-generated cache updates (review these)
   history.jsonl  one line per run: trends (commit recommended)
   reports/       latest.html / latest.json (gitignore these)
-features/        your .feature files
+  artifacts/     failure screenshots from the latest run (gitignore these)
+features/        your .saffron and .feature files
 ```
 
 ## Roadmap
 
-See the roadmap on **[the site](https://saffron-ai.lovable.app/#roadmap)**, the single source of truth for
+See the roadmap on **[the site](https://saffron-ai.io/#roadmap)**, the single source of truth for
 milestones, statuses, and decision gates. Headlines:
 action-vocabulary additions as real-world failures surface them, the
 Level-2 shared step library (gated on Level-1 divergence data), LLM-less
@@ -143,10 +145,10 @@ far is in the roadmap's "Where we are" table.
 
 ## Documentation
 
-- [Website](https://saffron-ai.lovable.app): what Saffron is, the honesty rules, economics, quickstart
-- [Documentation](https://saffron-ai.lovable.app/docs): the cache lifecycle, step sets & the `.saffron` dialect, the step vocabulary, and the guarantees, with diagrams
-- [Changelog](CHANGELOG.md), also at https://saffron-ai.lovable.app/docs/changelog
-- [Issues & questions](https://github.com/s-chathuranga-j/saffron-ai/issues)
+- [Website](https://saffron-ai.io): what Saffron is, the honesty rules, economics, quickstart
+- [Documentation](https://saffron-ai.io/docs): the cache lifecycle, step sets & the `.saffron` dialect, the step vocabulary, and the guarantees, with diagrams
+- [Changelog](CHANGELOG.md), also at https://saffron-ai.io/docs/changelog
+- [Issues & questions](https://github.com/s-chathuranga-j/saffron-community/issues)
 - **Docs site**: `npm run docs` builds a self-contained page at
 ## License