saffron-ai 0.8.3 → 0.8.5

package/CHANGELOG.md

@@ -4,6 +4,158 @@ All notable changes to Saffron, in one place: the `saffron-ai` runner (npm),
 the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
 JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates.
 
+## 0.8.5
+
+Released 2026-09-24.
+
+Runner
+
+- **A proposal is bound to what you reviewed.** Every proposal now carries
+  digests of the scenario's steps, of the committed recording it was
+  diffed against and of its own recording, taken when it was filed.
+  `saffron accept` recomputes them and refuses a proposal, by reason, when
+  any moved since: a step edited, a colleague's heal accepted, a locator
+  fix propagated, a pull, a hand edit ("step 3 now reads ... and was
+  ..."). There is no override; run the scenario again for a proposal
+  bound to what is there now, or reject it. `saffron accept`, `saffron
+  diff`, `saffron status --json` (`stale`) and both IDE panels show STALE
+  with the reason first. A run that finds a stale pending proposal records
+  afresh instead of proving it again. Fingerprints are advisory and are
+  not bound; data tables, doc strings and the feature edits that
+  `--with-feature-edit` would apply are. Proposals filed before this
+  release carry no digests and are accepted with a note.
+- **Recordings and proposals are validated on read.** The schema version,
+  every step, every action type and field are checked, along with the
+  operands each action needs (a click without a usable target, an
+  `expectUrl` without a URL, which used to compare "" and pass), and the
+  first problem is named by path. An unreadable recording makes its scenario red
+  with the reason instead of replaying as garbage or being recorded over;
+  `saffron status` marks it `invalid` with the reason (`cacheProblem`);
+  an unreadable proposal is reported and left out of listings
+  (`unreadableProposals`) instead of blanking them. A file from a newer
+  runner, a schema version or an action type this one does not know, says
+  to update `saffron-ai`. Fields added within the schema version stay
+  optional, so newer caches read on older runners. Hand-written caches may
+  omit `recordedAt` and `recordedBy`.
+
+Editors (VS Code extension 0.2.10, JetBrains plugin 0.2.5)
+
+- Both Proposals panels mark a stale proposal STALE, with the reason and
+  what to do, and note a proposal filed by an older runner.
+
+## 0.8.4
+
+Released 2026-09-24.
+
+Runner
+
+- **A check could point at the wrong element.** In a live heal the agent
+  recorded "the Morning Ritual package is applied" against `role=definition`
+  with no name. The booking summary has one definition per row, so replay
+  read the first one (the room) and the proof replay failed. `saffron_assert`
+  and `saffron_capture` now refuse a bare role unless it is one that occurs
+  once per page (`status`, `alert`, `dialog`, `main` and similar), and ask
+  the agent for a name, text, label or selector instead.
+- **A baseURL's query string and path were lost on replay.** With
+  `baseURL` set to `http://localhost:5173/?flow=checkout-guest`, a recorded
+  visit to the start page replayed as `http://localhost:5173/`, silently
+  dropping the flag that picks the variant under test. A baseURL with a
+  path fared no better: under `http://host/app`, a cached `/rooms`
+  replayed as `http://host/rooms`, and a recorded `http://host/apple` was
+  cached as `/le`. Under `http://h/#/home`, a recorded visit to `/` with
+  no hash replayed back to `#/home`, a different page for a hash router;
+  it is now kept as `/#`, the way a missing query is kept as `?`.
+  The baseURL is now a prefix in both directions: cached
+  navigations keep its path and inherit its query, so a cache recorded
+  against `?flow=standard` replays against `--base-url
+  "...?flow=checkout-guest"`. A cached URL with a query of its own keeps
+  it, other origins stay untouched, and existing caches replay as before.
+- **Heal summaries could describe a changed flow as a locator tweak.**
+  When an A/B variant folded the guest information page into checkout,
+  the heal was right (fields re-targeted, the "Continue to payment" click
+  dropped) but its note only said the selects were now found by label.
+  The heal prompt now asks the agent to name the kind of change (a locator
+  or label change, a page removed or merged, a page added or reordered)
+  and the page it happened on. Saffron also compares the healed recording
+  with the committed one and adds its own `flow change:` notes for any
+  click, key press, goto or tab move that was dropped or added, together
+  with the page replay stopped on. A renamed button still reads as a
+  locator change and adds no note.
+- **Two data references holding the same value could bind a field to the
+  wrong one.** With `users.owner` and `users.recipient` both "Alice", the
+  recorder tokenized every typed "Alice" as the first reference, so a
+  recipient field read the owner. Change the recipient to "Bob" and the
+  scenario correctly went stale, but step reuse seeded the mis-bound
+  actions straight back and the proof replay agreed: verified, zero AI
+  calls, wrong field. The recorder now resolves an equal value by the
+  reference the step itself names, leaves the literal when the step names
+  both or neither (so the stale check catches it), and step reuse refuses
+  a step that names a reference its recording does not read.
+- **The report could list `{unique:...}` values that were never typed.**
+  A recording seeded from earlier steps, and the proof replay of a pending
+  proposal, each drew their own seed while the report showed the values of
+  a third. One seed per scenario run now reaches every replay, recording
+  and proof, so the report's values are the ones in your application.
+- **A second Ctrl-C during teardown exited before the cleanup finished.**
+  The first Ctrl-C started teardown; the next one fell through to Node's
+  default and killed the run on top of it. Saffron now answers it with a
+  note and keeps waiting.
+- **Re-merging shards after a shard was re-run left the old result in
+  history.** The merge treated the run as already recorded and skipped it.
+  It now replaces that run's history line, and still never counts a run
+  twice. In CI the run is identified by the CI run id, the suite and the
+  base URL rather than by the earliest start time, so this holds whichever
+  shard was re-run; outside CI, `SAFFRON_RUN_ID` gives the same stability.
+- **The proof replay of a recording reused the recording's `{unique:...}`
+  values.** Fixed alongside the report fix above: the proof is a second
+  execution, so creating the same record again failed, filed the proposal
+  unverified and paid for a refinement pass. Each proof now draws its own
+  values, and the report lists them separately from the recording's.
+- **A heal or a refinement could redo earlier steps on the wrong tab and
+  file them under step 1.** When the agent takes over, it attaches to the
+  browser over CDP and sees every Saffron page as one tab list, starting
+  on the first. During a refinement the scenario's own page was still open
+  beside the proof replay's, so the agent started on the wrong page,
+  selected a tab, and in one run went back to the start page and redid the
+  search. Every action performed before its first `saffron_step` was filed
+  under step 1, and the merge adopted it over the recording replay had
+  already passed: proposals whose step 1 held only "switch to tab 1", or a
+  second copy of the search, and a proof replay that failed on them.
+  Saffron now closes the pages of other browser contexts before an agent
+  session, so the agent sees only the live tabs, and tells it which tab
+  replay left off on: the page replay was really acting on, not the newest
+  tab with a matching URL, which is a different page when two tabs share
+  one. A heal or refinement records nothing until the agent
+  announces the failed step: anything it clicks or navigates before that is
+  diagnosis and is discarded, and the prompt tells it to look with
+  snapshots instead. A seeded recording segment files early actions under
+  the first step it may record, never a step that already ran, and drops
+  an early tab selection. Actions the agent files under an already
+  completed step are discarded, and the merge keeps completed steps'
+  recordings whatever the session holds for them.
+  Each of these leaves a `recording:` note in the proposal's adaptations.
+  Fresh recordings from step 1 behave as before.
+  The live tab also follows the agent through a seeded recording: a tab
+  it selects, opens or closes in one segment is where the next seeded
+  island replays and the next segment starts, read from the browser
+  tool's own tab list rather than guessed from a URL.
+- **A role with a text no longer matched any element showing that text.**
+  Role and text together now mean the element of that role that shows the
+  text, and nothing else.
+- **Shards from different environments merged silently.** A staging shard
+  and a production shard combined into one report labelled with whichever
+  came first. The merge now refuses shards with different base URLs, by
+  name.
+
+Editors (VS Code extension 0.2.9)
+
+- **`{data:...}` diagnostics stayed stale after editing a data file.** The
+  extension watched feature files and caches but not the data folder or
+  `saffron.config.json`, so fixing a missing key in `users.json` left the
+  error on the feature until the feature itself was touched. Edits to data
+  files and to the config now refresh every open feature file, including a
+  custom data folder and environment overlays.
+
 ## 0.8.3
 
 Released 2026-09-22.