saffron-ai 0.9.2 → 0.9.3

package/CHANGELOG.md

@@ -2,9 +2,155 @@
 
 All notable changes to Saffron, in one place: the `saffron-ai` runner (npm),
 the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
-JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates.
+JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
+release heading names its main feature: `## <version>: <title>`.
 
-## 0.9.2
+## 0.9.3: Review CI heals on your machine
+
+Released 2026-10-04.
+
+Runner
+
+- **A heal made in CI can be reviewed and accepted on your machine.**
+  `saffron export` packs the pending proposals, the run's report and the
+  screenshots and traces it names into one zip, with where the run
+  happened (repository, branch, commit, CI run, shard); CI uploads it as
+  an artifact. `saffron import` checks it against your checkout and files
+  it where a local run would have, so `saffron diff`, `saffron accept`,
+  `saffron trace`, the report and both IDE panels work on it unchanged. A
+  bundle from another repository (by its path; the same path under
+  another host name, such as an SSH host alias, imports with a warning),
+  one that does not say its repository (unless
+  `--allow-unknown-repository`), of a newer format, with a damaged file or
+  a report it cannot show, or with a file over 1 GB (or over 100 times
+  its size unzipped) is refused whole, before anything is written, and so
+  are bundles that add up to over 8 GB unzipped together; a proposal made
+  against a different scenario text or committed recording is refused by
+  name, and the rest import. Bundles are read from disk one file at a
+  time, a URL's downloaded to disk first, so every shard of a large run
+  imports at once. Text from a bundle, in errors and reasons too, is
+  printed without control characters. Export keeps to the same limits:
+  it refuses a file over 1 GB and, when a bundle would be over 4 GB,
+  leaves the traces of green scenarios out first and says so. Shard
+  bundles merge into one report. The imported report says where it came
+  from and links to the CI run. Examples for GitHub Actions and Azure
+  Pipelines are in the guide (§19).
+- **Or CI opens a draft pull request with the heals.**
+  `saffron export --branch <name>` commits the proposal files, not the
+  recordings, onto the commit the run tested (on a pull request build, the
+  head of its branch, where each proposal's binding is checked again and
+  one that does not fit is left out, judged by that commit's own StepSets
+  and data) as a new branch, pushes it and, on GitHub, opens a
+  draft pull request through `gh` whose description is each proposal's
+  verification and diff, with the commands to accept, and a staging line
+  that stages exactly what accept changed (`git add -A .saffron/cache &&
+  git add -u .saffron/proposals`). After a tag build, which tested no
+  branch, name the pull request's branch with `--base`. Only the
+  checkout's HEAD, a commit it merged in or one it descends from is ever
+  built on; a `pull_request_target` build records the base branch and the
+  commit it checked out, never the fork's. The reviewer
+  switches to the branch, runs `saffron accept`, commits and marks it
+  ready: no agent session on their machine. The CI checkout is left as it
+  was, and an existing branch is never pushed onto.
+- **`saffron status --json` says which CI run a proposal was imported
+  from** (`importedFrom`: repository, branch, commit, run and its link,
+  shard), for as long as it is the proposal that was imported, so the
+  editors can show it past the next local run.
+- **`saffron runs` and `saffron import --run <id>`, for GitHub Actions and
+  Azure Pipelines alike.** `runs` lists the branch's CI runs (through `gh`,
+  or `az` with its azure-devops extension), with how many of each run's
+  proposals are pending here; `import --run` downloads a run's bundles and
+  imports them. The repository's host picks the service; a `ci` key in the
+  config sets it, and the Azure DevOps organization and project, for a
+  GitHub repository built by Azure Pipelines. An Azure pull request build
+  is listed under the branch it validates, and only the repository's own
+  builds are listed. Bundles of a re-run take each shard from its latest
+  attempt. `import --run` downloads into a folder under `.saffron/reports`
+  and removes it when done. A download that takes over 10 minutes is
+  stopped and says so. A tag build is listed under its tag on both
+  services. An Azure DevOps SSH remote through a host alias counts as
+  Azure. With `--json`, every `import` failure, an invalid config
+  included, is `{"error": ...}` and exit 2. A `runs --json` whose `gh` or
+  `az` is missing, signed out or failing answers `{provider, branch,
+  runs: [], problem: {kind, message}, error}`, `error` holding the same
+  message, so a client reading either field works; any other failure is
+  `{"error": ...}` alone; both exit 2. On Azure Pipelines the pull
+  request builds are asked for once (`--top 500`), so a listing usually takes
+  three `az` calls, not seven.
+  `export --branch` opens the draft pull request on Azure DevOps too,
+  through `az repos pr create`.
+- **Export streams the bundle to disk.** Screenshots and traces are
+  copied into the zip in chunks, never all held in memory, and the zip
+  is renamed into place only when complete; a file that changes while it
+  is packed is refused.
+- **Failures are one line.** `export` exits 2 for every failure that
+  pushed nothing (an `--out` that is a folder, an invalid config with
+  `--branch`), keeping exit 1 for a branch pushed without its pull
+  request. Any command's expected failure prints `saffron: <message>`
+  and exits 2, never a stack; `SAFFRON_DEBUG=1` shows it.
+- **Safe branch names in the pull request.** The description quotes the
+  branch in its commands, and `--branch` or `--base` with a space, a
+  quote or any of `` $ ` ; | & < > ( ) \ `` is refused up front.
+- **A `pull_request_review` build records its pull request's branch**,
+  read from the event, not `12/merge`.
+- **`import --json` passes on only the manifest fields it knows**, cleaned
+  of control characters.
+- **The pull request's review steps run saffron-ai by name**
+  (`npx -y -p saffron-ai saffron diff`): in a checkout without
+  `node_modules`, a bare `npx saffron` would fetch the unrelated `saffron`
+  package.
+- **A bundle names its run as the CI service does** (`github:<run id>`,
+  `azure:<build id>`), even with `SAFFRON_RUN_ID` set, so `saffron runs`
+  and both editors count its proposals under the right run.
+- **A green scenario's trace over 1 GB is left out of the bundle** like
+  the other green traces, instead of failing the export.
+- **Import hashes, compares and copies traces in chunks**: a 1 GB trace is
+  never held in memory. A bundle that changes on disk while it is
+  imported no longer says "nothing was imported" after some files were
+  written.
+- **`saffron runs` on Azure asks for 50 pull request builds first**, and
+  for more only while too few are the branch's, not 500 on every list.
+
+VS Code extension 0.4.0
+
+- **CI runs**: a new view lists the current branch's CI runs, GitHub
+  Actions or Azure Pipelines, through `saffron runs`. **Import** runs
+  `saffron import --run`, then lists under the run what happened to each
+  proposal and any warning (a checkout at another commit, a scenario that
+  changed since). Imported proposals are marked with the run they came
+  from, linked in the tooltip. From a red CI run to an accepted repair
+  without leaving the editor; commit and push with the Source Control
+  view. The CI's own tool keeps the credentials.
+- **Text from CI cannot act in a tooltip.** Run titles, branch names and
+  a bundle's notes are escaped so they never form links, headings or
+  other markup; a run links out only to GitHub or Azure DevOps, and never
+  when the address carries a user name. Import results are kept per
+  project. An import can be cancelled, and on Windows cancelling stops
+  the whole process tree. The list follows a branch switch, and makes no
+  CI call while its view is hidden.
+- **Cancelling an import stops all of it, and says what it left.** On
+  Windows the whole process tree is stopped however the CLI was started;
+  on macOS and Linux a process that outlives SIGTERM gets SIGKILL. The
+  message says a cancelled import may have written part of the run.
+
+JetBrains plugin 0.4.0
+
+- **CI Runs tab**: the same runs and the same Import, through the same
+  two commands, with each proposal's outcome under its run. The Proposals
+  tab names the CI run an imported proposal came from.
+- **Text from CI is shown as plain text**, in CI Runs and in Last Run
+  after an import, never as markup. Import is a background task you can
+  cancel, with no time limit; a run is imported once even if you switch
+  projects meanwhile; Refresh never starts a second list. The tab loads
+  as soon as the project is trusted, and its processes stop when the
+  project closes.
+- **Choosing another project clears the tabs at once**: a run listed for
+  the last project can no longer be imported into the new one, and a
+  status that arrives late for the last project is dropped.
+- **CI Runs lists again when you check out another branch.** A cancelled
+  import says it may have written part of the run.
+
+## 0.9.2: Bring your own AI agent
 
 Released 2026-09-30.
 
@@ -256,7 +402,7 @@ Editors (VS Code extension 0.3.3, JetBrains plugin 0.3.3)
   CSV; Run Tagged skips tags `--filter` cannot express and says which;
   diagnostics clear when a file closes or is deleted.
 
-## 0.9.1
+## 0.9.1: Tag expressions: and, or, not
 
 Released 2026-09-29.
 
@@ -320,7 +466,7 @@ Editors (VS Code extension 0.3.2, JetBrains plugin 0.3.2)
   tabs, under any file icon theme that has no icon of its own for them
   (Seti, the default, included).
 
-## 0.9.0
+## 0.9.0: Time-travel replay
 
 Released 2026-09-27.
 
@@ -515,7 +661,7 @@ Editors (VS Code extension 0.3.0, JetBrains plugin 0.3.0)
   what is pending when it runs, skipping UNVERIFIED proposals and
   refusing STALE ones.
 
-## 0.8.5
+## 0.8.5: Proposals bound to what you reviewed
 
 Released 2026-09-24.
 
@@ -554,7 +700,7 @@ Editors (VS Code extension 0.2.10, JetBrains plugin 0.2.5)
 - Both Proposals panels mark a stale proposal STALE, with the reason and
   what to do, and note a proposal filed by an older runner.
 
-## 0.8.4
+## 0.8.4: Heal and replay fixes
 
 Released 2026-09-24.
 
@@ -667,7 +813,7 @@ Editors (VS Code extension 0.2.9)
   files and to the config now refresh every open feature file, including a
   custom data folder and environment overlays.
 
-## 0.8.3
+## 0.8.3: Data references in table cells
 
 Released 2026-09-22.
 
@@ -683,7 +829,7 @@ Runner
   Code currently defaults to, which drifts) and which values it takes, so a
   project can pin its recording model.
 
-## 0.8.2
+## 0.8.2: Split a suite across CI machines
 
 Released 2026-09-22.
 
@@ -706,7 +852,7 @@ Runner
   twice does not count the run twice. Setup and teardown run on each shard
   and receive `SAFFRON_SHARD`. The docs have a GitHub Actions example.
 
-## 0.8.1
+## 0.8.1: Chrome, Edge, and setup and teardown
 
 Released 2026-09-22.
 
@@ -769,7 +915,7 @@ Editors (VS Code extension 0.2.8)
   hover and go-to-definition now follow the runner: the environment's CSV
   replaces the base one, and the hover names the file it read.
 
-## 0.8.0
+## 0.8.0: Test data outside the feature file
 
 Released 2026-09-21.
 
@@ -835,7 +981,7 @@ Editors (VS Code extension 0.2.7, JetBrains plugin 0.2.4)
 - Both Orphans lists show the screenshot folders of scenarios that no longer
   exist, so "Remove All" never deletes something it did not show.
 
-## 0.7.2
+## 0.7.2: Failure screenshots
 
 Released 2026-09-21.
 
@@ -890,7 +1036,7 @@ Editors (JetBrains plugin 0.2.2)
   notification says so when it is missing. The presence check uses no
   plugin-manager API, so it stays clean as that API turns internal.
 
-## 0.7.1
+## 0.7.1: Truthful checks and safer edits
 
 Released 2026-09-20.
 
@@ -961,7 +1107,7 @@ Editors (VS Code extension 0.2.5)
   passes the tree item, not the file path the row click passes; both forms
   are accepted, and the project guard applies to both.
 
-## 0.7.0
+## 0.7.0: saffron diff, saffron prune, and an external review
 
 Released 2026-09-19.
 
@@ -1154,7 +1300,7 @@ Editors (JetBrains plugin 0.2.2, VS Code extension 0.2.4)
   longer shell syntax, and a file from a second workspace folder runs
   against that folder's config and install.
 
-## 0.6.0
+## 0.6.0: Multi-tab scenarios
 
 Released 2026-09-18.
 
@@ -1176,7 +1322,7 @@ Editors
   its short description fits without being cut off, and the overview
   introduces the tool and how to get started before listing features.
 
-## 0.5.7
+## 0.5.7: Claude Code plugin
 
 Released 2026-09-17.
 
@@ -1200,7 +1346,7 @@ Agents
   points at `saffron status`. Run `npx saffron init` again to refresh both
   in a project.
 
-## 0.5.6
+## 0.5.6: Fewer false duplicate wordings
 
 Released 2026-09-17.
 
@@ -1211,7 +1357,7 @@ Runner
   duplicate wordings; steps are compared by their quote-generalised
   pattern.
 
-## 0.5.5
+## 0.5.5: Duplicate wordings proven by behaviour
 
 Released 2026-09-17.
 
@@ -1223,7 +1369,7 @@ Runner
   "enter X in the username field" against "enter Y in the password
   field", which are different steps.
 
-## 0.5.4
+## 0.5.4: saffron status for the IDE panels
 
 Released 2026-09-17.
 
@@ -1243,7 +1389,7 @@ Editors
   Health sections and an in-editor dashboard. All read
   `saffron status --json`.
 
-## 0.5.3
+## 0.5.3: Cost by how you pay, and running from JetBrains
 
 Released 2026-09-17.
 
@@ -1284,7 +1430,7 @@ Editors
   extension, the last run's totals with the report one click away, and
   the proposals pending review with Accept all.
 
-## 0.5.2
+## 0.5.2: Scenario Outlines record correctly
 
 Released 2026-09-15.
 
@@ -1313,7 +1459,7 @@ Editors
   hover says where it is defined and never shows the value. In the VS
   Code extension and in `saffron lsp` for JetBrains and other editors.
 
-## 0.5.1
+## 0.5.1: The example suite in .saffron files
 
 Released 2026-09-14.
 
@@ -1330,7 +1476,7 @@ Editors
   parameters instead of the whole step line. Ships in the VS Code
   extension, the JetBrains plugin and the package's TextMate bundle.
 
-## 0.5.0
+## 0.5.0: Guided first run
 
 Released 2026-09-14.
 
@@ -1343,7 +1489,7 @@ Runner
   matching `saffron.config.json` with `retries: 1`. Only installs into an
   empty `features/` directory. Documented as "Your first run".
 
-## 0.4.2
+## 0.4.2: Recover from a wrong recording
 
 Released 2026-09-13.
 
@@ -1375,7 +1521,7 @@ Docs and site
 - Assertion policy documented on the Features page; cache lifecycle gained
   the staleness and wrong-recording stages.
 
-## 0.4.1
+## 0.4.1: JetBrains plugin
 
 Released 2026-09-13.
 
@@ -1397,7 +1543,7 @@ Editors
 - VS Code extension 0.1.3: plain punctuation in README, hover and
   diagnostic messages.
 
-## 0.4.0
+## 0.4.0: Agent skill for AI assistants
 
 Released 2026-09-13.
 
@@ -1423,13 +1569,13 @@ Editors
 - VS Code extension 0.1.2: warning and quick fix when a set name is written
   without the `StepSet` keyword.
 
-## 0.3.1
+## 0.3.1: Version fix
 
 Released 2026-08-15.
 
 - `saffron --version` no longer reports a hardcoded value.
 
-## 0.3.0
+## 0.3.0: Network-aware steps
 
 Released 2026-08-12.
 
@@ -1451,7 +1597,7 @@ Editors
 
 - VS Code extension 0.1.0 published to the Marketplace.
 
-## 0.2.0
+## 0.2.0: Secrets, cross-browser and parallel replay
 
 Released 2026-07-19.
 
@@ -1467,7 +1613,7 @@ Runner
   workers (`--workers N`), and a cheaper model for heal sessions
   (`--heal-model`).
 
-## 0.1.0
+## 0.1.0: First public release
 
 Released 2026-07-10.
 

package/README.md

@@ -76,6 +76,7 @@ node dist/cli/index.js -p examples accept --all
 | `saffron run [paths] [--headed] [--filter expr] [--rerecord] [--no-agent] [--strict] [--browser b] [--no-hooks] [--workers n] [--heal-model m] [--no-verify] [--no-reuse] [--model m] [--storage-state f] [--trace mode]` | Run features. Cached replays are deterministic; misses/failures escalate to the agent (unless `--no-agent`). Replay cross-browser with `--browser firefox\|webkit`, parallelize with `--workers N`, heal on a cheaper model with `--heal-model`, keep an execution trace with `--trace retain-on-failure`. Exit 1 on red (and on yellow with `--strict`). |
 | `saffron accept [files... \| --all] [--include-unverified] [--with-feature-edit] [--propagate]` | Promote cache proposals to committed caches (`--all` skips UNVERIFIED ones unless `--include-unverified`); `--with-feature-edit` also rewrites the adapted steps in the `.feature` file (and keeps the cache in sync); `--propagate` applies the heal's locator fixes to every other cache using the same locator. One heal repairs N scenarios before they ever fail. No args: list pending proposals. |
 | `saffron reject [files... \| --all]` | Discard proposals; the agent will try again next run. |
+| `saffron export [-o file] [--branch name]` / `saffron runs` / `saffron import <bundles...> \| --run <id>` | Carry a CI heal to your machine: export packs the run's proposals, report, screenshots and traces into one bundle for CI to upload; `runs` lists the branch's CI runs (GitHub Actions or Azure Pipelines); import (of a file, or of a run with `--run`) checks the bundle against your checkout (repository, each proposal's binding) and files it there, ready for `saffron diff` and `saffron accept`. `--branch` also has CI commit the proposals to a new branch and open a draft pull request with their diffs. |
 | `saffron status [--json]` | Project overview for people, IDE panels and dashboards: feature files and scenarios with cache state, tags, pending proposals, last run, history, vocabulary health (divergent steps, duplicate wordings proven by identical recordings), effective config. |
 | `saffron steps [search] [--json] [--snippets]` | List/search the step vocabulary (files + caches) with recorded/divergent/unrecorded badges and usage; `--snippets` writes `.vscode/saffron.code-snippets` for native VS Code completion. |
 | `saffron mcp` | Serve the vocabulary and the project overview to AI assistants over stdio MCP (`search_steps`, `list_step_sets`, `project_status`), e.g. `claude mcp add saffron -- npx saffron mcp`. |