saffron-ai 0.9.5 → 0.9.7

package/CHANGELOG.md

@@ -5,6 +5,135 @@ the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
 JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
 release heading names its main feature: `## <version>: <title>`.
 
+## 0.9.7: CI results without importing
+
+Released 2026-10-05.
+
+Runner
+
+- **See what a CI run found before you import it.** `saffron report --run
+  <id>` downloads the run's Saffron bundles once into
+  `.saffron/ci/<provider>-<id>/attempt-<n>/`, checks them as `saffron
+  import` does (repository, digests, sizes, format, the run, attempts and
+  shards), judges each proposal against this checkout without filing it,
+  and opens the run's report page. Your `.saffron/reports/`,
+  `.saffron/proposals/`, `.saffron/artifacts/` and import ledger are left
+  as they were. `--json` gives the run, its totals, each scenario with its
+  failing step, first error line, screenshot and trace, each proposal as
+  importable or not with the reason, and warnings: what the VS Code and
+  JetBrains CI runs views show under a run.
+- **A run without results says why.** `running` (not finished),
+  `expired` (the service deleted the artifact) or `none` (nothing
+  uploaded; the guide's workflow exports only on failure, and `if:
+  always()` on the export and upload steps keeps every run's results), as
+  `reasonKind` and `reason`, exit 0.
+- **Large runs ask first.** `--max-download <MB>` refuses a run whose
+  bundles add up to more, with `{"error", "kind": "too-large",
+  "sizeBytes"}` and exit 2, so an editor can ask and run again.
+- **Replay a CI run, view only.** `saffron trace --run <id> [scenario]`
+  replays the run's traces from `.saffron/ci/`, taking the same scenario
+  forms as `saffron trace` (`<feature>:<scenario>` included). The page has
+  no Accept or Reject and its server no review routes: nothing was filed.
+- **View, then import, downloads once.** `saffron import --run <id>`
+  reuses the viewed download of the run's latest attempt.
+- **A cache that looks after itself.** `.saffron/ci/` keeps the newest 10
+  runs and at most 1 GB, removing the oldest first; a download goes
+  through a `.partial` folder and is renamed whole, so a cancelled one
+  leaves nothing behind. The folder writes its own `.gitignore` (`*`), so
+  it never shows in `git status`, even in a project set up before it
+  existed, and `saffron init` adds `.saffron/ci/` to the entries it
+  manages.
+
+VS Code extension 0.6.0
+
+- **Expand a CI run to see what it found, without importing it.** The
+  run lists its totals, each failed and healed scenario with its failing
+  step and first error line (the tooltip has what the agent changed), and
+  "N proposals: M importable" with the reason each one would not import.
+  Your reports, proposals and evidence are left alone. Needs saffron-ai
+  0.9.7 or later; an older runner says so.
+- **Open Report and Open Trace on a viewed run.** Open Report shows the
+  run's report in your browser; Open Trace replays a traced scenario,
+  view only. Import stays the only action that files proposals.
+- **Large results ask first.** Results above the new
+  `saffron.ciMaxDownloadMB` setting (default 100, 0 for no limit) ask
+  "This run's results are N MB. Download?". Loading can be cancelled,
+  which stops the whole download, and never runs while the CI runs view
+  is hidden.
+- **A run with no results says why:** still running, its artifacts
+  expired, or it uploaded none.
+
+JetBrains plugin 0.6.0
+
+- **CI Runs shows what a run found without importing it.** Select a run
+  to see its totals, each failed and healed scenario with its failing
+  step and first error line (the narrative in the tooltip), and
+  "N proposals: M importable" with the reason for each one that is not.
+  It loads in a background task you can cancel, only while the tab is on
+  screen. Needs saffron-ai 0.9.7 or later; an older runner says so.
+- **Open Report and Open Trace on a viewed run.** Open Trace (or a
+  double-click) replays a failed or healed scenario's trace, view only.
+  Import Run stays the only action that brings proposals into the
+  checkout.
+- **Large results ask first**, above the limit in Settings, Tools, Saffron
+  (default 100 MB, 0 for no limit), the same as the VS Code setting.
+- **A run with no results says why:** still running, expired, or none
+  uploaded.
+
+## 0.9.6: Dashboard figures you can trust
+
+Released 2026-10-05.
+
+Runner
+
+- **Three dashboard figures no longer flatter the suite.** *Recorded*
+  counted a stale recording, one whose scenario was edited since and
+  needs the agent again, so the dashboard could say 100% while `saffron
+  status` said otherwise; it now counts current recordings only, and at
+  100% says "every scenario has a current recording". *Passing* and the
+  last run's pass rate counted a heal whose zero-AI proof had failed;
+  such a heal is now "healed, unverified", out of the pass rate, with its
+  own stripe in the health bar, a count on the tiles and a mark in the
+  healed-again list. *Flaky* counted a scenario that was red on one base
+  URL and green on another as flipping; flips are now counted between
+  consecutive runs on the same base URL, and a first recording that
+  failed is left out. `status --json` gains `quality.suite.unverified`,
+  `quality.lastRun.unverified`, and `unverified` on tags and healing
+  entries. Both IDE dashboards show the runner's page, so they follow
+  without a plugin update.
+- **An upload whose file does not exist says so.** Under a project path
+  reached through a link (a temp folder on macOS), the message was
+  "outside the project". The repository's own example now names its
+  upload file relative to the project, as replay has resolved it since
+  0.9.4.
+- **Uploads are single files.** A folder passed to a file input was let
+  through, and a folder upload sends everything inside it, dotfiles such
+  as `.env` included. The agent's upload and replay both refuse a folder
+  now.
+- **`saffron report --merge` never hides another environment's result.**
+  A re-run attempt replaced the earlier attempt of the same shard number
+  before checking that both ran the same suite against the same base
+  URL, so a green production attempt could silently drop a red staging
+  one. An attempt now replaces only the same shard of the same suite,
+  split and base URL; anything else is refused, naming the base URLs.
+- **`saffron export --branch` works with `"features": "."`.** With the
+  scenarios at the project root, the shared StepSets were left out of the
+  check against the commit the branch starts on, so a correctly bound
+  proposal was refused as using a StepSet "not defined anywhere".
+- A replay that finds its recording stale no longer leaves a listener on
+  the browser context.
+
+VS Code extension 0.5.1
+
+- **The dashboard tab shows the Saffron mark in dark themes.** It used the
+  activity bar icon, one dark shape that VS Code tints there and a tab
+  draws as is, so it was dark on a dark tab.
+- **Cancelling an import stops every process it started, on macOS and
+  Linux too.** The CLI runs in a process group of its own, so a `gh` or
+  `az` download that outlives it is stopped as well, with a forced stop
+  for one that ignores the first request. Output over 16 MB ends the
+  command with a clear message.
+
 ## 0.9.5: CI heals you can trust
 
 Released 2026-10-04.