saffron-ai 0.9.2 → 0.9.4

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "saffron-ai",
-  "version": "0.9.2",
+  "version": "0.9.4",
   "description": "AI-fallback test runner for end-to-end tests written in plain language: zero-token cached replay, runtime AI healing, honest reports",
   "scripts": {
     "test": "vitest run",

package/skills/saffron/SKILL.md

@@ -132,7 +132,7 @@ Scenario: Checkout happy path
 
 ```bash
 npx saffron run                          # record misses, replay hits, heal failures
-npx saffron run --filter @smoke          # by tag (comma-separate several; any of them)
+npx saffron run --filter @smoke          # by tag; or an expression: "@e2e and not @wip", "(@smoke or @sanity) and not @flaky"
 npx saffron run --no-agent               # replay only (CI without AI access)
 npx saffron status                       # what is cached, pending, tagged; vocabulary health
 npx saffron diff                         # what a proposal changes, action by action: the evidence
@@ -141,7 +141,10 @@ npx saffron accept --all                 # promote verified proposals to caches
 npx saffron accept <file> [<file>...]    # promote chosen ones
 npx saffron reject <file> [<file>...]    # discard chosen ones; the agent retries next run
 npx saffron report                       # open the HTML report
+npx saffron dashboard                    # suite quality: failing, flaky, never run, slowest, per tag
 npx saffron run --rerecord --filter @t   # a recording is wrong: record it fresh
+npx saffron trace "<scenario>"           # a red or healed scenario, step by step, with the page at every action ("trace" on in config)
+npx saffron runs                         # this branch's CI runs; a heal made there: saffron import --run <id>
 ```
 
 Results: **green** = cached pass · **yellow** = pending review (an AI
@@ -168,6 +171,15 @@ unless deliberately authoring a manual cache (`"recordedBy": "manual"`).
 To change behavior, change the `.feature`/`.saffron` text and let the run
 re-record (mostly seeded from existing step recordings).
 
+**Review a CI heal.** A heal made in CI travels as a bundle (`saffron
+export` in the workflow, uploaded as an artifact). On the reviewer's
+checkout, `saffron runs` lists the branch's CI runs and `saffron import
+--run <id>` (or `saffron import <bundle>`) files the run's proposals,
+report and traces here, each checked against this checkout; then
+`saffron diff`, `saffron accept` and commit `.saffron/cache/` as usual.
+With `saffron export --branch <name>` CI pushes the proposals as a branch
+with a draft pull request instead: switch to it, accept, commit, push.
+
 ## Anti-patterns that cost money or hide bugs
 
 | Don't | Do |

package/skills/saffron/references/config.md

@@ -63,13 +63,14 @@ your-project/
 | `provider` | Whose agent records and heals, on its login on this machine: `claude` (default), `codex` (ChatGPT plan), `antigravity` (Google AI Pro/Ultra via the `agy` CLI, experimental), `cursor`. Also `--provider` |
 | `model` | Model in the provider's own names (unset: the provider's default). A `--provider` run on another provider ignores it |
 | `healModel` | Cheaper model for heal sessions only |
+| `ci` | Where `saffron runs` and `import --run` find CI runs: `{"provider": "github" \| "azure", "organization": ..., "project": ...}`. Unset, an Azure Repos remote means Azure Pipelines (through `az`), anything else GitHub Actions (through `gh`). `organization` and `project` are for a GitHub repository built by Azure Pipelines |
 
 ## CLI
 
 | Command | Purpose |
 |---|---|
-| `saffron run [paths] [--filter @tag] [--no-agent] [--strict] [--browser b] [--workers n] [--shard i/n] [--shard-by scenario\|file] [--provider p] [--heal-model m] [--headed] [--trace mode]` | Run; cached replay, agent on misses/failures. `--shard 2/4` runs one part of the suite on one CI machine and writes `shard-2-of-4.json` instead of `latest.json`; `--trace retain-on-failure` keeps an execution trace of each scenario that is not green |
-| `saffron trace [scenario] [--port n] [--no-open] [--json]` | Open the execution replay of a scenario from the last run (scenario name, feature path or `feature:scenario`; default: every traced scenario in one replay, opening on a red one, else a yellow one); `--json` prints the parsed trace |
+| `saffron run [paths] [--filter <tag expression>] [--no-agent] [--strict] [--browser b] [--workers n] [--shard i/n] [--shard-by scenario\|file] [--provider p] [--heal-model m] [--headed] [--trace mode]` | Run; cached replay, agent on misses/failures. `--filter` takes tags or an expression (`"@e2e and not @wip"`). `--shard 2/4` runs one part of the suite on one CI machine and writes `shard-2-of-4.json` instead of `latest.json`; `--trace retain-on-failure` keeps an execution trace of each scenario that is not green |
+| `saffron trace [scenario] [--port n] [--no-open] [--json] [--run startedAt]` | Open the execution replay of a scenario from the last run (scenario name, feature path or `feature:scenario`; default: every traced scenario in one replay, opening on a red one, else a yellow one); `--json` prints the parsed trace; `--run` opens it only while the last run is that one |
 | `saffron accept [files... \| --all] [--include-unverified] [--with-feature-edit] [--propagate]` | Promote proposals (`--all` skips UNVERIFIED ones unless `--include-unverified`); `--with-feature-edit` rewrites adapted steps in the feature file; `--propagate` applies a heal's locator fix to every cache using that locator |
 | `saffron reject [files... \| --all]` | Discard proposals |
 | `saffron prune [--yes] [--check] [--json]` | List recordings no scenario owns any more; `--yes` deletes them, `--check` exits 1 for CI |
@@ -80,9 +81,14 @@ your-project/
 | `saffron mcp` / `saffron lsp` | MCP tools (`search_steps`, `list_step_sets`, `project_status`) for AI assistants / language server for editors |
 | `saffron init [--examples] [--agents list]` | Install this skill into the project's agent directories, register MCP, add an AGENTS.md block, scaffold config; `--examples` adds the Saucedemo example suite |
 | `saffron report [--merge [paths]]` | Open the latest HTML report; `--merge` combines a complete set of shard reports into `latest.json`, `latest.html` and one history line |
+| `saffron dashboard [--stdout] [--no-open]` | The suite's quality from the run history (failing and for how long, flaky, repeatedly healed, never run, slowest, trends, per tag); writes and opens `.saffron/reports/dashboard.html`, `--stdout` prints it |
+| `saffron export [-o file] [--branch name [--base branch]]` | In CI: pack the pending proposals, the run's report, screenshots and traces into `.saffron/reports/saffron-bundle.zip` to upload as an artifact. `--branch` also pushes the proposals as a new branch and opens a draft pull request (GitHub through `gh`, Azure DevOps through `az`); exit 1 only when the branch is pushed but the pull request could not be opened |
+| `saffron runs [--branch b] [--limit n] [--provider github\|azure] [--json]` | The branch's CI runs, with how many of each run's proposals are pending here; exit 2 when `gh` or `az` is missing or signed out |
+| `saffron import <bundles...> \| --run <id> [--overwrite] [--allow-unknown-repository] [--json]` | Bring a CI run's bundle into this checkout: its proposals (each checked against the files here), report and traces. Then `saffron diff` and `saffron accept` as usual. Exit 1 when a proposal was not imported, 2 when the bundle was refused |
 
 Exit codes: 0 green/yellow, 1 red (or yellow with `--strict`), 2 usage /
-preflight (e.g. a missing `{env:VAR}`).
+preflight (e.g. a missing `{env:VAR}`) or any expected failure, printed as
+one line (`SAFFRON_DEBUG=1` shows the stack).
 
 ## AI access
 
@@ -114,8 +120,37 @@ npx playwright install chromium
 npx saffron run --no-agent --strict     # replay committed caches; no AI, no surprises
 ```
 
-Record and heal on developer machines (or a dedicated job with
-credentials), review proposals in PRs like snapshot updates.
+Record new scenarios on developer machines. A CI job that may heal (it
+has the provider's login or key) runs without `--no-agent`, and its heals
+travel to the reviewer as a bundle, never lost with the CI workspace:
+
+```yaml
+# GitHub Actions
+- run: npx saffron run --strict
+- name: Saffron bundle
+  if: failure()
+  run: npx saffron export --out saffron-bundle.zip
+- uses: actions/upload-artifact@v4
+  if: failure()
+  with:
+    name: saffron-bundle
+    path: saffron-bundle.zip
+```
+
+Azure Pipelines is the same: `saffron export --out
+$(Build.ArtifactStagingDirectory)/saffron-bundle.zip`, then publish it as
+the `saffron-bundle` artifact. Keep the artifact name starting with
+`saffron-bundle`: that is what `saffron import --run` and both editors'
+CI runs views download. Upload a file outside `.saffron/` (`--out`), as
+`upload-artifact` leaves hidden folders out.
+
+On the reviewer's checkout: `npx saffron runs`, then `npx saffron import
+--run <id>`, then `saffron diff`, `saffron accept` and commit
+`.saffron/cache/`. For review in a pull request instead, add `--branch
+saffron/heal-<run id>` to the export step: CI pushes the proposals (not
+the caches) as a branch and opens a draft pull request whose description
+says how to accept them; it needs a token that may push and open pull
+requests.
 
 ## Reading a report
 

package/skills/saffron/references/syntax.md

@@ -232,4 +232,9 @@ Scenario: Checkout happy path
 Scenario: ...
 ```
 
-`saffron run --filter @smoke`.
+`saffron run --filter @smoke`. `--filter` takes a tag expression:
+`and`, `or`, `not` and parentheses, with `and` binding tighter than `or`
+(`--filter "@e2e and not @wip"`, `--filter "(@smoke or @sanity) and not
+@flaky"`). A comma, `|`, a space or a repeated `--filter` between plain
+tags means `or`; once `and`, `not` or a parenthesis appears, join every
+tag with `and` or `or`. The `@` may be left out.