saffron-ai 0.9.1 → 0.9.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +426 -28
- package/README.md +14 -2
- package/dist-pkg/cli.js +240 -183
- package/package.json +16 -2
- package/skills/saffron/SKILL.md +10 -0
- package/skills/saffron/references/config.md +25 -11
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
|
|
5
5
|
Free to use · [npm: saffron-ai](https://www.npmjs.com/package/saffron-ai) · [site](https://saffron-ai.io)
|
|
6
6
|
|
|
7
|
-
Write tests in plain language, as `.saffron` files or standard `.feature` files. `.saffron` is Saffron's own test language and goes further than Gherkin: step sets for reusable flows, `{env:VAR}` tokens that keep secrets out of recordings and reports, and `{data:...}` references to data files. There are no step definitions and no glue code. On the first run, an AI agent (Claude
|
|
7
|
+
Write tests in plain language, as `.saffron` files or standard `.feature` files. `.saffron` is Saffron's own test language and goes further than Gherkin: step sets for reusable flows, `{env:VAR}` tokens that keep secrets out of recordings and reports, and `{data:...}` references to data files. There are no step definitions and no glue code. On the first run, an AI agent (Claude by default, or Codex, Google AI or Cursor on the subscription you already have, driving Playwright MCP) executes each scenario in a real browser and records what it did into a **cache**. A human-reviewable JSON action list committed to git. Every run after that, Saffron replays the cache in a real browser: **zero AI calls, zero tokens**, at the speed of the browser and your app. The cache is Saffron's own JSON, not a generated script; replay runs on Playwright's browser engine, and Playwright MCP is only how the agent operates the browser while it records or heals.
|
|
8
8
|
|
|
9
9
|
When the UI changes and a cached step fails at runtime, the agent takes over mid-execution with full scenario context, adapts, and finishes the run, then files a **cache proposal** you accept or reject, like a snapshot test update. Reports state exactly what was adapted and what it cost.
|
|
10
10
|
|
|
@@ -16,6 +16,7 @@ When the UI changes and a cached step fails at runtime, the agent takes over mid
|
|
|
16
16
|
- **Caches are git artifacts.** Proposals show diffs, including a suggested feature-file edit when the written steps no longer match reality, and nothing is committed without you.
|
|
17
17
|
- **Verified proposals.** Every recording/heal is proof-replayed zero-AI before it's filed (with one bounded refinement pass on failure), so proposals arrive stamped `verified ✓` or honestly `UNVERIFIED ✗`.
|
|
18
18
|
- **Honest cost reporting.** Reports break out prompt-cache reads/writes (the real bill of agent sessions) next to the in+out token count, per scenario and in totals, and name the models used. On an API key the dollars are what was billed; on a Claude subscription the headline is the 5-hour plan window before and after the run, with the dollars as the API-equivalent.
|
|
19
|
+
- **Your subscription, your choice of agent.** Recording and healing run on the plan already logged in on the machine: Claude (default), Codex on a ChatGPT plan, Google AI Pro/Ultra through the Antigravity CLI (experimental), or Cursor. Set `"provider"` in the config; each one runs through its vendor's official client, and every provider records through the same recorder, so the caches are identical in kind.
|
|
19
20
|
- **Trend memory.** Every run appends to `.saffron/history.jsonl`; reports show deltas vs the previous run and 20-run sparklines, and Saffron flags **chronic scenarios** (healing repeatedly. Re-record instead of paying again) plus recurring failure themes.
|
|
20
21
|
|
|
21
22
|
## Step reuse: new scenarios get cheaper as your suite grows
|
|
@@ -62,7 +63,7 @@ node examples/demo-app/server.mjs 4173 &
|
|
|
62
63
|
node dist/cli/index.js -p examples run
|
|
63
64
|
|
|
64
65
|
# delete a cache and watch the agent re-record it (needs Claude Code auth
|
|
65
|
-
# or ANTHROPIC_API_KEY)
|
|
66
|
+
# or ANTHROPIC_API_KEY, or another provider's login: see "provider" below)
|
|
66
67
|
rm examples/.saffron/cache/login/successful-login.json
|
|
67
68
|
node dist/cli/index.js -p examples run
|
|
68
69
|
node dist/cli/index.js -p examples accept --all
|
|
@@ -75,6 +76,7 @@ node dist/cli/index.js -p examples accept --all
|
|
|
75
76
|
| `saffron run [paths] [--headed] [--filter expr] [--rerecord] [--no-agent] [--strict] [--browser b] [--no-hooks] [--workers n] [--heal-model m] [--no-verify] [--no-reuse] [--model m] [--storage-state f] [--trace mode]` | Run features. Cached replays are deterministic; misses/failures escalate to the agent (unless `--no-agent`). Replay cross-browser with `--browser firefox\|webkit`, parallelize with `--workers N`, heal on a cheaper model with `--heal-model`, keep an execution trace with `--trace retain-on-failure`. Exit 1 on red (and on yellow with `--strict`). |
|
|
76
77
|
| `saffron accept [files... \| --all] [--include-unverified] [--with-feature-edit] [--propagate]` | Promote cache proposals to committed caches (`--all` skips UNVERIFIED ones unless `--include-unverified`); `--with-feature-edit` also rewrites the adapted steps in the `.feature` file (and keeps the cache in sync); `--propagate` applies the heal's locator fixes to every other cache using the same locator. One heal repairs N scenarios before they ever fail. No args: list pending proposals. |
|
|
77
78
|
| `saffron reject [files... \| --all]` | Discard proposals; the agent will try again next run. |
|
|
79
|
+
| `saffron export [-o file] [--branch name]` / `saffron runs` / `saffron import <bundles...> \| --run <id>` | Carry a CI heal to your machine: export packs the run's proposals, report, screenshots and traces into one bundle for CI to upload; `runs` lists the branch's CI runs (GitHub Actions or Azure Pipelines); import (of a file, or of a run with `--run`) checks the bundle against your checkout (repository, each proposal's binding) and files it there, ready for `saffron diff` and `saffron accept`. `--branch` also has CI commit the proposals to a new branch and open a draft pull request with their diffs. |
|
|
78
80
|
| `saffron status [--json]` | Project overview for people, IDE panels and dashboards: feature files and scenarios with cache state, tags, pending proposals, last run, history, vocabulary health (divergent steps, duplicate wordings proven by identical recordings), effective config. |
|
|
79
81
|
| `saffron steps [search] [--json] [--snippets]` | List/search the step vocabulary (files + caches) with recorded/divergent/unrecorded badges and usage; `--snippets` writes `.vscode/saffron.code-snippets` for native VS Code completion. |
|
|
80
82
|
| `saffron mcp` | Serve the vocabulary and the project overview to AI assistants over stdio MCP (`search_steps`, `list_step_sets`, `project_status`), e.g. `claude mcp add saffron -- npx saffron mcp`. |
|
|
@@ -95,6 +97,7 @@ Configuration lives in `saffron.config.json` at your project root:
|
|
|
95
97
|
"pollIntervalMs": 100,
|
|
96
98
|
"retries": 1,
|
|
97
99
|
"screenshot": "viewport",
|
|
100
|
+
"provider": "claude",
|
|
98
101
|
"model": "claude-sonnet-5",
|
|
99
102
|
"maxTurns": 100,
|
|
100
103
|
"storageState": ".auth/state.json",
|
|
@@ -109,6 +112,15 @@ Configuration lives in `saffron.config.json` at your project root:
|
|
|
109
112
|
}
|
|
110
113
|
```
|
|
111
114
|
|
|
115
|
+
`provider` (also `--provider`) picks whose agent records and heals:
|
|
116
|
+
`claude` (default), `codex` (ChatGPT plan; `npm i -D @openai/codex-sdk`),
|
|
117
|
+
`antigravity` (Google AI Pro/Ultra through Google's `agy` CLI,
|
|
118
|
+
experimental; `npx saffron login antigravity` once) or `cursor`
|
|
119
|
+
(`npm i -D @cursor/sdk`, then `npx saffron login cursor`). `model` takes
|
|
120
|
+
that provider's model names. `npx saffron login` says what a provider
|
|
121
|
+
still needs on this machine. Every sign-in flow is in
|
|
122
|
+
[docs/PROVIDERS.md](docs/PROVIDERS.md).
|
|
123
|
+
|
|
112
124
|
`storageState` (also `--storage-state`) points to a Playwright storage-state
|
|
113
125
|
JSON (cookies + localStorage) applied to every scenario's browser context, so
|
|
114
126
|
cached replays *and* the agent start authenticated. Generate one with
|