saffron-ai 0.7.1 → 0.7.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +56 -1
- package/README.md +12 -10
- package/dist-pkg/cli.js +103 -95
- package/package.json +3 -3
- package/skills/saffron/SKILL.md +5 -3
- package/skills/saffron/references/config.md +3 -0
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,61 @@ All notable changes to Saffron, in one place: the `saffron-ai` runner (npm),
|
|
|
4
4
|
the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
|
|
5
5
|
JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates.
|
|
6
6
|
|
|
7
|
+
## 0.7.2
|
|
8
|
+
|
|
9
|
+
Released 2026-09-21.
|
|
10
|
+
|
|
11
|
+
Runner
|
|
12
|
+
|
|
13
|
+
- **Failure screenshots.** When a step fails, Saffron saves a screenshot of
|
|
14
|
+
the tab it failed in and shows it in the HTML report, embedded so the
|
|
15
|
+
report stays one file. A healed scenario keeps the picture of the moment
|
|
16
|
+
its cached step stopped working, for whoever reviews the proposal; a
|
|
17
|
+
scenario still red after the agent ran is photographed again at the end.
|
|
18
|
+
`latest.json` carries the paths under `evidence`. New config key
|
|
19
|
+
`screenshot`: `viewport` (default), `full-page` or `off`. Files live in
|
|
20
|
+
`.saffron/artifacts/`, latest run only, and `saffron init` git-ignores
|
|
21
|
+
it: **add `.saffron/artifacts/` to your `.gitignore` if your project was
|
|
22
|
+
initialised earlier.** A screenshot shows whatever your page shows,
|
|
23
|
+
including a secret it displays; `{env:VAR}` masking cannot reach pixels.
|
|
24
|
+
|
|
25
|
+
- **A feature file that fails to parse says which file, and why.** Invalid
|
|
26
|
+
Gherkin reached the terminal as a raw stack trace with positions and no
|
|
27
|
+
file name. It is now one line per problem, `file:line:column`. A standard
|
|
28
|
+
`.feature` file that uses `StepSet` gets the actual answer instead of a
|
|
29
|
+
parser dump: rename it to `.saffron`, nothing else needs to change.
|
|
30
|
+
|
|
31
|
+
- `saffron status --json` gains `lastRun.attention`: every scenario of the
|
|
32
|
+
last run that was not green, red first, with the failed step, the first
|
|
33
|
+
line of the error and the screenshots that still exist on disk.
|
|
34
|
+
|
|
35
|
+
Editors (VS Code extension 0.2.6, JetBrains plugin 0.2.3)
|
|
36
|
+
|
|
37
|
+
- **Last run, with the screenshot one click away.** Both IDEs list what the
|
|
38
|
+
last run left to look at: failed scenarios, then healed ones, with the
|
|
39
|
+
step that failed. VS Code opens the screenshot in its image preview on
|
|
40
|
+
click; JetBrains opens it in the image viewer on double-click. Each also
|
|
41
|
+
jumps to the scenario. With a runner that predates failure screenshots,
|
|
42
|
+
both say so instead of reporting that everything passed.
|
|
43
|
+
|
|
44
|
+
Docs
|
|
45
|
+
|
|
46
|
+
- **A page for "`.saffron` and `.feature` files"**: what each is, a
|
|
47
|
+
side-by-side table, which to write, and what renaming a file means for its
|
|
48
|
+
recordings. The rest of the docs say "feature file" for both, where they
|
|
49
|
+
used to say `.feature` alone.
|
|
50
|
+
|
|
51
|
+
Editors (JetBrains plugin 0.2.2)
|
|
52
|
+
|
|
53
|
+
- **LSP4IJ is an optional dependency.** A first upload of 0.2.2 was rejected
|
|
54
|
+
by the Marketplace verifier on 2026.3 EAP because LSP4IJ had no release
|
|
55
|
+
for that build yet, and a mandatory dependency that cannot be resolved
|
|
56
|
+
blocks the whole plugin. Highlighting, run configurations and the tool
|
|
57
|
+
window now work without it; completion, go-to-definition, hover and
|
|
58
|
+
diagnostics switch on when LSP4IJ is installed, and a one-time
|
|
59
|
+
notification says so when it is missing. The presence check uses no
|
|
60
|
+
plugin-manager API, so it stays clean as that API turns internal.
|
|
61
|
+
|
|
7
62
|
## 0.7.1
|
|
8
63
|
|
|
9
64
|
Released 2026-09-20.
|
|
@@ -588,7 +643,7 @@ Released 2026-07-10.
|
|
|
588
643
|
First public release, free for any use under the Saffron Free Use License.
|
|
589
644
|
|
|
590
645
|
- Gherkin-native runner: an AI agent records each scenario once; later runs
|
|
591
|
-
|
|
646
|
+
are replayed by Saffron in a real browser at zero tokens; runtime healing files
|
|
592
647
|
reviewable proposals; assertions are never healed.
|
|
593
648
|
- Verified proposals (zero-AI proof replay), honest cost reporting, run
|
|
594
649
|
history and trends, chronic-scenario detection.
|
package/README.md
CHANGED
|
@@ -2,9 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
**Gherkin-native test runner with zero-token cached replay and runtime AI healing.**
|
|
4
4
|
|
|
5
|
-
Free to use · [npm: saffron-ai](https://www.npmjs.com/package/saffron-ai) · [site](https://saffron-ai.
|
|
5
|
+
Free to use · [npm: saffron-ai](https://www.npmjs.com/package/saffron-ai) · [site](https://saffron-ai.io)
|
|
6
6
|
|
|
7
|
-
Write tests as
|
|
7
|
+
Write tests as Gherkin feature files: `.saffron`, Saffron's superset of Gherkin with step sets, or standard `.feature`. On the first run, an AI agent (Claude Agent SDK + Playwright MCP) executes each scenario in a real browser and records what it did into a **cache**. A human-reviewable JSON action list committed to git. Every run after that, Saffron replays the cache in a real browser: **zero AI calls, zero tokens**, at the speed of the browser and your app. The cache is Saffron's own JSON, not a generated script; replay runs on Playwright's browser engine, and Playwright MCP is only how the agent operates the browser while it records or heals.
|
|
8
8
|
|
|
9
9
|
When the UI changes and a cached step fails at runtime, the agent takes over mid-execution with full scenario context, adapts, and finishes the run, then files a **cache proposal** you accept or reject, like a snapshot test update. Reports state exactly what was adapted and what it cost.
|
|
10
10
|
|
|
@@ -13,7 +13,7 @@ When the UI changes and a cached step fails at runtime, the agent takes over mid
|
|
|
13
13
|
- **Assertions are sacred.** By default no `Then` is ever healed: the AI may help *reach* an assertion, never make it pass; enforced mechanically, not by prompt. Projects may opt into `adaptable-mid` for mid-scenario checkpoints, but the **final assertion block** of a scenario is strict under every policy, forever.
|
|
14
14
|
- **Three result states.** Green = cached pass. Yellow = passed with AI adaptation (pending your review). Red = failed.
|
|
15
15
|
- **Diagnose before adapting.** On a step failure the agent first decides: UI drift (heal) or application defect (fail with a diagnosis).
|
|
16
|
-
- **Caches are git artifacts.** Proposals show diffs, including a suggested
|
|
16
|
+
- **Caches are git artifacts.** Proposals show diffs, including a suggested feature-file edit when the written steps no longer match reality, and nothing is committed without you.
|
|
17
17
|
- **Verified proposals.** Every recording/heal is proof-replayed zero-AI before it's filed (with one bounded refinement pass on failure), so proposals arrive stamped `verified ✓` or honestly `UNVERIFIED ✗`.
|
|
18
18
|
- **Honest cost reporting.** Reports break out prompt-cache reads/writes (the real bill of agent sessions) next to the in+out token count, per scenario and in totals, and name the models used. On an API key the dollars are what was billed; on a Claude subscription the headline is the 5-hour plan window before and after the run, with the dollars as the API-equivalent.
|
|
19
19
|
- **Trend memory.** Every run appends to `.saffron/history.jsonl`; reports show deltas vs the previous run and 20-run sparklines, and Saffron flags **chronic scenarios** (healing repeatedly. Re-record instead of paying again) plus recurring failure themes.
|
|
@@ -92,6 +92,7 @@ Configuration lives in `saffron.config.json` at your project root:
|
|
|
92
92
|
"actionTimeoutMs": 5000,
|
|
93
93
|
"pollIntervalMs": 100,
|
|
94
94
|
"retries": 1,
|
|
95
|
+
"screenshot": "viewport",
|
|
95
96
|
"model": "claude-sonnet-5",
|
|
96
97
|
"maxTurns": 100,
|
|
97
98
|
"storageState": ".auth/state.json",
|
|
@@ -113,7 +114,7 @@ cached replays *and* the agent start authenticated. Generate one with
|
|
|
113
114
|
|
|
114
115
|
## How it works
|
|
115
116
|
|
|
116
|
-
The full execution-model diagram is on the site: https://saffron-ai.
|
|
117
|
+
The full execution-model diagram is on the site: https://saffron-ai.io/docs
|
|
117
118
|
|
|
118
119
|
- Caches store per-step action lists with semantic targets (`role` + accessible name, `nameRegex` for volatile labels, CSS fallbacks) and `<param>` placeholders, so one cache serves every Examples row of a Scenario Outline.
|
|
119
120
|
- Dynamic-content vocabulary for real-world apps: `captureText` stores displayed values (prices, counters) under a name; `expectDiffers` compares against captured values; `expectMatches`/`expectAttribute` assert patterns instead of literals; `{date±N}` templates keep date-picker locators valid across days; the replayer follows links that open new tabs, and multi-tab scenarios are plain prose ("switch back to the first tab", "close the preview tab", "open a new tab at …") recorded as tab actions; native dialogs, file uploads, drag-and-drop, and iframe-scoped elements replay on all three engines; network-aware steps wait on or assert backend API responses (`waitForResponse` / sacred `expectResponse`) with URL/method/status/body matchers, "wait for the order API to return 201" instead of a sleep.
|
|
@@ -128,12 +129,13 @@ The full execution-model diagram is on the site: https://saffron-ai.lovable.app/
|
|
|
128
129
|
proposals/ pending AI-generated cache updates (review these)
|
|
129
130
|
history.jsonl one line per run: trends (commit recommended)
|
|
130
131
|
reports/ latest.html / latest.json (gitignore these)
|
|
131
|
-
|
|
132
|
+
artifacts/ failure screenshots from the latest run (gitignore these)
|
|
133
|
+
features/ your .saffron and .feature files
|
|
132
134
|
```
|
|
133
135
|
|
|
134
136
|
## Roadmap
|
|
135
137
|
|
|
136
|
-
See the roadmap on **[the site](https://saffron-ai.
|
|
138
|
+
See the roadmap on **[the site](https://saffron-ai.io/#roadmap)**, the single source of truth for
|
|
137
139
|
milestones, statuses, and decision gates. Headlines:
|
|
138
140
|
action-vocabulary additions as real-world failures surface them, the
|
|
139
141
|
Level-2 shared step library (gated on Level-1 divergence data), LLM-less
|
|
@@ -143,10 +145,10 @@ far is in the roadmap's "Where we are" table.
|
|
|
143
145
|
|
|
144
146
|
## Documentation
|
|
145
147
|
|
|
146
|
-
- [Website](https://saffron-ai.
|
|
147
|
-
- [Documentation](https://saffron-ai.
|
|
148
|
-
- [Changelog](CHANGELOG.md), also at https://saffron-ai.
|
|
149
|
-
- [Issues & questions](https://github.com/s-chathuranga-j/saffron-
|
|
148
|
+
- [Website](https://saffron-ai.io): what Saffron is, the honesty rules, economics, quickstart
|
|
149
|
+
- [Documentation](https://saffron-ai.io/docs): the cache lifecycle, step sets & the `.saffron` dialect, the step vocabulary, and the guarantees, with diagrams
|
|
150
|
+
- [Changelog](CHANGELOG.md), also at https://saffron-ai.io/docs/changelog
|
|
151
|
+
- [Issues & questions](https://github.com/s-chathuranga-j/saffron-community/issues)
|
|
150
152
|
- **Docs site**: `npm run docs` builds a self-contained page at
|
|
151
153
|
## License
|
|
152
154
|
|