saffron-ai 1.2.0 → 1.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,7 +5,375 @@ the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
5
5
  JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
6
6
  release heading names its main feature: `## <version>: <title>`.
7
7
 
8
- ## 1.2.0: API test automation
8
+ ## Unreleased
9
+
10
+ ## 1.2.2: Triage in the report, a new dashboard, and verification for coding agents
11
+
12
+ Released 2026-10-28.
13
+
14
+ Runner
15
+
16
+ - **Proposals are reviewed in the run report.** A scenario with a proposal
17
+ has a `run | review` tab pair. The review tab holds what used to sit
18
+ among the run's panels: every step beside what it records today and what
19
+ is proposed (unchanged steps dimmed), the notes on a step, the agent's
20
+ account, the page before the agent took over, the proof and stability
21
+ proofs, and the commands to accept or reject, each with a copy button.
22
+ The header has a "N proposals to review" chip that opens the first one.
23
+ The page before the agent took over is embedded once, as before.
24
+ `saffron diff --html` is unchanged. The report stays read only.
25
+ - **A new dashboard, in both editors and `saffron dashboard`.** The page
26
+ opens on the suite's health and a history grid: each scenario that is
27
+ failing, flaky or healed again and again gets one cell per run (last 20),
28
+ so a new break, a flaky test and a drifting page read at a glance. Click
29
+ a row for its error, and a scenario's name to open it in the editor.
30
+ A scenario that fits several chips (failing and flaky, say) is listed
31
+ once, under the most urgent group, and is found under each chip; a
32
+ group's count follows the chip, and a chip with nothing under it cannot
33
+ be pressed. The grid's columns are the last 20 runs a listed
34
+ scenario ran in, so runs from an editor's Run button of other scenarios
35
+ no longer stretch it with empty columns.
36
+ Slowest, tags and coverage gaps sit beside it. The pass-rate and
37
+ execution-time bar charts are gone. The editors need no update: they show
38
+ the runner's page.
39
+ - **Found in a wider test pass.**
40
+ - A day the page spells with its weekday ("Thursday, October 29,
41
+ 2026") in a scenario that writes that day as its own token
42
+ (`{date+1}`) is recorded as one token. It was recorded as
43
+ "Thursday, {date+1:MMMM D, YYYY}", which the next day's page no
44
+ longer shows.
45
+ - `saffron run --plan` counts stability proofs among the proof replays.
46
+ - A drag the repair no longer makes is a behavior change only, not also
47
+ a "requirement not kept" note.
48
+ - **After a review of the unreleased work.**
49
+ - `verify` says when the run produced no results (`runProblem`, and an
50
+ MCP error), instead of answering with an empty pass, and reads only
51
+ the rows its own run wrote.
52
+ - A recording deleted outright counts as losing every check it made, in
53
+ `saffron coverage --base` and in `verify`.
54
+ - Every copy of a request sent at once has its credentials masked in
55
+ the transcript, not only the copy the step's checks read.
56
+ - `maxCostUsd` warns that Codex, Antigravity and Cursor report no price,
57
+ so it cannot stop their sessions.
58
+ - A transport module's `browser()` and `api()` hooks are asked once per
59
+ run, however many workers start.
60
+ - The command a run records hides a URL's password and writes an
61
+ environment variable's value as its `{env:NAME}`.
62
+ - `reset` runs before a recording only when an agent session can follow,
63
+ including when the agent's provider cannot start (a missing SDK).
64
+ - `verify` hides every environment value in the run's output it quotes,
65
+ so a secret a failed setup printed never reaches the agent.
66
+ - The recorded command masks only secret-like variables (a path under
67
+ `HOME` stays as typed) and keeps a glob quoted.
68
+ - A transport hook that failed is asked again by the next caller.
69
+ - The `maxCostUsd` warning for an unpriced provider is said once, at its
70
+ first agent session, not in every run.
71
+ - **A pluggable transport for isolated environments.** `transport` in
72
+ saffron.config.json connects to a browser already running elsewhere (a
73
+ Chromium by `cdpEndpoint`, which records and replays, or a Playwright
74
+ server by `wsEndpoint`, which replays), sends API requests through a
75
+ proxy with extra headers and self-signed certificates accepted, and
76
+ takes a project module whose `browser()` and `api()` hooks decide both
77
+ at run time, so a Docker setup needs no custom integration.
78
+ - **Verification for coding agents over MCP.** The new `verify` tool
79
+ replays the scenarios an agent names, or the whole suite, at zero AI
80
+ (`saffron run --no-agent`), and answers in a versioned shape: the commit
81
+ and environment, each check's verdict and evidence, what could not be
82
+ verified, proposals waiting for a person, and the spec's changes against
83
+ the reviewed coverage map committed at a git ref, so a weakened or
84
+ removed check shows even when everything is green.
85
+ - **Requests sent at once, and prerequisites an environment lacks.** An
86
+ API step can send a request several times at once (`concurrent`, with
87
+ the `statuses` the copies answer in any order), so a double submission
88
+ that books twice is a red; the agent records it for steps such as "I
89
+ submit the booking twice at once". Such recordings are version 12. A
90
+ scenario tagged `@requires:<name>` is not run where `capabilities` in
91
+ saffron.config.json sets that name to false: it is red, blocked by the
92
+ environment, naming the missing prerequisite, instead of being healed
93
+ around it.
94
+ - **A run report built for triage.** The HTML report lists every
95
+ scenario in one scrollable list, failures first, then proposals, then
96
+ passes, with a search box and a filter per status; the scenario chosen
97
+ shows beside it. Failures with the same error, and proposals with the
98
+ same change, are linked to each other, with one command for them all
99
+ (`saffron run` for the failures, `saffron accept` with their files for
100
+ the proposals). A run map below shows one cell per scenario by feature
101
+ file, coloured by status, AI cost or duration, so a run of 200
102
+ scenarios stays one page; a cell opens its scenario in an inspector
103
+ under the map (its steps as cards, the failure or the change, and who
104
+ shares it), leaving the list's choice as it was. Without scripts the page still reads: every
105
+ scenario is shown in turn. Under the map, widgets show setup and
106
+ teardown (a failed command first and flagged in the header, the ones
107
+ that went well folded), the workers' timing (saying when they did very
108
+ different amounts of work) and the pass rate of the last runs, the
109
+ newest marked. A duration over a minute reads as minutes and seconds,
110
+ and scenarios sharing a cause get a button for the command that deals
111
+ with them all, the command itself still on the page.
112
+ - **A compatibility page per release.** [Compatibility](COMPATIBILITY.md)
113
+ lists the Node.js, operating systems, Playwright and browsers, each
114
+ agent's SDK (the range accepted and the version tested) and the recording
115
+ versions this release reads and writes, generated from the release's own
116
+ code and lockfile. The agent skill names the saffron-ai version it
117
+ describes.
118
+ - **A recording-quality benchmark.** `npm run benchmark` replays the
119
+ example suite's recordings several times on the demo app (flaky
120
+ scenarios), and against deliberately broken versions of it (false
121
+ greens); `--record` also records the suite afresh with the agent and
122
+ measures the share verified on the first attempt, the share held for
123
+ review, and the time and cost. The results are published in the docs
124
+ ([benchmark](BENCHMARK.md)): today 66 of 66 replays green and no false
125
+ green on 8 broken apps.
126
+ - **A coverage map per pull request.** `saffron coverage` maps each
127
+ requirement ID (a tag matching `requirementTag`, such as `@REQ-12`) to
128
+ its scenarios, the checks their recordings make and the last run's
129
+ verdict. `--write` saves it as `.saffron/coverage.json` to commit, and
130
+ `--base origin/main` compares with the target branch's map: scenarios
131
+ added and removed, requirements no longer covered and checks a recording
132
+ no longer makes, with `--markdown` for the pull request and
133
+ `--fail-on-loss` to gate on it.
134
+ - **Side effects are counted, and can be reset.** `saffron run --plan`
135
+ marks the scenarios that change the app's data (a write request in
136
+ their recording, a unique value, or an `@side-effects` tag) and how many
137
+ times the run executes each: a recording runs once, its proof, every
138
+ stability proof and every sensitivity check again. The new `reset`
139
+ setting runs your commands before each of those fresh executions (never
140
+ before an ordinary replay), with `SAFFRON_SCENARIO`, `SAFFRON_FEATURE`
141
+ and `SAFFRON_RESET_FOR`. A failed reset stops that recording, or fails
142
+ that proof, and the run goes on.
143
+ - **Stability proofs.** `stabilityProofs` (`--stability-proofs n`) replays
144
+ a recording that passed its proof n more times, each in a fresh
145
+ execution, before it is filed. The proposal, the run summary and the
146
+ review page tell "passed one proof" from "passed the stability proofs",
147
+ and a recording that fails one gets an `unstable recording:` note, left
148
+ out of `accept --all` as the editor plugins mark it.
149
+ - **A recording plan and spending limits.** `saffron run --plan` prints
150
+ what the run would do and stops, with no browser or agent: what replays,
151
+ what records or records again (with the reason), what is refused, how
152
+ many agent sessions and proof replays that takes, and new steps several
153
+ scenarios share, to record and accept first. `--json` and the MCP tool
154
+ `run_plan` give the same as data. `maxCostUsd` (`--max-cost`) and
155
+ `maxAgentMinutes` (`--max-agent-minutes`) stop new agent sessions once a
156
+ run reaches them, and the summary splits AI cost into recording, repair
157
+ and failed attempts.
158
+ - **Every repair is checked against what its recording bound.** From the
159
+ committed recording, Saffron lists each value a step enters, each request
160
+ it sends and each check it makes, and finds the repaired action that
161
+ keeps each one. `saffron diff`, its JSON (`contract`) and the review page
162
+ show the list with that evidence. A repair that no longer sends a request
163
+ or makes a check gets a `requirement not kept:` note, and `accept --all`
164
+ leaves it out for review, as the editor plugins mark it.
165
+ - **The run summary says which run it counts.** It names the invocation
166
+ (`this run (\`saffron run ...\`, 12 scenarios, 10:04)`), splits pending
167
+ review by proof passed, failed or not run, and splits failures into the
168
+ app's and those the recorder, the agent's provider or the environment
169
+ caused (`blocked` in the report JSON). A run of named scenarios also
170
+ prints the combined report's counts. The report shows the command too.
171
+ - **A review page for proposals.** `saffron diff --html` writes
172
+ `.saffron/reports/review.html`: each Gherkin step beside the action it
173
+ records today and the one proposed, the target, what the proof captured
174
+ and any note about that step, so "enter an expiry date" becoming "check
175
+ an expiry hint" shows in the step's own row. The last run's screenshot of
176
+ the page before the agent took over is included. `saffron diff` and
177
+ `--json` show those notes under their step too.
178
+ - **Dependencies.** saffron-ai now needs `zod` 4.6.7 or later, so
179
+ upgrading it in a project with a lockfile also brings the fixed `zod`.
180
+ The other packages `npm audit` may report after an upgrade
181
+ (`body-parser`, `qs`, `hono`, `ip-address`, `fast-uri`) come through
182
+ Saffron's dependencies and are already allowed at their fixed versions:
183
+ `npm audit fix` updates them.
184
+
185
+ Editors (VS Code extension 0.7.5, JetBrains plugin 0.7.6)
186
+
187
+ - **Two more notes hold a proposal for review.** A proposal with a
188
+ "requirement not kept:" note (a repair that no longer sends a request or
189
+ makes a check its recording did) or an "unstable recording:" note (a
190
+ recording that failed an extra stability proof) is held from Accept
191
+ All, as `saffron accept --all` holds it. The new dashboard needs no
192
+ editor update: both show the runner's page.
193
+
194
+ ## 1.2.1: Sturdier recordings, and API keys for Codex and Antigravity
195
+
196
+ Released 2026-10-28.
197
+
198
+ Runner
199
+
200
+ - **After a review of the unreleased work.**
201
+ - An Outline row's date that the page spells out (`Friday, October 30,
202
+ 2026`) is recorded as that row's own date token, so another row never
203
+ replays on it and the same row follows the day without recording
204
+ again. A day the page works out from it (a checkout two days on)
205
+ follows the row too. A recording that kept the date as text records
206
+ once more. Such a recording is schema version 11, which 1.2.0 refuses.
207
+ Another row with a different date there fails with its reason; it
208
+ needs a Scenario of its own. A day shown in a step that does not use
209
+ the column (today's date in a banner) no longer ties the Outline to
210
+ one row.
211
+ - A date the scenario writes itself (`2026-10-30`) stays that date when
212
+ the page spells it otherwise ("Friday, October 30, 2026"), instead of
213
+ moving with the calendar from the next day on.
214
+ - A token used as a key in a whole-JSON Examples column is refused
215
+ instead of being sent as a broken field name.
216
+ - `saffron author` makes the checks a recording makes before it starts
217
+ the agent, except agy's rules for Saffron's tools, which it does not
218
+ use: with `antigravityApiKey` set and no key it refuses instead
219
+ of signing in with the Google login, and a failed session prints its
220
+ reason instead of a stack trace.
221
+ - A token outside any string in a whole-JSON Examples column
222
+ (`{"qty": {captured:count}}`) is a value of its own, so a secret
223
+ beside it can no longer add a field. A cell that is neither JSON nor
224
+ one whole token is refused instead of being filled in first. The same
225
+ holds in exported tests.
226
+ - Captured values in a proposal mask a credential an API handed out.
227
+ - A secret inside a whole-JSON Examples column can no longer rewrite
228
+ the payload.
229
+ - "Should differ" compares references and counts (`ABC-123`,
230
+ `1 night`) as text, refusing only amounts in different currencies or
231
+ a price against a label, and a brief bad reading no longer ends the
232
+ check.
233
+ - A heal that swaps a tick, untick, upload or drag for another input is
234
+ flagged as a behavior change.
235
+ - The sensitivity check gives each check a second, skips a `Then` with
236
+ no check or in a session not yet open, and cannot abort a run. The
237
+ report counts only the proposals it checked.
238
+ - `expectDisabled` is decided by the recorded element, not a fallback.
239
+ - Slashed dates are read in the locale's order, a UK date with its
240
+ weekday becomes a token, and a month the scenario names is not a
241
+ volatile locator.
242
+ - **The npm page says what Saffron is.** The package description no longer
243
+ calls it an AI-fallback test runner, its keywords cover the terms people
244
+ search for, it names its author, and the README names Antigravity among
245
+ the agents.
246
+ - **`prune --json` no longer calls a broken file's recordings orphans.**
247
+ While a feature file fails to parse, prune already deleted nothing, but
248
+ its JSON listed the hidden scenarios' live recordings as orphans. Now
249
+ they are in `orphanScan.ownershipUnknown`, beside the parse `errors`;
250
+ other files' orphans stay in `orphans`, and the JSON has the same shape
251
+ on a healthy project. `saffron status --json` splits them the same way,
252
+ where it listed no orphans at all.
253
+ - **Volatile locators are caught while recording.** A date the agent
254
+ picked that the app spells out (`October 30, 2026`, `30.10.2026`) is
255
+ recorded as a `{date}` token, as ISO dates already were, and a day the
256
+ scenario's own token wrote keeps that token. A generated id in a
257
+ selector, frame or test id, a UUID in a name, or a month heading is
258
+ refused once at `saffron_done` so the agent finds a stable target; what
259
+ is left gets a `volatile locator:` note, and `accept --all` leaves that
260
+ proposal to be accepted by file.
261
+ - **An Examples column can hold a whole JSON payload.** A request whose
262
+ body, or a part of it, is the JSON a column the step names holds is
263
+ recorded as that column, so each row sends its own payload. Before, the
264
+ payload was kept as fixed fields and every other row was refused. The
265
+ proposal names the body each row sends, and a cell that is not JSON fails
266
+ its row by name. Such a recording is schema version 10.
267
+ - **Comparisons compare like with like.** A "should differ" check between
268
+ two amounts compares the numbers, so the same price drawn in another
269
+ format has not changed. Two values of different kinds fail at once and
270
+ say why: `SEK 1,295` against `NOK 1,347` differ by currency alone, and a
271
+ whole button label against a price by its label. `saffron diff` and the
272
+ report show what each capture read in the proof replay beside its step.
273
+ A step that says an element is selectable, available or disabled records
274
+ an enabled or disabled check (`expectEnabled`, `expectDisabled`), where it
275
+ recorded visibility only; such a recording is schema version 9.
276
+ - **Recorded checks must be able to fail.** A proof shows a recording
277
+ replays, not that its checks notice anything. After a proof passes,
278
+ Saffron replays the scenario once more with no AI and runs each `Then`
279
+ on the page as it was just before the action step it follows. A check
280
+ that passes there too gets an `insensitive check:` note (picking the day
281
+ that was already picked, a price compared with a value it always had),
282
+ and `accept --all` leaves that proposal to be accepted by file. The
283
+ report shows "checks proven sensitive in N of M" beside the proofs that
284
+ passed.
285
+ - **Heals keep what the scenario tests.** A heal serves the scenario's
286
+ intent first, then the step's, and the step's wording last: it may change
287
+ how a step is done, never what the scenario exercises. The agent is told
288
+ never to replace typed data with a check of a hint, or a real payment or
289
+ sign-in route with a simulator's button, even when the final `Then`
290
+ passes, and to fail and name a missing prerequisite instead of working
291
+ around it. Saffron checks this itself: a healed step that no longer
292
+ enters, picks or uploads what the committed recording did gets a
293
+ `behavior change:` note naming what it checks instead, and
294
+ `accept --all` leaves that proposal to be accepted by file. A value moved
295
+ to another control (a text field that became a dropdown) is still
296
+ entered and raises no note.
297
+ - **iframes are recorded with their whole scope.** An element in an iframe
298
+ inside another iframe keeps both, and one of several alike iframes
299
+ (previews, cards) keeps its position, so replay acts in the iframe the
300
+ agent used. Before, an iframe picked with `first()` or `nth()` lost its
301
+ position and replay could act in another one, and an element in a nested
302
+ iframe recorded no action at all. Code the recorder cannot read, such as
303
+ an element reached through a variable or an iframe picked by a filter, now
304
+ fails the recording and names the interaction instead of being left out.
305
+ `saffron_assert` takes the same frame forms. A recording with a nested or
306
+ positioned iframe is written as schema version 8, which saffron-ai 1.2.0
307
+ and earlier ask to be updated for; others keep their version.
308
+ - **Codex and Antigravity on an API key.** A machine with no ChatGPT or
309
+ Google login, a CI job say, records and heals with Codex on
310
+ `CODEX_API_KEY` (or `OPENAI_API_KEY`) and with Antigravity on
311
+ `GEMINI_API_KEY` (or `GOOGLE_API_KEY`), billed to that API account. A
312
+ login on the machine wins over a key, so a key set for other tools never
313
+ moves your runs off your plan. Whether there is a login is asked only
314
+ when the first session starts, and only when a key is set, without
315
+ holding up other sessions; only the vendor saying you are not signed in
316
+ counts, so a timeout or a network error keeps the login. The run's first
317
+ line says how sessions will sign in, and the first session says which one
318
+ it used ("Codex: signed in with API key (CODEX_API_KEY)"); a key stored by
319
+ `codex login --with-api-key` reads as one, not as a ChatGPT login.
320
+ `saffron login codex` and `saffron login antigravity` say which sign-in
321
+ sessions will use. An expired login still counts as a login: the session
322
+ it fails says the login won over the key, and how to sign in again or use
323
+ the key. Codex's `forced_login_method` decides alone: a key never counts
324
+ as a sign-in under `"chatgpt"`, `"api"` with no key in the environment
325
+ or stored by `codex login --with-api-key` (a stored ChatGPT login does
326
+ not count) is reported as a missing key, and a refused session names the setting. Antigravity's
327
+ equivalent is `"antigravityApiKey": true` in `saffron.config.json`: the
328
+ key is used even beside an agy login, by every command, and a missing
329
+ key is reported and starts no session. A key reaches the
330
+ vendor alone: an error that quotes it, whole or cut short, shows its
331
+ variable's name, and it is never written into a recording or report.
332
+ Codex on a key is verified live; Antigravity on a key is not yet.
333
+ - **A numeric credential is never kept in a recording.** A login's token
334
+ that is only digits (a one-time code, a PIN) is written as its
335
+ `{captured:name}` in a form field, a multipart part or a JSON number
336
+ wherever it stands, whatever the field is called. Before, a field whose
337
+ name did not name an id kept the value.
338
+ - **An API is named by the words that call it.** The check that the APIs a
339
+ scenario needs are up reads a named API from its recording, or from steps
340
+ that call it one ("the orders API", "the orders service"): "I list orders
341
+ through the default API" no longer waits on a service named `orders`.
342
+ - **A changed contract the API breaks is red.** When a recording is checked
343
+ again against a changed API contract and the API no longer answers as the
344
+ new contract says, the scenario is red and the run exits 1, as before
345
+ 1.2.0, with no proposal filed (nothing was recorded again). In 1.2.0 it
346
+ was filed as an unverified proposal and the run passed. One that passes is
347
+ filed for review as before, and its run line says it replayed from the
348
+ recording, not that an agent recorded it. `saffron status` says such a
349
+ recording is checked again on the next run instead of "will re-record",
350
+ and the Saffron skill describes it.
351
+ - **A malformed `{unique:...}` stops the run before it starts.** An
352
+ unclosed token, an empty or invalid name, or one an Examples row fills in
353
+ with an empty cell (`{unique:<id>}`) is named and the run exits 2, as a
354
+ bad `{data:...}` does, instead of costing an agent session per row.
355
+ - **Failure headings name the row's values.** `Check "the customer is on
356
+ the "Gold" plan" failed`, as the detail below it already did.
357
+ - **A form field that is only a number** is written as a captured value's
358
+ token only where its name names an id or the capture, as in a JSON body:
359
+ `quantity=100` beside order 100 stays 100.
360
+ - **A status code with its own reason** names that status whatever
361
+ follows: "expect 409 Conflict and a retry hint".
362
+
363
+ Editors (VS Code extension 0.7.4, JetBrains plugin 0.7.5)
364
+
365
+ - **Proposals Accept All skips are marked.** A proposal with a "behavior
366
+ change:", "insensitive check:" or "volatile locator:" note, or one
367
+ imported from a CI run or not verified, is marked as held (VS Code:
368
+ "held for review"; JetBrains: "held from Accept All").
369
+ - **Orphans while a file fails to parse.** The Orphans view lists the
370
+ other files' orphans and groups the broken file's recordings under
371
+ "Ownership unknown until it parses"; prune waits until it parses. Needs
372
+ saffron-ai 1.2.1.
373
+ - **No more endless Loading.** A status or diff that never answers stops
374
+ after 60 seconds with its error, and later refreshes run again.
375
+
376
+ ## 1.2.0: API test automation (beta)
9
377
 
10
378
  Released 2026-10-26.
11
379
 
@@ -171,6 +539,14 @@ Runner
171
539
  proposals, proofs, heals and the verdict) and drives the browser and the
172
540
  HTTP medium through one contract. Nothing changes for a project.
173
541
 
542
+ Editors (VS Code extension 0.7.3, JetBrains plugin 0.7.4)
543
+
544
+ - **An API scenario's requests in Last Run.** Last Run never opens an API
545
+ scenario's request transcript as a screenshot: only failure and
546
+ before-heal pictures open as one, as the runner picks them. Such a row
547
+ offers Open Replay, which steps through the requests it sent. Needs
548
+ saffron-ai 1.2.0.
549
+
174
550
  ## 1.1.0: Secrets the AI agent never sees
175
551
 
176
552
  Released 2026-10-22.
package/README.md CHANGED
@@ -4,7 +4,7 @@
4
4
 
5
5
  Free to use · [npm: saffron-ai](https://www.npmjs.com/package/saffron-ai) · [site](https://saffron-ai.io)
6
6
 
7
- Write tests in plain language, as `.saffron` files or standard `.feature` files. `.saffron` is Saffron's own test language and goes further than Gherkin: step sets for reusable flows, `{env:VAR}` tokens that keep secrets out of recordings and reports, and `{data:...}` references to data files. There are no step definitions and no glue code. On the first run, an AI agent (Claude by default, or Codex, Google AI or Cursor on the subscription you already have, driving Playwright MCP) executes each scenario in a real browser and records what it did into a **cache**. A human-reviewable JSON action list committed to git. Every run after that, Saffron replays the cache in a real browser: **zero AI calls, zero tokens**, at the speed of the browser and your app. The cache is Saffron's own JSON, not a generated script; replay runs on Playwright's browser engine, and Playwright MCP is only how the agent operates the browser while it records or heals.
7
+ Write tests in plain language, as `.saffron` files or standard `.feature` files. `.saffron` is Saffron's own test language and goes further than Gherkin: step sets for reusable flows, `{env:VAR}` tokens that keep secrets out of recordings and reports, and `{data:...}` references to data files. There are no step definitions and no glue code. On the first run, an AI agent (Claude by default, or Codex, Antigravity or Cursor on the subscription you already have, driving Playwright MCP) executes each scenario in a real browser and records what it did into a **cache**: a human-reviewable JSON action list committed to git. Every run after that, Saffron replays the cache in a real browser: **zero AI calls, zero tokens**, at the speed of the browser and your app. The cache is Saffron's own JSON, not a generated script; replay runs on Playwright's browser engine, and Playwright MCP is only how the agent operates the browser while it records or heals.
8
8
 
9
9
  When the UI changes and a cached step fails at runtime, the agent takes over mid-execution with full scenario context, adapts, and finishes the run, then files a **cache proposal** you accept or reject, like a snapshot test update. Reports state exactly what was adapted and what it cost.
10
10
 
@@ -17,7 +17,7 @@ When the UI changes and a cached step fails at runtime, the agent takes over mid
17
17
  - **Verified proposals.** Every recording/heal is proof-replayed zero-AI before it's filed (with one bounded refinement pass on failure), so proposals arrive stamped `verified ✓` or honestly `UNVERIFIED ✗`.
18
18
  - **Honest cost reporting.** Reports break out prompt-cache reads/writes (the real bill of agent sessions) next to the in+out token count, per scenario and in totals, and name the models used. On an API key the dollars are what was billed; on a Claude subscription the headline is the 5-hour plan window before and after the run, with the dollars as the API-equivalent.
19
19
  - **Your subscription, your choice of agent.** Recording and healing run on the plan already logged in on the machine: Claude (default), Codex on a ChatGPT plan, Google AI Pro/Ultra through the Antigravity CLI (experimental), or Cursor. Set `"provider"` in the config; each one runs through its vendor's official client, and every provider records through the same recorder, so the caches are identical in kind.
20
- - **Trend memory.** Every run appends to `.saffron/history.jsonl`; reports show deltas vs the previous run and 20-run sparklines, and Saffron flags **chronic scenarios** (healing repeatedly. Re-record instead of paying again) plus recurring failure themes.
20
+ - **Trend memory.** Every run appends to `.saffron/history.jsonl`; reports show deltas vs the previous run and 20-run sparklines, and Saffron flags **chronic scenarios** (healing repeatedly: re-record instead of paying again) plus recurring failure themes.
21
21
 
22
22
  ## Step reuse: new scenarios get cheaper as your suite grows
23
23
 
@@ -128,8 +128,8 @@ Configuration lives in `saffron.config.json` at your project root:
128
128
  ```
129
129
 
130
130
  `provider` (also `--provider`) picks whose agent records and heals:
131
- `claude` (default), `codex` (ChatGPT plan; `npm i -D "@openai/codex-sdk@>=0.158.0"`),
132
- `antigravity` (Google AI Pro/Ultra through Google's `agy` CLI,
131
+ `claude` (default), `codex` (ChatGPT plan or `CODEX_API_KEY`; `npm i -D "@openai/codex-sdk@>=0.158.0"`),
132
+ `antigravity` (Google AI Pro/Ultra or `GEMINI_API_KEY`, through Google's `agy` CLI,
133
133
  experimental; `npx saffron login antigravity` once) or `cursor`
134
134
  (`npm i -D "@cursor/sdk@^1.0.32"`, then `npx saffron login cursor`). `model` takes
135
135
  that provider's model names. `npx saffron login` says what a provider