saffron-ai 0.9.1 → 0.9.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +426 -28
- package/README.md +14 -2
- package/dist-pkg/cli.js +240 -183
- package/package.json +16 -2
- package/skills/saffron/SKILL.md +10 -0
- package/skills/saffron/references/config.md +25 -11
package/CHANGELOG.md
CHANGED
|
@@ -2,9 +2,407 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to Saffron, in one place: the `saffron-ai` runner (npm),
|
|
4
4
|
the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
|
|
5
|
-
JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates.
|
|
5
|
+
JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
|
|
6
|
+
release heading names its main feature: `## <version>: <title>`.
|
|
6
7
|
|
|
7
|
-
## 0.9.
|
|
8
|
+
## 0.9.3: Review CI heals on your machine
|
|
9
|
+
|
|
10
|
+
Released 2026-10-04.
|
|
11
|
+
|
|
12
|
+
Runner
|
|
13
|
+
|
|
14
|
+
- **A heal made in CI can be reviewed and accepted on your machine.**
|
|
15
|
+
`saffron export` packs the pending proposals, the run's report and the
|
|
16
|
+
screenshots and traces it names into one zip, with where the run
|
|
17
|
+
happened (repository, branch, commit, CI run, shard); CI uploads it as
|
|
18
|
+
an artifact. `saffron import` checks it against your checkout and files
|
|
19
|
+
it where a local run would have, so `saffron diff`, `saffron accept`,
|
|
20
|
+
`saffron trace`, the report and both IDE panels work on it unchanged. A
|
|
21
|
+
bundle from another repository (by its path; the same path under
|
|
22
|
+
another host name, such as an SSH host alias, imports with a warning),
|
|
23
|
+
one that does not say its repository (unless
|
|
24
|
+
`--allow-unknown-repository`), of a newer format, with a damaged file or
|
|
25
|
+
a report it cannot show, or with a file over 1 GB (or over 100 times
|
|
26
|
+
its size unzipped) is refused whole, before anything is written, and so
|
|
27
|
+
are bundles that add up to over 8 GB unzipped together; a proposal made
|
|
28
|
+
against a different scenario text or committed recording is refused by
|
|
29
|
+
name, and the rest import. Bundles are read from disk one file at a
|
|
30
|
+
time, a URL's downloaded to disk first, so every shard of a large run
|
|
31
|
+
imports at once. Text from a bundle, in errors and reasons too, is
|
|
32
|
+
printed without control characters. Export keeps to the same limits:
|
|
33
|
+
it refuses a file over 1 GB and, when a bundle would be over 4 GB,
|
|
34
|
+
leaves the traces of green scenarios out first and says so. Shard
|
|
35
|
+
bundles merge into one report. The imported report says where it came
|
|
36
|
+
from and links to the CI run. Examples for GitHub Actions and Azure
|
|
37
|
+
Pipelines are in the guide (§19).
|
|
38
|
+
- **Or CI opens a draft pull request with the heals.**
|
|
39
|
+
`saffron export --branch <name>` commits the proposal files, not the
|
|
40
|
+
recordings, onto the commit the run tested (on a pull request build, the
|
|
41
|
+
head of its branch, where each proposal's binding is checked again and
|
|
42
|
+
one that does not fit is left out, judged by that commit's own StepSets
|
|
43
|
+
and data) as a new branch, pushes it and, on GitHub, opens a
|
|
44
|
+
draft pull request through `gh` whose description is each proposal's
|
|
45
|
+
verification and diff, with the commands to accept, and a staging line
|
|
46
|
+
that stages exactly what accept changed (`git add -A .saffron/cache &&
|
|
47
|
+
git add -u .saffron/proposals`). After a tag build, which tested no
|
|
48
|
+
branch, name the pull request's branch with `--base`. Only the
|
|
49
|
+
checkout's HEAD, a commit it merged in or one it descends from is ever
|
|
50
|
+
built on; a `pull_request_target` build records the base branch and the
|
|
51
|
+
commit it checked out, never the fork's. The reviewer
|
|
52
|
+
switches to the branch, runs `saffron accept`, commits and marks it
|
|
53
|
+
ready: no agent session on their machine. The CI checkout is left as it
|
|
54
|
+
was, and an existing branch is never pushed onto.
|
|
55
|
+
- **`saffron status --json` says which CI run a proposal was imported
|
|
56
|
+
from** (`importedFrom`: repository, branch, commit, run and its link,
|
|
57
|
+
shard), for as long as it is the proposal that was imported, so the
|
|
58
|
+
editors can show it past the next local run.
|
|
59
|
+
- **`saffron runs` and `saffron import --run <id>`, for GitHub Actions and
|
|
60
|
+
Azure Pipelines alike.** `runs` lists the branch's CI runs (through `gh`,
|
|
61
|
+
or `az` with its azure-devops extension), with how many of each run's
|
|
62
|
+
proposals are pending here; `import --run` downloads a run's bundles and
|
|
63
|
+
imports them. The repository's host picks the service; a `ci` key in the
|
|
64
|
+
config sets it, and the Azure DevOps organization and project, for a
|
|
65
|
+
GitHub repository built by Azure Pipelines. An Azure pull request build
|
|
66
|
+
is listed under the branch it validates, and only the repository's own
|
|
67
|
+
builds are listed. Bundles of a re-run take each shard from its latest
|
|
68
|
+
attempt. `import --run` downloads into a folder under `.saffron/reports`
|
|
69
|
+
and removes it when done. A download that takes over 10 minutes is
|
|
70
|
+
stopped and says so. A tag build is listed under its tag on both
|
|
71
|
+
services. An Azure DevOps SSH remote through a host alias counts as
|
|
72
|
+
Azure. With `--json`, every `import` failure, an invalid config
|
|
73
|
+
included, is `{"error": ...}` and exit 2. A `runs --json` whose `gh` or
|
|
74
|
+
`az` is missing, signed out or failing answers `{provider, branch,
|
|
75
|
+
runs: [], problem: {kind, message}, error}`, `error` holding the same
|
|
76
|
+
message, so a client reading either field works; any other failure is
|
|
77
|
+
`{"error": ...}` alone; both exit 2. On Azure Pipelines the pull
|
|
78
|
+
request builds are asked for once (`--top 500`), so a listing usually takes
|
|
79
|
+
three `az` calls, not seven.
|
|
80
|
+
`export --branch` opens the draft pull request on Azure DevOps too,
|
|
81
|
+
through `az repos pr create`.
|
|
82
|
+
- **Export streams the bundle to disk.** Screenshots and traces are
|
|
83
|
+
copied into the zip in chunks, never all held in memory, and the zip
|
|
84
|
+
is renamed into place only when complete; a file that changes while it
|
|
85
|
+
is packed is refused.
|
|
86
|
+
- **Failures are one line.** `export` exits 2 for every failure that
|
|
87
|
+
pushed nothing (an `--out` that is a folder, an invalid config with
|
|
88
|
+
`--branch`), keeping exit 1 for a branch pushed without its pull
|
|
89
|
+
request. Any command's expected failure prints `saffron: <message>`
|
|
90
|
+
and exits 2, never a stack; `SAFFRON_DEBUG=1` shows it.
|
|
91
|
+
- **Safe branch names in the pull request.** The description quotes the
|
|
92
|
+
branch in its commands, and `--branch` or `--base` with a space, a
|
|
93
|
+
quote or any of `` $ ` ; | & < > ( ) \ `` is refused up front.
|
|
94
|
+
- **A `pull_request_review` build records its pull request's branch**,
|
|
95
|
+
read from the event, not `12/merge`.
|
|
96
|
+
- **`import --json` passes on only the manifest fields it knows**, cleaned
|
|
97
|
+
of control characters.
|
|
98
|
+
- **The pull request's review steps run saffron-ai by name**
|
|
99
|
+
(`npx -y -p saffron-ai saffron diff`): in a checkout without
|
|
100
|
+
`node_modules`, a bare `npx saffron` would fetch the unrelated `saffron`
|
|
101
|
+
package.
|
|
102
|
+
- **A bundle names its run as the CI service does** (`github:<run id>`,
|
|
103
|
+
`azure:<build id>`), even with `SAFFRON_RUN_ID` set, so `saffron runs`
|
|
104
|
+
and both editors count its proposals under the right run.
|
|
105
|
+
- **A green scenario's trace over 1 GB is left out of the bundle** like
|
|
106
|
+
the other green traces, instead of failing the export.
|
|
107
|
+
- **Import hashes, compares and copies traces in chunks**: a 1 GB trace is
|
|
108
|
+
never held in memory. A bundle that changes on disk while it is
|
|
109
|
+
imported no longer says "nothing was imported" after some files were
|
|
110
|
+
written.
|
|
111
|
+
- **`saffron runs` on Azure asks for 50 pull request builds first**, and
|
|
112
|
+
for more only while too few are the branch's, not 500 on every list.
|
|
113
|
+
|
|
114
|
+
VS Code extension 0.4.0
|
|
115
|
+
|
|
116
|
+
- **CI runs**: a new view lists the current branch's CI runs, GitHub
|
|
117
|
+
Actions or Azure Pipelines, through `saffron runs`. **Import** runs
|
|
118
|
+
`saffron import --run`, then lists under the run what happened to each
|
|
119
|
+
proposal and any warning (a checkout at another commit, a scenario that
|
|
120
|
+
changed since). Imported proposals are marked with the run they came
|
|
121
|
+
from, linked in the tooltip. From a red CI run to an accepted repair
|
|
122
|
+
without leaving the editor; commit and push with the Source Control
|
|
123
|
+
view. The CI's own tool keeps the credentials.
|
|
124
|
+
- **Text from CI cannot act in a tooltip.** Run titles, branch names and
|
|
125
|
+
a bundle's notes are escaped so they never form links, headings or
|
|
126
|
+
other markup; a run links out only to GitHub or Azure DevOps, and never
|
|
127
|
+
when the address carries a user name. Import results are kept per
|
|
128
|
+
project. An import can be cancelled, and on Windows cancelling stops
|
|
129
|
+
the whole process tree. The list follows a branch switch, and makes no
|
|
130
|
+
CI call while its view is hidden.
|
|
131
|
+
- **Cancelling an import stops all of it, and says what it left.** On
|
|
132
|
+
Windows the whole process tree is stopped however the CLI was started;
|
|
133
|
+
on macOS and Linux a process that outlives SIGTERM gets SIGKILL. The
|
|
134
|
+
message says a cancelled import may have written part of the run.
|
|
135
|
+
|
|
136
|
+
JetBrains plugin 0.4.0
|
|
137
|
+
|
|
138
|
+
- **CI Runs tab**: the same runs and the same Import, through the same
|
|
139
|
+
two commands, with each proposal's outcome under its run. The Proposals
|
|
140
|
+
tab names the CI run an imported proposal came from.
|
|
141
|
+
- **Text from CI is shown as plain text**, in CI Runs and in Last Run
|
|
142
|
+
after an import, never as markup. Import is a background task you can
|
|
143
|
+
cancel, with no time limit; a run is imported once even if you switch
|
|
144
|
+
projects meanwhile; Refresh never starts a second list. The tab loads
|
|
145
|
+
as soon as the project is trusted, and its processes stop when the
|
|
146
|
+
project closes.
|
|
147
|
+
- **Choosing another project clears the tabs at once**: a run listed for
|
|
148
|
+
the last project can no longer be imported into the new one, and a
|
|
149
|
+
status that arrives late for the last project is dropped.
|
|
150
|
+
- **CI Runs lists again when you check out another branch.** A cancelled
|
|
151
|
+
import says it may have written part of the run.
|
|
152
|
+
|
|
153
|
+
## 0.9.2: Bring your own AI agent
|
|
154
|
+
|
|
155
|
+
Released 2026-09-30.
|
|
156
|
+
|
|
157
|
+
Runner
|
|
158
|
+
|
|
159
|
+
- **Browser tool calls are checked call by call.** A security review of
|
|
160
|
+
the agent's tools found three ways a prompt-injected page could reach
|
|
161
|
+
the machine: `view-source:file://` slipped past Playwright's `file:`
|
|
162
|
+
check and read any file; the `filename` option of the snapshot,
|
|
163
|
+
screenshot, console and network tools wrote page content over any
|
|
164
|
+
project file (a response body into a bin is code execution on the next
|
|
165
|
+
run); and a file upload could hand `.env`, with every secret, to the
|
|
166
|
+
page. Navigation now opens web pages only, uploads take existing files
|
|
167
|
+
inside the project by their real path (a link out of the project is
|
|
168
|
+
out) and never dotfiles, `filename` is dropped, and anything the
|
|
169
|
+
vendor's tools write goes to a private folder of the session, removed
|
|
170
|
+
when it ends. The checks sit in the one
|
|
171
|
+
proxy every provider and both paths use, and replay applies the same
|
|
172
|
+
ones to a cache, which is data anyone can commit. A project reached
|
|
173
|
+
through a symlinked path (`/var` and `/private/var` on macOS) counts
|
|
174
|
+
as the project. Claude sessions also set
|
|
175
|
+
`strictMcpConfig`, so a `.mcp.json` or user-level MCP server of yours
|
|
176
|
+
never joins a recording. Codex sessions turn off its remaining
|
|
177
|
+
execution features (`unified_exec`, `multi_agent`, `code_mode_host`,
|
|
178
|
+
the in-app and external browsers) besides the shell.
|
|
179
|
+
- **Go-to-definition works under a folder with a space, `#` or `%` in
|
|
180
|
+
its name, and on Windows.** The language server sent unencoded file
|
|
181
|
+
paths as URIs; a strict client rejected them. A nested project without
|
|
182
|
+
a `features` key now gets the default folder instead of the outer
|
|
183
|
+
project's, and on Windows a StepSet in a subfolder is no longer flagged
|
|
184
|
+
as defined twice.
|
|
185
|
+
- **`--filter "not @wip @slow"` is refused** instead of reading as
|
|
186
|
+
`not @wip or @slow`: once `and`, `not` or parentheses appear, every
|
|
187
|
+
tag needs its `and` or `or`. The plain list forms are unchanged.
|
|
188
|
+
- **Outline values no longer replace inside a test id or frame selector
|
|
189
|
+
by coincidence** (`add` in `add-to-cart`), while a short id with a
|
|
190
|
+
digit still does (`1234` in `order-1234`), and a `<name>` that opens a
|
|
191
|
+
regex named group is left alone at replay.
|
|
192
|
+
- **The cost trend leaves a gap for runs whose provider reported no cost**
|
|
193
|
+
instead of plotting them as $0, and every other run keeps its place. Cursor's turn limit counts tool calls,
|
|
194
|
+
not the pieces their arguments stream in. An Antigravity session that
|
|
195
|
+
hits the turn limit says so, and how to raise it, even when agy
|
|
196
|
+
reports the session a success.
|
|
197
|
+
|
|
198
|
+
- **No agent can run code on your machine through Playwright.**
|
|
199
|
+
Playwright MCP's `browser_run_code_unsafe` runs any JavaScript in the
|
|
200
|
+
Playwright server process (its own docs call it RCE-equivalent); a
|
|
201
|
+
Codex test session used it to read a file from the home directory. It
|
|
202
|
+
is now hidden from every provider's agent, Claude's included, and a call
|
|
203
|
+
to it by name is refused. Nothing recorded depends on it.
|
|
204
|
+
`browser_evaluate`, which runs in the page, stays.
|
|
205
|
+
- **Claude recordings no longer carry your claude.ai connectors.** On a
|
|
206
|
+
Claude subscription login, Claude Code loads the account's claude.ai
|
|
207
|
+
connectors (Gmail, Figma, Canva, ElevenLabs and the like) into every
|
|
208
|
+
session from its second request on. On one account that added 294 tool
|
|
209
|
+
definitions, 1.16 MB of schema, to every model call: recording the
|
|
210
|
+
guestbook example cost $15.74 at API rates and 11.5M cache-read tokens.
|
|
211
|
+
Saffron's sessions now set `disableClaudeAiConnectors`, and the same
|
|
212
|
+
recording costs $0.37 with 279k cache-read tokens, in 70 s instead of
|
|
213
|
+
180 s. The agent never needed those tools; your connectors in Claude
|
|
214
|
+
itself are untouched.
|
|
215
|
+
- **Record and heal on the subscription you already have.** Besides
|
|
216
|
+
Claude (still the default), `"provider"` in `saffron.config.json` (or
|
|
217
|
+
`--provider`) now takes `codex` (a ChatGPT plan, through
|
|
218
|
+
`@openai/codex-sdk`), `antigravity` (Google AI Pro/Ultra, through
|
|
219
|
+
Google's `agy` CLI; experimental) and `cursor` (through `@cursor/sdk`).
|
|
220
|
+
Each runs on the login its official client already has on the machine.
|
|
221
|
+
These agents reach the browser through a local MCP gateway that records
|
|
222
|
+
every call with the same hooks as Claude, so their recordings, heals and
|
|
223
|
+
proof replays work the same. `model` takes the provider's own model
|
|
224
|
+
names; `saffron author` takes `--provider` too. The two SDKs are
|
|
225
|
+
optional peer dependencies: install the one you use. Codex runs without
|
|
226
|
+
your own `~/.codex/config.toml` apart from its login settings (a keyring
|
|
227
|
+
login works), which needs macOS or Linux for now: on Windows the Codex
|
|
228
|
+
provider refuses to start rather than run with your own MCP servers.
|
|
229
|
+
`agy` gets an explicit `--print-timeout` sized to the turn budget, and
|
|
230
|
+
an `agy` that cannot start fails its scenario instead of the run.
|
|
231
|
+
Antigravity is verified live with agy 1.2.13 (record, proof replay and
|
|
232
|
+
heal): its MCP calls go through agy's `call_mcp_tool`, allowed by two
|
|
233
|
+
rules in agy's settings and limited by a hook to Saffron's two servers.
|
|
234
|
+
- **`saffron login [provider]`** says what a provider still needs on this
|
|
235
|
+
machine, and does the one-time setup where one is needed:
|
|
236
|
+
`saffron login cursor` runs Cursor's sign-in, and
|
|
237
|
+
`saffron login antigravity` checks that agy is signed in and allows
|
|
238
|
+
Saffron's tools in agy's settings (two rules, everything else kept).
|
|
239
|
+
- **New docs page: AI providers and sign-in** (`docs/PROVIDERS.md`), with
|
|
240
|
+
every provider's sign-in, what `saffron login` sets up, what Saffron does
|
|
241
|
+
in its sessions, how to undo it, and troubleshooting.
|
|
242
|
+
- **A pass can no longer leave an assertion unchecked.** `saffron_done`
|
|
243
|
+
with success is refused once while an announced `Then` step has
|
|
244
|
+
recorded no check, and Saffron's tool refusals are now MCP errors, so an
|
|
245
|
+
agent that batches its calls stops and fixes the step instead of filing
|
|
246
|
+
an unverified proposal. A second pass stands (the proof replay still
|
|
247
|
+
marks it unverified), so no session burns its turns on it. A step
|
|
248
|
+
announced twice keeps one adaptation note, and an agent that fails
|
|
249
|
+
after `saffron_done` keeps its verdict.
|
|
250
|
+
- **Node.js 20.6 or newer** is now required (22.13 for the Cursor
|
|
251
|
+
provider).
|
|
252
|
+
- **Cost not reported, not $0.** Runs whose provider reports no price say
|
|
253
|
+
so in the summary line and the report instead of showing $0.0000.
|
|
254
|
+
- **The editor finds a project nested inside a bigger folder.** When the
|
|
255
|
+
IDE is opened above the Saffron project (a solution folder with the
|
|
256
|
+
tests in a subfolder), `saffron lsp` now resolves each `.saffron` file
|
|
257
|
+
against the nearest `saffron.config.json` above it. StepSets defined in
|
|
258
|
+
a sibling file and `{data:...}` references were reported as undefined
|
|
259
|
+
in that layout; they resolve now, and completion, hover and
|
|
260
|
+
go-to-definition follow the same root.
|
|
261
|
+
- **A browser tool result can no longer flood the agent's context.** A
|
|
262
|
+
recording fetched a 1.5 MB API response body with
|
|
263
|
+
`browser_network_request` (about 400k tokens). Claude Code then compacted
|
|
264
|
+
the session in a loop and failed the scenario after a few dollars of
|
|
265
|
+
tokens. Every browser tool result is now cut at 60,000 characters, with
|
|
266
|
+
a note asking for less (a snapshot of one region, a filtered request
|
|
267
|
+
list, or a `saffron_assert` on the field). The Playwright code Saffron
|
|
268
|
+
records comes first in a result, so a cut result still records its
|
|
269
|
+
action. The cap applies with and without the trace bridge, and to Codex,
|
|
270
|
+
Antigravity and Cursor sessions through the tool gateway.
|
|
271
|
+
- **Each Outline row acts on its own element through a locator chain.** A
|
|
272
|
+
chain keeps the row's value inside its selector; that value now becomes
|
|
273
|
+
the row's `<param>` (or data-table reference) there too, so another row
|
|
274
|
+
no longer clicks the recorded row's card. The same goes for every other
|
|
275
|
+
field replay resolves: an assertion's regex pattern (an `href`, a
|
|
276
|
+
heading's name), a test id, a prompt answer, a frame and a drag's drop
|
|
277
|
+
target. An Outline checking a link's `href` failed as written on every
|
|
278
|
+
row but the recorded one.
|
|
279
|
+
- **A changed table or data value still written in the recording is found
|
|
280
|
+
escaped too** (in a selector's quoted or regex body, or a pattern), so
|
|
281
|
+
the recording is made again instead of replaying the old value.
|
|
282
|
+
- **Chains with a comma, an apostrophe or a role option record the element
|
|
283
|
+
they clicked.** `filter({ hasText: 'Paris, France' })` and
|
|
284
|
+
`getByText(/^Don't$/)` recorded nothing, a regex with an apostrophe made
|
|
285
|
+
a selector Playwright read differently, and `getByRole('heading', {
|
|
286
|
+
level: 2 })` recorded any heading. Role options are kept as selector
|
|
287
|
+
attributes; one with no selector form records nothing.
|
|
288
|
+
- **Tag lists read the way 0.9.0 read them.** An empty entry (`@a,@b,`) is
|
|
289
|
+
skipped again, and a tag's own parentheses (`@issue(42)`) stay part of
|
|
290
|
+
it; a parenthesis after a space or a keyword still groups.
|
|
291
|
+
- **Replay fixes.** A heal that fixes a network assertion is checked
|
|
292
|
+
against the response the agent's session produced, instead of failing
|
|
293
|
+
"as written" while waiting for a new one. A replay no longer inherits an
|
|
294
|
+
upload armed on its page by an earlier replay (a pending proposal that
|
|
295
|
+
failed), which failed its first click. A value resolved into a regex
|
|
296
|
+
pattern (`{env:VAR}`, `<param>`, `{data:...}`) matches as written: the
|
|
297
|
+
`+` in `a+b@x.com` or the `$` in `$9.99` no longer change the pattern.
|
|
298
|
+
- **A failed tool call records nothing** in Codex, Antigravity and Cursor
|
|
299
|
+
sessions, as in Claude's: an upload refused because no file chooser was
|
|
300
|
+
open was recorded, and broke replay.
|
|
301
|
+
- **Antigravity gets its prompt on stdin.** The prompt carries resolved
|
|
302
|
+
`{env:VAR}` values, and `agy -p` put it in the command line, readable by
|
|
303
|
+
every account on the machine.
|
|
304
|
+
- **Tools a recording cannot replay are hidden from every agent:** going
|
|
305
|
+
back or forward, reloading, resizing and dropping files.
|
|
306
|
+
`browser_navigate` reaches any page instead, and is recorded.
|
|
307
|
+
- **`saffron_done` is refused once when a `Then` step recorded only setup**
|
|
308
|
+
(a click that opens a panel) and no check, the rule replay applies.
|
|
309
|
+
- **Playwright's MCP server gets the whole environment again** when the
|
|
310
|
+
trace bridge is off, so `PLAYWRIGHT_MCP_*` settings apply there too.
|
|
311
|
+
- **A proposal lock held by a live process on this machine is broken after
|
|
312
|
+
an hour**: a pid reused after its holder died kept it forever.
|
|
313
|
+
- **`saffron status` says "cost not reported"** for a provider with no
|
|
314
|
+
price, instead of $0.00. `saffron status --json` leaves such a cost
|
|
315
|
+
out (the last run's, history's and proposals'), and the editors'
|
|
316
|
+
last-run lines say "cost not reported" too (VS Code and JetBrains, next
|
|
317
|
+
editor release).
|
|
318
|
+
- **A Codex or Cursor session stopped at the turn limit says so**, and to
|
|
319
|
+
raise `maxTurns`, instead of "ended without finalizing".
|
|
320
|
+
- **`saffron login cursor` on Node below 22.13 names the Node version it
|
|
321
|
+
needs**, and a vendor SDK missing a part of its own reports that part
|
|
322
|
+
instead of asking to install the SDK again.
|
|
323
|
+
- **A single locator with options records the element it acted on.**
|
|
324
|
+
`getByText('Save', { exact: true })`, `getByLabel('Email', { exact:
|
|
325
|
+
true })` and `getByText(/^Save$/)` recorded nothing, so the click or
|
|
326
|
+
fill was missing from the recording, and `getByRole('checkbox', {
|
|
327
|
+
checked: false })` recorded any checkbox. They record as a selector now,
|
|
328
|
+
as in a chain.
|
|
329
|
+
- **An Outline row's value inside a regex matches the row replaying.**
|
|
330
|
+
Playwright writes exact text as a regex (`filter({ hasText: /^Garden
|
|
331
|
+
Queen$/ })`); that value becomes the row's `<param>` as well, and replay
|
|
332
|
+
writes the next row's value escaped, quotes, slashes and brackets
|
|
333
|
+
included.
|
|
334
|
+
- **A secret inside a selector or an assertion's pattern is masked where
|
|
335
|
+
it is written escaped** (a quote in a quoted name, a `.` in a regex), and
|
|
336
|
+
replay writes it back the same way. A short one next to an escape
|
|
337
|
+
(`\buat\b`) counts as standing alone.
|
|
338
|
+
- **The recorder reads only the code Playwright ran.** A snapshot or a
|
|
339
|
+
response body that showed Playwright code (a docs page, a code viewer)
|
|
340
|
+
was recorded as actions.
|
|
341
|
+
- **Only tools a recording can hold reach the agent**, whatever
|
|
342
|
+
`PLAYWRIGHT_MCP_CAPS` turns on: an allowlist of the tools that record or
|
|
343
|
+
only read replaces the list of hidden ones, so coordinate mouse,
|
|
344
|
+
storage, cookie and route tools stay out.
|
|
345
|
+
- **A heal checks a network assertion against the response that came
|
|
346
|
+
late.** The failed replay's response log keeps listening through the
|
|
347
|
+
heal, so a response that arrived just after replay gave up counts.
|
|
348
|
+
- **A cut browser result never splits an emoji**, which made Codex drop
|
|
349
|
+
the whole result.
|
|
350
|
+
- **`saffron login antigravity` writes through a settings link** (a
|
|
351
|
+
dotfiles repo, even before the file exists there, through chains and
|
|
352
|
+
linked folders) and keeps the file's permissions; a link into a
|
|
353
|
+
read-only place (a Nix store) is left as it is, with the two rules to
|
|
354
|
+
add named.
|
|
355
|
+
- **The editor keeps each nested project's vocabulary**, built once, in
|
|
356
|
+
editors that watch its files for it (VS Code, JetBrains), and a changed
|
|
357
|
+
file refreshes every open project's diagnostics. A project whose build
|
|
358
|
+
fails keeps its last good one, and coming back to a project checks its
|
|
359
|
+
documents again. Opening a file it already read (go to definition)
|
|
360
|
+
rebuilds nothing.
|
|
361
|
+
- **A browser action a Codex, Antigravity or Cursor agent gave up on is
|
|
362
|
+
still recorded.** Playwright finishes a click or a goto whatever the
|
|
363
|
+
agent does meanwhile, so the recording keeps what the browser did.
|
|
364
|
+
|
|
365
|
+
Editors (VS Code extension 0.3.3, JetBrains plugin 0.3.3)
|
|
366
|
+
|
|
367
|
+
- **A project nested inside the folder the IDE opened.** Both editors
|
|
368
|
+
now find the nearest `saffron.config.json` above a file and treat that
|
|
369
|
+
folder as the project: runs start there, status, proposals, `.env` and
|
|
370
|
+
the step vocabulary come from there, and the project's own `saffron`
|
|
371
|
+
install is used. JetBrains shows a chooser in the tool window title
|
|
372
|
+
when the IDE root holds several projects, and adds a Working directory
|
|
373
|
+
to run configurations. Before, a run from a nested project wrote
|
|
374
|
+
`.saffron/` into the parent folder with default settings and recorded
|
|
375
|
+
everything again.
|
|
376
|
+
- **JetBrains: nothing runs in a project you have not trusted.** No
|
|
377
|
+
status, runs, diffs or language server until the project is trusted;
|
|
378
|
+
the tool window says so and loads as soon as you trust it. The
|
|
379
|
+
language server and the Run menu entries appear only inside a Saffron
|
|
380
|
+
project (a `saffron.config.json`, an installed `saffron-ai`, or
|
|
381
|
+
`.saffron` files), so a Cucumber or Behat folder of `.feature` files
|
|
382
|
+
no longer launches `npx -y`. A nested project is left out of the outer
|
|
383
|
+
project's file list and never run from the outer root.
|
|
384
|
+
- **VS Code on Windows: runs no longer pass through a `.cmd` shim**,
|
|
385
|
+
which cmd.exe re-parses. The installed CLI runs with node directly;
|
|
386
|
+
the npx fallback escapes every argument.
|
|
387
|
+
- **JetBrains: reused run configurations no longer keep a Re-record tick
|
|
388
|
+
from an earlier use.** `.feature` files appear in the Files tab and
|
|
389
|
+
run from the context menu. A new `login` command takes the provider.
|
|
390
|
+
Links in the Dashboard open in your browser. The tool window refreshes
|
|
391
|
+
only for Saffron runs and changes inside the project.
|
|
392
|
+
- **Both editors list proposal files the runner cannot read**, with the
|
|
393
|
+
reason, and count them as pending. A run with no reported cost shows
|
|
394
|
+
as not reported, never as $0.
|
|
395
|
+
- **VS Code:** the panel keeps the project you were on when a webview,
|
|
396
|
+
diff or image takes focus, and Select Project lists nested projects;
|
|
397
|
+
Run Tagged keeps tags written with parentheses such as `@issue(42)`;
|
|
398
|
+
stale status after a run is gone; status errors show the
|
|
399
|
+
error line, not the last stack frame; lines inside docstrings are not
|
|
400
|
+
read as steps; `Example:` is a scenario keyword; Re-record works with
|
|
401
|
+
Replay only ticked; `{data:...}` flags an empty list and a header-only
|
|
402
|
+
CSV; Run Tagged skips tags `--filter` cannot express and says which;
|
|
403
|
+
diagnostics clear when a file closes or is deleted.
|
|
404
|
+
|
|
405
|
+
## 0.9.1: Tag expressions: and, or, not
|
|
8
406
|
|
|
9
407
|
Released 2026-09-29.
|
|
10
408
|
|
|
@@ -68,7 +466,7 @@ Editors (VS Code extension 0.3.2, JetBrains plugin 0.3.2)
|
|
|
68
466
|
tabs, under any file icon theme that has no icon of its own for them
|
|
69
467
|
(Seti, the default, included).
|
|
70
468
|
|
|
71
|
-
## 0.9.0
|
|
469
|
+
## 0.9.0: Time-travel replay
|
|
72
470
|
|
|
73
471
|
Released 2026-09-27.
|
|
74
472
|
|
|
@@ -263,7 +661,7 @@ Editors (VS Code extension 0.3.0, JetBrains plugin 0.3.0)
|
|
|
263
661
|
what is pending when it runs, skipping UNVERIFIED proposals and
|
|
264
662
|
refusing STALE ones.
|
|
265
663
|
|
|
266
|
-
## 0.8.5
|
|
664
|
+
## 0.8.5: Proposals bound to what you reviewed
|
|
267
665
|
|
|
268
666
|
Released 2026-09-24.
|
|
269
667
|
|
|
@@ -302,7 +700,7 @@ Editors (VS Code extension 0.2.10, JetBrains plugin 0.2.5)
|
|
|
302
700
|
- Both Proposals panels mark a stale proposal STALE, with the reason and
|
|
303
701
|
what to do, and note a proposal filed by an older runner.
|
|
304
702
|
|
|
305
|
-
## 0.8.4
|
|
703
|
+
## 0.8.4: Heal and replay fixes
|
|
306
704
|
|
|
307
705
|
Released 2026-09-24.
|
|
308
706
|
|
|
@@ -415,7 +813,7 @@ Editors (VS Code extension 0.2.9)
|
|
|
415
813
|
files and to the config now refresh every open feature file, including a
|
|
416
814
|
custom data folder and environment overlays.
|
|
417
815
|
|
|
418
|
-
## 0.8.3
|
|
816
|
+
## 0.8.3: Data references in table cells
|
|
419
817
|
|
|
420
818
|
Released 2026-09-22.
|
|
421
819
|
|
|
@@ -431,7 +829,7 @@ Runner
|
|
|
431
829
|
Code currently defaults to, which drifts) and which values it takes, so a
|
|
432
830
|
project can pin its recording model.
|
|
433
831
|
|
|
434
|
-
## 0.8.2
|
|
832
|
+
## 0.8.2: Split a suite across CI machines
|
|
435
833
|
|
|
436
834
|
Released 2026-09-22.
|
|
437
835
|
|
|
@@ -454,7 +852,7 @@ Runner
|
|
|
454
852
|
twice does not count the run twice. Setup and teardown run on each shard
|
|
455
853
|
and receive `SAFFRON_SHARD`. The docs have a GitHub Actions example.
|
|
456
854
|
|
|
457
|
-
## 0.8.1
|
|
855
|
+
## 0.8.1: Chrome, Edge, and setup and teardown
|
|
458
856
|
|
|
459
857
|
Released 2026-09-22.
|
|
460
858
|
|
|
@@ -517,7 +915,7 @@ Editors (VS Code extension 0.2.8)
|
|
|
517
915
|
hover and go-to-definition now follow the runner: the environment's CSV
|
|
518
916
|
replaces the base one, and the hover names the file it read.
|
|
519
917
|
|
|
520
|
-
## 0.8.0
|
|
918
|
+
## 0.8.0: Test data outside the feature file
|
|
521
919
|
|
|
522
920
|
Released 2026-09-21.
|
|
523
921
|
|
|
@@ -583,7 +981,7 @@ Editors (VS Code extension 0.2.7, JetBrains plugin 0.2.4)
|
|
|
583
981
|
- Both Orphans lists show the screenshot folders of scenarios that no longer
|
|
584
982
|
exist, so "Remove All" never deletes something it did not show.
|
|
585
983
|
|
|
586
|
-
## 0.7.2
|
|
984
|
+
## 0.7.2: Failure screenshots
|
|
587
985
|
|
|
588
986
|
Released 2026-09-21.
|
|
589
987
|
|
|
@@ -638,7 +1036,7 @@ Editors (JetBrains plugin 0.2.2)
|
|
|
638
1036
|
notification says so when it is missing. The presence check uses no
|
|
639
1037
|
plugin-manager API, so it stays clean as that API turns internal.
|
|
640
1038
|
|
|
641
|
-
## 0.7.1
|
|
1039
|
+
## 0.7.1: Truthful checks and safer edits
|
|
642
1040
|
|
|
643
1041
|
Released 2026-09-20.
|
|
644
1042
|
|
|
@@ -709,7 +1107,7 @@ Editors (VS Code extension 0.2.5)
|
|
|
709
1107
|
passes the tree item, not the file path the row click passes; both forms
|
|
710
1108
|
are accepted, and the project guard applies to both.
|
|
711
1109
|
|
|
712
|
-
## 0.7.0
|
|
1110
|
+
## 0.7.0: saffron diff, saffron prune, and an external review
|
|
713
1111
|
|
|
714
1112
|
Released 2026-09-19.
|
|
715
1113
|
|
|
@@ -902,7 +1300,7 @@ Editors (JetBrains plugin 0.2.2, VS Code extension 0.2.4)
|
|
|
902
1300
|
longer shell syntax, and a file from a second workspace folder runs
|
|
903
1301
|
against that folder's config and install.
|
|
904
1302
|
|
|
905
|
-
## 0.6.0
|
|
1303
|
+
## 0.6.0: Multi-tab scenarios
|
|
906
1304
|
|
|
907
1305
|
Released 2026-09-18.
|
|
908
1306
|
|
|
@@ -924,7 +1322,7 @@ Editors
|
|
|
924
1322
|
its short description fits without being cut off, and the overview
|
|
925
1323
|
introduces the tool and how to get started before listing features.
|
|
926
1324
|
|
|
927
|
-
## 0.5.7
|
|
1325
|
+
## 0.5.7: Claude Code plugin
|
|
928
1326
|
|
|
929
1327
|
Released 2026-09-17.
|
|
930
1328
|
|
|
@@ -948,7 +1346,7 @@ Agents
|
|
|
948
1346
|
points at `saffron status`. Run `npx saffron init` again to refresh both
|
|
949
1347
|
in a project.
|
|
950
1348
|
|
|
951
|
-
## 0.5.6
|
|
1349
|
+
## 0.5.6: Fewer false duplicate wordings
|
|
952
1350
|
|
|
953
1351
|
Released 2026-09-17.
|
|
954
1352
|
|
|
@@ -959,7 +1357,7 @@ Runner
|
|
|
959
1357
|
duplicate wordings; steps are compared by their quote-generalised
|
|
960
1358
|
pattern.
|
|
961
1359
|
|
|
962
|
-
## 0.5.5
|
|
1360
|
+
## 0.5.5: Duplicate wordings proven by behaviour
|
|
963
1361
|
|
|
964
1362
|
Released 2026-09-17.
|
|
965
1363
|
|
|
@@ -971,7 +1369,7 @@ Runner
|
|
|
971
1369
|
"enter X in the username field" against "enter Y in the password
|
|
972
1370
|
field", which are different steps.
|
|
973
1371
|
|
|
974
|
-
## 0.5.4
|
|
1372
|
+
## 0.5.4: saffron status for the IDE panels
|
|
975
1373
|
|
|
976
1374
|
Released 2026-09-17.
|
|
977
1375
|
|
|
@@ -991,7 +1389,7 @@ Editors
|
|
|
991
1389
|
Health sections and an in-editor dashboard. All read
|
|
992
1390
|
`saffron status --json`.
|
|
993
1391
|
|
|
994
|
-
## 0.5.3
|
|
1392
|
+
## 0.5.3: Cost by how you pay, and running from JetBrains
|
|
995
1393
|
|
|
996
1394
|
Released 2026-09-17.
|
|
997
1395
|
|
|
@@ -1032,7 +1430,7 @@ Editors
|
|
|
1032
1430
|
extension, the last run's totals with the report one click away, and
|
|
1033
1431
|
the proposals pending review with Accept all.
|
|
1034
1432
|
|
|
1035
|
-
## 0.5.2
|
|
1433
|
+
## 0.5.2: Scenario Outlines record correctly
|
|
1036
1434
|
|
|
1037
1435
|
Released 2026-09-15.
|
|
1038
1436
|
|
|
@@ -1061,7 +1459,7 @@ Editors
|
|
|
1061
1459
|
hover says where it is defined and never shows the value. In the VS
|
|
1062
1460
|
Code extension and in `saffron lsp` for JetBrains and other editors.
|
|
1063
1461
|
|
|
1064
|
-
## 0.5.1
|
|
1462
|
+
## 0.5.1: The example suite in .saffron files
|
|
1065
1463
|
|
|
1066
1464
|
Released 2026-09-14.
|
|
1067
1465
|
|
|
@@ -1078,7 +1476,7 @@ Editors
|
|
|
1078
1476
|
parameters instead of the whole step line. Ships in the VS Code
|
|
1079
1477
|
extension, the JetBrains plugin and the package's TextMate bundle.
|
|
1080
1478
|
|
|
1081
|
-
## 0.5.0
|
|
1479
|
+
## 0.5.0: Guided first run
|
|
1082
1480
|
|
|
1083
1481
|
Released 2026-09-14.
|
|
1084
1482
|
|
|
@@ -1091,7 +1489,7 @@ Runner
|
|
|
1091
1489
|
matching `saffron.config.json` with `retries: 1`. Only installs into an
|
|
1092
1490
|
empty `features/` directory. Documented as "Your first run".
|
|
1093
1491
|
|
|
1094
|
-
## 0.4.2
|
|
1492
|
+
## 0.4.2: Recover from a wrong recording
|
|
1095
1493
|
|
|
1096
1494
|
Released 2026-09-13.
|
|
1097
1495
|
|
|
@@ -1123,7 +1521,7 @@ Docs and site
|
|
|
1123
1521
|
- Assertion policy documented on the Features page; cache lifecycle gained
|
|
1124
1522
|
the staleness and wrong-recording stages.
|
|
1125
1523
|
|
|
1126
|
-
## 0.4.1
|
|
1524
|
+
## 0.4.1: JetBrains plugin
|
|
1127
1525
|
|
|
1128
1526
|
Released 2026-09-13.
|
|
1129
1527
|
|
|
@@ -1145,7 +1543,7 @@ Editors
|
|
|
1145
1543
|
- VS Code extension 0.1.3: plain punctuation in README, hover and
|
|
1146
1544
|
diagnostic messages.
|
|
1147
1545
|
|
|
1148
|
-
## 0.4.0
|
|
1546
|
+
## 0.4.0: Agent skill for AI assistants
|
|
1149
1547
|
|
|
1150
1548
|
Released 2026-09-13.
|
|
1151
1549
|
|
|
@@ -1171,13 +1569,13 @@ Editors
|
|
|
1171
1569
|
- VS Code extension 0.1.2: warning and quick fix when a set name is written
|
|
1172
1570
|
without the `StepSet` keyword.
|
|
1173
1571
|
|
|
1174
|
-
## 0.3.1
|
|
1572
|
+
## 0.3.1: Version fix
|
|
1175
1573
|
|
|
1176
1574
|
Released 2026-08-15.
|
|
1177
1575
|
|
|
1178
1576
|
- `saffron --version` no longer reports a hardcoded value.
|
|
1179
1577
|
|
|
1180
|
-
## 0.3.0
|
|
1578
|
+
## 0.3.0: Network-aware steps
|
|
1181
1579
|
|
|
1182
1580
|
Released 2026-08-12.
|
|
1183
1581
|
|
|
@@ -1199,7 +1597,7 @@ Editors
|
|
|
1199
1597
|
|
|
1200
1598
|
- VS Code extension 0.1.0 published to the Marketplace.
|
|
1201
1599
|
|
|
1202
|
-
## 0.2.0
|
|
1600
|
+
## 0.2.0: Secrets, cross-browser and parallel replay
|
|
1203
1601
|
|
|
1204
1602
|
Released 2026-07-19.
|
|
1205
1603
|
|
|
@@ -1215,7 +1613,7 @@ Runner
|
|
|
1215
1613
|
workers (`--workers N`), and a cheaper model for heal sessions
|
|
1216
1614
|
(`--heal-model`).
|
|
1217
1615
|
|
|
1218
|
-
## 0.1.0
|
|
1616
|
+
## 0.1.0: First public release
|
|
1219
1617
|
|
|
1220
1618
|
Released 2026-07-10.
|
|
1221
1619
|
|