saffron-ai 1.3.2 → 1.3.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +382 -0
- package/dist-pkg/cli.js +428 -409
- package/package.json +1 -1
- package/skills/saffron/SKILL.md +2 -2
- package/skills/saffron/references/api.md +37 -1
- package/templates/playwright/saffron-helpers.ts +133 -10
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,388 @@ the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
|
|
|
5
5
|
JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
|
|
6
6
|
release heading names its main feature: `## <version>: <title>`.
|
|
7
7
|
|
|
8
|
+
## 1.3.4: Checks that wait for a settled page, and credentials kept masked
|
|
9
|
+
|
|
10
|
+
Released 2026-11-07.
|
|
11
|
+
|
|
12
|
+
Runner
|
|
13
|
+
|
|
14
|
+
- An Outline row no longer passes on another row's data in three more
|
|
15
|
+
ways: a check naming a column that no step of the recording writes ("the
|
|
16
|
+
`<plan>` plan shows ...") binds the recording to its row, a row's number
|
|
17
|
+
is found where the page shows it grouped or with a decimal comma ("Up
|
|
18
|
+
to $1,000" for 1000), and a request body's number is taken for a
|
|
19
|
+
constant only where the body itself writes the column. A style check's
|
|
20
|
+
property is a value a row can hold. A check run in the session a column
|
|
21
|
+
names (`As "<who>"`) follows that column; one run in another session
|
|
22
|
+
(`As "admin"`) does not, and a step writing the column counts only when
|
|
23
|
+
it comes before the check in the same session. A row's number is also
|
|
24
|
+
found negative ("−$1,000") and grouped by an apostrophe or a no-break
|
|
25
|
+
space ("CHF 1'000").
|
|
26
|
+
- An amount check fails when the target shows another currency than the
|
|
27
|
+
captured amounts, and captures in two currencies are refused. A symbol
|
|
28
|
+
is compared with the codes it can stand for: "$5" may be USD 5 or CAD 5,
|
|
29
|
+
never NOK 5 or EUR 5, "€" is EUR, and a dollar its letters name is that
|
|
30
|
+
dollar ("US$", "CA$", "A$", "R$" and others); every capture is compared
|
|
31
|
+
with every other. A lowercase word that is also a code ("Top 10",
|
|
32
|
+
"150 eur") is compared with codes only. A URL check reads a
|
|
33
|
+
path whole past `-`, `.`, `_`, `~` and `+`, and past `:` before a letter
|
|
34
|
+
("/checkout" is not "/checkout-failed" or "/checkout_failed"; a host
|
|
35
|
+
before its port, "localhost:3000", still matches), and a word starts as
|
|
36
|
+
it ends: after a letter and a separator it is part of another ("paid"
|
|
37
|
+
is not in "/orders/not-paid"; in a query "+" and a space still separate
|
|
38
|
+
words, so "shoes" is in "?q=red+shoes"), a number also after a
|
|
39
|
+
digit ("42" is not in "/1-42"), while an id-prefixed slug keeps its words
|
|
40
|
+
("blue-shirt" is in "/products/42-blue-shirt") and "example.com" is in
|
|
41
|
+
"shop.example.com". A doubled separator ("/checkout--failed") is one, in
|
|
42
|
+
replay and in specs `saffron eject` writes. A committed check of a path
|
|
43
|
+
the app shows with a slug after it ("/products/42" on
|
|
44
|
+
"/products/42-blue-shirt"), or of one word inside a slug ("shirt" or
|
|
45
|
+
"US" in "/en-US/cart"), fails after upgrading: record it again with the
|
|
46
|
+
whole slug, or check the id another way. A currency code is read beside the
|
|
47
|
+
number, capitals first, so "Top up NOK 150" is in NOK, and "NET 150
|
|
48
|
+
EUR", "150 eur" and "EUR (150.00)" are in euros. Text inside `<noscript>`, or inside an SVG
|
|
49
|
+
definition never drawn where it stands, is not shown text, whether the
|
|
50
|
+
check's target is the container or that element itself.
|
|
51
|
+
- API tests: a group's `same` check compares values exactly (two int64
|
|
52
|
+
payment references a digit apart differ, as do 0.1 and
|
|
53
|
+
0.10000000000000001, and a number and a string of its digits), a number
|
|
54
|
+
in a body that JSON cannot hold as written is refused, in any spelling
|
|
55
|
+
(`9007199254740993`, `9007199254740993.0`, `9.007199254740993e15`,
|
|
56
|
+
`0.10000000000000001`; it would be sent rounded to another value: send
|
|
57
|
+
it as a string). A row or capture that held such a number and passed
|
|
58
|
+
before is refused after upgrading, a `differs` check whose row value is
|
|
59
|
+
empty is refused, an expected decimal with more digits than a number
|
|
60
|
+
holds is refused, and a group's contract check ties the recording to
|
|
61
|
+
the contract, so `api pin` re-records it.
|
|
62
|
+
- A provider's daily quota, "exhausted your capacity" and "hit your
|
|
63
|
+
limit" stop the run's AI sessions as other usage limits do. A key a
|
|
64
|
+
vendor quotes with its middle starred or elided (`sk-...XwVu`) is
|
|
65
|
+
masked, every Cursor error masks CURSOR_API_KEY, including one the SDK
|
|
66
|
+
throws, and an Antigravity error that ends in JSON names its
|
|
67
|
+
message. A `verify` scenario path that leaves the project through a
|
|
68
|
+
link is refused.
|
|
69
|
+
- Specs `saffron eject` writes open web pages only (never `data:`,
|
|
70
|
+
`file:` or `javascript:`) and upload only the project's own files (no
|
|
71
|
+
dotfiles, nothing outside the project, no link to a dotfile or out of
|
|
72
|
+
the project, no hard link, no folders), as replay does; they refuse a
|
|
73
|
+
`differs` whose value a blank row or capture empties, and a body number
|
|
74
|
+
they cannot send exactly. An Examples cell holding such a
|
|
75
|
+
number marks its test fixme instead of stopping the eject.
|
|
76
|
+
- A `"""` or a triple backtick at the start of a description line no
|
|
77
|
+
longer opens a doc string, which left every `StepSet` line after it
|
|
78
|
+
unread: as in Gherkin, a doc string opens only after a step in a
|
|
79
|
+
scenario, Background, StepSet or Fixture, never in a Feature, Rule or
|
|
80
|
+
Examples description, in the runner and the language server alike.
|
|
81
|
+
- An API that answers with a refused connection in its body or a field
|
|
82
|
+
(its own database's) is a failure, never "not run, environment
|
|
83
|
+
missing": only a request that got no response, or the browser's own
|
|
84
|
+
navigation error, says nothing answered.
|
|
85
|
+
The dashboard's recent failures pass over reds the app did not cause
|
|
86
|
+
when telling whether a scenario still fails and for how long.
|
|
87
|
+
- Recording: a credential an API handed out during the session is masked
|
|
88
|
+
in everything the browser tools show the agent (a token in a page URL
|
|
89
|
+
was shown whole, and an earlier value of a credential that changed); a
|
|
90
|
+
page the agent opens with an `{env:...}` value in its URL must be on the
|
|
91
|
+
app's origin or one the scenario writes, a URL with no scheme read as
|
|
92
|
+
the browser opens it; code a check runs through `browser_evaluate` that
|
|
93
|
+
clicks, submits, sets a value, navigates or sends a writing request is
|
|
94
|
+
refused, as a click in a check is;
|
|
95
|
+
with a browser already running (`transport.browser` or a
|
|
96
|
+
`transport.module`), recordings run one at a time, as two sessions on
|
|
97
|
+
one browser closed each other's pages.
|
|
98
|
+
- A password or user name in a base URL is masked wherever a URL quotes
|
|
99
|
+
it (`https://qa:***@staging.test`): the report's errors, the run
|
|
100
|
+
history, proposals, `status`, `diff`, bundles, traces and the API
|
|
101
|
+
transcript, and the agent is never shown it. Only there: the same word
|
|
102
|
+
elsewhere in a message is left as it is.
|
|
103
|
+
- A local notation beside the number (`100 kr`, `Rs. 500`, `5,00 zł`) is a
|
|
104
|
+
currency of its own in an amount check, so captures in "kr" fail a total
|
|
105
|
+
shown in euros. A URL check reads a percent-encoded letter beyond a
|
|
106
|
+
separator as a letter ("/checkout" is not in "/checkout-%C3%A9chou%C3%A9").
|
|
107
|
+
- A proposal or recording the run cannot read (no permission, a folder in
|
|
108
|
+
its place) is listed as unreadable instead of stopping `status`, `diff`,
|
|
109
|
+
`accept --all` and the run's closing summary. The editor flags a
|
|
110
|
+
`{data:...}` reference only where the runner reads it (a step, its table
|
|
111
|
+
or doc string), never in a title, a description or a comment.
|
|
112
|
+
- A Then recorded again under the same words that now expects another
|
|
113
|
+
value ("Pending" where it checked "Paid", a status 500 where it expected
|
|
114
|
+
201, another JSON value or URL), or reads another element named the same
|
|
115
|
+
way (test id "status" become "badge"), is held for review as a weakened
|
|
116
|
+
check (a row or data token holding the same value is the same): a stale re-record no longer replaces a check that just
|
|
117
|
+
failed. A heal that changed nothing is red, never a proposal, also with
|
|
118
|
+
`--no-verify`. The usage an agent session spent stays counted when its
|
|
119
|
+
proof replay cannot start.
|
|
120
|
+
- An Outline row's short value (a word under 5 characters with no digit,
|
|
121
|
+
such as "dev") is written as its `<column>` only where it stands alone:
|
|
122
|
+
inside "devices panel" it stays as recorded, so another row no longer
|
|
123
|
+
checks a text never on the page. A secret Playwright cut short with "…"
|
|
124
|
+
in a trace line, or a captured credential cut short in a message, is
|
|
125
|
+
masked.
|
|
126
|
+
- Replay: a "not shown" check looks inside the app's own iframes too (its
|
|
127
|
+
origin, or one it writes, shown on the page), as a "shown" check does, so
|
|
128
|
+
an error drawn in one no longer passes both; another origin's frame (a
|
|
129
|
+
chat widget's "Close") and a hidden one (a silent sign-in renewal) are
|
|
130
|
+
not searched. A "not shown", count, "differs" or "URL does
|
|
131
|
+
not contain" check, and a response check, passes only once the requests
|
|
132
|
+
the last action started have answered and the passing state has held for
|
|
133
|
+
half a second: a list still growing, a placeholder drawn during a
|
|
134
|
+
refresh or a row gone for a moment no longer passes. A long-poll the page
|
|
135
|
+
opened earlier is not waited for. A response check fails when the very
|
|
136
|
+
request it matched (its URL and method, and the body or JSON it checks)
|
|
137
|
+
was answered again with another status (a POST answered 200 and then
|
|
138
|
+
500); a refetch, a child path or a GraphQL query on the same endpoint is
|
|
139
|
+
another request.
|
|
140
|
+
- `accept --step-set` refuses a StepSet that ran in a named browser session
|
|
141
|
+
(`As "seller"`) and a proposal that fails an Examples row: filed for
|
|
142
|
+
reuse, its steps would seed other sessions' or rows' steps. A Nordic
|
|
143
|
+
price ("150,- kr", "150:- kr") keeps its currency. A base URL's Basic
|
|
144
|
+
header an API echoes is masked; a URL in a query beside an email is not
|
|
145
|
+
taken for a password; step text accept writes into the feature file
|
|
146
|
+
keeps its URLs whole.
|
|
147
|
+
- Recording: a capture's target is read as replay reads it before it is
|
|
148
|
+
recorded, and a value it does not show is refused, so a later check never
|
|
149
|
+
compares with another element; a status the step names exactly ("answer
|
|
150
|
+
404") is expected exactly, never as its class, and a status check names a
|
|
151
|
+
status the step states; a step performed again drops its requests'
|
|
152
|
+
responses too, so a later check never reads a dropped one; a response
|
|
153
|
+
check is verified with the step's table values, as replay reads it.
|
|
154
|
+
- Config: a key that names a known one misspelt (`maxCostUSD`, `stict`) is
|
|
155
|
+
refused, naming the key meant, and any other unknown key is said; a
|
|
156
|
+
`capabilities` value in quotes, an unknown `snapshotMode` and a
|
|
157
|
+
`hookTimeoutMs` of 0 are refused; an empty `SAFFRON_ENV` keeps the
|
|
158
|
+
config's `env`. `status --json` and the MCP `project_status` mask a
|
|
159
|
+
password in any URL and a header value written whole. `run --plan`
|
|
160
|
+
judges a pending proposal as the run does: a stale one is a recording,
|
|
161
|
+
one filed on another machine is not replayed, a row-bound one is
|
|
162
|
+
refused. The MCP `verify` tool says when the run did not finish cleanly
|
|
163
|
+
(interrupted, scenarios not run, a failed hook).
|
|
164
|
+
- A heal whose agent session could not start keeps the replay's failure,
|
|
165
|
+
never "not run"; a fixture that failed in the app fails the scenarios
|
|
166
|
+
behind it, never "environment missing"; the usage earlier agent
|
|
167
|
+
sessions of a recording spent stays counted when a later one fails.
|
|
168
|
+
- A password in the base URL, or a user name alone (`https://TOKEN@host`),
|
|
169
|
+
is masked in the report, the run history and bundles. A run with `--filter` or of part of the features folder is
|
|
170
|
+
partial in the history, so trends compare suite runs only. A red the
|
|
171
|
+
app did not cause never makes a scenario flaky on the dashboard.
|
|
172
|
+
- A CSV data file with a quote never closed is refused, naming the row,
|
|
173
|
+
instead of reading every line after it as one field. The language
|
|
174
|
+
server no longer stalls on a line holding a long run of spaces.
|
|
175
|
+
- Replay lets the first candidate that finds the recorded element decide a
|
|
176
|
+
check, shown or not: a hidden recorded element no longer passes a text,
|
|
177
|
+
visible, value or enabled check on a fallback elsewhere, a list check
|
|
178
|
+
(`every {data:...}`) no longer passes on a fallback or another frame
|
|
179
|
+
listing more, and a watched message counts only on the element watched,
|
|
180
|
+
never on a log line elsewhere. A currency-first value (`$5.00`) in a
|
|
181
|
+
pattern no longer matches its negative (`-$5.00`).
|
|
182
|
+
- A refused API request's reply to the agent masks a credential an API
|
|
183
|
+
handed out in the URL, as the reply to a request that answered does.
|
|
184
|
+
- Restoring a column over a secret no longer splices a short env value
|
|
185
|
+
into a word (`dev` in `/devices`); a row value an API request's path
|
|
186
|
+
holds URL-encoded is written back as its column, as in a browser URL.
|
|
187
|
+
- A red the agent cannot fix (an unreadable recording, a row refused, the
|
|
188
|
+
committed recording failing while another row's proposal waits) counts
|
|
189
|
+
as a failure in the report and the run summary, never as "needs
|
|
190
|
+
recording" with a command that would change nothing.
|
|
191
|
+
- The HTML report sets the reds the app did not cause apart from
|
|
192
|
+
failures: a stale recording in a run with `--no-agent`, a missing
|
|
193
|
+
environment, an agent that could not start or a run over its budget.
|
|
194
|
+
They get their own *not run* group under the failures (*needs
|
|
195
|
+
recording* when that is the only reason), a ○ on the run map, and a
|
|
196
|
+
*failed* count that says how many more did not run and why. Every
|
|
197
|
+
scenario that needs recording names the others, with one `saffron run`
|
|
198
|
+
command for their feature files. The run map counts them per feature
|
|
199
|
+
file, and the recurring failure themes leave them out (run history now
|
|
200
|
+
keeps the mark). They are still red, and the run still exits 1. Found in
|
|
201
|
+
real-world use, where one edit to a shared StepSet made 45 of 48 reds
|
|
202
|
+
stale recordings.
|
|
203
|
+
- A request group's step seeds another scenario that sends the same
|
|
204
|
+
requests with its own values: the same table header, request names and
|
|
205
|
+
rows, each cell read through `<row:column>`, and so does a group check
|
|
206
|
+
that reads each response's own row. Before, a group step was never
|
|
207
|
+
seeded, so every scenario recorded it with the agent.
|
|
208
|
+
- A StepSet accepted on its own (`accept --step-set`) after a scenario's
|
|
209
|
+
recording was made seeds the steps that come from that StepSet over the
|
|
210
|
+
scenario's older recording, with no `seeded from another recording:`
|
|
211
|
+
note; the same words written straight in a scenario keep its own.
|
|
212
|
+
- `saffron status --json` lists the StepSets each proposal recorded
|
|
213
|
+
(`stepSets`), for an editor to offer `accept --step-set`.
|
|
214
|
+
- `saffron eject` exports a count check as Playwright's count of shown
|
|
215
|
+
elements, `expect(locator.filter({ visible: true })).toHaveCount(n)`,
|
|
216
|
+
the number as replay reads it (`010` is 10).
|
|
217
|
+
- A pattern check that fails where the pattern writes an optional part
|
|
218
|
+
right after a value says the value is matched whole and how to write the
|
|
219
|
+
part instead.
|
|
220
|
+
- A currency code after a number counts as a price in a locator's name
|
|
221
|
+
only after two digits or more: "Section 4 SEK" is no price.
|
|
222
|
+
|
|
223
|
+
Editors (VS Code extension 0.8.1, JetBrains plugin 0.8.1)
|
|
224
|
+
|
|
225
|
+
- Both editors offer `saffron export` (the review bundle), a draft pull
|
|
226
|
+
request with the pending proposals (`export --branch`, after asking for
|
|
227
|
+
a branch and confirming), `saffron eject` (after asking, for the open
|
|
228
|
+
file or every one) and `accept --step-set` for one ticked proposal's
|
|
229
|
+
StepSet, from the Proposals view's menu (VS Code) or toolbar
|
|
230
|
+
(JetBrains).
|
|
231
|
+
|
|
232
|
+
## 1.3.3: Concurrent API requests, and recordings that follow their data
|
|
233
|
+
|
|
234
|
+
Released 2026-11-05.
|
|
235
|
+
|
|
236
|
+
Runner
|
|
237
|
+
|
|
238
|
+
Upgrading from 1.3.2:
|
|
239
|
+
|
|
240
|
+
- Recordings with a request group (several API requests sent at once) are
|
|
241
|
+
schema version 19, and recordings with a count check use an action
|
|
242
|
+
1.3.2 does not know: 1.3.2 refuses both rather than replaying them
|
|
243
|
+
wrong. Upgrade every machine that replays them, CI included.
|
|
244
|
+
- Two new notes hold a proposal from `accept --all`: `check weakened:`
|
|
245
|
+
(a `Then` recorded again with a looser check than the committed one)
|
|
246
|
+
and `data not followed:` (a step that names a `{data:...}` value its
|
|
247
|
+
recording keeps as text). Accept such a proposal by its file after
|
|
248
|
+
reading why.
|
|
249
|
+
- While recording, each browser check the agent writes is run on its page
|
|
250
|
+
by replay's own engine, and refused when it does not hold there. A
|
|
251
|
+
recording that used to fail its proof on such a check now fixes it
|
|
252
|
+
before the proof.
|
|
253
|
+
- A scenario recorded again because its recording is stale seeds the
|
|
254
|
+
steps it did not change from its own recording, never from another
|
|
255
|
+
scenario's newer one.
|
|
256
|
+
|
|
257
|
+
- A downloaded file's check resolves a `{captured:...}` value, while
|
|
258
|
+
recording and at replay, so "the calendar file contains the booking
|
|
259
|
+
number `{captured:bookingNumber}`" checks this run's booking; one that
|
|
260
|
+
nothing captured fails saying so. Before, replay checked for the token
|
|
261
|
+
as text and the proof failed every file.
|
|
262
|
+
- A date the scenario writes as a token stays that token when the agent
|
|
263
|
+
types it glued to a time (`20261111T140000Z` for `{date+7:YYYYMMDD}`),
|
|
264
|
+
and a pattern that writes the time after it (`{date+7:YYYYMMDD}T[0-9]{6}`)
|
|
265
|
+
matches it. A time written as optional (`(?:T[0-9]{6})?`) still ends the
|
|
266
|
+
date, so the step writes it as required. A date left as text
|
|
267
|
+
in the format the scenario's own token writes gets `date not following
|
|
268
|
+
the clock:` and is held from `accept --all`.
|
|
269
|
+
- A short data value the step names (`{data:party.size}`, 2) is written
|
|
270
|
+
back as its reference where it stands alone once in a check's or fill's
|
|
271
|
+
value, a text pattern or an amount ("Table for 2 people", `"price" *
|
|
272
|
+
2`), never in a selector, URL or path, nor a digit of regex syntax, a
|
|
273
|
+
time or a date. A step whose recording still holds a value it
|
|
274
|
+
names as text, or a list's items one by one, gets a new `data not
|
|
275
|
+
followed:` note and is held from `accept --all`. A price in a locator's
|
|
276
|
+
name (`Info Lunch SEK 150`) is a `volatile locator:`.
|
|
277
|
+
- The language server sees a StepSet's steps filed with `accept
|
|
278
|
+
--step-set` (`.saffron/step-sets`) as recorded when they change.
|
|
279
|
+
- The plugins' keyword tables carry each language's Feature, Background
|
|
280
|
+
and Rule, so a file in another Gherkin language ends a scenario at them.
|
|
281
|
+
- A `StepSet:` or `Fixture:` in a file written in another Gherkin
|
|
282
|
+
language (`# language: de`) defines its set or fixture: it was read as
|
|
283
|
+
the English `Scenario:`, which that language does not know, so no set
|
|
284
|
+
was found. A StepSet is called in a language with no `*` step too
|
|
285
|
+
(Slovenian), and the `# language:` line is read as Gherkin reads it:
|
|
286
|
+
`# language: de # Deutsch` is English to both.
|
|
287
|
+
- The language server reads an `As "..."` session through Background and
|
|
288
|
+
Rule headers in the file's own language, and ends one at a `Fixture:`,
|
|
289
|
+
as the VS Code extension does.
|
|
290
|
+
- A file the agent uploads while recording is found in the project when
|
|
291
|
+
its path is relative, wherever the run was started: `saffron -p app
|
|
292
|
+
run` from a monorepo's root refused `demo-app/sample-upload.txt` as
|
|
293
|
+
outside the project.
|
|
294
|
+
- The benchmark's recording measure counts every note held from `accept
|
|
295
|
+
--all`, and names the scenarios not verified on the first attempt or
|
|
296
|
+
held, with why.
|
|
297
|
+
- A browser check the agent writes is run on its page by replay's own
|
|
298
|
+
engine before it is recorded, with replay's values, captures and clock,
|
|
299
|
+
and refused with replay's reason when it does not hold: a value not
|
|
300
|
+
whole in its text or an element not shown fails while recording, not
|
|
301
|
+
at the proof.
|
|
302
|
+
- A `Then` recorded again under the same words with a looser check than
|
|
303
|
+
the committed one gets a `check weakened:` note and is held from
|
|
304
|
+
`accept --all`: a value it no longer reads (the same value written
|
|
305
|
+
another way still counts), a pattern's or name pattern's lost `^` or
|
|
306
|
+
`$`, a text it checks less of, a kind of check it no longer makes or
|
|
307
|
+
covers, a JSON field it no longer compares, an element it no longer
|
|
308
|
+
names. `--rerecord` keeps the recording it removes for this comparison.
|
|
309
|
+
- A scenario recorded again because its recording is stale seeds each
|
|
310
|
+
step it did not change from its own committed recording where that fits
|
|
311
|
+
and the page shows what it acts on, never from another scenario's newer
|
|
312
|
+
one (a shared StepSet's update brought those in). A step that replayed
|
|
313
|
+
another scenario's recording is named in a `seeded from another
|
|
314
|
+
recording:` note.
|
|
315
|
+
- A check of an element that runs out of time (shown, text, pattern,
|
|
316
|
+
value, style, attribute) says what replay saw last: nothing matching
|
|
317
|
+
the target, an element there but not shown, or the text or value it
|
|
318
|
+
showed, with secrets masked before it is cut.
|
|
319
|
+
- A count check: "3 rooms should be listed" is one check of how many
|
|
320
|
+
shown elements the target finds in the page (or the frame it names),
|
|
321
|
+
from a number, a `<column>` or a `{data:...}` value; a hidden element
|
|
322
|
+
never counts, and none is written as "not shown". `saffron eject` does
|
|
323
|
+
not export it yet, and saffron-ai 1.3.2 refuses a recording with one as
|
|
324
|
+
an action it does not know, never replaying it wrong.
|
|
325
|
+
- The report page's headline counts and cost are this run's own after a
|
|
326
|
+
run of named scenarios, with the number of rows kept from earlier runs
|
|
327
|
+
beside them; before, it added up the kept rows' results and AI calls.
|
|
328
|
+
|
|
329
|
+
- Several API requests at once, each with its own values, and checks over
|
|
330
|
+
all their responses: an idempotency test that registers one charge from
|
|
331
|
+
several requests at the same moment. A step's table names the requests,
|
|
332
|
+
a header row and a row per request (`When I send these registrations
|
|
333
|
+
concurrently as "registrations":`); the request is recorded once and
|
|
334
|
+
sent once per row, every row before any answers, each row's cell
|
|
335
|
+
written as `<row:column>`, so editing a cell needs no recording again;
|
|
336
|
+
a `<row:column>` naming no column of the table is refused rather than
|
|
337
|
+
sent as text, and the group's name may come from an Examples column.
|
|
338
|
+
Every response stays with its row whatever order it arrives in, a
|
|
339
|
+
request that gets none is kept and fails the step once all have
|
|
340
|
+
settled, and a group is never sent again. `Then` steps check every
|
|
341
|
+
response, those of one status, a named one, at least some, or all of
|
|
342
|
+
them sharing a field's value (`all 200 responses should have the same
|
|
343
|
+
"/paymentReference"`); a response's own cell in a check is its row's.
|
|
344
|
+
The common checks are exact phrases recorded with no agent. A failure
|
|
345
|
+
lists each request with its status and why it failed. "The response"
|
|
346
|
+
after a group fails instead of reading one of them. Recordings with a
|
|
347
|
+
group are schema version 19: saffron-ai 1.3.2 and earlier refuse them.
|
|
348
|
+
`saffron eject` does not export a group yet.
|
|
349
|
+
|
|
350
|
+
- `saffron accept --propagate` fixes a healed API request everywhere it
|
|
351
|
+
is sent, as it already did for a locator. Scenarios that share a step's
|
|
352
|
+
wording hold copies of its request; when a heal rewrites that request
|
|
353
|
+
(a field the API now requires, a renamed path), accepting it with
|
|
354
|
+
`--propagate` rewrites every other cache that sends the same request
|
|
355
|
+
under the same step text, so one heal fixes them all instead of each
|
|
356
|
+
scenario healing on its own run. `--propagate` goes on the accept itself:
|
|
357
|
+
without it, accept names the caches it left as they are, which heal on
|
|
358
|
+
their own run. Either way it prints what the request was and is now. A
|
|
359
|
+
fix is not copied when it writes a value of the healed scenario's own (a
|
|
360
|
+
table cell, a doc string, its Outline row), adds a `<column>` or
|
|
361
|
+
`{captured:...}` value, drops a value the request extracted, sends the
|
|
362
|
+
request to another API, or when the heal kept the request as it was in
|
|
363
|
+
one place and rewrote it in another; accept says which. A step whose
|
|
364
|
+
heal added or removed an action is not copied either.
|
|
365
|
+
- `saffron accept` without `--propagate` no longer tells you to re-run it
|
|
366
|
+
with `--propagate`: the proposal is accepted by then, and a re-run had
|
|
367
|
+
nothing left to fix. It says the other caches were left to heal on
|
|
368
|
+
their own run.
|
|
369
|
+
|
|
370
|
+
Editors (VS Code extension 0.8.0, JetBrains plugin 0.8.0)
|
|
371
|
+
|
|
372
|
+
- Both editors mark a proposal with a `check weakened:` or `data not
|
|
373
|
+
followed:` note, or one that fails an Examples row, as held from Accept
|
|
374
|
+
All, as `saffron accept --all` skips it. JetBrains asks before Accept
|
|
375
|
+
All runs, as VS Code does.
|
|
376
|
+
- In a file written in another Gherkin language (`# language: de`),
|
|
377
|
+
Right-click Run reads that language's Feature, Background and Rule
|
|
378
|
+
headers (JetBrains), and VS Code's completion, recording badges, hovers
|
|
379
|
+
and wording hints read its step keywords and headers, a keyword with no
|
|
380
|
+
space of its own ("假如") included.
|
|
381
|
+
- A `Fixture:` block is a block of its own: it ends the StepSet above it
|
|
382
|
+
(VS Code), and Right-click Run inside it runs no scenario (JetBrains).
|
|
383
|
+
- VS Code shows the steps a StepSet's proposal filed for reuse
|
|
384
|
+
(`.saffron/step-sets`) as recorded, and no longer marks `Examples:
|
|
385
|
+
{data: roles}` as an error when a description line and then a table
|
|
386
|
+
follow it.
|
|
387
|
+
- JetBrains passes a run's files after `--`, so a file whose name starts
|
|
388
|
+
with a dash is a file, never an option.
|
|
389
|
+
|
|
8
390
|
## 1.3.2: Exact checks for amounts, URLs and numbers, and secrets kept masked
|
|
9
391
|
|
|
10
392
|
Released 2026-11-04.
|