saffron-ai 0.9.9 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +969 -0
- package/README.md +5 -6
- package/dist-pkg/cli.js +384 -249
- package/package.json +4 -3
- package/skills/saffron/SKILL.md +208 -95
- package/skills/saffron/references/config.md +6 -3
- package/skills/saffron/references/syntax.md +128 -5
- package/textmate/saffron/syntaxes/saffron.tmLanguage.json +12 -3
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,975 @@ the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
|
|
|
5
5
|
JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
|
|
6
6
|
release heading names its main feature: `## <version>: <title>`.
|
|
7
7
|
|
|
8
|
+
## 1.0.1: Amounts, downloads and dates as the app shows them
|
|
9
|
+
|
|
10
|
+
Released 2026-10-21.
|
|
11
|
+
|
|
12
|
+
**Upgrading from 1.0.0: some recordings can turn red.** Checks are
|
|
13
|
+
stricter, so a recording 1.0.0 passed can fail after `npm update`, with
|
|
14
|
+
nothing changed in the application. It never turns a failing check green.
|
|
15
|
+
What changed:
|
|
16
|
+
|
|
17
|
+
- A role's name matches exactly (a partial name needs an explicit name
|
|
18
|
+
regex), attribute patterns are case-sensitive, and an input-value check
|
|
19
|
+
needs a visible input.
|
|
20
|
+
- A text or input-value check records an explicit expected value. Expected
|
|
21
|
+
text, a text pattern, a download check, an expected URL, or a target's
|
|
22
|
+
text, label or placeholder that a blank Examples or data value leaves
|
|
23
|
+
partly empty fails instead of matching more than it means.
|
|
24
|
+
- A response check counts only requests started since the latest browser
|
|
25
|
+
action, so an earlier successful save no longer satisfies it.
|
|
26
|
+
- An Outline recording that cannot follow another row (short values,
|
|
27
|
+
dates without a year, a role taken from a row, an unscoped action,
|
|
28
|
+
check or capture in a section a column names, a value typed in a
|
|
29
|
+
column's browser session, a step naming a column that recorded no action
|
|
30
|
+
writing it, a state check naming a column whose value it does not show,
|
|
31
|
+
as in `Then <user> sees the dashboard`) refuses that row with guidance
|
|
32
|
+
instead of passing on row 1's data. Login Outlines of that shape now
|
|
33
|
+
pass only on the recorded row: write the user in the text the check
|
|
34
|
+
expects (`Then the dashboard greets "<user>"`) and record again.
|
|
35
|
+
|
|
36
|
+
Re-record what fails (`saffron run --rerecord <file>:<line>`) or correct
|
|
37
|
+
the step the message names. Recordings 1.0.1 writes with its new features
|
|
38
|
+
(localized dates, several downloads per click, the new row bindings) are
|
|
39
|
+
refused by 1.0.0 by name, so a team updates together.
|
|
40
|
+
|
|
41
|
+
Agent plugins
|
|
42
|
+
|
|
43
|
+
- Declare documentation, privacy, license terms and support links for the
|
|
44
|
+
Claude directory listing, with explicit README links as a fallback.
|
|
45
|
+
|
|
46
|
+
- Prepare the agent plugin listings with the Saffron icon and documented
|
|
47
|
+
data handling. The Codex listing adds a shorter subtitle, local
|
|
48
|
+
execution requirements, prerequisites and links to support, data handling
|
|
49
|
+
and the software license.
|
|
50
|
+
|
|
51
|
+
Runner
|
|
52
|
+
|
|
53
|
+
- A row refused because its check only shows that something is visible or
|
|
54
|
+
in a state (`Then <user> sees the dashboard`) now says so, and suggests
|
|
55
|
+
writing the value in the text the check expects
|
|
56
|
+
(`Then the dashboard greets "<user>"`).
|
|
57
|
+
- A fallback selector's text (`:has-text("Order <id>")`, `text=Order
|
|
58
|
+
<id>`) finds row 1 only as a whole number or word, not inside "Order
|
|
59
|
+
12", and a blank row value at its edge is refused, as in other checks.
|
|
60
|
+
- A pattern that joins letters or digits to a row value itself
|
|
61
|
+
(`^<prefix>\d+$`) keeps them joined at replay.
|
|
62
|
+
- A secret shown lower-cased and percent-encoded in a URL is masked, and
|
|
63
|
+
so is a number secret shown with spaces or dashes between its digits.
|
|
64
|
+
- A day a step writes as plain `{date+1}` and the app shows in its own
|
|
65
|
+
format (`21/10/2026`, `Oct 21, 2026`) is recorded as the token in that
|
|
66
|
+
format, so the check moves with the calendar.
|
|
67
|
+
- A quoted CSV field keeps its own spaces. A double-quoted `.env` value
|
|
68
|
+
takes `\"` and `\\` escapes. A count in parentheses ("Cart (3)") is no
|
|
69
|
+
longer read as a negative amount.
|
|
70
|
+
|
|
71
|
+
- Mask mixed-case percent escapes in capture names, reports and traces
|
|
72
|
+
through the shared secret masker.
|
|
73
|
+
- Keep state checks bound to their recorded row even when the assertion
|
|
74
|
+
step also presses a key or performs another browser action.
|
|
75
|
+
- Match row values with Unicode-aware regex boundaries, including selector
|
|
76
|
+
fallbacks and downloads. Non-ASCII suffixes cannot satisfy a shorter value.
|
|
77
|
+
- Refuse tokens inside regex repeat counts instead of silently changing
|
|
78
|
+
the assertion. Use a separate Scenario with a literal repeat count.
|
|
79
|
+
|
|
80
|
+
- Mask URL-encoded secrets with lowercase or mixed-case percent escapes,
|
|
81
|
+
including escaped patterns, before saving recordings.
|
|
82
|
+
- Bind state checks to their recorded Outline row when the check does not
|
|
83
|
+
follow its column. A column at the start of the sentence no longer
|
|
84
|
+
bypasses this safeguard.
|
|
85
|
+
- Apply row-value boundaries to selector regex fallbacks and download
|
|
86
|
+
checks. Preserve character-class syntax when resolving regex tokens.
|
|
87
|
+
- Classify Given and inherited setup steps using the feature's language,
|
|
88
|
+
including Latvian keywords shared with other languages.
|
|
89
|
+
|
|
90
|
+
- Rendered-text capture tests allow time for capture stability and browser
|
|
91
|
+
round trips on CI. Hidden and empty text must still fail within the same
|
|
92
|
+
short test timeout.
|
|
93
|
+
- Runner CI installs WebKit alongside Chromium and Firefox so the existing
|
|
94
|
+
tests for all three browser engines can run on GitHub Actions.
|
|
95
|
+
|
|
96
|
+
- Review fixes: a provider or agent error is masked for secrets in full
|
|
97
|
+
before it is cut to its first line, so a secret spanning lines no
|
|
98
|
+
longer shows in the console or a later scenario's error. A typed value
|
|
99
|
+
submitted with Enter presses it on the field typed into, not whatever
|
|
100
|
+
has focus after an input handler ran. A date token with a locale tag
|
|
101
|
+
Node.js rejects (`en-x`) is marked in the editor instead of stopping the
|
|
102
|
+
diagnostics. A report row's `ranAt` is validated like the run's times,
|
|
103
|
+
and the HTML page is rendered before either report file is replaced.
|
|
104
|
+
A quoted value containing a word like "clear" no longer hides a lost
|
|
105
|
+
typed value. An amount with its minus before the currency (`-$5.00`)
|
|
106
|
+
reads as negative, so doubling it is never checked against $10.00. A
|
|
107
|
+
capture named like a token (`price {date}`) keeps its name in an amount
|
|
108
|
+
check at replay. A click that starts several downloads stores the count
|
|
109
|
+
as a number of its own, which no Outline pass rewrites, and the
|
|
110
|
+
recording is written as version 4, so an earlier runner refuses it
|
|
111
|
+
instead of checking the first file. Recording judges a download check
|
|
112
|
+
by replay's blank rule and resolves `{unique:...}` inside Examples
|
|
113
|
+
values, as replay does.
|
|
114
|
+
|
|
115
|
+
- Fourth review round. Outlines: a step that names a column but recorded
|
|
116
|
+
nothing that writes it (the page already showed "myself", or a box was
|
|
117
|
+
ticked) binds the recording to its row, so the "someone else" row is
|
|
118
|
+
refused instead of replaying row 1's page; a recording made before this
|
|
119
|
+
is recorded again once on a row that differs, and the agent is asked to
|
|
120
|
+
pick the row's value even where the page already shows it. A blank
|
|
121
|
+
first row no longer refuses every filled row: a filled row records once
|
|
122
|
+
more, and that recording serves the blank rows. A typed value equal to
|
|
123
|
+
the section's name no longer passes for scoping where the step says it
|
|
124
|
+
is a section (`in the "Double" room card`); captures and an assertion
|
|
125
|
+
step's checks must name the section too, and `the country selector in
|
|
126
|
+
"<section>"` and a hyphenated container word (`time-selection dialog`)
|
|
127
|
+
read as sections. A dialog's button the scoped action opened, scoped to
|
|
128
|
+
the dialog, and a wait, need not name the section. Literal braces in a
|
|
129
|
+
typed value (a JSON address) are text in a person's browser session, and
|
|
130
|
+
a short `{data:...}` value the step writes is the step's own. A date cell
|
|
131
|
+
written with its month's name (`{date+1:MMM D}`) follows the row in any
|
|
132
|
+
step, instead of being recorded again every day. Recording: performing
|
|
133
|
+
several flagged actions again replaces each in its own place, one by
|
|
134
|
+
one, and one not performed again stays where it was. A name Playwright's
|
|
135
|
+
MCP shortened (`'26. Oktober'` for "26. Oktober 2026", `'Room'` for
|
|
136
|
+
"Room 101") is recorded in full from the snapshot the agent clicked in,
|
|
137
|
+
as replay matches names exactly. Through the trace bridge, no page
|
|
138
|
+
fingerprint is taken before a browser action, which had made every
|
|
139
|
+
click by snapshot ref fail. Replay: a captured value is taken once the
|
|
140
|
+
page has stopped changing it for half a second (sooner for a short
|
|
141
|
+
timeout), so a total drawn again after the last change is not read
|
|
142
|
+
stale. Proposals: a proposal git checked out with CRLF line ends is
|
|
143
|
+
still the one this machine filed. Review: a proposal whose only change
|
|
144
|
+
is beside the steps (a row binding, the row it was recorded on, a data
|
|
145
|
+
value) says so in `saffron diff` and the report. `saffron status` judges
|
|
146
|
+
an Outline row the recording refuses as refused, not stale. A CI run's
|
|
147
|
+
results say when the run was interrupted. Step reuse records afresh
|
|
148
|
+
instead of keeping a capture under another value's name, and a fix
|
|
149
|
+
carried to other scenarios never copies a `<column>` into one without
|
|
150
|
+
it. The editor checks date tokens in doc strings too, never on a
|
|
151
|
+
comment line.
|
|
152
|
+
|
|
153
|
+
- Fifth review round. A step naming a column is judged by its own
|
|
154
|
+
actions: another step writing the column, or another column sharing its
|
|
155
|
+
value, no longer lets an empty step pass on other rows. Step reuse never
|
|
156
|
+
puts such a step back for a row whose value differs, so the row records
|
|
157
|
+
it afresh. A check on a bare role (`the error message in "<language>"`)
|
|
158
|
+
takes no section from the quote, and a wait in a section step never
|
|
159
|
+
binds the row. `saffron status` marks a refused row with ✗ and says why,
|
|
160
|
+
instead of calling its recording unreadable and counting it as replaying.
|
|
161
|
+
|
|
162
|
+
- Sixth review round. Recording: a full button name is read only from the
|
|
163
|
+
page snapshot a browser call shows, never from a response body or the
|
|
164
|
+
console. Corrections a step performs again, one by one or in one call,
|
|
165
|
+
each take the place of the flagged action of their kind and element, so
|
|
166
|
+
a fill stays before the Save after it; one that reopens a dialog
|
|
167
|
+
replaces the click that first opened it instead of clicking twice. A
|
|
168
|
+
check spelling its own value (`Then ... "Guest: Tor"`) keeps it, and no
|
|
169
|
+
longer reads another step's table cell. A value typed from
|
|
170
|
+
`{data:users.<who>.email}` is recorded as that reference, so an edited
|
|
171
|
+
data file reaches replay. `the "<room>" room card should show ...` is a
|
|
172
|
+
section without "in". Replay: a capture waits for the page's fetch and
|
|
173
|
+
XHR requests to be answered (up to 3 seconds) before reading.
|
|
174
|
+
|
|
175
|
+
- Seventh review round. One value two references in a step read
|
|
176
|
+
(shipping and billing both "Home") stays as typed and binds the row,
|
|
177
|
+
instead of becoming one of them; in different steps, each keeps its own
|
|
178
|
+
reference. An alert or other bare-role check inside a section the step
|
|
179
|
+
names explicitly must be scoped there. Step reuse keeps out a `When`
|
|
180
|
+
step that checks something but sets nothing of its column. A dialog
|
|
181
|
+
reopened while correcting a step replaces the click that first opened
|
|
182
|
+
it, across the waits between them, and a wait performed again is filed
|
|
183
|
+
once. Only the very same action counts as reopening (the same target in
|
|
184
|
+
the same scope, with the same click options): a click focusing a field
|
|
185
|
+
never replaces its fill, nor a click on a checkbox of the same name. A
|
|
186
|
+
short shared value ("NY"), a blank one, or one written escaped in a
|
|
187
|
+
pattern or selector, binds the row like any other. A `{data:...<column>...}`
|
|
188
|
+
value blank on the recorded row records once more on a filled row,
|
|
189
|
+
as a blank cell does, and one kept in another spelling (`ship-home`)
|
|
190
|
+
binds the row. A recording made before these bindings that still holds
|
|
191
|
+
such a value literally records once more on a row reading another. A
|
|
192
|
+
later check still holding a value another step's reference read (a
|
|
193
|
+
summary showing "Home") binds the row too, and a step that reads a
|
|
194
|
+
value and also spells it as its own constant (`Shipping "{data:...}"
|
|
195
|
+
and Billing "Home"`) binds the row, as which field was which cannot be
|
|
196
|
+
told. A short value ("M", "NY") counts only as a whole word, never
|
|
197
|
+
inside "My cart", and one inside the step's own text ("Shipping to NY")
|
|
198
|
+
becomes its reference. A recording made before this that typed the
|
|
199
|
+
reference for the step's own constant records once more. A value the
|
|
200
|
+
step spells inside a longer word ("Johnson" beside a first name
|
|
201
|
+
"John") binds the row too, and step reuse never puts back a check still
|
|
202
|
+
holding a value a reference read on a row that differs.
|
|
203
|
+
|
|
204
|
+
- Twelfth review round. Recording: a data value never rewrites a
|
|
205
|
+
`<column>` placeholder or a `{data:...}` reference's path ("email" in
|
|
206
|
+
`<email>`), and a tag in a selector (`button`) is never data. A value a
|
|
207
|
+
step spells itself stays its own when another step reads it ("Johnson"
|
|
208
|
+
beside a first name "John"). A check naming a column nothing in the
|
|
209
|
+
recording follows (`Then the banner is <state>`, recorded as "not
|
|
210
|
+
visible") binds the row. A role, an amount written another way
|
|
211
|
+
("12,50" checked as 12.50) or another spelling (`address-home-street`)
|
|
212
|
+
of a `{data:...<column>...}` value binds the row too, and a password
|
|
213
|
+
read through `{data:users.<who>.password}` from `{env:...}` follows the
|
|
214
|
+
row instead of keeping the first row's variable. Step reuse: a `When`
|
|
215
|
+
recording never seeds a `Then` of the same words, and a check holding a
|
|
216
|
+
value a joined key (`zone-<s>`) read, or expecting nothing for a blank
|
|
217
|
+
one, is never put back, also from a recording that names no row.
|
|
218
|
+
Replay: a `{date...}` inside a data value resolves to its day, as it
|
|
219
|
+
did while recording. The review that followed: secrets are masked
|
|
220
|
+
before any other pass, so a row, table or data value inside one never
|
|
221
|
+
cuts it up and leaves the rest in the recording. A state check ("not
|
|
222
|
+
visible", "disabled") naming a column binds the row even where another
|
|
223
|
+
step sets that state, and so does a check showing a date column's day
|
|
224
|
+
in the page's own format; a check stating its own text may name a
|
|
225
|
+
column to describe the row. Another step's value in an action step
|
|
226
|
+
("Ship to NY") binds the row, a session's column (`As "<who>"`) is
|
|
227
|
+
never cut out of a longer word ("Hannah"), a data value holding a
|
|
228
|
+
formatted date becomes its reference, and a constant spelled inside a
|
|
229
|
+
longer word stays however often the step names a reference. A changed
|
|
230
|
+
data file is noticed where a check holds the old value as a role or an
|
|
231
|
+
amount. Step reuse refuses joined keys without a separator
|
|
232
|
+
(`zone<s>`) and empty checks in any spelling (`^(?:)$`), and a
|
|
233
|
+
recording whose data holds a date is version 3, which saffron-ai 1.0.0
|
|
234
|
+
refuses rather than typing `{date+1}`.
|
|
235
|
+
|
|
236
|
+
- Thirteenth review round. Outlines: a row value inside a run of letters
|
|
237
|
+
("red" in "Delivered", "Pro" in "Products") no longer binds the row; a
|
|
238
|
+
camelCase part ("SelectedRed") still does. Rows whose data reads blank
|
|
239
|
+
on every row no longer record each other forever. A row value holding a
|
|
240
|
+
secret's value ("admin@shop.test" beside a variable "admin") is the
|
|
241
|
+
row's `<column>`, and a secret equal to a row's lookup key leaves the
|
|
242
|
+
step's `{data:...}` reference alone. A state check ("Promo" hidden, an
|
|
243
|
+
attribute's pattern) binds the row whatever its element is named. A
|
|
244
|
+
dialog answered after announcing the next step is armed before its
|
|
245
|
+
trigger, in the trigger's step. The fingerprint a row binding keeps of
|
|
246
|
+
a secret is salted and slow, so a short one is not read back from the
|
|
247
|
+
recording (version 3); older plain ones still match. A blank Examples
|
|
248
|
+
cell beside a step with something to check is version 3, and step reuse
|
|
249
|
+
refuses a one-character joined key (`zone0`). An older recording is
|
|
250
|
+
recorded again once a data value it reads holds a date. Replay: a row
|
|
251
|
+
value starting or ending an expected text, URL, response URL or text
|
|
252
|
+
target is never found inside a longer word or number ("Order 1" never
|
|
253
|
+
passes on "Order 12"). A pending proposal that fails to replay does so
|
|
254
|
+
in a browser of its own, so the agent records in a clean one. CLI:
|
|
255
|
+
`saffron prune --yes` names what it could not delete and exits 1; a
|
|
256
|
+
hook killed by a signal says so; `.env` reads `export` lines and drops
|
|
257
|
+
inline comments; "$.50" is 0.50. The language server runs a scenario
|
|
258
|
+
sharing its name with another by line, reads `SAFFRON_ENV` from the
|
|
259
|
+
project's `.env`, and checks date tokens only in steps, tables and doc
|
|
260
|
+
strings. Then: a check that an element found by its text is absent
|
|
261
|
+
("Promo" not visible) binds the row like any state check; a role with
|
|
262
|
+
its text never finds a row value inside a longer one either; re-masking
|
|
263
|
+
a secret never cuts into a `<column>` it sits in ("admin" in
|
|
264
|
+
`<adminEmail>`); and "-$.50" and "($.50)" are negative. The review after
|
|
265
|
+
that: a secret percent-encoded in a recorded URL (`admin%40shop.test`)
|
|
266
|
+
is masked too, and a URL check also reads the page's URL decoded;
|
|
267
|
+
another step's data value shown in another case or as a slug ("MUG")
|
|
268
|
+
binds the row; two columns holding one `{env:...}` password record once
|
|
269
|
+
more for a row typing them apart instead of being refused for good; a
|
|
270
|
+
column opening a Then step that another step writes (`Then <user> sees
|
|
271
|
+
the dashboard`) describes the row; a three-letter data value inside
|
|
272
|
+
text ("1 x Mug") follows its reference. A response URL pattern bounds a
|
|
273
|
+
row value wherever it stands, `saffron status` judges every Examples row
|
|
274
|
+
and finds an install in a parent folder, a pending proposal failing
|
|
275
|
+
for another row keeps its screenshot and trace, a quoted `.env` value
|
|
276
|
+
followed by a comment loses its quotes, and an empty Given in any
|
|
277
|
+
language is not sent back. The full review after that: a secret
|
|
278
|
+
percent-encoded in a pattern, written in another case, or in an
|
|
279
|
+
`encodeURI` URL is masked; a data value that is a variable follows its
|
|
280
|
+
reference inside text too; an amount a text check shows in another
|
|
281
|
+
format, a state check found by its text (visible or not), and a number
|
|
282
|
+
only inside a longer one are each judged as they should be; a column
|
|
283
|
+
opening a check is its subject only when another step writes it and the
|
|
284
|
+
check does not state it (`<state> is ...`); a multi-word value in a URL
|
|
285
|
+
(`q=red+shoes`) follows the row. Replay bounds a row value in every
|
|
286
|
+
pattern (text, name, attribute, response URL and body) and refuses a
|
|
287
|
+
blank one left inside an attribute or body pattern, reads response URLs
|
|
288
|
+
decoded, and keeps a regex's own named groups. A fresh recording that
|
|
289
|
+
never announced its last steps is sent back instead of filed as
|
|
290
|
+
verified. A quoted CSV field after a space is read whole, and a row with
|
|
291
|
+
more fields than its header is refused. `.env`: the last of a repeated
|
|
292
|
+
key wins, quotes match non-greedily, and inherited names are ordinary
|
|
293
|
+
keys; neither data nor rows read inherited properties. `saffron status`
|
|
294
|
+
counts unreadable recordings apart from stale ones, `saffron trace`
|
|
295
|
+
prints report names without terminal escapes, and a run without the
|
|
296
|
+
agent says so. A click performed twice on purpose while correcting a
|
|
297
|
+
step is kept twice; a reopened download trigger takes its download wait
|
|
298
|
+
with it, so the count is not doubled; and an armed action goes before
|
|
299
|
+
its trigger even when the same call waited for something after it.
|
|
300
|
+
|
|
301
|
+
- An Outline whose browser session comes from a column (`As "<who>" ...`)
|
|
302
|
+
no longer lets another row's person type the recorded person's value: a
|
|
303
|
+
step in that session typing a value the step does not write binds the
|
|
304
|
+
recording to its row (written as version 3), so the other rows are
|
|
305
|
+
refused with guidance instead of passing on the first person's data.
|
|
306
|
+
|
|
307
|
+
- Third review round. Recording: each action and check in a section a
|
|
308
|
+
column names must be scoped to it (one scoped action no longer covers an
|
|
309
|
+
unscoped fill beside it), a quoted value the step types is no section,
|
|
310
|
+
and performing a flagged action again replaces only that action, in its
|
|
311
|
+
place, with its download wait. A download is counted from the browser's
|
|
312
|
+
own events only, never twice. An amount the step writes itself stays the
|
|
313
|
+
step's, a column two written amounts match stays literal, a date cell's
|
|
314
|
+
day stays literal only in the steps that name its column, tokens that
|
|
315
|
+
always read alike (`{date+7:dddd}`, `{date+14:dddd}`) are one, and "an
|
|
316
|
+
empty" field reads as nothing to type. Format: a recording made before
|
|
317
|
+
a new-syntax date token was added to its scenario is recorded again
|
|
318
|
+
once as version 3, a "container" row binding is version 3, and
|
|
319
|
+
`rowsUseNewDates` is part of what a proposal binds. Replay: a download
|
|
320
|
+
check, an expected URL, or a target's text, label or placeholder that a
|
|
321
|
+
blank row value leaves partly empty fails instead of matching more; a
|
|
322
|
+
newer dialog answer replaces one no dialog used; a targeted upload that
|
|
323
|
+
starts a download is waited for; a retry sets the chosen files again;
|
|
324
|
+
an amount check's quoted names are never resolved. Runs: `--rerecord`
|
|
325
|
+
refuses, deleting nothing, when no browser of the run can record or the
|
|
326
|
+
agent cannot start; an offset time zone is refused; "retry in 41.8s"
|
|
327
|
+
is a passing throttle; Claude overage use is no limit.
|
|
328
|
+
|
|
329
|
+
- Second review round. Recording: a section a column names that the
|
|
330
|
+
recording ignores binds the row, so another row is refused rather than
|
|
331
|
+
acting in row 1's section; a place or manner (`in "Oslo"`) is no longer
|
|
332
|
+
taken for a section, and a `Given` the page already satisfies is not
|
|
333
|
+
asked for an action. Amounts: accounting, trailing and en-dash negatives,
|
|
334
|
+
decimals such as `0.125`, and an amount written in another format than its
|
|
335
|
+
Examples cell follow the row; a misplaced quote in an expression is
|
|
336
|
+
refused. Dates: a day the recorded row's cell holds stays literal, two
|
|
337
|
+
tokens that read alike stay literal, only whole dates are made tokens
|
|
338
|
+
outside their own step, and an Outline whose other rows use the new date
|
|
339
|
+
syntax is written as version 3. `{date-of-birth}` is typed as written,
|
|
340
|
+
offsets past a hundred years and locale-like typos are named, and a time
|
|
341
|
+
zone must be spelled as browsers spell it. Downloads: one started in a
|
|
342
|
+
popup tab or after its click returned is credited to that click, one
|
|
343
|
+
reported by a later result raises the count, opening a file's URL and a
|
|
344
|
+
chosen file that starts a download replay, and a response body is never
|
|
345
|
+
read as actions. Runs: a throttle that clears in seconds does not stop
|
|
346
|
+
AI sessions, "Quota exceeded" and a Claude subscription limit do, and
|
|
347
|
+
Ctrl-C before any scenario finished keeps the last report.
|
|
348
|
+
|
|
349
|
+
- **Amounts that add up.** A step such as `Then the basket total should be
|
|
350
|
+
"before" plus "breakfast"` records an `expectAmount` check: the target
|
|
351
|
+
must show the amount an expression over captured values states
|
|
352
|
+
(`"before" + "breakfast"`, `"total" - "discount"`, `"night" * 2`),
|
|
353
|
+
compared to the cent. Amounts are read in either decimal style
|
|
354
|
+
(`NOK 1 234,50`, `$1,234.56`, `CHF 1'234.50`); an element showing two
|
|
355
|
+
numbers is refused rather than guessed. A total that changed by the
|
|
356
|
+
wrong amount now fails, where a "differs" check passed. The agent
|
|
357
|
+
records it with `saffron_assert` kind `amount`, which refuses a name no
|
|
358
|
+
step captures. Capture names in the expression are identifiers: no
|
|
359
|
+
Outline, data, date or secret pass rewrites them, and a secret-bearing
|
|
360
|
+
one is renamed together with its capture.
|
|
361
|
+
|
|
362
|
+
- Agents are told that other scenarios reuse only accepted recordings:
|
|
363
|
+
the run's "proposals pending" line, the `project_status` MCP tool and
|
|
364
|
+
the bundled skill say to accept a shared setup before recording more
|
|
365
|
+
scenarios that use it, instead of paying for it in every one. The skill
|
|
366
|
+
also asks for a mid-scenario `Then` at each checkpoint of a route (a 3DS
|
|
367
|
+
challenge, a redirect), the exact control a variant uses, and the
|
|
368
|
+
section a step acts in when labels repeat.
|
|
369
|
+
|
|
370
|
+
- Fixes from a field report on recording a checkout suite with Codex.
|
|
371
|
+
`saffron_done` refuses a pass once while an action step recorded no
|
|
372
|
+
browser action: announcing a step performs nothing, and a step's action
|
|
373
|
+
can land under the next step. A step that acts in a named container
|
|
374
|
+
(`in "Your contact information"`) with a target that ignores it is sent
|
|
375
|
+
back too, since replay acts on the first match; performing it again
|
|
376
|
+
replaces the unscoped action. The step reply and the agent's
|
|
377
|
+
instructions say that announcing performs nothing, ask readiness steps
|
|
378
|
+
to record the condition they wait for, and ask for a fresh snapshot when
|
|
379
|
+
a ref no longer resolves. A second pass stands, so a step that needs
|
|
380
|
+
nothing still finishes.
|
|
381
|
+
|
|
382
|
+
- The lost-value check no longer flags a step that clears a field or
|
|
383
|
+
passes `""`: a quoted field name is not a typed value. An empty fill on a
|
|
384
|
+
step that types a value is still flagged. A blank first Examples row
|
|
385
|
+
binds to `<column>`, and an expected message held in a row's
|
|
386
|
+
`{data:...}` token follows each row.
|
|
387
|
+
|
|
388
|
+
- `--rerecord` records afresh over a pending proposal from an earlier
|
|
389
|
+
run, not only a row-bound one, instead of replaying it at zero AI. Rows
|
|
390
|
+
of the same run share the new recording.
|
|
391
|
+
|
|
392
|
+
- When an agent session fails on its provider's usage limit, no further
|
|
393
|
+
agent session starts in that run, on any worker (the agent describing
|
|
394
|
+
such a message in the application does not count, and the message is
|
|
395
|
+
stored with the scenario's secrets masked): the scenarios that need one
|
|
396
|
+
fail at once with the provider's message, and replays carry on. A
|
|
397
|
+
session in which the agent made no tool call at all says its tools did
|
|
398
|
+
not reach it.
|
|
399
|
+
|
|
400
|
+
- The run summary counts pending proposals whose proof replay failed
|
|
401
|
+
apart from the others, and the report splits them into proof passed,
|
|
402
|
+
proof failed and not proved. A run of named scenarios marks each report
|
|
403
|
+
row kept from an earlier run with when that run started; shards of one
|
|
404
|
+
run are not marked. A pending proposal that does not replay for another Examples
|
|
405
|
+
row names the columns that row types differently, and which of them the
|
|
406
|
+
failing step uses, instead of guessing at the cause.
|
|
407
|
+
|
|
408
|
+
- Typing with submission checks Enter in its own response window, so a
|
|
409
|
+
successful autosave cannot hide a failed submission during healing.
|
|
410
|
+
- Completed form fields and typing stay in the recording if a later field
|
|
411
|
+
or submission fails, including when the agent retries only the failed part.
|
|
412
|
+
Compound calls are validated before their first edit.
|
|
413
|
+
- Agent shutdown allows five seconds for started calls to finish recording,
|
|
414
|
+
then cancels stuck work and fails the incomplete recording with a clear
|
|
415
|
+
timeout. Queued calls stay blocked and provider usage is retained.
|
|
416
|
+
|
|
417
|
+
- Healing checks each field of a multi-field form against its own response
|
|
418
|
+
window, matching replay. The full batch is validated before any field
|
|
419
|
+
changes. Capture tracking preserves named regex groups when step parameters
|
|
420
|
+
change and prefers a unique test ID over app-generated DOM IDs for remounts.
|
|
421
|
+
|
|
422
|
+
- Healing response assertions follow the latest agent interaction in their
|
|
423
|
+
browser session, so an earlier successful retry cannot hide a later failure.
|
|
424
|
+
Agent shutdown waits for started browser calls to finish recording before
|
|
425
|
+
closing their clients or finalizing the failed session; queued calls stay
|
|
426
|
+
blocked.
|
|
427
|
+
|
|
428
|
+
- Codex recording and healing enable the code-mode tool host while retaining
|
|
429
|
+
Saffron's isolated browser tools. Tool-call limits are enforced at the
|
|
430
|
+
gateway, including calls nested in code-mode batches, before an excess
|
|
431
|
+
call can reach the browser. No dependency patch is needed.
|
|
432
|
+
|
|
433
|
+
- Text patterns reject partly blank row, data and environment values, including
|
|
434
|
+
nested tokens. The healing assertion guard retains captured element and
|
|
435
|
+
frame identity, so another frame cannot stand in for a removed target.
|
|
436
|
+
|
|
437
|
+
- Follow-up assertion fixes: redirects retain the original request's action
|
|
438
|
+
window, and partly blank text is refused even when the blank comes through
|
|
439
|
+
an Examples cell or data file. Live comparisons retain the captured target
|
|
440
|
+
and frame when that target becomes hidden or disappears. A unique ID or
|
|
441
|
+
test ID allows ordinary DOM replacement in the same frame; duplicate
|
|
442
|
+
matches remain tied to the original element.
|
|
443
|
+
|
|
444
|
+
- Response assertions check requests started since the latest browser action,
|
|
445
|
+
so an earlier successful save cannot hide a later failed save, even when
|
|
446
|
+
the earlier response arrives late. Waits and consecutive assertions keep
|
|
447
|
+
that window open. Check a request before the next interaction or navigation.
|
|
448
|
+
- Assertions use exact role names and case-sensitive attribute patterns;
|
|
449
|
+
input value checks require a visible input. Use an explicit name regex
|
|
450
|
+
for a partial role name. Text expectations containing a blank row or data
|
|
451
|
+
value alongside other text now fail with guidance. Recording a text or
|
|
452
|
+
input-value assertion requires an explicit expected value, including `""`
|
|
453
|
+
for an intentional empty check. Existing recordings relying on these
|
|
454
|
+
looser checks must be corrected or re-recorded.
|
|
455
|
+
|
|
456
|
+
- Outline recordings refuse other rows when they retain short values in
|
|
457
|
+
action steps, dates without a year or glued to digits, an unparameterized
|
|
458
|
+
blank cell, or an element role taken from the recorded row. A whole
|
|
459
|
+
`{env:...}` or `{data:...}` field equal to a row's value is treated as
|
|
460
|
+
ambiguous even when the step names that token. These cases fail with
|
|
461
|
+
guidance instead of passing on another row's data or repeatedly recording.
|
|
462
|
+
- Step reuse refuses changes to quoted arguments containing several
|
|
463
|
+
placeholders, where it cannot safely separate their values. Row comparisons
|
|
464
|
+
keep date tokens inside environment values literal, matching replay.
|
|
465
|
+
|
|
466
|
+
- Setup hooks can create or refresh files used by `{data:...}` steps. Data
|
|
467
|
+
and referenced environment variables are validated after setup, and replay
|
|
468
|
+
reads the refreshed values. Invalid data stops the run before a browser
|
|
469
|
+
opens and still runs teardown.
|
|
470
|
+
|
|
471
|
+
- Ctrl-C during `saffron run` writes the report of what finished, with or
|
|
472
|
+
without setup and teardown configured, instead of leaving the last run's
|
|
473
|
+
report in place. The report has `"interrupted": true` and `notRun`, the
|
|
474
|
+
scenarios the run selected that did not finish, which count as neither
|
|
475
|
+
passed nor failed; the HTML report, the run's output and `saffron status`
|
|
476
|
+
say it is partial, and its history line is marked partial. The browsers
|
|
477
|
+
close and teardown runs first, then Saffron exits with code 130.
|
|
478
|
+
Playwright's own Ctrl-C handler is off, so it no longer exits before
|
|
479
|
+
teardown has run. `saffron report --merge` refuses an interrupted shard,
|
|
480
|
+
and a run of named scenarios replaces an interrupted report whole.
|
|
481
|
+
|
|
482
|
+
- The run's first line and the report's `provenance` say what produced
|
|
483
|
+
it: the provider, each model asked for and whether it came from
|
|
484
|
+
`--model`, `--heal-model` or saffron.config.json, and the installed
|
|
485
|
+
version of the vendor package (`@openai/codex-sdk`,
|
|
486
|
+
`@anthropic-ai/claude-agent-sdk`, `@cursor/sdk`) beside saffron-ai's. A
|
|
487
|
+
model nobody asked for is left unnamed; the models that answered stay in
|
|
488
|
+
`totals.models`.
|
|
489
|
+
|
|
490
|
+
- **Dates as the app shows them.** A `{date...}` format writes month and
|
|
491
|
+
weekday names (`MMMM`, `MMM`, `dddd`, `ddd`) and two-digit years
|
|
492
|
+
(`YY`), in the locale the token names (`{date+7:D. MMM:de-DE}` is "24.
|
|
493
|
+
Okt.") or in `dateLocale` from `saffron.config.json` (en-US when
|
|
494
|
+
absent). A range is one token, `{date+7 - +9:MMM D, YYYY}`: what is
|
|
495
|
+
written between the offsets is written between the days, and what both
|
|
496
|
+
share is written once ("Oct 24 - 26, 2026", "Oct 30 - Nov 1, 2026",
|
|
497
|
+
"Dec 31, 2026 - Jan 2, 2027"). A suite no longer needs a setup script to
|
|
498
|
+
compute the stay it picks and the range it checks.
|
|
499
|
+
|
|
500
|
+
- **One clock.** `timezone` in `saffron.config.json` sets the calendar
|
|
501
|
+
date tokens count days on and the time zone every browser runs in, so
|
|
502
|
+
the page and the tokens agree on today, around midnight and through
|
|
503
|
+
clock changes. Every token of one replayed scenario resolves on the
|
|
504
|
+
clock the scenario started with.
|
|
505
|
+
|
|
506
|
+
- The agent is given each date of a scenario resolved, on the clock its
|
|
507
|
+
recording started with, instead of the token; the recording turns each
|
|
508
|
+
of those days back into its token, so a date written in a step's text in
|
|
509
|
+
any format moves with the calendar, not only an ISO one.
|
|
510
|
+
|
|
511
|
+
- A `{date...}` token Saffron cannot read, or with a locale this Node.js
|
|
512
|
+
does not know, stops the run before any browser opens, naming it and the
|
|
513
|
+
scenario; the language server marks it in `.feature` and `.saffron`
|
|
514
|
+
files. The TextMate grammar colours ranges.
|
|
515
|
+
|
|
516
|
+
- A recording with a range, a locale, a name or a two-digit year in a date
|
|
517
|
+
token is written as version 3 of the recording format, so saffron-ai
|
|
518
|
+
1.0.0, which would resolve it wrong, refuses it by name.
|
|
519
|
+
|
|
520
|
+
- **What a download says.** A booking confirmation's "Add to Apple
|
|
521
|
+
Calendar" link could only be checked by its `href` and file name, never
|
|
522
|
+
by the dates and identifiers inside the file. Now the click that
|
|
523
|
+
downloads a file records with the download armed before it
|
|
524
|
+
(`waitForDownload`, as a dialog is), and replay waits for that
|
|
525
|
+
download, fails the click when none starts, and keeps the file in
|
|
526
|
+
`.saffron/artifacts/<feature>/<scenario>/downloads/`. A `Then` step
|
|
527
|
+
such as `Then the calendar file should contain
|
|
528
|
+
"DTSTART;VALUE=DATE:{date+7:YYYYMMDD}"` records a content check
|
|
529
|
+
(`expectDownload`): text the scenario's latest download contains, a
|
|
530
|
+
regular expression it matches (case-sensitive, `^` and `$` at each
|
|
531
|
+
line), or both, with `{date...}`, `{data:...}`, `{env:...}` and
|
|
532
|
+
Examples values resolved.
|
|
533
|
+
Line breaks read as `\n`, a calendar or contact file's folded lines
|
|
534
|
+
read whole, and only UTF-8 text files up to 5 MB are checked. The check
|
|
535
|
+
is never healed. While recording, Saffron checks it against the file
|
|
536
|
+
the agent downloaded (`saffron_assert` kind `download`) and shows the
|
|
537
|
+
agent the start of the file when it does not hold. A click that
|
|
538
|
+
downloads a file is no longer lost from a recording made without
|
|
539
|
+
tracing: Playwright's MCP, attached over CDP, could not copy the file
|
|
540
|
+
and reported the click as failed, so nothing was recorded. Saffron now
|
|
541
|
+
keeps the file through its own connection and drops that one error.
|
|
542
|
+
A large page no longer hides a download from the recorder either: a
|
|
543
|
+
tool result cut to fit the agent's context keeps the sections the
|
|
544
|
+
recorder reads (its code, tabs and Events) whole, even where the cut
|
|
545
|
+
would split one, and the parts of a compound call share one allowance.
|
|
546
|
+
A click that starts several downloads records how many; replay waits for
|
|
547
|
+
that many and checks the latest, the file the recording checked. While
|
|
548
|
+
recording, a download check resolves `{env:...}`, `{data:...}` and
|
|
549
|
+
`{unique:...}` values, dates on the recording's clock, and values in a
|
|
550
|
+
pattern escaped, exactly as replay does.
|
|
551
|
+
A recording with a download needs this release; an older runner
|
|
552
|
+
refuses it by name.
|
|
553
|
+
|
|
554
|
+
## 1.0.0: Several people in one scenario
|
|
555
|
+
|
|
556
|
+
Released 2026-10-14.
|
|
557
|
+
|
|
558
|
+
VS Code extension 0.7.0 and JetBrains plugin 0.7.0 (with saffron-ai
|
|
559
|
+
1.0.0): a CI run's row says what it left to view, and a results panel
|
|
560
|
+
shows a run as one page, with its screenshots and buttons for its traces,
|
|
561
|
+
scenarios, report and Import (VS Code: **Show Results**; JetBrains: below
|
|
562
|
+
the runs in the CI Runs tab).
|
|
563
|
+
|
|
564
|
+
Runner
|
|
565
|
+
|
|
566
|
+
- Capture names containing resolved environment secrets become stable opaque
|
|
567
|
+
identifiers in recordings and proposals, with every comparison updated to
|
|
568
|
+
match. Ambiguous names are refused instead of merging distinct captures.
|
|
569
|
+
- Live captured-value comparisons search child frames when no frame selector
|
|
570
|
+
was recorded, matching captures; explicit frame selectors stay scoped.
|
|
571
|
+
Comparisons select the first displayed non-empty value by locator order
|
|
572
|
+
and keep that source while polling, rather than passing on another
|
|
573
|
+
frame or fallback merely because its value differs. Removing a hidden
|
|
574
|
+
duplicate cannot shift the comparison onto another price. A removed or
|
|
575
|
+
replaced selected element fails the comparison rather than switching targets.
|
|
576
|
+
- Captured-value comparisons use displayed text, ignoring hidden descendants.
|
|
577
|
+
Changing only hidden text no longer passes a price-change assertion.
|
|
578
|
+
Captures and comparisons against a live element require a non-empty
|
|
579
|
+
displayed value; hidden or blank elements do not prove a value changed.
|
|
580
|
+
- **Browser sessions.** A step that starts `As "buyer"` runs in a
|
|
581
|
+
browser session of its own: a browser context with its own cookies,
|
|
582
|
+
storage, login and tabs. The session carries over to the steps after it
|
|
583
|
+
until another `As "..."` names one; steps before the first one run in
|
|
584
|
+
the default session (`As "default"` goes back to it), and the name may
|
|
585
|
+
come from an Examples column. A step that reads as a session step and
|
|
586
|
+
names none (`As "" ...`) stops the run with its file and line; `As
|
|
587
|
+
"Jane", I log in` stays an ordinary step. One
|
|
588
|
+
agent records, heals and refines across every session, its browser
|
|
589
|
+
tools following the session of the step it is on (every provider);
|
|
590
|
+
replay opens each session the first time a step needs it and goes back
|
|
591
|
+
to the page a session was on. Tab indexes and network checks count
|
|
592
|
+
within a session. Recordings store nothing new: a step's session
|
|
593
|
+
follows from the step texts, so recordings made before stay valid,
|
|
594
|
+
except for a step whose text already began `As "name" ` (with a space
|
|
595
|
+
after the quote): it now runs in that browser session, which starts
|
|
596
|
+
from `storageState` unless `sessions` gives it a login of its own.
|
|
597
|
+
Reword such a step, or configure the session.
|
|
598
|
+
- **A login per session.** `sessions` in `saffron.config.json`
|
|
599
|
+
(`{ "buyer": { "storageState": ".auth/buyer.json" } }`) starts a session
|
|
600
|
+
signed in as its own person; a session without an entry takes
|
|
601
|
+
`storageState`, as does the default session. Every file is checked
|
|
602
|
+
when the run starts, and so is the shape of each entry: a misspelt key
|
|
603
|
+
stops the run instead of starting that person as the default one, as
|
|
604
|
+
does a `sessions.default` naming a file other than `storageState` (the
|
|
605
|
+
default session never reads it). When the whole suite runs, a
|
|
606
|
+
configured session no step names gets a warning, and when two or more
|
|
607
|
+
sessions the steps name have no entry while `storageState` is set, the
|
|
608
|
+
run warns that they all start signed in as that one person.
|
|
609
|
+
- **A trace per session.** Each session is traced on its own
|
|
610
|
+
(`trace.buyer.zip` beside `trace.zip`, `proof-1.buyer.zip` beside
|
|
611
|
+
`proof-1.zip`), listed under its session's name in `saffron trace`, which
|
|
612
|
+
opens on the session a failure is in and marks the other sessions' steps
|
|
613
|
+
as theirs. The report tags the step where each session takes over and
|
|
614
|
+
names every trace file.
|
|
615
|
+
- **Editors.** Completion, hover, the similar-wording hint and the step
|
|
616
|
+
vocabulary (`saffron steps`, the MCP `search_steps`) offer a step without
|
|
617
|
+
its `As "..."`, so picking one never moves it into someone else's session.
|
|
618
|
+
Accepting a completion replaces the step's text, before and after the cursor, keeping
|
|
619
|
+
the keyword and any `As "..."`, and each suggestion's badge is the status
|
|
620
|
+
of the step as it will read there. A StepSet's preview keeps each step's
|
|
621
|
+
`As "..."`.
|
|
622
|
+
- Step reuse stays in a step's browser session, whether the step names
|
|
623
|
+
it or carries it over from an earlier `As "..."`: another person's
|
|
624
|
+
recording may type that person's own values (an address, a card), so it
|
|
625
|
+
never seeds a step for someone else, not even through a quoted value.
|
|
626
|
+
- A duplicate-wording rename, or a step edit the agent suggests, never
|
|
627
|
+
adds, drops or changes a step's `As "..."`; `saffron accept
|
|
628
|
+
--with-feature-edit` skips an edit that would, with the reason, and
|
|
629
|
+
`saffron diff` says so beside it.
|
|
630
|
+
- **The test folder is yours to name.** `saffron init` creates
|
|
631
|
+
the folder the `features` key names, and `init --examples` installs
|
|
632
|
+
the example suite into it, with its data in `dataDir`, instead of
|
|
633
|
+
`features/` and `data/`. `saffron status --json` names the folder as
|
|
634
|
+
`featuresDir`.
|
|
635
|
+
- **CI runs say what they left.** On GitHub, `saffron runs`
|
|
636
|
+
marks each run with its results and their size, a summary only,
|
|
637
|
+
results expired, none uploaded, or still running, from one listing of
|
|
638
|
+
the repository's artifacts (`results` in `--json`, with the words both
|
|
639
|
+
editors show as `results.label`). Azure Pipelines runs say so when
|
|
640
|
+
opened.
|
|
641
|
+
- **A summary of every run.** `saffron export --report-only` writes
|
|
642
|
+
`saffron-summary.zip`: the last run's report and screenshots, no
|
|
643
|
+
proposals and no traces, small enough to upload on every run. Viewing a
|
|
644
|
+
run (`report --run`) uses, for each shard and attempt, its full bundle
|
|
645
|
+
when there is one and its summary otherwise (`bundle.summaryOnly` when
|
|
646
|
+
only summaries were left); importing a run that left only summaries
|
|
647
|
+
says there is nothing to import. A full bundle, or a later attempt,
|
|
648
|
+
stands only for the same browser, base URL and shard: beside another
|
|
649
|
+
browser's or environment's results the view and import refuse the mix
|
|
650
|
+
by name instead of hiding a failure.
|
|
651
|
+
- **A CI run as one page.** Viewing a run writes its results page beside
|
|
652
|
+
its report (`panelHtml` in `report --run --json`; `--panel` prints it): totals, failed and pending-review scenarios as
|
|
653
|
+
cards with their screenshots, what passed, the proposals, and buttons
|
|
654
|
+
an editor wires to Open trace, Open scenario, Open report, Open run
|
|
655
|
+
page and Import.
|
|
656
|
+
- Agents other than Claude Code (Codex, Cursor, Antigravity) have their
|
|
657
|
+
tool calls run one at a time in the order they arrive, as
|
|
658
|
+
Claude Code runs them, so a step announced before a click is in effect
|
|
659
|
+
when the click runs; a call the vendor gave up on while it waited never
|
|
660
|
+
starts.
|
|
661
|
+
- **The Saffron plugin for Codex.** The plugin repo that serves Claude
|
|
662
|
+
Code is now a Codex marketplace too: `codex plugin marketplace add
|
|
663
|
+
s-chathuranga-j/saffron-claude-plugin`, then `codex plugin add
|
|
664
|
+
saffron@saffron`, installs the MCP server and the skill. Codex plugins
|
|
665
|
+
have no language server slot, so diagnostics and navigation stay
|
|
666
|
+
Claude Code only.
|
|
667
|
+
- **Node.js 22.12 or later.** The command-line library Saffron depends
|
|
668
|
+
on needs it, so an install that enforces engine versions failed on the
|
|
669
|
+
Node 20.6 the docs promised. Node 20 is past its end of life.
|
|
670
|
+
- **Playwright 1.64.** A recording session refuses `browser_tabs` with
|
|
671
|
+
`isolatedContext`: such a tab would run without the scenario's sign-in
|
|
672
|
+
and trace, and replay as a plain tab in the shared context.
|
|
673
|
+
|
|
674
|
+
Fixes:
|
|
675
|
+
|
|
676
|
+
- **An Outline row never passes on another row's data.** Recording and
|
|
677
|
+
the check that decides whether a shared recording fits a row now use
|
|
678
|
+
one rule for where a row value sits, so whatever the check would flag
|
|
679
|
+
is written as `<column>`: a value glued to other letters or digits
|
|
680
|
+
(`order<id>`, `2 <item>s booked`, `<name>님`), in any normalization, and
|
|
681
|
+
a value holding markup (`<script>...</script>`, `Alice <alice@x.com>`).
|
|
682
|
+
A value two columns share on the recorded row stays as typed (which
|
|
683
|
+
column it was cannot be told): a row typing them apart is recorded again
|
|
684
|
+
once, and where no row does (`ship` and `bill` both France, both Spain),
|
|
685
|
+
the recording is bound to its row. A value glued to other
|
|
686
|
+
text is written as `<column>` only in steps that use that column. Where one
|
|
687
|
+
recording cannot carry a row's value, it is bound to its row: a field
|
|
688
|
+
that still holds other rows' values beside the recorded row's
|
|
689
|
+
(`Red, Green, Blue`, `Plan changed: Basic to Pro`) stays as recorded,
|
|
690
|
+
and so does the row's value glued inside other text in a step that does
|
|
691
|
+
not name its column (`alice2024@example.com`). The recorded row, and
|
|
692
|
+
rows with the same value, replay; every other row fails red at zero
|
|
693
|
+
tokens, without healing, recording again or calling the agent, with a
|
|
694
|
+
message naming the column, the step and the fix: put `<column>` in the
|
|
695
|
+
step, give the text its own Examples column, or give that check its own
|
|
696
|
+
Scenario. A spelling recording cannot write binds the recording to its
|
|
697
|
+
row the same way, wherever the check that decides whether a recording
|
|
698
|
+
fits a row would find it: another case (`Welcome, Alice` for `alice`),
|
|
699
|
+
a slug (`/weather/new-york`), URL-encoded (`?q=C%2B%2B`), words joined
|
|
700
|
+
(`data-isopen` for `Is Open`). The other rows fail with guidance
|
|
701
|
+
instead of recording each other again forever. So does a value of one
|
|
702
|
+
or two characters that an assertion not naming its column checks
|
|
703
|
+
(`?page=1`, `Qty: 1`), which would otherwise pass on 10 or 12. A date
|
|
704
|
+
typed in ISO for a cell in another format makes the other rows record
|
|
705
|
+
again rather than pass, and so does a row filling columns the
|
|
706
|
+
recorded row left blank, and one whose shared value came from an
|
|
707
|
+
environment variable that has changed since. A recording that reads a
|
|
708
|
+
value through
|
|
709
|
+
`{data:...}` or `{env:...}` stays current when the data changes. The
|
|
710
|
+
recording format version no longer depends on the environment or clock.
|
|
711
|
+
The check also holds for a row added after recording, for `{data:...}`
|
|
712
|
+
and `{env:...}` cells, which count by the value they read, and for an
|
|
713
|
+
environment variable changed since recording, named in the cell or in
|
|
714
|
+
its `{data:...}` value (the recording keeps only a one-way hash of the
|
|
715
|
+
value, never the value). Recording and the check read a `{data:...}` value that is itself
|
|
716
|
+
`{env:...}` the way replay types it, data first, so such a row neither
|
|
717
|
+
records over the others forever nor passes on their values. A value the
|
|
718
|
+
step writes itself as the whole field, such as a login with `{env:...}`
|
|
719
|
+
or through its own `{data:...}` reference, never binds the row; a row
|
|
720
|
+
value glued to such a value (`{env:ADMIN}@example.com` typed as the
|
|
721
|
+
row's mail) still does. A login, a base URL (`{env:BASE_URL}/cart`) or text
|
|
722
|
+
the step spells in full counts as the step's own, so a row value that
|
|
723
|
+
happens to sit inside it never binds the row. A `{data:...}` reference
|
|
724
|
+
keyed by a column (`{data:users.<user>}`) is recorded as written, so each
|
|
725
|
+
row reads its own entry instead of the rows recording over each other;
|
|
726
|
+
when two such references choose the same entry on the recorded row, a
|
|
727
|
+
step that reads it without writing either one binds the recording to its
|
|
728
|
+
row, and editing that entry in the data file no longer makes the
|
|
729
|
+
recording stale. An expected response status or method, or an
|
|
730
|
+
attribute's name, follows its column in a step that names it (`the
|
|
731
|
+
response status is <status>`), where it was recorded as the first row's
|
|
732
|
+
and a row expecting another status could pass. A pending
|
|
733
|
+
proposal bound to its row refuses the other rows the same way, so they
|
|
734
|
+
never record over it, and the message names `saffron run --rerecord`
|
|
735
|
+
when the row it was recorded on is gone.
|
|
736
|
+
- Assertions read the first element a target matches. A first match that
|
|
737
|
+
is hidden (including `visibility:hidden`, a closed `<details>` or
|
|
738
|
+
`content-visibility:hidden`) gives way only to a shown one with the same
|
|
739
|
+
text, so "You have unsaved changes" can no longer pass a check for a
|
|
740
|
+
hidden "Saved". Expected text matches with whitespace collapsed first,
|
|
741
|
+
then as written; a blank expected text passes on a target that shows
|
|
742
|
+
none; a list check also matches the list as the page shows it
|
|
743
|
+
(`text-transform`). A column inside a frame selector is escaped on
|
|
744
|
+
replay.
|
|
745
|
+
- Linked folders inside the test folder are followed, each folder once,
|
|
746
|
+
so a link that loops no longer stops the run. A folder that cannot be
|
|
747
|
+
read, directly or through a link, stops it with the error, as in 0.9.9, instead of dropping its
|
|
748
|
+
scenarios and calling their recordings orphaned.
|
|
749
|
+
- An Outline row whose Examples cell is a `{data:...}`, `{unique:...}` or
|
|
750
|
+
`{date+N}` token, in any date format (`{date+2:DD.MM.YYYY}`), replays
|
|
751
|
+
its own value, not the first row's.
|
|
752
|
+
- A blank expected text or pattern (a `{data:...}` value or an Examples
|
|
753
|
+
cell that is now empty) passes only on an element that shows no text,
|
|
754
|
+
never on one showing any text.
|
|
755
|
+
- A response's expected status or method, or an attribute's name, follows
|
|
756
|
+
its `{data:...}` or `{env:...}` value: a data file changing 201 to 422
|
|
757
|
+
makes the recording read 422, and an older recording still holding 201
|
|
758
|
+
there is recorded again. A step reused for another quoted value (`"201"`
|
|
759
|
+
to `"422"`) takes the new value there too, or is not reused.
|
|
760
|
+
- An attribute check whose pattern takes only an empty value fails when
|
|
761
|
+
the attribute is missing: `disabled` checked as `^$` no longer passes
|
|
762
|
+
once the button is enabled, and neither does one taking any value (`.*`)
|
|
763
|
+
or anything but false (`^(?!false$)` for "is pressed"). A pattern that
|
|
764
|
+
takes an empty value and `false` but not `true`, such as `^(false)?$`
|
|
765
|
+
for "not invalid", still holds on a missing attribute, as in 0.9.9. A
|
|
766
|
+
recording that relied on another pattern holding on a missing attribute
|
|
767
|
+
now fails and needs recording again.
|
|
768
|
+
- `expectDiffers` without a target keeps the capture it compares as
|
|
769
|
+
recorded, like the capture that saves it, so other Outline rows no
|
|
770
|
+
longer fail with "no captured value named".
|
|
771
|
+
- A field that a blank cell or `{data:...}` value leaves empty no longer
|
|
772
|
+
makes a check hold for anything. An attribute's pattern left blank
|
|
773
|
+
passes only on an empty or missing attribute. An expected URL, a
|
|
774
|
+
response's URL, status, method or body, or the name or text of the
|
|
775
|
+
element a step finds, left blank for a row fails with a message naming
|
|
776
|
+
the field. A field that keeps other text around the blank value
|
|
777
|
+
(`Total: <amount>`, a pattern `\b<v>\b`) checks only that text.
|
|
778
|
+
- A list check (`Then the plan lists every {data:plans.<tier>}`) reads the
|
|
779
|
+
row's own list, and one recorded before, still reading the first row's,
|
|
780
|
+
is recorded again.
|
|
781
|
+
- A column a step names follows its row inside an attribute's name too
|
|
782
|
+
(`data-<tier>`), in any case (`disabled` for a row's `Disabled`), so two
|
|
783
|
+
rows no longer record each other again forever. A response's status or
|
|
784
|
+
method, or an attribute's name, takes a `{data:...}` reference only when
|
|
785
|
+
it is that whole value and the step names the reference: a 200 is not
|
|
786
|
+
an order's quantity of 200. A captured value's name (`captureText` and
|
|
787
|
+
`expectDiffers`) no longer counts as a row's value, which also had two
|
|
788
|
+
rows recording each other again forever.
|
|
789
|
+
- A `{data:...}` reference that two references chose alike on the
|
|
790
|
+
recorded row binds the recording to its row when a response status,
|
|
791
|
+
method or attribute holds it too, as it does elsewhere.
|
|
792
|
+
- A step reused for another quoted `{data:...}` entry, or for an edited
|
|
793
|
+
doc string, is not reused while the old entry's value or the old doc
|
|
794
|
+
string is still written in other text (a test id `row-alice-smith`) or
|
|
795
|
+
the old reference is. An edited doc string still written that way makes
|
|
796
|
+
the recording stale too.
|
|
797
|
+
- An Outline recording made by 0.9.x, which does not name its row, is
|
|
798
|
+
recorded again once on a row it holds another row's value for. Under
|
|
799
|
+
`--no-agent`, as in CI, that row stays red, with a message saying the
|
|
800
|
+
recording comes from an older saffron-ai, until it is recorded once.
|
|
801
|
+
- A response status, method or attribute that a step not naming the
|
|
802
|
+
column checks as the row's value (`Then the API answers as expected`,
|
|
803
|
+
recorded as 201 on the row whose `<code>` is 201) binds the recording to
|
|
804
|
+
its row for every column holding that value, so the other rows fail with
|
|
805
|
+
guidance instead of checking 201. So does one holding what a `{data:...}`
|
|
806
|
+
reference keyed by columns reads on that row
|
|
807
|
+
(`{data:users.<region>.<user>}` reading 201), for the columns it reads.
|
|
808
|
+
Both are judged on the recorded row alone, so a row added after the
|
|
809
|
+
recording was accepted is refused too. So does an attribute's name or a
|
|
810
|
+
list check's data path holding the row's value, in any case or as a
|
|
811
|
+
slug: an attribute's name as a whole part (`aria-expanded` for a row's
|
|
812
|
+
`Expanded`, `data-is-open` for `Is Open`, never `aria-invalid` for
|
|
813
|
+
`Ali`), a data path inside a key too (`{data:plans.basicPlan}` for
|
|
814
|
+
`basic`). Where the step names the column, the value follows the row
|
|
815
|
+
instead when it is a whole part of the name or path: an attribute's
|
|
816
|
+
name in any case (`data-<tier>`), a data path spelled as the cell is
|
|
817
|
+
(`{data:plans.<tier>}`). Inside a word it is never written as
|
|
818
|
+
`<column>` (`aria-invalid` stays as it is on a row whose value is
|
|
819
|
+
`valid`). A value that only coincides (an
|
|
820
|
+
amount of 200 beside a check of status 200, a `post` beside method POST,
|
|
821
|
+
a `disabled` state beside the `disabled` attribute) binds too: give the
|
|
822
|
+
check its own Scenario, or give the expected value its own column.
|
|
823
|
+
- The text or regex inside a fallback or frame selector that is blank for
|
|
824
|
+
a row (`button:has-text("<v>")`, an XPath `contains(., '<v>')`,
|
|
825
|
+
`text=<v>`, anywhere in a `>>` chain) fails with a message naming the
|
|
826
|
+
selector, instead of matching any element. An exact attribute match
|
|
827
|
+
such as `input[value="<v>"]`, with or without the `i` or `s` flag, still
|
|
828
|
+
checks for an empty value.
|
|
829
|
+
- A failed `Then` stays red when a pending proposal from `--rerecord`
|
|
830
|
+
checks something else there; it no longer turns yellow. The same holds
|
|
831
|
+
when the run fails before reaching the `Then`: a pending proposal covers
|
|
832
|
+
it only if it checks every `Then` the assertion policy protects as the
|
|
833
|
+
committed recording does. A heal never files an edit to the text of a
|
|
834
|
+
`Then` step.
|
|
835
|
+
- An `{env:...}` value with a `'` or regex characters in it (an email) is
|
|
836
|
+
masked in a failed step's message, reports, proposals and traces, where
|
|
837
|
+
Playwright writes it escaped.
|
|
838
|
+
- Data-table and doc-string parameterization preserves captured-value names
|
|
839
|
+
in comparisons. Editing a table value no longer makes replay look for a
|
|
840
|
+
different capture that was never saved.
|
|
841
|
+
- Network body assertions wait for a successfully read, complete response.
|
|
842
|
+
A pending or failed body read no longer counts as an empty body, and
|
|
843
|
+
responses longer than 262,144 characters are checked in full instead of
|
|
844
|
+
letting a truncated prefix pass the assertion. A body check against a
|
|
845
|
+
response that is not JSON, text or XML (an image, a binary file) fails
|
|
846
|
+
instead of matching an empty body.
|
|
847
|
+
- **Recording format version 2.** A recording with several people
|
|
848
|
+
(`As "buyer"` steps), or one that uses a double, right or modified
|
|
849
|
+
click, a multi-select or a cleared selection (all now recorded and
|
|
850
|
+
replayed as made), is written as version 2, so an older runner refuses
|
|
851
|
+
it instead of replaying everyone in one browser or a plain click. So is
|
|
852
|
+
an Outline recording bound to its row or keeping a recorded row's value
|
|
853
|
+
as typed, which only a runner that checks the Examples row replays
|
|
854
|
+
safely, and one whose response status, method or attribute name follows
|
|
855
|
+
a column or holds a `{data:...}`, `{env:...}`, `{date...}` or
|
|
856
|
+
`{unique:...}` token, which a 0.9.x runner would read literally. Every
|
|
857
|
+
other recording stays version 1.
|
|
858
|
+
- Recordings 0.9.9 saved for names with no ASCII letters (`unnamed.json`)
|
|
859
|
+
are still found, a damaged older recording is reported as invalid
|
|
860
|
+
instead of recorded again, and `saffron prune` keeps an older evidence
|
|
861
|
+
folder its live scenario still owns until the scenario has run under
|
|
862
|
+
the new name. `saffron runs` works with a `gh` too old to report when a
|
|
863
|
+
re-run started.
|
|
864
|
+
- `saffron steps`, `status` and the MCP `search_steps` count a step
|
|
865
|
+
recorded in any person's session as recorded, and divergence is counted
|
|
866
|
+
per session; `search_steps` never shows a session's `As "..."` text.
|
|
867
|
+
The MCP tools read `saffron.config.json` on every call, so a changed
|
|
868
|
+
test folder is seen without restarting the server.
|
|
869
|
+
- A dropdown option whose text holds `]` (`Large [L]`) is recorded whole.
|
|
870
|
+
A `{date+N}` Examples cell is matched only as a whole value, only in
|
|
871
|
+
the steps that use it, in any format including two-character ones
|
|
872
|
+
(`{date+1:DD}`) and inside regex and selector fields.
|
|
873
|
+
- `expectEach` reads the same shown text as the other text checks: a
|
|
874
|
+
visible value is found even when a hidden option has the same text, and
|
|
875
|
+
a hidden line break no longer splits a word in two. Only a line the
|
|
876
|
+
page draws ends an entry (a line break in the HTML source does not), and
|
|
877
|
+
text inside a closed `<details>`, `hidden="until-found"` or
|
|
878
|
+
`content-visibility: hidden` is not shown. A check aimed at a dropdown
|
|
879
|
+
option, a `<textarea>`'s text, or a visible part of a hidden container
|
|
880
|
+
passes again, and text checks are about ten times faster on deep pages.
|
|
881
|
+
A text or visibility check reads the first copy of its target the page
|
|
882
|
+
shows, skipping a hidden duplicate (a hidden mobile menu before the
|
|
883
|
+
desktop one), and never another copy further down. A two-line doc
|
|
884
|
+
string or a value with doubled spaces matches the page's text as it is
|
|
885
|
+
shown, and text below the fold in a `content-visibility: auto` box is
|
|
886
|
+
found in WebKit too.
|
|
887
|
+
- A `{date+N}` Examples cell passed through a step's table or doc string
|
|
888
|
+
replays each row's own date. When two Examples columns (or two keys of a
|
|
889
|
+
table) hold the same value in the recorded row, that value stays as
|
|
890
|
+
typed rather than being guessed into one of them: a later row with
|
|
891
|
+
different values records again instead of passing on the wrong column.
|
|
892
|
+
A recording keeps the Examples row it was made on (`examples` in the
|
|
893
|
+
recording), so a row whose value differs from one left as typed is
|
|
894
|
+
recorded again rather than replayed. A date spelled with its year
|
|
895
|
+
follows its row in later steps too; dates resolve on the clock the
|
|
896
|
+
recording started on. Step reuse applies the same check, so a step
|
|
897
|
+
still holding another row's value is never reused for a row that
|
|
898
|
+
differs, and a one- or two-character value (`1`, `2`) that fills a
|
|
899
|
+
whole field follows its row too, including as the whole of a regex
|
|
900
|
+
(`^12$`). An Examples cell left blank on purpose replays blank, in the
|
|
901
|
+
fill and in the assertion that checks it, instead of being reported as
|
|
902
|
+
a lost value.
|
|
903
|
+
- An Outline assertion keeps its whole recorded text with only the row's
|
|
904
|
+
value made a placeholder (`Order for <user>: cancelled`, not just
|
|
905
|
+
`<user>`), so a row cannot pass on a check cut down to its value. A
|
|
906
|
+
value the step writes out itself (`I log in as "alice"`) is never bound
|
|
907
|
+
to an Examples column, a value inside a CSS selector (`#row-alice`)
|
|
908
|
+
follows its row, and step reuse never carries one column's placeholder
|
|
909
|
+
into another scenario, nor reuses a step whose recording still names
|
|
910
|
+
the old value in another spelling (`Red Shoe` as `/product/red-shoe`).
|
|
911
|
+
Recordings made by 0.9.9 are not rewritten; a newly recorded Outline
|
|
912
|
+
whose rows differ in such text records each row's check as it is.
|
|
913
|
+
- A CSV line of only commas or spaces is no row (write `"",""` for a
|
|
914
|
+
deliberate empty row). An empty `"features"` means `features/`
|
|
915
|
+
everywhere.
|
|
916
|
+
- A re-run GitHub run's results label describes what its latest attempt
|
|
917
|
+
left, shard by shard. In a sharded workflow, name each summary
|
|
918
|
+
`saffron-summary-<shard>` (the docs' snippet now says so).
|
|
919
|
+
- Text checks pass only on text the page shows: hidden text, a hidden part
|
|
920
|
+
of a visible element or a hidden dropdown option no longer satisfies
|
|
921
|
+
"should be shown", and text in a `display: contents` wrapper does. A
|
|
922
|
+
check against a `{data:...}` list (`expectEach`) leaves out hidden
|
|
923
|
+
options and hidden option groups too.
|
|
924
|
+
- A filled value that looks like code (`text .click('x')`) is recorded as
|
|
925
|
+
the fill it is; two confirm dialogs in one step each keep their own
|
|
926
|
+
handler.
|
|
927
|
+
- Step reuse refuses a quoted-value match when the old value survives in
|
|
928
|
+
a test id, selector or frame (`I delete "ann"` never clicks
|
|
929
|
+
`delete-bob`), and when the recording's locators name a placeholder
|
|
930
|
+
the scenario lacks.
|
|
931
|
+
- `saffron report --merge` checks every shard history line field by
|
|
932
|
+
field, and the dashboard escapes every value it shows.
|
|
933
|
+
- A proposal's feature edit that would leave a step empty is refused.
|
|
934
|
+
- Healing a mid-scenario assertion (`adaptable-mid`) keeps what the step
|
|
935
|
+
already did, such as a remembered value.
|
|
936
|
+
- Secrets stay masked in a failed proof's tab and response URLs (report,
|
|
937
|
+
bundle) and in both terminal lines a pending proposal prints when it no
|
|
938
|
+
longer replays.
|
|
939
|
+
- `saffron author` with Claude runs with no built-in tools.
|
|
940
|
+
- A recording without `recordedAt` no longer stops step reuse.
|
|
941
|
+
- A date the scenario writes out (`I enter 2026-10-10`, a table, a data
|
|
942
|
+
value, an `{env:...}` variable) is entered as written; only a date the
|
|
943
|
+
agent picked moves with the calendar.
|
|
944
|
+
- Typed text keeps its line breaks and tabs; double, right and modified
|
|
945
|
+
clicks replay as recorded; a multi-select keeps every option.
|
|
946
|
+
- A browser-matrix bundle's evidence (`.../chromium/failure.png`) is
|
|
947
|
+
accepted on view and import.
|
|
948
|
+
- `features: "."` finds only real files, never `.saffron/`, `node_modules`
|
|
949
|
+
or `.git`, and `admin/login.feature` and `shop/login.feature` get a
|
|
950
|
+
recording folder each instead of sharing one (so does a file outside
|
|
951
|
+
the `features` folder). Recordings saved in the shared folder are
|
|
952
|
+
still found by the feature they name; files directly in the folder
|
|
953
|
+
keep their paths.
|
|
954
|
+
- A browser session named by an Examples column (`As "<who>"`) never
|
|
955
|
+
seeds another row's step: the recording made for the buyer's row is
|
|
956
|
+
not reused for the seller's.
|
|
957
|
+
- Scenario names with letters outside ASCII (`Login Успешный`, `Café
|
|
958
|
+
ouvert`) get distinct recording files and screenshot folders;
|
|
959
|
+
recordings saved under the earlier name are still found.
|
|
960
|
+
- Completing a step in the middle of a line, or before its `As "..."`,
|
|
961
|
+
never duplicates or drops text.
|
|
962
|
+
- `saffron status --json` says when the orphan scan could not finish
|
|
963
|
+
(`orphanScan: { complete, errors }`); both editors show it instead of
|
|
964
|
+
"no orphaned recordings".
|
|
965
|
+
|
|
966
|
+
Agent skill and Claude Code plugin
|
|
967
|
+
|
|
968
|
+
- The skill now teaches a loop for writing a new test: survey the
|
|
969
|
+
project, search the existing steps, place the scenario (StepSet or
|
|
970
|
+
inline), fix the language server's diagnostics, run only that
|
|
971
|
+
scenario, review and accept. It says to write steps as a user sees the
|
|
972
|
+
page, never as selectors or test ids, and the syntax reference ends in
|
|
973
|
+
one complete example project.
|
|
974
|
+
- The plugin's listing describes what Saffron does and its README states
|
|
975
|
+
the license terms in plain words.
|
|
976
|
+
|
|
8
977
|
## 0.9.9: A browser matrix in one run
|
|
9
978
|
|
|
10
979
|
Released 2026-10-06.
|