saffron-ai 1.0.0 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,659 @@ the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
5
5
  JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
6
6
  release heading names its main feature: `## <version>: <title>`.
7
7
 
8
+ ## 1.1.0: Secrets the AI agent never sees
9
+
10
+ Released 2026-10-22.
11
+
12
+ **Upgrading from 1.0.1.** Nothing to change in a project. What can be
13
+ different after `npm update`:
14
+
15
+ - Row value boundaries are stricter: a regex atom that may match nothing
16
+ or anything (`\W*`, `.+`, `[^,]+`), a lookaround, or a group edge or
17
+ alternation next to a row value (`Order <id>(?:st|)`) no longer drops
18
+ the value's boundary, and row values are matched in every selector-chain
19
+ container. A row value at the edge of a role's name, a label, a
20
+ placeholder, an attribute (`alt`, `title`, `[aria-label*=...]`, XPath
21
+ `contains(@href, ...)`), `:text(...)`, `:text-matches(...)`, an inner
22
+ `has` locator or a frame's selector is whole too, as in text: row `1`
23
+ never finds `Order 12`. Beside a digit, a decimal or thousands part
24
+ counts as part of the number (`1` is not in `1.5` or `1,000`) unless the
25
+ pattern writes it (`<n>\.\d{2}`), and a pattern with a quantifier right
26
+ after a row value (`<id>+`) is refused. Text a selector says must be
27
+ absent (`:not(...)`, XPath `not(...)`, `has-not-text`) is no longer
28
+ looked for as present.
29
+ A recording that passed in 1.0.1 because of that can turn red after
30
+ `npm update`; it never turns a failing check green. Re-record what fails
31
+ (`saffron run --rerecord <file>:<line>`).
32
+ - Recordings that use JSON response field checks, or single-quoted row
33
+ values in any part of a selector chain (an XPath part first too), are
34
+ format version 5. Saffron 1.0.1 refuses them by name, so a team updates
35
+ together.
36
+ - An Outline with `As "<who>"` steps recorded before 1.0.1, where a step
37
+ types a value of that person's own, records again once on another
38
+ person's row (red under `--no-agent`, saying why). The new recording
39
+ then refuses the other rows by name, as 1.0.1 recordings of that shape
40
+ do: give each person's values a column, or each person a Scenario.
41
+ - `saffron run` exits 2 when the files it is given hold no scenario (only
42
+ step sets, or nothing yet), where it said "0 passed" and exited 0.
43
+ - Amount checks read a minus spaced off at the start (`- €5.00`) or after
44
+ a currency (`CHF-5.00`) as negative, and a hyphen after another word
45
+ (`ID-123`) as no sign.
46
+
47
+ - Agent-blind secrets: the recording agent no longer sees `{env:...}`
48
+ values. It is given the token and types the token; Saffron's browser
49
+ tools put the value into the page, and every tool result (snapshots,
50
+ echoed code, console, network, `browser_evaluate`) and every Saffron
51
+ reply comes back with the value as its token, as do the prompt's base
52
+ URL and a heal prompt's replay failure and page address. That covers a
53
+ snapshot line Playwright single-quotes, a value Playwright's call log
54
+ cuts with "…", and a downloaded file quoted in a reply. A token counts
55
+ only where the browser types, picks, opens or uploads it (`{env:...}` in
56
+ `browser_evaluate` code is refused), and a word the scenario or a data
57
+ value it uses writes with a short value glued in (`qa{env:PIN}`) is
58
+ shown back as written. The
59
+ recording reads the browser's own results, so a value quoted in
60
+ Playwright's echoed code is masked to its token there too. This holds
61
+ for Claude, Codex, Antigravity and Cursor alike. The guide has a diagram
62
+ of the flow and lists what remains: values under three characters, a
63
+ page that shows the value transformed or only part of it, code that
64
+ reads a field the value was typed into, same-text matches masked along
65
+ with the value, and a check on the value shown in another case.
66
+
67
+ - `saffron eject` writes the committed recordings as Playwright Test specs
68
+ you own from then on: one spec per feature file, each Gherkin step a
69
+ `test.step`, a helper file beside them, and `playwright.config.ts` when
70
+ there is none. An exported test is never looser than replay: it may fail
71
+ where replay would pass, never the other way, and what has no faithful
72
+ form yet (amount checks, patterns or selectors holding a row value, JSON
73
+ response checks) is written as `test.fixme` with the reason. `{env:...}`
74
+ values are read when the test runs and never written into a file. The
75
+ exported checks read an element's text exactly as replay does, so CSS
76
+ `text-transform` or text split over blocks cannot pass there and fail in
77
+ replay. Two feature files that would share a spec file stop the command
78
+ before anything is written.
79
+
80
+ - UI tests can check JSON response fields by path, including nested fields
81
+ and array entries. Equality checks preserve string, number, boolean and
82
+ null types; presence checks distinguish a missing field from null.
83
+ Failures name the field and expected and actual values, with secrets
84
+ masked. These checks replay without AI and Then assertions stay strict.
85
+ Recordings using them require format version 5, so older runners refuse
86
+ them with update guidance instead of skipping the JSON checks. A field's
87
+ path is kept as recorded: a row value that sits in it by chance
88
+ (`/errors/0`) never makes another row check another field.
89
+
90
+ - Match row values in every selector-chain container, including single-quoted
91
+ text. Such a recording is format version 5, since 1.0.1 would fill a
92
+ single-quoted value in without a row boundary.
93
+ - A row value keeps its boundary in a pattern next to an atom that may
94
+ match nothing or anything (`\W*`, `\d*`, `.+`, `\S+`, `[^,]+`, `\P{L}+`),
95
+ a lookaround (`(?!-)`, `(?<!-)`), or a group edge or alternation
96
+ (`(x|<id>)`): `Order <id>.+` fails row "1" on "Order 12 shipped".
97
+ - An XPath selector (`xpath=//button`) replays: it was read as a regex
98
+ with the flags "button" and threw.
99
+ - An Outline with `As "<who>"` steps recorded before 1.0.1 records again
100
+ once on a row of another person, instead of replaying the first person's
101
+ values; the new recording refuses the other rows by name.
102
+ - Amounts: a minus spaced off at the start (`- €5.00`, `- 5,00 €`) or
103
+ glued after a currency (`CHF-5.00`, as Swiss formats write it) is
104
+ negative, and a hyphen after another word (`ID-123`) is no sign.
105
+ - A run of feature files with no scenario in them exits 2 and says so,
106
+ instead of "0 passed". A step set file that does not parse is named in
107
+ the error. Feature edits apply to files written with CRLF and keep their
108
+ line endings. A recording is written beside and renamed over, so a
109
+ stopped run never leaves half of one.
110
+ - Mask case-changed encoded secrets in capture names, reports and traces.
111
+ - Read labelled counts such as `Guests (2 pax)` as positive, also where
112
+ the label is a currency code in lower case (`Pens (2 pen)`). Accounting
113
+ parentheses still denote negative amounts with currency notation.
114
+
115
+ ## 1.0.1: Amounts, downloads and dates as the app shows them
116
+
117
+ Released 2026-10-21.
118
+
119
+ **Upgrading from 1.0.0: some recordings can turn red.** Checks are
120
+ stricter, so a recording 1.0.0 passed can fail after `npm update`, with
121
+ nothing changed in the application. It never turns a failing check green.
122
+ What changed:
123
+
124
+ - A role's name matches exactly (a partial name needs an explicit name
125
+ regex), attribute patterns are case-sensitive, and an input-value check
126
+ needs a visible input.
127
+ - A text or input-value check records an explicit expected value. Expected
128
+ text, a text pattern, a download check, an expected URL, or a target's
129
+ text, label or placeholder that a blank Examples or data value leaves
130
+ partly empty fails instead of matching more than it means.
131
+ - A response check counts only requests started since the latest browser
132
+ action, so an earlier successful save no longer satisfies it.
133
+ - An Outline recording that cannot follow another row (short values,
134
+ dates without a year, a role taken from a row, an unscoped action,
135
+ check or capture in a section a column names, a value typed in a
136
+ column's browser session, a step naming a column that recorded no action
137
+ writing it, a state check naming a column whose value it does not show,
138
+ as in `Then <user> sees the dashboard`) refuses that row with guidance
139
+ instead of passing on row 1's data. Login Outlines of that shape now
140
+ pass only on the recorded row: write the user in the text the check
141
+ expects (`Then the dashboard greets "<user>"`) and record again.
142
+
143
+ Re-record what fails (`saffron run --rerecord <file>:<line>`) or correct
144
+ the step the message names. Recordings 1.0.1 writes with its new features
145
+ (localized dates, several downloads per click, the new row bindings) are
146
+ refused by 1.0.0 by name, so a team updates together.
147
+
148
+ Agent plugins
149
+
150
+ - Declare documentation, privacy, license terms and support links for the
151
+ Claude directory listing, with explicit README links as a fallback.
152
+
153
+ - Prepare the agent plugin listings with the Saffron icon and documented
154
+ data handling. The Codex listing adds a shorter subtitle, local
155
+ execution requirements, prerequisites and links to support, data handling
156
+ and the software license.
157
+
158
+ Runner
159
+
160
+ - A row refused because its check only shows that something is visible or
161
+ in a state (`Then <user> sees the dashboard`) now says so, and suggests
162
+ writing the value in the text the check expects
163
+ (`Then the dashboard greets "<user>"`).
164
+ - A fallback selector's text (`:has-text("Order <id>")`, `text=Order
165
+ <id>`) finds row 1 only as a whole number or word, not inside "Order
166
+ 12", and a blank row value at its edge is refused, as in other checks.
167
+ - A pattern that joins letters or digits to a row value itself
168
+ (`^<prefix>\d+$`) keeps them joined at replay.
169
+ - A secret shown lower-cased and percent-encoded in a URL is masked, and
170
+ so is a number secret shown with spaces or dashes between its digits.
171
+ - A day a step writes as plain `{date+1}` and the app shows in its own
172
+ format (`21/10/2026`, `Oct 21, 2026`) is recorded as the token in that
173
+ format, so the check moves with the calendar.
174
+ - A quoted CSV field keeps its own spaces. A double-quoted `.env` value
175
+ takes `\"` and `\\` escapes. A count in parentheses ("Cart (3)") is no
176
+ longer read as a negative amount.
177
+
178
+ - Mask mixed-case percent escapes in capture names, reports and traces
179
+ through the shared secret masker.
180
+ - Keep state checks bound to their recorded row even when the assertion
181
+ step also presses a key or performs another browser action.
182
+ - Match row values with Unicode-aware regex boundaries, including selector
183
+ fallbacks and downloads. Non-ASCII suffixes cannot satisfy a shorter value.
184
+ - Refuse tokens inside regex repeat counts instead of silently changing
185
+ the assertion. Use a separate Scenario with a literal repeat count.
186
+
187
+ - Mask URL-encoded secrets with lowercase or mixed-case percent escapes,
188
+ including escaped patterns, before saving recordings.
189
+ - Bind state checks to their recorded Outline row when the check does not
190
+ follow its column. A column at the start of the sentence no longer
191
+ bypasses this safeguard.
192
+ - Apply row-value boundaries to selector regex fallbacks and download
193
+ checks. Preserve character-class syntax when resolving regex tokens.
194
+ - Classify Given and inherited setup steps using the feature's language,
195
+ including Latvian keywords shared with other languages.
196
+
197
+ - Rendered-text capture tests allow time for capture stability and browser
198
+ round trips on CI. Hidden and empty text must still fail within the same
199
+ short test timeout.
200
+ - Runner CI installs WebKit alongside Chromium and Firefox so the existing
201
+ tests for all three browser engines can run on GitHub Actions.
202
+
203
+ - Review fixes: a provider or agent error is masked for secrets in full
204
+ before it is cut to its first line, so a secret spanning lines no
205
+ longer shows in the console or a later scenario's error. A typed value
206
+ submitted with Enter presses it on the field typed into, not whatever
207
+ has focus after an input handler ran. A date token with a locale tag
208
+ Node.js rejects (`en-x`) is marked in the editor instead of stopping the
209
+ diagnostics. A report row's `ranAt` is validated like the run's times,
210
+ and the HTML page is rendered before either report file is replaced.
211
+ A quoted value containing a word like "clear" no longer hides a lost
212
+ typed value. An amount with its minus before the currency (`-$5.00`)
213
+ reads as negative, so doubling it is never checked against $10.00. A
214
+ capture named like a token (`price {date}`) keeps its name in an amount
215
+ check at replay. A click that starts several downloads stores the count
216
+ as a number of its own, which no Outline pass rewrites, and the
217
+ recording is written as version 4, so an earlier runner refuses it
218
+ instead of checking the first file. Recording judges a download check
219
+ by replay's blank rule and resolves `{unique:...}` inside Examples
220
+ values, as replay does.
221
+
222
+ - Fourth review round. Outlines: a step that names a column but recorded
223
+ nothing that writes it (the page already showed "myself", or a box was
224
+ ticked) binds the recording to its row, so the "someone else" row is
225
+ refused instead of replaying row 1's page; a recording made before this
226
+ is recorded again once on a row that differs, and the agent is asked to
227
+ pick the row's value even where the page already shows it. A blank
228
+ first row no longer refuses every filled row: a filled row records once
229
+ more, and that recording serves the blank rows. A typed value equal to
230
+ the section's name no longer passes for scoping where the step says it
231
+ is a section (`in the "Double" room card`); captures and an assertion
232
+ step's checks must name the section too, and `the country selector in
233
+ "<section>"` and a hyphenated container word (`time-selection dialog`)
234
+ read as sections. A dialog's button the scoped action opened, scoped to
235
+ the dialog, and a wait, need not name the section. Literal braces in a
236
+ typed value (a JSON address) are text in a person's browser session, and
237
+ a short `{data:...}` value the step writes is the step's own. A date cell
238
+ written with its month's name (`{date+1:MMM D}`) follows the row in any
239
+ step, instead of being recorded again every day. Recording: performing
240
+ several flagged actions again replaces each in its own place, one by
241
+ one, and one not performed again stays where it was. A name Playwright's
242
+ MCP shortened (`'26. Oktober'` for "26. Oktober 2026", `'Room'` for
243
+ "Room 101") is recorded in full from the snapshot the agent clicked in,
244
+ as replay matches names exactly. Through the trace bridge, no page
245
+ fingerprint is taken before a browser action, which had made every
246
+ click by snapshot ref fail. Replay: a captured value is taken once the
247
+ page has stopped changing it for half a second (sooner for a short
248
+ timeout), so a total drawn again after the last change is not read
249
+ stale. Proposals: a proposal git checked out with CRLF line ends is
250
+ still the one this machine filed. Review: a proposal whose only change
251
+ is beside the steps (a row binding, the row it was recorded on, a data
252
+ value) says so in `saffron diff` and the report. `saffron status` judges
253
+ an Outline row the recording refuses as refused, not stale. A CI run's
254
+ results say when the run was interrupted. Step reuse records afresh
255
+ instead of keeping a capture under another value's name, and a fix
256
+ carried to other scenarios never copies a `<column>` into one without
257
+ it. The editor checks date tokens in doc strings too, never on a
258
+ comment line.
259
+
260
+ - Fifth review round. A step naming a column is judged by its own
261
+ actions: another step writing the column, or another column sharing its
262
+ value, no longer lets an empty step pass on other rows. Step reuse never
263
+ puts such a step back for a row whose value differs, so the row records
264
+ it afresh. A check on a bare role (`the error message in "<language>"`)
265
+ takes no section from the quote, and a wait in a section step never
266
+ binds the row. `saffron status` marks a refused row with ✗ and says why,
267
+ instead of calling its recording unreadable and counting it as replaying.
268
+
269
+ - Sixth review round. Recording: a full button name is read only from the
270
+ page snapshot a browser call shows, never from a response body or the
271
+ console. Corrections a step performs again, one by one or in one call,
272
+ each take the place of the flagged action of their kind and element, so
273
+ a fill stays before the Save after it; one that reopens a dialog
274
+ replaces the click that first opened it instead of clicking twice. A
275
+ check spelling its own value (`Then ... "Guest: Tor"`) keeps it, and no
276
+ longer reads another step's table cell. A value typed from
277
+ `{data:users.<who>.email}` is recorded as that reference, so an edited
278
+ data file reaches replay. `the "<room>" room card should show ...` is a
279
+ section without "in". Replay: a capture waits for the page's fetch and
280
+ XHR requests to be answered (up to 3 seconds) before reading.
281
+
282
+ - Seventh review round. One value two references in a step read
283
+ (shipping and billing both "Home") stays as typed and binds the row,
284
+ instead of becoming one of them; in different steps, each keeps its own
285
+ reference. An alert or other bare-role check inside a section the step
286
+ names explicitly must be scoped there. Step reuse keeps out a `When`
287
+ step that checks something but sets nothing of its column. A dialog
288
+ reopened while correcting a step replaces the click that first opened
289
+ it, across the waits between them, and a wait performed again is filed
290
+ once. Only the very same action counts as reopening (the same target in
291
+ the same scope, with the same click options): a click focusing a field
292
+ never replaces its fill, nor a click on a checkbox of the same name. A
293
+ short shared value ("NY"), a blank one, or one written escaped in a
294
+ pattern or selector, binds the row like any other. A `{data:...<column>...}`
295
+ value blank on the recorded row records once more on a filled row,
296
+ as a blank cell does, and one kept in another spelling (`ship-home`)
297
+ binds the row. A recording made before these bindings that still holds
298
+ such a value literally records once more on a row reading another. A
299
+ later check still holding a value another step's reference read (a
300
+ summary showing "Home") binds the row too, and a step that reads a
301
+ value and also spells it as its own constant (`Shipping "{data:...}"
302
+ and Billing "Home"`) binds the row, as which field was which cannot be
303
+ told. A short value ("M", "NY") counts only as a whole word, never
304
+ inside "My cart", and one inside the step's own text ("Shipping to NY")
305
+ becomes its reference. A recording made before this that typed the
306
+ reference for the step's own constant records once more. A value the
307
+ step spells inside a longer word ("Johnson" beside a first name
308
+ "John") binds the row too, and step reuse never puts back a check still
309
+ holding a value a reference read on a row that differs.
310
+
311
+ - Twelfth review round. Recording: a data value never rewrites a
312
+ `<column>` placeholder or a `{data:...}` reference's path ("email" in
313
+ `<email>`), and a tag in a selector (`button`) is never data. A value a
314
+ step spells itself stays its own when another step reads it ("Johnson"
315
+ beside a first name "John"). A check naming a column nothing in the
316
+ recording follows (`Then the banner is <state>`, recorded as "not
317
+ visible") binds the row. A role, an amount written another way
318
+ ("12,50" checked as 12.50) or another spelling (`address-home-street`)
319
+ of a `{data:...<column>...}` value binds the row too, and a password
320
+ read through `{data:users.<who>.password}` from `{env:...}` follows the
321
+ row instead of keeping the first row's variable. Step reuse: a `When`
322
+ recording never seeds a `Then` of the same words, and a check holding a
323
+ value a joined key (`zone-<s>`) read, or expecting nothing for a blank
324
+ one, is never put back, also from a recording that names no row.
325
+ Replay: a `{date...}` inside a data value resolves to its day, as it
326
+ did while recording. The review that followed: secrets are masked
327
+ before any other pass, so a row, table or data value inside one never
328
+ cuts it up and leaves the rest in the recording. A state check ("not
329
+ visible", "disabled") naming a column binds the row even where another
330
+ step sets that state, and so does a check showing a date column's day
331
+ in the page's own format; a check stating its own text may name a
332
+ column to describe the row. Another step's value in an action step
333
+ ("Ship to NY") binds the row, a session's column (`As "<who>"`) is
334
+ never cut out of a longer word ("Hannah"), a data value holding a
335
+ formatted date becomes its reference, and a constant spelled inside a
336
+ longer word stays however often the step names a reference. A changed
337
+ data file is noticed where a check holds the old value as a role or an
338
+ amount. Step reuse refuses joined keys without a separator
339
+ (`zone<s>`) and empty checks in any spelling (`^(?:)$`), and a
340
+ recording whose data holds a date is version 3, which saffron-ai 1.0.0
341
+ refuses rather than typing `{date+1}`.
342
+
343
+ - Thirteenth review round. Outlines: a row value inside a run of letters
344
+ ("red" in "Delivered", "Pro" in "Products") no longer binds the row; a
345
+ camelCase part ("SelectedRed") still does. Rows whose data reads blank
346
+ on every row no longer record each other forever. A row value holding a
347
+ secret's value ("admin@shop.test" beside a variable "admin") is the
348
+ row's `<column>`, and a secret equal to a row's lookup key leaves the
349
+ step's `{data:...}` reference alone. A state check ("Promo" hidden, an
350
+ attribute's pattern) binds the row whatever its element is named. A
351
+ dialog answered after announcing the next step is armed before its
352
+ trigger, in the trigger's step. The fingerprint a row binding keeps of
353
+ a secret is salted and slow, so a short one is not read back from the
354
+ recording (version 3); older plain ones still match. A blank Examples
355
+ cell beside a step with something to check is version 3, and step reuse
356
+ refuses a one-character joined key (`zone0`). An older recording is
357
+ recorded again once a data value it reads holds a date. Replay: a row
358
+ value starting or ending an expected text, URL, response URL or text
359
+ target is never found inside a longer word or number ("Order 1" never
360
+ passes on "Order 12"). A pending proposal that fails to replay does so
361
+ in a browser of its own, so the agent records in a clean one. CLI:
362
+ `saffron prune --yes` names what it could not delete and exits 1; a
363
+ hook killed by a signal says so; `.env` reads `export` lines and drops
364
+ inline comments; "$.50" is 0.50. The language server runs a scenario
365
+ sharing its name with another by line, reads `SAFFRON_ENV` from the
366
+ project's `.env`, and checks date tokens only in steps, tables and doc
367
+ strings. Then: a check that an element found by its text is absent
368
+ ("Promo" not visible) binds the row like any state check; a role with
369
+ its text never finds a row value inside a longer one either; re-masking
370
+ a secret never cuts into a `<column>` it sits in ("admin" in
371
+ `<adminEmail>`); and "-$.50" and "($.50)" are negative. The review after
372
+ that: a secret percent-encoded in a recorded URL (`admin%40shop.test`)
373
+ is masked too, and a URL check also reads the page's URL decoded;
374
+ another step's data value shown in another case or as a slug ("MUG")
375
+ binds the row; two columns holding one `{env:...}` password record once
376
+ more for a row typing them apart instead of being refused for good; a
377
+ column opening a Then step that another step writes (`Then <user> sees
378
+ the dashboard`) describes the row; a three-letter data value inside
379
+ text ("1 x Mug") follows its reference. A response URL pattern bounds a
380
+ row value wherever it stands, `saffron status` judges every Examples row
381
+ and finds an install in a parent folder, a pending proposal failing
382
+ for another row keeps its screenshot and trace, a quoted `.env` value
383
+ followed by a comment loses its quotes, and an empty Given in any
384
+ language is not sent back. The full review after that: a secret
385
+ percent-encoded in a pattern, written in another case, or in an
386
+ `encodeURI` URL is masked; a data value that is a variable follows its
387
+ reference inside text too; an amount a text check shows in another
388
+ format, a state check found by its text (visible or not), and a number
389
+ only inside a longer one are each judged as they should be; a column
390
+ opening a check is its subject only when another step writes it and the
391
+ check does not state it (`<state> is ...`); a multi-word value in a URL
392
+ (`q=red+shoes`) follows the row. Replay bounds a row value in every
393
+ pattern (text, name, attribute, response URL and body) and refuses a
394
+ blank one left inside an attribute or body pattern, reads response URLs
395
+ decoded, and keeps a regex's own named groups. A fresh recording that
396
+ never announced its last steps is sent back instead of filed as
397
+ verified. A quoted CSV field after a space is read whole, and a row with
398
+ more fields than its header is refused. `.env`: the last of a repeated
399
+ key wins, quotes match non-greedily, and inherited names are ordinary
400
+ keys; neither data nor rows read inherited properties. `saffron status`
401
+ counts unreadable recordings apart from stale ones, `saffron trace`
402
+ prints report names without terminal escapes, and a run without the
403
+ agent says so. A click performed twice on purpose while correcting a
404
+ step is kept twice; a reopened download trigger takes its download wait
405
+ with it, so the count is not doubled; and an armed action goes before
406
+ its trigger even when the same call waited for something after it.
407
+
408
+ - An Outline whose browser session comes from a column (`As "<who>" ...`)
409
+ no longer lets another row's person type the recorded person's value: a
410
+ step in that session typing a value the step does not write binds the
411
+ recording to its row (written as version 3), so the other rows are
412
+ refused with guidance instead of passing on the first person's data.
413
+
414
+ - Third review round. Recording: each action and check in a section a
415
+ column names must be scoped to it (one scoped action no longer covers an
416
+ unscoped fill beside it), a quoted value the step types is no section,
417
+ and performing a flagged action again replaces only that action, in its
418
+ place, with its download wait. A download is counted from the browser's
419
+ own events only, never twice. An amount the step writes itself stays the
420
+ step's, a column two written amounts match stays literal, a date cell's
421
+ day stays literal only in the steps that name its column, tokens that
422
+ always read alike (`{date+7:dddd}`, `{date+14:dddd}`) are one, and "an
423
+ empty" field reads as nothing to type. Format: a recording made before
424
+ a new-syntax date token was added to its scenario is recorded again
425
+ once as version 3, a "container" row binding is version 3, and
426
+ `rowsUseNewDates` is part of what a proposal binds. Replay: a download
427
+ check, an expected URL, or a target's text, label or placeholder that a
428
+ blank row value leaves partly empty fails instead of matching more; a
429
+ newer dialog answer replaces one no dialog used; a targeted upload that
430
+ starts a download is waited for; a retry sets the chosen files again;
431
+ an amount check's quoted names are never resolved. Runs: `--rerecord`
432
+ refuses, deleting nothing, when no browser of the run can record or the
433
+ agent cannot start; an offset time zone is refused; "retry in 41.8s"
434
+ is a passing throttle; Claude overage use is no limit.
435
+
436
+ - Second review round. Recording: a section a column names that the
437
+ recording ignores binds the row, so another row is refused rather than
438
+ acting in row 1's section; a place or manner (`in "Oslo"`) is no longer
439
+ taken for a section, and a `Given` the page already satisfies is not
440
+ asked for an action. Amounts: accounting, trailing and en-dash negatives,
441
+ decimals such as `0.125`, and an amount written in another format than its
442
+ Examples cell follow the row; a misplaced quote in an expression is
443
+ refused. Dates: a day the recorded row's cell holds stays literal, two
444
+ tokens that read alike stay literal, only whole dates are made tokens
445
+ outside their own step, and an Outline whose other rows use the new date
446
+ syntax is written as version 3. `{date-of-birth}` is typed as written,
447
+ offsets past a hundred years and locale-like typos are named, and a time
448
+ zone must be spelled as browsers spell it. Downloads: one started in a
449
+ popup tab or after its click returned is credited to that click, one
450
+ reported by a later result raises the count, opening a file's URL and a
451
+ chosen file that starts a download replay, and a response body is never
452
+ read as actions. Runs: a throttle that clears in seconds does not stop
453
+ AI sessions, "Quota exceeded" and a Claude subscription limit do, and
454
+ Ctrl-C before any scenario finished keeps the last report.
455
+
456
+ - **Amounts that add up.** A step such as `Then the basket total should be
457
+ "before" plus "breakfast"` records an `expectAmount` check: the target
458
+ must show the amount an expression over captured values states
459
+ (`"before" + "breakfast"`, `"total" - "discount"`, `"night" * 2`),
460
+ compared to the cent. Amounts are read in either decimal style
461
+ (`NOK 1 234,50`, `$1,234.56`, `CHF 1'234.50`); an element showing two
462
+ numbers is refused rather than guessed. A total that changed by the
463
+ wrong amount now fails, where a "differs" check passed. The agent
464
+ records it with `saffron_assert` kind `amount`, which refuses a name no
465
+ step captures. Capture names in the expression are identifiers: no
466
+ Outline, data, date or secret pass rewrites them, and a secret-bearing
467
+ one is renamed together with its capture.
468
+
469
+ - Agents are told that other scenarios reuse only accepted recordings:
470
+ the run's "proposals pending" line, the `project_status` MCP tool and
471
+ the bundled skill say to accept a shared setup before recording more
472
+ scenarios that use it, instead of paying for it in every one. The skill
473
+ also asks for a mid-scenario `Then` at each checkpoint of a route (a 3DS
474
+ challenge, a redirect), the exact control a variant uses, and the
475
+ section a step acts in when labels repeat.
476
+
477
+ - Fixes from a field report on recording a checkout suite with Codex.
478
+ `saffron_done` refuses a pass once while an action step recorded no
479
+ browser action: announcing a step performs nothing, and a step's action
480
+ can land under the next step. A step that acts in a named container
481
+ (`in "Your contact information"`) with a target that ignores it is sent
482
+ back too, since replay acts on the first match; performing it again
483
+ replaces the unscoped action. The step reply and the agent's
484
+ instructions say that announcing performs nothing, ask readiness steps
485
+ to record the condition they wait for, and ask for a fresh snapshot when
486
+ a ref no longer resolves. A second pass stands, so a step that needs
487
+ nothing still finishes.
488
+
489
+ - The lost-value check no longer flags a step that clears a field or
490
+ passes `""`: a quoted field name is not a typed value. An empty fill on a
491
+ step that types a value is still flagged. A blank first Examples row
492
+ binds to `<column>`, and an expected message held in a row's
493
+ `{data:...}` token follows each row.
494
+
495
+ - `--rerecord` records afresh over a pending proposal from an earlier
496
+ run, not only a row-bound one, instead of replaying it at zero AI. Rows
497
+ of the same run share the new recording.
498
+
499
+ - When an agent session fails on its provider's usage limit, no further
500
+ agent session starts in that run, on any worker (the agent describing
501
+ such a message in the application does not count, and the message is
502
+ stored with the scenario's secrets masked): the scenarios that need one
503
+ fail at once with the provider's message, and replays carry on. A
504
+ session in which the agent made no tool call at all says its tools did
505
+ not reach it.
506
+
507
+ - The run summary counts pending proposals whose proof replay failed
508
+ apart from the others, and the report splits them into proof passed,
509
+ proof failed and not proved. A run of named scenarios marks each report
510
+ row kept from an earlier run with when that run started; shards of one
511
+ run are not marked. A pending proposal that does not replay for another Examples
512
+ row names the columns that row types differently, and which of them the
513
+ failing step uses, instead of guessing at the cause.
514
+
515
+ - Typing with submission checks Enter in its own response window, so a
516
+ successful autosave cannot hide a failed submission during healing.
517
+ - Completed form fields and typing stay in the recording if a later field
518
+ or submission fails, including when the agent retries only the failed part.
519
+ Compound calls are validated before their first edit.
520
+ - Agent shutdown allows five seconds for started calls to finish recording,
521
+ then cancels stuck work and fails the incomplete recording with a clear
522
+ timeout. Queued calls stay blocked and provider usage is retained.
523
+
524
+ - Healing checks each field of a multi-field form against its own response
525
+ window, matching replay. The full batch is validated before any field
526
+ changes. Capture tracking preserves named regex groups when step parameters
527
+ change and prefers a unique test ID over app-generated DOM IDs for remounts.
528
+
529
+ - Healing response assertions follow the latest agent interaction in their
530
+ browser session, so an earlier successful retry cannot hide a later failure.
531
+ Agent shutdown waits for started browser calls to finish recording before
532
+ closing their clients or finalizing the failed session; queued calls stay
533
+ blocked.
534
+
535
+ - Codex recording and healing enable the code-mode tool host while retaining
536
+ Saffron's isolated browser tools. Tool-call limits are enforced at the
537
+ gateway, including calls nested in code-mode batches, before an excess
538
+ call can reach the browser. No dependency patch is needed.
539
+
540
+ - Text patterns reject partly blank row, data and environment values, including
541
+ nested tokens. The healing assertion guard retains captured element and
542
+ frame identity, so another frame cannot stand in for a removed target.
543
+
544
+ - Follow-up assertion fixes: redirects retain the original request's action
545
+ window, and partly blank text is refused even when the blank comes through
546
+ an Examples cell or data file. Live comparisons retain the captured target
547
+ and frame when that target becomes hidden or disappears. A unique ID or
548
+ test ID allows ordinary DOM replacement in the same frame; duplicate
549
+ matches remain tied to the original element.
550
+
551
+ - Response assertions check requests started since the latest browser action,
552
+ so an earlier successful save cannot hide a later failed save, even when
553
+ the earlier response arrives late. Waits and consecutive assertions keep
554
+ that window open. Check a request before the next interaction or navigation.
555
+ - Assertions use exact role names and case-sensitive attribute patterns;
556
+ input value checks require a visible input. Use an explicit name regex
557
+ for a partial role name. Text expectations containing a blank row or data
558
+ value alongside other text now fail with guidance. Recording a text or
559
+ input-value assertion requires an explicit expected value, including `""`
560
+ for an intentional empty check. Existing recordings relying on these
561
+ looser checks must be corrected or re-recorded.
562
+
563
+ - Outline recordings refuse other rows when they retain short values in
564
+ action steps, dates without a year or glued to digits, an unparameterized
565
+ blank cell, or an element role taken from the recorded row. A whole
566
+ `{env:...}` or `{data:...}` field equal to a row's value is treated as
567
+ ambiguous even when the step names that token. These cases fail with
568
+ guidance instead of passing on another row's data or repeatedly recording.
569
+ - Step reuse refuses changes to quoted arguments containing several
570
+ placeholders, where it cannot safely separate their values. Row comparisons
571
+ keep date tokens inside environment values literal, matching replay.
572
+
573
+ - Setup hooks can create or refresh files used by `{data:...}` steps. Data
574
+ and referenced environment variables are validated after setup, and replay
575
+ reads the refreshed values. Invalid data stops the run before a browser
576
+ opens and still runs teardown.
577
+
578
+ - Ctrl-C during `saffron run` writes the report of what finished, with or
579
+ without setup and teardown configured, instead of leaving the last run's
580
+ report in place. The report has `"interrupted": true` and `notRun`, the
581
+ scenarios the run selected that did not finish, which count as neither
582
+ passed nor failed; the HTML report, the run's output and `saffron status`
583
+ say it is partial, and its history line is marked partial. The browsers
584
+ close and teardown runs first, then Saffron exits with code 130.
585
+ Playwright's own Ctrl-C handler is off, so it no longer exits before
586
+ teardown has run. `saffron report --merge` refuses an interrupted shard,
587
+ and a run of named scenarios replaces an interrupted report whole.
588
+
589
+ - The run's first line and the report's `provenance` say what produced
590
+ it: the provider, each model asked for and whether it came from
591
+ `--model`, `--heal-model` or saffron.config.json, and the installed
592
+ version of the vendor package (`@openai/codex-sdk`,
593
+ `@anthropic-ai/claude-agent-sdk`, `@cursor/sdk`) beside saffron-ai's. A
594
+ model nobody asked for is left unnamed; the models that answered stay in
595
+ `totals.models`.
596
+
597
+ - **Dates as the app shows them.** A `{date...}` format writes month and
598
+ weekday names (`MMMM`, `MMM`, `dddd`, `ddd`) and two-digit years
599
+ (`YY`), in the locale the token names (`{date+7:D. MMM:de-DE}` is "24.
600
+ Okt.") or in `dateLocale` from `saffron.config.json` (en-US when
601
+ absent). A range is one token, `{date+7 - +9:MMM D, YYYY}`: what is
602
+ written between the offsets is written between the days, and what both
603
+ share is written once ("Oct 24 - 26, 2026", "Oct 30 - Nov 1, 2026",
604
+ "Dec 31, 2026 - Jan 2, 2027"). A suite no longer needs a setup script to
605
+ compute the stay it picks and the range it checks.
606
+
607
+ - **One clock.** `timezone` in `saffron.config.json` sets the calendar
608
+ date tokens count days on and the time zone every browser runs in, so
609
+ the page and the tokens agree on today, around midnight and through
610
+ clock changes. Every token of one replayed scenario resolves on the
611
+ clock the scenario started with.
612
+
613
+ - The agent is given each date of a scenario resolved, on the clock its
614
+ recording started with, instead of the token; the recording turns each
615
+ of those days back into its token, so a date written in a step's text in
616
+ any format moves with the calendar, not only an ISO one.
617
+
618
+ - A `{date...}` token Saffron cannot read, or with a locale this Node.js
619
+ does not know, stops the run before any browser opens, naming it and the
620
+ scenario; the language server marks it in `.feature` and `.saffron`
621
+ files. The TextMate grammar colours ranges.
622
+
623
+ - A recording with a range, a locale, a name or a two-digit year in a date
624
+ token is written as version 3 of the recording format, so saffron-ai
625
+ 1.0.0, which would resolve it wrong, refuses it by name.
626
+
627
+ - **What a download says.** A booking confirmation's "Add to Apple
628
+ Calendar" link could only be checked by its `href` and file name, never
629
+ by the dates and identifiers inside the file. Now the click that
630
+ downloads a file records with the download armed before it
631
+ (`waitForDownload`, as a dialog is), and replay waits for that
632
+ download, fails the click when none starts, and keeps the file in
633
+ `.saffron/artifacts/<feature>/<scenario>/downloads/`. A `Then` step
634
+ such as `Then the calendar file should contain
635
+ "DTSTART;VALUE=DATE:{date+7:YYYYMMDD}"` records a content check
636
+ (`expectDownload`): text the scenario's latest download contains, a
637
+ regular expression it matches (case-sensitive, `^` and `$` at each
638
+ line), or both, with `{date...}`, `{data:...}`, `{env:...}` and
639
+ Examples values resolved.
640
+ Line breaks read as `\n`, a calendar or contact file's folded lines
641
+ read whole, and only UTF-8 text files up to 5 MB are checked. The check
642
+ is never healed. While recording, Saffron checks it against the file
643
+ the agent downloaded (`saffron_assert` kind `download`) and shows the
644
+ agent the start of the file when it does not hold. A click that
645
+ downloads a file is no longer lost from a recording made without
646
+ tracing: Playwright's MCP, attached over CDP, could not copy the file
647
+ and reported the click as failed, so nothing was recorded. Saffron now
648
+ keeps the file through its own connection and drops that one error.
649
+ A large page no longer hides a download from the recorder either: a
650
+ tool result cut to fit the agent's context keeps the sections the
651
+ recorder reads (its code, tabs and Events) whole, even where the cut
652
+ would split one, and the parts of a compound call share one allowance.
653
+ A click that starts several downloads records how many; replay waits for
654
+ that many and checks the latest, the file the recording checked. While
655
+ recording, a download check resolves `{env:...}`, `{data:...}` and
656
+ `{unique:...}` values, dates on the recording's clock, and values in a
657
+ pattern escaped, exactly as replay does.
658
+ A recording with a download needs this release; an older runner
659
+ refuses it by name.
660
+
8
661
  ## 1.0.0: Several people in one scenario
9
662
 
10
663
  Released 2026-10-14.