saffron-ai 1.3.0 → 1.3.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,794 @@ the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
5
5
  JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
6
6
  release heading names its main feature: `## <version>: <title>`.
7
7
 
8
+ ## 1.3.2: Exact checks for amounts, URLs and numbers, and secrets kept masked
9
+
10
+ Released 2026-11-04.
11
+
12
+ Runner
13
+
14
+ Upgrading from 1.3.1:
15
+
16
+ - A URL check (`expectUrl`, contains) reads its text as a whole part of
17
+ the URL, and a path (text with a `/`) outside the query, unless the text
18
+ names a query (`?`, `=`, `&`): `/dashboard` no longer holds on
19
+ `/login?next=%2Fdashboard`, nor `/orders/42` on `/orders/421`. A word
20
+ with no slash (`shoes`) is still found in `/search?q=red+shoes`, but not
21
+ in a query value that is a path (`dashboard` in `/login?next=/dashboard`),
22
+ nor in a query value whose encoded `#` or `&` makes it look like a
23
+ fragment or another field (`/dashboard` in `?next=%2F%23%2Fdashboard`).
24
+ A check that the URL does not hold a text still reads it anywhere:
25
+ `error` is in `?errorCode=5`. A text that names a query is the page's
26
+ own: its `?` is where the URL's query starts and a field it names is
27
+ one of the URL's own, so `/dashboard?tab=settings` and `tab=settings`
28
+ no longer hold on `/login?next=%2Fdashboard%3Ftab%3Dsettings`, nor
29
+ `/dashboard` on an OAuth callback's `#state=%2Fdashboard`. A fragment's
30
+ fields are the page's own too: `#tab=settings`, `/app#tab=settings` and
31
+ `tab=settings` hold on `/app#tab=settings`. A recording
32
+ that passed only that way turns red; the exported test checks the same.
33
+ - `Examples: {data: roles}` (a space, a capital `Data`) is an error naming
34
+ the file and line, where the Outline used to run no row at all. The
35
+ language server marks it while it is typed.
36
+ - An expected text never matches inside a longer word or number, whether
37
+ a row value or written as it is (SAF-197): `"$10"` no longer holds on
38
+ `$100`, `"1 item"` on `11 items`, nor a text target `Saved` on "You have
39
+ unsaved changes". A role's name may still have grown a suffix. A number
40
+ written in the step may be followed by a zero decimal part (`"$10"` in
41
+ `$10.00`, never `$10.50`, `₹1,00,000` or `2.0.1`); a row value keeps its
42
+ decimals whole (`<id>` 1 is not in `1.00`), and neither is found before
43
+ a digit in an inline tag (`Order 1<b>2</b>`, `$10<sup>.50</sup>`). A
44
+ text ending in a number's point or comma takes no more digits
45
+ (`"Refund of $10."` is not in `Refund of $10.50`), and a number is not
46
+ found in its negative (`"5.00"` is not in `-5.00` or `−5.00`; `"5"` is
47
+ still in `pages 3-5`). A value that starts or ends in Chinese, Japanese,
48
+ Thai, Lao, Khmer or Myanmar script is not bounded there, as those
49
+ scripts put no space between words, but for a negating prefix:
50
+ `"成功"` is not in `不成功` nor `没有成功`, `"保存"` not in `无法保存`.
51
+ Only a negation right before the value is read: a short value in a
52
+ sentence that negates it further back can still be found. A recording that passed only that way turns red, and the agent
53
+ records the whole word next time.
54
+ - An expected text never runs across a block, a table cell or a `<br>`
55
+ (SAF-198): `"15.00"` no longer holds on the cells `1` and `5.00`, nor
56
+ `"Qty 10"` on the items `Qty 1` and `0 left in stock`. Items of a flex
57
+ row or a grid read as one line, a space apart: `"Total: $10"` holds on
58
+ the items `Total:` and `$10`. A value written on several lines still
59
+ matches the lines the page draws.
60
+ - A style or attribute check needs its element shown (SAF-199): "the
61
+ error is shown in red" no longer holds on an error that is not on
62
+ screen, nor on a shown fallback when the recorded element is on the page
63
+ but hidden. A check of what hides an element (`display`, `visibility`,
64
+ `opacity`, `content-visibility`; `hidden`, `aria-hidden`, `inert`) still
65
+ holds on a hidden one, and so does one of a `class`, a `style`, `open`
66
+ or `data-state` attribute, or a height, width, transform or clip: how
67
+ many apps hide an element (`d-none`, a closed dialog).
68
+
69
+ The exported Playwright tests check text, blocks and styles the same way
70
+ (SAF-197, SAF-198, SAF-199).
71
+
72
+ - A text check of an amount written with its currency (`$5.00`, `€5`)
73
+ no longer holds on the negative amount (`-$5.00`, `−€5`), as `5.00`
74
+ already did not on `-5.00`. A URL check of a number takes no decimal
75
+ part (`amount=10` is not in `?amount=10.50`, nor `/orders/42` in
76
+ `/orders/42.5`), and a letter or digit the URL writes percent-encoded
77
+ belongs to the word beside it (`name=Jos` is not in `?name=Jos%C3%A9`,
78
+ nor `id=42` in `?id=42%35`). The exported test checks the same.
79
+
80
+ Fixed, from a review of the runner:
81
+
82
+ - A credential stays masked wherever it travels: a fixture's token that a
83
+ later capture replaced, or one inside an extracted object, in the
84
+ scenarios the fixture seeds; and a replaced token in what the agent
85
+ reads while recording, which it used to show. An exported test masks a
86
+ secret as a URL encodes it (`%2B`, `%2F`, `%3D`) and in the errors of
87
+ its JSON checks, as replay does.
88
+ - A `differs` check on an object holding an integer beyond JavaScript's
89
+ precision no longer passes against the same object a capture kept.
90
+ - The dashboard's suite health no longer reads 100% while a scenario
91
+ failed (473 of 475 reads 99.58%).
92
+ - The language server no longer marks `Examples: {data: roles}` as an
93
+ error inside a doc string, nor when a description comes before its
94
+ table: the runner reads both.
95
+
96
+ - No false green from a fallback selector or an iframe: the first locator
97
+ that finds a shown element decides a check, so a fallback that happens
98
+ to show the expected text no longer vouches for the recorded element
99
+ that failed it. A recorded element that is gone, or hidden for a check
100
+ other than a style or attribute check, still gives way to its fallback.
101
+ - No false green from an API check whose pattern matches any text: a
102
+ field, header or `until` pattern such as `.*`, `^` or an empty one is
103
+ refused while recording, as page checks already were.
104
+ - No Outline row passes on another row's data when the app writes a row
105
+ value without its accents (`José` as `status-jose`), or when a column
106
+ names a section without quotes (`the <plan> card shows "Popular"`).
107
+ - A captured id of one or two characters is recorded as its token where
108
+ a path segment or field names its capture (`/orders/42` for `order`), so
109
+ replay no longer reads the record made while recording. A field that
110
+ names another id (`productId: 1`) is left as it is.
111
+ - A secret in an API query that holds `/` together with `+`, `=` or `&`
112
+ (a base64 key) is masked in errors, transcripts, reports and the
113
+ agent's replies.
114
+ - A page or API message cut for a report is cut with `…`, so a secret
115
+ across the cut is masked too, also when the message carries it
116
+ URL-encoded.
117
+ - A request method taken from a column (`a <verb> request`) is kept as
118
+ sent, which binds the recording to its row, instead of an invalid
119
+ recording made again on every run.
120
+ - An integer above 2^53 extracted from a response (a snowflake id) keeps
121
+ the digits the API sent, instead of being rounded to another id. A JSON
122
+ check reads the same digits, so "no longer the same id" fails on the
123
+ same id, and a request naming the captured id still sends its token.
124
+ - A differs check finds the element again when the app draws it anew
125
+ (a React remount), where it used to fail for good.
126
+ - A shard run removes its own earlier report as it starts, so a shard
127
+ that then fails setup, crashes or is stopped is missing from
128
+ `report --merge` instead of standing in with the earlier run's results.
129
+ It also goes when the run stops before running anything (a feature file
130
+ error, a `--filter` that matches nothing).
131
+ - `saffron eject` warns when the project's own `playwright.config` records
132
+ a trace and the exported tests read secrets, `{env:...}` values or
133
+ credentials the API hands out, which Playwright's trace keeps unmasked
134
+ (SAF-200). It no longer only checked on the first eject, and it leaves
135
+ the file as it is. A config it writes leaves the trace out for
136
+ captured credentials too.
137
+ - A pending proposal stays with the Outline row whose values it was
138
+ recorded on when a row is inserted above it or the rows are reordered
139
+ (SAF-201), where another row used to take its "(example N)" and record
140
+ over it. `saffron status` and the plugins name that row as the rows
141
+ stand now, and the review page shows that row's pictures, where they
142
+ used to name and show the row that took its place.
143
+ - An API credential is masked wherever it could still show: a token a
144
+ polled response handed out before the one waited for, a token a later
145
+ capture replaced (a poll that rotates it, then quotes the old one as
146
+ `previous_token`), and an error raised while a request is built (a body
147
+ value that is no number quoting the captured token it resolves to). The
148
+ exported test learns a token from every polled response too, where it
149
+ learnt only the last one's and an error quoted the one before.
150
+ - A compiled `Then` keeps a quoted value as written: `the /name field is
151
+ "Ada Lovelace"` checks for both spaces, where it used to check for one.
152
+ - `saffron accept <file>#<revision> --step-set` refuses a proposal replaced
153
+ since it was listed, as a whole accept does, where it took the
154
+ replacement's steps unseen.
155
+ - A request sent several times at once waits for every copy when one gets
156
+ no response, and keeps the copies that answered in the transcript, where
157
+ the failure came first and a late write went unrecorded.
158
+ - An exported test extracts an integer beyond JavaScript's precision as the
159
+ digits the API sent, as replay does, where it rounded an int64 id into
160
+ another record's.
161
+ - Each API response is let go once read, in replay and in exported tests,
162
+ so a long poll no longer holds every response it read until the scenario
163
+ ends.
164
+ - An exported text check reads the copy replay reads, a shown one after a
165
+ hidden copy with the same text (a mobile menu's), where it read the
166
+ hidden one and failed.
167
+ - A step naming its column for the statuses of copies sent at once
168
+ (`refused with <refusal>`) records them as that column, as it records a
169
+ single status, so the other rows replay where they were refused as
170
+ recorded for one row.
171
+ - A JSON check reads a number JavaScript cannot hold as written as a
172
+ number, written as the API sent it: a `min` or `max` compares it exactly
173
+ with the bound as written, in any notation (`9007199254740993`,
174
+ `9007199254740993.0` and `9.007199254740993e15` are all over a maximum
175
+ of `9007199254740992`, `9007199254740991.1` over one of
176
+ `9007199254740991`, and `1e9999999999` over `1`, where it stopped the
177
+ run), and a `pattern` (which reads text) does not hold on it. An `equals` still checks its digits, as the error for a typed
178
+ number advises.
179
+ - A compiled `Then` leaves a number JSON cannot hold (`the /n field is
180
+ 007`) to the agent, where it compiled a check that could never pass.
181
+
182
+ Fixed, from real-world use of 1.3.1:
183
+
184
+ - The report's pass rate never reads 100% while a scenario failed: 473 of
185
+ 475 reads 99.58%, where it was rounded up to 100%. The dashboard's
186
+ suite health, recorded and tag shares read the same way, and 0% means
187
+ none. `passRate` in
188
+ `saffron status --json` and the history stays a whole number, now never
189
+ 100 unless every scenario passed (99 for 473 of 475).
190
+
191
+ - An Outline row that fails in a step before any step uses its own values
192
+ (a lunch sold out in a shared checkout step) is blamed on the app's
193
+ state, blocked by the environment, where the report used to call it
194
+ the recording's fault and point at a column the step never read. A
195
+ value used in a step's data table or doc string counts as used.
196
+ - A pending proposal stays with its row when a column is added to the
197
+ Examples: only the columns its row had tell rows apart, where every row
198
+ was called another row's and none recorded again.
199
+ - The sensitivity check judges a check against the action before a step
200
+ that only reads (a captured value, a watch armed), as it already did
201
+ for a wait: reading a booking number is not what confirms a payment.
202
+ - `api_contract` (the MCP tool) and the agent's contract summary list a
203
+ request body's alternatives with their fields (`one of: { ... } | { ...
204
+ }`) and an `allOf`'s parts as one body, where they said `any`.
205
+ - The Saffron skill's syntax reference describes the negative and exact
206
+ URL checks 1.3.1 added, where it said there were none.
207
+ - A recorded API step no longer keeps the agent's failed attempts
208
+ (SAF-202). A request whose body the pinned contract refuses is not
209
+ recorded, even when the API answered 2xx, unless the step expects a
210
+ refusal. The body is judged as sent, its tokens resolved: `{date+1}` as
211
+ a date, and `{captured:id}` as the text it sends, so an integer field
212
+ takes the number itself. A
213
+ request sent again to the same operation and expected answer right
214
+ after the step's last one takes its place, unless the step asks for
215
+ several ("twice", "two customers", "for each", "once more"), the earlier
216
+ one sent a value the step names that this one does not ("the currency
217
+ to SEK and guest names on" is two writes), this one sends the same
218
+ fields with other values the step does not name ("a mug and a hat"), or
219
+ both carry the same idempotency key. A value counts as named in any
220
+ case and in the step's data table or doc string, and as sent in a form
221
+ or the query too (`?status=draft`, then `?status=archived`); a step
222
+ with a table of several rows keeps every request.
223
+ - A `Then` that names a section in plain words (`the confirmation Summary
224
+ shows the package`, `the order card shows Paid`) has its checks scoped
225
+ to that section, as a quoted name already did (SAF-203): a check that
226
+ reads the whole page is refused while recording, instead of passing on
227
+ text that was there before the action.
228
+ - An Outline whose recorded row held one value in two columns
229
+ (`currency` and `display` both `NOK`) no longer refuses the other rows
230
+ for a step that names only one of them (`I choose "<currency>"`): the
231
+ recorder writes the column the step names. A step naming both, or
232
+ neither, still records again on a row that types them apart.
233
+ - A request that sends its header's credential in the body or URL as well
234
+ (`Authorization: Bearer {env:KEY}` and a body field `apiKey`) is refused
235
+ while recording, unless the step names both the body and the header.
236
+ Only an auth header counts: `X-Auth-Token`, not `X-Author` or
237
+ `X-Auth-User`.
238
+ - A "not shown" check naming a control by the start of its name (a
239
+ textbox `Firstname`, where the page shows `Firstname Required`) gets an
240
+ insensitive check note saying no element has that name, and naming the
241
+ control shown before the step.
242
+ - A check that finds its element by the text it checks (an amount found as
243
+ the text `SEK 495.00` anywhere on the page) is refused while recording:
244
+ it holds wherever that text shows. The agent finds the field the step
245
+ names, or records a visible check when the step only says the text is
246
+ shown.
247
+
248
+ Docs
249
+
250
+ - **How a recording is checked.** A new section of the guide (§24), and a
251
+ page of its own on the site, maps every check Saffron runs on a
252
+ recording before it is filed, in order: what each note means, which ones
253
+ hold a proposal from `accept --all` and when to accept it by file, what
254
+ is on by default, and how to turn on stability proofs.
255
+
256
+ Editors (VS Code extension 0.7.8, JetBrains plugin 0.7.9)
257
+
258
+ - Both editors mark `Examples: {data: roles}` or `{Data:roles}` with no
259
+ table under it as an error where it is written, as the runner stops on
260
+ it: VS Code in every project, JetBrains through the runner's language
261
+ server. Neither marks one inside a doc string or with a description
262
+ before its table.
263
+ - VS Code: the "Proposals pending review" row asks before it runs
264
+ `saffron accept --all`.
265
+ - JetBrains: Accept All and an accept or reject of selected proposals no
266
+ longer become the selected run configuration, so the Run button never
267
+ repeats them.
268
+
269
+ ## 1.3.1: Checks for what vanishes, URL and style checks, and a StepSet accepted on its own
270
+
271
+ Released 2026-11-02.
272
+
273
+ Runner
274
+
275
+ Upgrading from 1.3.0:
276
+
277
+ - A recording with a negative or exact URL check, a check of a message
278
+ that vanishes, a computed-style check, or a value captured earlier
279
+ inside the element a step looks for is cache schema version 14, 15, 16
280
+ or 17, which 1.3.0 and earlier refuse ("Update saffron-ai"). An Outline
281
+ recording whose cell is a `{data:...}` value holding a date is version
282
+ 18. Upgrade every machine and CI job that replays the suite together.
283
+ - `saffron.config.json` refuses a quoted number (`"actionTimeoutMs":
284
+ "10000"`), a negative one, or a `screenshot` other than `off`,
285
+ `viewport` or `full-page`, where it used to take them as they were.
286
+ - A row, data or captured value inside an API request's path is
287
+ percent-encoded where it would change the URL (`#`, `?`, a space; `&`,
288
+ `=` and `+` in the query): `A#1` is sent as `A%231`, no longer cut at
289
+ the `#`. A `/` and an existing `%20` are sent as they are.
290
+ - A `*` that opens a scenario after a Background ending in a `Then` is an
291
+ action now, as Cucumber reads it, not a check: a recording of such a
292
+ step records once more.
293
+ - A data table or doc string written under a `StepSet <name>` call is an
294
+ error naming the file and line, where it used to be dropped.
295
+ - With `reset` commands, recordings run one at a time when `agentWorkers`
296
+ is above 1 and the agent is on, and the run says so: one session's
297
+ reset would wipe the data another is recording on. Replays still use
298
+ `workers` as before.
299
+ - `strict`, `verifyProposals` and `reuseSteps` in `saffron.config.json`
300
+ take `true` or `false` without quotes; a quoted `"false"` used to read
301
+ as on.
302
+ - In an Outline, a number or boolean a request body or JSON check holds
303
+ only by coincidence with a row's value (a field not named like the
304
+ column, or a step not naming it) is kept as recorded and binds the
305
+ recording to its row, where it used to be sent and checked as each
306
+ row's value. Other rows are refused with a message until the field is
307
+ named like the column or the step names it. A constant beside a column
308
+ the step writes (`active: true` beside `admin: <admin>`) is kept as
309
+ recorded without binding the row.
310
+ - The recorder refuses checks that cannot fail: a URL check every URL
311
+ holds (`/`), a text, style, download or response pattern that matches
312
+ any text (`.*`), and a watch for a message other than the one the
313
+ `Then` names.
314
+ - A watched message counts as seen only when it appears after the watch
315
+ was armed (an element already showing it must stop showing it once),
316
+ and a seen check uses its watch up: a second check of the same message
317
+ needs a watch of its own. A recording that relied on a message left on
318
+ screen goes red and heals.
319
+ - `saffron diff --json` prints a list for one proposal too: a script that
320
+ read an object for one proposal reads the first entry now.
321
+ - SIGTERM stops a run as Ctrl-C does and exits 143, not 2.
322
+ - `--workers 0`, `--agent-workers 0` and a worker count that is not a
323
+ whole number exit 2 instead of running one worker.
324
+ - `init --examples` no longer writes into an existing
325
+ `saffron.config.json`.
326
+
327
+ - **From real-world use of Saffron.**
328
+ - A data value an API request takes through its step's table (`|
329
+ lastName | {data:signup.lastName} |`, sent as `<table:lastName>`) can
330
+ change and the recording replays at zero AI: the freshness check
331
+ follows the table cell to the data it names, and a header name HTTP
332
+ defines (`User-Agent`) is never taken for an old value ("User").
333
+ (SAF-177)
334
+ - A URL check with no URL is refused while recording, saying that a URL
335
+ check reads what the URL contains, and that there is no "does not
336
+ contain" or "is exactly" URL check yet: four such checks had been
337
+ recorded empty, and only the paid proof refused them. (SAF-178)
338
+ - `saffron run --plan --json` prints only the JSON on stdout, with a
339
+ shard too: the `shard 1/2 ...` line goes to stderr. (SAF-179)
340
+ - Fewer false sensitivity notes: a check is judged against the last step
341
+ that acted on the page, a step that only waits passed over; a
342
+ StepSet's exit guard is not probed against the set's own last action,
343
+ as it says the whole set got where it should (a set of checks only is
344
+ probed against the step before it); and a step that reads a whole
345
+ body, page or message gets no `broad check:` note. (SAF-180)
346
+ - From a review of these fixes: `--plan --json` also keeps the note on
347
+ Examples rows that share a recording off stdout, and an empty shard's
348
+ plan is an empty JSON plan, with no shard report written; a StepSet
349
+ used twice in a row has each use's exit guard left unprobed; an old
350
+ data value written into a body key or a custom header's name, whole or
351
+ in part, still makes the recording stale; "the body of a card", "the
352
+ entire row" and a card's body are not read as the whole page; and a URL
353
+ check's text is recorded trimmed. A header's value is still searched
354
+ for an old data value (`Authorization: Bearer <old token>`); only the
355
+ header names HTTP defines are left out, and a header an extract reads
356
+ by name is not taken for data.
357
+
358
+ - **From a three-round review of the runner.**
359
+ - No false green from a blank row value: a role name pattern, a response
360
+ URL, a JSON field or header pattern, a watched message and a style
361
+ pattern that a blank value leaves matching more than they mean are
362
+ refused, as text and URL checks already were. An expected status left
363
+ blank is refused, never read as any 2xx.
364
+ - Outline rows never record over each other's proposal: a row whose
365
+ pending proposal was recorded for another row is red, with a committed
366
+ recording too, and a proposal from another machine never turns a
367
+ failing protected `Then` yellow.
368
+ - A fixture takes values only from a proof that passed, and a proposal
369
+ from another machine is never replayed for its values.
370
+ - A scenario whose run throws (a busy proposal lock, a full disk) is that
371
+ scenario's red; the run goes on and writes its report. Reports, shard
372
+ reports and the history are written whole, never half.
373
+ - A contract check repaired by a heal keeps every operand of the check:
374
+ a URL, pattern or attribute changed is review, not repair.
375
+ - Secrets: a value set with a trailing newline is masked as the page
376
+ shows it, trimmed; credential headers and cookies in a trace are
377
+ replaced whole (`Authorization: Basic ...`); names written as one word
378
+ (`authtoken`, `JSESSIONID`) count as credentials; a header the
379
+ transcript keeps is judged by the same rule as the trace.
380
+ - A pattern holding a harmless escape (`\#`, `\:`) next to a row value
381
+ no longer fails as an invalid regular expression.
382
+ - `{captured:...}` in the element a step looks for is the captured value,
383
+ and a retried "message was shown" check can still see the message.
384
+ - API scenarios: YAML merge keys (`<<: *base`) in a contract are applied,
385
+ the response schema is chosen by the response's content type,
386
+ `differs` compares lists and objects, `the /x field is null` compiles,
387
+ a phrase-compiled `Then` serves every Outline row, and with
388
+ `api.transcript: "off"` no error quotes a response body.
389
+ - Recording: a multi-line typed text keeps its `{captured:...}`, a
390
+ `{unique:...}` inside an Examples cell is typed resolved, a feature
391
+ edit writes `<plan>` only for the whole word (`Production` stays),
392
+ and Antigravity stops when the run aborts.
393
+ - `saffron eject` writes tests never looser than replay: an exact path
394
+ or style is compared whole, enabled and disabled checks need the
395
+ element shown, `differs` compares amounts, a not-visible check reads
396
+ every locator at once, timed waits are capped, the device's own
397
+ engine is dropped, and Playwright's trace is left out when the tests
398
+ read `{env:...}` values, since it would record them unmasked.
399
+ - CLI: `run --plan` validates its flags as `run` does, `coverage --write
400
+ --json` prints only JSON, the HTML report escapes every proposal
401
+ field, and `init` sets up the rest before naming an MCP file it
402
+ cannot merge.
403
+ - Amounts and dates: `(1.5 kg)` is positive, and a date range that ends
404
+ before it starts is refused.
405
+
406
+ - **From a second two-round review of the runner.**
407
+ - Outline rows: `accept --propagate` no longer copies a healed locator
408
+ holding its Outline row's value into other recordings, and `saffron
409
+ status` marks an Outline stale when any row would record again, naming
410
+ the row.
411
+ - Secrets: a value set with a trailing newline is masked where the page
412
+ holds it trimmed (in a URL, in another case, escaped in a selector or
413
+ glued into a word), and a warning about an unhandled rejection masks
414
+ the credentials a session captured.
415
+ - Recording: a target with nothing to find the element by is refused, a
416
+ `saffron_step` whose text names another step is refused so actions are
417
+ never filed a step late, the first `saffron_done` verdict stands, a
418
+ pass past the time budget that leaves a step unrecorded is a failure,
419
+ and a Claude session stopped at its turn limit says to raise
420
+ `maxTurns`; Claude token counts include compaction.
421
+ - Files and processes: `prune` keeps a merge-conflicted recording of a
422
+ live scenario outside `features` and the recording of an Outline whose
423
+ Examples have no rows for now; a scenario name of any length can be
424
+ saved, and recordings saved under the whole name are still found;
425
+ Ctrl-C stops a running `reset`; a `hookTimeoutMs` past a timer's limit
426
+ no longer ends every hook at once; a hook timed out on Windows ends with
427
+ its child processes; a run killed while writing history no longer
428
+ spoils the next line; a browser that fails to close or a history file
429
+ that cannot be written no longer costs the run its report; writes ride
430
+ out a file briefly locked on Windows; a failed browser close leaves no
431
+ video folder behind; `init` git-ignores the temp and lock files a
432
+ killed run leaves.
433
+ - `saffron eject` writes tests never looser than replay: a schema check
434
+ finds its operation with the row's value in the path, a response with
435
+ a schema per JSON media type is refused, a blank expected status and
436
+ the NAT64 and IPv4-compatible link-local forms are refused, attribute
437
+ and style checks read the element replay reads, a blank captured value
438
+ in a checked text fails, a dialog answer from a row is that row's, a
439
+ whitespace-only text pattern is fixme, and header patterns compile as
440
+ replay compiles them.
441
+ - Status, plan and coverage: a feature in another Gherkin language shows
442
+ its name, `run --plan` counts a feature's fixture, `coverage` refuses a
443
+ suite with a file it cannot parse and `verify` says so, and a scenario
444
+ named like an Outline row keeps its own last result.
445
+ - Editors and reports: the language server reads doc string lines as
446
+ data, not StepSet calls, and the CI run panel shows a cost under a cent
447
+ as the report does.
448
+
449
+ - **From a two-round review of the runner.**
450
+ - No false green from another row or a leftover: an Outline cell that
451
+ reads a date through a data value has its day checked as a date cell
452
+ does; a step reused by swapping a quoted value no longer offers the old
453
+ value's other recordings; a message already on screen before an action
454
+ no longer counts as seen; a capture pattern a blank row value leaves
455
+ open is refused.
456
+ - Recording types a captured value as plain text, as replay does, and
457
+ refuses a browser call whose captured value holds an `{env:...}`
458
+ token rather than type the secret.
459
+ - Checks without a frame also find what an iframe attached after the
460
+ check began shows; checks with fallback locators end at their timeout;
461
+ a drag finds its drop zone by its fallbacks; the heal guard runs a
462
+ check on the tab an earlier action moved to or opened.
463
+ - Seeded recording keeps a watch armed in one stretch for its `Then` in
464
+ the next, and seeded steps and the sensitivity check read values
465
+ another step's table or doc string gives.
466
+ - API: a schema check finds the operation for a path with a parameter
467
+ inside a segment (`/files/{name}.json`); a row value after a captured
468
+ URL that brings its own query is encoded as a query value, in replay
469
+ and in exported tests; with no agent, an Outline's rows file one
470
+ contract re-check, not one each; `api pin` passes over a fallback URL
471
+ that does not answer with OpenAPI 3.
472
+ - Proposals: `accept` notices a hand edit of a proposal's API contract
473
+ fingerprints, `accept --step-set` refuses to replace another StepSet
474
+ whose name makes the same file name, and `run` reports two StepSets or
475
+ a fixture and a scenario whose recordings would share a file; an
476
+ accepted heal no longer copies a captured value into other scenarios;
477
+ a "capture differs" note leaves credentials out.
478
+ - Reports: a password holding an `@` in a URL on the command line is
479
+ hidden whole, `report --merge` refuses a shard it cannot show, naming
480
+ the field, an exported or imported bundle's report names only the
481
+ evidence the bundle carries, and the trace page marks a request the
482
+ agent did not record.
483
+ - CLI and editors: `saffron author` fails with the provider's error
484
+ instead of writing it as the draft; `init` refuses an instructions
485
+ block with no end marker instead of deleting the text after it; a
486
+ config file holding something other than an object is refused by
487
+ name; an exported timed wait written with a row value waits as long
488
+ as replay; feature edits apply to steps whose keyword is several
489
+ words (`Étant donné que`); the language server started by an editor
490
+ reads a project with no `features` key as the default folder.
491
+
492
+ - **A `Then` is read-only while recording.** A check refuses a browser
493
+ action that enters, presses keys, picks, ticks, uploads, drags or
494
+ navigates, and a click on a button that submits or commits (Log in,
495
+ Save, Submit, Confirm and the like), before the browser runs it: the
496
+ agent is told the step is a check and to fail it, suggesting the action
497
+ as a `When`, if it does not hold as the page stands. Before, such a
498
+ click was recorded and proved, and held for review afterwards; a login
499
+ suite had a check click Log in, and others submit again to bring back a
500
+ toast that had vanished. A click that opens what is checked (a menu, a
501
+ tab) is still recorded and held as before, and proposals filed before
502
+ this stay held from `accept --all`. (SAF-181)
503
+
504
+ - **A "not shown" check must see something disappear.** The sensitivity
505
+ check after a proof runs each "not shown" check in a `Then` that holds
506
+ other checks on its own, before the step it follows; one that holds
507
+ there too (its element was never on the page, as with an invented
508
+ selector) gets an `insensitive check:` note naming it, held from
509
+ `accept --all`. Before, the step's other checks failing there hid it,
510
+ and it stayed green whatever the page showed. The recording agent is
511
+ also told to target the element a named field labels (the booker, not
512
+ any paragraph showing the name) and only elements it saw for a "not
513
+ shown" check. (SAF-183)
514
+
515
+ - **A StepSet's own data value stays its own in an Outline.** A step that
516
+ enters a `{data:...}` value itself, while not naming the column (a
517
+ StepSet typing `{data:guests.first.surname}`), no longer has the rows
518
+ refused because one row's cell reads the same: the email row of a login
519
+ suite was refused for it. A value a step only checks is still refused
520
+ for the rows that differ, as the page may show it from the row.
521
+ (SAF-182)
522
+
523
+ - **Trial navigations stay out of the recording.** A navigation the
524
+ agent leaves for the same page with another query or fragment, before
525
+ doing anything there, is discarded and noted in the proposal: a login
526
+ suite kept each query format the agent tried as a replay action. A
527
+ navigation to another page, or one acted on, is kept. (SAF-191)
528
+
529
+ - **A part of a name the agent wrote replays.** A click through a locator
530
+ the agent wrote, naming a part of the element's name, is recorded with
531
+ the full name the page it saw last shows for the one element of that
532
+ role holding it: replay matches a name whole, and timed out on "Sign
533
+ in". Where the page showed none or several, the proposal notes it
534
+ while recording, before the proof. (SAF-184)
535
+
536
+ - **Negative and exact URL checks.** A URL check can now say that the
537
+ URL does not contain a value, or that its path is exactly one, recorded
538
+ from the step's own words: before, a URL check was containment only,
539
+ and steps saying the URL no longer holds the login could not be
540
+ recorded. Replay and `saffron eject` check them alike. A recording
541
+ holding one is cache schema version 14, so an older runner refuses it
542
+ instead of checking containment. (SAF-186)
543
+
544
+ - **Checks for messages that vanish.** A toast or error flash gone
545
+ before its `Then` runs can be checked: the agent arms a watch
546
+ (`saffron_watch`) in the `When` step before the action that shows it,
547
+ and the `Then` checks that it was seen since. Replay watches the same
548
+ way. Before, a login error toast vanished before the check, and
549
+ recordings submitted again inside the `Then` to bring it back. A
550
+ recording holding a watch is cache schema version 15; `saffron eject`
551
+ writes its scenario as `test.fixme` for now. (SAF-185)
552
+
553
+ - **Computed-style checks.** A step can check an element's computed CSS
554
+ property (an invalid input's `border-color` is `rgb(220, 38, 38)`), or
555
+ match it to a pattern: before, only classes, attributes and the visible
556
+ error could be checked. Replay and `saffron eject` check it alike, in
557
+ Chromium, Firefox and WebKit. A recording holding one is cache schema
558
+ version 16. `saffron eject` also writes the negative and exact URL
559
+ checks now, which it had written as `test.fixme`. (SAF-187)
560
+
561
+ - **A navigation to a captured URL stays the capture.** When the agent
562
+ goes to `{captured:signin}` and the value is an address on the base URL,
563
+ the recording keeps `{captured:signin}`: before, it kept the day's path
564
+ relative to the base, and replay went to the old link instead of the one
565
+ its own request got. (SAF-188)
566
+
567
+ - **Accept one StepSet from a proposal.** `saffron accept <file>
568
+ --step-set "Sign in"` files only that set's steps, which its proof
569
+ replayed green, for every other scenario to reuse at zero AI, while the
570
+ proposal and its failing check stay pending. Before, a failed check kept
571
+ a verified sign-in from being reused, and each scenario paid to record it
572
+ again. A proposal whose proof failed records how many steps passed before
573
+ it (`meta.provedSteps`), and a set the proof did not get through is
574
+ refused. (SAF-189)
575
+
576
+ - **Reuse picks the recording the page's state fits.** A step recorded in
577
+ two page states (a promo field opened first, or found open) keeps both
578
+ recordings, and seeding replays the one whose controls the page shows
579
+ now: before, it always took the newest, which closed a field already open
580
+ and paid the agent to record the step again. (SAF-190)
581
+
582
+ - **CLI and files, from the 1.0 review.** (SAF-133)
583
+ - A scenario with no steps is red, saying so, and starts no agent
584
+ session: it had recorded nothing and passed.
585
+ - A broken link or a folder named `*.json` under `.saffron/proposals`
586
+ no longer stops `status`, `diff` and the end of a run.
587
+ - `saffron diff --json` prints a list for one proposal too, as it does
588
+ for several.
589
+ - `accept`, `reject` and `diff` find a project-relative file (as
590
+ `status` prints it) under `--project-root`, not only from the current
591
+ folder.
592
+ - `accept --with-feature-edit` edits a step line that ends in spaces.
593
+ - `saffron status` prints an unreadable recording's warning on a line of
594
+ its own.
595
+ - `saffron init` keeps a skill you edited under `.saffron/backups/skills/`
596
+ before installing the shipped one.
597
+ - Already fixed and checked: an empty run exits 2 naming why, a StepSet
598
+ file that does not parse is named, recordings are written atomically,
599
+ a CRLF file takes feature edits, and status counts an unreadable
600
+ recording apart.
601
+
602
+ - **A check's selector is tried on the page while recording.** A check
603
+ whose CSS selector matches nothing on the page the agent sees is refused
604
+ then, at no cost, naming the likely cause (a link styled as a button is
605
+ an `<a>`): before, only the paid proof caught `p:has(button)`. The other
606
+ half of this story, a `broad check:` note for a check held by a whole
607
+ card, shipped in 1.3.0. (SAF-171)
608
+
609
+ - **Exact `Then` phrases need no agent.** In an API scenario, `Then the
610
+ response should answer 404`, `Then the /status field is active` and
611
+ `Then the response matches the Customer schema` are recorded as those
612
+ checks without the agent, which records only the steps around them. Any
613
+ other wording goes to the agent as before, never guessed. An option, not
614
+ a required syntax. (SAF-162)
615
+
616
+ - **Live progress, and a time budget per agent session.** While an agent
617
+ records, the run prints on stderr each step it records and each proof as
618
+ it ends (`· Checkout: recording step 41 of 92: ...`, `· Checkout: proof
619
+ 1 passed`): a 92-step recording ran 340 seconds with no word until it
620
+ finished. `maxSessionMinutes` (`--max-session-minutes`) ends one session
621
+ that has gone on that long: its tools refuse, and it ends unfinished
622
+ with the agent's account of where it got to. (SAF-194)
623
+
624
+ - **Row proofs show their time and each row's latest result.** `saffron
625
+ diff` and the review page say when an Outline's rows were proved, and
626
+ beside each row its latest result in the last run (`latest run: red at
627
+ ...`, or `no run since`): a proposal's passing row proofs were read as
628
+ readiness while later runs of those rows timed out. (SAF-192)
629
+
630
+ - **A failed response is named beside the step it broke.** A step that
631
+ fails after one of the app's own requests reported failure (an error
632
+ status, or a JSON body with `success: false`, `ok: false` or an
633
+ `error`) says so in its error, which the console, the report and the
634
+ heal prompt all show: an HTTP 200 holding "overbooked" before a timeout
635
+ reaching checkout no longer reads like a locator problem. (SAF-193)
636
+
637
+ - **Outline rows, from the 1.0 review.** (SAF-128)
638
+ - A list check whose values come from `{data:...}` or `{env:...}` cells
639
+ ("Alice, Bobby" from `{data:guests.a}` and `{data:guests.b}`) is tied
640
+ to its row, as one of plain cells is, so another row fails it with the
641
+ reason and no agent session: it was written as the row's `<column>`,
642
+ and every other row failed it and paid the agent again on every run.
643
+ - The guide says what a row-bound recording keeps of a short secret
644
+ read through an Examples cell, and why a PIN belongs elsewhere.
645
+ - Checked and kept with tests: a value inside a word of what a
646
+ Background step types (`test` in `qa-tester@example.com`) binds no row;
647
+ a date cell typed as ISO, a value seen only as a slug or in another
648
+ case, and a column naming a selector's role refuse the other rows
649
+ instead of recording again; a URL-encoded value is the row's
650
+ `<column>`; a `{data:...}` value naming an unset variable stops the run
651
+ before it starts; a short secret glued to other text is masked.
652
+
653
+ - **Visible text, from the 1.0 review.** (SAF-129)
654
+ - A check whose value is written on several lines (a two-line doc
655
+ string) holds where the page draws those lines, split by a `<br>` or
656
+ by blocks: it never matched "Line one<br>Line two", which read "Line
657
+ oneLine two". A value on one line still reads the text as one run, so
658
+ two blocks drawn apart never make it.
659
+ - A list check (`expectEach`) no longer passes on a hidden option, or
660
+ hidden text, that differs from a shown one only in case: the page's
661
+ lines in another case count only where CSS `text-transform` changes
662
+ them.
663
+ - A check on a large container is faster: 22 ms instead of 56 ms on a
664
+ 4,000-row section. 0.9.9 takes 4 ms there, as it did not read what is
665
+ drawn; the rest is the cost of reading only drawn text.
666
+ - Kept as it is: an element without text whose first match is hidden
667
+ gives way to a displayed one, as a hidden mobile copy does to the
668
+ desktop one; a check that means one row's badge targets that row.
669
+
670
+ - **CLI, reports and the trace viewer, from the 1.0 review.** (SAF-130)
671
+ - `saffron diff` names the text a role with no name is found by:
672
+ `expect listitem with text "Milk" to be visible`, not every listitem.
673
+ - The trace viewer titles another browser session's step by the keyword
674
+ it was written with (`Given`, `And`), not by its kind, and a session
675
+ opens on a step of its own rather than on another session's first one.
676
+ - The results page embeds at most 20 MB of screenshots, failures first;
677
+ past that it names each file instead (300 failures made a 78 MB page).
678
+ - A feature reached through a linked folder's alias, or named in another
679
+ case on a case-insensitive disk, is the suite's feature, with its
680
+ recordings, not a new one.
681
+ - The MCP tools keep answering while `saffron.config.json` cannot be read,
682
+ on the config they last read, and say what is wrong with it.
683
+ - `init --examples` leaves an existing `saffron.config.json` as it is:
684
+ the examples' `retries`, `strict` and `trace` no longer change the
685
+ project's own runs, and a missing `baseURL` is named in a note.
686
+ - A bundle made in a git checkout with no remote is refused for that, not
687
+ as one made outside a checkout.
688
+ - SIGTERM stops a run as Ctrl-C does (teardown, the report of what
689
+ finished) and exits 143, instead of a red line and exit 2.
690
+ - A red scenario that replayed no step is labelled `not run` in the run
691
+ output, not `cache`.
692
+ - `--workers 0`, or a typo in `--agent-workers` or in the config's
693
+ `workers` or `agentWorkers`, exits 2 instead of running one worker.
694
+ - Fewer temporary folders are left behind by the test suite.
695
+ - Kept as they are: `saffron status` already marks an Examples row a
696
+ row-bound recording refuses (✗, with the reason), and the chronic hint
697
+ counts only reds that cost an agent session, which a refused row never
698
+ does. `saffron report --no-open` already writes the report without
699
+ opening a browser (1.3.0).
700
+
701
+ - **Language server and editor parity, from the 1.0 review.** (SAF-131)
702
+ - The language server reads data Examples under every Examples keyword
703
+ of the file's language (`Scenarios:`, `Exemples:`), as the parser
704
+ does, instead of flagging them; a bare `{data:file}` in a step is no
705
+ reference to the runner, so it gets no hover and no error; and the
706
+ `# language:` header is read with Gherkin's own pattern.
707
+ - Its completion rates a step after `As "<who>"` (a session an Examples
708
+ column names) unrecorded, as step reuse never seeds it.
709
+
710
+ - A text check that fails on a value the page holds but does not show says
711
+ where it sits: `Timed out waiting for text "Total: 42 NOK": the page holds
712
+ it, but not shown (inside a closed <details>)`, or in an element with
713
+ `display: none`, the `hidden` attribute, `visibility: hidden` or
714
+ `content-visibility: hidden`, naming the element. A total that is gone and one behind a panel that did not open read apart. Text
715
+ the page does not hold reads as before. (SAF-195)
716
+ - A text check on a large container is fast again: 3.5 ms on a
717
+ 4,000-row container (0.9.9 took 2.4 ms, reading hidden text too). The value is looked for in the container's text first,
718
+ and only the text it covers is asked whether it is drawn; anything that
719
+ question cannot settle (a textarea, a dropdown, a shadow host) is read
720
+ the full way, so the check passes on exactly what it passed on before.
721
+ (SAF-129)
722
+ - A range of days the page shows with its year (`20–22 Oct 2026`,
723
+ `Oct 30 - Nov 1, 2026`, `31 Dec 2026–2 Jan 2027`, German
724
+ `20.-22.10.2026`), up to 62 days long, is recorded as a range token
725
+ (`{date+7–+9:D MMM YYYY}`), as a single day already was. A step that
726
+ names a stay by intent ("check-in in 7 days for 2 nights") now records
727
+ a check that follows the clock instead of a date frozen on the day of
728
+ recording. The agent is no longer told to match dates with a pattern,
729
+ which passes on any day. (SAF-124)
730
+ - The agent skill and the block `saffron init` writes into `AGENTS.md` and
731
+ `CLAUDE.md` say when a small supporting script is the right tool (only
732
+ for a value Saffron cannot express, asked for first) and how to keep it
733
+ trustworthy: expected values never taken from the output being checked,
734
+ run as a `setup` command writing git-ignored `data/` files, assumptions
735
+ stated, and a report of what could replace it. Run `saffron init` again
736
+ to update the block. (SAF-125)
737
+ - A video of each scenario's run: `"video": "retain-on-failure"` keeps
738
+ the video of every scenario that is not green, `"on"` every one, in
739
+ `video.webm` beside its screenshots; also `--video` for one run. Off
740
+ by default. It works in Chromium, Firefox and WebKit, and a heal or a
741
+ recording is in the video of the run it happened in; proof replays are
742
+ not filmed. A browser that cannot record (no ffmpeg) stops the run once,
743
+ at the start. A summary export carries no video. A video shows what is
744
+ typed, secrets included, so keep it off for scenarios that type them.
745
+ (SAF-51)
746
+ - The limits of the CI runs viewed into `.saffron/ci/` are settings:
747
+ `"ci": {"cacheRuns": 20, "cacheMB": 4096}`. The defaults stay 10 runs
748
+ and 1 GB. (SAF-121)
749
+ - Each feature edit a proposal suggests names the file and line it
750
+ rewrites, in `saffron diff` (`file` and `line` in `--json`), the review
751
+ page and the run report. A StepSet's step names the StepSet's
752
+ definition, where accept rewrites it for every scenario using the set,
753
+ and an edit to a Background step is marked skipped, as accept leaves it
754
+ to you. (SAF-31)
755
+ - `saffron status --json` lists each pending proposal's feature edits
756
+ (`featureEdits`, as `diff --json` gives them, with file and line), and
757
+ the VS Code and JetBrains panels show them in a proposal's details
758
+ without another command. (SAF-30)
759
+ - Each step a proposal changes is marked with its class: `repair` (only
760
+ how the step finds the same element again or waits for it: a fallback
761
+ selector, a timeout, a wait), `review` (what it does changed, an
762
+ element named another way, or one of Saffron's notes names it) or, for a
763
+ suggested feature edit, `protected` (the wording, which only you
764
+ apply). It shows in `saffron diff` (`adaptation` in `--json`, with
765
+ counts in `summary`), the review page and the run report, and the
766
+ guide has a section on the three classes. (SAF-32)
767
+ - From a review of this release: a step whose target is named by a
768
+ selector inside an iframe is `review` when the selector changes (the
769
+ frame names the iframe, not the element); the heal branch's pull request
770
+ body, the run report and the editors' trace panel class each step as
771
+ `saffron diff` does, reading the proposal's notes, and name the file and
772
+ line of each feature edit; a video with no viewport setting is filmed at
773
+ the page's own 1280x720, not Playwright's 800x800 fit; a range that
774
+ writes the month twice (`Oct 20 - Oct 22, 2026`), one ending exactly 31
775
+ days back, or one from before today to after it (`Room 7 - 22 Oct 2026`
776
+ is a room) is kept whole, never with only its end rewritten.
777
+
778
+ Editors (VS Code extension 0.7.7, JetBrains plugin 0.7.8)
779
+
780
+ - Both editors list the feature file edits a proposal suggests, each with
781
+ the file and line `saffron accept --with-feature-edit` rewrites and why
782
+ accept would skip one: VS Code in the proposal tooltip, JetBrains in the
783
+ Proposals tab details, read from `saffron status --json`. (SAF-30)
784
+ - VS Code: a `# language:` header Gherkin rejects is a plain comment; a
785
+ file a link inside the test folder leads back to is listed once; a
786
+ StepSet under a nested `saffron.config.json` in the test folder is
787
+ defined. (SAF-131)
788
+ - JetBrains: the Files tab fails on a folder it cannot read, as the runner
789
+ does, instead of hiding it; right-click Run reads the IDE's view of the
790
+ files, not the disk through links, each time the menu opens; a Windows
791
+ junction is walked as a link. (SAF-131)
792
+ - Both editors open a report path through a link outside the test folder
793
+ only when it really leads into the project, as the runner follows links
794
+ inside the test folder only. (SAF-131)
795
+
8
796
  ## 1.3.0: Recordings checked by machine, and a review every view agrees on
9
797
 
10
798
  Released 2026-10-30.