saffron-ai 1.3.5 → 1.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,1812 @@ the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
5
5
  JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
6
6
  release heading names its main feature: `## <version>: <title>`.
7
7
 
8
+ ## 1.4.1: Database steps on seven more databases
9
+
10
+ Released 2026-11-17.
11
+
12
+ Runner
13
+
14
+ - Database steps on six more databases (beta), beside MongoDB below, each
15
+ tested against the real database or its emulator:
16
+ - Oracle (`oracle://user:pass@host:1521/service`, `npm install oracledb@6
17
+ node-sql-parser`; the driver's thin mode needs no Oracle client). A
18
+ statement is read as plain SQL: `FETCH FIRST`, `OFFSET` and `ROWNUM`
19
+ count as a `LIMIT`, and `CONNECT BY`, `(+)`, flashback, `q'...'`
20
+ strings, `MERGE` and PL/SQL are refused. Reads run in a `READ ONLY`
21
+ transaction; a write is read back by its primary key, the key Oracle
22
+ gave an inserted row read from that row itself. Names are in capitals,
23
+ so a row's columns are checked as `/0/NAME`.
24
+ - SQLite (`sqlite:path/to/app.db`, `npm install node-sql-parser`; the
25
+ driver is Node's own, from Node 22.13). The file is opened read-only
26
+ for a check and never created. `INSERT OR REPLACE` and the like are
27
+ refused. A "not in the database" check that only a type keeps empty
28
+ (`json_extract(...) = $1` with text where the JSON holds a number) is
29
+ refused. `db.allow` lists the file's name or path.
30
+ - MariaDB (`mariadb://` or `mysql://`). It runs as MySQL, and did not
31
+ before: every statement failed on a session setting MariaDB does not
32
+ have, and cleanup put a `JSON` column back respelt (`1.10` as `1.1`).
33
+ - CockroachDB (`cockroachdb://` or `postgres://`). It runs as PostgreSQL,
34
+ and did not before: every write failed, as CockroachDB does not say
35
+ which table a written row is in. A table with no primary key (a hidden
36
+ `rowid`) counts as having none, and a read the database asks to run
37
+ again is read again.
38
+ - DynamoDB (`dynamodb://<region>`, or `dynamodb://<host>:<port>` for
39
+ DynamoDB Local; `npm install @aws-sdk/client-dynamodb`). A step holds
40
+ one JSON command, as on MongoDB: `get`, `query`, `put` or `update`,
41
+ with no scan and no delete. A key attribute is sent as the type the
42
+ table declares; a "not in the database" check is red when an item is
43
+ found with a bind sent as another type, and never reads a global
44
+ secondary index. A `put` never replaces an item. Tested on DynamoDB
45
+ Local; AWS itself is not part of the test suite.
46
+ - Firestore (`firestore://<project-id>`, or
47
+ `firestore://<host>:<port>/<project-id>` for the emulator; `npm
48
+ install @google-cloud/firestore@7`). One JSON command per step: `get`,
49
+ `query`, `create`, `add` or `update`. Placeholders name Firestore's
50
+ types (`"$1:timestamp"`, `"$1:reference"`); a "not in the database"
51
+ check is red when a document is found with a bind sent as another
52
+ type. A `create` never writes over a document. Tested on the
53
+ emulator; Google Cloud itself is not part of the test suite.
54
+ - What tells AI agents how to use Saffron is brought up to this release:
55
+ the authoring skill and its references (the new databases and how to
56
+ word a step for each, exact headers and keywords, `--filter` with
57
+ `and`, `or` and `not`, amounts compared to the decimals shown), the
58
+ block `saffron init` writes into `AGENTS.md` and `CLAUDE.md`, and the
59
+ MCP server's `project_status`, which names the database's engine and
60
+ how its steps are worded. Run `npx saffron init` again to take the new
61
+ block and skill.
62
+ - A recording with a database step on one of the new engines is written
63
+ as schema version 22, so an older runner asks for an update instead of
64
+ a re-record it could not make.
65
+ - Database steps on MongoDB (beta): set `db.url` to a `mongodb://` or
66
+ `mongodb+srv://` URL that names its database, and `npm install mongodb`
67
+ (no SQL parser needed). A step holds one JSON command where a SQL step
68
+ holds its statement: `{"find": "customers", "filter": {"email": "$1"}}`
69
+ or `{"count": ...}` for a check, `{"insert": ..., "documents": [...]}`
70
+ or `{"update": ..., "filter": {...}, "set": {...}}` for a write. The
71
+ rules are the SQL ones: every value a placeholder bound from the step
72
+ (typed where the field needs it: `"$1:objectId"`, `"$1:uuid"`,
73
+ `"$1:long"`, `"$1:decimal"`, `"$1:date"`), a check tied to a value the
74
+ scenario wrote, filters limited to comparison operators (`$where`,
75
+ `$expr`, `$regex` and aggregation are refused), and cleanup that deletes
76
+ inserted documents and puts back, type for type, only the paths an
77
+ update set. A "not in the database" check is judged again with each
78
+ bind sent as each other type it reads as (text, ObjectId, UUID, number,
79
+ decimal, long, date), at its start and at the end of its window: a
80
+ document found so is red, as is a field no document has (compared with
81
+ a bind, in a range, under `$elemMatch` or `$exists` true) or a
82
+ collection that is not there. A UUID reads as its dashed text. A
83
+ whole document or list compared as a value is refused (compare its
84
+ fields with dotted paths), and so is a projection beside a check that a
85
+ field is missing. An update that fails part way puts back what it
86
+ changed first. A failed MongoDB check is never repaired. Recordings
87
+ with MongoDB steps are cache version 21, so an older runner asks to be
88
+ updated instead of advising a re-record it cannot do.
89
+ - On every engine, an `every` check under `LIMIT`, `TOP` or `DISTINCT`
90
+ is refused, as a count is: it cannot see a second, wrong row.
91
+ - A value extracted from inside a credential object (`/session/id`,
92
+ `/session/0/id`, `/credentials/access`, kept as `ref`) is now a
93
+ credential too when it is a text of 6 characters or more: it is masked,
94
+ and never sent to another origin, as the body already showed it hidden
95
+ (`/auth/role` holding "admin" stays plain). So is a credential inside an
96
+ extracted object (`extract "/data"` holding `session: { id }`), a list
97
+ under such an object (`auth.backup[0]`), the texts of a list or object
98
+ whose name ends in `tokens` (`refreshTokens`, while `total_tokens`
99
+ holding a number stays plain), and a name in the plural (`apiKeys`,
100
+ `passwords`). Before, the
101
+ session id or token could reach a second host. Exported tests read
102
+ credentials the same way.
103
+ - Postgres: `db.cleanup` now puts back a json or jsonb column whose change
104
+ was only past 15 digits (`9007199254740992` to `9007199254740993`).
105
+ Both sides of the comparison were read as JavaScript numbers, so the
106
+ change looked like none and the new value stayed. A write's `extract`
107
+ of such a number now keeps every digit instead of a neighbouring id.
108
+ - An API scenario's database checks now read again at `pollIntervalMs`,
109
+ as a browser scenario's do, instead of every 250 ms.
110
+ - An interaction that ran is no longer run again by `retries`: an action
111
+ recorded as opening a tab when none opens, a click armed for a download
112
+ or a file chooser that starts none, a click or key press whose page load
113
+ outlasts the action timeout, and a checkbox the click did not change all
114
+ fail at once. Before, a retry clicked again, and a submit could send its
115
+ order twice (and, on a reloaded form, pass). A `newTab` whose page fails
116
+ to load is closed, so a retry's tab is the only one.
117
+ - A response header sent more than once is checked as it means: each
118
+ Set-Cookie on its own (of several cookies, a pattern names its cookie
119
+ first, `^session=.*HttpOnly`), and a header that holds one value
120
+ (X-Frame-Options, Content-Type, Location) with every value, so a `DENY`
121
+ beside an `ALLOWALL` is red. A list-valued header (Vary, Cache-Control)
122
+ is still read joined. Exported tests check headers the same way.
123
+ - A JSON check whose path holds an empty row value (`/stock/<id>` with an
124
+ empty cell) is refused instead of reading a field no response has, where
125
+ "not there" passed on anything. A check typed as a string fails on a big
126
+ number, so an id the API turned from text into a number is red.
127
+ - Recordings that may turn red with this release, and need
128
+ `saffron run --rerecord`: a check typed as a string on a big number
129
+ (the old advice for such numbers; check it with no type), and a
130
+ Set-Cookie pattern that names no cookie on a response that sets several.
131
+ - Credentials in a form-encoded answer (an OAuth token endpoint's
132
+ `access_token=...&scope=...`) and in JSON after an anti-XSSI prefix are
133
+ hidden from the agent, errors and transcripts.
134
+ - An Examples table with a header row and no rows is now an error. Before,
135
+ the outline ran nothing, so deleting its last row removed it from the
136
+ run while CI stayed green. Comment a draft out until it has a row.
137
+ - `--max-session-minutes` (`maxSessionMinutes`) now limits API recording
138
+ and healing sessions too, and they print their progress as browser ones
139
+ do.
140
+ - A header pattern (`expectHeader`) or a downloaded file's pattern that
141
+ backtracks (`(a+)+$`) now fails the check instead of freezing the run,
142
+ as JSON and page patterns already did.
143
+ - A JSON data file with a number a JavaScript number reads as another
144
+ value (an id past 2^53, more digits than a double holds, `1e400` read
145
+ as Infinity or `1e-400` as 0) is now an error that asks for it in
146
+ quotes, instead of sending and checking another value.
147
+ - A feature file whose scenario header Gherkin does not know (`scenario:`,
148
+ `Scenario :`, `Scenario Name` with no colon, another language's keyword
149
+ under an English file) is now an error. Gherkin reads such a line, and
150
+ the steps under it, as the feature's description, so the scenario, or
151
+ the whole file, left the run without a word while the suite stayed
152
+ green. So is a step above a scenario's first step that Gherkin cannot
153
+ read as one (a tab or a no-break space after its keyword, an invisible
154
+ character before it), a `StepSet` call there in a `.feature` file, and
155
+ `stepset` in the wrong case: each was left out of its scenario. Prose is
156
+ left alone: one description line that starts with "When", or a wrapped
157
+ line that starts with "and", stays description. The language server
158
+ reports these, and an Examples table with no rows or no table, in every
159
+ editor that uses it.
160
+ - A recording is recorded again when a step reads a table value another
161
+ step gave it (`<table:name>`) and an edit leaves two tables disagreeing
162
+ on that value. Before, the token resolved to nothing, replay looked for
163
+ the text `<table:name>` itself, and a "not shown" check passed on it
164
+ with the value on the page.
165
+ - A "not shown" check ending in a row's whole number (`<amt>` holding 10)
166
+ now counts `10.00` on the page as shown. It passed as gone.
167
+ - A "not shown" check in a step that names no column, holding a word of
168
+ the row's value in another order or spelling ("Lee, Ann" for the guest
169
+ "Ann Lee"), now binds its recording to that row, so the other rows fail
170
+ with guidance instead of asking whether the first row's guest is gone.
171
+ - An amount check compares to the unit the amount is shown with: a rate
172
+ of 0.125 no longer passes on 0.129. Amounts shown with two decimals or
173
+ fewer still compare to the cent, so a tax worked out as 7.9992 is the
174
+ 8.00 shown. Two captures that differ under a cent (0.125 and 0.129) now
175
+ count as differing. The unit is read from the text shown, so a rate
176
+ shown as 0.1200 does not pass on 0.1204, nor 0.0000002 on 0.0000001.
177
+ - `saffron accept --all` now holds a `Then` recorded again with another
178
+ pattern (`^Paid$` become `^Pending$` or `^.*$`), another attribute or
179
+ style property, another response URL pattern, a URL check turned from
180
+ "contains" to "excludes", or another compared capture, under "check
181
+ weakened:". Only values, statuses, URLs and JSON checks were compared.
182
+ The same pattern with an anchor gained, or grouped another way, is not
183
+ held.
184
+ - Database checks: a query whose WHERE picks one of its table's own rows
185
+ by a subquery (`WHERE id = (SELECT max(id) FROM orders WHERE ref = $1)`,
186
+ `created_at >= ALL (...)`) is read as cutting the rows down, as `LIMIT 1`
187
+ is: a count of the rows or "no second row" under it is refused, and a
188
+ repair may not add it. A subquery on another table is left alone.
189
+ - Database checks: a "not in the database" check refuses a bind of spaces
190
+ alone, as an empty one, and a bind that is a whole number past 2^53,
191
+ which a recording had already rounded to a neighbouring id (bind it as
192
+ text).
193
+ - Database rows shown in a failed check and in the report hide a column
194
+ named as a credential (`api_token`, `password_hash`), as a response
195
+ body's fields are hidden. `SELECT *` on a users table put its hashes and
196
+ tokens in the report. A name ending in `hash` or `digest` after a
197
+ credential's (`passwordHash`, `token_digest`) now names a credential
198
+ everywhere.
199
+ - A repair of a database check bound to a number an earlier step kept
200
+ (an id) is no longer refused: the recorded bind carries its type, which
201
+ a repair cannot write, so the two never compared equal. The recorded
202
+ bind is the one kept, and the one the repair is checked with on the
203
+ live data before its proposal is filed.
204
+ - MongoDB, beyond the first entry above: a whole document or list compared under
205
+ `$eq`, `$ne`, `$in`, `$nin` or `$all` is refused, as it is written
206
+ plainly; a projection or a set path through `$`, `$[]` or `$[name]` is
207
+ refused (name a list's item by its number, `items.0.qty`); `$0` is no
208
+ placeholder; a `:date` that is no day of the calendar (`2026-02-30`,
209
+ hour 24) is refused instead of read as the next day. A "not in the
210
+ database" check is judged on each bound field alone as well, so two
211
+ binds sent as the wrong type at once no longer pass, and a check that a
212
+ field is missing is refused under a projection at any depth. A write
213
+ whose documents cannot be read back is undone at once (inserted
214
+ documents deleted, set paths put back) instead of left for a cleanup
215
+ that never heard of it. An update that set an item past the end of a
216
+ list puts the list back whole, also when it changed an item of that
217
+ list in the same update, and a path named as an object's own
218
+ member (`constructor`) is removed again.
219
+ - MySQL: options in `db.url`'s query can no longer turn the adapter's own
220
+ off. `?multipleStatements=true` or `?decimalNumbers=true` in a URL
221
+ shared with the app won over them.
222
+ - Exported tests hide a credential field in a response body they quote
223
+ and never quote a credential header's value, and read a token pair's
224
+ access token as a credential, all as replay does.
225
+ - The reset commands (`reset` in the config) no longer run before a
226
+ pending proposal is only replayed: every later Examples row, and a
227
+ scenario waiting for review, reset the project's data and then recorded
228
+ nothing. They run when a recording really follows.
229
+ - `saffron author` with the Antigravity provider no longer fails at "the
230
+ session's turn limit" after the draft came back.
231
+ - In a heal, an assert, a watch or a network wait the agent makes before
232
+ its first step, or under a step that had run, is answered as not
233
+ recorded, with why. It was answered "Assertion recorded.", and surfaced
234
+ only as a refused finish.
235
+ - `saffron diff --html <file>` no longer writes the review page over the
236
+ file named after it. `--html` takes its file only as `.html`: a proposal
237
+ or feature file there is a file to review, and the page goes to its
238
+ default place. Before, a recording that cost AI usage, or a feature
239
+ file, was replaced by the page.
240
+ - `saffron report --merge` folds the last report's rows in only for
241
+ shards of a run that named its scenarios (`file:line`, `file:name`).
242
+ Shards of a `--filter` or path run were read the same way, so
243
+ `latest.json` held another run's scenarios and reds.
244
+ - `saffron status` (and the dashboard and the editors' panels) reads a
245
+ last report whose rows are not rows as no report, instead of crashing.
246
+ - `saffron eject` refreshes its own specs when they have CRLF line
247
+ endings (a Windows checkout), instead of keeping them for good as
248
+ edited.
249
+ - `saffron accept --all` and `reject --all` exit 1, saying it once, when
250
+ a pending proposal cannot be read.
251
+ - Config: `actionTimeoutMs: 0` is refused (Playwright reads 0 as no
252
+ timeout; here every check timed out at once), and so is a `transport`
253
+ that is not an object or `capabilities` given as a list.
254
+ - `--filter` refuses `&&`, `!` and `~` from other tools instead of reading
255
+ them as tags: `"@a && @b"` ran `@a` or `@b`.
256
+ - An Outline with thousands of rows (an Examples data file) no longer
257
+ costs memory by the square of its rows: 20,000 rows ran out of memory.
258
+ - Found in a further review, before release:
259
+ - A "not shown" check, and a check that the URL does not hold a text,
260
+ passed on the page a click was about to leave (a redirect a moment
261
+ after the click, to a page that answers slowly). The moment a passing
262
+ state must hold is now timed between two looks, and the URL is read
263
+ once a navigation on its way has arrived.
264
+ - A Set-Cookie pattern is run only on the cookie it names. Written with
265
+ no `^`, or with a second branch (`^id=.*Secure|HttpOnly`), it held on
266
+ another cookie. Exported tests check it the same way.
267
+ - An Outline with several Examples blocks: a `<column>` that only
268
+ another block has is an error. Before, the block without it replayed
269
+ with the text `<column>` as written.
270
+ - "The price differs from before" no longer passes when the element
271
+ stopped showing a price ("Price unavailable"): a price against a text
272
+ that is no amount is refused. Exported tests refuse it the same way.
273
+ - A row value followed by a point or comma at the end of a selector's
274
+ text (`"Qty <n>."`) is kept whole: row 1 no longer finds `Qty 1.5 kg`.
275
+ A minus written as another hyphen (U+2010, U+2011) is read as one.
276
+ - "Not shown" finds text the page writes in the other Unicode form (`é`
277
+ as `e` and an accent).
278
+ - A click whose handler keeps the page busy past the action timeout is
279
+ not clicked again by `retries`.
280
+ - Oracle: text is bound as `CHAR`, so it matches a `CHAR` or `NCHAR`
281
+ column as a literal does. Before, `code = $1` found no row of a
282
+ `CHAR(10)` column, and "not in the database" passed with the row
283
+ there. A `NUMBER` reads as a number whatever its decimals (`12.5`
284
+ read as text, `12` as a number, so no one check served both rows of
285
+ an Outline); only one past 15 digits reads as text. `SELECT UNIQUE`,
286
+ `NEXTVAL` in a check, and the `UTL_` and `DBMS_` packages are refused.
287
+ - Database checks: more aggregates are read as the one row they always
288
+ return (`json_group_array`, `total`, `median`, `jsonb_agg_strict`,
289
+ `corr`, `regr_*`, `range_agg`, `APPROX_COUNT_DISTINCT` and the like),
290
+ so a count or an exists on that row is refused.
291
+ - SQLite: a write to a table declared `ON CONFLICT REPLACE` is refused
292
+ while cleanup is on, as it deletes the row in its way for good. A
293
+ "not in the database" check that sends `true` or `false` as text
294
+ against a boolean is refused, as a number sent as text is.
295
+ - MariaDB: `/*M! ... */` comments, which MariaDB runs, are refused as
296
+ `/*! ... */` is.
297
+ - DynamoDB and Firestore: a "not in the database" check is red on an
298
+ attribute no item has that the filter needs to exist (`$exists`
299
+ true), and on a set or a list compared with a value by equality.
300
+ - MongoDB: a "not in the database" check is red on a field no document
301
+ has in a range, `$exists` true or `$elemMatch` too; a document found
302
+ as the bind is sent is the check's own red, with no type blamed; a
303
+ seed under a number `_id` past int32 no longer fails after writing
304
+ its document; `"$1:uuid"` binds a UUID, which reads as its dashed
305
+ text; and a suite whose MongoDB is down waits 2 seconds for each
306
+ scenario after the first, not 10.
307
+ - A response body's credentials stay hidden when a masked number left
308
+ the body no JSON (`"id": {env:USER_ID}`); before, every token in that
309
+ response stayed in the transcript. A credential repeated inside
310
+ another text (a link, a hint) or in JSON kept as text is hidden too.
311
+ `auth_key`, `authKey` and `X-Auth-Key` are read as credentials, and a
312
+ key in a base URL's query no longer reaches the agent's prompt.
313
+ - A contract's `pattern` that only JavaScript's unicode flag refuses
314
+ (`[\w-\.]`, `\_`) no longer makes every schema check fail to compile,
315
+ and an operation is found by a path the URL encodes (`/price list/{id}`).
316
+ - Masking no longer takes seconds on one long run of letters joined by
317
+ `-` or `.`, and a request with a large body is masked once, not per
318
+ field.
319
+ - A repaired database check's proposal is replayed on later runs and
320
+ on an Outline's other rows. Before, the agent repaired it again on
321
+ every run, and the other rows went red.
322
+ - A pending heal proposal is replayed once the failed replay's own
323
+ database rows are undone, so a `Given` that seeds a row under a
324
+ literal unique value no longer makes it "not replay" and heal again
325
+ on every run.
326
+ - A heal whose proof marks an action as opening a tab files a proposal
327
+ bound to the committed recording as its file holds it. Before, it was
328
+ stale the moment it was filed: `saffron accept` refused it and the
329
+ next run healed again.
330
+ - A heal that changed nothing shows the step the committed recording
331
+ failed on, and names it in the error. Before, every step read as
332
+ passed on a red.
333
+ - A proof of refined steps that cannot open stops the scenario on that
334
+ error. Before, the refined steps were filed with the first proof's
335
+ failure.
336
+ - A browser that dies while a recording is set up is no longer reported
337
+ as the AI provider's fault ("not run: provider").
338
+ - `saffron accept --all` holds more checks recorded again looser: a
339
+ longer time limit, another header, schema or JSON field of a group, a
340
+ group check that gained a filter, another bind or query of a database
341
+ check, and a URL pattern that lost its `^` or `$`.
342
+ - A scenario with no name is an error. It was recorded, then its
343
+ recording and proposal could never be read.
344
+ - A fixture's proposal can apply its feature edits
345
+ (`accept --with-feature-edit`).
346
+ - A URL holding a row's value as the browser writes it
347
+ (`Smith,%20John`) is recorded as its `<column>`, so each row checks
348
+ its own; a recording that kept the recorded row's value there is
349
+ recorded again.
350
+ - The MCP `verify` tool runs one call at a time: two at once could
351
+ answer with each other's scenarios. The MCP and language servers
352
+ follow a `.env` that changed while they run.
353
+ - `run --plan` no longer says a MongoDB, DynamoDB or Firestore insert is
354
+ kept under `db.cleanup: "inserts"`.
355
+ - `saffron coverage --fail-on-loss` (or `--markdown`) with no `--base`,
356
+ and `saffron report --strict` with no `--merge`, exit 2 instead of
357
+ passing over the flag and exiting 0.
358
+ - `saffron init` leaves a `CLAUDE.md` that only reads `AGENTS.md` in
359
+ (`@AGENTS.md`) as it is, where it wrote a second copy of the block,
360
+ and adds to a `.gitignore` in the file's own line endings.
361
+ - Found in a third review, before release:
362
+ - Ctrl-C while the agent records starts no new agent session, ends the
363
+ one running, and files no proposal for the scenario it was on. A run
364
+ stopped by its terminal closing (SIGHUP) stops as Ctrl-C does: it
365
+ writes its report, runs `teardown` and exits 129.
366
+ - `--rerecord` sets each recording aside instead of deleting it, and
367
+ puts it back when the run is killed before recording it again. The
368
+ next run puts back what a killed run left aside, unless a recording
369
+ or proposal for that scenario was written since.
370
+ - A fixture the replay phase ran is not run again when a scenario of
371
+ its feature goes on to the agent.
372
+ - Two runs writing `.saffron/reports/latest.json` at once no longer
373
+ lose each other's scenarios.
374
+ - `saffron import` refuses a bundle whose evidence or proposal would be
375
+ written through a symbolic link out of `.saffron`.
376
+ - Recording: a response field or header checked against the value that
377
+ same response gave (extracted in the When) is refused, as it holds
378
+ whatever the API answered; so is a header checked against a value
379
+ extracted in the same Then. A group's status check names a status
380
+ the step names, as a status check does.
381
+ - Recording: an `{env:...}`, `{data:...}` or kept value that is also a
382
+ column's name is never written inside its `<column>` placeholder,
383
+ where replay typed the literal text `<user>`. A `{data:...}` value
384
+ inside a password no longer leaves the rest of the password in the
385
+ recording. A capture whose name holds an env value is renamed with
386
+ its references, and a request's extract names are renamed too.
387
+ - Recording: a kept value of tens of thousands of characters no longer
388
+ makes every later request fail to record. An action no saved
389
+ recording could load is refused when recorded: a capture named `""`,
390
+ a blank text check (an empty field is checked by its value), a
391
+ status written `" 404"`. A status class written `4XX` is kept as
392
+ `4xx`. A row value a JSON check's pattern holds escaped is its
393
+ column, as in other patterns. A time limit written in words ("within
394
+ two seconds") is the bound it names.
395
+ - Exported tests check as replay does: a pattern or capture never reads
396
+ across a block's edge ("Qty 1" over "0 left" is not "Qty 10"), a
397
+ download holds the value as written, never across a line, "not
398
+ shown" reads words as replay reads them, a role's name in another
399
+ case counts as shown and an option a closed dropdown does not show
400
+ does not, credentials in a response are hidden and a URL's secrets masked as
401
+ replay does, a captured credential encoded in a URL goes to the APIs
402
+ under test only, and a multipart request never sends
403
+ `saffron.config.json`. A check on a target recorded outside an
404
+ iframe is exported as stricter, as the export looks for it in the
405
+ page only. `saffron eject` warns that `setup`, `teardown` and `reset`
406
+ are not run by the exported tests, and a data list the data does not
407
+ hold refuses only the scenario that reads it.
408
+ - Database checks: `count(*)` reads as text on PostgreSQL, MySQL,
409
+ MariaDB and CockroachDB, and as a number on SQL Server, Oracle and
410
+ SQLite, as the guide now says. Oracle refuses more network and
411
+ scheduling packages (`UTL_URL`, `DBMS_LDAP`, `HTTPURITYPE` and the
412
+ like). SQLite's refusal of `true` or `false` sent as text says to
413
+ send 1 or 0.
414
+ - `saffron accept --all` holds a check recorded again without its
415
+ contract check, or on another request group.
416
+ - Found in a fourth review, before release:
417
+ - A replay that failed, then passed when the recording was replayed
418
+ again for a heal, is red with a note that it failed on replay and then
419
+ passed, where it was green. Without an agent it is not replayed again.
420
+ - Database checks: a "not in the database" check is judged again with
421
+ its binds as the other types the database may keep them as, on every
422
+ engine: on MongoDB, DynamoDB and Firestore a range's two ends and the
423
+ values of `$all` together; on PostgreSQL, CockroachDB, MySQL, MariaDB
424
+ and SQLite, text that reads as a number, `true` or `false` as 1 or 0,
425
+ and the values inside a JSON bind. A `count(*)` checked as 0 on SQLite
426
+ and an "at most" count are judged too. Each held with the row there.
427
+ - MongoDB: `$elemMatch` on a field held as one value is refused in a
428
+ "not in the database" check; so are, on DynamoDB and Firestore, a list
429
+ or set compared by `in`, `array-contains`, `$in` or `$beginsWith`, and
430
+ one value compared by `array-contains`. Each held whatever the item
431
+ held.
432
+ - MongoDB, DynamoDB and Firestore take a `:binary` bind (`0x` hex), so
433
+ a field held as bytes can be checked. MongoDB and DynamoDB take a null
434
+ bind, which finds a field held as null, where it was refused and the
435
+ agent was told to write `$exists: false`, which never finds one.
436
+ - DynamoDB: a probe value DynamoDB will not compare (BETWEEN a number
437
+ and a text) no longer fails a valid check. With `db.url` naming a
438
+ region, an endpoint set in the environment (`AWS_ENDPOINT_URL_DYNAMODB`)
439
+ or a profile is no longer used. On DynamoDB and Firestore, a read that
440
+ runs past its time on a database that answers is the check's red,
441
+ never an outage.
442
+ - SQL: in a "not in the database" check, a `LIKE` bind holding a
443
+ backslash (a `[` on SQL Server) is refused, as it never finds the text
444
+ it holds. `$0` is refused. The latest row written as an anti-join, a
445
+ `max()` read through a derived table, and SQL Server's `FOR JSON` and
446
+ `FOR XML` count as keeping one row. On SQL Server, an alias that is a
447
+ word a statement starts with (`SHUTDOWN`) is refused, and a date
448
+ written as text reads year, month, day whatever the login's language.
449
+ Oracle refuses every `UTL_` and `DBMS_` package called (`DBMS_LOB`
450
+ aside) and `APEX_WEB_SERVICE`, as the guide says.
451
+ - MySQL and MariaDB: an `UPDATE` that changed rows its `WHERE`, read
452
+ first, did not find (a `COLLATE` the snapshot lost) is rolled back,
453
+ where cleanup left them changed; a write waiting on a table's lock
454
+ gives up at the step's time, where it waited a year; an `UPDATE` of a
455
+ `FLOAT` under cleanup is refused, as the driver reads it to 6 digits.
456
+ - Schema repairs: a `count(*)`, `LIMIT` or `DISTINCT` check can be
457
+ repaired (each repair was red); a repair of a step's second or later
458
+ check is taken from the repaired check alone, as the agent is told to
459
+ record it; a check that expects no rows, which no repair could be held
460
+ to, starts no agent session.
461
+ - A recording made again (`--rerecord`, or a stale one) whose step now
462
+ writes to the database is held for review against the one it replaces,
463
+ and a first recording whose When writes a table a later Then reads is
464
+ held too. Its note keeps the name `database write added by the heal:`,
465
+ so Accept All in an editor holds it as before. The block `saffron init`
466
+ writes into `AGENTS.md` says so to agents: a `When` acts through the
467
+ app, never the database.
468
+ - Recording: a Then that checked the very field the When had typed into,
469
+ for the value typed and with no page loaded since, passed whether or
470
+ not the app saved it: it is refused, and the agent checks where the
471
+ app shows the saved value, or the field once the page loads again. An
472
+ item or every check that compared a list's entries with a value taken
473
+ from that same list is refused, as the same check on the taken field
474
+ was. A database check bound to a `{unique:...}` no Given or When
475
+ writes, a `{data:...}` no step writes, or a `<table:key>` no table
476
+ holds is refused (a misspelt `{unique:email}` held "not in the
477
+ database" for good).
478
+ - Recording: a time limit in words or in parts is the bound it names:
479
+ "twenty-five seconds" was read as 5 s, "1 minute 30 seconds" as 60 s,
480
+ and "within a second" or "half a minute" as no limit. A response header
481
+ kept as `Header:Location` is kept as `header:Location`, which the
482
+ recording format loads. A group check filtered by a status the step
483
+ does not spell out is recorded: only a status a check expects must be
484
+ named.
485
+ - Recording: a tab closed or opened under a Then was recorded and not
486
+ held, so "the payment window closes" passed when the agent closed it:
487
+ it is held for review, as a click there is. A value of tens of
488
+ thousands of characters (an upload's base64, a long `{data:...}` text)
489
+ made recording fail with "Regular expression too large" after its
490
+ request was sent: it is compared as it stands. Two writes a step asks
491
+ for to one operation were kept as one when they sent a form, a
492
+ multipart, or JSON bodies with other fields (`{sku, quantity}`, then
493
+ `{sku, amount}`): only a field renamed, retyped or added to the same
494
+ values marks a corrected attempt. A capture named with a space whose
495
+ name held an `{env:...}` value had its references garbled: they follow
496
+ its new name. An Outline's database or response field check that
497
+ states a value of the row an earlier step wrote, read through
498
+ `{captured:...}`, follows the row, where every other row was refused.
499
+ - Browser checks: a text check read the items of a flex row, a grid or
500
+ floated boxes a space apart, so "$15" passed on a price drawn as `$15`
501
+ `.99`, and "$50.00" on `-` `$50.00`: a number that goes on across items
502
+ reads as one, and "$12.50" holds on `$` `12` `.50`. Text drawn with no
503
+ area (font-size 0, or in a box scaled to nothing such as a closed
504
+ menu's `transform: scaleY(0)`) counted as shown, so "I see Delete
505
+ account" and "Delete account is not shown" both passed on one page: it
506
+ counts as not shown, and the failure says why. A page rewriting its
507
+ history every few hundred milliseconds kept every settled check from
508
+ ever passing: only a navigation to another document starts the hold
509
+ again. Exported tests read the same.
510
+ - An Outline row value before a point in an XPath function replay
511
+ cannot tie (`contains(translate(., ...), 'qty <n>.')`) is refused, as
512
+ it was without the point, where row 1 passed on "Qty 1.5 kg". An
513
+ Outline whose database or API check names a column but follows nothing
514
+ of it is refused in that check's own words, where it was told to show
515
+ the value on the page.
516
+ - `saffron accept --all` holds a Then recorded again on a wider element
517
+ when only its selectors, its role or its frame changed (a total read on
518
+ `body`, a "not shown" that lost its payment frame). A hand-edited
519
+ recording with an HTTP field on a browser action is refused by name,
520
+ where the staleness check crashed on it.
521
+ - API steps: a request carried an `{env:...}` value or a captured
522
+ credential to another host when it came inside a captured value (a
523
+ `Location` that echoed `?key=` back on another host, captured and
524
+ followed): it is refused, in replay and in exported tests. An exported
525
+ test sent a captured credential written as a query value (`s.urlValue`)
526
+ to another host: it is refused, as replay refuses it. A Set-Cookie
527
+ pattern with an unescaped dot in the cookie's name ran on another
528
+ cookie the dot also matched: the name is read as written. A schema
529
+ check passed on a JSON body served as `text/plain`, a type the
530
+ contract does not give that response: it is red, naming the type.
531
+ - `run --plan` and the MCP `run_plan` tool plan a stale recording with a
532
+ pending proposal as the proposal replay the run makes, where they said
533
+ "record again (AI)". A `baseURL` or `--base-url` that is not an
534
+ `http://` or `https://` URL is refused, naming the key, where every
535
+ API scenario failed with "Invalid URL". `saffron import` refuses to
536
+ write through a `.saffron`, `proposals` or `artifacts` folder that is
537
+ itself a symbolic link.
538
+ - The language server marks a `{unique:...}` that is no token
539
+ (`{unique:e mail}`, `{unique:}`) where it is written, as `saffron run`
540
+ refuses to start for it; JetBrains showed nothing until the run.
541
+ - Found while publishing, and in the review that followed:
542
+ - Playwright colours its error log wherever `TERM`, `COLORTERM`,
543
+ `FORCE_COLOR` or the CI vendor (GitLab CI, CircleCI) give colours, also
544
+ into a pipe or a log file and with `NO_COLOR` set. GitHub Actions and a
545
+ shell without `TERM` get none, which is where this project's tests ran.
546
+ Saffron read that text as it came:
547
+ - The rules that keep an interaction from being sent twice matched
548
+ nothing with the codes in it, so a click whose handler kept the page
549
+ busy past the action timeout was clicked again by the retry.
550
+ - A typed `{env:...}` value that holds a line break (a PEM key, a JSON
551
+ service account) was written in clear to the HTML and JSON reports
552
+ when its fill failed, and was shown to the recording agent, in the
553
+ error's own text where its line breaks stand as typed: the codes
554
+ inside the value kept it from being masked there. A value of 3 to 5
555
+ characters right after a code was left as well.
556
+ - Every failed step's error carried the codes into reports and logs.
557
+
558
+ Playwright's colours are now turned off for the run, and every error is
559
+ read, masked and reported without the codes. The tests run with colours
560
+ on.
561
+ - Known limit, as in 1.4.0: a typed `{env:...}` value of several lines (a
562
+ PEM key, a pasted JSON key file) is masked where its line breaks stand
563
+ as typed, JSON-escaped, percent-encoded or inside base64, and not where
564
+ they are written another way: as single spaces, as a page shows the
565
+ text and the recording agent's snapshot reads it; as CR LF, as a form
566
+ post sends it (`%0D%0A` in a form body); as `↵`, in Playwright's call
567
+ log. There it reaches the agent, the terminal, the reports, the history
568
+ and the trace. Do not type a secret of several lines into a field in a
569
+ scenario. A one-line value with a tab or two spaces in a row is not
570
+ masked where a page shows one space there.
571
+ - A value inside base64 was missed when other text was glued before the
572
+ encoding by `/`, `-` or `_` (`/v/<base64>` in a path, `sess-<base64>`
573
+ in an id), or when the encoding's `+`, `/` or `=` were written `%2B`,
574
+ `%2F` or `%3D` in a query string: the trace kept it and the recording
575
+ agent saw it. Such a run is decoded from after each of those characters
576
+ too. Also in 1.4.0.
577
+ - Outline rows: a check that something is no longer shown, in a step that
578
+ does not name the date column, that held the recorded row's day by its
579
+ month's name without the year ("Nov 20", "20 November", "Fri, Nov 20"
580
+ for a cell `2026-11-20` or `{date+5}`) was bound to no row. Every other
581
+ row then asked whether row 1's day was gone, which holds whatever became
582
+ of its own entry. Such a check binds the recording to its row: the other
583
+ rows are refused, with the step to change. A recording made before this
584
+ is recorded once more on the first row with another day. Also in 1.4.0.
585
+ - A database check that a value "differs" passed on the unchanged value
586
+ when the database reads the column as text (a PostgreSQL `numeric`,
587
+ `bigint` or `count(*)`, a MySQL `DECIMAL` or `BIGINT`): `12.50` was
588
+ never the text `12.5`. Text that spells a number is now compared as the
589
+ number, with no type or with `"type": "number"`; `"type": "string"`
590
+ compares the text. A "differs" check typed as a number or a boolean on a
591
+ value of another type is refused, since it held whatever the row held.
592
+ A text column that holds digits is compared as numbers too (`1.10` is
593
+ "the same" as `1.1`, `5` as `5.0`), except a code written with a leading
594
+ zero on either side (a zip `1234` differs from `01234`): where the text
595
+ itself is what matters, give the check `"type": "string"`. Also in
596
+ 1.4.0.
597
+ - An "at most" database check (`count(*) AS n` with `max`) passed with any
598
+ number of rows there when its bind was empty or null (an Outline row
599
+ with an empty cell, an empty capture): the count was 0. It is refused,
600
+ as a "not in the database" check is. To count rows whose column is
601
+ rightly empty, compare its length with the number 0
602
+ (`length(note) = $1`: run on PostgreSQL, MySQL, MariaDB and SQLite; on
603
+ Oracle an empty text is NULL, so write `IS NULL`, beside a column
604
+ compared with a bind). Also in 1.4.0. The
605
+ refusal names the way out of its own engine: it named
606
+ `length(note) = $1` on Oracle too, where an empty text and its length
607
+ are NULL, so the check it led the recording agent to counted no row and
608
+ held whatever the table held. On Oracle it says `IS NULL` beside a
609
+ bound column, on SQL
610
+ Server and MongoDB it names no function they do not have, and a null
611
+ bind is called the "at most" check it is.
612
+ - MySQL and MariaDB: a "not in the database" or "at most" check that
613
+ binds an unquoted `true` where the rows are found by its text (a
614
+ `BOOLEAN` or number column holding false or 0, which MySQL reads the
615
+ text as, or a text column holding 'true') was told only that the
616
+ column holds the step's value as another type. It is told what to
617
+ send: 1 or 0 for a `BOOLEAN` or number column (a `CAST` of the column
618
+ to text holds there too), the quoted text for a text one.
619
+ Such a check on a `BOOLEAN` or number column whose row holds false or
620
+ 0 is refused though it is right, where 1.4.0 passed it on MySQL: bind 1
621
+ or 0.
622
+ - `npm install saffron-ai` no longer fails with ERESOLVE in a project
623
+ whose own app depends on another version of a database driver (MongoDB
624
+ 6, Firestore 8 or 9, Oracle 5 or 26, a pinned AWS SDK, mssql 11, mysql2
625
+ 3.11): the optional drivers accept any version at install, and the
626
+ version is checked where a database step loads the driver. Another
627
+ major version than the tests run is refused with the engine, the major
628
+ needed and the version found ("database steps on MongoDB need mongodb
629
+ 7; this project has 6.22.0: npm install mongodb@7"). The messages for a
630
+ missing driver name the version to install.
631
+ - An interaction that reached the page is never sent again by a retry: a
632
+ select, a fill, a hover, a drag, an upload, and a key press (a captured
633
+ credential's own included) whose handler kept the page busy past the
634
+ action timeout ran it twice (a dropdown that submits its form sent it
635
+ twice, a drop was dropped twice). It fails as it is, as a click does. So
636
+ does a drag whose drop target came only after the timeout, the mouse
637
+ being down already. An element that is only covered, disabled, hidden,
638
+ read-only or not there yet is still tried again. A step that passed only
639
+ on such a second try is now red: give the page more time with
640
+ `actionTimeoutMs`.
641
+ - A "not in the database" check, and an "at most" check, no longer pass
642
+ with the row there when the read that judges them (the same query with a
643
+ bind sent as another type) fails for a reason that is not the type: the
644
+ app's lock on a row only that read reaches, a timeout, a deadlock or a
645
+ lost connection. Only the database refusing the value as that type counts
646
+ as "not kept as that type"; a lock is read again and is red if held to
647
+ the end of `db.timeout`, and any other error fails the check. Seen on SQL
648
+ Server with a bytes column and a join.
649
+ - An exported Playwright test printed the `{env:...}` value it typed, the
650
+ element it named or the URL it opened when that action failed (a
651
+ disabled or missing field), in the terminal and the CI log, with the
652
+ line, list, dot, json and junit reporters too. Exported actions on such
653
+ a value, or on a captured credential, now run through the helper's
654
+ `masked`, as the checks do, and name the token. Export again to get it.
655
+ (Also in 1.4.0.)
656
+ - A pattern check of an API response, a header or a database row that
657
+ holds `{unique:<column>}` (which recording writes for an Outline) was
658
+ refused on every row with "A regex quantifier cannot contain a row or
659
+ data token": each row is checked against its own generated value.
660
+ - Known cost of the wait a check makes for its action's answer (new in
661
+ this release, below): a page whose latest action left a request open (a
662
+ long-poll or a stream started on load, with the checks straight after
663
+ the page load) makes each of those checks wait half the action timeout,
664
+ 3 seconds at most, and hold for half a second: four checks take about
665
+ 12 seconds. In replay the checks after the next interaction do not wait
666
+ for it. In an exported test a count, "not shown", "differs", "URL does
667
+ not contain" or response check on such a page waits that long wherever
668
+ it stands, as in 1.4.0, and an exported test of nine checks that wait
669
+ fails on Playwright Test's 30 second test timeout. An answer slower
670
+ than that wait is not waited
671
+ for by a single check, whatever `actionTimeoutMs` is: write the wait
672
+ into the step ("When I pay and wait for the payment API to return
673
+ 402"), then check what the answer shows before what it must leave as it
674
+ was.
675
+ - A key press with no target ends at the action's timeout: on a page whose
676
+ key handler never returns the run hung with no verdict and no report. It
677
+ fails saying the page did not answer, and the key is not pressed again.
678
+ - A run whose output goes away (`saffron run | head`, a pager quit) no
679
+ longer dies with exit 1 and a stack
680
+ trace before its teardown and report: it goes on without output, runs
681
+ `teardown`, writes its report and exits with its own verdict. Known
682
+ limit: when the error stream is gone too (`saffron run 2>&1 | head`), a
683
+ `setup`, `reset` or `teardown` command that prints fails on its own
684
+ write, so a
685
+ `setup` that prints stops the run and `echo done && cleanup` does not
686
+ reach the cleanup. Send such a hook's output to a file.
687
+ - With `--browser webkit`, an app that is not running is counted as
688
+ blocked by the environment, as in Chromium and Firefox. It was counted
689
+ as a failure in the app, and with an agent on, a heal session started
690
+ for every scenario.
691
+ - Database outages said as they are: SQL Server with a certificate the
692
+ machine does not trust names the certificate and
693
+ `?trustServerCertificate=true`, where it said the database did not
694
+ answer; an Oracle host that does not resolve is the database not
695
+ answering; a missing SQLite file under a folder with a bracket in its
696
+ name is the environment; an error the app's own trigger raises that says
697
+ "Connection lost" is the statement failing.
698
+ - `saffron runs`, `import --run`, `report --run` and `trace <run>` no
699
+ longer fail when the shell exports `CLICOLOR_FORCE=1` or `GH_FORCE_TTY`:
700
+ gh and az are run with colours and the forced terminal off.
701
+ - `saffron steps` marks a divergent step `◐`, where it shared `●` with a
702
+ recorded one and differed only by a colour that a pipe, a log file and
703
+ an AI agent's shell never get. The language server's hover does too.
704
+ - `saffron init` reads end of input at its question about the examples
705
+ (Ctrl-D, a closed input) as "no" and goes on, where it ended with exit 0
706
+ and nothing set up; it no longer asks when `CI` is set; it says
707
+ `updated` for a `.gitignore` that was already there.
708
+ - `saffron mcp` typed into a terminal says it is a server for an AI
709
+ assistant and how to add it, and exits 2, where it printed nothing and
710
+ waited. The `--filter` refusal no longer ends with two full stops, and a
711
+ recording's target with nothing to find it by is told
712
+ `fallbackSelectors`, the field's name.
713
+ - Playwright 1.65.1 (from 1.64.0; the range is now `^1.65.1`). Run
714
+ `npx playwright install` after upgrading: it brings Chromium 159,
715
+ Firefox 159 and WebKit 27.2. A project with its own `@playwright/test`
716
+ or `playwright` updates that to the same version first
717
+ (`npm install -D @playwright/test@1.65.1`, or
718
+ `npm install -D playwright@1.65.1` where the project depends on
719
+ `playwright`): beside an older one npm
720
+ installs a second Playwright for Saffron, `npx playwright install`
721
+ installs the browsers of the project's own, and the run goes on
722
+ saying a browser is missing. Its trace no longer says which load state a
723
+ wait waited for, so the trace view names it "Wait for load state".
724
+ Playwright 1.65 itself waits for an element inside an `inert` part of
725
+ the page to stop being inert before acting, leaves text the browser does
726
+ not draw (a closed `<details>`) out of accessible names, and matches
727
+ cookies to paths as RFC 6265 says (`Path=/api` no longer goes to
728
+ `/apiv2`): a recording that relied on the old behaviour turns red. The
729
+ recording agent's browser still names elements as Playwright 1.62 does,
730
+ so a new recording can hold a name with such undrawn text too: its
731
+ action or positive check is red on replay, and a "not shown" check on
732
+ that name passes whatever the page shows. Target such an element by a
733
+ test id or a selector.
734
+ - The release gate (`npm run gate`, which `npm publish` runs): the unit
735
+ tests, every database engine one file at a time (it fails where Docker
736
+ does not answer, where it skipped them all), then both end-to-end
737
+ scripts on the bundle that is published.
738
+ - Found in a fifth review, before release:
739
+ - Database checks: a "not in the database" check held whatever the
740
+ database held when the value was kept as a type its bind was never tried
741
+ as: a JSON string, for a bind the statement reads as JSON
742
+ (`data->'ref' = $1`, a `jsonb` column, MySQL's `CAST($1 AS JSON)`), as
743
+ the text 4471 read as JSON is the number; a JSON boolean, for a MySQL
744
+ bind of 1 or 0, as MySQL compares a JSON true with TRUE and never with
745
+ 1; and bytes, on every SQL engine, for the `0x` hex a row shows them as.
746
+ Each is tried now, so such a check is red, saying what found the rows
747
+ and how to compare: as text (`data->>'ref' = $1`), `true` or `false`
748
+ unquoted for a JSON boolean (a bind of "true" was told to send 1 or 0)
749
+ and 1 or 0 for a `BOOLEAN` column, the column's hex on that engine. A
750
+ bind the SQL makes JSON of itself (`to_jsonb(CAST($1 AS text))`,
751
+ `JSON_QUOTE($1)`) is refused in such a check.
752
+ - Database checks: a bind only a negation compares (`<> $2`, `NOT IN`,
753
+ `$ne`, `$nin`, `!=`, `not-in`) made a right "not in the database" check
754
+ red, saying to send that bind as another type, which would have broken
755
+ it: it is left as sent, since sent as another type it finds more rows,
756
+ never fewer. On Firestore a query that finds documents as sent blames no
757
+ type for them. A type probe that met the app's lock on the rows (a busy
758
+ SQLite file) failed the check at once: the check reads again, as its
759
+ read loop does.
760
+ - Database checks: a check that a column is not in a row (`/0/discount`
761
+ exists false) on a SQL engine held whatever the column held once the
762
+ `SELECT` left it out or misspelt it, and failed whatever it held once
763
+ the `SELECT` named it: it is refused, saying to write
764
+ `discount IS NULL`; a field inside a JSON column is still checked so. On
765
+ MongoDB, DynamoDB and Firestore a field checked as not in a document is
766
+ red when no document of the collection holds it (a misspelt name), as a
767
+ filter's field already was.
768
+ - Database checks: a count, or a check that there is no second row, under
769
+ a `WHERE` that keeps one row of each group
770
+ (`id IN (SELECT max(id) FROM orders GROUP BY ref)`, a
771
+ `row_number() OVER (...)` column compared in the `WHERE`) passed on a
772
+ double submit: it is refused, as under `LIMIT`. A JSON number past 2^53
773
+ (an external id in a `json`, `jsonb` or Oracle `JSON` column) read
774
+ rounded, so "differs" passed on the unchanged value and "equals" was red
775
+ on the right one: it reads as its digits, as MySQL reads it, and
776
+ Oracle's cleanup, which put it back rounded, puts the JSON back as it
777
+ was.
778
+ - Database checks: the repair of a step's second database check after a
779
+ schema change was refused before any session when an earlier check of
780
+ the step, which still runs as recorded, expects no rows or was recorded
781
+ without its row count: only the check that failed and the checks the
782
+ repair changes are held to those rules. A recording whose `Then` holds a
783
+ database write (a hand edit, a merge) loaded and ran it, so the `Then`
784
+ checked what it had just written: it is refused, and so is a database
785
+ check under a `Given` or `When`.
786
+ - Database steps, while recording: during a heal the agent could seed the
787
+ database, or send a request that writes, under a step replay had already
788
+ run: the write ran, then was refused, and the extra row made the
789
+ `Then`'s right check fail. Such a write, and one the recording format
790
+ refuses, is refused before it runs. A check bound to a value of more
791
+ than about 32,000 characters that the steps spell out (a doc string's
792
+ JSON document) could not be recorded, the tool answering "Regular
793
+ expression too large": such a value is found as it stands.
794
+ - Outline rows: a literal date cell (`2026-11-20`) that a check showed in
795
+ the page's own format ("Friday, November 20, 2026", "20/11/2026"), an
796
+ amount cell an element showed with other decimals ("Pay $12.50" for
797
+ 12.5, in a button's name or a text target), and a data row's value with
798
+ outer spaces (`" Oslo "` in a CSV) that the page showed trimmed were
799
+ found by no rule, so every other row checked row 1's value and a "not
800
+ shown" check passed whatever became of its row: the recording is bound
801
+ to its row. The page's ISO spelling of a cell such as `20.11.2026` no
802
+ longer becomes a date that moves with the calendar.
803
+ - Outline rows: a data table cell of three letters or more, or a
804
+ `{data:...}` value of four or more, was written into unrelated words a
805
+ check held ("Pro" in "Products", "Mark" in "Marketing"): after the cell
806
+ or the file changed, or on another row, a check looked for "Maxducts"
807
+ and a "not shown" check passed whatever the page showed. A value is
808
+ written only where it stands as a value, and an edit whose old value
809
+ stays in such a word records the step once more. A `{data:...}` value
810
+ shown inside angle brackets ("Ann <ann@shop.test>") was kept as recorded
811
+ and read as a placeholder, so the recording stayed fresh when the file
812
+ changed and checked the old address: it is written as its reference, and
813
+ a recording made before is recorded again.
814
+ - Outline rows: a data table value equal to the row's value ("alice"
815
+ seeded by a table, and the row's `<user>`) was written as the table's
816
+ cell in a later check ("Signed in as alice"), so every other row checked
817
+ alice's name: it is the row's `<user>`, or kept as recorded and bound to
818
+ its row where the check lists every row's value. A `{unique:<column>}`
819
+ the agent typed as this row's value was recorded as `{unique:alice}`,
820
+ and every other row was refused: it is recorded as the step writes it,
821
+ and each row types its own generated value. Such a recording is format
822
+ version 23, which older runners refuse: they typed the token as text. A
823
+ capture the agent named after the row's value (`alicebooking`) refused
824
+ every other row once a step naming the column read it: a capture's name
825
+ is no row data. An Outline value of about 33,000 characters or more (a
826
+ maximum-length text from a data file) crashed the recording beside an
827
+ attribute check or a response status: it records.
828
+ - Outline rows: a block beside `Examples: {data:roles}` that lacked one of
829
+ the data file's columns ran `<menu>` as written, on the data rows'
830
+ recording, so a row could pass on another row's data (two data files
831
+ missed it the same way when the placeholder was only in a doc string or
832
+ a table), and a placeholder with a space inside its brackets (`<role >`,
833
+ `< role >`) stayed as written on every row, so the rows replayed each
834
+ other's recording. Both are refused when the file is read, naming the
835
+ placeholder.
836
+ - Recording: a `Then` that read back what a `When` had typed was still
837
+ recorded when it checked the text of a textarea, a rich-text editor or a
838
+ select, named the field another way than the fill did (a selector, a
839
+ test id, a name pattern), or checked an Outline row's value the agent
840
+ had typed as its column: each passed whether or not the app saved
841
+ anything. The recording tool decides on the page which field a check
842
+ reads, and refuses these as it refused the value check. A check against
843
+ a value captured in the same `Then` was recorded when it named the
844
+ captured element another way (`[data-testid=total]` for the `h1`
845
+ captured) or a box holding it, comparing the element with itself: it is
846
+ refused, decided on the page too.
847
+ - Recording: a check found by the text it checks, beside a role (role
848
+ `cell`, text "Ada") or a selector scoped only by tags
849
+ (`table td:has-text('Ada')`), held on any cell showing that text; a
850
+ "starts with" or "ends with" check that let any text come between the
851
+ anchor and the value (`^.*ORD`, `2026.*$`) matched the value anywhere; a
852
+ URL check on what the app's base URL holds (`localhost:3000`, its base
853
+ path) held on every page; and a browser response check or network wait
854
+ passed with a JSON pattern matching any text, a status no step names
855
+ (the 500 the app happened to answer), or a class where the step names
856
+ the code (2xx for "201 Created"). Each is refused, the last as the API
857
+ medium's own checks refuse it.
858
+ - Recording: a password typed as `{env:...}` reached the recording agent
859
+ base64-encoded, in a Basic Authorization header that
860
+ `browser_network_request` shows or from page code that encoded the
861
+ field, and reports and traces kept such a value too, outside a trace's
862
+ request headers: a value inside base64 is masked to its token wherever
863
+ text is masked for display. A heal prompt's live tab line showed a page
864
+ URL holding a credential an API step had captured, while the line below
865
+ masked it: the credential is its `{captured:...}` token there and in the
866
+ completed steps. A first recording, not only a heal, filed the agent's
867
+ rewording of a `Then`, which `accept --all --with-feature-edit` then
868
+ wrote into the feature file: a `Then`'s wording is never filed as a
869
+ feature edit.
870
+ - Browser checks: text inside a box closed to no height or no width with
871
+ overflow hidden (a menu or an accordion closed with `max-height: 0`,
872
+ `height: 0` or `width: 0`) counted as shown in every browser, so "I see
873
+ Delete account" and "Delete account is not shown" both passed on one
874
+ page: it counts as not shown, and so do the fields, buttons and list
875
+ items such a box holds, in replay and exported tests alike, while what
876
+ is positioned against a box outside it still counts, as the browser
877
+ draws it. Text in a zero-sized inline `<svg>` is not shown either. A
878
+ target recorded inside an iframe was read as shown while the iframe was
879
+ hidden, zero-sized, off the page or in such a box, and its "not shown"
880
+ check passed too: it counts only while every iframe it was recorded in
881
+ is shown.
882
+ - Browser checks: a check that something is shown or holds a text, a
883
+ value, an amount, a style or an attribute, or that the URL holds a text,
884
+ was judged at its first look, before the request the `When` sent was
885
+ answered, so "the total is still $50.00", "I am still on the sign-up
886
+ page" and an optimistic "1 item" the server refused all passed: when the
887
+ action before it still has a request out, the check waits for the answer
888
+ (half the action timeout, 3 seconds at most: a slower answer is not
889
+ waited for) and passes only on a state that holds for half a second, as a count or a
890
+ "not shown" check does, in replay and exported tests alike. A check
891
+ whose action sent nothing is judged at once, as before.
892
+ - Browser checks: an attribute check in the absence form ("not disabled"
893
+ as `^(false)?$`) passed on an HTML boolean attribute that was there
894
+ (`disabled`, `checked`, `required`, `hidden`, `open` and the like),
895
+ whose empty value the pattern also takes, so "the Save button is not
896
+ disabled" held on a disabled button: such an attribute is set whenever
897
+ it is there, whatever its text. A style check on a shorthand (`outline`,
898
+ `background`) or a font list, recorded in Chromium, was red in WebKit or
899
+ Firefox on a page that looked the same: it is compared by the longhands
900
+ the value sets. Exported tests read the same.
901
+ - Replay: a page whose own script never yields (an endless loop after a
902
+ click) left every check, and the run, waiting for ever, with no red and
903
+ no report: the check fails at its time limit, saying the page did not
904
+ answer, its own script kept it busy, and the run goes on and writes its
905
+ report. A wait whose target has several ways to be found waited the
906
+ whole action timeout on each in turn before failing (four ways, four
907
+ timeouts): they are polled together under one deadline, the first that
908
+ finds an element deciding.
909
+ - API steps: a status a step named only to rule it out ("the API should
910
+ not answer 500", "a 4xx but not 404") counted as named: the agent was
911
+ told to expect it, and a recording that passed only when the API
912
+ answered 500 was kept. Such a status is no longer named, by a request's
913
+ expected status or by a `Then`'s status check. After copies that
914
+ answered otherwise, or a poll that ran out, the agent's next check was
915
+ verified against that unrecorded response, and the reply said the
916
+ request could not be sent: no check reads it now, the transcript marks
917
+ it as not recorded, and the reply says what it answered. A kept value
918
+ was quoted whole in the tool's reply, past the cap on a tool result (a
919
+ list of 6,000 products made a 418 KB reply): it is cut as a body is.
920
+ - API steps: a capture the scenario sent as its credential (an API key
921
+ kept as `key` and sent as `X-Api-Key`, a `Bearer {captured:result}`) was
922
+ a credential only when its name said so: the key reached the transcript,
923
+ the report, error text and the agent, and was sent to another host. A
924
+ capture sent in a credential header, or in a query parameter, form field
925
+ or body field a credential's name names, is a credential whatever its
926
+ name, from the moment it is extracted, in a browser scenario's API
927
+ steps too, where it became one only when a request first sent it, so a
928
+ check failing before then quoted it. An exported test keeps it as one
929
+ too: masked, sent only to the APIs under test, and with the trace left
930
+ out of the generated config.
931
+ - API steps: a schema check ignored a discriminator, so a valid response
932
+ of a `oneOf` picked by `discriminator` (the OpenAPI specification's own
933
+ Pet example) was always red, and such a request body could not be
934
+ recorded: the discriminator's property picks the schema, by its mapping
935
+ or the component's name, in replay and exported tests alike, and a value
936
+ neither names is red. A schema check was red on a response the contract
937
+ allows when its media type had no JSON schema (`"application/json": {}`,
938
+ or a `text/plain` response the contract gives a text schema): a media
939
+ type with no schema leaves the body free (a JSON one must still be
940
+ JSON), and a non-JSON type the contract gives is checked as text against
941
+ its schema. Behind `transport.api.proxy` the API was probed directly
942
+ before an agent session started, so an API whose host resolves only
943
+ through the proxy was never recorded ("did not answer (ENOTFOUND)"): the
944
+ probe goes to the proxy, as every request does, and names a proxy that
945
+ is down.
946
+ - Exported tests: an exported `@api` scenario sent its requests through
947
+ the test's browser context, which Playwright starts with the project's
948
+ `storageState`, so a request whose own token failed passed on the
949
+ signed-in session's cookie, where replay is red: each session of an
950
+ exported `@api` scenario sends through a request client of its own with
951
+ no cookies, as replay does, and a browser action in an `@api` scenario,
952
+ which replay fails, is exported as `test.fixme`. A failed exported
953
+ element check printed the resolved `{env:...}` value it waited for (a
954
+ text, a field's value, a URL, a download's text, or a target named by
955
+ one) in the terminal, the CI log and Playwright's reports: it names the
956
+ `{env:...}` token, as replay's errors do. `saffron eject` warned only of
957
+ a Playwright trace, and said turning it off was enough, while
958
+ Playwright's HTML report keeps every typed `{env:...}` value as a step's
959
+ title: it warns of an HTML or blob reporter in the project's own
960
+ Playwright config too, naming `playwright-report` and `blob-report`
961
+ beside `test-results`.
962
+ - Runs: a heal that recorded the same actions as the committed recording,
963
+ differing only in the page fingerprint a browser recording stamps, was
964
+ filed as a proposal, so a failing step turned yellow: it is red, as any
965
+ heal that changed nothing is. A scenario with no steps of its own (its
966
+ lines read as its description) ran its Background alone and could pass:
967
+ it is red, as a scenario with no steps is. With the app down, a pending
968
+ proposal's replay was blamed on the recording (for another Examples row,
969
+ with advice to reject it and record again) or, under `--no-agent`, on
970
+ the recorder: it is blocked by the environment, as a committed
971
+ recording's replay is. A scenario that needed recording in a browser
972
+ that only replays (Firefox, WebKit) was counted as blocked by the
973
+ agent's provider: it is blocked by the recorder. With no agent and
974
+ several workers, checking a recording again against a changed API
975
+ contract ran the `reset` commands while other workers replayed: it runs
976
+ none, as an ordinary replay.
977
+ - Review: a held note whose own words held a value the scenario names with
978
+ `{env:...}` (`stable` in `unstable recording:`, `base` in
979
+ `database write added`) was masked there too, and `accept --all` then
980
+ promoted the proposal: a note's prefix is never masked. A heal that
981
+ changed or dropped the status a request expects (201 to 409, or to any
982
+ 2xx) kept that request's requirement and was not held for review: the
983
+ expected status is part of it. An import that stopped part way (a write
984
+ that failed, Ctrl-C) left the proposals it had already written out of
985
+ the import ledger, and `accept --all` promoted them as if filed here:
986
+ each is in the ledger before it is written. `saffron run --rerecord`
987
+ lost track of the recording it replaced when that one was kept under its
988
+ 0.9.9 file name (a scenario name with no ASCII letters, or a long name
989
+ saved whole), so a looser `Then` recorded again was not held: it is
990
+ found. A `--rerecord` run that recorded nothing for a scenario deleted
991
+ the recording it had set aside when the scenario's pending proposal was
992
+ rejected or replaced during the run: it is put back.
993
+ - Configuration: a count or limit written in quotes
994
+ (`"stabilityProofs": "2"`) was read as a number by `saffron run` and as
995
+ text by the MCP `run_plan`, which planned 1121 executions: every command
996
+ refuses it as the config loads, naming the key. A bad value in
997
+ `saffron.config.json` was refused as a flag the user never passed
998
+ (`invalid --trace "yes"`): the message names the key
999
+ (`invalid trace "yes" in saffron.config.json`), and the flag only when
1000
+ one was given. A config saved with a byte order mark (Windows Notepad,
1001
+ PowerShell 5.1) was refused as not valid JSON, and `saffron init` filled
1002
+ `features/` instead of the configured folder: the mark is read as no
1003
+ part of the JSON, as in a data file. `saffron mcp` did not start while
1004
+ `saffron.config.json` was broken, so the agent fixing it had no tools:
1005
+ the server starts, and each tool answers on the defaults with
1006
+ `configProblem` saying what is wrong.
1007
+ - `saffron init` replaced a `saffron` MCP server you had set up yourself
1008
+ (a local CLI, an `env`) with its `npx` entry and said only "updated": it
1009
+ keeps one that runs another command and says so, keeps any field you
1010
+ added to its own entry, and brings an earlier init's `npx saffron mcp`
1011
+ up to date. `saffron report --merge` of a named run's shards wrote
1012
+ `latest.json` and `history.jsonl` without the lock a run takes, so a run
1013
+ writing at the same time could lose its rows, or the merge its own: both
1014
+ write under one lock. `saffron status`, the editors' panels and the MCP
1015
+ `project_status` crashed on a last report whose row held steps or
1016
+ evidence that were not a list, or an error that was not text, and on one
1017
+ with no totals: such a row is read without those fields, and such a
1018
+ report as none.
1019
+ - Parser: a mistyped header (`scenario:`, `Background :`, or
1020
+ `Scenario Name` and `Background` with no colon) over a single step, or
1021
+ over `*` steps, was read as description and silently did not run, and a
1022
+ file of one such scenario ran nothing: it is refused, naming the header.
1023
+ A `*` line counts as a step only right under a header misspelt with its
1024
+ colon, and one step under a header word with no colon only when it is
1025
+ indented deeper, so bullets and prose in a description stay description.
1026
+ A StepSet call written after a step keyword (`When StepSet Login`,
1027
+ `* StepSet Login`) ran as an ordinary step the agent made sense of, and
1028
+ the set never ran: it is refused, saying a call is a line of its own. A
1029
+ StepSet at the top of a `.feature` file was told to fix its header's
1030
+ spelling, or with one step under it ran nothing: it is told to rename
1031
+ the file to `.saffron`. A refusal of a line read as description names
1032
+ that line, where it named its block's header (the language server marks
1033
+ it there too), and Gherkin's parse errors quote the line as written, at
1034
+ file:line:column, where they quoted it as Saffron rewrites it
1035
+ (`* StepSet Login`) and lost the file:line form in a file that defines a
1036
+ StepSet.
1037
+ - The language server, which JetBrains reads: one StepSet problem anywhere
1038
+ (a nested call, a name defined twice, a set with Examples, a set file
1039
+ that does not parse) left it no step sets, so every call in every file
1040
+ was marked "not defined anywhere", completion offered nothing, and the
1041
+ problem the runner stops on was marked nowhere: each problem is marked
1042
+ on its own line, and every other set is kept. A `.saffron` description
1043
+ line starting with "StepSet " is no longer marked as a call. Two
1044
+ scenarios of one name in a file (one per Rule), or names that slug
1045
+ alike, which every run of the file refuses as sharing a recording, are
1046
+ marked on the later one. A step with a long run of spaces inside it took
1047
+ more than a second per keystroke: the pattern sessions are read with is
1048
+ linear.
1049
+ - Exported tests: a check of a target inside an iframe passed while the
1050
+ iframe was hidden, zero-sized or inside a box clipped to nothing, where
1051
+ replay fails it: the exported test reads such a target as not shown. On
1052
+ a page that holds a request open (a long-poll, a notifications stream)
1053
+ an exported test waited up to 3 seconds before each check and failed a
1054
+ message shown briefly that replay passed: every action now starts its
1055
+ own window, as in replay.
1056
+ - A DynamoDB "not in the database" check filtering on a top-level
1057
+ attribute whose name holds a dot (`meta.tier`) was refused as naming an
1058
+ attribute no item has: it is read as the attribute it names.
1059
+ - Recordings and files that may turn red or stop a run with this release:
1060
+ a feature file with a mistyped scenario header (fix the header), a
1061
+ database check that counts rows under a one-row subquery (re-record,
1062
+ `saffron run --rerecord`), a "not shown" check on a whole number shown
1063
+ with zero decimals, an amount check on a value shown with three or more
1064
+ decimals that was a little off, a config with `actionTimeoutMs: 0`, an
1065
+ Outline that uses a `<column>` one of its Examples blocks lacks (add the
1066
+ column), a scenario with no name (name it), a Set-Cookie pattern written
1067
+ with a second branch, a "differs" check between a price and a text
1068
+ that is no amount (capture the amount alone), exported tests, once
1069
+ ejected again, that passed on text across a block's edge or a download's
1070
+ lines, a "not in the database" check whose bind is of another type than
1071
+ the database keeps the value as (give the bind its type), a `LIKE` bind
1072
+ with a backslash in such a check, a `$0` placeholder, a SQL Server alias
1073
+ that is a statement's word, an Oracle check that calls a `UTL_` or
1074
+ `DBMS_` package, a count of rows under a latest-row anti-join, a
1075
+ MySQL `UPDATE` of a `FLOAT` under cleanup (set `db.cleanup` to
1076
+ `"inserts"`), a config whose `baseURL` is not an `http://` or
1077
+ `https://` URL, a schema check on a response served as a type that is
1078
+ not JSON, a mistyped header over a single step or `*` steps, or a
1079
+ description line starting with Scenario, Example or Background over a
1080
+ step indented deeper (fix the header, or reword or unindent the line), a
1081
+ StepSet call after a step keyword (drop the keyword), a placeholder with
1082
+ a space inside its brackets (remove the space), an Examples block beside
1083
+ a `{data:...}` one that lacks one of its columns (add the column), a
1084
+ scenario with no steps of its own, only the Background's (give it steps,
1085
+ or remove it), a count or limit in quotes in `saffron.config.json`
1086
+ (`"stabilityProofs": "2"`: write the number), a recording whose `Then`
1087
+ writes to the database or whose `Given` or `When` checks it (re-record),
1088
+ a SQL check that a column is not in a row (re-record, and it is written
1089
+ `IS NULL`), a "not in the database" check on a field no document of the
1090
+ collection holds (fix a misspelt name, or let a document hold the field
1091
+ first), one that compares a JSON value with a bind read as JSON, a MySQL
1092
+ JSON boolean with 1 or 0, or a binary column with `0x` hex, or that
1093
+ binds JSON the SQL makes itself (compare as its message says), a count
1094
+ or a "no second row" check under a `WHERE` that keeps one row of each
1095
+ group (re-record), a check of a state shown only while the `When`'s
1096
+ request is out, such as a spinner (re-record, so it is watched for from
1097
+ the `When`), a check of text in a box closed to no height or width, or
1098
+ in an iframe the page hides (open it in a `When` first), an absence
1099
+ check on an HTML boolean attribute that is there with a text such as
1100
+ `false` (the browser reads it as set: fix the page), an Outline whose
1101
+ check shows a row's date, amount or spaced value in the page's own form
1102
+ (give the shown value an Examples column of its own), and exported
1103
+ `@api` tests, once ejected again, that passed on the signed-in session's
1104
+ cookie (give the request credentials of its own). From the last review:
1105
+ an Outline whose not-shown check holds the recorded row's day by its
1106
+ month's name without the year (the other rows are refused, and a run with
1107
+ no agent calls an older such recording stale: name the date column in the
1108
+ step, or re-record); a database "differs" check on text that spells a
1109
+ number (`"type": "string"` compares the text); a database "at most" check
1110
+ with an empty or null value (give the step a value, or count empty
1111
+ columns by `length(note) = $1` with 0; on Oracle an empty text is NULL,
1112
+ so write `IS NULL` beside a column compared with a bind); database steps with a driver
1113
+ of another major version than the tests run (the refusal names the one
1114
+ to install); a step whose select, fill, hover, drag, upload, check or
1115
+ key press passed only on its second try, after its handler kept the page
1116
+ busy past the action timeout (it is no longer sent twice: raise
1117
+ `actionTimeoutMs`); a click, a key press, a check, a drag or a hover that closes
1118
+ its own tab and is rejected by Playwright for it, where a second try used to
1119
+ land on the tab behind (the step is red at once); a recording that
1120
+ relied on Playwright 1.64 acting inside an `inert` part of the page, on
1121
+ an accessible name that holds text the browser does not draw, or on a
1122
+ cookie whose `Path` is only a prefix of the request's (re-record); a
1123
+ database "differs" check typed as a number or a boolean on a value the
1124
+ row holds as another type (give it the type the row shows, or none); a
1125
+ MySQL or MariaDB "not in the database" or "at most" check that binds an
1126
+ unquoted `true` on a `BOOLEAN` or number column holding false or 0 (bind
1127
+ 1 or 0); an
1128
+ Outline whose not-shown check holds a word that is also a month's name
1129
+ ("may", "march") beside a number that is a row's day (the other rows are
1130
+ refused: name the date column in the step); and exported tests, once
1131
+ ejected again, whose nine or more checks straight after a page load that
1132
+ keeps a request open now each wait about 3 seconds (raise the test's
1133
+ `timeout` in `playwright.config.ts`).
1134
+
1135
+ Editors (VS Code extension 0.8.3, JetBrains plugin 0.8.3)
1136
+
1137
+ - VS Code: the same headers and steps Gherkin reads as description are
1138
+ marked as errors; a StepSet the file itself defines is no longer called
1139
+ undefined, and a description line starting with "StepSet " is text; a
1140
+ `Background:` line under a scenario's header no longer hides its
1141
+ Examples errors; a `.feature` file in a folder with no
1142
+ `saffron.config.json` gets no Saffron errors; a row of a report written
1143
+ on Windows opens its scenario; with no folder open the panel says so
1144
+ instead of loading for ever; `{data:...}` hints read a data file once
1145
+ until it changes. More of what the runner's parser refuses is marked
1146
+ where it is written: a scenario with no name, an outline placeholder
1147
+ no Examples row fills (a column only another Examples block has, a
1148
+ near miss of a column, a column named twice, placeholders with no
1149
+ Examples, a column a data file lacks), a `{unique:...}` that is no
1150
+ token, a StepSet line anywhere in a `.feature` file, and a fixture or
1151
+ step set the runner cannot run.
1152
+ - JetBrains: the CI Runs results page runs none of the scripts a run's
1153
+ bundle carries; a row of a report written on Windows opens its scenario
1154
+ at its line; a feature file that is not valid UTF-8 is listed with its
1155
+ scenarios; Run Selected on one file names its configuration by the
1156
+ file's path; a test folder outside the project is watched; a selected
1157
+ scenario row in CI Runs stays selected through a reload, told apart
1158
+ from another feature's of the same name. A folder of feature files
1159
+ added to the test folder is listed without a Refresh, and on Windows a
1160
+ test folder on another drive is listed at all; a status reload keeps
1161
+ the proposal being read and the Last Run row selected; while the status
1162
+ loads, the tabs say so instead of claiming nothing is pending; the
1163
+ Accept All tooltips name a database write a new recording added.
1164
+
1165
+ ## 1.4.0: Database steps
1166
+
1167
+ Released 2026-11-10.
1168
+
1169
+ Runner
1170
+
1171
+ - Database steps (beta): a step like `Then the customer should
1172
+ be saved in the database` works with no query written by hand. While
1173
+ recording, the agent reads the schema and records one parameterized
1174
+ `SELECT`, its values as binds; every later run replays it at zero
1175
+ tokens. A check must be bound to a value the scenario wrote; a value that
1176
+ is new on each run (`{unique:...}`, an id an earlier step kept) keeps it
1177
+ from passing on rows an earlier run left. With `"writes": true` and the
1178
+ database's host or name in `allow`, a `Given` or `When` may seed data with
1179
+ one `INSERT` or `UPDATE ... WHERE`. When the scenario ends, green or
1180
+ red, `db.cleanup` undoes them: `"all"` (default) deletes the rows it
1181
+ inserted and puts the rows it updated back as they were, `"inserts"`
1182
+ deletes inserts only, `"none"` leaves everything. `DELETE`, `TRUNCATE`,
1183
+ DDL and several statements are refused by a SQL parser. Set `db` in
1184
+ `saffron.config.json` and `npm install node-sql-parser` with the
1185
+ engine's driver: neither comes with saffron-ai, so a project without
1186
+ database steps does not carry the parser's 88 MB. Recordings with
1187
+ database steps are cache version 20.
1188
+ - Database steps on PostgreSQL, MySQL and SQL Server, named by db.url's
1189
+ scheme (`postgres://`, `mysql://`, `mssql://`), with the driver `pg`,
1190
+ `mysql2` or `mssql`. Each engine takes the same steps, rules and `$1` binds; MySQL and SQL Server writes
1191
+ are read back by their primary key, one row per `INSERT` when the key
1192
+ is auto-numbered. A recording made on one engine is refused on another.
1193
+ A write to more than one table (MySQL's `UPDATE a JOIN b`) is refused,
1194
+ and a write whose rows cannot be read back by their key is rolled back,
1195
+ so cleanup never misses a row or touches one Saffron did not write. A
1196
+ SQL Server check that meets a row the app is still writing reads again
1197
+ instead of failing. A table hint (`WITH (NOLOCK)`, `READPAST`) is refused,
1198
+ so a check never reads rows the app did not commit or skips rows it
1199
+ holds. A write goes only to the database `db.url` names (MySQL's
1200
+ `otherdb.t` and SQL Server's `db.schema.t` are refused), an `UPDATE` with
1201
+ `ORDER BY` or `LIMIT` is refused, and a SQL Server `UPDATE` that changes a
1202
+ time finer than milliseconds, or with an offset, is refused while cleanup
1203
+ would put it back. MySQL's `INSERT IGNORE` is refused, as it skips rows
1204
+ without saying so, and a check whose rows the app holds locked for the
1205
+ whole `db.timeout` fails saying so.
1206
+ - A database check reads again within `db.timeout` (default 3000 ms): a
1207
+ presence check passes as soon as its rows are there, and a "not in the
1208
+ database" check waits for a row a worker deletes late, then must hold
1209
+ to the end of the window, so a row a worker writes late fails it.
1210
+ - On PostgreSQL and MySQL every statement a database step runs goes to
1211
+ the database as exactly one statement, so a query the SQL check reads
1212
+ as one SELECT can never run a second statement (a quote read differently
1213
+ by the two could hide a COMMIT and a DELETE in a look at the data). SQL
1214
+ Server runs a batch, so there the parser's one-statement rule and the
1215
+ refused backslash before a quote hold it.
1216
+ - Cleanup puts a json or jsonb column back as JSON, an array or a string
1217
+ value included. Placeholders numbered with a gap ($1 and $3, no $2) are
1218
+ refused while recording.
1219
+ - Only a scenario whose recording has database steps, or whose steps name
1220
+ the database before it is recorded, needs the database to answer before
1221
+ recording starts.
1222
+ - A database check that the database refused (a permission, a timeout)
1223
+ says so, apart from one that found other rows; neither starts an agent.
1224
+ - A database check whose stored query fails because the schema changed
1225
+ is repaired by the agent, the one exception to a `Then` never being
1226
+ healed: only its table, columns and joins may change, its binds and
1227
+ expected rows are enforced unchanged, it must return as many rows as
1228
+ the recording did (a join that fans out is refused), and the repair must
1229
+ pass a proof replay. The step is reported healed and a proposal names the old and
1230
+ new SQL. A query that returns the wrong rows stays red, with no agent.
1231
+ - The run report shows the rows each database check found under its
1232
+ step, masked, the first five.
1233
+ - A recorded check must tie its rows to the scenario's value: a column
1234
+ compared with a bind by `=`, `IN` or `LIKE`, on each side of an `OR`,
1235
+ and a subquery on the tying side ties its own rows the same way.
1236
+ `UNION`, `INTERSECT` and `EXCEPT`, `email <> $1`, `NOT (...)`, a
1237
+ comparison inside `COALESCE` or `CASE`, a range alone
1238
+ (`created_at > $1`) and a bind in `LIMIT` or `OFFSET` are refused, as
1239
+ each could hold on rows the scenario never wrote. The same holds for an
1240
+ `UPDATE`'s `WHERE`. A JSON field can tie a check
1241
+ (`data->>'orderRef' = $1`, `data @> CAST($1 AS jsonb)`). A null bind,
1242
+ or an empty one in a "not in the database" check, is refused while
1243
+ recording and at replay: `= NULL` matches no row.
1244
+ - A repaired database check, and a heal that adds or changes a database
1245
+ write, are held for review: `accept --all` leaves them out (`DATABASE
1246
+ CHECK REPAIRED`, `DATABASE WRITE ADDED`).
1247
+ - A database that is down, restarts mid-step or does not answer within 10
1248
+ seconds is the environment: no agent session starts, and the message
1249
+ says the database did not answer, not the app. Before recording, an
1250
+ unset `db.url` variable, a connection string that is not a URL, a wrong
1251
+ password, a missing database and a missing driver (`pg`, `mysql2` or
1252
+ `mssql`) or `node-sql-parser` are said at once, without starting an
1253
+ agent.
1254
+ - `saffron status --json` and the `project_status` MCP tool say whether
1255
+ database steps are configured (`database`: the engine, whether writes
1256
+ are on, the cleanup mode), never the connection string; they and the
1257
+ editors' config views mask a password in a `db.url` written in keyword
1258
+ form (`host=... password=...`) or in its query (`?password=`). The
1259
+ bundled skill, `saffron init`'s AGENTS.md block and the guides cover
1260
+ database steps and the one Then whose query may be repaired.
1261
+ - `run --plan` and the MCP `run_plan` tool list a recorded database write
1262
+ as a side effect when `db.cleanup` keeps it (`"inserts"` keeps an
1263
+ `UPDATE`, `"none"` every write).
1264
+ - An `@api` scenario whose recording sends no request (database steps
1265
+ alone) replays without the API answering.
1266
+ - The language server (JetBrains completion and hover) says a step whose
1267
+ recordings all read or write the database replays at zero tokens only in
1268
+ the scenario that recorded it, per `As "..."` session, as VS Code does,
1269
+ and its similar-wording hint no longer suggests such a step.
1270
+ - Found in the release review, before release:
1271
+ - A write inside a `WITH` passed when both statements were `UPDATE`s,
1272
+ and its table was never put back. Any `WITH` in a write is refused,
1273
+ and so is a `WHERE` that reads no column (`WHERE true`).
1274
+ - A check could echo its bind back (`SELECT $1 AS email`), or hold on
1275
+ any row (`OR true`, `OR email IS NOT NULL`, `(email = $1) IS NOT NULL`,
1276
+ `$1 = $1`), and a schema repair could add
1277
+ `LIMIT 1`, `DISTINCT` or `max()` to hide a join that fans out. Both
1278
+ are refused, and so are values written in as `E'...'`, `$$...$$` or
1279
+ `DATE '...'`.
1280
+ - Under `"assertionPolicy": "adaptable-mid"`, a heal rewrote a database
1281
+ check in the middle of a scenario, its binds and expected rows too.
1282
+ A database check is never healed, under every policy.
1283
+ - A database check recorded by another scenario was reused with that
1284
+ scenario's value, so it could pass on rows an earlier run left. A
1285
+ database step is seeded only from the scenario's own recording.
1286
+ - A `db.url` written as `host=... password=...` was shown whole in the
1287
+ error, password included. The message no longer shows the value, and
1288
+ the rows an error shows are masked before they are cut.
1289
+ - A database connection dropped mid-step crashed the run, a host that
1290
+ does not answer now fails within 10 seconds, and a step still running
1291
+ after Ctrl-C opens no new connection.
1292
+ - Ctrl-C during a database write left its row behind. Closing now
1293
+ waits for the write and undoes it.
1294
+ - A check on an aggregate's one row passed whatever the table held:
1295
+ `count(*)` with no `GROUP BY`, an aggregate the parser reads as a
1296
+ plain function (`json_agg`, `JSON_ARRAYAGG`, `bool_or`), `GROUP BY
1297
+ ROLLUP`, and a query whose only `FROM` or `WITH` source is an
1298
+ aggregate. A count, exists or item check on that row is refused while
1299
+ recording and at replay; only a value checked on it is taken.
1300
+ - A count above 0, or a check that there is no second row, could not see
1301
+ a second row under `LIMIT`, `TOP`, `DISTINCT` or a `GROUP BY` with no
1302
+ aggregate, and `OFFSET` (SQL Server's `OFFSET ... FETCH` too) hid rows
1303
+ from a count and let a "no rows" check pass: each is refused. On MySQL
1304
+ and SQL Server a check never saw `DISTINCT` (the parser writes it
1305
+ differently there), so such checks passed on a duplicate row and a
1306
+ repair could add it; it is now read on every engine.
1307
+ - A check that expects no rows, or that a field must not be there, is no
1308
+ longer repaired: a repaired query finding none proved nothing, and a
1309
+ repair could stop selecting that field.
1310
+ - A field that must not be in a row (`/0/data/discount`) passed when the
1311
+ app saved no row at all; it now needs the row to be there.
1312
+ - A captured id of one or two characters was recorded as itself in a
1313
+ database bind, so replay checked the recording run's row; it must be
1314
+ written as `{captured:name}`, and one written as itself beside another
1315
+ bind is refused.
1316
+ - On MySQL, `--` with no space after it let a look at the data run what
1317
+ Saffron read as a comment; it is refused. On SQL Server, the agent's
1318
+ `UPDATE` was read as PostgreSQL when checking that its `WHERE` names
1319
+ the scenario's rows, and a write to a table whose key the driver reads
1320
+ inexactly (a decimal past 15 digits) is refused while cleanup is on.
1321
+ - A page fingerprint left by a browser tool that failed went onto the
1322
+ next database or HTTP action, and that recording was refused on every
1323
+ load after.
1324
+ - A MySQL write could read back a row that was there before (`7`
1325
+ finding `'07'`) for one its key missed, and cleanup deleted it: each
1326
+ key now reads back exactly its own row, or the write is rolled back.
1327
+ A JSON column is put back as the text it held, never read through a
1328
+ JS number.
1329
+ - A secret a driver quotes cut short (MySQL's duplicate entry at 64
1330
+ characters, SQL Server's truncated value), one holding a parenthesis
1331
+ or cut to 4 characters too, is masked, and a secret in a row no longer
1332
+ breaks the rows the report shows. A lock held for the whole window
1333
+ says it timed out, not that it found other rows.
1334
+ - A database check was seeded from the scenario's own recording after
1335
+ an earlier step changed the value it binds; it is recorded again. A
1336
+ stale recording whose edited steps name the database now asks the
1337
+ database to answer before recording.
1338
+ - A SQL Server `BIGINT` identity past 2^53 is read exactly, so cleanup
1339
+ deletes the scenario's row, never the app's.
1340
+ - A check or an agent `UPDATE` with an `OR` whose other side is a fixed
1341
+ value (`email = $1 OR role = 'admin'`) passed on rows earlier runs
1342
+ left: each side of an `OR`, and every value of an `IN`, must hold a
1343
+ value the scenario wrote. A `LIKE` value of only `%` or `_` (from an
1344
+ Examples cell too) is refused unless another condition ties the rows,
1345
+ and so are `LIMIT 0`, `TOP 0` and `FETCH ... 0`.
1346
+ - A quoted, qualified column (`` `o`.`total` ``, `"o"."total"`) was
1347
+ refused as a value written into the SQL; it is read as a name.
1348
+ - Stability proofs and an Outline's row proofs failed on rows the first
1349
+ proof seeded: its database writes are now undone before they run.
1350
+ - The MCP `search_steps` tool showed the agent a database step's
1351
+ session-prefixed text (`As "name" ...`), which would move that step
1352
+ into the named session; it no longer does.
1353
+ - An Outline check that holds a row's value as the page sets it in type
1354
+ (a curly apostrophe or quotes, spaces collapsed or no-break, an
1355
+ ellipsis) let another row pass on it; that row now records again or is
1356
+ refused.
1357
+ - A key under `api` or `api.services.<name>` that Saffron does not read
1358
+ (`"transcripts"`, `"contracts"`) is refused with the keys it takes,
1359
+ instead of leaving that setting at its default.
1360
+ - A check that something is shown (text, visible, a field's value) no
1361
+ longer passes on content inside a frame the page hides (`visibility:
1362
+ hidden`, zero-sized, placed off the page), or inside a frame in one.
1363
+ - A base URL's password, which Chromium keeps in the page's URL, is
1364
+ masked in what the recording agent reads: snapshots, tool errors,
1365
+ Saffron tool replies and the heal prompt's live page and failure.
1366
+ Navigating to the masked URL still signs in.
1367
+ - A remote browser's endpoint (`transport.browser`) no longer prints its
1368
+ password or `?token=` key when the connection fails.
1369
+ - `saffron status --json`, the `project_status` and `api_contract` MCP
1370
+ tools mask a password in a base URL and a key in a URL's query or
1371
+ fragment (`?token=`, `?api_key=`, `?key=`, `#access_token=`); `api_contract` showed the password
1372
+ to the agent. The MCP `verify` tool masks them in a failed run's last
1373
+ lines, and `saffron api pin` in what it prints.
1374
+ - `saffron api pin` says why a URL could not be fetched (connection
1375
+ refused, host not found) instead of only "fetch failed".
1376
+ - `saffron run --json` without `--plan` exits 2 instead of starting a run
1377
+ that prints no JSON.
1378
+ - `saffron eject --out <dir>` warns when the project's Playwright config
1379
+ looks for tests in another folder, so `npx playwright test` would not
1380
+ run the new specs.
1381
+ - `saffron init` registers the MCP server as `npx -y -p saffron-ai saffron
1382
+ mcp`: a bare `npx saffron` fetched npm's unrelated `saffron` package in
1383
+ a project without saffron-ai installed.
1384
+ - A `saffron.config.json` that is not valid JSON is named in the error.
1385
+ - `saffron status --json` and the `project_status` MCP tool name, in each
1386
+ `lastRun.attention[]` entry, why a red was not run (`blocked`:
1387
+ `recorder`, `environment`, `provider` or `budget`), so editors list it
1388
+ as not run instead of failed.
1389
+ - `--project-root` naming a folder that does not exist stops with exit 2,
1390
+ instead of reading it as an empty project, and `saffron run` names the
1391
+ configured features folder when it does not exist.
1392
+ - A run notes when the selected data environment (`--env`, `SAFFRON_ENV`
1393
+ or `env`) matches no data file, so a misspelled name no longer uses the
1394
+ base data in silence.
1395
+ - `Examples: {data:...}` refuses a CSV file that names a column twice, as
1396
+ an Examples table does; one of them was read for both. A data JSON file
1397
+ saved with a byte order mark is read.
1398
+ - The recording agent can no longer send an `{env:...}` value to another
1399
+ site by writing the site as a reference (`{captured:link}?p={env:...}`,
1400
+ a row's `<column>`, a `{data:...}` value): the site is checked as the
1401
+ browser will open it.
1402
+ - A capture, and a check that a value changed, wait while the element
1403
+ they were recorded on is still empty, instead of reading the same test
1404
+ id in an iframe or a fallback that happens to show text: "the price
1405
+ changed" passed on an embedded widget's copy.
1406
+ - When a tab closes (a step closes it, or it closes itself, as a sign-in
1407
+ popup does), replay and an exported Playwright test go where the
1408
+ recording's browser went: they stay where they were when another tab
1409
+ closed, and take the tab that slid into the closed one's place, never
1410
+ simply the last tab.
1411
+ - `saffron eject` no longer writes a base URL's password, or a key in its
1412
+ query or fragment (`?api_key=`, `#access_token=`), into
1413
+ `playwright.config.ts` or `saffron-helpers.ts`: it writes `***` and
1414
+ warns, naming the variable to set to the full URL (`BASE_URL`,
1415
+ `API_URL` or `API_URL_<NAME>`).
1416
+ - When a click opens a new tab and the agent keeps working on the page it
1417
+ clicked (the browser it records with stays there), the recording now
1418
+ switches back to that page, so replay runs those steps there instead of
1419
+ on the new tab, where a "not shown" check could pass on the wrong page.
1420
+ A tab closed after such a click is closed from the agent's page too, and
1421
+ a heal that takes over on a tab replay followed records a switch to the
1422
+ tab the agent works on (its browser starts on the first tab). A
1423
+ recording made earlier keeps its behavior: re-record a scenario that
1424
+ opens a tab and then acts on the page that opened it.
1425
+ - Replay follows only a tab a recorded action opened. One that opens
1426
+ without such an action (a popup more than 5 seconds after its click, a
1427
+ page timer) is not followed, as the agent's browser stayed on its page:
1428
+ the checks after it ran on the new tab, where they could pass. Exported
1429
+ tests follow tabs the same way.
1430
+ - An older recording that uses tabs without tab marks still follows any
1431
+ new tab, also one that opens later than 150 ms after a click: the check
1432
+ after it moves to the new tab, and an action waits for the tab or its
1433
+ element, a `{unique:...}` target included. A click that opens no tab is
1434
+ not slowed. A "not shown" check there must hold on the page it started
1435
+ on, then waits for the tab and must hold on it too, so neither page
1436
+ passes it for the other (record the scenario again to add tab marks and
1437
+ drop the wait).
1438
+ - Exported tests wait for a tab a background click opened, and take a tab
1439
+ that already opened before a switch or close, as replay does.
1440
+ - Reports and run history mask a key in the base URL's query or fragment
1441
+ (`?api_key=`, `#access_token=`), as they already masked its password.
1442
+ - A signed storage URL's signature (an Azure SAS `?sig=`) is masked
1443
+ wherever a key in a URL's query is.
1444
+ - `saffron eject` warns when the exported tests send API requests while
1445
+ `transport.api` or `transport.module` is set: they go without its
1446
+ headers, proxy or certificate setting.
1447
+ - A feature path a Windows machine recorded (`features\login.feature`)
1448
+ is found on macOS and Linux: `saffron prune` deleted such a recording of
1449
+ a live feature outside the features folder, `saffron import` refused a
1450
+ Windows CI run's proposals as "not in this checkout", `saffron trace`
1451
+ called them stale, `accept` could not apply their feature edits, and
1452
+ viewing that run hid "Open scenario". `saffron coverage` writes feature
1453
+ paths with forward slashes, and `--base` reads a map committed on
1454
+ Windows as the same files instead of every scenario removed.
1455
+ - Feature paths written on Windows also match elsewhere in the run: a
1456
+ sharded run split across Windows and Linux machines gives every
1457
+ scenario to exactly one shard (some ran twice and some never), `saffron
1458
+ diff <feature file>` finds a proposal filed on Windows, `run --plan`
1459
+ treats such a pending proposal as the run does instead of planning a
1460
+ recording, and `status --json` names an Examples row's scenario
1461
+ (`baseScenario`) from a report written on Windows. `saffron init` keeps
1462
+ the Windows line endings of an AGENTS.md or CLAUDE.md that uses them.
1463
+ - A dropdown option picked in a form is replayed by its label when
1464
+ another option's value is the same text (that earlier option was chosen
1465
+ instead). Exported tests do the same.
1466
+ - Recording with many agent workers no longer prints a
1467
+ MaxListenersExceededWarning: the sessions share one exit listener.
1468
+ - A fixture whose pending proposal is deleted or unreadable mid-run (with
1469
+ proofs off) makes its scenarios red, naming why, instead of ending the
1470
+ run without a report.
1471
+ - `saffron accept` writes a feature edit atomically, keeping the file's
1472
+ line endings, permissions and symlink, so a crash mid-write no longer
1473
+ leaves it cut short.
1474
+ - One StepSet file that does not parse, or defines a set twice, no longer
1475
+ stops `saffron steps` and the MCP `search_steps` and `list_step_sets`
1476
+ tools: the other files' steps are offered and that file is named.
1477
+ - The language server refreshes `{data:...}` hints when files in a
1478
+ configured `dataDir` change, not only in `data/`.
1479
+ - Recording keeps right, middle and Ctrl or Shift clicks: Playwright MCP
1480
+ writes their options over several lines, and they were left out of the
1481
+ recording.
1482
+ - A `{unique:name:digits}` value sent as a number (in a request body or a
1483
+ database check bind) is recorded as its token: it was kept as the
1484
+ recording run's number, so later runs checked that run's row.
1485
+ - An abbreviated amount (`€1.5M`, `$1.2K`), or one written in words or
1486
+ spaced off ("3 bn", "€1.5 million", "1,5 Mio. €", "5 lakh"), is refused
1487
+ instead of read as its digits, and Indian lakh grouping
1488
+ (`₹1,23,456.00`) reads as 123456, in replay and exported tests.
1489
+ - Copies of a request sent at once (`concurrent`) are never sent again by
1490
+ `retries`: a retry could pass a race the first copies lost.
1491
+ - Response bodies are read in the charset their Content-Type names, for
1492
+ API requests, the browser's response checks and exported tests, and a
1493
+ leading byte order mark is ignored: a Latin-1 body read garbled, so a
1494
+ `differs` check could pass on an unchanged value.
1495
+ - A recording the agent's provider failed (a sign-in that expired, the
1496
+ network, a usage limit) is counted as blocked by the agent's provider,
1497
+ not as failed in the app.
1498
+ - Stopping a run (Ctrl-C, a cancelled CI job) also stops a running Codex
1499
+ or Antigravity session, which kept running and spending tokens.
1500
+ - `maxSessionMinutes` also stops a session that makes no tool call (a
1501
+ vendor waiting on a sign-in, a stalled model call), two minutes past its
1502
+ budget, and says why. Such a recording counts as failed, not as blocked
1503
+ by the provider, as the app may be what kept it waiting.
1504
+ - `saffron import` refuses bundles whose manifests, reports and proposals
1505
+ would unzip to more than 1 GB together, before reading any, and masks a
1506
+ password or token in a bundle URL wherever it prints it.
1507
+ - Recording keeps the checkboxes and radio buttons the agent sets while
1508
+ filling a form, a key press of `{`, `(`, `[`, `,`, `'` or `\`, and a
1509
+ navigation to an address with an apostrophe; each was dropped without a
1510
+ warning, and replay never did it.
1511
+ - `saffron trace` refuses a trace file that unzips past its declared size
1512
+ or over 1 GB, so a damaged or hostile trace from CI cannot fill memory.
1513
+ - A "not shown" check passed while the text was on screen with an
1514
+ invisible format character in it: the bidi isolates Fluent and ICU put
1515
+ around a value, a soft hyphen, a word joiner. These now count as shown.
1516
+ - A step seeded from another with a new quoted value wrote it into a
1517
+ pattern unescaped: `matches "Paid (partial)"` passed on "Paid partial".
1518
+ - A text check whose expected value holds a captured credential
1519
+ (`{captured:...}`) is no longer handed to the page's own scripts, as
1520
+ `{env:...}` values already were, so a script on the page or in an iframe
1521
+ cannot read it.
1522
+ - A closed single dropdown reads as its selected option only: a text,
1523
+ pattern, capture or "differs" check no longer passes on another option,
1524
+ and a "not shown" check fails while the dropdown shows the option it
1525
+ names. A text target ("I see Cancelled") never matches an option the
1526
+ dropdown only offers. A list check still reads every option.
1527
+ - A "not shown" check by role and name fails when the page shows that name
1528
+ in another case ("CANCEL ORDER").
1529
+ - Seeding by quoted value keeps every date token form (`{date}`,
1530
+ `{date+1}`, `{date+1:D MMM}`) as written in a pattern.
1531
+ - A multipart request never sends `saffron.config.json` (in any letter
1532
+ case), which may hold a base URL's password.
1533
+ - A CI run's results never offer to open a feature path outside the
1534
+ project.
1535
+ - The language server reads the data environment as a run does: a
1536
+ `SAFFRON_ENV` set in the shell, even empty, keeps the `.env` one out.
1537
+ - A request the page started before an action (a long-poll opened on
1538
+ load) is no longer taken for the action's own when its event arrives a
1539
+ moment late, so "not shown", count and response checks no longer wait it
1540
+ out: four such checks took 14 seconds instead of 2.
1541
+ - A check that a dropdown offers an option reads the option itself
1542
+ (`expectText` on `option "Cancelled"`). A 1.3.x check on the dropdown
1543
+ that passed on one of its other options is now red: record it again.
1544
+ - Under `--no-agent`, a run where the app or the database does not answer
1545
+ marks those scenarios blocked by the environment, as a run with the
1546
+ agent does, so CI tells an outage from a regression.
1547
+ - Colour only on a terminal, and never with `NO_COLOR` set: a log file or
1548
+ a pipe gets plain text.
1549
+ - An exported "not shown" check counts as shown what replay does: a text
1550
+ with invisible format characters in it, a role's name in another case,
1551
+ the selected option of a closed dropdown.
1552
+ - `saffron eject` says to install `@playwright/test` before
1553
+ `npx playwright test`.
1554
+ - A message watched for with "and see" (a toast, a flash, a status) is also
1555
+ watched inside the page when its target is a test id or plain CSS: one
1556
+ shown for a few milliseconds is seen, even on a busy machine or when the
1557
+ page moves on at once, where a look every 50 ms could miss it. A watched
1558
+ text that holds a secret is never handed to the page.
1559
+ - A {unique:...} value the page shows in another case (uppercase through
1560
+ CSS) goes back to its token in the recording, so a "not shown" check
1561
+ never keeps last run's value; a {unique:name:digits} value the page
1562
+ groups ("984,905,203") is refused when the recording is saved, as no
1563
+ token stands for it.
1564
+ - The check after a heal follows only a tab its action was recorded
1565
+ opening, as replay does: a popup a page timer opened mid-check no longer
1566
+ passes the check on another page.
1567
+ - A secret in a URL's query or fragment typed on the command line
1568
+ (`--base-url "https://app.test/?access_token=..."`) is masked in the
1569
+ report's command, the HTML report, the console summary and export
1570
+ bundles, as a URL password already was.
1571
+ - A run whose only reds are scenarios the recorder could not run (a stale
1572
+ or missing recording under --no-agent) no longer reads "all clear" in
1573
+ the HTML report or "0 failed" alone in `saffron status`: both say the
1574
+ run failed.
1575
+ - saffron.config.json refuses a number or other non-string for a path or
1576
+ name setting (baseURL, features, dataDir, storageState, timezone, model
1577
+ and others, and a session's storageState) at load, instead of failing
1578
+ later with a raw Node error after agent work.
1579
+ - `saffron report --merge` exits 1 when the merged run has a red, or a
1580
+ scenario pending review with `strict` on, as `saffron run` does.
1581
+ - `saffron diff` with a proposal file that is not there, and `saffron
1582
+ author` with a requirements file that is not there, say "<file> does not
1583
+ exist" and exit 2 instead of crashing.
1584
+ - A placeholder an Outline's Examples have no column for (a misspelt
1585
+ `<rol>` beside `| role |`), in a step, doc string or table cell, is now
1586
+ an error naming the columns (with inline Examples, only a near miss of a
1587
+ column: other tags such as `"<script>"` or `List<String>` stay
1588
+ literal), instead of every row running the same text and later rows
1589
+ passing on row 1's recording.
1590
+ - An Examples block with no table (`Examples: data:roles`, `Examples:
1591
+ {roles}`, or a table forgotten) is now an error instead of running no
1592
+ row without a word. A header row with no rows yet still runs nothing.
1593
+ - A Scenario Outline with no Examples whose steps use a placeholder is
1594
+ now refused ("add Examples or use Scenario:") instead of running once
1595
+ with its placeholders as literal text.
1596
+ - A `*` step opening a scenario after a Background `Given` is no longer
1597
+ counted as part of the Background's setup.
1598
+ - A .feature file whose description or doc string has a line starting
1599
+ `Fixture:` is no longer told to rename to .saffron. Only a `Fixture:`
1600
+ line with steps under it is.
1601
+ - Editor snippets escape `$`, `\` and `}` in step texts and StepSet names,
1602
+ so a name holding one completes as written.
1603
+ - The AGENTS.md block `saffron init` writes names the features and data
1604
+ folders `saffron.config.json` sets, not always `features/` and `data/`.
1605
+ - `saffron init` reads a VS Code `.vscode/mcp.json` with comments and
1606
+ trailing commas (JSONC) instead of calling it invalid JSON; one already
1607
+ holding the saffron entry is left as is, and one that needs it says what
1608
+ to add by hand rather than drop the comments.
1609
+ - A database check written as count(*) and checked as 0 is now treated as
1610
+ "not in the database": an empty bind is refused, the 0 must keep holding
1611
+ for all of db.timeout, and a schema repair of it is refused.
1612
+ - A schema repair of a database check whose row count comes from an
1613
+ Outline value (a count of `<n>`) is refused, since that value may be 0.
1614
+ - On MySQL, a check that locks rows (FOR UPDATE, FOR SHARE, SKIP LOCKED,
1615
+ LOCK IN SHARE MODE) is refused.
1616
+ - Database writes through a SQL Server synonym or view, or a MySQL view,
1617
+ are refused, since they may reach a table in another database.
1618
+ - On MySQL, INSERT ... SET is read with its columns, as INSERT ... VALUES
1619
+ is.
1620
+ - A token in an API response that no step extracts (an access_token or
1621
+ refresh_token field, a Location carrying #access_token=) no longer
1622
+ reaches the transcript, reports, traces, error lines or the recording
1623
+ agent in clear.
1624
+ - A polled request (until) keeps polling after a lost connection (a
1625
+ reset) or an attempt that timed out, and fails with that error only once
1626
+ the poll's time is up. A stopped run, a DNS failure or a TLS failure
1627
+ ends the poll at once.
1628
+ - The recording agent can no longer record a response-time limit (maxMs)
1629
+ or a poll limit (until.timeoutMs) other than the one the scenario states
1630
+ in figures: the bound must be a duration the step or the checks right
1631
+ after it name, in any unit.
1632
+ - A Then whose only check is the 2xx class, or the very status its
1633
+ request already required, is refused as a check that could never fail:
1634
+ the agent records a field or schema check instead.
1635
+ - Two requests at once in a new API session no longer open two clients,
1636
+ one of which was never closed.
1637
+ - In a parallel run with the agent, a scenario whose replay failed and
1638
+ then passed when run again for the agent is red, as with one worker: it
1639
+ was reported green, and a failure that comes and goes passed with
1640
+ nothing in the report.
1641
+ - `saffron run --plan` refuses an invalid `--workers` or
1642
+ `--agent-workers`, as a run does.
1643
+ - A failed check on a field named as a credential (a token, a session id)
1644
+ says `[credential, not kept]` instead of its value, in the error, the
1645
+ report and a poll's last reason.
1646
+ - Hiding a credential field in a response keeps the rest of the body as
1647
+ the API wrote it: large ids, decimals like 1.10 and spacing are not
1648
+ rewritten.
1649
+ - Exported response checks wait for the page's requests to settle and fail
1650
+ when the same request is answered again with a status the check rejects,
1651
+ as replay does.
1652
+ - Exported "URL does not contain", count and differs checks hold only once
1653
+ the page settles, as replay's do: a navigation on its way, a list still
1654
+ growing or a value that changes for a moment no longer passes them.
1655
+ - Exported captures and differs checks read the displayed copy of an
1656
+ element after a hidden one with the same text, as replay does.
1657
+ - Exported value checks require a shown input that holds the value in the
1658
+ same look.
1659
+ - Exported "not shown" checks on a literal name pattern also find the name
1660
+ with invisible format characters in it, as replay does.
1661
+ - The exported helper file writes the project root with forward slashes on
1662
+ Windows, so it reads on macOS and Linux too.
1663
+ - A heal of a step that failed partway through is told which of the step's
1664
+ actions replay already performed, and records only the rest; a healed
1665
+ recording no longer repeats them (an Add click run twice before
1666
+ Proceed).
1667
+ - Behavior change: a heal that changes nothing in the recording no longer
1668
+ files a proposal when its proof passes. Such a run used to be a yellow
1669
+ proposal and is now red, saying the recording failed on replay and then
1670
+ passed again (timing or the app's state), as it already was with
1671
+ --no-verify.
1672
+ - A proposal not filed on this machine no longer turns a failed committed
1673
+ recording yellow when it checks the Then steps differently from the
1674
+ committed recording: the run is red and names the pending file.
1675
+ - A check in a browser session that has no page yet (As "Bob") is checked
1676
+ on that session's own page, as replay does, never on the default
1677
+ session's page.
1678
+ - A seeded step that an agent segment went past and recorded nothing for
1679
+ keeps its seeded recording instead of an empty one.
1680
+ - "Checks proven sensitive" is claimed only when every Then was probed; a
1681
+ Then the sensitivity check leaves out (one that clicks, for example)
1682
+ leaves it unclaimed.
1683
+ - A row value checked through XPath `normalize-space(...)`, `string(...)`
1684
+ or `./text()`, or through an unquoted CSS attribute value
1685
+ (`[data-k*=x-<id>]`), is kept whole, so row 1 no longer passes or clicks
1686
+ on "Order 12". A row value inside another XPath function, such as
1687
+ `translate` or `concat`, is refused instead of matched loosely.
1688
+ - A "not shown" check whose selector holds the text itself
1689
+ (`:has-text(...)`, `text=...`, XPath `contains(., ...)`) fails when the
1690
+ page shows that text with invisible characters around the row value,
1691
+ such as the bidi isolates Fluent and ICU add.
1692
+ - An invisible character (soft hyphen, zero-width space, bidi isolate) no
1693
+ longer counts as a word boundary, so "saved" does not pass on "unsaved"
1694
+ written with a soft hyphen, and row 1 does not pass on "Order 12" with a
1695
+ zero-width space inside.
1696
+ - A recorded response, attribute, style or JSON field pattern that
1697
+ backtracks fails the step at once with a message saying so, instead of
1698
+ freezing the run.
1699
+ - A "not shown" check treats straight and typographic quotes alike, so
1700
+ "Can't save" fails while the page shows "Can’t save".
1701
+ - Exported Playwright tests fail a recorded pattern that backtracks (a
1702
+ URL, a response body or JSON field, an attribute, a style) saying so, as
1703
+ replay does, no longer take an invisible format character as a word
1704
+ boundary, and read straight and typographic quotes alike in "not shown"
1705
+ checks.
1706
+ - A credential in an API base URL's query (`api.baseURL` or `--api-url`
1707
+ with `?api_key=`) is masked in request errors, the transcript and what
1708
+ the recording agent reads, as the URL's password already was; one in the
1709
+ browser `baseURL` query (`?token=`) is masked in traces too.
1710
+ - A failed header check never quotes a credential header's value
1711
+ (Set-Cookie) and masks a token in a Location it quotes.
1712
+ - A row value inside an XPath `or` (`contains(., "Order <id>") or
1713
+ contains(., "Order #<id>")`) is refused as one replay cannot tie to its
1714
+ row, as inside `translate` or `concat`: a "not shown" check passed on
1715
+ the other branch.
1716
+ - A "not shown" check through `:text(...)`, `:text-is(...)`,
1717
+ `:text-matches(...)`, XPath `normalize-space(.)="..."` or
1718
+ `text()="..."`, or an exact `internal:text="..."s` also finds the text
1719
+ with invisible characters around the row value.
1720
+ - A numeric one-time code or PIN in an API response (`"otp": 123456`) is
1721
+ hidden in the transcript and error lines, as a text one is.
1722
+ - A recorded pattern bounds a row value before `.*`, `[^,]*` or `.{0,9}`
1723
+ and more text: `/Order <id>.*Shipped/` for row 1 passed on "Order 12
1724
+ Shipped". An optional or loose point after the value (`<id>\.?`,
1725
+ `<id>[,;]`) no longer lets "1.5" or "1,000" pass for 1.
1726
+ - "Not shown" finds text in markup with no space between its elements
1727
+ (React, Vue, minified HTML): "Order 1" passed as gone while the row read
1728
+ "Order 1Pending", and "Bread" while it read "BreadIn stock". A "not
1729
+ shown" by role and name on a role whose name never comes from its text (a
1730
+ list item, a status, an alert, a paragraph) reads its text: it passed
1731
+ with the text still shown.
1732
+ - An attribute check on `checked`, `selected` or `value` reads the control
1733
+ as it is now, in replay and in exported tests: a box the app unticked
1734
+ passed as ticked.
1735
+ - Patterns, captures and amounts never read across a block's edge, as text
1736
+ checks and exported tests do: "Qty 1" over "0 left" no longer reads "Qty
1737
+ 10", nor "$1" over "5 nights" the amount 15. A price split into flex items
1738
+ ("$" "12" ".50") still reads "$12.50".
1739
+ - A row value written unquoted inside an XPath function
1740
+ (`contains(@data-order, <id>)`) is refused: row 1 found order 12.
1741
+ - A row's number is never written as `<column>` inside a longer number
1742
+ ("9.5" in "€9.50", "104" in "order-000104"): another row's "not shown"
1743
+ looked for "€12.250" and passed. A zero-padded id in a locator now binds
1744
+ the recording to its row, so other rows are red until the step names the
1745
+ id as the page writes it.
1746
+ - A two-column table with a repeated first cell is judged whole, so an
1747
+ edit to an earlier row re-records and seeds no other scenario.
1748
+ - A long number an Outline row's page sets in its own way ("+4712345678"
1749
+ shown "+47 123 45 678", an IBAN in blocks) binds the recording to its
1750
+ row, with guidance, where another row passed on it.
1751
+ - The recording agent cannot check a value against one captured in the same
1752
+ Then, nor a response by a pattern every URL matches, nor bind a database
1753
+ check to part of a row's value ("ann" of `ann@example.test`). A watch armed
1754
+ in a step replay ran before the session can be checked.
1755
+ - A recording the post-process refuses is told to the agent and ends as an
1756
+ ordinary red, its cost counted, not as the provider failing.
1757
+ - An API request whose path segment comes out blank (`/orders/{captured:id}`
1758
+ with an empty id) is refused, in replay and in exported tests: it went to
1759
+ the collection URL and passed. A credential a response echoes under a
1760
+ plain name is kept from other origins.
1761
+ - Database checks refuse MySQL `||` (run as OR), a WITH that selects a bind
1762
+ itself, and on PostgreSQL a backslash before a quote; "at most 0" on a
1763
+ count is an absence check, as "equals 0" is.
1764
+ - A kept trace no longer holds the saved login (`storageState`), basic-auth
1765
+ credentials or extra headers, and every cookie in its network log is
1766
+ redacted whatever its name. A `{refresh, access}` token pair's access
1767
+ token is a credential: hidden from the agent, the transcript and check
1768
+ failures, and kept from other origins.
1769
+ - A pinned contract applies `required` on a `readOnly` property to
1770
+ responses only and on a `writeOnly` one to requests only, so one shared
1771
+ model no longer refuses every request body. A response the contract
1772
+ gives as `*/*` is checked against its schema.
1773
+ - `saffron status`, `saffron coverage` and the editors keep the last run's
1774
+ result of each scenario when the report came from a CI machine on another
1775
+ OS.
1776
+ - `--rerecord` puts back a recording it removed when the scenario was not
1777
+ recorded again (the app down, a capability off, the budget spent,
1778
+ Ctrl-C). `run --plan --rerecord` replays a fixture, as the run does.
1779
+ - `saffron report --merge --strict` exits 1 on a pending review, as shards
1780
+ run with `--strict` do.
1781
+ - A parallel run's kept red no longer names evidence its second try
1782
+ deleted.
1783
+ - A feature edit that would drop an Outline `<column>` is skipped, and a
1784
+ file with mixed line endings keeps every other line's ending.
1785
+ - A short config key of the team's own (`app`, `id`) is ignored, not
1786
+ refused as a misspelling; a short key's letters swapped (`evn`) still is.
1787
+ - `differs from` compares `$5` and `5 USD` as one currency.
1788
+
1789
+ Editors (VS Code extension 0.8.2, JetBrains plugin 0.8.2)
1790
+
1791
+ - Database steps: a step recorded with a database check or write is not
1792
+ offered as a zero-token wording to reuse (completion, the step hover and
1793
+ StepSet expansion say it replays at zero tokens only in its own
1794
+ scenario), and a step is marked as one only when every recording of it
1795
+ reads or writes the database.
1796
+ - A proposal with the runner's new "database check repaired after a schema
1797
+ change:" or "database write added by the heal:" note is held from Accept
1798
+ All, as `saffron accept --all` skips it.
1799
+ - The last run's line, a CI run's totals and its rows count a red the app
1800
+ did not cause (a stale recording without the agent, a missing
1801
+ environment, the agent or the budget) as not run, as the runner's status
1802
+ does.
1803
+ - Open trace finds a CI scenario whose feature path the runner left out,
1804
+ by its name.
1805
+ - Eject's This file runs in the open file's own Saffron project, with
1806
+ forward slashes on Windows, and is not offered for a file outside every
1807
+ project.
1808
+ - VS Code: a status or diff that times out is stopped with its child
1809
+ processes, a cut diff says so, and `{data:...}` hints read the
1810
+ environment as the runner does. JetBrains: the Dashboard reloads only on
1811
+ new runner output, and proposals and lenses no longer read the disk on
1812
+ the UI thread.
1813
+
8
1814
  ## 1.3.5: Scenarios kept from running at once
9
1815
 
10
1816
  Released 2026-11-08.
@@ -1606,7 +3412,7 @@ Runner
1606
3412
  newest marked. A duration over a minute reads as minutes and seconds,
1607
3413
  and scenarios sharing a cause get a button for the command that deals
1608
3414
  with them all, the command itself still on the page.
1609
- - **A compatibility page per release.** [Compatibility](COMPATIBILITY.md)
3415
+ - **A compatibility page per release.** [Compatibility](docs/COMPATIBILITY.md)
1610
3416
  lists the Node.js, operating systems, Playwright and browsers, each
1611
3417
  agent's SDK (the range accepted and the version tested) and the recording
1612
3418
  versions this release reads and writes, generated from the release's own
@@ -1618,7 +3424,7 @@ Runner
1618
3424
  greens); `--record` also records the suite afresh with the agent and
1619
3425
  measures the share verified on the first attempt, the share held for
1620
3426
  review, and the time and cost. The results are published in the docs
1621
- ([benchmark](BENCHMARK.md)): today 66 of 66 replays green and no false
3427
+ ([benchmark](docs/BENCHMARK.md)): today 66 of 66 replays green and no false
1622
3428
  green on 8 broken apps.
1623
3429
  - **A coverage map per pull request.** `saffron coverage` maps each
1624
3430
  requirement ID (a tag matching `requirementTag`, such as `@REQ-12`) to
@@ -2383,7 +4189,7 @@ Runner
2383
4189
  ("red" in "Delivered", "Pro" in "Products") no longer binds the row; a
2384
4190
  camelCase part ("SelectedRed") still does. Rows whose data reads blank
2385
4191
  on every row no longer record each other forever. A row value holding a
2386
- secret's value ("admin@shop.test" beside a variable "admin") is the
4192
+ secret's value (`admin@shop.test` beside a variable "admin") is the
2387
4193
  row's `<column>`, and a secret equal to a row's lookup key leaves the
2388
4194
  step's `{data:...}` reference alone. A state check ("Promo" hidden, an
2389
4195
  attribute's pattern) binds the row whatever its element is named. A