saffron-ai 1.3.1 → 1.3.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +419 -0
- package/dist-pkg/cli.js +417 -400
- package/package.json +1 -1
- package/skills/saffron/SKILL.md +2 -2
- package/skills/saffron/references/api.md +37 -1
- package/skills/saffron/references/syntax.md +8 -4
- package/templates/playwright/saffron-helpers.ts +202 -64
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,425 @@ the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
|
|
|
5
5
|
JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
|
|
6
6
|
release heading names its main feature: `## <version>: <title>`.
|
|
7
7
|
|
|
8
|
+
## 1.3.3: Concurrent API requests, and recordings that follow their data
|
|
9
|
+
|
|
10
|
+
Released 2026-11-05.
|
|
11
|
+
|
|
12
|
+
Runner
|
|
13
|
+
|
|
14
|
+
Upgrading from 1.3.2:
|
|
15
|
+
|
|
16
|
+
- Recordings with a request group (several API requests sent at once) are
|
|
17
|
+
schema version 19, and recordings with a count check use an action
|
|
18
|
+
1.3.2 does not know: 1.3.2 refuses both rather than replaying them
|
|
19
|
+
wrong. Upgrade every machine that replays them, CI included.
|
|
20
|
+
- Two new notes hold a proposal from `accept --all`: `check weakened:`
|
|
21
|
+
(a `Then` recorded again with a looser check than the committed one)
|
|
22
|
+
and `data not followed:` (a step that names a `{data:...}` value its
|
|
23
|
+
recording keeps as text). Accept such a proposal by its file after
|
|
24
|
+
reading why.
|
|
25
|
+
- While recording, each browser check the agent writes is run on its page
|
|
26
|
+
by replay's own engine, and refused when it does not hold there. A
|
|
27
|
+
recording that used to fail its proof on such a check now fixes it
|
|
28
|
+
before the proof.
|
|
29
|
+
- A scenario recorded again because its recording is stale seeds the
|
|
30
|
+
steps it did not change from its own recording, never from another
|
|
31
|
+
scenario's newer one.
|
|
32
|
+
|
|
33
|
+
- A downloaded file's check resolves a `{captured:...}` value, while
|
|
34
|
+
recording and at replay, so "the calendar file contains the booking
|
|
35
|
+
number `{captured:bookingNumber}`" checks this run's booking; one that
|
|
36
|
+
nothing captured fails saying so. Before, replay checked for the token
|
|
37
|
+
as text and the proof failed every file.
|
|
38
|
+
- A date the scenario writes as a token stays that token when the agent
|
|
39
|
+
types it glued to a time (`20261111T140000Z` for `{date+7:YYYYMMDD}`),
|
|
40
|
+
and a pattern that writes the time after it (`{date+7:YYYYMMDD}T[0-9]{6}`)
|
|
41
|
+
matches it. A time written as optional (`(?:T[0-9]{6})?`) still ends the
|
|
42
|
+
date, so the step writes it as required. A date left as text
|
|
43
|
+
in the format the scenario's own token writes gets `date not following
|
|
44
|
+
the clock:` and is held from `accept --all`.
|
|
45
|
+
- A short data value the step names (`{data:party.size}`, 2) is written
|
|
46
|
+
back as its reference where it stands alone once in a check's or fill's
|
|
47
|
+
value, a text pattern or an amount ("Table for 2 people", `"price" *
|
|
48
|
+
2`), never in a selector, URL or path, nor a digit of regex syntax, a
|
|
49
|
+
time or a date. A step whose recording still holds a value it
|
|
50
|
+
names as text, or a list's items one by one, gets a new `data not
|
|
51
|
+
followed:` note and is held from `accept --all`. A price in a locator's
|
|
52
|
+
name (`Info Lunch SEK 150`) is a `volatile locator:`.
|
|
53
|
+
- The language server sees a StepSet's steps filed with `accept
|
|
54
|
+
--step-set` (`.saffron/step-sets`) as recorded when they change.
|
|
55
|
+
- The plugins' keyword tables carry each language's Feature, Background
|
|
56
|
+
and Rule, so a file in another Gherkin language ends a scenario at them.
|
|
57
|
+
- A `StepSet:` or `Fixture:` in a file written in another Gherkin
|
|
58
|
+
language (`# language: de`) defines its set or fixture: it was read as
|
|
59
|
+
the English `Scenario:`, which that language does not know, so no set
|
|
60
|
+
was found. A StepSet is called in a language with no `*` step too
|
|
61
|
+
(Slovenian), and the `# language:` line is read as Gherkin reads it:
|
|
62
|
+
`# language: de # Deutsch` is English to both.
|
|
63
|
+
- The language server reads an `As "..."` session through Background and
|
|
64
|
+
Rule headers in the file's own language, and ends one at a `Fixture:`,
|
|
65
|
+
as the VS Code extension does.
|
|
66
|
+
- A file the agent uploads while recording is found in the project when
|
|
67
|
+
its path is relative, wherever the run was started: `saffron -p app
|
|
68
|
+
run` from a monorepo's root refused `demo-app/sample-upload.txt` as
|
|
69
|
+
outside the project.
|
|
70
|
+
- The benchmark's recording measure counts every note held from `accept
|
|
71
|
+
--all`, and names the scenarios not verified on the first attempt or
|
|
72
|
+
held, with why.
|
|
73
|
+
- A browser check the agent writes is run on its page by replay's own
|
|
74
|
+
engine before it is recorded, with replay's values, captures and clock,
|
|
75
|
+
and refused with replay's reason when it does not hold: a value not
|
|
76
|
+
whole in its text or an element not shown fails while recording, not
|
|
77
|
+
at the proof.
|
|
78
|
+
- A `Then` recorded again under the same words with a looser check than
|
|
79
|
+
the committed one gets a `check weakened:` note and is held from
|
|
80
|
+
`accept --all`: a value it no longer reads (the same value written
|
|
81
|
+
another way still counts), a pattern's or name pattern's lost `^` or
|
|
82
|
+
`$`, a text it checks less of, a kind of check it no longer makes or
|
|
83
|
+
covers, a JSON field it no longer compares, an element it no longer
|
|
84
|
+
names. `--rerecord` keeps the recording it removes for this comparison.
|
|
85
|
+
- A scenario recorded again because its recording is stale seeds each
|
|
86
|
+
step it did not change from its own committed recording where that fits
|
|
87
|
+
and the page shows what it acts on, never from another scenario's newer
|
|
88
|
+
one (a shared StepSet's update brought those in). A step that replayed
|
|
89
|
+
another scenario's recording is named in a `seeded from another
|
|
90
|
+
recording:` note.
|
|
91
|
+
- A check of an element that runs out of time (shown, text, pattern,
|
|
92
|
+
value, style, attribute) says what replay saw last: nothing matching
|
|
93
|
+
the target, an element there but not shown, or the text or value it
|
|
94
|
+
showed, with secrets masked before it is cut.
|
|
95
|
+
- A count check: "3 rooms should be listed" is one check of how many
|
|
96
|
+
shown elements the target finds in the page (or the frame it names),
|
|
97
|
+
from a number, a `<column>` or a `{data:...}` value; a hidden element
|
|
98
|
+
never counts, and none is written as "not shown". `saffron eject` does
|
|
99
|
+
not export it yet, and saffron-ai 1.3.2 refuses a recording with one as
|
|
100
|
+
an action it does not know, never replaying it wrong.
|
|
101
|
+
- The report page's headline counts and cost are this run's own after a
|
|
102
|
+
run of named scenarios, with the number of rows kept from earlier runs
|
|
103
|
+
beside them; before, it added up the kept rows' results and AI calls.
|
|
104
|
+
|
|
105
|
+
- Several API requests at once, each with its own values, and checks over
|
|
106
|
+
all their responses: an idempotency test that registers one charge from
|
|
107
|
+
several requests at the same moment. A step's table names the requests,
|
|
108
|
+
a header row and a row per request (`When I send these registrations
|
|
109
|
+
concurrently as "registrations":`); the request is recorded once and
|
|
110
|
+
sent once per row, every row before any answers, each row's cell
|
|
111
|
+
written as `<row:column>`, so editing a cell needs no recording again;
|
|
112
|
+
a `<row:column>` naming no column of the table is refused rather than
|
|
113
|
+
sent as text, and the group's name may come from an Examples column.
|
|
114
|
+
Every response stays with its row whatever order it arrives in, a
|
|
115
|
+
request that gets none is kept and fails the step once all have
|
|
116
|
+
settled, and a group is never sent again. `Then` steps check every
|
|
117
|
+
response, those of one status, a named one, at least some, or all of
|
|
118
|
+
them sharing a field's value (`all 200 responses should have the same
|
|
119
|
+
"/paymentReference"`); a response's own cell in a check is its row's.
|
|
120
|
+
The common checks are exact phrases recorded with no agent. A failure
|
|
121
|
+
lists each request with its status and why it failed. "The response"
|
|
122
|
+
after a group fails instead of reading one of them. Recordings with a
|
|
123
|
+
group are schema version 19: saffron-ai 1.3.2 and earlier refuse them.
|
|
124
|
+
`saffron eject` does not export a group yet.
|
|
125
|
+
|
|
126
|
+
- `saffron accept --propagate` fixes a healed API request everywhere it
|
|
127
|
+
is sent, as it already did for a locator. Scenarios that share a step's
|
|
128
|
+
wording hold copies of its request; when a heal rewrites that request
|
|
129
|
+
(a field the API now requires, a renamed path), accepting it with
|
|
130
|
+
`--propagate` rewrites every other cache that sends the same request
|
|
131
|
+
under the same step text, so one heal fixes them all instead of each
|
|
132
|
+
scenario healing on its own run. `--propagate` goes on the accept itself:
|
|
133
|
+
without it, accept names the caches it left as they are, which heal on
|
|
134
|
+
their own run. Either way it prints what the request was and is now. A
|
|
135
|
+
fix is not copied when it writes a value of the healed scenario's own (a
|
|
136
|
+
table cell, a doc string, its Outline row), adds a `<column>` or
|
|
137
|
+
`{captured:...}` value, drops a value the request extracted, sends the
|
|
138
|
+
request to another API, or when the heal kept the request as it was in
|
|
139
|
+
one place and rewrote it in another; accept says which. A step whose
|
|
140
|
+
heal added or removed an action is not copied either.
|
|
141
|
+
- `saffron accept` without `--propagate` no longer tells you to re-run it
|
|
142
|
+
with `--propagate`: the proposal is accepted by then, and a re-run had
|
|
143
|
+
nothing left to fix. It says the other caches were left to heal on
|
|
144
|
+
their own run.
|
|
145
|
+
|
|
146
|
+
Editors (VS Code extension 0.8.0, JetBrains plugin 0.8.0)
|
|
147
|
+
|
|
148
|
+
- Both editors mark a proposal with a `check weakened:` or `data not
|
|
149
|
+
followed:` note, or one that fails an Examples row, as held from Accept
|
|
150
|
+
All, as `saffron accept --all` skips it. JetBrains asks before Accept
|
|
151
|
+
All runs, as VS Code does.
|
|
152
|
+
- In a file written in another Gherkin language (`# language: de`),
|
|
153
|
+
Right-click Run reads that language's Feature, Background and Rule
|
|
154
|
+
headers (JetBrains), and VS Code's completion, recording badges, hovers
|
|
155
|
+
and wording hints read its step keywords and headers, a keyword with no
|
|
156
|
+
space of its own ("假如") included.
|
|
157
|
+
- A `Fixture:` block is a block of its own: it ends the StepSet above it
|
|
158
|
+
(VS Code), and Right-click Run inside it runs no scenario (JetBrains).
|
|
159
|
+
- VS Code shows the steps a StepSet's proposal filed for reuse
|
|
160
|
+
(`.saffron/step-sets`) as recorded, and no longer marks `Examples:
|
|
161
|
+
{data: roles}` as an error when a description line and then a table
|
|
162
|
+
follow it.
|
|
163
|
+
- JetBrains passes a run's files after `--`, so a file whose name starts
|
|
164
|
+
with a dash is a file, never an option.
|
|
165
|
+
|
|
166
|
+
## 1.3.2: Exact checks for amounts, URLs and numbers, and secrets kept masked
|
|
167
|
+
|
|
168
|
+
Released 2026-11-04.
|
|
169
|
+
|
|
170
|
+
Runner
|
|
171
|
+
|
|
172
|
+
Upgrading from 1.3.1:
|
|
173
|
+
|
|
174
|
+
- A URL check (`expectUrl`, contains) reads its text as a whole part of
|
|
175
|
+
the URL, and a path (text with a `/`) outside the query, unless the text
|
|
176
|
+
names a query (`?`, `=`, `&`): `/dashboard` no longer holds on
|
|
177
|
+
`/login?next=%2Fdashboard`, nor `/orders/42` on `/orders/421`. A word
|
|
178
|
+
with no slash (`shoes`) is still found in `/search?q=red+shoes`, but not
|
|
179
|
+
in a query value that is a path (`dashboard` in `/login?next=/dashboard`),
|
|
180
|
+
nor in a query value whose encoded `#` or `&` makes it look like a
|
|
181
|
+
fragment or another field (`/dashboard` in `?next=%2F%23%2Fdashboard`).
|
|
182
|
+
A check that the URL does not hold a text still reads it anywhere:
|
|
183
|
+
`error` is in `?errorCode=5`. A text that names a query is the page's
|
|
184
|
+
own: its `?` is where the URL's query starts and a field it names is
|
|
185
|
+
one of the URL's own, so `/dashboard?tab=settings` and `tab=settings`
|
|
186
|
+
no longer hold on `/login?next=%2Fdashboard%3Ftab%3Dsettings`, nor
|
|
187
|
+
`/dashboard` on an OAuth callback's `#state=%2Fdashboard`. A fragment's
|
|
188
|
+
fields are the page's own too: `#tab=settings`, `/app#tab=settings` and
|
|
189
|
+
`tab=settings` hold on `/app#tab=settings`. A recording
|
|
190
|
+
that passed only that way turns red; the exported test checks the same.
|
|
191
|
+
- `Examples: {data: roles}` (a space, a capital `Data`) is an error naming
|
|
192
|
+
the file and line, where the Outline used to run no row at all. The
|
|
193
|
+
language server marks it while it is typed.
|
|
194
|
+
- An expected text never matches inside a longer word or number, whether
|
|
195
|
+
a row value or written as it is (SAF-197): `"$10"` no longer holds on
|
|
196
|
+
`$100`, `"1 item"` on `11 items`, nor a text target `Saved` on "You have
|
|
197
|
+
unsaved changes". A role's name may still have grown a suffix. A number
|
|
198
|
+
written in the step may be followed by a zero decimal part (`"$10"` in
|
|
199
|
+
`$10.00`, never `$10.50`, `₹1,00,000` or `2.0.1`); a row value keeps its
|
|
200
|
+
decimals whole (`<id>` 1 is not in `1.00`), and neither is found before
|
|
201
|
+
a digit in an inline tag (`Order 1<b>2</b>`, `$10<sup>.50</sup>`). A
|
|
202
|
+
text ending in a number's point or comma takes no more digits
|
|
203
|
+
(`"Refund of $10."` is not in `Refund of $10.50`), and a number is not
|
|
204
|
+
found in its negative (`"5.00"` is not in `-5.00` or `−5.00`; `"5"` is
|
|
205
|
+
still in `pages 3-5`). A value that starts or ends in Chinese, Japanese,
|
|
206
|
+
Thai, Lao, Khmer or Myanmar script is not bounded there, as those
|
|
207
|
+
scripts put no space between words, but for a negating prefix:
|
|
208
|
+
`"成功"` is not in `不成功` nor `没有成功`, `"保存"` not in `无法保存`.
|
|
209
|
+
Only a negation right before the value is read: a short value in a
|
|
210
|
+
sentence that negates it further back can still be found. A recording that passed only that way turns red, and the agent
|
|
211
|
+
records the whole word next time.
|
|
212
|
+
- An expected text never runs across a block, a table cell or a `<br>`
|
|
213
|
+
(SAF-198): `"15.00"` no longer holds on the cells `1` and `5.00`, nor
|
|
214
|
+
`"Qty 10"` on the items `Qty 1` and `0 left in stock`. Items of a flex
|
|
215
|
+
row or a grid read as one line, a space apart: `"Total: $10"` holds on
|
|
216
|
+
the items `Total:` and `$10`. A value written on several lines still
|
|
217
|
+
matches the lines the page draws.
|
|
218
|
+
- A style or attribute check needs its element shown (SAF-199): "the
|
|
219
|
+
error is shown in red" no longer holds on an error that is not on
|
|
220
|
+
screen, nor on a shown fallback when the recorded element is on the page
|
|
221
|
+
but hidden. A check of what hides an element (`display`, `visibility`,
|
|
222
|
+
`opacity`, `content-visibility`; `hidden`, `aria-hidden`, `inert`) still
|
|
223
|
+
holds on a hidden one, and so does one of a `class`, a `style`, `open`
|
|
224
|
+
or `data-state` attribute, or a height, width, transform or clip: how
|
|
225
|
+
many apps hide an element (`d-none`, a closed dialog).
|
|
226
|
+
|
|
227
|
+
The exported Playwright tests check text, blocks and styles the same way
|
|
228
|
+
(SAF-197, SAF-198, SAF-199).
|
|
229
|
+
|
|
230
|
+
- A text check of an amount written with its currency (`$5.00`, `€5`)
|
|
231
|
+
no longer holds on the negative amount (`-$5.00`, `−€5`), as `5.00`
|
|
232
|
+
already did not on `-5.00`. A URL check of a number takes no decimal
|
|
233
|
+
part (`amount=10` is not in `?amount=10.50`, nor `/orders/42` in
|
|
234
|
+
`/orders/42.5`), and a letter or digit the URL writes percent-encoded
|
|
235
|
+
belongs to the word beside it (`name=Jos` is not in `?name=Jos%C3%A9`,
|
|
236
|
+
nor `id=42` in `?id=42%35`). The exported test checks the same.
|
|
237
|
+
|
|
238
|
+
Fixed, from a review of the runner:
|
|
239
|
+
|
|
240
|
+
- A credential stays masked wherever it travels: a fixture's token that a
|
|
241
|
+
later capture replaced, or one inside an extracted object, in the
|
|
242
|
+
scenarios the fixture seeds; and a replaced token in what the agent
|
|
243
|
+
reads while recording, which it used to show. An exported test masks a
|
|
244
|
+
secret as a URL encodes it (`%2B`, `%2F`, `%3D`) and in the errors of
|
|
245
|
+
its JSON checks, as replay does.
|
|
246
|
+
- A `differs` check on an object holding an integer beyond JavaScript's
|
|
247
|
+
precision no longer passes against the same object a capture kept.
|
|
248
|
+
- The dashboard's suite health no longer reads 100% while a scenario
|
|
249
|
+
failed (473 of 475 reads 99.58%).
|
|
250
|
+
- The language server no longer marks `Examples: {data: roles}` as an
|
|
251
|
+
error inside a doc string, nor when a description comes before its
|
|
252
|
+
table: the runner reads both.
|
|
253
|
+
|
|
254
|
+
- No false green from a fallback selector or an iframe: the first locator
|
|
255
|
+
that finds a shown element decides a check, so a fallback that happens
|
|
256
|
+
to show the expected text no longer vouches for the recorded element
|
|
257
|
+
that failed it. A recorded element that is gone, or hidden for a check
|
|
258
|
+
other than a style or attribute check, still gives way to its fallback.
|
|
259
|
+
- No false green from an API check whose pattern matches any text: a
|
|
260
|
+
field, header or `until` pattern such as `.*`, `^` or an empty one is
|
|
261
|
+
refused while recording, as page checks already were.
|
|
262
|
+
- No Outline row passes on another row's data when the app writes a row
|
|
263
|
+
value without its accents (`José` as `status-jose`), or when a column
|
|
264
|
+
names a section without quotes (`the <plan> card shows "Popular"`).
|
|
265
|
+
- A captured id of one or two characters is recorded as its token where
|
|
266
|
+
a path segment or field names its capture (`/orders/42` for `order`), so
|
|
267
|
+
replay no longer reads the record made while recording. A field that
|
|
268
|
+
names another id (`productId: 1`) is left as it is.
|
|
269
|
+
- A secret in an API query that holds `/` together with `+`, `=` or `&`
|
|
270
|
+
(a base64 key) is masked in errors, transcripts, reports and the
|
|
271
|
+
agent's replies.
|
|
272
|
+
- A page or API message cut for a report is cut with `…`, so a secret
|
|
273
|
+
across the cut is masked too, also when the message carries it
|
|
274
|
+
URL-encoded.
|
|
275
|
+
- A request method taken from a column (`a <verb> request`) is kept as
|
|
276
|
+
sent, which binds the recording to its row, instead of an invalid
|
|
277
|
+
recording made again on every run.
|
|
278
|
+
- An integer above 2^53 extracted from a response (a snowflake id) keeps
|
|
279
|
+
the digits the API sent, instead of being rounded to another id. A JSON
|
|
280
|
+
check reads the same digits, so "no longer the same id" fails on the
|
|
281
|
+
same id, and a request naming the captured id still sends its token.
|
|
282
|
+
- A differs check finds the element again when the app draws it anew
|
|
283
|
+
(a React remount), where it used to fail for good.
|
|
284
|
+
- A shard run removes its own earlier report as it starts, so a shard
|
|
285
|
+
that then fails setup, crashes or is stopped is missing from
|
|
286
|
+
`report --merge` instead of standing in with the earlier run's results.
|
|
287
|
+
It also goes when the run stops before running anything (a feature file
|
|
288
|
+
error, a `--filter` that matches nothing).
|
|
289
|
+
- `saffron eject` warns when the project's own `playwright.config` records
|
|
290
|
+
a trace and the exported tests read secrets, `{env:...}` values or
|
|
291
|
+
credentials the API hands out, which Playwright's trace keeps unmasked
|
|
292
|
+
(SAF-200). It no longer only checked on the first eject, and it leaves
|
|
293
|
+
the file as it is. A config it writes leaves the trace out for
|
|
294
|
+
captured credentials too.
|
|
295
|
+
- A pending proposal stays with the Outline row whose values it was
|
|
296
|
+
recorded on when a row is inserted above it or the rows are reordered
|
|
297
|
+
(SAF-201), where another row used to take its "(example N)" and record
|
|
298
|
+
over it. `saffron status` and the plugins name that row as the rows
|
|
299
|
+
stand now, and the review page shows that row's pictures, where they
|
|
300
|
+
used to name and show the row that took its place.
|
|
301
|
+
- An API credential is masked wherever it could still show: a token a
|
|
302
|
+
polled response handed out before the one waited for, a token a later
|
|
303
|
+
capture replaced (a poll that rotates it, then quotes the old one as
|
|
304
|
+
`previous_token`), and an error raised while a request is built (a body
|
|
305
|
+
value that is no number quoting the captured token it resolves to). The
|
|
306
|
+
exported test learns a token from every polled response too, where it
|
|
307
|
+
learnt only the last one's and an error quoted the one before.
|
|
308
|
+
- A compiled `Then` keeps a quoted value as written: `the /name field is
|
|
309
|
+
"Ada Lovelace"` checks for both spaces, where it used to check for one.
|
|
310
|
+
- `saffron accept <file>#<revision> --step-set` refuses a proposal replaced
|
|
311
|
+
since it was listed, as a whole accept does, where it took the
|
|
312
|
+
replacement's steps unseen.
|
|
313
|
+
- A request sent several times at once waits for every copy when one gets
|
|
314
|
+
no response, and keeps the copies that answered in the transcript, where
|
|
315
|
+
the failure came first and a late write went unrecorded.
|
|
316
|
+
- An exported test extracts an integer beyond JavaScript's precision as the
|
|
317
|
+
digits the API sent, as replay does, where it rounded an int64 id into
|
|
318
|
+
another record's.
|
|
319
|
+
- Each API response is let go once read, in replay and in exported tests,
|
|
320
|
+
so a long poll no longer holds every response it read until the scenario
|
|
321
|
+
ends.
|
|
322
|
+
- An exported text check reads the copy replay reads, a shown one after a
|
|
323
|
+
hidden copy with the same text (a mobile menu's), where it read the
|
|
324
|
+
hidden one and failed.
|
|
325
|
+
- A step naming its column for the statuses of copies sent at once
|
|
326
|
+
(`refused with <refusal>`) records them as that column, as it records a
|
|
327
|
+
single status, so the other rows replay where they were refused as
|
|
328
|
+
recorded for one row.
|
|
329
|
+
- A JSON check reads a number JavaScript cannot hold as written as a
|
|
330
|
+
number, written as the API sent it: a `min` or `max` compares it exactly
|
|
331
|
+
with the bound as written, in any notation (`9007199254740993`,
|
|
332
|
+
`9007199254740993.0` and `9.007199254740993e15` are all over a maximum
|
|
333
|
+
of `9007199254740992`, `9007199254740991.1` over one of
|
|
334
|
+
`9007199254740991`, and `1e9999999999` over `1`, where it stopped the
|
|
335
|
+
run), and a `pattern` (which reads text) does not hold on it. An `equals` still checks its digits, as the error for a typed
|
|
336
|
+
number advises.
|
|
337
|
+
- A compiled `Then` leaves a number JSON cannot hold (`the /n field is
|
|
338
|
+
007`) to the agent, where it compiled a check that could never pass.
|
|
339
|
+
|
|
340
|
+
Fixed, from real-world use of 1.3.1:
|
|
341
|
+
|
|
342
|
+
- The report's pass rate never reads 100% while a scenario failed: 473 of
|
|
343
|
+
475 reads 99.58%, where it was rounded up to 100%. The dashboard's
|
|
344
|
+
suite health, recorded and tag shares read the same way, and 0% means
|
|
345
|
+
none. `passRate` in
|
|
346
|
+
`saffron status --json` and the history stays a whole number, now never
|
|
347
|
+
100 unless every scenario passed (99 for 473 of 475).
|
|
348
|
+
|
|
349
|
+
- An Outline row that fails in a step before any step uses its own values
|
|
350
|
+
(a lunch sold out in a shared checkout step) is blamed on the app's
|
|
351
|
+
state, blocked by the environment, where the report used to call it
|
|
352
|
+
the recording's fault and point at a column the step never read. A
|
|
353
|
+
value used in a step's data table or doc string counts as used.
|
|
354
|
+
- A pending proposal stays with its row when a column is added to the
|
|
355
|
+
Examples: only the columns its row had tell rows apart, where every row
|
|
356
|
+
was called another row's and none recorded again.
|
|
357
|
+
- The sensitivity check judges a check against the action before a step
|
|
358
|
+
that only reads (a captured value, a watch armed), as it already did
|
|
359
|
+
for a wait: reading a booking number is not what confirms a payment.
|
|
360
|
+
- `api_contract` (the MCP tool) and the agent's contract summary list a
|
|
361
|
+
request body's alternatives with their fields (`one of: { ... } | { ...
|
|
362
|
+
}`) and an `allOf`'s parts as one body, where they said `any`.
|
|
363
|
+
- The Saffron skill's syntax reference describes the negative and exact
|
|
364
|
+
URL checks 1.3.1 added, where it said there were none.
|
|
365
|
+
- A recorded API step no longer keeps the agent's failed attempts
|
|
366
|
+
(SAF-202). A request whose body the pinned contract refuses is not
|
|
367
|
+
recorded, even when the API answered 2xx, unless the step expects a
|
|
368
|
+
refusal. The body is judged as sent, its tokens resolved: `{date+1}` as
|
|
369
|
+
a date, and `{captured:id}` as the text it sends, so an integer field
|
|
370
|
+
takes the number itself. A
|
|
371
|
+
request sent again to the same operation and expected answer right
|
|
372
|
+
after the step's last one takes its place, unless the step asks for
|
|
373
|
+
several ("twice", "two customers", "for each", "once more"), the earlier
|
|
374
|
+
one sent a value the step names that this one does not ("the currency
|
|
375
|
+
to SEK and guest names on" is two writes), this one sends the same
|
|
376
|
+
fields with other values the step does not name ("a mug and a hat"), or
|
|
377
|
+
both carry the same idempotency key. A value counts as named in any
|
|
378
|
+
case and in the step's data table or doc string, and as sent in a form
|
|
379
|
+
or the query too (`?status=draft`, then `?status=archived`); a step
|
|
380
|
+
with a table of several rows keeps every request.
|
|
381
|
+
- A `Then` that names a section in plain words (`the confirmation Summary
|
|
382
|
+
shows the package`, `the order card shows Paid`) has its checks scoped
|
|
383
|
+
to that section, as a quoted name already did (SAF-203): a check that
|
|
384
|
+
reads the whole page is refused while recording, instead of passing on
|
|
385
|
+
text that was there before the action.
|
|
386
|
+
- An Outline whose recorded row held one value in two columns
|
|
387
|
+
(`currency` and `display` both `NOK`) no longer refuses the other rows
|
|
388
|
+
for a step that names only one of them (`I choose "<currency>"`): the
|
|
389
|
+
recorder writes the column the step names. A step naming both, or
|
|
390
|
+
neither, still records again on a row that types them apart.
|
|
391
|
+
- A request that sends its header's credential in the body or URL as well
|
|
392
|
+
(`Authorization: Bearer {env:KEY}` and a body field `apiKey`) is refused
|
|
393
|
+
while recording, unless the step names both the body and the header.
|
|
394
|
+
Only an auth header counts: `X-Auth-Token`, not `X-Author` or
|
|
395
|
+
`X-Auth-User`.
|
|
396
|
+
- A "not shown" check naming a control by the start of its name (a
|
|
397
|
+
textbox `Firstname`, where the page shows `Firstname Required`) gets an
|
|
398
|
+
insensitive check note saying no element has that name, and naming the
|
|
399
|
+
control shown before the step.
|
|
400
|
+
- A check that finds its element by the text it checks (an amount found as
|
|
401
|
+
the text `SEK 495.00` anywhere on the page) is refused while recording:
|
|
402
|
+
it holds wherever that text shows. The agent finds the field the step
|
|
403
|
+
names, or records a visible check when the step only says the text is
|
|
404
|
+
shown.
|
|
405
|
+
|
|
406
|
+
Docs
|
|
407
|
+
|
|
408
|
+
- **How a recording is checked.** A new section of the guide (§24), and a
|
|
409
|
+
page of its own on the site, maps every check Saffron runs on a
|
|
410
|
+
recording before it is filed, in order: what each note means, which ones
|
|
411
|
+
hold a proposal from `accept --all` and when to accept it by file, what
|
|
412
|
+
is on by default, and how to turn on stability proofs.
|
|
413
|
+
|
|
414
|
+
Editors (VS Code extension 0.7.8, JetBrains plugin 0.7.9)
|
|
415
|
+
|
|
416
|
+
- Both editors mark `Examples: {data: roles}` or `{Data:roles}` with no
|
|
417
|
+
table under it as an error where it is written, as the runner stops on
|
|
418
|
+
it: VS Code in every project, JetBrains through the runner's language
|
|
419
|
+
server. Neither marks one inside a doc string or with a description
|
|
420
|
+
before its table.
|
|
421
|
+
- VS Code: the "Proposals pending review" row asks before it runs
|
|
422
|
+
`saffron accept --all`.
|
|
423
|
+
- JetBrains: Accept All and an accept or reject of selected proposals no
|
|
424
|
+
longer become the selected run configuration, so the Run button never
|
|
425
|
+
repeats them.
|
|
426
|
+
|
|
8
427
|
## 1.3.1: Checks for what vanishes, URL and style checks, and a StepSet accepted on its own
|
|
9
428
|
|
|
10
429
|
Released 2026-11-02.
|