saffron-ai 1.3.2 → 1.3.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +158 -0
- package/dist-pkg/cli.js +411 -394
- package/package.json +1 -1
- package/skills/saffron/SKILL.md +2 -2
- package/skills/saffron/references/api.md +37 -1
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,164 @@ the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
|
|
|
5
5
|
JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
|
|
6
6
|
release heading names its main feature: `## <version>: <title>`.
|
|
7
7
|
|
|
8
|
+
## 1.3.3: Concurrent API requests, and recordings that follow their data
|
|
9
|
+
|
|
10
|
+
Released 2026-11-05.
|
|
11
|
+
|
|
12
|
+
Runner
|
|
13
|
+
|
|
14
|
+
Upgrading from 1.3.2:
|
|
15
|
+
|
|
16
|
+
- Recordings with a request group (several API requests sent at once) are
|
|
17
|
+
schema version 19, and recordings with a count check use an action
|
|
18
|
+
1.3.2 does not know: 1.3.2 refuses both rather than replaying them
|
|
19
|
+
wrong. Upgrade every machine that replays them, CI included.
|
|
20
|
+
- Two new notes hold a proposal from `accept --all`: `check weakened:`
|
|
21
|
+
(a `Then` recorded again with a looser check than the committed one)
|
|
22
|
+
and `data not followed:` (a step that names a `{data:...}` value its
|
|
23
|
+
recording keeps as text). Accept such a proposal by its file after
|
|
24
|
+
reading why.
|
|
25
|
+
- While recording, each browser check the agent writes is run on its page
|
|
26
|
+
by replay's own engine, and refused when it does not hold there. A
|
|
27
|
+
recording that used to fail its proof on such a check now fixes it
|
|
28
|
+
before the proof.
|
|
29
|
+
- A scenario recorded again because its recording is stale seeds the
|
|
30
|
+
steps it did not change from its own recording, never from another
|
|
31
|
+
scenario's newer one.
|
|
32
|
+
|
|
33
|
+
- A downloaded file's check resolves a `{captured:...}` value, while
|
|
34
|
+
recording and at replay, so "the calendar file contains the booking
|
|
35
|
+
number `{captured:bookingNumber}`" checks this run's booking; one that
|
|
36
|
+
nothing captured fails saying so. Before, replay checked for the token
|
|
37
|
+
as text and the proof failed every file.
|
|
38
|
+
- A date the scenario writes as a token stays that token when the agent
|
|
39
|
+
types it glued to a time (`20261111T140000Z` for `{date+7:YYYYMMDD}`),
|
|
40
|
+
and a pattern that writes the time after it (`{date+7:YYYYMMDD}T[0-9]{6}`)
|
|
41
|
+
matches it. A time written as optional (`(?:T[0-9]{6})?`) still ends the
|
|
42
|
+
date, so the step writes it as required. A date left as text
|
|
43
|
+
in the format the scenario's own token writes gets `date not following
|
|
44
|
+
the clock:` and is held from `accept --all`.
|
|
45
|
+
- A short data value the step names (`{data:party.size}`, 2) is written
|
|
46
|
+
back as its reference where it stands alone once in a check's or fill's
|
|
47
|
+
value, a text pattern or an amount ("Table for 2 people", `"price" *
|
|
48
|
+
2`), never in a selector, URL or path, nor a digit of regex syntax, a
|
|
49
|
+
time or a date. A step whose recording still holds a value it
|
|
50
|
+
names as text, or a list's items one by one, gets a new `data not
|
|
51
|
+
followed:` note and is held from `accept --all`. A price in a locator's
|
|
52
|
+
name (`Info Lunch SEK 150`) is a `volatile locator:`.
|
|
53
|
+
- The language server sees a StepSet's steps filed with `accept
|
|
54
|
+
--step-set` (`.saffron/step-sets`) as recorded when they change.
|
|
55
|
+
- The plugins' keyword tables carry each language's Feature, Background
|
|
56
|
+
and Rule, so a file in another Gherkin language ends a scenario at them.
|
|
57
|
+
- A `StepSet:` or `Fixture:` in a file written in another Gherkin
|
|
58
|
+
language (`# language: de`) defines its set or fixture: it was read as
|
|
59
|
+
the English `Scenario:`, which that language does not know, so no set
|
|
60
|
+
was found. A StepSet is called in a language with no `*` step too
|
|
61
|
+
(Slovenian), and the `# language:` line is read as Gherkin reads it:
|
|
62
|
+
`# language: de # Deutsch` is English to both.
|
|
63
|
+
- The language server reads an `As "..."` session through Background and
|
|
64
|
+
Rule headers in the file's own language, and ends one at a `Fixture:`,
|
|
65
|
+
as the VS Code extension does.
|
|
66
|
+
- A file the agent uploads while recording is found in the project when
|
|
67
|
+
its path is relative, wherever the run was started: `saffron -p app
|
|
68
|
+
run` from a monorepo's root refused `demo-app/sample-upload.txt` as
|
|
69
|
+
outside the project.
|
|
70
|
+
- The benchmark's recording measure counts every note held from `accept
|
|
71
|
+
--all`, and names the scenarios not verified on the first attempt or
|
|
72
|
+
held, with why.
|
|
73
|
+
- A browser check the agent writes is run on its page by replay's own
|
|
74
|
+
engine before it is recorded, with replay's values, captures and clock,
|
|
75
|
+
and refused with replay's reason when it does not hold: a value not
|
|
76
|
+
whole in its text or an element not shown fails while recording, not
|
|
77
|
+
at the proof.
|
|
78
|
+
- A `Then` recorded again under the same words with a looser check than
|
|
79
|
+
the committed one gets a `check weakened:` note and is held from
|
|
80
|
+
`accept --all`: a value it no longer reads (the same value written
|
|
81
|
+
another way still counts), a pattern's or name pattern's lost `^` or
|
|
82
|
+
`$`, a text it checks less of, a kind of check it no longer makes or
|
|
83
|
+
covers, a JSON field it no longer compares, an element it no longer
|
|
84
|
+
names. `--rerecord` keeps the recording it removes for this comparison.
|
|
85
|
+
- A scenario recorded again because its recording is stale seeds each
|
|
86
|
+
step it did not change from its own committed recording where that fits
|
|
87
|
+
and the page shows what it acts on, never from another scenario's newer
|
|
88
|
+
one (a shared StepSet's update brought those in). A step that replayed
|
|
89
|
+
another scenario's recording is named in a `seeded from another
|
|
90
|
+
recording:` note.
|
|
91
|
+
- A check of an element that runs out of time (shown, text, pattern,
|
|
92
|
+
value, style, attribute) says what replay saw last: nothing matching
|
|
93
|
+
the target, an element there but not shown, or the text or value it
|
|
94
|
+
showed, with secrets masked before it is cut.
|
|
95
|
+
- A count check: "3 rooms should be listed" is one check of how many
|
|
96
|
+
shown elements the target finds in the page (or the frame it names),
|
|
97
|
+
from a number, a `<column>` or a `{data:...}` value; a hidden element
|
|
98
|
+
never counts, and none is written as "not shown". `saffron eject` does
|
|
99
|
+
not export it yet, and saffron-ai 1.3.2 refuses a recording with one as
|
|
100
|
+
an action it does not know, never replaying it wrong.
|
|
101
|
+
- The report page's headline counts and cost are this run's own after a
|
|
102
|
+
run of named scenarios, with the number of rows kept from earlier runs
|
|
103
|
+
beside them; before, it added up the kept rows' results and AI calls.
|
|
104
|
+
|
|
105
|
+
- Several API requests at once, each with its own values, and checks over
|
|
106
|
+
all their responses: an idempotency test that registers one charge from
|
|
107
|
+
several requests at the same moment. A step's table names the requests,
|
|
108
|
+
a header row and a row per request (`When I send these registrations
|
|
109
|
+
concurrently as "registrations":`); the request is recorded once and
|
|
110
|
+
sent once per row, every row before any answers, each row's cell
|
|
111
|
+
written as `<row:column>`, so editing a cell needs no recording again;
|
|
112
|
+
a `<row:column>` naming no column of the table is refused rather than
|
|
113
|
+
sent as text, and the group's name may come from an Examples column.
|
|
114
|
+
Every response stays with its row whatever order it arrives in, a
|
|
115
|
+
request that gets none is kept and fails the step once all have
|
|
116
|
+
settled, and a group is never sent again. `Then` steps check every
|
|
117
|
+
response, those of one status, a named one, at least some, or all of
|
|
118
|
+
them sharing a field's value (`all 200 responses should have the same
|
|
119
|
+
"/paymentReference"`); a response's own cell in a check is its row's.
|
|
120
|
+
The common checks are exact phrases recorded with no agent. A failure
|
|
121
|
+
lists each request with its status and why it failed. "The response"
|
|
122
|
+
after a group fails instead of reading one of them. Recordings with a
|
|
123
|
+
group are schema version 19: saffron-ai 1.3.2 and earlier refuse them.
|
|
124
|
+
`saffron eject` does not export a group yet.
|
|
125
|
+
|
|
126
|
+
- `saffron accept --propagate` fixes a healed API request everywhere it
|
|
127
|
+
is sent, as it already did for a locator. Scenarios that share a step's
|
|
128
|
+
wording hold copies of its request; when a heal rewrites that request
|
|
129
|
+
(a field the API now requires, a renamed path), accepting it with
|
|
130
|
+
`--propagate` rewrites every other cache that sends the same request
|
|
131
|
+
under the same step text, so one heal fixes them all instead of each
|
|
132
|
+
scenario healing on its own run. `--propagate` goes on the accept itself:
|
|
133
|
+
without it, accept names the caches it left as they are, which heal on
|
|
134
|
+
their own run. Either way it prints what the request was and is now. A
|
|
135
|
+
fix is not copied when it writes a value of the healed scenario's own (a
|
|
136
|
+
table cell, a doc string, its Outline row), adds a `<column>` or
|
|
137
|
+
`{captured:...}` value, drops a value the request extracted, sends the
|
|
138
|
+
request to another API, or when the heal kept the request as it was in
|
|
139
|
+
one place and rewrote it in another; accept says which. A step whose
|
|
140
|
+
heal added or removed an action is not copied either.
|
|
141
|
+
- `saffron accept` without `--propagate` no longer tells you to re-run it
|
|
142
|
+
with `--propagate`: the proposal is accepted by then, and a re-run had
|
|
143
|
+
nothing left to fix. It says the other caches were left to heal on
|
|
144
|
+
their own run.
|
|
145
|
+
|
|
146
|
+
Editors (VS Code extension 0.8.0, JetBrains plugin 0.8.0)
|
|
147
|
+
|
|
148
|
+
- Both editors mark a proposal with a `check weakened:` or `data not
|
|
149
|
+
followed:` note, or one that fails an Examples row, as held from Accept
|
|
150
|
+
All, as `saffron accept --all` skips it. JetBrains asks before Accept
|
|
151
|
+
All runs, as VS Code does.
|
|
152
|
+
- In a file written in another Gherkin language (`# language: de`),
|
|
153
|
+
Right-click Run reads that language's Feature, Background and Rule
|
|
154
|
+
headers (JetBrains), and VS Code's completion, recording badges, hovers
|
|
155
|
+
and wording hints read its step keywords and headers, a keyword with no
|
|
156
|
+
space of its own ("假如") included.
|
|
157
|
+
- A `Fixture:` block is a block of its own: it ends the StepSet above it
|
|
158
|
+
(VS Code), and Right-click Run inside it runs no scenario (JetBrains).
|
|
159
|
+
- VS Code shows the steps a StepSet's proposal filed for reuse
|
|
160
|
+
(`.saffron/step-sets`) as recorded, and no longer marks `Examples:
|
|
161
|
+
{data: roles}` as an error when a description line and then a table
|
|
162
|
+
follow it.
|
|
163
|
+
- JetBrains passes a run's files after `--`, so a file whose name starts
|
|
164
|
+
with a dash is a file, never an option.
|
|
165
|
+
|
|
8
166
|
## 1.3.2: Exact checks for amounts, URLs and numbers, and secrets kept masked
|
|
9
167
|
|
|
10
168
|
Released 2026-11-04.
|