saffron-ai 1.3.4 → 1.3.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,100 @@ the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
|
|
|
5
5
|
JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
|
|
6
6
|
release heading names its main feature: `## <version>: <title>`.
|
|
7
7
|
|
|
8
|
+
## 1.3.5: Scenarios kept from running at once
|
|
9
|
+
|
|
10
|
+
Released 2026-11-08.
|
|
11
|
+
|
|
12
|
+
Runner
|
|
13
|
+
|
|
14
|
+
- `@serial:<name>` keeps scenarios that would trip over each other's data
|
|
15
|
+
from running at the same time. With `--workers` or `--agent-workers`,
|
|
16
|
+
scenarios that share a name run one at a time, in file order, while the
|
|
17
|
+
rest of the suite runs beside them; a plain `@serial` is one shared name.
|
|
18
|
+
On a Feature line it covers the whole file. `--shard` keeps every
|
|
19
|
+
scenario under one name on the same machine, so two machines never run
|
|
20
|
+
them at once either.
|
|
21
|
+
- A value is no longer found inside a number grouped by spaces or
|
|
22
|
+
apostrophes: `250 kr` passed on `1 250 kr`, and `1 250` on
|
|
23
|
+
`1 250 000`. Nor is an amount found inside another currency written with
|
|
24
|
+
the same symbol: `$5.00` passed on `R$5.00` and `CA$5.00`. Both were
|
|
25
|
+
false greens, in text, pattern and download checks.
|
|
26
|
+
- A response check's row value is no longer found inside a longer word
|
|
27
|
+
written percent-encoded in the URL: `Jos` passed on `/users/Jos%C3%A9`.
|
|
28
|
+
- A captured decimal keeps the digits the API sent: `0.123456789012345678`
|
|
29
|
+
was rounded, so a `differs from {captured:...}` check passed on an
|
|
30
|
+
unchanged value, and a later request sent another number.
|
|
31
|
+
- `should have a non-empty "/id"` holds on a number: a numeric id was red
|
|
32
|
+
on every run.
|
|
33
|
+
- The recording agent no longer sees a base URL's password: the APIs'
|
|
34
|
+
summary in its prompt, a refused request and a request's errors quoted
|
|
35
|
+
it. A secret percent-encoded the way Python writes it (`%28`, `%2A`) is
|
|
36
|
+
masked too.
|
|
37
|
+
- The dashboard, `saffron status` and the CI run panel count reds the app
|
|
38
|
+
did not cause (a stale recording in a run with `--no-agent`) as *not
|
|
39
|
+
run*, as the HTML report does. The dashboard counted one as failed, and
|
|
40
|
+
said it had been red since before its grid. A scenario whose latest
|
|
41
|
+
result is such a red now counts as not run, never as passing on an older
|
|
42
|
+
green: the dashboard said 100% passing after a run that exited 1.
|
|
43
|
+
- "Not shown" no longer passes while the text is on screen beside a
|
|
44
|
+
number: `iPhone 15` was taken for part of `15 128` in
|
|
45
|
+
`iPhone 15 128 GB`. A positive check reads a number as grouped with the
|
|
46
|
+
next only after one to three digits that are not part of a word, so
|
|
47
|
+
`Galaxy S24` is found in `Galaxy S24 256 GB` again.
|
|
48
|
+
- Thai, Lao, Khmer and Myanmar digits keep a number whole (`1` passed on
|
|
49
|
+
`12` written in Myanmar digits), as do the Arabic decimal point and
|
|
50
|
+
thousands separator (`١٫٥`) and a thin space between thousands in a
|
|
51
|
+
downloaded file.
|
|
52
|
+
- A base URL password holding a raw `@` is masked whole: its tail after
|
|
53
|
+
the `@` reached the recording agent.
|
|
54
|
+
- An untyped `differs from` holds a number however the step writes it: an
|
|
55
|
+
unchanged `10.5` passed against `10.50`, and `100` against `1e2`.
|
|
56
|
+
- A captured list or object keeps each number as the API wrote it. A
|
|
57
|
+
decimal JavaScript cannot hold was sent on as a string; it is now
|
|
58
|
+
refused, as the same number captured alone is.
|
|
59
|
+
- `@serial` holds while a heal proves an Outline's other rows: a row with
|
|
60
|
+
its own `@serial` Examples tag was replayed without its name. Tags are
|
|
61
|
+
read in any case, with commas between names (`@Serial:Rooms` and
|
|
62
|
+
`@serial:a,b` were ignored), and `saffron eject` writes `workers: 1`
|
|
63
|
+
for a suite with `@serial` scenarios, or warns when the project has its
|
|
64
|
+
own Playwright config.
|
|
65
|
+
- "Not shown" counts a number beside a comma as shown: `2` passed while
|
|
66
|
+
`Tags: 1,2,3` was on screen, and `42` beside `1001,42`. It also counts
|
|
67
|
+
`$5.00` as shown in `US$5.00`, and a role name pattern with a row value
|
|
68
|
+
reads `15 128` as one number, as a text target does.
|
|
69
|
+
- `256 GB` is found in `Galaxy S24 256 GB`: a model number heads no
|
|
70
|
+
grouped number.
|
|
71
|
+
- The dashboard counts a scenario whose latest run was not run against
|
|
72
|
+
the pass rate: one scenario passing while the rest were not run read
|
|
73
|
+
100%, 1 of 1.
|
|
74
|
+
- A `differs from` checked as a string compares a number too large for
|
|
75
|
+
JavaScript by its digits: the same int64 id passed.
|
|
76
|
+
- A base URL password holding a quote or a space is masked before the
|
|
77
|
+
recording agent reads the base URL.
|
|
78
|
+
- The exported helper refuses `$10.50` for a role name recorded as
|
|
79
|
+
`Refund of $10.`, keeps a captured object's numbers as the API wrote
|
|
80
|
+
them, and `saffron eject` gives the trace warning beside the
|
|
81
|
+
`@serial` one, where it gave only the `@serial` one.
|
|
82
|
+
- An Outline's "not shown" check on a row value the recording glued into
|
|
83
|
+
a word reads the word in any form. Recorded on `apples` as `<item>s`,
|
|
84
|
+
row `box` checked that `boxs` was not shown and passed while `boxes`
|
|
85
|
+
was listed; it now counts `boxes` as shown.
|
|
86
|
+
- An exported "not shown" check looks in the app's own frames, as replay
|
|
87
|
+
does: it passed while `Card declined` showed in a same-origin iframe. It
|
|
88
|
+
also waits for the page's requests and for the hidden state to hold, as
|
|
89
|
+
replay does: it passed before an error drawn 300 ms after a click.
|
|
90
|
+
- The dashboard keeps every browser of the last CI run's matrix, where
|
|
91
|
+
each browser's shards are merged on their own: the browser merged last
|
|
92
|
+
hid the others' reds. A red from before browsers were recorded stands
|
|
93
|
+
when a later run is not run in one browser and passes in another.
|
|
94
|
+
- A `differs from` checked as a string fails on the same number, `true`
|
|
95
|
+
or `null`: it passed on any of them.
|
|
96
|
+
- `saffron eject` run again no longer warns about `@serial` when the
|
|
97
|
+
config it wrote already runs one test at a time.
|
|
98
|
+
- `run_plan` (MCP) and `saffron run --plan` name each `@serial` name and
|
|
99
|
+
how many tests share it, and the agent skill's syntax reference lists the
|
|
100
|
+
tags that change how a scenario runs.
|
|
101
|
+
|
|
8
102
|
## 1.3.4: Checks that wait for a settled page, and credentials kept masked
|
|
9
103
|
|
|
10
104
|
Released 2026-11-07.
|