saffron-ai 1.3.4 → 1.3.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,100 @@ the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
5
5
  JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
6
6
  release heading names its main feature: `## <version>: <title>`.
7
7
 
8
+ ## 1.3.5: Scenarios kept from running at once
9
+
10
+ Released 2026-11-08.
11
+
12
+ Runner
13
+
14
+ - `@serial:<name>` keeps scenarios that would trip over each other's data
15
+ from running at the same time. With `--workers` or `--agent-workers`,
16
+ scenarios that share a name run one at a time, in file order, while the
17
+ rest of the suite runs beside them; a plain `@serial` is one shared name.
18
+ On a Feature line it covers the whole file. `--shard` keeps every
19
+ scenario under one name on the same machine, so two machines never run
20
+ them at once either.
21
+ - A value is no longer found inside a number grouped by spaces or
22
+ apostrophes: `250 kr` passed on `1 250 kr`, and `1 250` on
23
+ `1 250 000`. Nor is an amount found inside another currency written with
24
+ the same symbol: `$5.00` passed on `R$5.00` and `CA$5.00`. Both were
25
+ false greens, in text, pattern and download checks.
26
+ - A response check's row value is no longer found inside a longer word
27
+ written percent-encoded in the URL: `Jos` passed on `/users/Jos%C3%A9`.
28
+ - A captured decimal keeps the digits the API sent: `0.123456789012345678`
29
+ was rounded, so a `differs from {captured:...}` check passed on an
30
+ unchanged value, and a later request sent another number.
31
+ - `should have a non-empty "/id"` holds on a number: a numeric id was red
32
+ on every run.
33
+ - The recording agent no longer sees a base URL's password: the APIs'
34
+ summary in its prompt, a refused request and a request's errors quoted
35
+ it. A secret percent-encoded the way Python writes it (`%28`, `%2A`) is
36
+ masked too.
37
+ - The dashboard, `saffron status` and the CI run panel count reds the app
38
+ did not cause (a stale recording in a run with `--no-agent`) as *not
39
+ run*, as the HTML report does. The dashboard counted one as failed, and
40
+ said it had been red since before its grid. A scenario whose latest
41
+ result is such a red now counts as not run, never as passing on an older
42
+ green: the dashboard said 100% passing after a run that exited 1.
43
+ - "Not shown" no longer passes while the text is on screen beside a
44
+ number: `iPhone 15` was taken for part of `15 128` in
45
+ `iPhone 15 128 GB`. A positive check reads a number as grouped with the
46
+ next only after one to three digits that are not part of a word, so
47
+ `Galaxy S24` is found in `Galaxy S24 256 GB` again.
48
+ - Thai, Lao, Khmer and Myanmar digits keep a number whole (`1` passed on
49
+ `12` written in Myanmar digits), as do the Arabic decimal point and
50
+ thousands separator (`١٫٥`) and a thin space between thousands in a
51
+ downloaded file.
52
+ - A base URL password holding a raw `@` is masked whole: its tail after
53
+ the `@` reached the recording agent.
54
+ - An untyped `differs from` holds a number however the step writes it: an
55
+ unchanged `10.5` passed against `10.50`, and `100` against `1e2`.
56
+ - A captured list or object keeps each number as the API wrote it. A
57
+ decimal JavaScript cannot hold was sent on as a string; it is now
58
+ refused, as the same number captured alone is.
59
+ - `@serial` holds while a heal proves an Outline's other rows: a row with
60
+ its own `@serial` Examples tag was replayed without its name. Tags are
61
+ read in any case, with commas between names (`@Serial:Rooms` and
62
+ `@serial:a,b` were ignored), and `saffron eject` writes `workers: 1`
63
+ for a suite with `@serial` scenarios, or warns when the project has its
64
+ own Playwright config.
65
+ - "Not shown" counts a number beside a comma as shown: `2` passed while
66
+ `Tags: 1,2,3` was on screen, and `42` beside `1001,42`. It also counts
67
+ `$5.00` as shown in `US$5.00`, and a role name pattern with a row value
68
+ reads `15 128` as one number, as a text target does.
69
+ - `256 GB` is found in `Galaxy S24 256 GB`: a model number heads no
70
+ grouped number.
71
+ - The dashboard counts a scenario whose latest run was not run against
72
+ the pass rate: one scenario passing while the rest were not run read
73
+ 100%, 1 of 1.
74
+ - A `differs from` checked as a string compares a number too large for
75
+ JavaScript by its digits: the same int64 id passed.
76
+ - A base URL password holding a quote or a space is masked before the
77
+ recording agent reads the base URL.
78
+ - The exported helper refuses `$10.50` for a role name recorded as
79
+ `Refund of $10.`, keeps a captured object's numbers as the API wrote
80
+ them, and `saffron eject` gives the trace warning beside the
81
+ `@serial` one, where it gave only the `@serial` one.
82
+ - An Outline's "not shown" check on a row value the recording glued into
83
+ a word reads the word in any form. Recorded on `apples` as `<item>s`,
84
+ row `box` checked that `boxs` was not shown and passed while `boxes`
85
+ was listed; it now counts `boxes` as shown.
86
+ - An exported "not shown" check looks in the app's own frames, as replay
87
+ does: it passed while `Card declined` showed in a same-origin iframe. It
88
+ also waits for the page's requests and for the hidden state to hold, as
89
+ replay does: it passed before an error drawn 300 ms after a click.
90
+ - The dashboard keeps every browser of the last CI run's matrix, where
91
+ each browser's shards are merged on their own: the browser merged last
92
+ hid the others' reds. A red from before browsers were recorded stands
93
+ when a later run is not run in one browser and passes in another.
94
+ - A `differs from` checked as a string fails on the same number, `true`
95
+ or `null`: it passed on any of them.
96
+ - `saffron eject` run again no longer warns about `@serial` when the
97
+ config it wrote already runs one test at a time.
98
+ - `run_plan` (MCP) and `saffron run --plan` name each `@serial` name and
99
+ how many tests share it, and the agent skill's syntax reference lists the
100
+ tags that change how a scenario runs.
101
+
8
102
  ## 1.3.4: Checks that wait for a settled page, and credentials kept masked
9
103
 
10
104
  Released 2026-11-07.