saffron-ai 1.0.1 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +107 -0
- package/README.md +1 -2
- package/dist-pkg/cli.js +329 -294
- package/package.json +2 -2
- package/skills/saffron/references/syntax.md +23 -3
- package/templates/playwright/saffron-helpers.ts +652 -0
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,113 @@ the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
|
|
|
5
5
|
JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
|
|
6
6
|
release heading names its main feature: `## <version>: <title>`.
|
|
7
7
|
|
|
8
|
+
## 1.1.0: Secrets the AI agent never sees
|
|
9
|
+
|
|
10
|
+
Released 2026-10-22.
|
|
11
|
+
|
|
12
|
+
**Upgrading from 1.0.1.** Nothing to change in a project. What can be
|
|
13
|
+
different after `npm update`:
|
|
14
|
+
|
|
15
|
+
- Row value boundaries are stricter: a regex atom that may match nothing
|
|
16
|
+
or anything (`\W*`, `.+`, `[^,]+`), a lookaround, or a group edge or
|
|
17
|
+
alternation next to a row value (`Order <id>(?:st|)`) no longer drops
|
|
18
|
+
the value's boundary, and row values are matched in every selector-chain
|
|
19
|
+
container. A row value at the edge of a role's name, a label, a
|
|
20
|
+
placeholder, an attribute (`alt`, `title`, `[aria-label*=...]`, XPath
|
|
21
|
+
`contains(@href, ...)`), `:text(...)`, `:text-matches(...)`, an inner
|
|
22
|
+
`has` locator or a frame's selector is whole too, as in text: row `1`
|
|
23
|
+
never finds `Order 12`. Beside a digit, a decimal or thousands part
|
|
24
|
+
counts as part of the number (`1` is not in `1.5` or `1,000`) unless the
|
|
25
|
+
pattern writes it (`<n>\.\d{2}`), and a pattern with a quantifier right
|
|
26
|
+
after a row value (`<id>+`) is refused. Text a selector says must be
|
|
27
|
+
absent (`:not(...)`, XPath `not(...)`, `has-not-text`) is no longer
|
|
28
|
+
looked for as present.
|
|
29
|
+
A recording that passed in 1.0.1 because of that can turn red after
|
|
30
|
+
`npm update`; it never turns a failing check green. Re-record what fails
|
|
31
|
+
(`saffron run --rerecord <file>:<line>`).
|
|
32
|
+
- Recordings that use JSON response field checks, or single-quoted row
|
|
33
|
+
values in any part of a selector chain (an XPath part first too), are
|
|
34
|
+
format version 5. Saffron 1.0.1 refuses them by name, so a team updates
|
|
35
|
+
together.
|
|
36
|
+
- An Outline with `As "<who>"` steps recorded before 1.0.1, where a step
|
|
37
|
+
types a value of that person's own, records again once on another
|
|
38
|
+
person's row (red under `--no-agent`, saying why). The new recording
|
|
39
|
+
then refuses the other rows by name, as 1.0.1 recordings of that shape
|
|
40
|
+
do: give each person's values a column, or each person a Scenario.
|
|
41
|
+
- `saffron run` exits 2 when the files it is given hold no scenario (only
|
|
42
|
+
step sets, or nothing yet), where it said "0 passed" and exited 0.
|
|
43
|
+
- Amount checks read a minus spaced off at the start (`- €5.00`) or after
|
|
44
|
+
a currency (`CHF-5.00`) as negative, and a hyphen after another word
|
|
45
|
+
(`ID-123`) as no sign.
|
|
46
|
+
|
|
47
|
+
- Agent-blind secrets: the recording agent no longer sees `{env:...}`
|
|
48
|
+
values. It is given the token and types the token; Saffron's browser
|
|
49
|
+
tools put the value into the page, and every tool result (snapshots,
|
|
50
|
+
echoed code, console, network, `browser_evaluate`) and every Saffron
|
|
51
|
+
reply comes back with the value as its token, as do the prompt's base
|
|
52
|
+
URL and a heal prompt's replay failure and page address. That covers a
|
|
53
|
+
snapshot line Playwright single-quotes, a value Playwright's call log
|
|
54
|
+
cuts with "…", and a downloaded file quoted in a reply. A token counts
|
|
55
|
+
only where the browser types, picks, opens or uploads it (`{env:...}` in
|
|
56
|
+
`browser_evaluate` code is refused), and a word the scenario or a data
|
|
57
|
+
value it uses writes with a short value glued in (`qa{env:PIN}`) is
|
|
58
|
+
shown back as written. The
|
|
59
|
+
recording reads the browser's own results, so a value quoted in
|
|
60
|
+
Playwright's echoed code is masked to its token there too. This holds
|
|
61
|
+
for Claude, Codex, Antigravity and Cursor alike. The guide has a diagram
|
|
62
|
+
of the flow and lists what remains: values under three characters, a
|
|
63
|
+
page that shows the value transformed or only part of it, code that
|
|
64
|
+
reads a field the value was typed into, same-text matches masked along
|
|
65
|
+
with the value, and a check on the value shown in another case.
|
|
66
|
+
|
|
67
|
+
- `saffron eject` writes the committed recordings as Playwright Test specs
|
|
68
|
+
you own from then on: one spec per feature file, each Gherkin step a
|
|
69
|
+
`test.step`, a helper file beside them, and `playwright.config.ts` when
|
|
70
|
+
there is none. An exported test is never looser than replay: it may fail
|
|
71
|
+
where replay would pass, never the other way, and what has no faithful
|
|
72
|
+
form yet (amount checks, patterns or selectors holding a row value, JSON
|
|
73
|
+
response checks) is written as `test.fixme` with the reason. `{env:...}`
|
|
74
|
+
values are read when the test runs and never written into a file. The
|
|
75
|
+
exported checks read an element's text exactly as replay does, so CSS
|
|
76
|
+
`text-transform` or text split over blocks cannot pass there and fail in
|
|
77
|
+
replay. Two feature files that would share a spec file stop the command
|
|
78
|
+
before anything is written.
|
|
79
|
+
|
|
80
|
+
- UI tests can check JSON response fields by path, including nested fields
|
|
81
|
+
and array entries. Equality checks preserve string, number, boolean and
|
|
82
|
+
null types; presence checks distinguish a missing field from null.
|
|
83
|
+
Failures name the field and expected and actual values, with secrets
|
|
84
|
+
masked. These checks replay without AI and Then assertions stay strict.
|
|
85
|
+
Recordings using them require format version 5, so older runners refuse
|
|
86
|
+
them with update guidance instead of skipping the JSON checks. A field's
|
|
87
|
+
path is kept as recorded: a row value that sits in it by chance
|
|
88
|
+
(`/errors/0`) never makes another row check another field.
|
|
89
|
+
|
|
90
|
+
- Match row values in every selector-chain container, including single-quoted
|
|
91
|
+
text. Such a recording is format version 5, since 1.0.1 would fill a
|
|
92
|
+
single-quoted value in without a row boundary.
|
|
93
|
+
- A row value keeps its boundary in a pattern next to an atom that may
|
|
94
|
+
match nothing or anything (`\W*`, `\d*`, `.+`, `\S+`, `[^,]+`, `\P{L}+`),
|
|
95
|
+
a lookaround (`(?!-)`, `(?<!-)`), or a group edge or alternation
|
|
96
|
+
(`(x|<id>)`): `Order <id>.+` fails row "1" on "Order 12 shipped".
|
|
97
|
+
- An XPath selector (`xpath=//button`) replays: it was read as a regex
|
|
98
|
+
with the flags "button" and threw.
|
|
99
|
+
- An Outline with `As "<who>"` steps recorded before 1.0.1 records again
|
|
100
|
+
once on a row of another person, instead of replaying the first person's
|
|
101
|
+
values; the new recording refuses the other rows by name.
|
|
102
|
+
- Amounts: a minus spaced off at the start (`- €5.00`, `- 5,00 €`) or
|
|
103
|
+
glued after a currency (`CHF-5.00`, as Swiss formats write it) is
|
|
104
|
+
negative, and a hyphen after another word (`ID-123`) is no sign.
|
|
105
|
+
- A run of feature files with no scenario in them exits 2 and says so,
|
|
106
|
+
instead of "0 passed". A step set file that does not parse is named in
|
|
107
|
+
the error. Feature edits apply to files written with CRLF and keep their
|
|
108
|
+
line endings. A recording is written beside and renamed over, so a
|
|
109
|
+
stopped run never leaves half of one.
|
|
110
|
+
- Mask case-changed encoded secrets in capture names, reports and traces.
|
|
111
|
+
- Read labelled counts such as `Guests (2 pax)` as positive, also where
|
|
112
|
+
the label is a currency code in lower case (`Pens (2 pen)`). Accounting
|
|
113
|
+
parentheses still denote negative amounts with currency notation.
|
|
114
|
+
|
|
8
115
|
## 1.0.1: Amounts, downloads and dates as the app shows them
|
|
9
116
|
|
|
10
117
|
Released 2026-10-21.
|
package/README.md
CHANGED
|
@@ -154,8 +154,7 @@ The roadmap is at **[saffron-ai.io/docs/roadmap](https://saffron-ai.io/docs/road
|
|
|
154
154
|
milestones, statuses and decision gates, and everything shipped so far in
|
|
155
155
|
its "Where we are" table.
|
|
156
156
|
Headlines ahead: API testing (reusable REST operations, fixtures and schema
|
|
157
|
-
validation), failure video,
|
|
158
|
-
agent-blind secrets, LLM-less MCP replay, the Level-2 shared step library
|
|
157
|
+
validation), failure video, LLM-less MCP replay, the Level-2 shared step library
|
|
159
158
|
(gated on Level-1 divergence data), and the long-term open-core SaaS layer.
|
|
160
159
|
|
|
161
160
|
## Documentation
|