saffron-ai 1.0.1 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,113 @@ the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
5
5
  JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates. Each
6
6
  release heading names its main feature: `## <version>: <title>`.
7
7
 
8
+ ## 1.1.0: Secrets the AI agent never sees
9
+
10
+ Released 2026-10-22.
11
+
12
+ **Upgrading from 1.0.1.** Nothing to change in a project. What can be
13
+ different after `npm update`:
14
+
15
+ - Row value boundaries are stricter: a regex atom that may match nothing
16
+ or anything (`\W*`, `.+`, `[^,]+`), a lookaround, or a group edge or
17
+ alternation next to a row value (`Order <id>(?:st|)`) no longer drops
18
+ the value's boundary, and row values are matched in every selector-chain
19
+ container. A row value at the edge of a role's name, a label, a
20
+ placeholder, an attribute (`alt`, `title`, `[aria-label*=...]`, XPath
21
+ `contains(@href, ...)`), `:text(...)`, `:text-matches(...)`, an inner
22
+ `has` locator or a frame's selector is whole too, as in text: row `1`
23
+ never finds `Order 12`. Beside a digit, a decimal or thousands part
24
+ counts as part of the number (`1` is not in `1.5` or `1,000`) unless the
25
+ pattern writes it (`<n>\.\d{2}`), and a pattern with a quantifier right
26
+ after a row value (`<id>+`) is refused. Text a selector says must be
27
+ absent (`:not(...)`, XPath `not(...)`, `has-not-text`) is no longer
28
+ looked for as present.
29
+ A recording that passed in 1.0.1 because of that can turn red after
30
+ `npm update`; it never turns a failing check green. Re-record what fails
31
+ (`saffron run --rerecord <file>:<line>`).
32
+ - Recordings that use JSON response field checks, or single-quoted row
33
+ values in any part of a selector chain (an XPath part first too), are
34
+ format version 5. Saffron 1.0.1 refuses them by name, so a team updates
35
+ together.
36
+ - An Outline with `As "<who>"` steps recorded before 1.0.1, where a step
37
+ types a value of that person's own, records again once on another
38
+ person's row (red under `--no-agent`, saying why). The new recording
39
+ then refuses the other rows by name, as 1.0.1 recordings of that shape
40
+ do: give each person's values a column, or each person a Scenario.
41
+ - `saffron run` exits 2 when the files it is given hold no scenario (only
42
+ step sets, or nothing yet), where it said "0 passed" and exited 0.
43
+ - Amount checks read a minus spaced off at the start (`- €5.00`) or after
44
+ a currency (`CHF-5.00`) as negative, and a hyphen after another word
45
+ (`ID-123`) as no sign.
46
+
47
+ - Agent-blind secrets: the recording agent no longer sees `{env:...}`
48
+ values. It is given the token and types the token; Saffron's browser
49
+ tools put the value into the page, and every tool result (snapshots,
50
+ echoed code, console, network, `browser_evaluate`) and every Saffron
51
+ reply comes back with the value as its token, as do the prompt's base
52
+ URL and a heal prompt's replay failure and page address. That covers a
53
+ snapshot line Playwright single-quotes, a value Playwright's call log
54
+ cuts with "…", and a downloaded file quoted in a reply. A token counts
55
+ only where the browser types, picks, opens or uploads it (`{env:...}` in
56
+ `browser_evaluate` code is refused), and a word the scenario or a data
57
+ value it uses writes with a short value glued in (`qa{env:PIN}`) is
58
+ shown back as written. The
59
+ recording reads the browser's own results, so a value quoted in
60
+ Playwright's echoed code is masked to its token there too. This holds
61
+ for Claude, Codex, Antigravity and Cursor alike. The guide has a diagram
62
+ of the flow and lists what remains: values under three characters, a
63
+ page that shows the value transformed or only part of it, code that
64
+ reads a field the value was typed into, same-text matches masked along
65
+ with the value, and a check on the value shown in another case.
66
+
67
+ - `saffron eject` writes the committed recordings as Playwright Test specs
68
+ you own from then on: one spec per feature file, each Gherkin step a
69
+ `test.step`, a helper file beside them, and `playwright.config.ts` when
70
+ there is none. An exported test is never looser than replay: it may fail
71
+ where replay would pass, never the other way, and what has no faithful
72
+ form yet (amount checks, patterns or selectors holding a row value, JSON
73
+ response checks) is written as `test.fixme` with the reason. `{env:...}`
74
+ values are read when the test runs and never written into a file. The
75
+ exported checks read an element's text exactly as replay does, so CSS
76
+ `text-transform` or text split over blocks cannot pass there and fail in
77
+ replay. Two feature files that would share a spec file stop the command
78
+ before anything is written.
79
+
80
+ - UI tests can check JSON response fields by path, including nested fields
81
+ and array entries. Equality checks preserve string, number, boolean and
82
+ null types; presence checks distinguish a missing field from null.
83
+ Failures name the field and expected and actual values, with secrets
84
+ masked. These checks replay without AI and Then assertions stay strict.
85
+ Recordings using them require format version 5, so older runners refuse
86
+ them with update guidance instead of skipping the JSON checks. A field's
87
+ path is kept as recorded: a row value that sits in it by chance
88
+ (`/errors/0`) never makes another row check another field.
89
+
90
+ - Match row values in every selector-chain container, including single-quoted
91
+ text. Such a recording is format version 5, since 1.0.1 would fill a
92
+ single-quoted value in without a row boundary.
93
+ - A row value keeps its boundary in a pattern next to an atom that may
94
+ match nothing or anything (`\W*`, `\d*`, `.+`, `\S+`, `[^,]+`, `\P{L}+`),
95
+ a lookaround (`(?!-)`, `(?<!-)`), or a group edge or alternation
96
+ (`(x|<id>)`): `Order <id>.+` fails row "1" on "Order 12 shipped".
97
+ - An XPath selector (`xpath=//button`) replays: it was read as a regex
98
+ with the flags "button" and threw.
99
+ - An Outline with `As "<who>"` steps recorded before 1.0.1 records again
100
+ once on a row of another person, instead of replaying the first person's
101
+ values; the new recording refuses the other rows by name.
102
+ - Amounts: a minus spaced off at the start (`- €5.00`, `- 5,00 €`) or
103
+ glued after a currency (`CHF-5.00`, as Swiss formats write it) is
104
+ negative, and a hyphen after another word (`ID-123`) is no sign.
105
+ - A run of feature files with no scenario in them exits 2 and says so,
106
+ instead of "0 passed". A step set file that does not parse is named in
107
+ the error. Feature edits apply to files written with CRLF and keep their
108
+ line endings. A recording is written beside and renamed over, so a
109
+ stopped run never leaves half of one.
110
+ - Mask case-changed encoded secrets in capture names, reports and traces.
111
+ - Read labelled counts such as `Guests (2 pax)` as positive, also where
112
+ the label is a currency code in lower case (`Pens (2 pen)`). Accounting
113
+ parentheses still denote negative amounts with currency notation.
114
+
8
115
  ## 1.0.1: Amounts, downloads and dates as the app shows them
9
116
 
10
117
  Released 2026-10-21.
package/README.md CHANGED
@@ -154,8 +154,7 @@ The roadmap is at **[saffron-ai.io/docs/roadmap](https://saffron-ai.io/docs/road
154
154
  milestones, statuses and decision gates, and everything shipped so far in
155
155
  its "Where we are" table.
156
156
  Headlines ahead: API testing (reusable REST operations, fixtures and schema
157
- validation), failure video,
158
- agent-blind secrets, LLM-less MCP replay, the Level-2 shared step library
157
+ validation), failure video, LLM-less MCP replay, the Level-2 shared step library
159
158
  (gated on Level-1 divergence data), and the long-term open-core SaaS layer.
160
159
 
161
160
  ## Documentation