saffron-ai 1.3.5 → 1.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -4,19 +4,19 @@
4
4
 
5
5
  Free to use · [npm: saffron-ai](https://www.npmjs.com/package/saffron-ai) · [site](https://saffron-ai.io)
6
6
 
7
- Write tests in plain language, as `.saffron` files or standard `.feature` files. `.saffron` is Saffron's own test language and goes further than Gherkin: step sets for reusable flows, `{env:VAR}` tokens that keep secrets out of recordings and reports, and `{data:...}` references to data files. There are no step definitions and no glue code. On the first run, an AI agent (Claude by default, or Codex, Antigravity or Cursor on the subscription you already have, driving Playwright MCP) executes each scenario in a real browser and records what it did into a **cache**: a human-reviewable JSON action list committed to git. Every run after that, Saffron replays the cache in a real browser: **zero AI calls, zero tokens**, at the speed of the browser and your app. The cache is Saffron's own JSON, not a generated script; replay runs on Playwright's browser engine, and Playwright MCP is only how the agent operates the browser while it records or heals.
7
+ Write tests in plain language, as `.saffron` files or standard `.feature` files. `.saffron` is Saffron's own test language and goes further than Gherkin: step sets for reusable flows, `{env:VAR}` tokens that keep secrets out of recordings and reports, and `{data:...}` references to data files. There are no step definitions and no glue code. On the first run, an AI agent (Claude by default, or Codex, with Antigravity (experimental) and Cursor (not yet verified live), on the subscription you already have, driving Playwright MCP) executes each scenario in a real browser and records what it did into a **cache**: a human-reviewable JSON action list committed to git. Every run after that, Saffron replays the cache in a real browser: **zero AI calls, zero tokens**, at the speed of the browser and your app. The cache is Saffron's own JSON, not a generated script; replay runs on Playwright's browser engine, and Playwright MCP is only how the agent operates the browser while it records or heals.
8
8
 
9
9
  When the UI changes and a cached step fails at runtime, the agent takes over mid-execution with full scenario context, adapts, and finishes the run, then files a **cache proposal** you accept or reject, like a snapshot test update. Reports state exactly what was adapted and what it cost.
10
10
 
11
11
  ## The rules that keep it honest
12
12
 
13
- - **Assertions are sacred.** By default no `Then` is ever healed: the AI may help *reach* an assertion, never make it pass; enforced mechanically, not by prompt. Projects may opt into `adaptable-mid` for mid-scenario checkpoints, but the **final assertion block** of a scenario is strict under every policy, forever.
13
+ - **Assertions are sacred.** By default no `Then` is ever healed: the AI may help *reach* an assertion, never make it pass; enforced mechanically, not by prompt. Projects may opt into `adaptable-mid` for mid-scenario checkpoints, but the **final assertion block** of a scenario is strict under every policy, forever. The one repair: a database check whose query breaks on a schema change may have its table, columns and joins fixed, never its values, expected rows or row count, and the repair is held for your review.
14
14
  - **Three result states.** Green = cached pass. Yellow = passed with AI adaptation (pending your review). Red = failed.
15
15
  - **Diagnose before adapting.** On a step failure the agent first decides: UI drift (heal) or application defect (fail with a diagnosis).
16
16
  - **Caches are git artifacts.** Proposals show diffs, including a suggested feature-file edit when the written steps no longer match reality, and nothing is committed without you.
17
17
  - **Verified proposals.** Every recording/heal is proof-replayed zero-AI before it's filed (with one bounded refinement pass on failure), so proposals arrive stamped `verified ✓` or honestly `UNVERIFIED ✗`.
18
18
  - **Honest cost reporting.** Reports break out prompt-cache reads/writes (the real bill of agent sessions) next to the in+out token count, per scenario and in totals, and name the models used. On an API key the dollars are what was billed; on a Claude subscription the headline is the 5-hour plan window before and after the run, with the dollars as the API-equivalent.
19
- - **Your subscription, your choice of agent.** Recording and healing run on the plan already logged in on the machine: Claude (default), Codex on a ChatGPT plan, Google AI Pro/Ultra through the Antigravity CLI (experimental), or Cursor. Set `"provider"` in the config; each one runs through its vendor's official client, and every provider records through the same recorder, so the caches are identical in kind.
19
+ - **Your subscription, your choice of agent.** Recording and healing run on the plan already logged in on the machine: Claude (default), Codex on a ChatGPT plan, Google AI Pro/Ultra through the Antigravity CLI (experimental), or Cursor (not yet verified live). Set `"provider"` in the config; each one runs through its vendor's official client, and every provider records through the same recorder, so the caches are identical in kind.
20
20
  - **Trend memory.** Every run appends to `.saffron/history.jsonl`; reports show deltas vs the previous run and 20-run sparklines, and Saffron flags **chronic scenarios** (healing repeatedly: re-record instead of paying again) plus recurring failure themes.
21
21
 
22
22
  ## Step reuse: new scenarios get cheaper as your suite grows
@@ -42,6 +42,23 @@ credentials an API hands out stay out of recordings, reports and traces.
42
42
  See "API tests" in the [guide](https://saffron-ai.io/docs). API test
43
43
  automation is in beta: it works, and it is still being shaped by use.
44
44
 
45
+ ## Database steps (beta)
46
+
47
+ A step can check what the database holds ("Then the customer should be
48
+ saved in the database") or seed the data a scenario starts from, with no
49
+ query written by hand: while recording, the agent reads the schema and
50
+ records one parameterized statement, and every later run replays it at
51
+ zero tokens, in browser and API scenarios alike. Checks are bound to the
52
+ scenario's own values and read in transactions that are rolled back; writes are off
53
+ by default, go only to an allowlisted database, and are undone when the
54
+ scenario ends. PostgreSQL, MySQL, SQL Server, Oracle, SQLite, MariaDB or
55
+ CockroachDB (`db` in `saffron.config.json`, `npm install node-sql-parser`
56
+ with `pg`, `mysql2`, `mssql` or `oracledb@6`), or MongoDB, DynamoDB or
57
+ Firestore (their driver alone, `@google-cloud/firestore@7` for
58
+ Firestore), whose steps hold one JSON command instead
59
+ of SQL. See "Database steps" in the
60
+ [guide](https://saffron-ai.io/docs).
61
+
45
62
  ## Install
46
63
 
47
64
  ```bash
@@ -92,7 +109,7 @@ node dist/cli/index.js -p examples accept --all
92
109
  | `saffron export [-o file] [--branch name]` / `saffron runs` / `saffron import <bundles...> \| --run <id>` | Carry a CI heal to your machine: export packs the run's proposals, report, screenshots and traces into one bundle for CI to upload; `runs` lists the branch's CI runs (GitHub Actions or Azure Pipelines); import (of a file, or of a run with `--run`) checks the bundle against your checkout (repository, each proposal's binding) and files it there, ready for `saffron diff` and `saffron accept`. `--branch` also has CI commit the proposals to a new branch and open a draft pull request with their diffs. |
93
110
  | `saffron status [--json]` | Project overview for people, IDE panels and dashboards: feature files and scenarios with cache state, tags, pending proposals, last run, history, vocabulary health (divergent steps, duplicate wordings proven by identical recordings), effective config, and suite quality from the run history (failing, flaky, never run, slowest, trends, per tag). |
94
111
  | `saffron steps [search] [--json] [--snippets]` | List/search the step vocabulary (files + caches) with recorded/divergent/unrecorded badges and usage; `--snippets` writes `.vscode/saffron.code-snippets` for native VS Code completion. |
95
- | `saffron mcp` | Serve the vocabulary, the project overview and the pinned API contract to AI assistants over stdio MCP (`search_steps`, `list_step_sets`, `project_status`, `api_contract`), e.g. `claude mcp add saffron -- npx saffron mcp`. |
112
+ | `saffron mcp` | Serve the vocabulary, the project overview and the pinned API contract to AI assistants over stdio MCP (`search_steps`, `list_step_sets`, `project_status`, `api_contract`, `run_plan`, `verify`), e.g. `claude mcp add saffron -- npx -y -p saffron-ai saffron mcp`. |
96
113
  | `saffron api pin <url or file> [--out file] [--service name]` | Pin an API's OpenAPI 3 document in the project (`api/openapi.json` by default) for `@api` scenarios' schema checks: from a Swagger UI page, the document's URL, or a file. Then set `"api": { "contract": "api/openapi.json" }` in `saffron.config.json`. `--service orders` pins a second API's contract, named under `api.services`. |
97
114
  | `saffron author <prose-file> [-o out]` | Draft a `.saffron` feature file from plain-paragraph requirements, reusing the project's step vocabulary (AI; reports how many lines are seedable). |
98
115
  | `saffron lsp` | Run the Saffron language server over stdio for JetBrains (LSP4IJ/Ultimate), Neovim, any LSP editor: badge completion, StepSet go-to-definition, hover, diagnostics. Highlighting for JetBrains comes from the bundled TextMate grammar (`textmate/saffron` in the package); run `npx saffron lsp` in a terminal for setup steps. |
@@ -167,7 +184,7 @@ features/ your .saffron and .feature files
167
184
  The roadmap is at **[saffron-ai.io/docs/roadmap](https://saffron-ai.io/docs/roadmap)**:
168
185
  milestones, statuses and decision gates, and everything shipped so far in
169
186
  its "Where we are" table.
170
- Headlines ahead: failure video, LLM-less MCP replay, the Level-2 shared step
187
+ Headlines ahead: LLM-less MCP replay, the Level-2 shared step
171
188
  library (gated on Level-1 divergence data), and the long-term open-core SaaS layer.
172
189
 
173
190
  ## Documentation
@@ -176,7 +193,6 @@ library (gated on Level-1 divergence data), and the long-term open-core SaaS lay
176
193
  - [Documentation](https://saffron-ai.io/docs): the cache lifecycle, step sets & the `.saffron` dialect, the step vocabulary, and the guarantees, with diagrams
177
194
  - [Changelog](CHANGELOG.md), also at https://saffron-ai.io/docs/changelog
178
195
  - [Issues & questions](https://github.com/s-chathuranga-j/saffron-community/issues)
179
- - **Docs site**: `npm run docs` builds a self-contained page at
180
196
  ## License
181
197
 
182
198
  Free to use, including commercially, under the [Saffron Free Use License](https://saffron-ai.io/docs/license) (the [LICENSE](LICENSE) file in this package). The runner's source is closed.