saffron-ai 0.5.6 → 0.5.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +24 -0
- package/README.md +7 -6
- package/dist-pkg/cli.js +97 -97
- package/package.json +4 -3
- package/skills/saffron/SKILL.md +31 -6
- package/skills/saffron/references/config.md +18 -5
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,30 @@ All notable changes to Saffron, in one place: the `saffron-ai` runner (npm),
|
|
|
4
4
|
the VS Code extension (`ChathurangaJayasinghe.saffron-vscode`) and the
|
|
5
5
|
JetBrains plugin (`ai.saffron.jetbrains`). Dates are publish dates.
|
|
6
6
|
|
|
7
|
+
## 0.5.7
|
|
8
|
+
|
|
9
|
+
Released 2026-09-17.
|
|
10
|
+
|
|
11
|
+
Runner
|
|
12
|
+
|
|
13
|
+
- `saffron mcp` gains a `project_status` tool (the `saffron status --json`
|
|
14
|
+
overview) next to `search_steps` and `list_step_sets`, and reports the
|
|
15
|
+
real package version.
|
|
16
|
+
|
|
17
|
+
Agents
|
|
18
|
+
|
|
19
|
+
- **Claude Code plugin**: `claude-plugin/` packages the language server,
|
|
20
|
+
the MCP server and the skill as one plugin (`/plugin marketplace add
|
|
21
|
+
s-chathuranga-j/saffron-claude-plugin`, `/plugin install
|
|
22
|
+
saffron@saffron`). `saffron lsp` now adopts the workspace root the
|
|
23
|
+
editor sends in `initialize`, so plugin-launched servers index the
|
|
24
|
+
right project.
|
|
25
|
+
- The bundled skill covers `saffron status`, accepting or rejecting chosen
|
|
26
|
+
proposals, subscription versus API-key cost reporting, Scenario Outline
|
|
27
|
+
recording, and the IDE integrations; the `AGENTS.md` managed block
|
|
28
|
+
points at `saffron status`. Run `npx saffron init` again to refresh both
|
|
29
|
+
in a project.
|
|
30
|
+
|
|
7
31
|
## 0.5.6
|
|
8
32
|
|
|
9
33
|
Released 2026-09-17.
|
package/README.md
CHANGED
|
@@ -15,7 +15,7 @@ When the UI changes and a cached step fails at runtime, the agent takes over mid
|
|
|
15
15
|
- **Diagnose before adapting.** On a step failure the agent first decides: UI drift (heal) or application defect (fail with a diagnosis).
|
|
16
16
|
- **Caches are git artifacts.** Proposals show diffs, including a suggested `.feature` edit when the written steps no longer match reality, and nothing is committed without you.
|
|
17
17
|
- **Verified proposals.** Every recording/heal is proof-replayed zero-AI before it's filed (with one bounded refinement pass on failure), so proposals arrive stamped `verified ✓` or honestly `UNVERIFIED ✗`.
|
|
18
|
-
- **Honest cost reporting.** Reports break out prompt-cache reads/writes (the real bill of agent sessions) next to the in+out token count, per scenario and in totals.
|
|
18
|
+
- **Honest cost reporting.** Reports break out prompt-cache reads/writes (the real bill of agent sessions) next to the in+out token count, per scenario and in totals, and name the models used. On an API key the dollars are what was billed; on a Claude subscription the headline is the 5-hour plan window before and after the run, with the dollars as the API-equivalent.
|
|
19
19
|
- **Trend memory.** Every run appends to `.saffron/history.jsonl`; reports show deltas vs the previous run and 20-run sparklines, and Saffron flags **chronic scenarios** (healing repeatedly. Re-record instead of paying again) plus recurring failure themes.
|
|
20
20
|
|
|
21
21
|
## Step reuse: new scenarios get cheaper as your suite grows
|
|
@@ -24,7 +24,7 @@ Like a maturing Cucumber suite, most of a new feature file is steps you already
|
|
|
24
24
|
|
|
25
25
|
**Step sets** (shipped): `.saffron` files, a superset dialect of Gherkin, add the `StepSet:` keyword for named, reusable step sequences invoked with `StepSet <name>` inside any scenario. Sets expand at parse time, so their steps cache and seed like ordinary steps; editing a set makes every invoking scenario honestly stale (re-recorded mostly seeded), and heal edits route to the set definition, one fix, every caller follows. Sets are **project-wide**: keep application-wide flows in a sets-only library file (convention: `features/shared.steps.saffron`) and invoke them from any feature.
|
|
26
26
|
|
|
27
|
-
**IDE integration & authoring** (shipped): `saffron steps` lists the project vocabulary with recorded/divergent/unrecorded badges (`--snippets` for native VS Code completion), `saffron mcp` serves it to AI assistants, `saffron lsp` brings completion/navigation/diagnostics to **JetBrains** (incl. Community editions via LSP4IJ) and Neovim, `saffron author` drafts feature files from prose in your own vocabulary, and duplicate wordings that record identical actions get **behavior-proven rename proposals**. The **Saffron VS Code extension** is on the [Marketplace](https://marketplace.visualstudio.com/items?itemName=ChathurangaJayasinghe.saffron-vscode) (`code --install-extension ChathurangaJayasinghe.saffron-vscode`). JetBrains users get the same in one click from the **Saffron JetBrains plugin** ([source](https://github.com/s-chathuranga-j/saffron-jetbrains-plugin): `.saffron` file type, bundled grammar, LSP4IJ wiring
|
|
27
|
+
**IDE integration & authoring** (shipped): `saffron steps` lists the project vocabulary with recorded/divergent/unrecorded badges (`--snippets` for native VS Code completion), `saffron mcp` serves it to AI assistants, `saffron lsp` brings completion/navigation/diagnostics to **JetBrains** (incl. Community editions via LSP4IJ) and Neovim, `saffron author` drafts feature files from prose in your own vocabulary, and duplicate wordings that record identical actions get **behavior-proven rename proposals**. The **Saffron VS Code extension** is on the [Marketplace](https://marketplace.visualstudio.com/items?itemName=ChathurangaJayasinghe.saffron-vscode) (`code --install-extension ChathurangaJayasinghe.saffron-vscode`). JetBrains users get the same in one click from the **Saffron JetBrains plugin** ([Marketplace](https://plugins.jetbrains.com/plugin/34240-saffron), [source](https://github.com/s-chathuranga-j/saffron-jetbrains-plugin)): `.saffron` file type, bundled grammar, LSP4IJ wiring. Both editors also **run from the IDE**: right-click Run on `.saffron` files and folders, Saffron run configurations (JetBrains), and a Saffron panel with feature files, tags to run, proposals to accept or reject, vocabulary health, config, and the run report as an in-editor dashboard, all fed by `saffron status --json`.
|
|
28
28
|
|
|
29
29
|
Data tables and doc strings are first-class: 2-column key/value tables parameterize the recording (`<table:username>`), multi-row record tables parameterize per cell (`<table:1:firstName>`), and `"""` doc strings record as `<docstring>`, so editing *values* or *content* replays at zero tokens, while structural changes (keys, headers, row counts) honestly re-record. Unambiguous params are scenario-wide, so a later assertion on a note's text follows content edits too. **Secrets** stay out of everything: `{env:VAR}` resolves from the environment (or a git-ignored `.env`) at replay, recordings and reports are masked back to the token, and missing variables fail fast by name.
|
|
30
30
|
|
|
@@ -43,7 +43,7 @@ of short `.saffron` scenarios against the public demo shop at saucedemo.com
|
|
|
43
43
|
the middle of scenarios) so the first run records and replays something
|
|
44
44
|
real.
|
|
45
45
|
|
|
46
|
-
**Built for the AI era:** the package ships an [Agent Skill](https://agentskills.io) that teaches Claude Code, Codex, Cursor, Copilot and any skill-aware agent to write Saffron tests efficiently. Reuse recorded wordings, keep assertions sacred, secrets as `{env:VAR}`, so the tests your agents write replay at zero tokens instead of paying for near-duplicate recordings.
|
|
46
|
+
**Built for the AI era:** Claude Code users install everything at once from the [Saffron plugin](https://github.com/s-chathuranga-j/saffron-claude-plugin) (`/plugin marketplace add s-chathuranga-j/saffron-claude-plugin`, then `/plugin install saffron@saffron`): the language server for diagnostics and navigation while Claude edits `.saffron` files, the MCP server, and the skill. For every other agent, the package ships an [Agent Skill](https://agentskills.io) that teaches Claude Code, Codex, Cursor, Copilot and any skill-aware agent to write Saffron tests efficiently. Reuse recorded wordings, keep assertions sacred, secrets as `{env:VAR}`, so the tests your agents write replay at zero tokens instead of paying for near-duplicate recordings.
|
|
47
47
|
|
|
48
48
|
CI tip: `saffron run --strict` treats yellow (passed-with-adaptation) as a
|
|
49
49
|
failure until its cache proposal is reviewed: cached-green-only builds.
|
|
@@ -73,10 +73,11 @@ node dist/cli/index.js -p examples accept --all
|
|
|
73
73
|
| Command | What it does |
|
|
74
74
|
|---|---|
|
|
75
75
|
| `saffron run [paths] [--headed] [--filter @tags] [--rerecord] [--no-agent] [--strict] [--browser b] [--workers n] [--heal-model m] [--no-verify] [--no-reuse] [--model m] [--storage-state f]` | Run features. Cached replays are deterministic; misses/failures escalate to the agent (unless `--no-agent`). Replay cross-browser with `--browser firefox\|webkit`, parallelize with `--workers N`, heal on a cheaper model with `--heal-model`. Exit 1 on red (and on yellow with `--strict`). |
|
|
76
|
-
| `saffron accept [
|
|
77
|
-
| `saffron reject [
|
|
76
|
+
| `saffron accept [files... \| --all] [--include-unverified] [--with-feature-edit] [--propagate]` | Promote cache proposals to committed caches (`--all` skips UNVERIFIED ones unless `--include-unverified`); `--with-feature-edit` also rewrites the adapted steps in the `.feature` file (and keeps the cache in sync); `--propagate` applies the heal's locator fixes to every other cache using the same locator. One heal repairs N scenarios before they ever fail. No args: list pending proposals. |
|
|
77
|
+
| `saffron reject [files... \| --all]` | Discard proposals; the agent will try again next run. |
|
|
78
|
+
| `saffron status [--json]` | Project overview for people, IDE panels and dashboards: feature files and scenarios with cache state, tags, pending proposals, last run, history, vocabulary health (divergent steps, duplicate wordings proven by identical recordings), effective config. |
|
|
78
79
|
| `saffron steps [search] [--json] [--snippets]` | List/search the step vocabulary (files + caches) with recorded/divergent/unrecorded badges and usage; `--snippets` writes `.vscode/saffron.code-snippets` for native VS Code completion. |
|
|
79
|
-
| `saffron mcp` | Serve the vocabulary to AI assistants over stdio MCP (`search_steps`, `list_step_sets`), e.g. `claude mcp add saffron -- npx saffron mcp`. |
|
|
80
|
+
| `saffron mcp` | Serve the vocabulary and the project overview to AI assistants over stdio MCP (`search_steps`, `list_step_sets`, `project_status`), e.g. `claude mcp add saffron -- npx saffron mcp`. |
|
|
80
81
|
| `saffron author <prose-file> [-o out]` | Draft a `.saffron` feature file from plain-paragraph requirements, reusing the project's step vocabulary (AI; reports how many lines are seedable). |
|
|
81
82
|
| `saffron lsp` | Run the Saffron language server over stdio for JetBrains (LSP4IJ/Ultimate), Neovim, any LSP editor: badge completion, StepSet go-to-definition, hover, diagnostics. Highlighting for JetBrains comes from the bundled TextMate grammar (`textmate/saffron` in the package); run `npx saffron lsp` in a terminal for setup steps. |
|
|
82
83
|
| `saffron init [--agents list] [--examples]` | Make the project agent-ready (`--examples` installs the Saucedemo example suite): installs the bundled **Saffron agent skill** into `.claude/skills` and `.agents/skills` (Copilot/Cursor via `--agents`), registers the `saffron mcp` server in `.mcp.json` / `.cursor/mcp.json` / `.vscode/mcp.json`, adds a managed block to `AGENTS.md`/`CLAUDE.md`, scaffolds config. |
|
package/dist-pkg/cli.js
CHANGED
|
@@ -1,40 +1,45 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
|
-
import D from"node:fs";import
|
|
3
|
-
`),n=new Map,r=new Map;return{source:t.map((o,i)=>{let a=o.match(sr);if(a)return n.set(i+1,a[2]),`${a[1]}Scenario: ${a[2]}`;let
|
|
4
|
-
`),definitions:n,invocations:r}}function ir(e){return new nr(new er(rr.uuid()),new tr).parse(e)}function
|
|
5
|
-
`)}function
|
|
6
|
-
`}import{Server as
|
|
2
|
+
import D from"node:fs";import E from"node:path";import{spawn as Os}from"node:child_process";import Ds from"node:readline";import{Command as Ms}from"commander";import cr from"node:fs";import lr from"node:path";import{AstBuilder as dr,GherkinClassicTokenMatcher as pr,Parser as ur}from"@cucumber/gherkin";import{IdGenerator as fr,StepKeywordType as Ge}from"@cucumber/messages";function ne(e,t){return e.replace(/<([^<>]+)>/g,(n,r)=>r in t?t[r]:n)}function U(e){if(!e||e.length===0)return{};let t={};if(e.every(n=>n.length===2)){for(let[n,r]of e)t[`table:${n}`]=r;return t}if(e.length>=2){let[n,...r]=e;if(n.some(s=>!s))return{};r.forEach((s,o)=>{n.forEach((i,a)=>{s[a]!==void 0&&(t[`table:${o+1}:${i}`]=s[a])})})}return t}function re(e){return e!==void 0?{docstring:e}:{}}function le(e){let t=new Map;for(let r of e){let s={...U(r.resolvedTable??r.table),...re(r.resolvedDocString??r.docString)};for(let[o,i]of Object.entries(s)){let a=t.get(o);a||t.set(o,a=new Set),a.add(i)}}let n={};for(let[r,s]of t)s.size===1&&(n[r]=[...s][0]);return n}import He from"node:fs";import bt from"node:path";import{AstBuilder as er,GherkinClassicTokenMatcher as tr,Parser as nr}from"@cucumber/gherkin";import{IdGenerator as rr}from"@cucumber/messages";var N=class extends Error{},sr=/^(\s*)StepSet:\s*(.+?)\s*$/,or=/^(\s*)StepSet\s+([^\s:].*?)\s*$/;function We(e){let t=e.split(`
|
|
3
|
+
`),n=new Map,r=new Map;return{source:t.map((o,i)=>{let a=o.match(sr);if(a)return n.set(i+1,a[2]),`${a[1]}Scenario: ${a[2]}`;let c=o.match(or);return c?(r.set(i+1,c[2]),`${c[1]}* StepSet ${c[2]}`):o}).join(`
|
|
4
|
+
`),definitions:n,invocations:r}}function ir(e){return new nr(new er(rr.uuid()),new tr).parse(e)}function vt(e,t){for(let n of e)n.scenario?t(n.scenario):"rule"in n&&n.rule&&vt(n.rule.children,t)}function Z(e,t){let n=new Map;for(let r of e){let s=bt.relative(t,r),o=We(He.readFileSync(r,"utf8"));if(o.definitions.size===0)continue;let i=ir(o.source);i.feature&&vt(i.feature.children,a=>{let c=o.definitions.get(a.location.line);if(c===void 0)return;let p=n.get(c);if(p)throw new N(`StepSet "${c}" is defined twice: ${p.file}:${p.line} and ${s}:${a.location.line}. Set names are unique across the project. If line ${a.location.line} was meant to INVOKE the set inside a scenario, remove the colon: "StepSet ${c}".`);if(a.examples.length>0)throw new N(`StepSet "${c}" (${s}:${a.location.line}) has an Examples table \u2014 step sets take their values from the invoking scenario.`);for(let h of a.steps)if(o.invocations.has(h.location.line))throw new N(`StepSet "${c}" (${s}:${a.location.line}) invokes another step set at line ${h.location.line} \u2014 step sets cannot be nested.`);n.set(c,{name:c,file:s,line:a.location.line,steps:[...a.steps]})})}return n}var ar=[".feature",".saffron"];function se(e){return ar.some(t=>e.endsWith(t))}function de(e){if(!He.existsSync(e))return[];let t=[];for(let n of He.readdirSync(e,{recursive:!0})){let r=String(n);r.endsWith(".saffron")&&t.push(bt.join(e,r))}return t.sort()}function wt(e,t,n){return e.map(({step:r,source:s})=>{let o;switch(r.keywordType){case Ge.OUTCOME:o="assertion";break;case Ge.CONTEXT:case Ge.ACTION:o="action";break;default:o=n.value}n.value=o;let i=r.dataTable?r.dataTable.rows.map(c=>c.cells.map(p=>p.value)):void 0,a=r.docString?.content;return{keyword:r.keyword.trim(),kind:o,text:r.text,resolvedText:ne(r.text,t),table:i,resolvedTable:i?.map(c=>c.map(p=>ne(p,t))),docString:a,resolvedDocString:a!==void 0?ne(a,t):void 0,source:s}})}function kt(e,t,n,r,s){let o=[];for(let i of e){let a=r.get(i.location.line);if(a===void 0){o.push({step:i,source:{file:t,line:i.location.line,fromBackground:n}});continue}let c=s?.get(a);if(!c)throw new N(`${t}:${i.location.line} invokes StepSet "${a}", which is not defined anywhere in the project.`);for(let p of c.steps)o.push({step:p,source:{file:c.file,line:p.location.line,fromStepSet:c.name,fromBackground:n}})}return o}function gr(e,t,n,r,s){let o=e.tags.map(a=>a.name),i=[];if(e.examples.length===0)i.push({params:{},label:""});else{let a=0;for(let c of e.examples){let p=c.tableHeader?.cells.map(h=>h.value)??[];for(let h of c.tableBody){a+=1;let d={};h.cells.forEach((l,m)=>{d[p[m]]=l.value}),i.push({params:d,label:` (example ${a})`})}}}return i.map(({params:a,label:c})=>{let p={value:"action"};return{featurePath:t,featureName:n,name:e.name,displayName:`${e.name}${c}`,tags:o,steps:[...wt(r,a,p),...wt(s,a,p)],params:a}})}function ee(e,t,n){let r=cr.readFileSync(e,"utf8"),o=e.endsWith(".saffron")?We(r):{source:r,definitions:new Map,invocations:new Map},a=new ur(new dr(fr.uuid()),new pr).parse(o.source);if(!a.feature)return[];let c=lr.relative(t,e),p=a.feature.name,h=[],d=(l,m)=>{let g=m;for(let u of l)if(u.background)g=[...m,...kt(u.background.steps,c,!0,o.invocations,n)];else if(u.scenario){if(o.definitions.has(u.scenario.location.line))continue;h.push(...gr(u.scenario,c,p,g,kt(u.scenario.steps,c,!1,o.invocations,n)))}else"rule"in u&&u.rule&&d(u.rule.children,g)};return d(a.feature.children,[]),h}import Ct from"node:fs";import Je from"node:path";import Rt from"node:fs";import Tt from"node:path";import Re from"node:fs";import he from"node:path";var $t=".saffron";function Te(e){return e.toLowerCase().replace(/[^a-z0-9]+/g,"-").replace(/^-+|-+$/g,"")||"unnamed"}function B(e,t,n){let r=Te(he.basename(t,".feature"));return he.join(e,$t,"cache",r,`${Te(n)}.json`)}function ye(e,t,n){let r=Te(he.basename(t,".feature"));return he.join(e,$t,"proposals",r,`${Te(n)}.json`)}function z(e){if(!Re.existsSync(e))return;let t=JSON.parse(Re.readFileSync(e,"utf8"));if(t.version!==1)throw new Error(`Unsupported cache version ${t.version} in ${e}`);return t}function pe(e,t){Re.mkdirSync(he.dirname(e),{recursive:!0}),Re.writeFileSync(e,`${JSON.stringify(t,null,2)}
|
|
5
|
+
`)}function Ee(e){let t=[];return{pattern:e.replace(/"((?:[^"\\]|\\.)*)"/g,(r,s)=>(t.push(s),`"\xAB${t.length-1}\xBB"`)),args:t}}function mr(e,t){let n=r=>JSON.stringify(r.map(({pageFingerprint:s,...o})=>o));return n(e)===n(t)}function hr(e){let t=new Set,n=r=>{if(r)for(let s of r.matchAll(/<([^<>]+)>/g))t.add(s[1])};for(let r of e.actions)n(r.value),n(r.url),n(r.pattern),n(r.target?.name),n(r.target?.nameRegex),n(r.target?.label),n(r.target?.text),n(r.target?.placeholder);return n(e.gherkin),t}function W(e){let t=Tt.join(e,".saffron","cache"),n=new Map,r=new Map,s=0;if(Rt.existsSync(t))for(let a of Rt.readdirSync(t,{recursive:!0})){let c=String(a);if(!c.endsWith(".json"))continue;let p=z(Tt.join(t,c));if(p){s++;for(let h of p.steps){let d=h.gherkin,l=JSON.stringify(h.actions.map(({pageFingerprint:u,...y})=>y)),m=r.get(d)??[];m.includes(l)||(m.push(l),r.set(d,m));let g=n.get(d);!g||p.recordedAt>g.recordedAt?n.set(d,{gherkin:h.gherkin,table:h.table,docString:h.docString,actions:h.actions,feature:p.feature,scenario:p.scenario,recordedAt:p.recordedAt,variants:m.length}):g.variants=m.length}}}let o=[...r.entries()].filter(([,a])=>a.length>1).map(([a])=>a),i=new Map;for(let a of n.values()){let{pattern:c,args:p}=Ee(a.gherkin);if(p.length===0)continue;let h=i.get(c);(!h||a.recordedAt>h.recordedAt)&&i.set(c,{...a,args:p})}return{steps:n,patterns:i,divergent:o,scannedCaches:s}}function Et(e,t,n){if(!!e.table!=!!t.table)return!1;if(e.table&&t.table){let o=Object.keys(U(e.table)),i=Object.keys(U(t.table));if(o.length>0&&i.length>0){if(JSON.stringify(o)!==JSON.stringify(i))return!1}else if(JSON.stringify(e.table)!==JSON.stringify(t.table))return!1}if(e.docString!==void 0!=(t.docString!==void 0)||e.docString!==void 0&&t.docString!==void 0&&e.docString!==t.docString&&!e.actions.some(o=>[o.value,o.url,o.pattern].some(i=>i?.includes("<docstring>"))))return!1;let r={...n,...U(t.resolvedTable??t.table),...re(t.resolvedDocString??t.docString)},s=hr({gherkin:e.gherkin,keyword:"",kind:"action",actions:e.actions});for(let o of s)if(!(o in r))return!1;return!0}function At(e,t,n){let r=e.steps.get(t.text);if(r)return Et(r,t,n)?r:void 0;let{pattern:s,args:o}=Ee(t.text);if(o.length===0)return;let i=e.patterns.get(s);if(!i)return;let a=new Map;for(let d=0;d<o.length;d++){let l=i.args[d],m=o[d];if(l===m)continue;let g=a.get(l);if(g!==void 0&&g!==m)return;a.set(l,m)}let c=d=>[d?.name,d?.nameRegex,d?.label,d?.text,d?.placeholder];for(let d of a.keys())if(!i.actions.some(m=>[m.value,m.url,m.pattern,...c(m.target)].includes(d)))return;let p=d=>d!==void 0&&a.has(d)?a.get(d):d,h={...i,gherkin:t.text,actions:i.actions.map(({pageFingerprint:d,...l})=>({...l,value:p(l.value),url:p(l.url),pattern:p(l.pattern),target:l.target?{...l.target,name:p(l.target.name),nameRegex:p(l.target.nameRegex),label:p(l.target.label),text:p(l.target.text),placeholder:p(l.target.placeholder)}:void 0}))};return Et(h,t,n)?h:void 0}function Pt(e,t){let n=[];return e.forEach((r,s)=>{if(r.actions.length!==0&&!t.steps.has(r.gherkin)){for(let o of t.steps.values())if(!(o.variants>1)&&o.actions.length!==0&&mr(r.actions,o.actions)){n.push({index:s,from:r.gherkin,to:o.gherkin,feature:o.feature,scenario:o.scenario});break}}}),n}function yr(e){if(!Ct.existsSync(e))return[];let t=[];for(let n of Ct.readdirSync(e,{recursive:!0})){let r=String(n);se(r)&&t.push(Je.join(e,r))}return t.sort()}function q(e,t,n){let r=Je.isAbsolute(t)?t:Je.join(e,t),s=yr(r),o=n??Z(s.filter(l=>l.endsWith(".saffron")),e),i=new Map,a=new Map;for(let l of s)for(let m of ee(l,e,o)){let g=`${m.featurePath}::${m.name}`;for(let u of m.steps){let y=i.get(u.text);if(y||(y={text:u.text,keywords:new Map,kind:u.kind,scenarios:new Set,files:new Set,fromStepSet:u.source?.fromStepSet},i.set(u.text,y)),y.keywords.set(u.keyword,(y.keywords.get(u.keyword)??0)+1),y.scenarios.add(g),u.source&&y.files.add(u.source.file),u.source?.fromStepSet){let $=a.get(u.source.fromStepSet);$||a.set(u.source.fromStepSet,$=new Set),$.add(g)}}}let c=W(e),p=new Set(c.divergent),h=[...i.values()].map(l=>({text:l.text,keyword:[...l.keywords.entries()].sort((m,g)=>g[1]-m[1])[0][0],kind:l.kind,status:p.has(l.text)?"divergent":c.steps.has(l.text)?"recorded":"unrecorded",usage:l.scenarios.size,files:[...l.files].sort(),fromStepSet:l.fromStepSet})).sort((l,m)=>m.usage-l.usage||l.text.localeCompare(m.text)),d=[...o.values()].map(l=>({name:l.name,file:l.file,line:l.line,steps:l.steps.map(m=>m.text),usage:a.get(l.name)?.size??0})).sort((l,m)=>m.usage-l.usage||l.name.localeCompare(m.name));return{steps:h,stepSets:d,scannedFiles:s.length,scannedCaches:c.scannedCaches}}function Ae(e,t){let n=t.toLowerCase();return{...e,steps:e.steps.filter(r=>r.text.toLowerCase().includes(n)),stepSets:e.stepSets.filter(r=>r.name.toLowerCase().includes(n)||r.steps.some(s=>s.toLowerCase().includes(n)))}}function Ft(e){let t={},n="saffron,feature,gherkin,cucumber";for(let r of e.steps)t[`step: ${r.text}`]={scope:n,prefix:r.text,body:[r.text.replace(/\$/g,"\\$")],description:`${r.status} \xB7 used in ${r.usage} scenario(s)`};for(let r of e.stepSets)t[`StepSet: ${r.name}`]={scope:n,prefix:`StepSet ${r.name}`,body:[`StepSet ${r.name}`],description:`step set (${r.steps.length} steps) \u2014 ${r.file}`};return JSON.stringify(t,null,2)+`
|
|
6
|
+
`}import{Server as Er}from"@modelcontextprotocol/sdk/server/index.js";import{StdioServerTransport as Ar}from"@modelcontextprotocol/sdk/server/stdio.js";import{CallToolRequestSchema as Pr,ListToolsRequestSchema as Cr}from"@modelcontextprotocol/sdk/types.js";import J from"node:fs";import Y from"node:path";import Ot from"node:fs";import xr from"node:path";import jt from"node:fs";import Sr from"node:path";var It=/\{env:([A-Za-z_][A-Za-z0-9_]*)\}/g;function Se(e){return e.replace(It,(t,n)=>{let r=process.env[n];if(r===void 0)throw new Error(`environment variable ${n} is not set (referenced as {env:${n}}) \u2014 export it or add it to .env`);return r})}function Ye(e){let t=new Set;for(let n of e)if(n)for(let r of n.matchAll(It))t.add(r[1]);return t}function Ke(e){let t=[];for(let n of e.steps){t.push(n.text,n.resolvedText),n.docString&&t.push(n.docString);for(let r of n.table??[])t.push(...r)}return t}function xe(e){let t=new Map;for(let n of Ye(Ke(e))){let r=process.env[n];r!==void 0&&t.set(n,r)}return t}function ue(e,t){let n=e;for(let[r,s]of[...t.entries()].sort((o,i)=>i[1].length-o[1].length))s.length<4||(n=n.split(s).join(`{env:${r}}`));return n}function _t(e){let t=Sr.join(e,".env");if(jt.existsSync(t))for(let n of jt.readFileSync(t,"utf8").split(`
|
|
7
|
+
`)){let r=n.trim();if(!r||r.startsWith("#"))continue;let s=r.indexOf("=");if(s<=0)continue;let o=r.slice(0,s).trim();if(!/^[A-Za-z_][A-Za-z0-9_]*$/.test(o))continue;let i=r.slice(s+1).trim();(i.startsWith('"')&&i.endsWith('"')||i.startsWith("'")&&i.endsWith("'"))&&(i=i.slice(1,-1)),process.env[o]===void 0&&(process.env[o]=i)}}function H(e){_t(e);let t=xr.join(e,"saffron.config.json");return Ot.existsSync(t)?JSON.parse(Ot.readFileSync(t,"utf8")):{}}var br={features:"features",actionTimeoutMs:5e3,retries:1,maxTurns:100,strict:!1,verifyProposals:!0,snapshotMode:"none",reuseSteps:!0,browser:"chromium",workers:1,assertionPolicy:"strict"};function Dt(e){return{...br,...Object.fromEntries(Object.entries(e).filter(([,t])=>t!==void 0))}}import Qe from"node:fs";import Pe from"node:path";import{fileURLToPath as vr}from"node:url";function Ce(){let e=Pe.dirname(vr(import.meta.url));for(let t=0;t<6;t++){let n=Pe.join(e,"package.json");if(Qe.existsSync(n))try{if(JSON.parse(Qe.readFileSync(n,"utf8")).name==="saffron-ai")return e}catch{}e=Pe.dirname(e)}}function te(){let e=Ce();if(!e)return"unknown";try{return JSON.parse(Qe.readFileSync(Pe.join(e,"package.json"),"utf8")).version??"unknown"}catch{return"unknown"}}import G from"node:fs";import Fe from"node:path";function Mt(e,t){G.mkdirSync(Fe.dirname(e),{recursive:!0}),G.writeFileSync(e,`${JSON.stringify(t,null,2)}
|
|
8
|
+
`)}function fe(e){return JSON.parse(G.readFileSync(e,"utf8"))}function oe(e){let t=Fe.join(e,".saffron","proposals");if(!G.existsSync(t))return[];let n=[];for(let r of G.readdirSync(t)){let s=Fe.join(t,r);if(G.statSync(s).isDirectory())for(let o of G.readdirSync(s)){if(!o.endsWith(".json"))continue;let i=Fe.join(s,o);n.push({file:i,proposal:fe(i)})}}return n}function Nt(e,t){let n=fe(t),r=B(e,n.cache.feature,n.cache.scenario);return pe(r,n.cache),G.rmSync(t),r}function Ut(e){G.rmSync(e)}import je from"node:fs";import Lt from"node:path";function Vt(e){return Lt.join(e,".saffron","history.jsonl")}function Bt(e,t,n,r){let s=e.map(o=>({feature:o.scenario.featurePath,scenario:o.scenario.displayName,tags:[...o.scenario.tags],status:o.status,source:o.source,durationMs:o.durationMs,verified:o.verified,seededSteps:o.seededSteps,adaptations:o.adaptations.length,driftSteps:o.stepResults.filter(i=>i.drift).length,aiCalls:o.usage.aiCalls,tokensInOut:o.usage.inputTokens+o.usage.outputTokens,cacheRead:o.usage.cacheReadTokens,cacheWrite:o.usage.cacheCreationTokens,costUsd:o.usage.costUsd??0,error:o.error?.split(`
|
|
9
|
+
`)[0]}));return{runId:t.toISOString(),startedAt:t.toISOString(),finishedAt:new Date().toISOString(),baseURL:r,green:s.filter(o=>o.status==="green").length,yellow:s.filter(o=>o.status==="yellow").length,red:s.filter(o=>o.status==="red").length,aiCalls:s.reduce((o,i)=>o+i.aiCalls,0),costUsd:s.reduce((o,i)=>o+i.costUsd,0),divergentSteps:n,scenarios:s}}function zt(e,t){let n=Vt(e);je.mkdirSync(Lt.dirname(n),{recursive:!0}),je.appendFileSync(n,`${JSON.stringify(t)}
|
|
10
|
+
`)}function Ie(e,t=50){let n=Vt(e);if(!je.existsSync(n))return[];let r=je.readFileSync(n,"utf8").trim().split(`
|
|
11
|
+
`).filter(Boolean),s=[];for(let o of r.slice(-t))try{s.push(JSON.parse(o))}catch{}return s}var Xe=e=>{let t=e.green+e.yellow+e.red;return t===0?0:Math.round((e.green+e.yellow)/t*100)};function qt(e,t){let n=[...e,t],r=n.slice(-20),s=n.slice(-10),o=e.at(-1),i=o?{passRatePp:Xe(t)-Xe(o),costUsd:t.costUsd-o.costUsd,aiCalls:t.aiCalls-o.aiCalls,red:t.red-o.red}:void 0,a=new Map;for(let d of s)for(let l of d.scenarios){let m=`${l.feature}::${l.scenario}`,g=a.get(m)??{feature:l.feature,scenario:l.scenario,heals:0,reds:0,runs:0};g.runs++,l.source==="agent-heal"&&l.status==="yellow"&&g.heals++,l.status==="red"&&g.reds++,a.set(m,g)}let c=[...a.values()].filter(d=>d.heals>=2||d.reds>=3).sort((d,l)=>l.heals+l.reds-(d.heals+d.reds)),p=new Map;for(let d of r)for(let l of d.scenarios)l.status==="red"&&l.error&&p.set(l.error,(p.get(l.error)??0)+1);let h=[...p.entries()].sort((d,l)=>l[1]-d[1]).slice(0,3).map(([d,l])=>({message:d,count:l}));return{prev:i,series:{passRate:r.map(Xe),costUsd:r.map(d=>Number(d.costUsd.toFixed(4))),aiCalls:r.map(d=>d.aiCalls)},chronic:c,divergentSteps:t.divergentSteps,topIssues:h}}var wr=/^\s*(?:Scenario(?: Outline| Template)?|Example):\s*(.+?)\s*$/,kr=/^\s*StepSet:/;function Ze(e,t){return Y.relative(e,t).split(Y.sep).join("/")}function $r(e,t){let n={green:0,yellow:1,red:2};return!e||n[t]>n[e]?t:e}function Ht(e,t){return`${e} :: ${t}`}function _e(e,t,n={}){let r=t.features??"features",s=Y.isAbsolute(r)?r:Y.join(e,r),o=Y.join(e,".saffron","reports","latest.json"),i=Y.join(e,".saffron","reports","latest.html"),a;try{a=JSON.parse(J.readFileSync(o,"utf8"))}catch{a=void 0}let c=new Map;for(let f of a?.scenarios??[]){let v=f.scenario.replace(/ \(example \d+\)$/,""),b=Ht(f.feature,v);c.set(b,$r(c.get(b),f.status))}let p=new Map,h;try{p=Z(de(s),e)}catch(f){h=f instanceof Error?f.message:String(f)}let d=J.existsSync(s)?[...J.readdirSync(s,{recursive:!0})].map(String).filter(se).map(f=>Y.join(s,f)).sort():[],l=[],m=new Map;for(let f of d){let v=Ze(e,f),b="";try{b=J.readFileSync(f,"utf8")}catch(C){l.push({path:v,name:v,scenarios:[],stepSets:0,error:String(C)});continue}let w=b.split(`
|
|
12
|
+
`),P=new Map,S=0;w.forEach((C,X)=>{let V=C.match(wr);V&&!P.has(V[1])?P.set(V[1],X+1):kr.test(C)&&S++});let k=w.find(C=>/^\s*Feature:/.test(C)),A=k?k.replace(/^\s*Feature:\s*/,"").trim():v,j;try{j=ee(f,e,p)}catch(C){l.push({path:v,name:A,scenarios:[],stepSets:S,error:h??(C instanceof Error?C.message:String(C))});continue}let O=new Map;for(let C of j){let X=O.get(C.name);if(X){X.rows+=1;continue}O.set(C.name,{name:C.name,line:P.get(C.name),tags:[...C.tags],outline:Object.keys(C.params).length>0,rows:1,cached:J.existsSync(B(e,C.featurePath,C.name)),proposal:J.existsSync(ye(e,C.featurePath,C.name)),lastStatus:c.get(Ht(C.featurePath,C.name))});for(let V of C.tags)m.set(V,(m.get(V)??0)+1)}l.push({path:v,name:A,scenarios:[...O.values()],stepSets:S})}let g=oe(e).map(({file:f,proposal:v})=>({file:Ze(e,f),feature:v.cache.feature,scenario:v.cache.scenario,mode:v.meta.mode,createdAt:v.meta.createdAt,verified:v.meta.verified,proofError:v.meta.proofError,adaptations:v.meta.adaptations??[],narrative:v.meta.narrative,suggestedFeatureEdit:v.meta.suggestedFeatureEdit,recordedFor:v.meta.recordedFor,aiCalls:v.meta.usage?.aiCalls??0,costUsd:v.meta.usage?.costUsd})).sort((f,v)=>f.file.localeCompare(v.file)),u;try{u=q(e,r,p)}catch{u={steps:[],stepSets:[],scannedFiles:0,scannedCaches:0}}let y=Tr(e),$=Ie(e,n.historyLimit??30).map(f=>({startedAt:f.startedAt,green:f.green,yellow:f.yellow,red:f.red,aiCalls:f.aiCalls,costUsd:f.costUsd})),x=Y.join(e,"saffron.config.json");return{tool:"saffron",version:te(),projectRoot:e,packageInstalled:J.existsSync(Y.join(e,"node_modules","saffron-ai")),config:{file:J.existsSync(x)?"saffron.config.json":void 0,effective:Dt(t)},features:l,tags:[...m.entries()].map(([f,v])=>({tag:f,scenarios:v})).sort((f,v)=>f.tag.localeCompare(v.tag)),proposals:g,lastRun:a?{startedAt:a.startedAt,finishedAt:a.finishedAt,totals:a.totals,reportHtml:J.existsSync(i)?Ze(e,i):void 0}:void 0,history:$,vocabulary:{steps:u.steps.length,recorded:u.steps.filter(f=>f.status==="recorded").length,unrecorded:u.steps.filter(f=>f.status==="unrecorded").length,stepSets:u.stepSets.length,divergent:u.steps.filter(f=>f.status==="divergent").map(f=>f.text),duplicateWordings:y.slice(0,50)}}}function Gt(e){let t=e.features.reduce((s,o)=>s+o.scenarios.length,0),n=e.features.reduce((s,o)=>s+o.scenarios.filter(i=>i.cached).length,0);console.log(`saffron ${e.version} \xB7 ${e.features.length} feature file(s) \xB7 ${t} scenario(s), ${n} cached`);for(let s of e.features){let o=s.scenarios.map(i=>i.proposal?"\u25D0":i.cached?"\u25CF":"\u25CB").join("");console.log(` ${s.path} ${o}${s.error?` \u2717 ${s.error.split(`
|
|
13
|
+
`)[0]}`:""}`)}if(e.tags.length&&console.log(`tags: ${e.tags.map(s=>`${s.tag} (${s.scenarios})`).join(", ")}`),console.log(e.proposals.length?`${e.proposals.length} proposal(s) pending review: ${e.proposals.map(s=>s.file).join(", ")}`:"no proposals pending review"),e.lastRun){let s=e.lastRun.totals;console.log(`last run ${e.lastRun.finishedAt}: ${s.green} passed \xB7 ${s.yellow} pending review \xB7 ${s.red} failed \xB7 $${s.costUsd.toFixed(2)}`)}let r=e.vocabulary;console.log(`vocabulary: ${r.steps} steps (${r.recorded} recorded, ${r.unrecorded} unrecorded), ${r.stepSets} step sets, ${r.divergent.length} divergent, ${r.duplicateWordings.length} duplicate wording group(s)`),console.log(`config: ${e.config.file??"(defaults, no saffron.config.json)"}`)}var Rr=/^(<[^>]+>|\{env:[^}]+\}|\{date[^}]*\})$/;function Wt(e){return e?e.role?`${e.role}${e.name?` "${e.name}"`:e.nameRegex?` /${e.nameRegex}/`:""}`:e.label?`label "${e.label}"`:e.text?`text "${e.text}"`:e.testId?`testid ${e.testId}`:e.placeholder?`placeholder "${e.placeholder}"`:e.fallbackSelectors?.length?e.fallbackSelectors[0]:"":""}function Tr(e){let t;try{t=W(e)}catch{return[]}let n=new Map;for(let r of t.steps.values()){if(r.actions.length===0)continue;let s=Ee(r.gherkin).pattern,o=[...r.gherkin.matchAll(/"([^"]*)"/g)].map(d=>d[1]),i=d=>d!==void 0&&(o.includes(d)||Rr.test(d))?"\xABarg\xBB":d,a=r.actions.map(d=>{let{pageFingerprint:l,value:m,promptText:g,...u}=d;return{...u,value:i(m),promptText:i(g)}}),c=JSON.stringify(a),p=r.actions.map(d=>`${d.action}${Wt(d.target)?` ${Wt(d.target)}`:d.url?` ${d.url}`:""}`).join(" \xB7 "),h=n.get(c)??{steps:new Map,actions:p};h.steps.has(s)||h.steps.set(s,r.gherkin),n.set(c,h)}return[...n.values()].filter(r=>r.steps.size>1).map(r=>({steps:[...r.steps.values()].sort(),actions:r.actions})).sort((r,s)=>s.steps.length-r.steps.length||r.steps[0].localeCompare(s.steps[0]))}async function Jt(e,t){let n=new Er({name:"saffron",version:te()},{capabilities:{tools:{}}});n.setRequestHandler(Cr,async()=>({tools:[{name:"search_steps",description:"Search the project's step vocabulary. Returns known Gherkin steps with status (recorded = replays at zero tokens, divergent, unrecorded), usage counts, and source files. ALWAYS reuse an existing wording exactly instead of inventing a near-duplicate: exact step text is the replay-cache identity, so a new wording costs a fresh AI recording.",inputSchema:{type:"object",properties:{query:{type:"string",description:"Case-insensitive substring filter over step text; omit to list everything."}}}},{name:"list_step_sets",description:"List the project's StepSet definitions (named reusable step sequences from .saffron files) with their steps, defining file, and usage. Invoke one inside a scenario with a line reading exactly: StepSet <name>",inputSchema:{type:"object",properties:{}}},{name:"project_status",description:"The project overview (same data as `saffron status --json`): every feature file with its scenarios (header line, tags, outline rows, cached / proposal pending / last run status), the tag list, pending proposals with narrative and proof-replay verdict, the last run's totals, run history, vocabulary health (divergent steps, duplicate wordings proven by identical recordings) and the effective config. Call it before editing tests to see what is already recorded and what awaits review.",inputSchema:{type:"object",properties:{}}}]})),n.setRequestHandler(Pr,async r=>Fr(r.params.name,r.params.arguments,e,t)),await n.connect(new Ar)}function Fr(e,t,n,r){let s=o=>({content:[{type:"text",text:JSON.stringify(o,null,2)}]});if(e==="search_steps"){let o=q(n,r),i=typeof t?.query=="string"?t.query:void 0,a=i?Ae(o,i):o;return s({steps:a.steps,scannedFiles:a.scannedFiles,scannedCaches:a.scannedCaches})}if(e==="list_step_sets"){let o=q(n,r);return s({stepSets:o.stepSets})}return e==="project_status"?s(_e(n,H(n))):{content:[{type:"text",text:`Unknown tool: ${e}`}],isError:!0}}import{query as jr}from"@anthropic-ai/claude-agent-sdk";function Ir(e,t){let n=t.steps.map(s=>`- [${s.status}] ${s.keyword} ${s.text}`).join(`
|
|
7
14
|
`),r=t.stepSets.map(s=>`- StepSet ${s.name}
|
|
8
15
|
${s.steps.map(o=>` ${o}`).join(`
|
|
9
16
|
`)}`).join(`
|
|
10
17
|
`);return["Draft a Saffron feature file from the requirements below.","","EXISTING STEP VOCABULARY \u2014 reuse these wordings EXACTLY (character for character) wherever they express what a step needs to do. Steps marked [recorded] or [divergent] already have replay recordings: reusing them makes the new file replay almost for free. Only invent a new wording when nothing in the vocabulary expresses the intent, and then follow the same style:",n.length>0?n:"(none yet \u2014 this is the project's first file)","",t.stepSets.length>0?`EXISTING STEP SETS \u2014 invoke with a line reading exactly \`StepSet <name>\` instead of repeating their steps:
|
|
11
18
|
${r}`:void 0,"","RULES:","1. Output ONLY the feature file content \u2014 no prose, no code fences, no explanations.","2. Gherkin/.saffron syntax: Feature:, Scenario:, Scenario Outline: + Examples, Given/When/Then/And/But. Prefer concrete scenarios; use an Outline only when the requirements list variations of the same flow.","3. Then steps are assertions \u2014 state exactly what must be true, one condition per step. End every scenario with at least one Then.","4. If the same 3+ step sequence would appear in two or more scenarios, define it once as a StepSet (first step a guard, last step an exit assertion) and invoke it. Definition uses a colon (StepSet: Name), invocation does not (StepSet Name).",'5. Use 2-column data tables for key/value input, header tables for lists of records, and """ doc strings for free text.',"6. Steps are natural language for an AI browser agent \u2014 never CSS selectors or code.","","REQUIREMENTS:",e.trim()].filter(s=>s!==void 0).join(`
|
|
12
|
-
`)}async function
|
|
13
|
-
`,usage:r}}import
|
|
14
|
-
`)
|
|
15
|
-
`);return{contents:{kind:
|
|
19
|
+
`)}async function Yt(e,t,n){let r={aiCalls:0,inputTokens:0,outputTokens:0,cacheReadTokens:0,cacheCreationTokens:0,costUsd:0},s="",o=jr({prompt:Ir(e,t),options:{model:n,maxTurns:1,permissionMode:"bypassPermissions",persistSession:!1,settingSources:[],allowedTools:[],systemPrompt:"You are Saffron's feature-file author. You write clean, assertive Gherkin that reuses the project's exact step vocabulary. You output only file content."}});for await(let i of o)if(i.type==="assistant")for(let a of i.message.content)a.type==="text"&&(s+=a.text);else i.type==="result"&&(r.aiCalls=i.num_turns,r.costUsd=i.total_cost_usd,"usage"in i&&i.usage&&(r.inputTokens=i.usage.input_tokens??0,r.outputTokens=i.usage.output_tokens??0,r.cacheReadTokens=i.usage.cache_read_input_tokens??0,r.cacheCreationTokens=i.usage.cache_creation_input_tokens??0));return s=s.replace(/^\s*```(?:gherkin|saffron)?\s*\n/,"").replace(/\n```\s*$/,"").trim(),{content:s+`
|
|
20
|
+
`,usage:r}}import De from"node:fs";import ie from"node:path";import{createConnection as _r,ProposedFeatures as Or,TextDocuments as Dr,TextDocumentSyncKind as Mr,CompletionItemKind as be,InsertTextFormat as Kt,DiagnosticSeverity as Oe,MarkupKind as Qt}from"vscode-languageserver/node";import{TextDocument as Nr}from"vscode-languageserver-textdocument";var Xt=/^(\s*)(Given|When|Then|And|But|\*)\s+(.+?)\s*$/,Ur=/^\s*StepSet:\s*(.+?)\s*$/,et=/^(\s*)(StepSet)\s+([^\s:].*?)\s*$/,tt={recorded:"\u25CF recorded \u2014 replays at zero tokens",divergent:"\u25CF divergent \u2014 same text, different recordings",unrecorded:"\u25CB written, not recorded yet"};function Lr(e,t){let n=i=>new Set(i.toLowerCase().replace(/"[^"]*"/g,'"x"').split(/[^a-z0-9"<>]+/).filter(Boolean)),r=n(e),s=n(t);if(r.size===0||s.size===0)return 0;let o=0;for(let i of r)s.has(i)&&o++;return o/(r.size+s.size-o)}function Vr(e){try{let t=JSON.parse(De.readFileSync(ie.join(e,"saffron.config.json"),"utf8"));return typeof t.features=="string"?t.features:void 0}catch{return}}function tn(e,t,n={}){let r=e,s=t,o=_r(Or.all,process.stdin,process.stdout),i=new Dr(Nr),a={steps:[],stepSets:[],scannedFiles:0,scannedCaches:0},c=new Map,p,h=()=>{try{let g=ie.isAbsolute(s)?s:ie.join(r,s);c=Z(de(g),r),a=q(r,s,c)}catch(g){o.console.warn(`saffron vocabulary rebuild: ${String(g)}`)}for(let g of i.all())m(g)},d=()=>{clearTimeout(p),p=setTimeout(h,300)};function l(g){return a.steps.find(u=>u.text===g)?.status}function m(g){if(!g.uri.endsWith(".saffron"))return;let u=[],y=g.getText().split(`
|
|
21
|
+
`),$=new Map;for(let x=0;x<y.length;x++){let f=y[x],v={start:{line:x,character:0},end:{line:x,character:f.length}},b=f.match(et);b&&!c.has(b[3])&&u.push({range:v,severity:Oe.Error,source:"saffron",message:`StepSet "${b[3]}" is not defined anywhere in the project.`});let w=f.match(Ur);if(w){let k=c.get(w[1]),A=ie.relative(r,new URL(g.uri).pathname).replace(/\\/g,"/");($.has(w[1])||k&&k.file!==A)&&u.push({range:v,severity:Oe.Error,source:"saffron",message:`StepSet "${w[1]}" is defined twice \u2014 set names are project-wide. If this line was meant to INVOKE the set, remove the colon: "StepSet ${w[1]}".`}),$.set(w[1],x)}let P=f.trim();if(P&&!b&&!w){for(let[k]of c)if(P.toLowerCase()===k.toLowerCase()){u.push({range:v,severity:Oe.Warning,source:"saffron",message:`Did you mean "StepSet ${k}"? Invoking a step set needs the StepSet keyword, like a step needs Given/When/Then.`});break}}let S=f.match(Xt);if(S&&l(S[3])!=="recorded"){for(let k of a.steps)if(!(k.text===S[3]||k.status==="unrecorded")&&Lr(k.text,S[3])>=.7){u.push({range:{start:{line:x,character:f.indexOf(S[3])},end:{line:x,character:f.length}},severity:Oe.Information,source:"saffron",message:`Similar to the ${k.status} step "${k.text}" (\xD7${k.usage}). Reusing the exact wording replays at zero tokens; a new wording costs a fresh AI recording.`});break}}}o.sendDiagnostics({uri:g.uri,diagnostics:u})}o.onInitialize(g=>{if(n.rootFromClient){let u=g.workspaceFolders?.[0]?.uri??g.rootUri??void 0;if(u&&u.startsWith("file:")){let y=decodeURIComponent(new URL(u).pathname);De.existsSync(y)&&(r=y,s=Vr(y)??t)}}return h(),{capabilities:{textDocumentSync:Mr.Incremental,completionProvider:{triggerCharacters:[" "]},definitionProvider:!0,hoverProvider:!0}}}),o.onCompletion(g=>{let u=i.get(g.textDocument.uri);if(!u)return[];let y=u.getText({start:{line:g.position.line,character:0},end:g.position});return Br(u.uri,y,g.position.line,a)}),o.onDefinition(g=>{let u=i.get(g.textDocument.uri);if(!u)return;let y=u.getText({start:{line:g.position.line,character:0},end:{line:g.position.line+1,character:0}}).replace(/\n$/,""),$=Zt(y,g.position.character);if($){let b=en(r,$.name);return b?{uri:`file://${b.file}`,range:{start:{line:b.line,character:0},end:{line:b.line,character:0}}}:void 0}let x=y.match(et);if(!x)return;let f=c.get(x[3]);return f?{uri:`file://${ie.join(r,f.file)}`,range:{start:{line:f.line-1,character:0},end:{line:f.line-1,character:0}}}:void 0}),o.onHover(g=>{let u=i.get(g.textDocument.uri);if(!u)return;{let f=u.getText({start:{line:g.position.line,character:0},end:{line:g.position.line+1,character:0}}).replace(/\n$/,""),v=Zt(f,g.position.character);if(v){let b=en(r,v.name),w=b?`defined in ${ie.relative(r,b.file)} (line ${b.line+1}); the value is resolved at replay and never shown here`:"not found in .env or .env.example; export it in the shell or add it to a git-ignored .env";return{contents:{kind:"markdown",value:`**{env:${v.name}}** ${w}.`}}}}let y=u.getText({start:{line:g.position.line,character:0},end:{line:g.position.line+1,character:0}}).replace(/\n$/,""),$=y.match(et);if($){let f=a.stepSets.find(b=>b.name===$[3]);if(!f)return;let v=f.steps.map(b=>`- ${tt[l(b)??"unrecorded"][0]} ${b}`).join(`
|
|
22
|
+
`);return{contents:{kind:Qt.Markdown,value:`**StepSet ${f.name}** \u2014 expands to:
|
|
16
23
|
|
|
17
|
-
${
|
|
24
|
+
${v}`}}}let x=y.match(Xt);if(x){let f=a.steps.find(v=>v.text===x[3]);return f?{contents:{kind:Qt.Markdown,value:`${tt[f.status]}
|
|
18
25
|
|
|
19
|
-
Used in ${
|
|
26
|
+
Used in ${f.usage} place(s): ${f.files.join(", ")}${f.fromStepSet?`
|
|
20
27
|
|
|
21
|
-
Defined in StepSet **${
|
|
22
|
-
`).findIndex(i=>new RegExp(`^\\s*(?:export\\s+)?${t}\\s*=`).test(i));if(o!==-1)return{file:r,line:o}}}function
|
|
23
|
-
`)}));if(/^\s*(Given|When|Then|And|But|\*)\s+/.test(t))return r.steps.map((a,
|
|
28
|
+
Defined in StepSet **${f.fromStepSet}**`:""}`}}:void 0}}),i.onDidOpen(g=>m(g.document)),i.onDidChangeContent(g=>m(g.document)),i.onDidSave(()=>d()),o.onDidChangeWatchedFiles(()=>d()),i.listen(o),o.listen()}function Zt(e,t){let n=/\{env:([A-Za-z_][A-Za-z0-9_]*)\}/g,r;for(;r=n.exec(e);)if(t>=r.index&&t<=r.index+r[0].length)return{name:r[1],start:r.index,end:r.index+r[0].length}}function en(e,t){for(let n of[".env",".env.example"]){let r=ie.join(e,n);if(!De.existsSync(r))continue;let o=De.readFileSync(r,"utf8").split(`
|
|
29
|
+
`).findIndex(i=>new RegExp(`^\\s*(?:export\\s+)?${t}\\s*=`).test(i));if(o!==-1)return{file:r,line:o}}}function Br(e,t,n,r){let s=e.endsWith(".saffron");if(/^\s*StepSet\s+/.test(t)&&s)return r.stepSets.map(a=>({label:a.name,kind:be.Function,detail:`step set \xB7 ${a.steps.length} step(s) \xB7 \xD7${a.usage}`,documentation:a.steps.join(`
|
|
30
|
+
`)}));if(/^\s*(Given|When|Then|And|But|\*)\s+/.test(t))return r.steps.map((a,c)=>({label:a.text,kind:be.Text,detail:`${tt[a.status]} \xB7 \xD7${a.usage}`,sortText:String(c).padStart(5,"0"),filterText:a.text}));let i=t.replace(/^\s*/,"").trimEnd().toLowerCase().replace(/\s+/g,"");if(i.length>0&&("stepset:".startsWith(i)||i.startsWith("stepset"))&&s){let a={start:{line:n,character:t.length-t.replace(/^\s*/,"").length},end:{line:n,character:t.length}},c=[{label:"StepSet: \u2026",kind:be.Keyword,detail:"define a reusable step set",documentation:`StepSet: Name
|
|
24
31
|
Given \u2026
|
|
25
32
|
When \u2026
|
|
26
33
|
|
|
27
|
-
Invoke it anywhere with: StepSet Name`,filterText:"step set stepset:",sortText:"0",insertTextFormat:
|
|
28
|
-
`),filterText:`step set ${p.name}`,sortText:`1${p.name}`,textEdit:{range:a,newText:`StepSet ${p.name}`}});return
|
|
29
|
-
`)){let
|
|
30
|
-
`)
|
|
31
|
-
`)
|
|
32
|
-
`)
|
|
33
|
-
`)}
|
|
34
|
-
`)
|
|
35
|
-
`)[0]})`:"")+". A row value is probably baked in as a literal instead of a <placeholder>; inspect the proposal, reject it, and re-record with saffron run --rerecord."}}await u.goto("about:blank")}if(p&&(S=await ve(u,t,p,{baseURL:o,actionTimeoutMs:i,retries:a}),S.status==="passed"))return{scenario:t,status:"green",source:"cache",stepResults:S.stepResults,adaptations:[],usage:fe,durationMs:Date.now()-r};if(!(this.options.agent!==void 0&&this.cdpEndpoint!==void 0&&!n.agentDisabled)){let g=this.options.agent!==void 0&&this.cdpEndpoint===void 0&&!n.agentDisabled?`recording/healing requires Chromium (the agent attaches over CDP). Record with --browser chromium (the default), then replay with --browser ${this.options.browser}.`:"no agent is configured.";return{scenario:t,status:"red",source:"cache",stepResults:S?.stepResults??[],adaptations:[],usage:fe,durationMs:Date.now()-r,error:p?S?.status==="stale"?`Cache is stale (feature file changed) and ${g}`:`Replay failed and ${g} ${S?.failure?.error??""}`.trimEnd():`No cache exists and ${g}`}}let h=p!==void 0&&S!==void 0&&S.status==="failed",w=0,y;try{if(h)y=await this.withDialogsVisible(()=>this.options.agent.run({scenario:t,baseURL:o,mode:"heal",model:this.options.healModel,cdpEndpoint:this.cdpEndpoint,assertionPolicy:this.options.assertionPolicy,completedSteps:S.stepResults.filter(g=>g.status==="passed").map(g=>g.gherkin),failedStep:{text:S.failure.stepText,kind:S.failure.kind,error:S.failure.error},takeFingerprint:async()=>{let g=we(this.browser)??u;return oe(g)}}));else if(await u.goto("about:blank"),this.options.reuseSteps!==!1){let g=await this.recordWithSeeding(t,u);w=g.seededCount,y=g}else y=await this.withDialogsVisible(()=>this.options.agent.run({scenario:t,baseURL:o,mode:"record",cdpEndpoint:this.cdpEndpoint,assertionPolicy:this.options.assertionPolicy,takeFingerprint:async()=>{let g=we(this.browser)??u;return oe(g)}}))}catch(g){let k=g instanceof Error?g.message:String(g);return{scenario:t,status:"red",source:h?"agent-heal":"agent-record",stepResults:S?.stepResults??[],adaptations:[],usage:fe,durationMs:Date.now()-r,error:`Agent invocation failed: ${k.split(`
|
|
36
|
-
`)[0]}`}}if(y.success&&h&&S.failure.kind==="assertion"&&(this.options.assertionPolicy!=="adaptable-mid"||an(p.steps,S.failure.stepIndex))){let g=p.steps[S.failure.stepIndex],k=we(this.browser)??u,v=new Map(Object.entries(S.captured)),b=nt(t,S.failure.stepIndex);try{for(let C of g.actions)await tt(k,C,b,this.options.actionTimeoutMs??5e3,o,v)}catch(C){let x=C instanceof Error?C.message:String(C);return{scenario:t,status:"red",source:"agent-heal",stepResults:S.stepResults,narrative:y.narrative,adaptations:y.adaptations,suggestedFeatureEdit:y.suggestedFeatureEdit,usage:y.usage,durationMs:Date.now()-r,error:`Assertion failed as written: "${g.gherkin}". Saffron never adapts assertions. If this change is intended, update the .feature file or re-record the scenario (saffron run --rerecord --filter @tag). If the recording itself looks wrong (an empty value, a wrong target), re-record it the same way. Verification error: ${x.split(`
|
|
37
|
-
`)[0]}`}}}return await this.finalizeAgentRun(t,y,h,p,r,w,h?S?.failure:void 0)}finally{await l.close()}}async finalizeAgentRun(t,n,r,s,o,i=0,a){let d=r?"agent-heal":"agent-record",p=n.recordedSteps.map(v=>({gherkin:v.gherkin,kind:v.kind,status:n.success?"passed":"failed"}));if(!n.success)return{scenario:t,status:"red",source:d,stepResults:p,narrative:n.narrative,adaptations:n.adaptations,usage:n.usage,durationMs:Date.now()-o,error:n.narrative};let m=r?cn(s.steps,n.recordedSteps,this.options.assertionPolicy==="adaptable-mid",a?{stepIndex:a.stepIndex,actionIndex:a.actionIndex}:void 0):n.recordedSteps,c=[...n.adaptations],l={...n.usage},u,S;if(this.options.verifyProposals!==!1){let v=await this.proofReplay(t,this.buildCandidateCache(t,m));try{if(v.result.status!=="passed"&&v.result.failure){let b=v.result.failure;if(this.options.agent&&!(r&&b.kind==="assertion"&&(this.options.assertionPolicy!=="adaptable-mid"||an(s.steps,b.stepIndex))))try{let x=await this.withDialogsVisible(()=>this.options.agent.run({scenario:t,baseURL:this.options.baseURL,mode:"refine",model:r?this.options.healModel:void 0,cdpEndpoint:this.cdpEndpoint,assertionPolicy:this.options.assertionPolicy,completedSteps:v.result.stepResults.filter(R=>R.status==="passed").map(R=>R.gherkin),failedStep:{text:b.stepText,kind:b.kind,error:b.error},takeFingerprint:async()=>oe(we(this.browser)??v.page)}));l=ln(l,x.usage),x.success&&(c.push(...x.adaptations),m=r?cn(m,x.recordedSteps,this.options.assertionPolicy==="adaptable-mid"):Qr(m,x.recordedSteps),await v.context.close(),v=await this.proofReplay(t,this.buildCandidateCache(t,m)))}catch{}}u=v.result.status==="passed",u||(S=v.result.failure?.error??`proof replay ${v.result.status}`)}finally{await v.context.close().catch(()=>{})}}let f=[...n.featureEdits];for(let v of At(m,q(this.options.projectRoot)))c.push(`duplicate wording: "${v.from}" recorded exactly the same actions as the existing step "${v.to}" (${v.feature} \u203A ${v.scenario}). Accept with --with-feature-edit to converge on the canonical wording.`),f.some(b=>b.index===v.index)||f.push({index:v.index,text:v.to});let h=be(t),w=v=>h.size>0?pe(v,h):v;c=c.map(w);let y={meta:{createdAt:new Date().toISOString(),mode:r?"heal":"record",narrative:w(n.narrative),adaptations:c,suggestedFeatureEdit:n.suggestedFeatureEdit?w(n.suggestedFeatureEdit):void 0,featureEdits:f.length>0?f:void 0,verified:u,proofError:S?w(S):void 0,seededSteps:i>0?i:void 0,recordedFor:t.displayName,usage:l},cache:{version:1,feature:t.featurePath,scenario:t.name,recordedAt:new Date().toISOString(),recordedBy:"agent",steps:m}},g=Xt(m);g.length>0&&(u=!1,S=[S,...g].filter(Boolean).join(" "));let k=Se(this.options.projectRoot,t.featurePath,t.name);return Bt(k,y),{scenario:t,status:"yellow",source:d,stepResults:p,narrative:n.narrative,adaptations:c,suggestedFeatureEdit:n.suggestedFeatureEdit,featureEdits:f.length>0?f:void 0,verified:u,proofError:S,seededSteps:i>0?i:void 0,proposalFile:k,usage:l,durationMs:Date.now()-o}}};import Sn from"node:path";import{createRequire as rs}from"node:module";import{query as ss}from"@anthropic-ai/claude-agent-sdk";import{createSdkMcpServer as Xr,tool as ge}from"@anthropic-ai/claude-agent-sdk";import{z as $}from"zod";var M=e=>({content:[{type:"text",text:e}]});function pn(e){let t={};return e.role&&(t.role=e.role),e.name&&(t.name=e.name),e.nameRegex&&(t.nameRegex=e.nameRegex),e.label&&(t.label=e.label),e.text&&(t.text=e.text),e.selector&&(t.fallbackSelectors=[e.selector]),e.frame&&(t.frame=e.frame),Object.keys(t).length>0?t:void 0}var un={role:$.string().optional().describe("ARIA role of the element"),name:$.string().optional().describe("Accessible name \u2014 use the SHORTEST stable substring; never include volatile content like prices or dates"),nameRegex:$.string().optional().describe("Regex for the accessible name when it embeds volatile content, e.g. 'Select \\\\{date\\\\+1\\\\} as checkin' cannot work \u2014 instead use nameRegex 'Select .* as checkin'"),label:$.string().optional().describe("Form-label text for getByLabel \u2014 the literal <label> text only, never a free-text description"),text:$.string().optional().describe("getByText locator text"),selector:$.string().optional().describe("CSS fallback selector"),frame:$.string().optional().describe(`CSS selector of the iframe hosting the element, e.g. 'iframe[title="Feedback widget"]' \u2014 REQUIRED whenever the element lives inside an iframe; without it replay looks only at the top-level page`)};function fn(e){return Xr({name:"saffron",version:"0.1.0",tools:[ge("saffron_step","Announce the Gherkin step you are about to perform. MUST be called before executing browser actions for each step, with the step's zero-based index and exact text.",{index:$.number().int().min(0),text:$.string(),adaptation:$.string().optional().describe("If the written step does not match the real UI and you are deviating from it, describe the deviation in one sentence.")},async t=>(e.currentStepIndex=t.index,e.announced.add(t.index),t.adaptation&&e.adaptations.push(`Step ${t.index+1} ("${t.text}"): ${t.adaptation}`),M(`Recording step ${t.index}: ${t.text}`))),ge("saffron_capture",`Record a captureText action for the current step: at replay time, the target element's text is read and stored under saveAs for later expectDiffers comparisons. Use for steps like 'I record the displayed price as "X"'. The target must select exactly the element whose text is the value.`,{saveAs:$.string().describe("Name to store the captured text under"),...un},async t=>{let n=pn(t);return n?(e.addActions([{action:"captureText",target:n,saveAs:t.saveAs}]),M(`Capture recorded as "${t.saveAs}".`)):M("ERROR: capture needs a target (role/name, nameRegex, label, text, or selector).")}),ge("saffron_dialog","Record how a native dialog (alert/confirm/prompt) must be handled at replay. Call this IMMEDIATELY whenever an action triggers a native dialog \u2014 whether you handled it via browser_handle_dialog or the browser reported it as already handled. At replay Saffron arms this handler BEFORE the triggering action, so without this call the replay will auto-dismiss the dialog and diverge from what you saw.",{accept:$.boolean().describe("true = accept/OK the dialog, false = dismiss/cancel"),promptText:$.string().optional().describe("Text typed into a prompt() before accepting")},async t=>(e.addArmedAction({action:"handleDialog",value:t.accept?"accept":"dismiss",...t.promptText?{promptText:t.promptText}:{}}),M(`Dialog handling recorded (${t.accept?"accept":"dismiss"}).`))),ge("saffron_wait_api","Record a network wait for the current step: replay will block until a matching API response is observed (responses from the triggering step count). Call AFTER you verified via browser_network_requests that the response actually occurred with the expected status/body. Use a path-only urlPattern regex and never literal volatile IDs (/api/orders/\\d+, not /api/orders/8842).",{urlPattern:$.string().describe("Regex matched against the response URL \u2014 path-only for portability"),method:$.string().optional().describe("HTTP method to match (any if omitted)"),status:$.string().optional().describe("Expected status \u2014 exact ('201') or class ('2xx'); default any 2xx"),bodyPattern:$.string().optional().describe(`Regex the response body must match, e.g. '"status"\\s*:\\s*"READY"' for polling waits`)},async t=>(e.addActions([{action:"waitForResponse",urlPattern:t.urlPattern,...t.method?{method:t.method}:{},...t.status?{status:t.status}:{},...t.bodyPattern?{pattern:t.bodyPattern}:{}}]),M("Network wait recorded."))),ge("saffron_assert","Record a verified assertion for the current Then step. Call ONLY after you have confirmed the condition holds via a page snapshot. If the condition does NOT hold, do not call this \u2014 call saffron_done with success=false instead. NEVER record volatile content (prices, dates, counters) as literal expected text \u2014 use kind=matches with a pattern, or kind=differs against captured values.",{kind:$.enum(["visible","notVisible","text","url","value","differs","matches","attribute","response"]),...un,expected:$.string().optional().describe("Expected text/value for kind=text|value \u2014 stable text only, never volatile values"),url:$.string().optional().describe("URL substring for kind=url (path only, no host)"),pattern:$.string().optional().describe("kind=matches|attribute: regex the target's text (or attribute) must match, e.g. 'NOK [\\\\d,]+' for a price"),attribute:$.string().optional().describe("kind=attribute: attribute name to check, e.g. 'href' for link-target assertions \u2014 use this for 'X should link to Y' instead of clicking through"),capture:$.string().optional().describe("kind=differs: name of the captured value on the LEFT of the comparison (omit when comparing the target element's live text)"),urlPattern:$.string().optional().describe("kind=response: regex matched against the response URL \u2014 path-only for portability, never literal volatile IDs (use /api/orders/\\d+)"),method:$.string().optional().describe("kind=response: HTTP method to match (any if omitted)"),status:$.string().optional().describe("kind=response: expected status \u2014 exact ('201') or class ('2xx'); default any 2xx"),bodyPattern:$.string().optional().describe("kind=response: regex the response body must match (verify via browser_network_requests first)"),compareTo:$.string().optional().describe("kind=differs: name of the captured value to compare against")},async t=>{let n=pn(t),r;switch(t.kind){case"visible":r={action:"expectVisible",target:n};break;case"notVisible":r={action:"expectNotVisible",target:n};break;case"text":r={action:"expectText",target:n,value:t.expected??""};break;case"value":r={action:"expectValue",target:n,value:t.expected??""};break;case"url":r={action:"expectUrl",url:t.url??""};break;case"differs":if(!t.compareTo)return M("ERROR: kind=differs requires compareTo.");if(!t.capture&&!n)return M("ERROR: kind=differs needs either capture (a stored name) or a target element.");r={action:"expectDiffers",target:n,value:t.capture,compareTo:t.compareTo};break;case"matches":if(!t.pattern)return M("ERROR: kind=matches requires pattern.");r={action:"expectMatches",target:n,pattern:t.pattern};break;case"attribute":if(!t.attribute||!t.pattern)return M("ERROR: kind=attribute requires attribute and pattern.");r={action:"expectAttribute",target:n,attribute:t.attribute,pattern:t.pattern};break;case"response":if(!t.urlPattern)return M("ERROR: kind=response requires urlPattern.");r={action:"expectResponse",urlPattern:t.urlPattern,...t.method?{method:t.method}:{},...t.status?{status:t.status}:{},...t.bodyPattern?{pattern:t.bodyPattern}:{}};break}return!["url","differs","response"].includes(t.kind)&&!n?M("ERROR: assertion needs a target (role/name, nameRegex, label, text, or selector)."):(e.addActions([r]),M("Assertion recorded."))}),ge("saffron_done","Finalize the run. Call exactly once, after the final step: success=true only if every step's intent was achieved and every assertion verified.",{success:$.boolean(),narrative:$.string().describe("2-4 sentence summary of the run: what happened, and why it failed if it did."),suggestedFeatureEdit:$.string().optional().describe("If you adapted any steps: the corrected Gherkin for those steps, as you would rewrite them in the .feature file (free text, for the human report)."),featureEdits:$.array($.object({index:$.number().int().min(0).describe("zero-based step index"),text:$.string().describe("replacement step text WITHOUT the Given/When/Then keyword")})).optional().describe("Machine-applicable version of the suggested edits: one entry per adapted step. Provide this whenever you adapted a step.")},async t=>(e.done=!0,e.success=t.success,e.narrative=t.narrative,e.suggestedFeatureEdit=t.suggestedFeatureEdit,e.featureEdits=t.featureEdits??[],M("Run finalized.")))]})}function gn(e={}){let t=e.deliberateSnapshots?`
|
|
34
|
+
Invoke it anywhere with: StepSet Name`,filterText:"step set stepset:",sortText:"0",insertTextFormat:Kt.Snippet,textEdit:{range:a,newText:"StepSet: ${1:Name}\n "}}];r.stepSets.length===0&&c.push({label:"StepSet \u2026",kind:be.Keyword,detail:"invoke a step set (none defined yet)",filterText:"step set stepset",sortText:"1",insertTextFormat:Kt.Snippet,textEdit:{range:a,newText:"StepSet ${1:Name}"}});for(let p of r.stepSets)c.push({label:`StepSet ${p.name}`,kind:be.Function,detail:`invoke step set \xB7 ${p.steps.length} step(s)`,documentation:p.steps.join(`
|
|
35
|
+
`),filterText:`step set ${p.name}`,sortText:`1${p.name}`,textEdit:{range:a,newText:`StepSet ${p.name}`}});return c}return[]}import hn from"node:fs";import ns from"node:net";import yn from"node:path";import{chromium as rs,firefox as ss,webkit as os}from"playwright";import rn from"node:path";var zr=/\{date([+-]\d+)?(?::([^}]+))?\}/g;function qr(e,t){let n=String(e.getFullYear()),r=String(e.getMonth()+1).padStart(2,"0"),s=String(e.getDate()).padStart(2,"0");return t.replaceAll("YYYY",n).replaceAll("MM",r).replaceAll("DD",s).replaceAll("M",String(e.getMonth()+1)).replaceAll("D",String(e.getDate()))}function nt(e,t=new Date){return e.replace(zr,(n,r,s)=>{let o=new Date(t);return o.setDate(o.getDate()+(r?parseInt(r,10):0)),qr(o,s??"YYYY-MM-DD")})}var Hr=/\b(\d{4})-(\d{2})-(\d{2})\b/g,Wr=400;function nn(e,t=new Date){let n=new Date(t.getFullYear(),t.getMonth(),t.getDate());return e.replace(Hr,(r,s,o,i)=>{let a=new Date(Number(s),Number(o)-1,Number(i));if(Number.isNaN(a.getTime()))return r;let c=Math.round((a.getTime()-n.getTime())/864e5);return Math.abs(c)>Wr?r:c===0?"{date}":`{date${c>0?"+":""}${c}}`})}import{createHash as Gr}from"node:crypto";async function ae(e){let t;try{t=await e.locator("body").ariaSnapshot({timeout:3e3})}catch{return"sha256:unavailable"}let n=t.replace(/\d+/g,"#").replace(/\s+/g," ").trim();return`sha256:${Gr("sha256").update(n).digest("hex")}`}var ot=e=>new Promise(t=>setTimeout(t,e)),st=class{entries=[];seq=0;prevStepSeq=0;curStepSeq=0;listener;context;attach(t,n){this.context=t,this.listener=r=>{let s={seq:++this.seq,url:r.url(),method:r.request().method(),status:r.status()};if(this.entries.push(s),n){let o=r.headers()["content-type"]??"";/json|text|xml/i.test(o)&&r.text().then(i=>{s.body=i.length>262144?i.slice(0,262144):i}).catch(()=>{})}},t.on("response",this.listener)}detach(){this.context&&this.listener&&this.context.off("response",this.listener)}markStep(){this.prevStepSeq=this.curStepSeq,this.curStepSeq=this.seq}sincePreviousStep(){return this.entries.filter(t=>t.seq>this.prevStepSeq)}all(){return this.entries}};function Jr(e,t){if(!t)return e>=200&&e<300;let n=/^([1-5])xx$/i.exec(t.trim());if(n){let r=Number(n[1])*100;return e>=r&&e<r+100}return e===Number(t.trim())}function sn(e,t,n,r,s){return!(!t.test(e.url)||n&&e.method.toUpperCase()!==n.toUpperCase()||!Jr(e.status,r)||s&&!s.test(e.body??""))}var Yr=(e,t)=>`response matching ${e.method??"any-method"} /${t}/ status ${e.status??"2xx"}${e.pattern?` body /${e.pattern}/`:""}`;function on(e,t,n,r=!1){let s=a=>Se(nt(ne(a,n))),o=[],i=a=>{if(t.role){let c=t.nameRegex?new RegExp(s(t.nameRegex),"i"):t.name?s(t.name):void 0;o.push(a.getByRole(t.role,{name:c,exact:!1}))}t.label&&o.push(a.getByLabel(s(t.label))),t.placeholder&&o.push(a.getByPlaceholder(s(t.placeholder))),t.text&&o.push(a.getByText(s(t.text))),t.testId&&o.push(a.getByTestId(s(t.testId)));for(let c of t.fallbackSelectors??[])o.push(a.locator(c))};if(t.frame)i(e.frameLocator(s(t.frame)));else if(i(e),r)for(let a of e.frames())a!==e.mainFrame()&&i(a);if(o.length===0)throw new Error("Action target has neither role nor fallback selectors");return o}async function K(e,t,n){let r=Date.now()+t,s;for(;Date.now()<r;){try{if(await e())return}catch(o){s=o}await ot(100)}throw new Error(`Timed out waiting for ${n}${s?` (${String(s)})`:""}`)}async function L(e,t){let n;for(let r of e)try{await t(r.first());return}catch(s){n=s}throw n}var rt=e=>e.replace(/\s+/g," ").trim();async function it(e,t,n,r,s,o=new Map,i){let a=u=>Se(nt(ne(u,n))),c=t.value!==void 0?a(t.value):void 0,p=t.url!==void 0?a(t.url):void 0,h=t.action==="expectVisible"||t.action==="expectText"||t.action==="expectValue"||t.action==="expectMatches"||t.action==="expectAttribute"||t.action==="captureText",d=t.target?on(e,t.target,n,h):[],l={timeout:r},m={timeout:Math.min(1e3,r)},g=u=>async()=>{for(let y of d)try{if(await u(y.first()))return!0}catch{}return!1};switch(t.action){case"goto":{if(!p)throw new Error("goto requires url");let u=/^https?:\/\//.test(p)?p:new URL(p,s).toString();await e.goto(u,l);return}case"click":return L(d,u=>u.click(l));case"fill":return L(d,u=>u.fill(c??"",l));case"press":return d.length>0?L(d,u=>u.press(c??"",l)):e.keyboard.press(c??"");case"selectOption":return L(d,async u=>{await u.selectOption(c??"",l)});case"check":return L(d,u=>u.check(l));case"uncheck":return L(d,u=>u.uncheck(l));case"hover":return L(d,u=>u.hover(l));case"waitFor":return d.length>0?L(d,u=>u.waitFor({state:"visible",timeout:r})):ot(Math.min(Number(c??"0"),r));case"expectVisible":return K(g(u=>u.isVisible()),r,"target to be visible");case"expectNotVisible":return L(d,u=>u.waitFor({state:"hidden",timeout:r}));case"expectText":return K(g(async u=>(await u.textContent(m)??"").includes(c??"")),r,`text "${c}"`);case"expectValue":return K(g(async u=>await u.inputValue(m)===(c??"")),r,`input value "${c}"`);case"expectUrl":{let u=p??c??"";return K(async()=>e.context().pages().some(y=>y.url().includes(u)),r,`URL containing "${u}" in any tab (current: ${e.context().pages().map(y=>y.url()).join(", ")})`)}case"handleDialog":{let u=(c??"accept")!=="dismiss",y=t.promptText?a(t.promptText):void 0;e.once("dialog",$=>{u?$.accept(y):$.dismiss()});return}case"uploadFile":{let u=(c??"").split(`
|
|
36
|
+
`).filter(Boolean).map(y=>rn.isAbsolute(y)?y:rn.resolve(y));if(u.length===0)throw new Error("uploadFile requires a path");if(d.length>0)return L(d,y=>y.setInputFiles(u,l));e.once("filechooser",y=>{y.setFiles(u)});return}case"dragTo":{if(!t.toTarget)throw new Error("dragTo requires toTarget");let u=on(e,t.toTarget,n);return L(d,y=>y.dragTo(u[0].first(),l))}case"captureText":{if(!t.saveAs)throw new Error("captureText requires saveAs");return K(g(async u=>{let y=rt(await u.textContent(m)??"");return y===""?!1:(o.set(t.saveAs,y),!0)}),r,`non-empty text to capture as "${t.saveAs}"`)}case"expectDiffers":{if(!t.compareTo)throw new Error("expectDiffers requires compareTo");let u=o.get(t.compareTo);if(u===void 0)throw new Error(`expectDiffers: no captured value named "${t.compareTo}"`);if(d.length>0)return K(g(async x=>rt(await x.textContent(m)??"")!==u),r,`target text to differ from "${t.compareTo}" (${u})`);let y=c??"",$=o.get(y);if($===void 0)throw new Error(`expectDiffers: no captured value named "${y}"`);if($===u)throw new Error(`expectDiffers: "${y}" (${$}) equals "${t.compareTo}" (${u})`);return}case"expectAttribute":{if(!t.attribute)throw new Error("expectAttribute requires attribute");if(!t.pattern)throw new Error("expectAttribute requires pattern");let u=new RegExp(t.pattern,"i");return K(g(async y=>u.test(await y.getAttribute(t.attribute,m)??"")),r,`attribute ${t.attribute} to match /${t.pattern}/i`)}case"expectMatches":{if(!t.pattern)throw new Error("expectMatches requires pattern");let u=new RegExp(t.pattern,"i");return K(g(async y=>u.test(rt(await y.textContent(m)??""))),r,`target text to match /${t.pattern}/i`)}case"waitForResponse":case"expectResponse":{if(!t.urlPattern)throw new Error(`${t.action} requires urlPattern`);let u=new RegExp(a(t.urlPattern)),y=t.pattern?new RegExp(a(t.pattern)):void 0;if(!i){await e.waitForResponse(x=>sn({seq:0,url:x.url(),method:x.request().method(),status:x.status()},u,t.method,t.status,void 0),l);return}let $=()=>t.action==="expectResponse"?i.all():i.sincePreviousStep();return K(async()=>$().some(x=>sn(x,u,t.method,t.status,y)),r,Yr(t,t.urlPattern))}default:throw new Error(`Unknown action type: ${t.action}`)}}function an(e,t){return e.actions.some(n=>[n.value,n.url,n.pattern,n.target?.name,n.target?.nameRegex,n.target?.label,n.target?.text,n.target?.placeholder].some(r=>r?.includes(t)))}function Kr(e,t){return t.steps.length!==e.steps.length?!1:t.steps.every((n,r)=>{let s=e.steps[r];if(n.gherkin!==s.text)return!1;let o=n.table,i=s.table;if(!!o!=!!i)return!1;if(o&&i){let a=Object.keys(U(o)),c=Object.keys(U(i));if(a.length>0&&c.length>0){if(JSON.stringify(a)!==JSON.stringify(c))return!1}else if(JSON.stringify(o)!==JSON.stringify(i))return!1}if(n.docString!==void 0!=(s.docString!==void 0))return!1;if(n.docString!==void 0&&s.docString!==void 0&&n.docString!==s.docString){let a=t.steps.some(p=>an(p,"<docstring>")),c=t.steps.some(p=>an(p,n.docString));if(!a||c)return!1}return!0})}function at(e,t){let n=e.steps[t],r=n?.resolvedTable??n?.table;return{...e.params,...le(e.steps),...U(r),...re(n?.resolvedDocString??n?.docString)}}async function ve(e,t,n,r={}){let s=r.actionTimeoutMs??5e3,o=r.retries??1,i=[],a=r.vars??new Map,c=()=>Object.fromEntries(a),p=n.steps.some(m=>m.actions.some(g=>(g.action==="waitForResponse"||g.action==="expectResponse")&&g.pattern!==void 0)),h=new st;h.attach(e.context(),p);let d=e,l=()=>{let m=e.context().pages(),g=m[m.length-1];g&&g!==d&&!g.isClosed()&&(d=g)};if(!Kr(t,n))return h.detach(),{status:"stale",stepResults:i,captured:c()};try{for(let m=0;m<n.steps.length;m++){let g=n.steps[m],u=at(t,m);h.markStep();let y=!1,$=0;for(let x=0;x<g.actions.length;x++){let f=g.actions[x];l(),f.pageFingerprint&&await ae(d)!==f.pageFingerprint&&(y=!0);let v,b=!1;for(let w=0;w<=o&&!b;w++){w>0&&($++,await ot(250*w),l());try{await it(d,f,u,s,r.baseURL,a,h),b=!0}catch(P){v=P}}if(!b){let w=v instanceof Error?v.message:String(v);i.push({gherkin:g.gherkin,fromStepSet:g.fromStepSet,kind:g.kind,status:"failed",drift:y,...$?{retries:$}:{},error:w});for(let P=m+1;P<n.steps.length;P++)i.push({gherkin:n.steps[P].gherkin,kind:n.steps[P].kind,status:"skipped"});return{status:"failed",failure:{stepIndex:m,actionIndex:x,stepText:g.gherkin,kind:g.kind,error:w},stepResults:i,captured:c()}}}i.push({gherkin:g.gherkin,fromStepSet:g.fromStepSet,kind:g.kind,status:"passed",drift:y,...$?{retries:$}:{}})}return{status:"passed",stepResults:i,captured:c()}}finally{h.detach()}}var cn=new Set(["expectVisible","expectNotVisible","expectText","expectUrl","expectValue","expectDiffers","expectMatches","expectAttribute","expectResponse"]);var Me=class{steps;currentStepIndex=-1;done=!1;success=!1;narrative="";adaptations=[];suggestedFeatureEdit;featureEdits=[];announced=new Set;pendingFingerprint;constructor(t){this.steps=t.steps.map(n=>({gherkin:n.text,keyword:n.keyword,kind:n.kind,table:n.table,docString:n.docString,fromStepSet:n.source?.fromStepSet,actions:[]}))}addArmedAction(t){if(this.done)return;let n=Math.max(this.currentStepIndex,0),r=this.steps[n];if(!r||r.actions.some(i=>i.action===t.action&&i.value===t.value&&i.promptText===t.promptText))return;let o=Math.max(r.actions.length-1,0);r.actions.splice(o,0,t)}addActions(t){if(this.done||t.length===0)return;let n=Math.max(this.currentStepIndex,0),r=this.steps[n];r&&(this.pendingFingerprint&&t[0]&&(t[0].pageFingerprint=this.pendingFingerprint,this.pendingFingerprint=void 0),r.actions.push(...t))}},ce=e=>e.replace(/\\(['"\\])/g,"$1");function Qr(e){let t=e.match(/'((?:[^'\\]|\\.)*)'/);return t?ce(t[1]):void 0}function ct(e){let t={},n=e.match(/(?:frameLocator\('((?:[^'\\]|\\.)*)'\)|locator\('((?:[^'\\]|\\.)*)'\)\.contentFrame\(\))/);n&&(t.frame=ce(n[1]??n[2]),e=e.slice((n.index??0)+n[0].length));let r=e.match(/getByRole\('([^']+)'(?:,\s*\{([^}]*)\})?\)/);if(r){t.role=r[1];let i=(r[2]??"").match(/name:\s*'((?:[^'\\]|\\.)*)'/);i&&(t.name=ce(i[1]))}let s=[[/getByLabel\('((?:[^'\\]|\\.)*)'\)/,"label"],[/getByText\('((?:[^'\\]|\\.)*)'\)/,"text"],[/getByTestId\('((?:[^'\\]|\\.)*)'\)/,"testId"],[/getByPlaceholder\('((?:[^'\\]|\\.)*)'\)/,"placeholder"]];for(let[i,a]of s){let c=e.match(i);c&&(t[a]=ce(c[1]))}let o=e.match(/(?:^|\.)locator\('((?:[^'\\]|\\.)*)'\)/);return o&&(t.fallbackSelectors=[ce(o[1])]),Object.keys(t).length>0?t:void 0}function Xr(e,t){if(t&&e.startsWith(t)){let n=e.slice(t.length);return n.startsWith("/")?n:`/${n}`}return e}var Zr={click:"click",dblclick:"click",fill:"fill",pressSequentially:"fill",type:"fill",press:"press",check:"check",uncheck:"uncheck",hover:"hover",selectOption:"selectOption"};function es(e,t){let n=e.match(/page\.goto\('((?:[^'\\]|\\.)*)'\)/);if(n)return{action:"goto",url:Xr(ce(n[1]),t)};let r=e.match(/page\.keyboard\.press\('((?:[^'\\]|\\.)*)'\)/);if(r)return{action:"press",value:ce(r[1])};let s=e.match(/await\s+page\.(.+)\.dragTo\(page\.(.+?)\);?\s*$/);if(s){let l=ct(s[1]),m=ct(s[2]);if(l&&m)return{action:"dragTo",target:l,toTarget:m}}let o=e.match(/await\s+page\.(.+)\.(click|dblclick|fill|pressSequentially|type|press|check|uncheck|hover|selectOption)\((.*)\);?\s*$/);if(!o)return;let[,i,a,c]=o,p=ct(i);if(!p)return;let h={action:Zr[a],target:p},d=Qr(c);return d!==void 0&&a!=="click"&&a!=="dblclick"&&(h.value=d),h}function ln(e,t){let n=[],r=Array.isArray(e)?e:e?.content;if(Array.isArray(r))for(let o of r){let i=o?.text;typeof i=="string"&&n.push(i)}else typeof e=="string"&&n.push(e);let s=[];for(let o of n)for(let i of o.split(`
|
|
37
|
+
`)){let a=es(i.trim(),t);a&&s.push(a)}return ts(s)}function ts(e){return e.filter((t,n)=>{let r=e[n+1];return!(t.action==="fill"&&(t.value??"")===""&&r?.action==="fill"&&JSON.stringify(r.target)===JSON.stringify(t.target))})}function dn(e){let t=[];for(let n of e)if(/"[^"]+"|<[^<>]+>/.test(n.gherkin)){for(let s of n.actions)if(s.action==="fill"&&(s.value??"")===""){t.push(`Step "${n.gherkin}" recorded a fill with an empty value although the step passes one; the typed value was lost. Re-record with: saffron run --rerecord`);break}}return t}function lt(e,t){let n=e;for(let[r,s]of Object.entries(t).filter(([,o])=>o.length>=3).sort((o,i)=>i[1].length-o[1].length))n=n.split(s).join(`<${r}>`);return n}function pn(e,t){let n=Object.entries(t).filter(([,i])=>i.length>=3);if(n.length===0)return e;let r=i=>i===void 0?void 0:lt(i,t),s=[...n].sort((i,a)=>a[1].length-i[1].length),o=i=>{if(i!==void 0){for(let[a,c]of s)if(i.includes(c))return`<${a}>`;return i}};return e.map(i=>({...i,actions:i.actions.map(a=>{let c=cn.has(a.action)&&a.action!=="expectValue";return{...a,value:c?o(a.value):r(a.value),url:r(a.url),target:a.target?{...a.target,name:c?o(a.target.name):r(a.target.name),label:r(a.target.label),text:c?o(a.target.text):r(a.target.text),placeholder:r(a.target.placeholder)}:void 0}})}))}function un(e,t){let n=le(t.steps);return e.map((r,s)=>{let o=t.steps[s],i=o?.resolvedTable??o?.table,a=o?.resolvedDocString??o?.docString,c=Object.entries({...n,...U(i),...re(a)}).filter(([,h])=>h.length>=3).sort((h,d)=>d[1].length-h[1].length);if(c.length===0)return r;let p=h=>{if(h===void 0)return;let d=h;for(let[l,m]of c)d=d.split(m).join(`<${l}>`);return d};return{...r,actions:r.actions.map(h=>({...h,value:p(h.value),url:p(h.url),target:h.target?{...h.target,name:p(h.target.name),label:p(h.target.label),text:p(h.target.text),placeholder:p(h.target.placeholder)}:void 0}))}})}function fn(e,t=new Date){let n=r=>r===void 0?void 0:nn(r,t);return e.map(r=>({...r,actions:r.actions.map(s=>({...s,value:n(s.value),url:n(s.url),target:s.target?{...s.target,name:n(s.target.name),nameRegex:n(s.target.nameRegex),label:n(s.target.label),text:n(s.target.text)}:void 0}))}))}function gn(e,t){if(t.size===0)return e;let n=r=>r===void 0?void 0:ue(r,t);return e.map(r=>({...r,actions:r.actions.map(s=>({...s,value:n(s.value),url:n(s.url),pattern:n(s.pattern),target:s.target?{...s.target,name:n(s.target.name),nameRegex:n(s.target.nameRegex),label:n(s.target.label),text:n(s.target.text),placeholder:n(s.target.placeholder)}:void 0}))}))}function mn(e,t){let n=t??{};if(e==="mcp__playwright__browser_handle_dialog"){let r={action:"handleDialog",value:n.accept?"accept":"dismiss"};return typeof n.promptText=="string"&&n.promptText&&(r.promptText=n.promptText),r}if(e==="mcp__playwright__browser_file_upload"){let r=Array.isArray(n.paths)?n.paths:[];if(r.length===0)return;let s=process.cwd()+"/";return{action:"uploadFile",value:r.map(i=>typeof i=="string"&&i.startsWith(s)?i.slice(s.length):i).join(`
|
|
38
|
+
`)}}}var ge={aiCalls:0,inputTokens:0,outputTokens:0,cacheReadTokens:0,cacheCreationTokens:0,costUsd:0};async function is(){return new Promise((e,t)=>{let n=ns.createServer();n.listen(0,"127.0.0.1",()=>{let{port:r}=n.address();n.close(()=>e(r))}),n.on("error",t)})}function we(e){let t=e.contexts().flatMap(n=>n.pages());return[...t].reverse().find(n=>n.url()!=="about:blank")??t.at(-1)}function vn(e){let t=e.length;for(let n=e.length-1;n>=0&&e[n].kind==="assertion";n--)t=n;return t}function Sn(e,t){return e[t]?.kind==="assertion"&&t>=vn(e)}function xn(e,t,n=!1,r){let s=vn(e);return e.map((o,i)=>{let a=t[i],c=a&&a.actions.length>0;return o.kind==="assertion"?n&&i<s&&c?a:o:c?r&&i===r.stepIndex&&r.actionIndex>0?{...a,actions:[...o.actions.slice(0,r.actionIndex),...a.actions]}:a:o})}function as(e,t){return e.map((n,r)=>{let s=t[r];return s&&s.actions.length>0?s:n})}function bn(e,t){return{aiCalls:e.aiCalls+t.aiCalls,inputTokens:e.inputTokens+t.inputTokens,outputTokens:e.outputTokens+t.outputTokens,cacheReadTokens:e.cacheReadTokens+t.cacheReadTokens,cacheCreationTokens:e.cacheCreationTokens+t.cacheCreationTokens,costUsd:(e.costUsd??0)+(t.costUsd??0),models:e.models||t.models?[...new Set([...e.models??[],...t.models??[]])]:void 0}}var Ne=class{constructor(t){this.options=t}options;browser;cdpEndpoint;stepIndex;get agent(){return this.options.agent}getStepIndex(){if(!this.stepIndex&&(this.stepIndex=W(this.options.projectRoot),this.stepIndex.steps.size>0)){let t=this.stepIndex.divergent.length>0?` \xB7 ${this.stepIndex.divergent.length} step(s) have divergent recordings (Level-2 signal)`:"";console.log(`step index: ${this.stepIndex.steps.size} reusable steps from ${this.stepIndex.scannedCaches} cache(s)${t}`)}return this.stepIndex}async recordWithSeeding(t,n){let r=this.getStepIndex(),s=t.steps.length,o={...t.params,...le(t.steps)},i=t.steps.map(x=>{let f=r.steps.size>0?At(r,x,o):void 0;return f?{gherkin:x.text,keyword:x.keyword,kind:x.kind,table:x.table,docString:x.docString,fromStepSet:x.source?.fromStepSet,actions:f.actions}:void 0}),a=t.steps.map(x=>({gherkin:x.text,keyword:x.keyword,kind:x.kind,table:x.table,docString:x.docString,fromStepSet:x.source?.fromStepSet,actions:[]})),c=new Map,p=[],h=[],d=[],l=[],m={...ge},g=0,u=0,y=0,$=x=>({success:x,narrative:h.join(" ")||"Composed entirely from seeded recordings.",adaptations:p,suggestedFeatureEdit:l.length>0?l.join(`
|
|
39
|
+
`):void 0,featureEdits:d,recordedSteps:a,announcedSteps:[],usage:m,seededCount:g});for(;y<s;){if(i[y]){let w=y;for(;w<s&&i[w];)w++;let P=i.slice(y,w),S={...t,steps:t.steps.slice(y,w)},k=await ve(n,S,this.buildCandidateCache(t,P),{baseURL:this.options.baseURL,actionTimeoutMs:this.options.actionTimeoutMs,retries:this.options.retries,vars:c}),A=k.status==="passed"?w-y:k.failure?.stepIndex??0;for(let O=0;O<A;O++)a[y+O]=P[O];if(g+=A,k.status==="passed"){y=w;continue}let j=y+A;p.push(`Seeded recording for "${t.steps[j].text}" did not replay in this scenario's context; the agent re-recorded it fresh. Divergence signal.`),i[j]=void 0,y=j;continue}if(!this.options.agent)return $(!1);u++;let x=y;for(;x<s&&!i[x];)x++;u>=3&&(x=s);let f=await this.withDialogsVisible(()=>this.options.agent.run({scenario:t,baseURL:this.options.baseURL,mode:"record",cdpEndpoint:this.cdpEndpoint,assertionPolicy:this.options.assertionPolicy,completedSteps:y>0?t.steps.slice(0,y).map(w=>w.resolvedText):void 0,recordOnly:x<s?{from:y,to:x-1}:void 0,takeFingerprint:async()=>ae(we(this.browser)??n)}));if(m=bn(m,f.usage),h.push(f.narrative),p.push(...f.adaptations),d.push(...f.featureEdits),f.suggestedFeatureEdit&&l.push(f.suggestedFeatureEdit),!f.success)return $(!1);let v=f.announcedSteps.filter(w=>w>=y&&w<s),b=Math.min(s-1,Math.max(v.length>0?Math.max(...v):x-1,y));for(let w=y;w<=b;w++)a[w]=f.recordedSteps[w];y=b+1}return $(!0)}async withDialogsVisible(t){let n=()=>{},r=[],s=[],o=i=>{i.on("dialog",n),r.push(i)};for(let i of this.browser?.contexts()??[]){i.on("page",o),s.push(i);for(let a of i.pages())o(a)}try{return await t()}finally{for(let i of s)i.off("page",o);for(let i of r)try{i.off("dialog",n)}catch{}}}async start(){let t=this.options.browser??"chromium";if(t==="chromium"){let n=await is();this.browser=await rs.launch({headless:!this.options.headed,args:[`--remote-debugging-port=${n}`]}),this.cdpEndpoint=`http://127.0.0.1:${n}`}else this.browser=await(t==="firefox"?ss:os).launch({headless:!this.options.headed}),this.cdpEndpoint=void 0}async stop(){await this.browser?.close(),this.browser=void 0}resolveStorageState(){if(!this.options.storageState)return;let t=yn.isAbsolute(this.options.storageState)?this.options.storageState:yn.join(this.options.projectRoot,this.options.storageState);if(!hn.existsSync(t))throw new Error(`storageState file not found: ${t}. Generate one with: npx playwright open --save-storage=${this.options.storageState} <url>`);return t}buildCandidateCache(t,n){return{version:1,feature:t.featurePath,scenario:t.name,recordedAt:new Date().toISOString(),recordedBy:"agent",steps:n}}async proofReplay(t,n){let r=await this.browser.newContext({storageState:this.resolveStorageState()}),s=await r.newPage();return{result:await ve(s,t,n,{baseURL:this.options.baseURL,actionTimeoutMs:this.options.actionTimeoutMs,retries:this.options.retries}),context:r,page:s}}async runScenario(t,n={}){let r=await this.runScenarioInner(t,n),s=xe(t);if(s.size>0){let o=i=>i===void 0?i:ue(i,s);r.error=o(r.error),r.narrative=o(r.narrative),r.proofError=o(r.proofError),r.suggestedFeatureEdit=o(r.suggestedFeatureEdit),r.adaptations=r.adaptations.map(i=>ue(i,s));for(let i of r.stepResults)i.error=o(i.error)}return r}async runScenarioInner(t,n={}){if(!this.browser)throw new Error("Orchestrator not started");let r=Date.now(),{projectRoot:s,baseURL:o,actionTimeoutMs:i,retries:a}=this.options,c=B(s,t.featurePath,t.name),p=z(c),h=ye(s,t.featurePath,t.name),d=!p&&hn.existsSync(h)?fe(h):void 0,l=await this.browser.newContext({storageState:this.resolveStorageState()}),m=await l.newPage();try{let g;if(d){let f=await ve(m,t,d.cache,{baseURL:o,actionTimeoutMs:i,retries:a}),v=d.meta.recordedFor??t.name;if(f.status==="passed")return{scenario:t,status:"yellow",source:"agent-record",stepResults:f.stepResults,narrative:`Replayed the pending proposal recorded for "${v}" at zero AI; accept it to go green.`,adaptations:[],verified:!0,proposalFile:h,usage:ge,durationMs:Date.now()-r};if(Object.keys(t.params).length>0&&v!==t.displayName){let w=f.failure;return{scenario:t,status:"red",source:"agent-record",stepResults:f.stepResults,adaptations:[],proposalFile:h,usage:ge,durationMs:Date.now()-r,error:`The pending proposal recorded for "${v}" does not replay for this Examples row`+(w?` (step "${w.stepText}": ${w.error.split(`
|
|
40
|
+
`)[0]})`:"")+". A row value is probably baked in as a literal instead of a <placeholder>; inspect the proposal, reject it, and re-record with saffron run --rerecord."}}await m.goto("about:blank")}if(p&&(g=await ve(m,t,p,{baseURL:o,actionTimeoutMs:i,retries:a}),g.status==="passed"))return{scenario:t,status:"green",source:"cache",stepResults:g.stepResults,adaptations:[],usage:ge,durationMs:Date.now()-r};if(!(this.options.agent!==void 0&&this.cdpEndpoint!==void 0&&!n.agentDisabled)){let f=this.options.agent!==void 0&&this.cdpEndpoint===void 0&&!n.agentDisabled?`recording/healing requires Chromium (the agent attaches over CDP). Record with --browser chromium (the default), then replay with --browser ${this.options.browser}.`:"no agent is configured.";return{scenario:t,status:"red",source:"cache",stepResults:g?.stepResults??[],adaptations:[],usage:ge,durationMs:Date.now()-r,error:p?g?.status==="stale"?`Cache is stale (feature file changed) and ${f}`:`Replay failed and ${f} ${g?.failure?.error??""}`.trimEnd():`No cache exists and ${f}`}}let y=p!==void 0&&g!==void 0&&g.status==="failed",$=0,x;try{if(y)x=await this.withDialogsVisible(()=>this.options.agent.run({scenario:t,baseURL:o,mode:"heal",model:this.options.healModel,cdpEndpoint:this.cdpEndpoint,assertionPolicy:this.options.assertionPolicy,completedSteps:g.stepResults.filter(f=>f.status==="passed").map(f=>f.gherkin),failedStep:{text:g.failure.stepText,kind:g.failure.kind,error:g.failure.error},takeFingerprint:async()=>{let f=we(this.browser)??m;return ae(f)}}));else if(await m.goto("about:blank"),this.options.reuseSteps!==!1){let f=await this.recordWithSeeding(t,m);$=f.seededCount,x=f}else x=await this.withDialogsVisible(()=>this.options.agent.run({scenario:t,baseURL:o,mode:"record",cdpEndpoint:this.cdpEndpoint,assertionPolicy:this.options.assertionPolicy,takeFingerprint:async()=>{let f=we(this.browser)??m;return ae(f)}}))}catch(f){let v=f instanceof Error?f.message:String(f);return{scenario:t,status:"red",source:y?"agent-heal":"agent-record",stepResults:g?.stepResults??[],adaptations:[],usage:ge,durationMs:Date.now()-r,error:`Agent invocation failed: ${v.split(`
|
|
41
|
+
`)[0]}`}}if(x.success&&y&&g.failure.kind==="assertion"&&(this.options.assertionPolicy!=="adaptable-mid"||Sn(p.steps,g.failure.stepIndex))){let f=p.steps[g.failure.stepIndex],v=we(this.browser)??m,b=new Map(Object.entries(g.captured)),w=at(t,g.failure.stepIndex);try{for(let P of f.actions)await it(v,P,w,this.options.actionTimeoutMs??5e3,o,b)}catch(P){let S=P instanceof Error?P.message:String(P);return{scenario:t,status:"red",source:"agent-heal",stepResults:g.stepResults,narrative:x.narrative,adaptations:x.adaptations,suggestedFeatureEdit:x.suggestedFeatureEdit,usage:x.usage,durationMs:Date.now()-r,error:`Assertion failed as written: "${f.gherkin}". Saffron never adapts assertions. If this change is intended, update the .feature file or re-record the scenario (saffron run --rerecord --filter @tag). If the recording itself looks wrong (an empty value, a wrong target), re-record it the same way. Verification error: ${S.split(`
|
|
42
|
+
`)[0]}`}}}return await this.finalizeAgentRun(t,x,y,p,r,$,y?g?.failure:void 0)}finally{await l.close()}}async finalizeAgentRun(t,n,r,s,o,i=0,a){let c=r?"agent-heal":"agent-record",p=n.recordedSteps.map(b=>({gherkin:b.gherkin,kind:b.kind,status:n.success?"passed":"failed"}));if(!n.success)return{scenario:t,status:"red",source:c,stepResults:p,narrative:n.narrative,adaptations:n.adaptations,usage:n.usage,durationMs:Date.now()-o,error:n.narrative};let h=r?xn(s.steps,n.recordedSteps,this.options.assertionPolicy==="adaptable-mid",a?{stepIndex:a.stepIndex,actionIndex:a.actionIndex}:void 0):n.recordedSteps,d=[...n.adaptations],l={...n.usage},m,g;if(this.options.verifyProposals!==!1){let b=await this.proofReplay(t,this.buildCandidateCache(t,h));try{if(b.result.status!=="passed"&&b.result.failure){let w=b.result.failure;if(this.options.agent&&!(r&&w.kind==="assertion"&&(this.options.assertionPolicy!=="adaptable-mid"||Sn(s.steps,w.stepIndex))))try{let S=await this.withDialogsVisible(()=>this.options.agent.run({scenario:t,baseURL:this.options.baseURL,mode:"refine",model:r?this.options.healModel:void 0,cdpEndpoint:this.cdpEndpoint,assertionPolicy:this.options.assertionPolicy,completedSteps:b.result.stepResults.filter(k=>k.status==="passed").map(k=>k.gherkin),failedStep:{text:w.stepText,kind:w.kind,error:w.error},takeFingerprint:async()=>ae(we(this.browser)??b.page)}));l=bn(l,S.usage),S.success&&(d.push(...S.adaptations),h=r?xn(h,S.recordedSteps,this.options.assertionPolicy==="adaptable-mid"):as(h,S.recordedSteps),await b.context.close(),b=await this.proofReplay(t,this.buildCandidateCache(t,h)))}catch{}}m=b.result.status==="passed",m||(g=b.result.failure?.error??`proof replay ${b.result.status}`)}finally{await b.context.close().catch(()=>{})}}let u=[...n.featureEdits];for(let b of Pt(h,W(this.options.projectRoot)))d.push(`duplicate wording: "${b.from}" recorded exactly the same actions as the existing step "${b.to}" (${b.feature} \u203A ${b.scenario}). Accept with --with-feature-edit to converge on the canonical wording.`),u.some(w=>w.index===b.index)||u.push({index:b.index,text:b.to});let y=xe(t),$=b=>y.size>0?ue(b,y):b;d=d.map($);let x={meta:{createdAt:new Date().toISOString(),mode:r?"heal":"record",narrative:$(n.narrative),adaptations:d,suggestedFeatureEdit:n.suggestedFeatureEdit?$(n.suggestedFeatureEdit):void 0,featureEdits:u.length>0?u:void 0,verified:m,proofError:g?$(g):void 0,seededSteps:i>0?i:void 0,recordedFor:t.displayName,usage:l},cache:{version:1,feature:t.featurePath,scenario:t.name,recordedAt:new Date().toISOString(),recordedBy:"agent",steps:h}},f=dn(h);f.length>0&&(m=!1,g=[g,...f].filter(Boolean).join(" "));let v=ye(this.options.projectRoot,t.featurePath,t.name);return Mt(v,x),{scenario:t,status:"yellow",source:c,stepResults:p,narrative:n.narrative,adaptations:d,suggestedFeatureEdit:n.suggestedFeatureEdit,featureEdits:u.length>0?u:void 0,verified:m,proofError:g,seededSteps:i>0?i:void 0,proposalFile:v,usage:l,durationMs:Date.now()-o}}};import An from"node:path";import{createRequire as fs}from"node:module";import{query as gs}from"@anthropic-ai/claude-agent-sdk";import{createSdkMcpServer as cs,tool as me}from"@anthropic-ai/claude-agent-sdk";import{z as R}from"zod";var M=e=>({content:[{type:"text",text:e}]});function wn(e){let t={};return e.role&&(t.role=e.role),e.name&&(t.name=e.name),e.nameRegex&&(t.nameRegex=e.nameRegex),e.label&&(t.label=e.label),e.text&&(t.text=e.text),e.selector&&(t.fallbackSelectors=[e.selector]),e.frame&&(t.frame=e.frame),Object.keys(t).length>0?t:void 0}var kn={role:R.string().optional().describe("ARIA role of the element"),name:R.string().optional().describe("Accessible name \u2014 use the SHORTEST stable substring; never include volatile content like prices or dates"),nameRegex:R.string().optional().describe("Regex for the accessible name when it embeds volatile content, e.g. 'Select \\\\{date\\\\+1\\\\} as checkin' cannot work \u2014 instead use nameRegex 'Select .* as checkin'"),label:R.string().optional().describe("Form-label text for getByLabel \u2014 the literal <label> text only, never a free-text description"),text:R.string().optional().describe("getByText locator text"),selector:R.string().optional().describe("CSS fallback selector"),frame:R.string().optional().describe(`CSS selector of the iframe hosting the element, e.g. 'iframe[title="Feedback widget"]' \u2014 REQUIRED whenever the element lives inside an iframe; without it replay looks only at the top-level page`)};function $n(e){return cs({name:"saffron",version:"0.1.0",tools:[me("saffron_step","Announce the Gherkin step you are about to perform. MUST be called before executing browser actions for each step, with the step's zero-based index and exact text.",{index:R.number().int().min(0),text:R.string(),adaptation:R.string().optional().describe("If the written step does not match the real UI and you are deviating from it, describe the deviation in one sentence.")},async t=>(e.currentStepIndex=t.index,e.announced.add(t.index),t.adaptation&&e.adaptations.push(`Step ${t.index+1} ("${t.text}"): ${t.adaptation}`),M(`Recording step ${t.index}: ${t.text}`))),me("saffron_capture",`Record a captureText action for the current step: at replay time, the target element's text is read and stored under saveAs for later expectDiffers comparisons. Use for steps like 'I record the displayed price as "X"'. The target must select exactly the element whose text is the value.`,{saveAs:R.string().describe("Name to store the captured text under"),...kn},async t=>{let n=wn(t);return n?(e.addActions([{action:"captureText",target:n,saveAs:t.saveAs}]),M(`Capture recorded as "${t.saveAs}".`)):M("ERROR: capture needs a target (role/name, nameRegex, label, text, or selector).")}),me("saffron_dialog","Record how a native dialog (alert/confirm/prompt) must be handled at replay. Call this IMMEDIATELY whenever an action triggers a native dialog \u2014 whether you handled it via browser_handle_dialog or the browser reported it as already handled. At replay Saffron arms this handler BEFORE the triggering action, so without this call the replay will auto-dismiss the dialog and diverge from what you saw.",{accept:R.boolean().describe("true = accept/OK the dialog, false = dismiss/cancel"),promptText:R.string().optional().describe("Text typed into a prompt() before accepting")},async t=>(e.addArmedAction({action:"handleDialog",value:t.accept?"accept":"dismiss",...t.promptText?{promptText:t.promptText}:{}}),M(`Dialog handling recorded (${t.accept?"accept":"dismiss"}).`))),me("saffron_wait_api","Record a network wait for the current step: replay will block until a matching API response is observed (responses from the triggering step count). Call AFTER you verified via browser_network_requests that the response actually occurred with the expected status/body. Use a path-only urlPattern regex and never literal volatile IDs (/api/orders/\\d+, not /api/orders/8842).",{urlPattern:R.string().describe("Regex matched against the response URL \u2014 path-only for portability"),method:R.string().optional().describe("HTTP method to match (any if omitted)"),status:R.string().optional().describe("Expected status \u2014 exact ('201') or class ('2xx'); default any 2xx"),bodyPattern:R.string().optional().describe(`Regex the response body must match, e.g. '"status"\\s*:\\s*"READY"' for polling waits`)},async t=>(e.addActions([{action:"waitForResponse",urlPattern:t.urlPattern,...t.method?{method:t.method}:{},...t.status?{status:t.status}:{},...t.bodyPattern?{pattern:t.bodyPattern}:{}}]),M("Network wait recorded."))),me("saffron_assert","Record a verified assertion for the current Then step. Call ONLY after you have confirmed the condition holds via a page snapshot. If the condition does NOT hold, do not call this \u2014 call saffron_done with success=false instead. NEVER record volatile content (prices, dates, counters) as literal expected text \u2014 use kind=matches with a pattern, or kind=differs against captured values.",{kind:R.enum(["visible","notVisible","text","url","value","differs","matches","attribute","response"]),...kn,expected:R.string().optional().describe("Expected text/value for kind=text|value \u2014 stable text only, never volatile values"),url:R.string().optional().describe("URL substring for kind=url (path only, no host)"),pattern:R.string().optional().describe("kind=matches|attribute: regex the target's text (or attribute) must match, e.g. 'NOK [\\\\d,]+' for a price"),attribute:R.string().optional().describe("kind=attribute: attribute name to check, e.g. 'href' for link-target assertions \u2014 use this for 'X should link to Y' instead of clicking through"),capture:R.string().optional().describe("kind=differs: name of the captured value on the LEFT of the comparison (omit when comparing the target element's live text)"),urlPattern:R.string().optional().describe("kind=response: regex matched against the response URL \u2014 path-only for portability, never literal volatile IDs (use /api/orders/\\d+)"),method:R.string().optional().describe("kind=response: HTTP method to match (any if omitted)"),status:R.string().optional().describe("kind=response: expected status \u2014 exact ('201') or class ('2xx'); default any 2xx"),bodyPattern:R.string().optional().describe("kind=response: regex the response body must match (verify via browser_network_requests first)"),compareTo:R.string().optional().describe("kind=differs: name of the captured value to compare against")},async t=>{let n=wn(t),r;switch(t.kind){case"visible":r={action:"expectVisible",target:n};break;case"notVisible":r={action:"expectNotVisible",target:n};break;case"text":r={action:"expectText",target:n,value:t.expected??""};break;case"value":r={action:"expectValue",target:n,value:t.expected??""};break;case"url":r={action:"expectUrl",url:t.url??""};break;case"differs":if(!t.compareTo)return M("ERROR: kind=differs requires compareTo.");if(!t.capture&&!n)return M("ERROR: kind=differs needs either capture (a stored name) or a target element.");r={action:"expectDiffers",target:n,value:t.capture,compareTo:t.compareTo};break;case"matches":if(!t.pattern)return M("ERROR: kind=matches requires pattern.");r={action:"expectMatches",target:n,pattern:t.pattern};break;case"attribute":if(!t.attribute||!t.pattern)return M("ERROR: kind=attribute requires attribute and pattern.");r={action:"expectAttribute",target:n,attribute:t.attribute,pattern:t.pattern};break;case"response":if(!t.urlPattern)return M("ERROR: kind=response requires urlPattern.");r={action:"expectResponse",urlPattern:t.urlPattern,...t.method?{method:t.method}:{},...t.status?{status:t.status}:{},...t.bodyPattern?{pattern:t.bodyPattern}:{}};break}return!["url","differs","response"].includes(t.kind)&&!n?M("ERROR: assertion needs a target (role/name, nameRegex, label, text, or selector)."):(e.addActions([r]),M("Assertion recorded."))}),me("saffron_done","Finalize the run. Call exactly once, after the final step: success=true only if every step's intent was achieved and every assertion verified.",{success:R.boolean(),narrative:R.string().describe("2-4 sentence summary of the run: what happened, and why it failed if it did."),suggestedFeatureEdit:R.string().optional().describe("If you adapted any steps: the corrected Gherkin for those steps, as you would rewrite them in the .feature file (free text, for the human report)."),featureEdits:R.array(R.object({index:R.number().int().min(0).describe("zero-based step index"),text:R.string().describe("replacement step text WITHOUT the Given/When/Then keyword")})).optional().describe("Machine-applicable version of the suggested edits: one entry per adapted step. Provide this whenever you adapted a step.")},async t=>(e.done=!0,e.success=t.success,e.narrative=t.narrative,e.suggestedFeatureEdit=t.suggestedFeatureEdit,e.featureEdits=t.featureEdits??[],M("Run finalized.")))]})}function Rn(e={}){let t=e.deliberateSnapshots?`
|
|
38
43
|
8. Tool responses do NOT include a page snapshot. Take snapshots deliberately with browser_snapshot, and only when you actually need to see the page: after navigation, before locating an element you haven't seen yet, and when verifying an assertion. Do NOT snapshot after actions whose outcome is predictable (filling a field you just located, pressing Enter, a click whose result you will verify later anyway). Every snapshot is expensive.
|
|
39
44
|
9. Minimize round-trips: batch tool calls in a single response whenever they don't depend on each other's results \u2014 saffron_step together with that step's browser actions, several fills in one turn, bookkeeping calls (saffron_capture, saffron_assert) alongside the actions they belong to. Only stop and wait when you genuinely need a result before deciding your next call. Do not write narration between tool calls.`:"";return`You are Saffron's test execution agent. You execute Gherkin scenarios in a real browser using the playwright MCP tools, while Saffron records your actions into a deterministic replay cache.
|
|
40
45
|
|
|
@@ -52,14 +57,14 @@ Non-negotiable rules:
|
|
|
52
57
|
5c. Real-world page furniture is supported: elements inside IFRAMES record and replay fine (interact normally) \u2014 but when a saffron_assert targets an element INSIDE an iframe, you MUST pass the iframe's CSS selector as the 'frame' argument (e.g. frame='iframe[title="Feedback widget"]'); replay cannot see into frames otherwise. File uploads: use browser_file_upload with the file path. Drag-and-drop: use browser_drag. All of these are recorded and replay deterministically.
|
|
53
58
|
5d. NATIVE DIALOGS (alert/confirm/prompt): when an action opens one, the page blocks and the snapshot reports a modal state \u2014 resolve it with browser_handle_dialog (accept true/false; promptText for prompt()). That call is recorded automatically as an armed handler replay places BEFORE the triggering action, so do NOT also call saffron_dialog. After handling, verify the post-dialog state as usual. If a step's wording says to confirm/accept, accept; if it says cancel/dismiss, dismiss.
|
|
54
59
|
5e. NETWORK WAITS: when a step says to wait for an API call ("wait for the order API to succeed", "until the job API reports READY"), do it verify-first: perform the trigger, check via browser_network_requests that the matching response actually occurred with the expected status/body, then record it with saffron_wait_api. For Then steps about API contracts ("the request should have returned 201") verify the same way and record with saffron_assert kind=response. urlPattern is a path-only regex \u2014 NEVER a literal volatile ID (/api/orders/\\d+, not /api/orders/8842) and never the host.
|
|
55
|
-
7. Never ask the user questions; you are unattended.${t}`}function
|
|
56
|
-
${
|
|
57
|
-
`)}`),p!==void 0&&(
|
|
60
|
+
7. Never ask the user questions; you are unattended.${t}`}function ls(e,t){if(e[t].kind!=="assertion")return"action";let n=e.length;for(let r=e.length-1;r>=0&&e[r].kind==="assertion";r--)n=r;return t>=n?"assertion:final":"assertion"}function Tn(e){let{scenario:t}=e,n=t.steps.map((s,o)=>{let i=d=>Se(d),a=`${o}. [${ls(t.steps,o)}] ${s.keyword} ${i(s.resolvedText)}`,c=s.resolvedTable??s.table,p=s.resolvedDocString??s.docString,h=a;return c&&(h+=`
|
|
61
|
+
${c.map(d=>` | ${d.map(i).join(" | ")} |`).join(`
|
|
62
|
+
`)}`),p!==void 0&&(h+=`
|
|
58
63
|
"""
|
|
59
64
|
${i(p).split(`
|
|
60
|
-
`).map(
|
|
65
|
+
`).map(d=>` ${d}`).join(`
|
|
61
66
|
`)}
|
|
62
|
-
"""`),
|
|
67
|
+
"""`),h}).join(`
|
|
63
68
|
`),r=[`Feature: ${t.featureName} (${t.featurePath})`,`Scenario: ${t.displayName}`,e.baseURL?`Base URL: ${e.baseURL}`:void 0,"","Steps (index. [kind] keyword text):",n,""];if(e.mode==="refine")r.push("MODE: REFINE. You (or a previous agent session) just recorded this scenario and it completed successfully \u2014 but the deterministic zero-AI replay of that recording FAILED at the step below. The application is working; the RECORDING is not replayable.","The browser tab shows the state where the proof replay stopped.","",`Steps that replayed fine:
|
|
64
69
|
${(e.completedSteps??[]).map(s=>` \u2713 ${s}`).join(`
|
|
65
70
|
`)||" (none)"}`,"",`Failed step: [${e.failedStep?.kind}] ${e.failedStep?.text}`,`Replay failure: ${e.failedStep?.error}`,"","Typical causes: a locator that only matched during your session (volatile accessible name \u2014 use nameRegex or a shorter stable name), a literal dynamic value baked into an assertion (use kind=matches/differs/captureText), or an assertion that cannot be checked without navigation (use kind=attribute for link targets).","Re-perform and re-record the failed step and every step after it in a REPLAYABLE form via saffron_step + browser tools / saffron_capture / saffron_assert. Express the SAME conditions the steps state \u2014 do not weaken them.","If a step genuinely cannot be expressed replayably, call saffron_done success=false and explain why.");else if(e.mode==="heal")r.push("MODE: HEAL. A cached deterministic replay of this scenario failed mid-run.","The browser tab already holds the page state reached so far \u2014 do NOT restart from step 0; verify where you are with a snapshot and continue from the failed step.","",`Steps already completed by replay:
|
|
@@ -68,62 +73,62 @@ ${(e.completedSteps??[]).map(s=>` \u2713 ${s}`).join(`
|
|
|
68
73
|
${e.completedSteps.map(i=>` \u2713 ${i}`).join(`
|
|
69
74
|
`)}`:"You are starting from a blank page.","",`Record ONLY steps ${s} through ${o} (inclusive).`,`CRITICAL STOP RULE: the steps AFTER step ${o} already have recordings and will be replayed automatically the moment you stop. After completing step ${o}, immediately call saffron_done \u2014 do NOT perform, verify, or navigate into any later step. Executing a later step yourself would make the automatic replay run it TWICE.`)}else if(e.completedSteps&&e.completedSteps.length>0){let s=e.completedSteps.length;r.push("MODE: RECORD (seeded). No cache exists for this scenario yet, but its first steps matched existing recordings and were ALREADY EXECUTED against the live browser by zero-AI replay:",e.completedSteps.map(o=>` \u2713 ${o}`).join(`
|
|
70
75
|
`),"",`The browser currently shows the state after those ${s} step(s) (verify with a snapshot if unsure). Do NOT redo them and do NOT navigate away from the current state.`,`Start with saffron_step for step index ${s} and record from there to the end.`)}else r.push("MODE: RECORD. No cache exists for this scenario yet.","Execute every step in order, starting at step 0. Navigate to the base URL first if the scenario implies a starting page.");return r.filter(s=>s!==void 0).join(`
|
|
71
|
-
`)}import{query as
|
|
72
|
-
`),a.set(
|
|
73
|
-
`));return o}import
|
|
74
|
-
<path d="${
|
|
75
|
-
<path d="${
|
|
76
|
-
</svg>`}function
|
|
76
|
+
`)}import{query as ds}from"@anthropic-ai/claude-agent-sdk";var ps="usage_EXPERIMENTAL_MAY_CHANGE_DO_NOT_RELY_ON_THIS_API_YET";async function dt(e,t=8e3){let n=e?.[ps];if(typeof n!="function")return;let r;try{let s=await Promise.race([n.call(e),new Promise(o=>{r=setTimeout(()=>o(void 0),t),r.unref?.()})]);return us(s)}catch{return}finally{r&&clearTimeout(r)}}function us(e){if(!e||!e.rate_limits_available||!e.subscription_type)return;let t=e.rate_limits?.five_hour;return{subscriptionType:e.subscription_type,utilization:typeof t?.utilization=="number"?t.utilization:void 0,resetsAt:t?.resets_at??void 0}}async function En(){let e=()=>{},t=new Promise(s=>e=s);async function*n(){await t}let r;try{let s=ds({prompt:n(),options:{maxTurns:1,tools:[],settingSources:[],persistSession:!1}});r=s;let o=(async()=>{try{for await(let a of s);}catch{}})(),i=await dt(s);return e(),r.close?.(),await o,i}catch{e(),r?.close?.();return}}var ms=fs(import.meta.url);function hs(){let e=ms.resolve("@playwright/mcp/package.json");return An.join(An.dirname(e),"cli.js")}var ys=new Set(["mcp__playwright__browser_navigate","mcp__playwright__browser_click","mcp__playwright__browser_type","mcp__playwright__browser_fill_form","mcp__playwright__browser_select_option","mcp__playwright__browser_press_key"]),Ue=class{constructor(t={}){this.opts=t}opts;planBefore;async planUsage(){if(!this.planBefore)return;let t=await this.planBefore;if(!t)return;let n=await En();return{subscriptionType:t.subscriptionType,fiveHourBefore:t.utilization,fiveHourAfter:n?.utilization,resetsAt:n?.resetsAt??t.resetsAt}}async run(t){let n=new Me(t.scenario),r={aiCalls:0,inputTokens:0,outputTokens:0,cacheReadTokens:0,cacheCreationTokens:0},s="",o=gs({prompt:Tn(t),options:{systemPrompt:Rn({deliberateSnapshots:(this.opts.snapshotMode??"none")==="none",adaptableMidAssertions:t.assertionPolicy==="adaptable-mid"}),model:t.model??this.opts.model,maxTurns:this.opts.maxTurns??100,tools:[],permissionMode:"bypassPermissions",allowDangerouslySkipPermissions:!0,persistSession:!1,settingSources:[],mcpServers:{playwright:{type:"stdio",command:process.execPath,args:[hs(),"--cdp-endpoint",t.cdpEndpoint,"--snapshot-mode",this.opts.snapshotMode??"none","--image-responses","omit"]},saffron:$n(n)},hooks:{PreToolUse:[{hooks:[async i=>{let a=i;if(ys.has(a.tool_name)&&t.takeFingerprint&&!n.pendingFingerprint)try{n.pendingFingerprint=await t.takeFingerprint()}catch{}return{continue:!0}}]}],PostToolUse:[{hooks:[async i=>{let a=i,c=mn(a.tool_name,a.tool_input);return c&&n.addArmedAction(c),a.tool_name.startsWith("mcp__playwright__")&&(n.addActions(ln(a.tool_response,t.baseURL)),n.pendingFingerprint=void 0),{continue:!0}}]}]}}});this.planBefore||(this.planBefore=dt(o));try{for await(let i of o)i.type==="result"&&(r.aiCalls=i.num_turns,r.costUsd=i.total_cost_usd,"modelUsage"in i&&i.modelUsage&&(r.models=Object.keys(i.modelUsage)),"usage"in i&&i.usage&&(r.inputTokens=i.usage.input_tokens??0,r.outputTokens=i.usage.output_tokens??0,r.cacheReadTokens=i.usage.cache_read_input_tokens??0,r.cacheCreationTokens=i.usage.cache_creation_input_tokens??0),i.subtype==="success"&&(s=i.result))}catch(i){let a=i instanceof Error?i.message:String(i);n.done=!0,n.success=!1,n.narrative=`Agent session error: ${a}`}return n.done||(n.success=!1,n.narrative=n.narrative||`Agent session ended without finalizing the run. Last output: ${s.slice(0,500)}`),{success:n.success,narrative:n.narrative,adaptations:n.adaptations,suggestedFeatureEdit:n.suggestedFeatureEdit,featureEdits:n.featureEdits.map(i=>({index:i.index,text:lt(i.text,t.scenario.params)})),recordedSteps:gn(fn(pn(un(n.steps,t.scenario),t.scenario.params)),xe(t.scenario)),announcedSteps:[...n.announced].sort((i,a)=>i-a),usage:r}}};function Le(e){return`$${e.toFixed(e>=1?2:4)}`}function pt(e,t){let n=e===void 0?"?":`${Math.round(e)}%`,r=t===void 0?"?":`${Math.round(t)}%`;return`${n} \u2192 ${r}`}function Pn(e){return e.plan?` \xB7 5-hour window ${pt(e.plan.fiveHourBefore,e.plan.fiveHourAfter)} (${e.plan.subscriptionType} plan) \xB7 \u2248 ${Le(e.costUsd)} at API rates`:` \xB7 ${Le(e.costUsd)}`}function Cn(e,t){return e?t?`\u2248 ${Le(e)}`:Le(e):""}function Fn(e){if(!e)return"";let t=new Date(e);return Number.isNaN(t.getTime())?"":`resets ${t.toLocaleTimeString([],{hour:"2-digit",minute:"2-digit"})}`}import jn from"node:fs";import Ve from"node:path";function Be(e){return JSON.stringify([e.role??null,e.name??null,e.nameRegex??null,e.label??null,e.text??null,e.testId??null,e.placeholder??null])}function In(e,t){let n=new Map,r=Math.min(e.steps.length,t.steps.length);for(let s=0;s<r;s++){let o=e.steps[s].actions,i=t.steps[s].actions;if(o.length===i.length)for(let a=0;a<o.length;a++){let c=o[a],p=i[a];if(c.action!==p.action||!c.target||!p.target)continue;let h=Be(c.target);h!==Be(p.target)&&n.set(h,{from:c.target,to:p.target})}}return[...n.values()]}function Ss(e){let t=Ve.join(e,".saffron","cache");if(!jn.existsSync(t))return[];let n=[];for(let r of jn.readdirSync(t,{recursive:!0})){let s=String(r);s.endsWith(".json")&&n.push(Ve.join(t,s))}return n}function _n(e,t,n,r){if(t.length===0)return[];let s=new Map(t.map(i=>[Be(i.from),i])),o=[];for(let i of Ss(e)){if(Ve.resolve(i)===Ve.resolve(n))continue;let a=z(i);if(!a)continue;let c=[],p=!1;for(let h of a.steps){let d=0;for(let l of h.actions){if(!l.target)continue;let m=s.get(Be(l.target));m&&(d++,r||(l.target=structuredClone(m.to),p=!0))}d>0&&c.push({gherkin:h.gherkin,count:d})}c.length>0&&(p&&pe(i,a),o.push({file:i,scenario:a.scenario,feature:a.feature,steps:c}))}return o}function ut(e){return e.role?`${e.role} "${e.name??e.nameRegex??""}"`:e.label?`label "${e.label}"`:e.text?`text "${e.text}"`:e.testId?`testId "${e.testId}"`:(e.fallbackSelectors??[]).join(",")||"(empty target)"}import On from"node:fs";import ft from"node:path";function Dn(e,t,n,r,s){let o={applied:[],skipped:[]},i=ee(ft.join(e,t),e,s).find(p=>p.name===n);if(!i){for(let p of r)o.skipped.push({index:p.index,reason:`scenario "${n}" not found in ${t}`});return o}let a=new Map,c=new Set;for(let p of r){let h=i.steps[p.index];if(!h?.source){o.skipped.push({index:p.index,reason:`no step at index ${p.index}`});continue}let{file:d,line:l,fromStepSet:m,fromBackground:g}=h.source;if(g){o.skipped.push({index:p.index,reason:`"${h.text}" is a Background step shared across scenarios; edit it manually`});continue}let u=`${d}:${l}`;if(c.has(u))continue;let y=a.get(d);y||(y=On.readFileSync(ft.join(e,d),"utf8").split(`
|
|
77
|
+
`),a.set(d,y));let $=y[l-1],x=$?.match(/^(\s*)(\S+)\s+(.*)$/);if(!x||x[3]!==h.text){o.skipped.push({index:p.index,reason:`${d}:${l} no longer matches the recorded step text`});continue}let f=`${x[1]}${x[2]} ${p.text}`;y[l-1]=f,c.add(u),o.applied.push({file:d,line:l,from:$.trim(),to:f.trim(),fromStepSet:m})}for(let[p,h]of a)o.applied.some(d=>d.file===p)&&On.writeFileSync(ft.join(e,p),h.join(`
|
|
78
|
+
`));return o}import mt from"node:fs";import ht from"node:path";var I=e=>e.replaceAll("&","&").replaceAll("<","<").replaceAll(">",">").replaceAll('"',"""),Nn="M11 14 L10 4.5 L17.5 9.5 L24 2.5 L30.5 9.5 L38 4.5 L37 14 Z",xs="M24.085 45.646L24.085 45.646Q20.719 45.646 18.407 44.575Q16.095 43.504 14.905 41.651Q13.715 39.798 13.715 37.384L13.715 37.384L19.121 37.384Q19.121 38.336 19.648 39.152Q20.175 39.968 21.263 40.461Q22.351 40.954 24.085 40.954L24.085 40.954Q25.649 40.954 26.703 40.546Q27.757 40.138 28.301 39.407Q28.845 38.676 28.845 37.724L28.845 37.724Q28.845 36.534 27.825 35.837Q26.805 35.140 24.527 34.936L24.527 34.936L22.657 34.766Q18.951 34.460 16.741 32.454Q14.531 30.448 14.531 27.150L14.531 27.150Q14.531 24.770 15.670 23.036Q16.809 21.302 18.849 20.367Q20.889 19.432 23.677 19.432L23.677 19.432Q26.601 19.432 28.692 20.435Q30.783 21.438 31.905 23.257Q33.027 25.076 33.027 27.524L33.027 27.524L27.587 27.524Q27.587 26.606 27.145 25.841Q26.703 25.076 25.836 24.600Q24.969 24.124 23.677 24.124L23.677 24.124Q22.453 24.124 21.620 24.532Q20.787 24.940 20.379 25.637Q19.971 26.334 19.971 27.150L19.971 27.150Q19.971 28.204 20.719 28.986Q21.467 29.768 23.167 29.904L23.167 29.904L25.071 30.074Q27.791 30.312 29.865 31.247Q31.939 32.182 33.112 33.797Q34.285 35.412 34.285 37.724L34.285 37.724Q34.285 40.104 33.044 41.889Q31.803 43.674 29.525 44.660Q27.247 45.646 24.085 45.646Z";function bs(e){return`<svg viewBox="0 0 48 48" width="${e}" height="${e}" aria-hidden="true">
|
|
79
|
+
<path d="${Nn}" fill="#F4A300"/>
|
|
80
|
+
<path d="${xs}" fill="#E7E9EC"/>
|
|
81
|
+
</svg>`}function vs(e){return`<svg viewBox="8 1 32 15" height="${e}" aria-hidden="true"><path d="${Nn}" fill="#F4A300"/></svg>`}var ws={green:"passed",yellow:"pending review",red:"failed"};function ze(e){return e>=1e3?`${(e/1e3).toFixed(1)}s`:`${e}ms`}function ke(e){return e>=1e6?`${(e/1e6).toFixed(1)}M`:e>=1e4?`${(e/1e3).toFixed(1)}k`:e.toLocaleString()}function Q(e,t,n,r="",s="",o=""){return`<div class="kpi">
|
|
77
82
|
<div class="kpi-label">${e}</div>
|
|
78
83
|
<div class="kpi-value ${r}">${t}</div>
|
|
79
84
|
<div class="kpi-sub ${s}">${n}</div>${o}
|
|
80
|
-
</div>`}function
|
|
81
|
-
${
|
|
82
|
-
${
|
|
83
|
-
${
|
|
84
|
-
${
|
|
85
|
-
${
|
|
86
|
-
${
|
|
87
|
-
${n.plan?
|
|
88
|
-
</section>`}function
|
|
89
|
-
`)}</section>`:""}function
|
|
85
|
+
</div>`}function gt(e,t){if(e.length<2)return"";let n=108,r=22,s=Math.min(...e),i=Math.max(...e)-s||1,a=e.map((c,p)=>`${(p/(e.length-1)*n).toFixed(1)},${(r-2-(c-s)/i*(r-4)).toFixed(1)}`).join(" ");return`<svg class="spark" viewBox="0 0 ${n} ${r}" width="${n}" height="${r}" aria-hidden="true"><polyline points="${a}" fill="none" stroke="${t}" stroke-width="1.8" stroke-linejoin="round" stroke-linecap="round"/></svg>`}var Mn=(e,t="",n=0)=>`${e>=0?"+":"\u2212"}${Math.abs(e).toFixed(n)}${t} vs prev`;function ks(e,t){let n=e.totals,r=e.trends,s=n.scenarios===0?0:Math.round((n.green+n.yellow)/n.scenarios*100),o=n.red>0?"red":n.yellow>0?"gold":"green",i=n.inputTokens+n.outputTokens,a=r?.prev?`${Mn(r.prev.passRatePp,"pp")} \xB7 ${n.green+n.yellow} of ${n.scenarios}`:`${n.green+n.yellow} of ${n.scenarios} scenarios`,c=r?.prev?`${Mn(r.prev.costUsd,"",4).replace("+","+$").replace("\u2212","\u2212$")} \xB7 ${ze(t)}`:`${ze(t)} total runtime`;return`<section class="kpis">
|
|
86
|
+
${Q("pass rate",`${s}<span class="unit">%</span>`,a,o,r?.prev?r.prev.passRatePp>=0?"green":"red":"",gt(r?.series.passRate??[],"var(--green)"))}
|
|
87
|
+
${Q("passed",String(n.green),"deterministic replay","green",n.green>0?"green":"")}
|
|
88
|
+
${Q("pending",String(n.yellow),n.yellow>0?"\u25B2 proposals to review":"nothing to review",n.yellow>0?"gold":"",n.yellow>0?"gold":"")}
|
|
89
|
+
${Q("failed",String(n.red),n.red>0?"\u25BC needs attention":"all clear",n.red>0?"red":"",n.red>0?"red":"green")}
|
|
90
|
+
${Q("ai calls",String(n.aiCalls),`${ke(i)} in+out tokens`,n.aiCalls>0?"":"green")}
|
|
91
|
+
${Q("cache traffic",ke(n.cacheReadTokens+n.cacheCreationTokens),n.cacheReadTokens+n.cacheCreationTokens>0?`${ke(n.cacheReadTokens)} read \xB7 ${ke(n.cacheCreationTokens)} written`:"zero (fully cached run)","",n.cacheReadTokens+n.cacheCreationTokens>0?"":"green")}
|
|
92
|
+
${n.plan?Q("plan usage",`${pt(n.plan.fiveHourBefore,n.plan.fiveHourAfter)}`,[`5-hour window \xB7 ${n.plan.subscriptionType} plan`,Fn(n.plan.resetsAt)].filter(Boolean).join(" \xB7 "),"gold","")+Q("api-equivalent",`<span class="unit">\u2248 $</span>${n.costUsd.toFixed(n.costUsd>=1?2:4)}`,`what this run costs on an API key \xB7 ${c}`,"","",gt(r?.series.costUsd??[],"var(--gold)")):Q("ai cost",`<span class="unit">$</span>${n.costUsd.toFixed(n.costUsd>=1?2:4)}`,c,"","",gt(r?.series.costUsd??[],"var(--gold)"))}
|
|
93
|
+
</section>`}function $s(e){let t=e.trends;if(!t)return"";let n=[];return t.chronic.length>0&&n.push(`<div class="panel accent-gold"><div class="panel-title gold">chronic scenarios: stop paying for heals</div><ul>${t.chronic.map(r=>`<li>${I(r.feature)} \u203A ${I(r.scenario)}: ${r.heals} heal(s), ${r.reds} red(s) in the last ${r.runs} run(s). Re-record it or fix the feature text.</li>`).join("")}</ul></div>`),t.divergentSteps>0&&n.push(`<div class="panel"><div class="panel-title">step divergence (level-2 signal)</div><p>${t.divergentSteps} step text(s) currently have divergent recordings across caches.</p></div>`),t.topIssues.length>0&&n.push(`<div class="panel accent-red"><div class="panel-title red">recurring failure themes (last 20 runs)</div><ul>${t.topIssues.map(r=>`<li>${r.count}\xD7: ${I(r.message)}</li>`).join("")}</ul></div>`),n.length>0?`<section class="trends">${n.join(`
|
|
94
|
+
`)}</section>`:""}function Rs(e){return`<ol class="steps">${e.steps.map(n=>{let r=n.status==="passed"?'<span class="dot green"></span>':n.status==="failed"?'<span class="dot red"></span>':'<span class="dot idle"></span>';return`<li class="step ${n.status}">
|
|
90
95
|
${r}
|
|
91
96
|
<div class="step-body">
|
|
92
|
-
<span class="step-text">${
|
|
97
|
+
<span class="step-text">${I(n.gherkin)}</span>
|
|
93
98
|
${n.kind==="assertion"?'<span class="tag">assert</span>':""}
|
|
94
|
-
${n.fromStepSet?`<span class="tag indigo" title="inlined from a StepSet definition: edit once, every invoking scenario follows">from StepSet: ${
|
|
99
|
+
${n.fromStepSet?`<span class="tag indigo" title="inlined from a StepSet definition: edit once, every invoking scenario follows">from StepSet: ${I(n.fromStepSet)}</span>`:""}
|
|
95
100
|
${n.drift?'<span class="tag gold" title="page fingerprint no longer matches the recording">drift</span>':""}
|
|
96
101
|
${n.retries?`<span class="tag gold" title="an action failed and was re-attempted with backoff before the step settled">retried \xD7${n.retries}</span>`:""}
|
|
97
|
-
${n.error?`<div class="step-error">${
|
|
102
|
+
${n.error?`<div class="step-error">${I(n.error)}</div>`:""}
|
|
98
103
|
</div>
|
|
99
104
|
</li>`}).join(`
|
|
100
|
-
`)}</ol>`}function
|
|
105
|
+
`)}</ol>`}function Ts(e){return e.featureEdits&&e.featureEdits.length>0?`<div class="panel">
|
|
101
106
|
<div class="panel-title gold">suggested .feature edit</div>
|
|
102
|
-
<div class="diff">${e.featureEdits.map(n=>{let r=e.steps[n.index]?.gherkin??`step ${n.index}`;return`<div class="diff-line del">\u2212 ${
|
|
107
|
+
<div class="diff">${e.featureEdits.map(n=>{let r=e.steps[n.index]?.gherkin??`step ${n.index}`;return`<div class="diff-line del">\u2212 ${I(r)}</div><div class="diff-line add">+ ${I(n.text)}</div>`}).join("")}</div>
|
|
103
108
|
<div class="panel-hint">apply with <code>saffron accept --with-feature-edit</code></div>
|
|
104
|
-
</div>`:e.suggestedFeatureEdit?`<div class="panel"><div class="panel-title gold">suggested .feature edit</div><pre>${
|
|
109
|
+
</div>`:e.suggestedFeatureEdit?`<div class="panel"><div class="panel-title gold">suggested .feature edit</div><pre>${I(e.suggestedFeatureEdit)}</pre></div>`:""}function Es(e,t){let n=e.usage.inputTokens+e.usage.outputTokens;if(e.usage.aiCalls===0)return'<span class="pill green">0 tokens</span>';let r=Cn(e.usage.costUsd,t),s=r?` \xB7 ${r}`:"";return`<span class="pill gold">${e.usage.aiCalls} ai \xB7 ${ke(n)} tok${s}</span>`}function As(e,t){return`<details class="card" ${e.status!=="green"?"open":""}>
|
|
105
110
|
<summary>
|
|
106
|
-
<span class="pill ${e.status==="green"?"green":e.status==="yellow"?"gold":"red"}">${
|
|
107
|
-
<span class="card-title">${
|
|
111
|
+
<span class="pill ${e.status==="green"?"green":e.status==="yellow"?"gold":"red"}">${ws[e.status]}</span>
|
|
112
|
+
<span class="card-title">${I(e.scenario)}</span>
|
|
108
113
|
<span class="card-meta">
|
|
109
|
-
<span class="m">${
|
|
114
|
+
<span class="m">${I(e.feature)}</span>
|
|
110
115
|
<span class="m-sep">\xB7</span>
|
|
111
|
-
<span class="m">${
|
|
116
|
+
<span class="m">${I(e.source)}</span>
|
|
112
117
|
<span class="m-sep">\xB7</span>
|
|
113
|
-
<span class="m">${
|
|
114
|
-
${
|
|
118
|
+
<span class="m">${ze(e.durationMs)}</span>
|
|
119
|
+
${Es(e,t)}
|
|
115
120
|
</span>
|
|
116
121
|
</summary>
|
|
117
122
|
<div class="card-body">
|
|
118
|
-
${
|
|
119
|
-
${e.narrative?`<div class="panel"><div class="panel-title">agent narrative</div><p>${
|
|
120
|
-
${e.adaptations.length>0?`<div class="panel accent-gold"><div class="panel-title gold">adaptations</div><ul>${e.adaptations.map(n=>`<li>${
|
|
121
|
-
${
|
|
122
|
-
${e.proposalFile?`<div class="panel"><div class="panel-title">cache proposal pending${e.verified===!0?' <span class="pill green">verified</span>':e.verified===!1?' <span class="pill red">unverified</span>':""}</div><p class="mono">${
|
|
123
|
+
${Rs(e)}
|
|
124
|
+
${e.narrative?`<div class="panel"><div class="panel-title">agent narrative</div><p>${I(e.narrative)}</p></div>`:""}
|
|
125
|
+
${e.adaptations.length>0?`<div class="panel accent-gold"><div class="panel-title gold">adaptations</div><ul>${e.adaptations.map(n=>`<li>${I(n)}</li>`).join("")}</ul></div>`:""}
|
|
126
|
+
${Ts(e)}
|
|
127
|
+
${e.proposalFile?`<div class="panel"><div class="panel-title">cache proposal pending${e.verified===!0?' <span class="pill green">verified</span>':e.verified===!1?' <span class="pill red">unverified</span>':""}</div><p class="mono">${I(e.proposalFile)}</p>${e.verified===!1&&e.proofError?`<p class="mono" style="color:var(--red)">proof replay failed: ${I(e.proofError.split(`
|
|
123
128
|
`)[0])}</p>`:""}<div class="panel-hint">review with <code>saffron accept</code> / <code>saffron reject</code></div></div>`:""}
|
|
124
|
-
${e.error&&e.status==="red"?`<div class="panel accent-red"><div class="panel-title red">failure</div><p>${
|
|
129
|
+
${e.error&&e.status==="red"?`<div class="panel accent-red"><div class="panel-title red">failure</div><p>${I(e.error)}</p></div>`:""}
|
|
125
130
|
</div>
|
|
126
|
-
</details>`}function
|
|
131
|
+
</details>`}function Un(e){let t=e.totals,n=new Date(e.startedAt),r=new Date(e.finishedAt).getTime()-n.getTime(),s=n.toLocaleString("en-GB",{day:"2-digit",month:"short",year:"numeric",hour:"2-digit",minute:"2-digit"});return`<!doctype html>
|
|
127
132
|
<html lang="en">
|
|
128
133
|
<head>
|
|
129
134
|
<meta charset="utf-8"/>
|
|
@@ -243,62 +248,57 @@ ${e.completedSteps.map(i=>` \u2713 ${i}`).join(`
|
|
|
243
248
|
<div class="wrap">
|
|
244
249
|
<header class="top">
|
|
245
250
|
<div class="brand">
|
|
246
|
-
${
|
|
251
|
+
${bs(38)}
|
|
247
252
|
<div>
|
|
248
253
|
<div class="brand-name">saffron</div>
|
|
249
254
|
<div class="brand-sub">test run report</div>
|
|
250
255
|
</div>
|
|
251
256
|
</div>
|
|
252
257
|
<div class="run-meta">
|
|
253
|
-
<span>${
|
|
258
|
+
<span>${I(s)}</span>
|
|
254
259
|
<span>\xB7</span>
|
|
255
|
-
<span>${
|
|
256
|
-
${e.baseURL?`<span class="chip base">${
|
|
260
|
+
<span>${ze(r)}</span>
|
|
261
|
+
${e.baseURL?`<span class="chip base">${I(e.baseURL)}</span>`:""}
|
|
257
262
|
</div>
|
|
258
263
|
</header>
|
|
259
264
|
|
|
260
|
-
${
|
|
261
|
-
${
|
|
265
|
+
${ks(e,r)}
|
|
266
|
+
${$s(e)}
|
|
262
267
|
|
|
263
268
|
<div class="section-head">
|
|
264
269
|
<span class="section-title">scenarios \xB7 ${t.scenarios}</span>
|
|
265
|
-
<span class="section-title">saffron v${
|
|
270
|
+
<span class="section-title">saffron v${I(e.version)}</span>
|
|
266
271
|
</div>
|
|
267
|
-
${e.scenarios.map(o=>
|
|
272
|
+
${e.scenarios.map(o=>As(o,!!e.totals.plan)).join(`
|
|
268
273
|
`)}
|
|
269
274
|
|
|
270
|
-
<footer>${
|
|
275
|
+
<footer>${vs(11)} generated by saffron v${I(e.version)} \xB7 ${I(new Date(e.finishedAt).toLocaleString("en-GB",{dateStyle:"medium",timeStyle:"medium"}))}</footer>
|
|
271
276
|
</div>
|
|
272
277
|
</body>
|
|
273
278
|
</html>
|
|
274
|
-
`}function
|
|
275
|
-
`),
|
|
276
|
-
`)
|
|
277
|
-
`)}function Ve(e,t=50){let n=Mn(e);if(!Le.existsSync(n))return[];let r=Le.readFileSync(n,"utf8").trim().split(`
|
|
278
|
-
`).filter(Boolean),s=[];for(let o of r.slice(-t))try{s.push(JSON.parse(o))}catch{}return s}var ft=e=>{let t=e.green+e.yellow+e.red;return t===0?0:Math.round((e.green+e.yellow)/t*100)};function Ln(e,t){let n=[...e,t],r=n.slice(-20),s=n.slice(-10),o=e.at(-1),i=o?{passRatePp:ft(t)-ft(o),costUsd:t.costUsd-o.costUsd,aiCalls:t.aiCalls-o.aiCalls,red:t.red-o.red}:void 0,a=new Map;for(let c of s)for(let l of c.scenarios){let u=`${l.feature}::${l.scenario}`,S=a.get(u)??{feature:l.feature,scenario:l.scenario,heals:0,reds:0,runs:0};S.runs++,l.source==="agent-heal"&&l.status==="yellow"&&S.heals++,l.status==="red"&&S.reds++,a.set(u,S)}let d=[...a.values()].filter(c=>c.heals>=2||c.reds>=3).sort((c,l)=>l.heals+l.reds-(c.heals+c.reds)),p=new Map;for(let c of r)for(let l of c.scenarios)l.status==="red"&&l.error&&p.set(l.error,(p.get(l.error)??0)+1);let m=[...p.entries()].sort((c,l)=>l[1]-c[1]).slice(0,3).map(([c,l])=>({message:c,count:l}));return{prev:i,series:{passRate:r.map(ft),costUsd:r.map(c=>Number(c.costUsd.toFixed(4))),aiCalls:r.map(c=>c.aiCalls)},chronic:d,divergentSteps:t.divergentSteps,topIssues:m}}var ws=/^\s*(?:Scenario(?: Outline| Template)?|Example):\s*(.+?)\s*$/,ks=/^\s*StepSet:/;function gt(e,t){return K.relative(e,t).split(K.sep).join("/")}function $s(e,t){let n={green:0,yellow:1,red:2};return!e||n[t]>n[e]?t:e}function Vn(e,t){return`${e} :: ${t}`}function zn(e,t,n={}){let r=t.features??"features",s=K.isAbsolute(r)?r:K.join(e,r),o=K.join(e,".saffron","reports","latest.json"),i=K.join(e,".saffron","reports","latest.html"),a;try{a=JSON.parse(J.readFileSync(o,"utf8"))}catch{a=void 0}let d=new Map;for(let g of a?.scenarios??[]){let k=g.scenario.replace(/ \(example \d+\)$/,""),v=Vn(g.feature,k);d.set(v,$s(d.get(v),g.status))}let p=new Map,m;try{p=X(ce(s),e)}catch(g){m=g instanceof Error?g.message:String(g)}let c=J.existsSync(s)?[...J.readdirSync(s,{recursive:!0})].map(String).filter(re).map(g=>K.join(s,g)).sort():[],l=[],u=new Map;for(let g of c){let k=gt(e,g),v="";try{v=J.readFileSync(g,"utf8")}catch(P){l.push({path:k,name:k,scenarios:[],stepSets:0,error:String(P)});continue}let b=v.split(`
|
|
279
|
-
`),C=new Map,x=0;b.forEach((P,Q)=>{let V=P.match(ws);V&&!C.has(V[1])?C.set(V[1],Q+1):ks.test(P)&&x++});let R=b.find(P=>/^\s*Feature:/.test(P)),E=R?R.replace(/^\s*Feature:\s*/,"").trim():k,I;try{I=Z(g,e,p)}catch(P){l.push({path:k,name:E,scenarios:[],stepSets:x,error:m??(P instanceof Error?P.message:String(P))});continue}let O=new Map;for(let P of I){let Q=O.get(P.name);if(Q){Q.rows+=1;continue}O.set(P.name,{name:P.name,line:C.get(P.name),tags:[...P.tags],outline:Object.keys(P.params).length>0,rows:1,cached:J.existsSync(B(e,P.featurePath,P.name)),proposal:J.existsSync(Se(e,P.featurePath,P.name)),lastStatus:d.get(Vn(P.featurePath,P.name))});for(let V of P.tags)u.set(V,(u.get(V)??0)+1)}l.push({path:k,name:E,scenarios:[...O.values()],stepSets:x})}let S=se(e).map(({file:g,proposal:k})=>({file:gt(e,g),feature:k.cache.feature,scenario:k.cache.scenario,mode:k.meta.mode,createdAt:k.meta.createdAt,verified:k.meta.verified,proofError:k.meta.proofError,adaptations:k.meta.adaptations??[],narrative:k.meta.narrative,suggestedFeatureEdit:k.meta.suggestedFeatureEdit,recordedFor:k.meta.recordedFor,aiCalls:k.meta.usage?.aiCalls??0,costUsd:k.meta.usage?.costUsd})).sort((g,k)=>g.file.localeCompare(k.file)),f;try{f=H(e,r,p)}catch{f={steps:[],stepSets:[],scannedFiles:0,scannedCaches:0}}let h=Ts(e),w=Ve(e,n.historyLimit??30).map(g=>({startedAt:g.startedAt,green:g.green,yellow:g.yellow,red:g.red,aiCalls:g.aiCalls,costUsd:g.costUsd})),y=K.join(e,"saffron.config.json");return{tool:"saffron",version:me(),projectRoot:e,packageInstalled:J.existsSync(K.join(e,"node_modules","saffron-ai")),config:{file:J.existsSync(y)?"saffron.config.json":void 0,effective:On(t)},features:l,tags:[...u.entries()].map(([g,k])=>({tag:g,scenarios:k})).sort((g,k)=>g.tag.localeCompare(k.tag)),proposals:S,lastRun:a?{startedAt:a.startedAt,finishedAt:a.finishedAt,totals:a.totals,reportHtml:J.existsSync(i)?gt(e,i):void 0}:void 0,history:w,vocabulary:{steps:f.steps.length,recorded:f.steps.filter(g=>g.status==="recorded").length,unrecorded:f.steps.filter(g=>g.status==="unrecorded").length,stepSets:f.stepSets.length,divergent:f.steps.filter(g=>g.status==="divergent").map(g=>g.text),duplicateWordings:h.slice(0,50)}}}function qn(e){let t=e.features.reduce((s,o)=>s+o.scenarios.length,0),n=e.features.reduce((s,o)=>s+o.scenarios.filter(i=>i.cached).length,0);console.log(`saffron ${e.version} \xB7 ${e.features.length} feature file(s) \xB7 ${t} scenario(s), ${n} cached`);for(let s of e.features){let o=s.scenarios.map(i=>i.proposal?"\u25D0":i.cached?"\u25CF":"\u25CB").join("");console.log(` ${s.path} ${o}${s.error?` \u2717 ${s.error.split(`
|
|
280
|
-
`)[0]}`:""}`)}if(e.tags.length&&console.log(`tags: ${e.tags.map(s=>`${s.tag} (${s.scenarios})`).join(", ")}`),console.log(e.proposals.length?`${e.proposals.length} proposal(s) pending review: ${e.proposals.map(s=>s.file).join(", ")}`:"no proposals pending review"),e.lastRun){let s=e.lastRun.totals;console.log(`last run ${e.lastRun.finishedAt}: ${s.green} passed \xB7 ${s.yellow} pending review \xB7 ${s.red} failed \xB7 $${s.costUsd.toFixed(2)}`)}let r=e.vocabulary;console.log(`vocabulary: ${r.steps} steps (${r.recorded} recorded, ${r.unrecorded} unrecorded), ${r.stepSets} step sets, ${r.divergent.length} divergent, ${r.duplicateWordings.length} duplicate wording group(s)`),console.log(`config: ${e.config.file??"(defaults, no saffron.config.json)"}`)}var Rs=/^(<[^>]+>|\{env:[^}]+\}|\{date[^}]*\})$/;function Bn(e){return e?e.role?`${e.role}${e.name?` "${e.name}"`:e.nameRegex?` /${e.nameRegex}/`:""}`:e.label?`label "${e.label}"`:e.text?`text "${e.text}"`:e.testId?`testid ${e.testId}`:e.placeholder?`placeholder "${e.placeholder}"`:e.fallbackSelectors?.length?e.fallbackSelectors[0]:"":""}function Ts(e){let t;try{t=q(e)}catch{return[]}let n=new Map;for(let r of t.steps.values()){if(r.actions.length===0)continue;let s=Ae(r.gherkin).pattern,o=[...r.gherkin.matchAll(/"([^"]*)"/g)].map(c=>c[1]),i=c=>c!==void 0&&(o.includes(c)||Rs.test(c))?"\xABarg\xBB":c,a=r.actions.map(c=>{let{pageFingerprint:l,value:u,promptText:S,...f}=c;return{...f,value:i(u),promptText:i(S)}}),d=JSON.stringify(a),p=r.actions.map(c=>`${c.action}${Bn(c.target)?` ${Bn(c.target)}`:c.url?` ${c.url}`:""}`).join(" \xB7 "),m=n.get(d)??{steps:new Map,actions:p};m.steps.has(s)||m.steps.set(s,r.gherkin),n.set(d,m)}return[...n.values()].filter(r=>r.steps.size>1).map(r=>({steps:[...r.steps.values()].sort(),actions:r.actions})).sort((r,s)=>s.steps.length-r.steps.length||r.steps[0].localeCompare(s.steps[0]))}import T from"node:fs";import F from"node:path";var $e={claude:".claude/skills",agents:".agents/skills",copilot:".github/skills",cursor:".cursor/skills"},As=["claude","agents"],Gn="<!-- saffron:start -->",ht="<!-- saffron:end -->";function Es(){return[Gn,"## Saffron end-to-end tests","","UI tests are plain-Gherkin `.feature` / `.saffron` files under `features/`,","run by Saffron (`npx saffron run`). No step definitions exist: an AI agent","records each scenario once; later runs replay at zero tokens.","","- Load the `saffron` skill before writing or editing scenarios.","- Exact step text is the cache identity: run `npx saffron steps` and reuse"," recorded wordings instead of inventing synonyms.","- `Then` steps are never healed: write them as the verdict, last.","- Never edit `.saffron/cache/` or `.saffron/proposals/` by hand; change the"," scenario text and re-run. Review proposals with `npx saffron accept`.","- Secrets go in `{env:VAR}` tokens, never in files.",ht,""].join(`
|
|
281
|
-
`)}function Hn(e,t){if(!T.existsSync(e))return T.writeFileSync(e,t),"created";let n=T.readFileSync(e,"utf8"),r=n.indexOf(Gn),s=n.indexOf(ht);if(r!==-1&&s!==-1&&s>r){let i=n.slice(0,r)+t.trimEnd()+n.slice(s+ht.length);return i===n?"unchanged":(T.writeFileSync(e,i),"updated")}let o=n.endsWith(`
|
|
279
|
+
`}function Ln(e,t,n){let r={scenarios:e.length,green:0,yellow:0,red:0,aiCalls:0,inputTokens:0,outputTokens:0,cacheReadTokens:0,cacheCreationTokens:0,costUsd:0},s=new Set;for(let o of e){for(let i of o.usage.models??[])s.add(i);r[o.status]+=1,r.aiCalls+=o.usage.aiCalls,r.inputTokens+=o.usage.inputTokens,r.outputTokens+=o.usage.outputTokens,r.cacheReadTokens+=o.usage.cacheReadTokens,r.cacheCreationTokens+=o.usage.cacheCreationTokens??0,r.costUsd+=o.usage.costUsd??0}return s.size>0&&(r.models=[...s].sort()),{tool:"saffron",version:te(),startedAt:t.toISOString(),finishedAt:new Date().toISOString(),baseURL:n,totals:r,scenarios:e.map(o=>({feature:o.scenario.featurePath,scenario:o.scenario.displayName,tags:[...o.scenario.tags],status:o.status,source:o.source,durationMs:o.durationMs,usage:o.usage,steps:o.stepResults,narrative:o.narrative,adaptations:o.adaptations,suggestedFeatureEdit:o.suggestedFeatureEdit,featureEdits:o.featureEdits,verified:o.verified,proofError:o.proofError,proposalFile:o.proposalFile,error:o.error}))}}function Vn(e,t){let n=ht.join(e,".saffron","reports");mt.mkdirSync(n,{recursive:!0});let r=ht.join(n,"latest.json"),s=ht.join(n,"latest.html");return mt.writeFileSync(r,`${JSON.stringify(t,null,2)}
|
|
280
|
+
`),mt.writeFileSync(s,Un(t)),{json:r,html:s}}function Bn(e){let t=e===void 0?[]:Array.isArray(e)?e:[e],n=new Set;for(let r of t)for(let s of r.split(/[\s,|]+/)){let o=s.trim();o&&n.add(o.startsWith("@")?o:`@${o}`)}return[...n]}function zn(e,t){return t.length===0?e:e.filter(n=>n.tags.some(r=>t.includes(r)))}function qn(e,t){return[...t??[],e]}import T from"node:fs";import F from"node:path";var $e={claude:".claude/skills",agents:".agents/skills",copilot:".github/skills",cursor:".cursor/skills"},Ps=["claude","agents"],Gn="<!-- saffron:start -->",St="<!-- saffron:end -->";function Cs(){return[Gn,"## Saffron end-to-end tests","","UI tests are plain-Gherkin `.feature` / `.saffron` files under `features/`,","run by Saffron (`npx saffron run`). No step definitions exist: an AI agent","records each scenario once; later runs replay at zero tokens.","","- Load the `saffron` skill before writing or editing scenarios.","- Exact step text is the cache identity: run `npx saffron steps` and reuse"," recorded wordings instead of inventing synonyms.","- `Then` steps are never healed: write them as the verdict, last.","- `npx saffron status` shows what is cached, what awaits review, the tags"," and vocabulary health; editing a cached scenario's text re-records it.","- Never edit `.saffron/cache/` or `.saffron/proposals/` by hand; change the"," scenario text and re-run. Review proposals with `npx saffron accept`"," (`--all`, or chosen files) and `npx saffron reject`.","- Secrets go in `{env:VAR}` tokens, never in files.",St,""].join(`
|
|
281
|
+
`)}function Hn(e,t){if(!T.existsSync(e))return T.writeFileSync(e,t),"created";let n=T.readFileSync(e,"utf8"),r=n.indexOf(Gn),s=n.indexOf(St);if(r!==-1&&s!==-1&&s>r){let i=n.slice(0,r)+t.trimEnd()+n.slice(s+St.length);return i===n?"unchanged":(T.writeFileSync(e,i),"updated")}let o=n.endsWith(`
|
|
282
282
|
`)?`
|
|
283
283
|
`:`
|
|
284
284
|
|
|
285
|
-
`;return T.writeFileSync(e,n+o+t),"updated"}var
|
|
286
|
-
`),s?"updated":"created")}function
|
|
287
|
-
`),n.scaffolded.push("saffron.config.json"));let o=F.join(e.projectRoot,"features");T.existsSync(o)||(T.mkdirSync(o,{recursive:!0}),n.scaffolded.push("features/"));let i=F.join(e.projectRoot,".gitignore"),a=e.examples?[".saffron/reports/",".env"]:[".saffron/reports/"],
|
|
288
|
-
`).some(
|
|
285
|
+
`;return T.writeFileSync(e,n+o+t),"updated"}var yt={command:"npx",args:["saffron","mcp"]},Fs={claude:{file:".mcp.json",key:"mcpServers",entry:yt},cursor:{file:".cursor/mcp.json",key:"mcpServers",entry:yt},copilot:{file:".vscode/mcp.json",key:"servers",entry:{type:"stdio",...yt}}},Wn={agents:"codex mcp add saffron -- npx saffron mcp (Codex; other hosts: see their MCP settings)"};function js(e,t){let n=F.join(e,t.file),r={},s=T.existsSync(n);if(s)try{r=JSON.parse(T.readFileSync(n,"utf8"))}catch{throw new Error(`${t.file} is not valid JSON: fix it or remove it, then re-run`)}let o=r[t.key]??{};return JSON.stringify(o.saffron)===JSON.stringify(t.entry)?"unchanged":(o.saffron=t.entry,r[t.key]=o,T.mkdirSync(F.dirname(n),{recursive:!0}),T.writeFileSync(n,JSON.stringify(r,null,2)+`
|
|
286
|
+
`),s?"updated":"created")}function Jn(e,t){T.mkdirSync(t,{recursive:!0});for(let n of T.readdirSync(e,{withFileTypes:!0})){let r=F.join(e,n.name),s=F.join(t,n.name);n.isDirectory()?Jn(r,s):T.copyFileSync(r,s)}}function Yn(e){let t=F.join(e.packageRoot,"skills","saffron");if(!T.existsSync(F.join(t,"SKILL.md")))throw new Error(`bundled skill not found at ${t}`);let n={skillDirs:[],instructionFiles:[],scaffolded:[],mcpFiles:[],mcpHints:[]},r=e.targets??Ps;for(let s of r){let o=F.join(e.projectRoot,$e[s],"saffron");T.existsSync(o)&&T.rmSync(o,{recursive:!0,force:!0}),Jn(t,o),n.skillDirs.push(F.relative(e.projectRoot,o))}if(e.mcp!==!1)for(let s of r){let o=Fs[s];o?n.mcpFiles.push({file:o.file,outcome:js(e.projectRoot,o)}):Wn[s]&&n.mcpHints.push(Wn[s])}if(e.instructions!==!1){let s=Cs(),o=F.join(e.projectRoot,"AGENTS.md");n.instructionFiles.push({file:"AGENTS.md",outcome:Hn(o,s)});let i=F.join(e.projectRoot,"CLAUDE.md");T.existsSync(i)&&n.instructionFiles.push({file:"CLAUDE.md",outcome:Hn(i,s)})}if(e.examples){n.examples=Is(e.projectRoot,e.packageRoot);for(let s of n.examples.files)n.scaffolded.push(s)}if(e.scaffold!==!1){let s=F.join(e.projectRoot,"saffron.config.json");T.existsSync(s)||(T.writeFileSync(s,JSON.stringify({baseURL:"http://localhost:3000",features:"features"},null,2)+`
|
|
287
|
+
`),n.scaffolded.push("saffron.config.json"));let o=F.join(e.projectRoot,"features");T.existsSync(o)||(T.mkdirSync(o,{recursive:!0}),n.scaffolded.push("features/"));let i=F.join(e.projectRoot,".gitignore"),a=e.examples?[".saffron/reports/",".env"]:[".saffron/reports/"],c=T.existsSync(i)?T.readFileSync(i,"utf8"):"",p=[];for(let h of a)c.split(`
|
|
288
|
+
`).some(d=>d.trim()===h)||(c+=(c&&!c.endsWith(`
|
|
289
289
|
`)?`
|
|
290
|
-
`:"")+
|
|
291
|
-
`,p.push(
|
|
292
|
-
`)}function
|
|
293
|
-
`))}async function
|
|
294
|
-
`)[0]}`),
|
|
295
|
-
${
|
|
296
|
-
${
|
|
290
|
+
`:"")+h+`
|
|
291
|
+
`,p.push(h));p.length>0&&(T.writeFileSync(i,c),n.scaffolded.push(`.gitignore (+ ${p.join(", ")})`))}return n}function Is(e,t){let n=F.join(t,"templates","saucedemo");if(!T.existsSync(F.join(n,"features")))return{installed:!1,files:[],reason:`bundled examples not found at ${n}`};let r=F.join(e,"features");if(T.existsSync(r)&&T.readdirSync(r).length>0)return{installed:!1,files:[],reason:"features/ is not empty; examples are only installed into an empty features directory"};let s=[];T.mkdirSync(r,{recursive:!0});for(let a of T.readdirSync(F.join(n,"features")))T.copyFileSync(F.join(n,"features",a),F.join(r,a)),s.push(`features/${a}`);let o=F.join(e,"saffron.config.json");T.existsSync(o)||(T.copyFileSync(F.join(n,"saffron.config.json"),o),s.push("saffron.config.json (baseURL https://www.saucedemo.com, retries 1)"));let i=F.join(e,".env.example");return T.existsSync(i)||(T.copyFileSync(F.join(n,".env.example"),i),s.push(".env.example")),{installed:!0,files:s}}import _s from"node:path";function Kn(e){return["saffron lsp is a language server: it speaks LSP over stdio and is meant","to be launched by your editor, not run by hand (nothing happens here).","","JetBrains (IntelliJ, WebStorm, PyCharm, Rider, incl. Community):"," Easiest: install the Saffron plugin from the JetBrains Marketplace"," (https://plugins.jetbrains.com/plugin/34240-saffron): highlighting,"," this language server, run configurations and the Saffron tool window,"," in one click. Manual setup instead:"," 1. Completion, diagnostics, hover, go-to-definition: install the free"," LSP4IJ plugin, then Settings \u2192 Languages & Frameworks \u2192 Language"," Servers \u2192 +: command `npx saffron lsp`, working dir = project root,"," file patterns *.saffron and *.feature."," 2. Syntax highlighting comes from a TextMate bundle, not the LSP:"," Settings \u2192 Editor \u2192 TextMate Bundles \u2192 + and pick this folder:",` ${e?_s.join(e,"textmate","saffron"):"node_modules/saffron-ai/textmate/saffron"}`,"","Neovim / other LSP editors: cmd = { 'npx', 'saffron', 'lsp' },","filetypes = { 'saffron', 'feature' }.","","VS Code: install the extension instead:"," code --install-extension ChathurangaJayasinghe.saffron-vscode"].join(`
|
|
292
|
+
`)}function Ns(e,t){let n=t.length>0?t:["features"],r=[];for(let s of n){let o=E.isAbsolute(s)?s:E.join(e,s);if(D.existsSync(o))if(D.statSync(o).isDirectory())for(let i of D.readdirSync(o,{recursive:!0})){let a=String(i);se(a)&&r.push(E.join(o,a))}else se(o)&&r.push(o)}return r.sort()}function Qn(e,t,n){let r=E.isAbsolute(t.features??"features")?t.features:E.join(e,t.features??"features"),s=new Set(de(r));for(let o of n)o.endsWith(".saffron")&&s.add(o);return Z([...s].sort(),e)}function Us(){let e=process.env.ANTHROPIC_API_KEY||process.env.CLAUDE_CODE_OAUTH_TOKEN||process.env.CLAUDE_CODE_USE_BEDROCK||process.env.CLAUDE_CODE_USE_VERTEX,t=process.env.HOME??"",n=t&&(D.existsSync(E.join(t,".claude",".credentials.json"))||D.existsSync(E.join(t,".claude.json")));e||n||console.error(["note: no Claude credentials detected. Cached scenarios replay fine","without AI, but recording and healing need one of:"," \xB7 ANTHROPIC_API_KEY=sk-ant-... (console.anthropic.com \u2192 API keys)"," \xB7 a Claude Code login on this machine","Replay-only runs: add --no-agent to silence this note.",""].join(`
|
|
293
|
+
`))}async function Ls(e){let t=Ds.createInterface({input:process.stdin,output:process.stdout});try{let n=await new Promise(r=>t.question(e,r));return/^y(es)?$/i.test(n.trim())}finally{t.close()}}function qe(e){console.error(`saffron: ${e.message}`),process.exit(2)}function xt(e){return e>=1e6?`${(e/1e6).toFixed(1)}M`:e>=1e4?`${(e/1e3).toFixed(1)}k`:e.toLocaleString()}var Vs={green:"\x1B[32m",yellow:"\x1B[33m",red:"\x1B[31m"},Bs="\x1B[0m",_=new Ms().name("saffron").description("Gherkin-native AI-fallback test runner: zero-token cached replay, runtime AI healing.").version(te()).option("-p, --project-root <dir>","project root",process.cwd());_.command("run").description("Run feature files (cached replay; AI records/heals on demand)").argument("[paths...]","feature files or directories (default: features/)").option("--base-url <url>","base URL of the app under test").option("--headed","run the browser headed").option("--filter <tags>",'only run scenarios carrying any of these @tags: comma-separated ("@smoke,@WA-TC1") or repeat the flag',qn).option("--no-agent","disable AI fallback (cache misses/failures go red)").option("--rerecord","discard the committed caches of the selected scenarios and let the agent record them fresh (the honest fix for a wrong recording)").option("--model <model>","model for the agent").option("--storage-state <file>","Playwright storageState JSON for authenticated sessions").option("--strict","treat passed-with-adaptation (yellow) as failure until its proposal is reviewed").option("--no-verify","skip the zero-AI proof replay of agent recordings before filing proposals").option("--no-reuse","disable Level-1 step reuse (seeding new recordings from existing step recordings)").option("--browser <name>",'replay browser: "chromium" (default), "firefox", or "webkit", recording/healing always use Chromium').option("--heal-model <model>","cheaper model for heal sessions").option("--workers <n>","parallel replay workers (agent recording/healing stays sequential)").option("--assertion-policy <policy>",'assertion adaptability: "strict" (default) or "adaptable-mid" (mid-scenario assertions may be adapted; the final assertion block never is)').action(async(e,t)=>{let n=E.resolve(_.opts().projectRoot),r=H(n),s=t.baseUrl??r.baseURL,o=Ns(n,e.length>0?e:r.features?[r.features]:[]);o.length===0&&(console.error("No .feature/.saffron files found."),process.exit(2)),t.agent!==!1&&Us();let i;try{let S=Qn(n,r,o);i=o.flatMap(k=>ee(k,n,S))}catch(S){throw S instanceof N&&qe(S),S}let a=Bn(t.filter);if(a.length>0&&(i=zn(i,a),i.length===0&&(console.error(`No scenarios match --filter ${a.join(",")} in ${o.length} feature file(s).`),process.exit(2))),t.rerecord){t.agent===!1&&(console.error("saffron: --rerecord needs the agent; drop --no-agent"),process.exit(2));let S=0;for(let k of i){let A=B(n,k.featurePath,k.name);D.existsSync(A)&&(D.rmSync(A),S++)}console.log(`rerecord: removed ${S} cache file(s) for ${i.length} scenario(s); the agent records them fresh (seeded from the other recordings). Restore with git if needed.`)}{let S=new Map;for(let k of i){let A=Ke(k),j=z(B(n,k.featurePath,k.name));j&&A.push(JSON.stringify(j.steps));for(let O of Ye(A))process.env[O]===void 0&&!S.has(O)&&S.set(O,k.displayName)}if(S.size>0){for(let[k,A]of S)console.error(`missing environment variable ${k} (referenced as {env:${k}} by "${A}")`);console.error("set the variable(s), or add them to a git-ignored .env in the project root"),process.exit(2)}}let c=new Ne({projectRoot:n,baseURL:s,headed:t.headed,actionTimeoutMs:r.actionTimeoutMs,retries:r.retries,storageState:t.storageState??r.storageState,browser:(()=>{let S=t.browser??r.browser??"chromium";return S!=="chromium"&&S!=="firefox"&&S!=="webkit"&&(console.error(`invalid --browser "${S}": use "chromium", "firefox", or "webkit"`),process.exit(2)),S})(),healModel:t.healModel??r.healModel,verifyProposals:t.verify===!1?!1:r.verifyProposals,reuseSteps:t.reuse===!1?!1:r.reuseSteps,assertionPolicy:(()=>{let S=t.assertionPolicy??r.assertionPolicy??"strict";return S!=="strict"&&S!=="adaptable-mid"&&(console.error(`invalid --assertion-policy "${S}": use "strict" or "adaptable-mid"`),process.exit(2)),S})(),agent:t.agent?new Ue({model:t.model??r.model,maxTurns:r.maxTurns,snapshotMode:r.snapshotMode}):void 0}),p=new Date,h=Math.max(1,Number.parseInt(String(t.workers??r.workers??1),10)||1),d=new Array(i.length),l=S=>{let k=S.scenario,A=Vs[S.status],j=S.usage.inputTokens+S.usage.outputTokens,O=S.usage.cacheReadTokens+S.usage.cacheCreationTokens,C=O>0?`, ${xt(O)} cache traffic`:"";console.log(`${A}${S.status.toUpperCase().padEnd(7)}${Bs}${k.displayName} (${S.source}, ${S.durationMs}ms, ${S.usage.aiCalls} AI calls, ${j} tokens${C})`);for(let V of S.adaptations)console.log(` \u26A0 ${V}`);S.seededSteps&&console.log(` \u26A1 ${S.seededSteps}/${S.stepResults.length} steps seeded from existing recordings (zero AI)`);let X=S.stepResults.reduce((V,Zn)=>V+(Zn.retries??0),0);X&&console.log(` \u21BB ${X} action ${X===1?"retry":"retries"} absorbed before settling (retries: ${r.retries??1} per action)`),S.verified===!0?console.log(" \u2713 proposal verified by zero-AI proof replay"):S.verified===!1&&console.log(` \u2717 proposal UNVERIFIED, proof replay failed: ${S.proofError?.split(`
|
|
294
|
+
`)[0]}`),S.error&&console.log(` ${S.error}`)};await c.start();try{if(h>1&&i.length>1){let S=i.map((A,j)=>({s:A,i:j})),k=[];await Promise.all(Array.from({length:Math.min(h,S.length)},async()=>{for(;;){let A=S.shift();if(!A)return;let j=await c.runScenario(A.s,{agentDisabled:!0});j.status==="red"?k.push(A):(d[A.i]=j,l(j))}}));for(let A of k.sort((j,O)=>j.i-O.i)){let j=await c.runScenario(A.s);d[A.i]=j,l(j)}}else for(let S=0;S<i.length;S++){let k=await c.runScenario(i[S]);d[S]=k,l(k)}}finally{await c.stop()}let m=Ln(d,p,s),g=W(n).divergent.length,u=Bt(d,p,g,s),y=Ie(n);m.trends=qt(y,u),zt(n,u),m.totals.aiCalls>0&&(m.totals.plan=await c.agent?.planUsage?.());let{html:$,json:x}=Vn(n,m),f=m.totals,v=f.cacheReadTokens+f.cacheCreationTokens;console.log(`
|
|
295
|
+
${f.green} passed, ${f.yellow} pending review, ${f.red} failed \xB7 ${f.aiCalls} AI calls \xB7 ${(f.inputTokens+f.outputTokens).toLocaleString()} tokens in+out${v>0?` \xB7 ${xt(f.cacheReadTokens)} cache read \xB7 ${xt(f.cacheCreationTokens)} cache written`:""}${Pn(f)}${f.models?.length?` \xB7 ${f.models.join(", ")}`:""}`);let b=m.trends;if(b?.prev){let S=b.prev.passRatePp,k=b.prev.costUsd;console.log(`vs previous run: pass rate ${S>=0?"+":""}${S}pp \xB7 cost ${k>=0?"+":"-"}$${Math.abs(k).toFixed(4)} \xB7 AI calls ${b.prev.aiCalls>=0?"+":""}${b.prev.aiCalls}`)}for(let S of b?.chronic??[])console.log(`\u26A0 chronic: ${S.feature} \u203A ${S.scenario}: ${S.heals} heal(s), ${S.reds} red(s) in last ${S.runs} run(s); re-record or fix the feature text instead of paying for more heals`);console.log(`report: ${$}
|
|
296
|
+
${x}`);let w=oe(E.resolve(_.opts().projectRoot));w.length>0&&console.log(`${w.length} cache proposal(s) pending. Review with: saffron accept | saffron reject`);let P=t.strict??r.strict??!1;P&&f.yellow>0&&console.log(`strict mode: ${f.yellow} scenario(s) pending review treated as failure. Review proposals to go green.`),process.exit(f.red>0||P&&f.yellow>0?1:0)});_.command("accept").description("Promote pending cache proposals to committed caches").argument("[files...]","proposal file(s) (default with --all: every proposal)").option("--all","accept all pending proposals").option("--with-feature-edit","also apply the proposal's suggested .feature step rewrites").option("--propagate","apply the same locator fixes to every other cache using the same locator").option("--include-unverified","with --all: also accept proposals whose proof replay failed").action((e,t)=>{let n=E.resolve(_.opts().projectRoot),r=H(n),s=e.length>0?e.map(o=>E.resolve(o)):t.all?oe(n).map(o=>o.file):[];if(s.length===0){Xn(n);return}for(let o of s){let i=fe(o);if(i.meta.verified===!1){if(t.all&&!t.includeUnverified){console.log(`skipped UNVERIFIED ${E.basename(o)} (${i.meta.proofError?.split(`
|
|
297
297
|
`)[0]}). Re-record it (saffron run --rerecord), or accept it explicitly by file, or pass --include-unverified.`);continue}console.log(`\u26A0 accepting UNVERIFIED proposal (proof replay failed: ${i.meta.proofError?.split(`
|
|
298
|
-
`)[0]}): expect a heal or red on the next run.`)}let a=z(B(n,i.cache.feature,i.cache.scenario)),
|
|
299
|
-
${s.steps.length} step(s) across ${s.scannedFiles} file(s) \xB7 ${s.scannedCaches} cache(s) \xB7 \x1B[32m\u25CF\x1B[0m recorded (replays free) \xB7 \x1B[33m\u25CF\x1B[0m divergent \xB7 \u25CB unrecorded`)});_.command("author").description("Draft a feature file from plain-paragraph requirements, reusing the project's step vocabulary (AI)").argument("<prose-file>","text/markdown file with the requirements").option("-o, --out <file>","output path (default: features/<name>.saffron)").option("--model <model>","model for authoring").action(async(e,t)=>{let n=
|
|
300
|
-
`).map(p=>p.trim().replace(/^(Given|When|Then|And|But|\*)\s+/,"")).filter(p=>o.steps.some(
|
|
301
|
-
`))});_.command("status").description("Project overview for dashboards and IDE panels: feature files and scenarios with cache state, tags, pending proposals, last run, history, vocabulary health, effective config").option("--json","machine-readable output").action(e=>{let t=
|
|
298
|
+
`)[0]}): expect a heal or red on the next run.`)}let a=z(B(n,i.cache.feature,i.cache.scenario)),c=Nt(n,o);console.log(`accepted \u2192 ${c}`);let p=a?In(a,i.cache):[];if(p.length>0){let l=_n(n,p,c,!t.propagate);for(let m of p)console.log(` locator fix: ${ut(m.from)} \u2192 ${ut(m.to)}`);if(l.length===0)console.log(" no other caches use this locator.");else if(t.propagate)for(let m of l)console.log(` propagated \u2192 ${m.feature} \u203A ${m.scenario} (${m.steps.map(g=>`"${g.gherkin}"`).join(", ")})`);else{console.log(` ${l.length} other cache(s) use the same locator: re-run with --propagate to fix them too:`);for(let m of l)console.log(` \xB7 ${m.feature} \u203A ${m.scenario}`)}}let h=i.meta.featureEdits??[];if(!t.withFeatureEdit||h.length===0)continue;let d;try{d=Dn(n,i.cache.feature,i.cache.scenario,h,Qn(n,r,[]))}catch(l){throw l instanceof N&&qe(l),l}for(let l of d.applied){let m=l.fromStepSet?`${l.file}:${l.line} (StepSet "${l.fromStepSet}", fixes every invoking scenario)`:`${l.file}:${l.line}`;console.log(` feature edit ${m}:`),console.log(` - ${l.from}`),console.log(` + ${l.to}`)}for(let l of d.skipped)console.log(` skipped edit [${l.index}]: ${l.reason}`);if(d.applied.length>0){let l=z(c);for(let m of h)l.steps[m.index]&&(l.steps[m.index].gherkin=m.text);pe(c,l)}}});_.command("steps").description("List/search the project's step vocabulary (files + caches): what exists, what's recorded, what to reuse").argument("[search]","case-insensitive substring filter").option("--json","machine-readable vocabulary").option("--snippets","write .vscode/saffron.code-snippets for native VS Code completion").action((e,t)=>{let n=E.resolve(_.opts().projectRoot),r=H(n),s;try{s=q(n,r.features??"features")}catch(i){throw i instanceof N&&qe(i),i}if(e&&(s=Ae(s,e)),t.json){console.log(JSON.stringify(s,null,2));return}if(t.snippets){let i=E.join(n,".vscode","saffron.code-snippets");D.mkdirSync(E.dirname(i),{recursive:!0}),D.writeFileSync(i,Ft(s)),console.log(`wrote ${s.steps.length} step + ${s.stepSets.length} StepSet snippet(s) \u2192 ${i}`),console.log("re-run after recording sessions to keep completions current");return}let o={recorded:"\x1B[32m\u25CF\x1B[0m",divergent:"\x1B[33m\u25CF\x1B[0m",unrecorded:"\u25CB"};if(s.stepSets.length>0){console.log("step sets:");for(let i of s.stepSets)console.log(` ${i.name} (${i.steps.length} steps, ${i.usage} scenario(s)), ${i.file}:${i.line}`);console.log("")}for(let i of s.steps){let a=i.fromStepSet?` [${i.fromStepSet}]`:"";console.log(`${o[i.status]} ${i.keyword} ${i.text} \xD7${i.usage}${a}`)}console.log(`
|
|
299
|
+
${s.steps.length} step(s) across ${s.scannedFiles} file(s) \xB7 ${s.scannedCaches} cache(s) \xB7 \x1B[32m\u25CF\x1B[0m recorded (replays free) \xB7 \x1B[33m\u25CF\x1B[0m divergent \xB7 \u25CB unrecorded`)});_.command("author").description("Draft a feature file from plain-paragraph requirements, reusing the project's step vocabulary (AI)").argument("<prose-file>","text/markdown file with the requirements").option("-o, --out <file>","output path (default: features/<name>.saffron)").option("--model <model>","model for authoring").action(async(e,t)=>{let n=E.resolve(_.opts().projectRoot),r=H(n),s=D.readFileSync(E.resolve(e),"utf8"),o;try{o=q(n,r.features??"features")}catch(p){throw p instanceof N&&qe(p),p}console.log(`authoring with ${o.steps.length} known step(s) and ${o.stepSets.length} step set(s) as vocabulary\u2026`);let i=await Yt(s,o,t.model??r.model),a=t.out?E.resolve(t.out):E.join(n,r.features??"features",`${E.basename(e).replace(/\.[^.]+$/,"")}.saffron`);D.existsSync(a)&&(console.error(`refusing to overwrite ${a}: pass --out for a new path`),process.exit(1)),D.mkdirSync(E.dirname(a),{recursive:!0}),D.writeFileSync(a,i.content);let c=i.content.split(`
|
|
300
|
+
`).map(p=>p.trim().replace(/^(Given|When|Then|And|But|\*)\s+/,"")).filter(p=>o.steps.some(h=>h.text===p)).length;console.log(`wrote ${a}`),console.log(`${c} step line(s) reuse the existing vocabulary (seedable) \xB7 $${(i.usage.costUsd??0).toFixed(4)}`),console.log("review the draft, then record it: saffron run "+E.relative(process.cwd(),a))});_.command("mcp").description("Serve the step vocabulary to AI assistants over stdio MCP (search_steps, list_step_sets)").action(async()=>{let e=E.resolve(_.opts().projectRoot),t=H(e);await Jt(e,t.features??"features")});_.command("lsp").description("Run the Saffron language server over stdio (completion, go-to-definition, hover, diagnostics). For JetBrains (LSP4IJ / Ultimate LSP API), Neovim, and any LSP-capable editor").action(()=>{process.stdin.isTTY&&(console.error(Kn(Ce())),process.exit(2));let e=E.resolve(_.opts().projectRoot),t=H(e),n=process.argv.includes("-p")||process.argv.some(r=>r.startsWith("--project-root"));tn(e,t.features??"features",{rootFromClient:!n})});_.command("reject").description("Discard pending cache proposals (AI will retry next run)").argument("[files...]","proposal file(s)").option("--all","reject all pending proposals").action((e,t)=>{let n=E.resolve(_.opts().projectRoot),r=e.length>0?e.map(s=>E.resolve(s)):t.all?oe(n).map(s=>s.file):[];if(r.length===0){Xn(n);return}for(let s of r)Ut(s),console.log(`rejected ${s}`)});_.command("init").description("Make the project agent-ready: install the bundled Saffron skill into agent directories, add instructions to AGENTS.md, scaffold config").option("--agents <list>",`comma-separated targets: ${Object.keys($e).join(", ")} (default: claude,agents)`).option("--no-instructions","do not touch AGENTS.md / CLAUDE.md").option("--no-scaffold","do not create saffron.config.json, features/, .gitignore entry").option("--no-mcp","do not register `saffron mcp` in project MCP configs (.mcp.json, .cursor/mcp.json, .vscode/mcp.json)").option("--examples","install the Saucedemo example suite (features, config, .env.example) for a guided first run").option("--no-examples","never install the example suite (skips the prompt)").action(async e=>{let t=E.resolve(_.opts().projectRoot),n=Ce();n||(console.error("saffron: cannot locate the installed package root"),process.exit(2));let r=e.agents?String(e.agents).split(",").map(i=>i.trim()).filter(Boolean):void 0;for(let i of r??[])i in $e||(console.error(`saffron: unknown agent target "${i}" (expected one of ${Object.keys($e).join(", ")})`),process.exit(2));let s=e.examples===!0;e.examples===void 0&&process.stdin.isTTY&&process.stdout.isTTY&&(s=await Ls("Install the Saucedemo example tests (login, cart, checkout) into features/ for a guided first run? [y/N] "));let o=Yn({projectRoot:t,packageRoot:n,targets:r,instructions:e.instructions,scaffold:e.scaffold,mcp:e.mcp,examples:s});for(let i of o.skillDirs)console.log(`skill \u2192 ${i}/`);for(let i of o.mcpFiles)console.log(`${i.outcome.padEnd(9)} \u2192 ${i.file} (saffron mcp server)`);for(let i of o.mcpHints)console.log(`mcp hint \u2192 ${i}`);for(let i of o.instructionFiles)console.log(`${i.outcome.padEnd(9)} \u2192 ${i.file} (managed Saffron block)`);for(let i of o.scaffolded)console.log(`created \u2192 ${i}`);o.examples&&!o.examples.installed&&console.log(`examples \u2192 not installed: ${o.examples.reason}`),console.log("\nAgents now load the saffron skill when writing tests. Re-run after `npm update saffron-ai` to refresh it."),o.examples?.installed&&console.log(["","Your guided first run (recording needs Claude credentials; replays never do):"," cp .env.example .env # public demo credentials, resolved at replay"," npx playwright install chromium"," npx saffron run --filter @smoke # the agent records two scenarios (about $3 of AI usage), files proposals"," npx saffron accept --all # review, then promote them to caches"," npx saffron run --filter @smoke # replays at zero tokens","Then try the rest: npx saffron run (login errors outline, checkout with a data table)"].join(`
|
|
301
|
+
`))});_.command("status").description("Project overview for dashboards and IDE panels: feature files and scenarios with cache state, tags, pending proposals, last run, history, vocabulary health, effective config").option("--json","machine-readable output").action(e=>{let t=E.resolve(_.opts().projectRoot),n=H(t),r=_e(t,n);if(e.json){console.log(JSON.stringify(r,null,2));return}Gt(r)});_.command("report").description("Open the latest HTML report").option("--open","open in the default browser",!0).action(()=>{let e=E.resolve(_.opts().projectRoot),t=E.join(e,".saffron","reports","latest.html");D.existsSync(t)||(console.error("No report found. Run `saffron run` first."),process.exit(2)),console.log(t);let n=process.platform==="darwin"?"open":"xdg-open";Os(n,[t],{detached:!0,stdio:"ignore"}).unref()});function Xn(e){let t=oe(e);if(t.length===0){console.log("No pending proposals.");return}console.log(`Pending proposals:
|
|
302
302
|
`);for(let{file:n,proposal:r}of t){console.log(` ${n}`);let s=r.meta.verified===!0?" \xB7 verified \u2713":r.meta.verified===!1?" \xB7 UNVERIFIED \u2717":"";console.log(` ${r.cache.feature} \u203A ${r.cache.scenario} (${r.meta.mode}, ${r.meta.createdAt}${s})`),console.log(` ${r.meta.narrative}`);for(let o of r.meta.adaptations)console.log(` \u26A0 ${o}`);r.meta.suggestedFeatureEdit&&console.log(` suggested .feature edit:
|
|
303
303
|
${r.meta.suggestedFeatureEdit.split(`
|
|
304
304
|
`).map(o=>` ${o}`).join(`
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "saffron-ai",
|
|
3
|
-
"version": "0.5.
|
|
3
|
+
"version": "0.5.7",
|
|
4
4
|
"description": "Gherkin-native AI-fallback test runner: zero-token cached replay, runtime AI healing, honest reports",
|
|
5
5
|
"scripts": {
|
|
6
6
|
"test": "vitest run",
|
|
@@ -8,8 +8,9 @@
|
|
|
8
8
|
"dev": "tsc --watch",
|
|
9
9
|
"e2e": "node scripts/e2e.mjs",
|
|
10
10
|
"prepublishOnly": "npm run build && npm test && npm run bundle",
|
|
11
|
-
"docs": "node scripts/build-docs.mjs && node scripts/build-site-docs.mjs",
|
|
12
|
-
"bundle": "esbuild src/cli/index.ts --bundle --platform=node --format=esm --packages=external --minify --legal-comments=none --outfile=dist-pkg/cli.js"
|
|
11
|
+
"docs": "node scripts/build-docs.mjs && node scripts/build-site-docs.mjs && node scripts/build-claude-plugin.mjs",
|
|
12
|
+
"bundle": "esbuild src/cli/index.ts --bundle --platform=node --format=esm --packages=external --minify --legal-comments=none --outfile=dist-pkg/cli.js",
|
|
13
|
+
"plugin": "node scripts/build-claude-plugin.mjs"
|
|
13
14
|
},
|
|
14
15
|
"keywords": [
|
|
15
16
|
"testing",
|
package/skills/saffron/SKILL.md
CHANGED
|
@@ -38,8 +38,20 @@ Prefer a `●` recorded wording verbatim. Quoted values may differ freely
|
|
|
38
38
|
(`I enter "admin"` seeds from `I enter "bob"`), so parameterize with
|
|
39
39
|
quotes rather than inventing new phrasings. `saffron init` registers the
|
|
40
40
|
`saffron` MCP server with Claude Code, Cursor and VS Code; its
|
|
41
|
-
`search_steps` / `list_step_sets` tools give the same
|
|
42
|
-
shell.
|
|
41
|
+
`search_steps` / `list_step_sets` / `project_status` tools give the same
|
|
42
|
+
answers without a shell.
|
|
43
|
+
|
|
44
|
+
**Know the project before touching it:**
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
npx saffron status # files and scenarios (● cached, ◐ proposal pending, ○ unrecorded), tags, proposals, last run, health
|
|
48
|
+
npx saffron status --json # the same as data (what the IDE panels and the project_status MCP tool read)
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Use it to pick tags that already exist, to see which scenarios are
|
|
52
|
+
cached (editing their text re-records them), which proposals await
|
|
53
|
+
review, and whether the vocabulary has divergent steps or duplicate
|
|
54
|
+
wordings to converge before adding more.
|
|
43
55
|
|
|
44
56
|
## 2. `Then` steps are sacred: write them as the verdict
|
|
45
57
|
|
|
@@ -114,20 +126,33 @@ Scenario: Checkout happy path
|
|
|
114
126
|
|
|
115
127
|
```bash
|
|
116
128
|
npx saffron run # record misses, replay hits, heal failures
|
|
117
|
-
npx saffron run --filter @smoke # by tag
|
|
129
|
+
npx saffron run --filter @smoke # by tag (comma-separate several; any of them)
|
|
118
130
|
npx saffron run --no-agent # replay only (CI without AI access)
|
|
119
|
-
npx saffron
|
|
131
|
+
npx saffron status # what is cached, pending, tagged; vocabulary health
|
|
132
|
+
npx saffron accept # list pending proposals with narratives
|
|
120
133
|
npx saffron accept --all # promote verified proposals to caches
|
|
134
|
+
npx saffron accept <file> [<file>...] # promote chosen ones
|
|
135
|
+
npx saffron reject <file> [<file>...] # discard chosen ones; the agent retries next run
|
|
121
136
|
npx saffron report # open the HTML report
|
|
122
137
|
npx saffron run --rerecord --filter @t # a recording is wrong: record it fresh
|
|
123
138
|
```
|
|
124
139
|
|
|
125
|
-
Results: **green** = cached pass · **yellow** =
|
|
126
|
-
or adaptation
|
|
140
|
+
Results: **green** = cached pass · **yellow** = pending review (an AI
|
|
141
|
+
recording or adaptation filed a proposal) · **red** = failed. Proposals
|
|
127
142
|
arrive stamped `verified ✓` (zero-AI proof replay passed) or
|
|
128
143
|
`UNVERIFIED ✗`; accept the former, investigate the latter. Commit
|
|
129
144
|
`.saffron/cache/` like snapshots; git-ignore `.saffron/reports/`.
|
|
130
145
|
|
|
146
|
+
Cost in the summary line and report: on an API key the dollars are what
|
|
147
|
+
was billed; on a Claude subscription (Claude Code login, no key) the
|
|
148
|
+
headline is the 5-hour plan window before and after the run, and the
|
|
149
|
+
dollars are the API-equivalent, marked `≈`. Both name the models used.
|
|
150
|
+
|
|
151
|
+
Scenario Outlines record once: the first Examples row records, later
|
|
152
|
+
rows replay that proposal at zero AI. Write the `Then` with the row's
|
|
153
|
+
value quoted exactly (`Then I should see "<message>"`); the recorder
|
|
154
|
+
keeps the placeholder, so every row satisfies it.
|
|
155
|
+
|
|
131
156
|
**Never hand-edit files under `.saffron/cache/` or `.saffron/proposals/`**
|
|
132
157
|
unless deliberately authoring a manual cache (`"recordedBy": "manual"`).
|
|
133
158
|
To change behavior, change the `.feature`/`.saffron` text and let the run
|
|
@@ -54,12 +54,13 @@ your-project/
|
|
|
54
54
|
| Command | Purpose |
|
|
55
55
|
|---|---|
|
|
56
56
|
| `saffron run [paths] [--filter @tag] [--no-agent] [--strict] [--browser b] [--workers n] [--heal-model m] [--headed]` | Run; cached replay, agent on misses/failures |
|
|
57
|
-
| `saffron accept [
|
|
58
|
-
| `saffron reject [
|
|
57
|
+
| `saffron accept [files... \| --all] [--include-unverified] [--with-feature-edit] [--propagate]` | Promote proposals (`--all` skips UNVERIFIED ones unless `--include-unverified`); `--with-feature-edit` rewrites adapted steps in the feature file; `--propagate` applies a heal's locator fix to every cache using that locator |
|
|
58
|
+
| `saffron reject [files... \| --all]` | Discard proposals |
|
|
59
|
+
| `saffron status [--json]` | Project overview: files and scenarios with cache state, tags, pending proposals, last run, history, vocabulary health, effective config |
|
|
59
60
|
| `saffron steps [search] [--json] [--snippets]` | The step vocabulary with recorded/divergent/unrecorded badges |
|
|
60
61
|
| `saffron author <prose-file>` | Draft a `.saffron` file from plain-paragraph requirements using the project vocabulary |
|
|
61
|
-
| `saffron mcp` / `saffron lsp` |
|
|
62
|
-
| `saffron init [--examples]` | Install this skill into the project's agent directories, register MCP, scaffold config; `--examples` adds the Saucedemo example suite |
|
|
62
|
+
| `saffron mcp` / `saffron lsp` | MCP tools (`search_steps`, `list_step_sets`, `project_status`) for AI assistants / language server for editors |
|
|
63
|
+
| `saffron init [--examples] [--agents list]` | Install this skill into the project's agent directories, register MCP, add an AGENTS.md block, scaffold config; `--examples` adds the Saucedemo example suite |
|
|
63
64
|
| `saffron report` | Open the latest HTML report |
|
|
64
65
|
|
|
65
66
|
Exit codes: 0 green/yellow, 1 red (or yellow with `--strict`), 2 usage /
|
|
@@ -69,7 +70,19 @@ preflight (e.g. a missing `{env:VAR}`).
|
|
|
69
70
|
|
|
70
71
|
Recording and healing need Claude credentials: `ANTHROPIC_API_KEY`, or a
|
|
71
72
|
Claude Code login on the machine. Replay-only runs (`--no-agent`) need
|
|
72
|
-
none: that is the normal CI mode once caches are committed.
|
|
73
|
+
none: that is the normal CI mode once caches are committed. With no
|
|
74
|
+
`model` configured the Agent SDK's default model is used; set `model`
|
|
75
|
+
(and `healModel`) to pin one. On a subscription login the report shows
|
|
76
|
+
the 5-hour plan window used and the API-equivalent dollars; on a key it
|
|
77
|
+
shows what was billed.
|
|
78
|
+
|
|
79
|
+
## IDE integration
|
|
80
|
+
|
|
81
|
+
The VS Code extension and the JetBrains plugin add right-click Run on
|
|
82
|
+
`.saffron` files, a Saffron panel (feature files, tags, proposals to
|
|
83
|
+
accept or reject, vocabulary health, config) and the report as an
|
|
84
|
+
in-editor dashboard, all reading `saffron status --json`. Nothing there
|
|
85
|
+
is required for agents; the CLI is the same surface.
|
|
73
86
|
|
|
74
87
|
## Recommended CI
|
|
75
88
|
|