saffron-ai 0.4.1 → 0.4.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -3
- package/dist-pkg/cli.js +92 -90
- package/package.json +1 -1
- package/skills/saffron/SKILL.md +2 -1
- package/skills/saffron/references/config.md +1 -0
package/README.md
CHANGED
|
@@ -24,7 +24,7 @@ Like a maturing Cucumber suite, most of a new feature file is steps you already
|
|
|
24
24
|
|
|
25
25
|
**Step sets** (shipped): `.saffron` files, a superset dialect of Gherkin, add the `StepSet:` keyword for named, reusable step sequences invoked with `StepSet <name>` inside any scenario. Sets expand at parse time, so their steps cache and seed like ordinary steps; editing a set makes every invoking scenario honestly stale (re-recorded mostly seeded), and heal edits route to the set definition, one fix, every caller follows. Sets are **project-wide**: keep application-wide flows in a sets-only library file (convention: `features/shared.steps.saffron`) and invoke them from any feature.
|
|
26
26
|
|
|
27
|
-
**IDE integration & authoring** (shipped): `saffron steps` lists the project vocabulary with recorded/divergent/unrecorded badges (`--snippets` for native VS Code completion), `saffron mcp` serves it to AI assistants, `saffron lsp` brings completion/navigation/diagnostics to **JetBrains** (incl. Community editions via LSP4IJ) and Neovim, `saffron author` drafts feature files from prose in your own vocabulary, and duplicate wordings that record identical actions get **behavior-proven rename proposals**. The **Saffron VS Code extension** is on the [Marketplace](https://marketplace.visualstudio.com/items?itemName=ChathurangaJayasinghe.saffron-vscode) (`code --install-extension ChathurangaJayasinghe.saffron-vscode`). JetBrains users get the same in one click from the **Saffron JetBrains plugin** (
|
|
27
|
+
**IDE integration & authoring** (shipped): `saffron steps` lists the project vocabulary with recorded/divergent/unrecorded badges (`--snippets` for native VS Code completion), `saffron mcp` serves it to AI assistants, `saffron lsp` brings completion/navigation/diagnostics to **JetBrains** (incl. Community editions via LSP4IJ) and Neovim, `saffron author` drafts feature files from prose in your own vocabulary, and duplicate wordings that record identical actions get **behavior-proven rename proposals**. The **Saffron VS Code extension** is on the [Marketplace](https://marketplace.visualstudio.com/items?itemName=ChathurangaJayasinghe.saffron-vscode) (`code --install-extension ChathurangaJayasinghe.saffron-vscode`). JetBrains users get the same in one click from the **Saffron JetBrains plugin** ([source](https://github.com/s-chathuranga-j/saffron-jetbrains-plugin): `.saffron` file type, bundled grammar, LSP4IJ wiring; Marketplace listing pending).
|
|
28
28
|
|
|
29
29
|
Data tables and doc strings are first-class: 2-column key/value tables parameterize the recording (`<table:username>`), multi-row record tables parameterize per cell (`<table:1:firstName>`), and `"""` doc strings record as `<docstring>`, so editing *values* or *content* replays at zero tokens, while structural changes (keys, headers, row counts) honestly re-record. Unambiguous params are scenario-wide, so a later assertion on a note's text follows content edits too. **Secrets** stay out of everything: `{env:VAR}` resolves from the environment (or a git-ignored `.env`) at replay, recordings and reports are masked back to the token, and missing variables fail fast by name.
|
|
30
30
|
|
|
@@ -66,8 +66,8 @@ node dist/cli/index.js -p examples accept --all
|
|
|
66
66
|
|
|
67
67
|
| Command | What it does |
|
|
68
68
|
|---|---|
|
|
69
|
-
| `saffron run [paths] [--headed] [--filter @tags] [--no-agent] [--strict] [--browser b] [--workers n] [--heal-model m] [--no-verify] [--no-reuse] [--model m] [--storage-state f]` | Run features. Cached replays are deterministic; misses/failures escalate to the agent (unless `--no-agent`). Replay cross-browser with `--browser firefox\|webkit`, parallelize with `--workers N`, heal on a cheaper model with `--heal-model`. Exit 1 on red (and on yellow with `--strict`). |
|
|
70
|
-
| `saffron accept [file \| --all] [--with-feature-edit] [--propagate]` | Promote cache proposals to committed caches; `--with-feature-edit` also rewrites the adapted steps in the `.feature` file (and keeps the cache in sync); `--propagate` applies the heal's locator fixes to every other cache using the same locator. One heal repairs N scenarios before they ever fail. No args: list pending proposals. |
|
|
69
|
+
| `saffron run [paths] [--headed] [--filter @tags] [--rerecord] [--no-agent] [--strict] [--browser b] [--workers n] [--heal-model m] [--no-verify] [--no-reuse] [--model m] [--storage-state f]` | Run features. Cached replays are deterministic; misses/failures escalate to the agent (unless `--no-agent`). Replay cross-browser with `--browser firefox\|webkit`, parallelize with `--workers N`, heal on a cheaper model with `--heal-model`. Exit 1 on red (and on yellow with `--strict`). |
|
|
70
|
+
| `saffron accept [file \| --all] [--include-unverified] [--with-feature-edit] [--propagate]` | Promote cache proposals to committed caches (`--all` skips UNVERIFIED ones unless `--include-unverified`); `--with-feature-edit` also rewrites the adapted steps in the `.feature` file (and keeps the cache in sync); `--propagate` applies the heal's locator fixes to every other cache using the same locator. One heal repairs N scenarios before they ever fail. No args: list pending proposals. |
|
|
71
71
|
| `saffron reject [file \| --all]` | Discard proposals; the agent will try again next run. |
|
|
72
72
|
| `saffron steps [search] [--json] [--snippets]` | List/search the step vocabulary (files + caches) with recorded/divergent/unrecorded badges and usage; `--snippets` writes `.vscode/saffron.code-snippets` for native VS Code completion. |
|
|
73
73
|
| `saffron mcp` | Serve the vocabulary to AI assistants over stdio MCP (`search_steps`, `list_step_sets`), e.g. `claude mcp add saffron -- npx saffron mcp`. |
|
package/dist-pkg/cli.js
CHANGED
|
@@ -1,34 +1,34 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
|
-
import I from"node:fs";import
|
|
3
|
-
`),
|
|
4
|
-
`),definitions:
|
|
5
|
-
`)}function
|
|
6
|
-
`}import{Server as
|
|
7
|
-
`),
|
|
2
|
+
import I from"node:fs";import R from"node:path";import{spawn as es}from"node:child_process";import{Command as ts}from"commander";import Ur from"node:fs";import Nr from"node:path";import{AstBuilder as Br,GherkinClassicTokenMatcher as Vr,Parser as zr}from"@cucumber/gherkin";import{IdGenerator as qr,StepKeywordType as Me}from"@cucumber/messages";function q(t,e){return t.replace(/<([^<>]+)>/g,(r,n)=>n in e?e[n]:r)}function L(t){if(!t||t.length===0)return{};let e={};if(t.every(r=>r.length===2)){for(let[r,n]of t)e[`table:${r}`]=n;return e}if(t.length>=2){let[r,...n]=t;if(r.some(s=>!s))return{};n.forEach((s,o)=>{r.forEach((i,a)=>{s[a]!==void 0&&(e[`table:${o+1}:${i}`]=s[a])})})}return e}function W(t){return t!==void 0?{docstring:t}:{}}function K(t){let e=new Map;for(let n of t){let s={...L(n.resolvedTable??n.table),...W(n.resolvedDocString??n.docString)};for(let[o,i]of Object.entries(s)){let a=e.get(o);a||e.set(o,a=new Set),a.add(i)}}let r={};for(let[n,s]of e)s.size===1&&(r[n]=[...s][0]);return r}import De from"node:fs";import at from"node:path";import{AstBuilder as Fr,GherkinClassicTokenMatcher as Ir,Parser as jr}from"@cucumber/gherkin";import{IdGenerator as _r}from"@cucumber/messages";var M=class extends Error{},Dr=/^(\s*)StepSet:\s*(.+?)\s*$/,Or=/^(\s*)StepSet\s+([^\s:].*?)\s*$/;function Oe(t){let e=t.split(`
|
|
3
|
+
`),r=new Map,n=new Map;return{source:e.map((o,i)=>{let a=o.match(Dr);if(a)return r.set(i+1,a[2]),`${a[1]}Scenario: ${a[2]}`;let d=o.match(Or);return d?(n.set(i+1,d[2]),`${d[1]}* StepSet ${d[2]}`):o}).join(`
|
|
4
|
+
`),definitions:r,invocations:n}}function Mr(t){return new jr(new Fr(_r.uuid()),new Ir).parse(t)}function ct(t,e){for(let r of t)r.scenario?e(r.scenario):"rule"in r&&r.rule&&ct(r.rule.children,e)}function X(t,e){let r=new Map;for(let n of t){let s=at.relative(e,n),o=Oe(De.readFileSync(n,"utf8"));if(o.definitions.size===0)continue;let i=Mr(o.source);i.feature&&ct(i.feature.children,a=>{let d=o.definitions.get(a.location.line);if(d===void 0)return;let p=r.get(d);if(p)throw new M(`StepSet "${d}" is defined twice: ${p.file}:${p.line} and ${s}:${a.location.line}. Set names are unique across the project. If line ${a.location.line} was meant to INVOKE the set inside a scenario, remove the colon: "StepSet ${d}".`);if(a.examples.length>0)throw new M(`StepSet "${d}" (${s}:${a.location.line}) has an Examples table \u2014 step sets take their values from the invoking scenario.`);for(let m of a.steps)if(o.invocations.has(m.location.line))throw new M(`StepSet "${d}" (${s}:${a.location.line}) invokes another step set at line ${m.location.line} \u2014 step sets cannot be nested.`);r.set(d,{name:d,file:s,line:a.location.line,steps:[...a.steps]})})}return r}var Lr=[".feature",".saffron"];function se(t){return Lr.some(e=>t.endsWith(e))}function ue(t){if(!De.existsSync(t))return[];let e=[];for(let r of De.readdirSync(t,{recursive:!0})){let n=String(r);n.endsWith(".saffron")&&e.push(at.join(t,n))}return e.sort()}function lt(t,e,r){return t.map(({step:n,source:s})=>{let o;switch(n.keywordType){case Me.OUTCOME:o="assertion";break;case Me.CONTEXT:case Me.ACTION:o="action";break;default:o=r.value}r.value=o;let i=n.dataTable?n.dataTable.rows.map(d=>d.cells.map(p=>p.value)):void 0,a=n.docString?.content;return{keyword:n.keyword.trim(),kind:o,text:n.text,resolvedText:q(n.text,e),table:i,resolvedTable:i?.map(d=>d.map(p=>q(p,e))),docString:a,resolvedDocString:a!==void 0?q(a,e):void 0,source:s}})}function dt(t,e,r,n,s){let o=[];for(let i of t){let a=n.get(i.location.line);if(a===void 0){o.push({step:i,source:{file:e,line:i.location.line,fromBackground:r}});continue}let d=s?.get(a);if(!d)throw new M(`${e}:${i.location.line} invokes StepSet "${a}", which is not defined anywhere in the project.`);for(let p of d.steps)o.push({step:p,source:{file:d.file,line:p.location.line,fromStepSet:d.name,fromBackground:r}})}return o}function Wr(t,e,r,n,s){let o=t.tags.map(a=>a.name),i=[];if(t.examples.length===0)i.push({params:{},label:""});else{let a=0;for(let d of t.examples){let p=d.tableHeader?.cells.map(m=>m.value)??[];for(let m of d.tableBody){a+=1;let l={};m.cells.forEach((c,g)=>{l[p[g]]=c.value}),i.push({params:l,label:` (example ${a})`})}}}return i.map(({params:a,label:d})=>{let p={value:"action"};return{featurePath:e,featureName:r,name:t.name,displayName:`${t.name}${d}`,tags:o,steps:[...lt(n,a,p),...lt(s,a,p)],params:a}})}function Z(t,e,r){let n=Ur.readFileSync(t,"utf8"),o=t.endsWith(".saffron")?Oe(n):{source:n,definitions:new Map,invocations:new Map},a=new zr(new Br(qr.uuid()),new Vr).parse(o.source);if(!a.feature)return[];let d=Nr.relative(e,t),p=a.feature.name,m=[],l=(c,g)=>{let h=g;for(let u of c)if(u.background)h=[...g,...dt(u.background.steps,d,!0,o.invocations,r)];else if(u.scenario){if(o.definitions.has(u.scenario.location.line))continue;m.push(...Wr(u.scenario,d,p,h,dt(u.scenario.steps,d,!1,o.invocations,r)))}else"rule"in u&&u.rule&&l(u.rule.children,h)};return l(a.feature.children,[]),m}import xt from"node:fs";import Le from"node:path";import ft from"node:fs";import gt from"node:path";import fe from"node:fs";import oe from"node:path";var pt=".saffron";function ge(t){return t.toLowerCase().replace(/[^a-z0-9]+/g,"-").replace(/^-+|-+$/g,"")||"unnamed"}function z(t,e,r){let n=ge(oe.basename(e,".feature"));return oe.join(t,pt,"cache",n,`${ge(r)}.json`)}function ut(t,e,r){let n=ge(oe.basename(e,".feature"));return oe.join(t,pt,"proposals",n,`${ge(r)}.json`)}function N(t){if(!fe.existsSync(t))return;let e=JSON.parse(fe.readFileSync(t,"utf8"));if(e.version!==1)throw new Error(`Unsupported cache version ${e.version} in ${t}`);return e}function ee(t,e){fe.mkdirSync(oe.dirname(t),{recursive:!0}),fe.writeFileSync(t,`${JSON.stringify(e,null,2)}
|
|
5
|
+
`)}function ht(t){let e=[];return{pattern:t.replace(/"((?:[^"\\]|\\.)*)"/g,(n,s)=>(e.push(s),`"\xAB${e.length-1}\xBB"`)),args:e}}function Hr(t,e){let r=n=>JSON.stringify(n.map(({pageFingerprint:s,...o})=>o));return r(t)===r(e)}function Gr(t){let e=new Set,r=n=>{if(n)for(let s of n.matchAll(/<([^<>]+)>/g))e.add(s[1])};for(let n of t.actions)r(n.value),r(n.url),r(n.pattern),r(n.target?.name),r(n.target?.nameRegex),r(n.target?.label),r(n.target?.text),r(n.target?.placeholder);return r(t.gherkin),e}function H(t){let e=gt.join(t,".saffron","cache"),r=new Map,n=new Map,s=0;if(ft.existsSync(e))for(let a of ft.readdirSync(e,{recursive:!0})){let d=String(a);if(!d.endsWith(".json"))continue;let p=N(gt.join(e,d));if(p){s++;for(let m of p.steps){let l=m.gherkin,c=JSON.stringify(m.actions.map(({pageFingerprint:u,...f})=>f)),g=n.get(l)??[];g.includes(c)||(g.push(c),n.set(l,g));let h=r.get(l);!h||p.recordedAt>h.recordedAt?r.set(l,{gherkin:m.gherkin,table:m.table,docString:m.docString,actions:m.actions,feature:p.feature,scenario:p.scenario,recordedAt:p.recordedAt,variants:g.length}):h.variants=g.length}}}let o=[...n.entries()].filter(([,a])=>a.length>1).map(([a])=>a),i=new Map;for(let a of r.values()){let{pattern:d,args:p}=ht(a.gherkin);if(p.length===0)continue;let m=i.get(d);(!m||a.recordedAt>m.recordedAt)&&i.set(d,{...a,args:p})}return{steps:r,patterns:i,divergent:o,scannedCaches:s}}function mt(t,e,r){if(!!t.table!=!!e.table)return!1;if(t.table&&e.table){let o=Object.keys(L(t.table)),i=Object.keys(L(e.table));if(o.length>0&&i.length>0){if(JSON.stringify(o)!==JSON.stringify(i))return!1}else if(JSON.stringify(t.table)!==JSON.stringify(e.table))return!1}if(t.docString!==void 0!=(e.docString!==void 0)||t.docString!==void 0&&e.docString!==void 0&&t.docString!==e.docString&&!t.actions.some(o=>[o.value,o.url,o.pattern].some(i=>i?.includes("<docstring>"))))return!1;let n={...r,...L(e.resolvedTable??e.table),...W(e.resolvedDocString??e.docString)},s=Gr({gherkin:t.gherkin,keyword:"",kind:"action",actions:t.actions});for(let o of s)if(!(o in n))return!1;return!0}function yt(t,e,r){let n=t.steps.get(e.text);if(n)return mt(n,e,r)?n:void 0;let{pattern:s,args:o}=ht(e.text);if(o.length===0)return;let i=t.patterns.get(s);if(!i)return;let a=new Map;for(let l=0;l<o.length;l++){let c=i.args[l],g=o[l];if(c===g)continue;let h=a.get(c);if(h!==void 0&&h!==g)return;a.set(c,g)}let d=l=>[l?.name,l?.nameRegex,l?.label,l?.text,l?.placeholder];for(let l of a.keys())if(!i.actions.some(g=>[g.value,g.url,g.pattern,...d(g.target)].includes(l)))return;let p=l=>l!==void 0&&a.has(l)?a.get(l):l,m={...i,gherkin:e.text,actions:i.actions.map(({pageFingerprint:l,...c})=>({...c,value:p(c.value),url:p(c.url),pattern:p(c.pattern),target:c.target?{...c.target,name:p(c.target.name),nameRegex:p(c.target.nameRegex),label:p(c.target.label),text:p(c.target.text),placeholder:p(c.target.placeholder)}:void 0}))};return mt(m,e,r)?m:void 0}function St(t,e){let r=[];return t.forEach((n,s)=>{if(n.actions.length!==0&&!e.steps.has(n.gherkin)){for(let o of e.steps.values())if(!(o.variants>1)&&o.actions.length!==0&&Hr(n.actions,o.actions)){r.push({index:s,from:n.gherkin,to:o.gherkin,feature:o.feature,scenario:o.scenario});break}}}),r}function Jr(t){if(!xt.existsSync(t))return[];let e=[];for(let r of xt.readdirSync(t,{recursive:!0})){let n=String(r);se(n)&&e.push(Le.join(t,n))}return e.sort()}function G(t,e,r){let n=Le.isAbsolute(e)?e:Le.join(t,e),s=Jr(n),o=r??X(s.filter(c=>c.endsWith(".saffron")),t),i=new Map,a=new Map;for(let c of s)for(let g of Z(c,t,o)){let h=`${g.featurePath}::${g.name}`;for(let u of g.steps){let f=i.get(u.text);if(f||(f={text:u.text,keywords:new Map,kind:u.kind,scenarios:new Set,files:new Set,fromStepSet:u.source?.fromStepSet},i.set(u.text,f)),f.keywords.set(u.keyword,(f.keywords.get(u.keyword)??0)+1),f.scenarios.add(h),u.source&&f.files.add(u.source.file),u.source?.fromStepSet){let y=a.get(u.source.fromStepSet);y||a.set(u.source.fromStepSet,y=new Set),y.add(h)}}}let d=H(t),p=new Set(d.divergent),m=[...i.values()].map(c=>({text:c.text,keyword:[...c.keywords.entries()].sort((g,h)=>h[1]-g[1])[0][0],kind:c.kind,status:p.has(c.text)?"divergent":d.steps.has(c.text)?"recorded":"unrecorded",usage:c.scenarios.size,files:[...c.files].sort(),fromStepSet:c.fromStepSet})).sort((c,g)=>g.usage-c.usage||c.text.localeCompare(g.text)),l=[...o.values()].map(c=>({name:c.name,file:c.file,line:c.line,steps:c.steps.map(g=>g.text),usage:a.get(c.name)?.size??0})).sort((c,g)=>g.usage-c.usage||c.name.localeCompare(g.name));return{steps:m,stepSets:l,scannedFiles:s.length,scannedCaches:d.scannedCaches}}function me(t,e){let r=e.toLowerCase();return{...t,steps:t.steps.filter(n=>n.text.toLowerCase().includes(r)),stepSets:t.stepSets.filter(n=>n.name.toLowerCase().includes(r)||n.steps.some(s=>s.toLowerCase().includes(r)))}}function bt(t){let e={},r="saffron,feature,gherkin,cucumber";for(let n of t.steps)e[`step: ${n.text}`]={scope:r,prefix:n.text,body:[n.text.replace(/\$/g,"\\$")],description:`${n.status} \xB7 used in ${n.usage} scenario(s)`};for(let n of t.stepSets)e[`StepSet: ${n.name}`]={scope:r,prefix:`StepSet ${n.name}`,body:[`StepSet ${n.name}`],description:`step set (${n.steps.length} steps) \u2014 ${n.file}`};return JSON.stringify(e,null,2)+`
|
|
6
|
+
`}import{Server as Yr}from"@modelcontextprotocol/sdk/server/index.js";import{StdioServerTransport as Qr}from"@modelcontextprotocol/sdk/server/stdio.js";import{CallToolRequestSchema as Kr,ListToolsRequestSchema as Xr}from"@modelcontextprotocol/sdk/types.js";async function vt(t,e){let r=new Yr({name:"saffron",version:"0.1.0"},{capabilities:{tools:{}}});r.setRequestHandler(Xr,async()=>({tools:[{name:"search_steps",description:"Search the project's step vocabulary. Returns known Gherkin steps with status (recorded = replays at zero tokens, divergent, unrecorded), usage counts, and source files. ALWAYS reuse an existing wording exactly instead of inventing a near-duplicate \u2014 exact step text is the replay-cache identity, so a new wording costs a fresh AI recording.",inputSchema:{type:"object",properties:{query:{type:"string",description:"Case-insensitive substring filter over step text; omit to list everything."}}}},{name:"list_step_sets",description:"List the project's StepSet definitions (named reusable step sequences from .saffron files) with their steps, defining file, and usage. Invoke one inside a scenario with a line reading exactly: StepSet <name>",inputSchema:{type:"object",properties:{}}}]})),r.setRequestHandler(Kr,async n=>{let s=G(t,e);if(n.params.name==="search_steps"){let o=n.params.arguments?.query,i=o?me(s,o):s;return{content:[{type:"text",text:JSON.stringify({steps:i.steps,scannedFiles:i.scannedFiles,scannedCaches:i.scannedCaches},null,2)}]}}return n.params.name==="list_step_sets"?{content:[{type:"text",text:JSON.stringify({stepSets:s.stepSets},null,2)}]}:{content:[{type:"text",text:`Unknown tool: ${n.params.name}`}],isError:!0}}),await r.connect(new Qr)}import{query as Zr}from"@anthropic-ai/claude-agent-sdk";function en(t,e){let r=e.steps.map(s=>`- [${s.status}] ${s.keyword} ${s.text}`).join(`
|
|
7
|
+
`),n=e.stepSets.map(s=>`- StepSet ${s.name}
|
|
8
8
|
${s.steps.map(o=>` ${o}`).join(`
|
|
9
9
|
`)}`).join(`
|
|
10
|
-
`);return["Draft a Saffron feature file from the requirements below.","","EXISTING STEP VOCABULARY \u2014 reuse these wordings EXACTLY (character for character) wherever they express what a step needs to do. Steps marked [recorded] or [divergent] already have replay recordings: reusing them makes the new file replay almost for free. Only invent a new wording when nothing in the vocabulary expresses the intent, and then follow the same style:",
|
|
11
|
-
${
|
|
12
|
-
`)}async function
|
|
13
|
-
`,usage:
|
|
14
|
-
`),h=new Map;for(let u=0;u<g.length;u++){let f=g[u],y={start:{line:u,character:0},end:{line:u,character:f.length}},x=f.match(
|
|
15
|
-
`)}));if(/^\s*(Given|When|Then|And|But|\*)\s+/.test(g))return s.steps.map((f,y)=>({label:f.text,kind:
|
|
16
|
-
`),filterText:`step set ${y.name}`,textEdit:{range:f,newText:`StepSet ${y.name}`}}))}return[]}),
|
|
17
|
-
`);return{contents:{kind:
|
|
10
|
+
`);return["Draft a Saffron feature file from the requirements below.","","EXISTING STEP VOCABULARY \u2014 reuse these wordings EXACTLY (character for character) wherever they express what a step needs to do. Steps marked [recorded] or [divergent] already have replay recordings: reusing them makes the new file replay almost for free. Only invent a new wording when nothing in the vocabulary expresses the intent, and then follow the same style:",r.length>0?r:"(none yet \u2014 this is the project's first file)","",e.stepSets.length>0?`EXISTING STEP SETS \u2014 invoke with a line reading exactly \`StepSet <name>\` instead of repeating their steps:
|
|
11
|
+
${n}`:void 0,"","RULES:","1. Output ONLY the feature file content \u2014 no prose, no code fences, no explanations.","2. Gherkin/.saffron syntax: Feature:, Scenario:, Scenario Outline: + Examples, Given/When/Then/And/But. Prefer concrete scenarios; use an Outline only when the requirements list variations of the same flow.","3. Then steps are assertions \u2014 state exactly what must be true, one condition per step. End every scenario with at least one Then.","4. If the same 3+ step sequence would appear in two or more scenarios, define it once as a StepSet (first step a guard, last step an exit assertion) and invoke it. Definition uses a colon (StepSet: Name), invocation does not (StepSet Name).",'5. Use 2-column data tables for key/value input, header tables for lists of records, and """ doc strings for free text.',"6. Steps are natural language for an AI browser agent \u2014 never CSS selectors or code.","","REQUIREMENTS:",t.trim()].filter(s=>s!==void 0).join(`
|
|
12
|
+
`)}async function wt(t,e,r){let n={aiCalls:0,inputTokens:0,outputTokens:0,cacheReadTokens:0,cacheCreationTokens:0,costUsd:0},s="",o=Zr({prompt:en(t,e),options:{model:r,maxTurns:1,permissionMode:"bypassPermissions",persistSession:!1,settingSources:[],allowedTools:[],systemPrompt:"You are Saffron's feature-file author. You write clean, assertive Gherkin that reuses the project's exact step vocabulary. You output only file content."}});for await(let i of o)if(i.type==="assistant")for(let a of i.message.content)a.type==="text"&&(s+=a.text);else i.type==="result"&&(n.aiCalls=i.num_turns,n.costUsd=i.total_cost_usd,"usage"in i&&i.usage&&(n.inputTokens=i.usage.input_tokens??0,n.outputTokens=i.usage.output_tokens??0,n.cacheReadTokens=i.usage.cache_read_input_tokens??0,n.cacheCreationTokens=i.usage.cache_creation_input_tokens??0));return s=s.replace(/^\s*```(?:gherkin|saffron)?\s*\n/,"").replace(/\n```\s*$/,"").trim(),{content:s+`
|
|
13
|
+
`,usage:n}}import he from"node:path";import{createConnection as tn,ProposedFeatures as rn,TextDocuments as nn,TextDocumentSyncKind as sn,CompletionItemKind as Ue,DiagnosticSeverity as ye,MarkupKind as kt}from"vscode-languageserver/node";import{TextDocument as on}from"vscode-languageserver-textdocument";var $t=/^(\s*)(Given|When|Then|And|But|\*)\s+(.+?)\s*$/,an=/^\s*StepSet:\s*(.+?)\s*$/,Ne=/^(\s*)(StepSet)\s+([^\s:].*?)\s*$/,Be={recorded:"\u25CF recorded \u2014 replays at zero tokens",divergent:"\u25CF divergent \u2014 same text, different recordings",unrecorded:"\u25CB written, not recorded yet"};function cn(t,e){let r=i=>new Set(i.toLowerCase().replace(/"[^"]*"/g,'"x"').split(/[^a-z0-9"<>]+/).filter(Boolean)),n=r(t),s=r(e);if(n.size===0||s.size===0)return 0;let o=0;for(let i of n)s.has(i)&&o++;return o/(n.size+s.size-o)}function Rt(t,e){let r=tn(rn.all,process.stdin,process.stdout),n=new nn(on),s={steps:[],stepSets:[],scannedFiles:0,scannedCaches:0},o=new Map,i,a=()=>{try{let l=he.isAbsolute(e)?e:he.join(t,e);o=X(ue(l),t),s=G(t,e,o)}catch(l){r.console.warn(`saffron vocabulary rebuild: ${String(l)}`)}for(let l of n.all())m(l)},d=()=>{clearTimeout(i),i=setTimeout(a,300)};function p(l){return s.steps.find(c=>c.text===l)?.status}function m(l){if(!l.uri.endsWith(".saffron"))return;let c=[],g=l.getText().split(`
|
|
14
|
+
`),h=new Map;for(let u=0;u<g.length;u++){let f=g[u],y={start:{line:u,character:0},end:{line:u,character:f.length}},x=f.match(Ne);x&&!o.has(x[3])&&c.push({range:y,severity:ye.Error,source:"saffron",message:`StepSet "${x[3]}" is not defined anywhere in the project.`});let w=f.match(an);if(w){let b=o.get(w[1]),j=he.relative(t,new URL(l.uri).pathname).replace(/\\/g,"/");(h.has(w[1])||b&&b.file!==j)&&c.push({range:y,severity:ye.Error,source:"saffron",message:`StepSet "${w[1]}" is defined twice \u2014 set names are project-wide. If this line was meant to INVOKE the set, remove the colon: "StepSet ${w[1]}".`}),h.set(w[1],u)}let F=f.trim();if(F&&!x&&!w){for(let[b]of o)if(F.toLowerCase()===b.toLowerCase()){c.push({range:y,severity:ye.Warning,source:"saffron",message:`Did you mean "StepSet ${b}"? Invoking a step set needs the StepSet keyword, like a step needs Given/When/Then.`});break}}let v=f.match($t);if(v&&p(v[3])!=="recorded"){for(let b of s.steps)if(!(b.text===v[3]||b.status==="unrecorded")&&cn(b.text,v[3])>=.7){c.push({range:{start:{line:u,character:f.indexOf(v[3])},end:{line:u,character:f.length}},severity:ye.Information,source:"saffron",message:`Similar to the ${b.status} step "${b.text}" (\xD7${b.usage}). Reusing the exact wording replays at zero tokens; a new wording costs a fresh AI recording.`});break}}}r.sendDiagnostics({uri:l.uri,diagnostics:c})}r.onInitialize(()=>(a(),{capabilities:{textDocumentSync:sn.Incremental,completionProvider:{triggerCharacters:[" "]},definitionProvider:!0,hoverProvider:!0}})),r.onCompletion(l=>{let c=n.get(l.textDocument.uri);if(!c)return[];let g=c.getText({start:{line:l.position.line,character:0},end:l.position});if(/^\s*StepSet\s+/.test(g)&&c.uri.endsWith(".saffron"))return s.stepSets.map(f=>({label:f.name,kind:Ue.Function,detail:`step set \xB7 ${f.steps.length} step(s) \xB7 \xD7${f.usage}`,documentation:f.steps.join(`
|
|
15
|
+
`)}));if(/^\s*(Given|When|Then|And|But|\*)\s+/.test(g))return s.steps.map((f,y)=>({label:f.text,kind:Ue.Text,detail:`${Be[f.status]} \xB7 \xD7${f.usage}`,sortText:String(y).padStart(5,"0"),filterText:f.text}));let u=g.replace(/^\s*/,"").trimEnd().toLowerCase().replace(/\s+/g,"");if(u.length>0&&("stepset".startsWith(u)||u.startsWith("stepset"))&&c.uri.endsWith(".saffron")){let f={start:{line:l.position.line,character:g.length-g.replace(/^\s*/,"").length},end:l.position};return s.stepSets.map(y=>({label:`StepSet ${y.name}`,kind:Ue.Function,detail:`invoke step set \xB7 ${y.steps.length} step(s)`,documentation:y.steps.join(`
|
|
16
|
+
`),filterText:`step set ${y.name}`,textEdit:{range:f,newText:`StepSet ${y.name}`}}))}return[]}),r.onDefinition(l=>{let c=n.get(l.textDocument.uri);if(!c)return;let h=c.getText({start:{line:l.position.line,character:0},end:{line:l.position.line+1,character:0}}).replace(/\n$/,"").match(Ne);if(!h)return;let u=o.get(h[3]);return u?{uri:`file://${he.join(t,u.file)}`,range:{start:{line:u.line-1,character:0},end:{line:u.line-1,character:0}}}:void 0}),r.onHover(l=>{let c=n.get(l.textDocument.uri);if(!c)return;let g=c.getText({start:{line:l.position.line,character:0},end:{line:l.position.line+1,character:0}}).replace(/\n$/,""),h=g.match(Ne);if(h){let f=s.stepSets.find(x=>x.name===h[3]);if(!f)return;let y=f.steps.map(x=>`- ${Be[p(x)??"unrecorded"][0]} ${x}`).join(`
|
|
17
|
+
`);return{contents:{kind:kt.Markdown,value:`**StepSet ${f.name}** \u2014 expands to:
|
|
18
18
|
|
|
19
|
-
${y}`}}}let u=g.match(
|
|
19
|
+
${y}`}}}let u=g.match($t);if(u){let f=s.steps.find(y=>y.text===u[3]);return f?{contents:{kind:kt.Markdown,value:`${Be[f.status]}
|
|
20
20
|
|
|
21
21
|
Used in ${f.usage} place(s): ${f.files.join(", ")}${f.fromStepSet?`
|
|
22
22
|
|
|
23
|
-
Defined in StepSet **${f.fromStepSet}**`:""}`}}:void 0}}),
|
|
24
|
-
`)){let r
|
|
25
|
-
`)}function xe(t){return JSON.parse(B.readFileSync(t,"utf8"))}function ce(t){let e=Se.join(t,".saffron","proposals");if(!B.existsSync(e))return[];let
|
|
26
|
-
`).filter(Boolean).map(f=>
|
|
27
|
-
`)
|
|
28
|
-
`)
|
|
29
|
-
`)
|
|
30
|
-
`)
|
|
31
|
-
`)}}}import{createSdkMcpServer as Tr,tool as ne}from"@anthropic-ai/claude-agent-sdk";import{z as R}from"zod";var D=t=>({content:[{type:"text",text:t}]});function Ht(t){let e={};return t.role&&(e.role=t.role),t.name&&(e.name=t.name),t.nameRegex&&(e.nameRegex=t.nameRegex),t.label&&(e.label=t.label),t.text&&(e.text=t.text),t.selector&&(e.fallbackSelectors=[t.selector]),t.frame&&(e.frame=t.frame),Object.keys(e).length>0?e:void 0}var Gt={role:R.string().optional().describe("ARIA role of the element"),name:R.string().optional().describe("Accessible name \u2014 use the SHORTEST stable substring; never include volatile content like prices or dates"),nameRegex:R.string().optional().describe("Regex for the accessible name when it embeds volatile content, e.g. 'Select \\\\{date\\\\+1\\\\} as checkin' cannot work \u2014 instead use nameRegex 'Select .* as checkin'"),label:R.string().optional().describe("Form-label text for getByLabel \u2014 the literal <label> text only, never a free-text description"),text:R.string().optional().describe("getByText locator text"),selector:R.string().optional().describe("CSS fallback selector"),frame:R.string().optional().describe(`CSS selector of the iframe hosting the element, e.g. 'iframe[title="Feedback widget"]' \u2014 REQUIRED whenever the element lives inside an iframe; without it replay looks only at the top-level page`)};function Jt(t){return Tr({name:"saffron",version:"0.1.0",tools:[ne("saffron_step","Announce the Gherkin step you are about to perform. MUST be called before executing browser actions for each step, with the step's zero-based index and exact text.",{index:R.number().int().min(0),text:R.string(),adaptation:R.string().optional().describe("If the written step does not match the real UI and you are deviating from it, describe the deviation in one sentence.")},async e=>(t.currentStepIndex=e.index,t.announced.add(e.index),e.adaptation&&t.adaptations.push(`Step ${e.index+1} ("${e.text}"): ${e.adaptation}`),D(`Recording step ${e.index}: ${e.text}`))),ne("saffron_capture",`Record a captureText action for the current step: at replay time, the target element's text is read and stored under saveAs for later expectDiffers comparisons. Use for steps like 'I record the displayed price as "X"'. The target must select exactly the element whose text is the value.`,{saveAs:R.string().describe("Name to store the captured text under"),...Gt},async e=>{let n=Ht(e);return n?(t.addActions([{action:"captureText",target:n,saveAs:e.saveAs}]),D(`Capture recorded as "${e.saveAs}".`)):D("ERROR: capture needs a target (role/name, nameRegex, label, text, or selector).")}),ne("saffron_dialog","Record how a native dialog (alert/confirm/prompt) must be handled at replay. Call this IMMEDIATELY whenever an action triggers a native dialog \u2014 whether you handled it via browser_handle_dialog or the browser reported it as already handled. At replay Saffron arms this handler BEFORE the triggering action, so without this call the replay will auto-dismiss the dialog and diverge from what you saw.",{accept:R.boolean().describe("true = accept/OK the dialog, false = dismiss/cancel"),promptText:R.string().optional().describe("Text typed into a prompt() before accepting")},async e=>(t.addArmedAction({action:"handleDialog",value:e.accept?"accept":"dismiss",...e.promptText?{promptText:e.promptText}:{}}),D(`Dialog handling recorded (${e.accept?"accept":"dismiss"}).`))),ne("saffron_wait_api","Record a network wait for the current step: replay will block until a matching API response is observed (responses from the triggering step count). Call AFTER you verified via browser_network_requests that the response actually occurred with the expected status/body. Use a path-only urlPattern regex and never literal volatile IDs (/api/orders/\\d+, not /api/orders/8842).",{urlPattern:R.string().describe("Regex matched against the response URL \u2014 path-only for portability"),method:R.string().optional().describe("HTTP method to match (any if omitted)"),status:R.string().optional().describe("Expected status \u2014 exact ('201') or class ('2xx'); default any 2xx"),bodyPattern:R.string().optional().describe(`Regex the response body must match, e.g. '"status"\\s*:\\s*"READY"' for polling waits`)},async e=>(t.addActions([{action:"waitForResponse",urlPattern:e.urlPattern,...e.method?{method:e.method}:{},...e.status?{status:e.status}:{},...e.bodyPattern?{pattern:e.bodyPattern}:{}}]),D("Network wait recorded."))),ne("saffron_assert","Record a verified assertion for the current Then step. Call ONLY after you have confirmed the condition holds via a page snapshot. If the condition does NOT hold, do not call this \u2014 call saffron_done with success=false instead. NEVER record volatile content (prices, dates, counters) as literal expected text \u2014 use kind=matches with a pattern, or kind=differs against captured values.",{kind:R.enum(["visible","notVisible","text","url","value","differs","matches","attribute","response"]),...Gt,expected:R.string().optional().describe("Expected text/value for kind=text|value \u2014 stable text only, never volatile values"),url:R.string().optional().describe("URL substring for kind=url (path only, no host)"),pattern:R.string().optional().describe("kind=matches|attribute: regex the target's text (or attribute) must match, e.g. 'NOK [\\\\d,]+' for a price"),attribute:R.string().optional().describe("kind=attribute: attribute name to check, e.g. 'href' for link-target assertions \u2014 use this for 'X should link to Y' instead of clicking through"),capture:R.string().optional().describe("kind=differs: name of the captured value on the LEFT of the comparison (omit when comparing the target element's live text)"),urlPattern:R.string().optional().describe("kind=response: regex matched against the response URL \u2014 path-only for portability, never literal volatile IDs (use /api/orders/\\d+)"),method:R.string().optional().describe("kind=response: HTTP method to match (any if omitted)"),status:R.string().optional().describe("kind=response: expected status \u2014 exact ('201') or class ('2xx'); default any 2xx"),bodyPattern:R.string().optional().describe("kind=response: regex the response body must match (verify via browser_network_requests first)"),compareTo:R.string().optional().describe("kind=differs: name of the captured value to compare against")},async e=>{let n=Ht(e),r;switch(e.kind){case"visible":r={action:"expectVisible",target:n};break;case"notVisible":r={action:"expectNotVisible",target:n};break;case"text":r={action:"expectText",target:n,value:e.expected??""};break;case"value":r={action:"expectValue",target:n,value:e.expected??""};break;case"url":r={action:"expectUrl",url:e.url??""};break;case"differs":if(!e.compareTo)return D("ERROR: kind=differs requires compareTo.");if(!e.capture&&!n)return D("ERROR: kind=differs needs either capture (a stored name) or a target element.");r={action:"expectDiffers",target:n,value:e.capture,compareTo:e.compareTo};break;case"matches":if(!e.pattern)return D("ERROR: kind=matches requires pattern.");r={action:"expectMatches",target:n,pattern:e.pattern};break;case"attribute":if(!e.attribute||!e.pattern)return D("ERROR: kind=attribute requires attribute and pattern.");r={action:"expectAttribute",target:n,attribute:e.attribute,pattern:e.pattern};break;case"response":if(!e.urlPattern)return D("ERROR: kind=response requires urlPattern.");r={action:"expectResponse",urlPattern:e.urlPattern,...e.method?{method:e.method}:{},...e.status?{status:e.status}:{},...e.bodyPattern?{pattern:e.bodyPattern}:{}};break}return!["url","differs","response"].includes(e.kind)&&!n?D("ERROR: assertion needs a target (role/name, nameRegex, label, text, or selector)."):(t.addActions([r]),D("Assertion recorded."))}),ne("saffron_done","Finalize the run. Call exactly once, after the final step: success=true only if every step's intent was achieved and every assertion verified.",{success:R.boolean(),narrative:R.string().describe("2-4 sentence summary of the run: what happened, and why it failed if it did."),suggestedFeatureEdit:R.string().optional().describe("If you adapted any steps: the corrected Gherkin for those steps, as you would rewrite them in the .feature file (free text, for the human report)."),featureEdits:R.array(R.object({index:R.number().int().min(0).describe("zero-based step index"),text:R.string().describe("replacement step text WITHOUT the Given/When/Then keyword")})).optional().describe("Machine-applicable version of the suggested edits: one entry per adapted step. Provide this whenever you adapted a step.")},async e=>(t.done=!0,t.success=e.success,t.narrative=e.narrative,t.suggestedFeatureEdit=e.suggestedFeatureEdit,t.featureEdits=e.featureEdits??[],D("Run finalized.")))]})}function Yt(t={}){let e=t.deliberateSnapshots?`
|
|
23
|
+
Defined in StepSet **${f.fromStepSet}**`:""}`}}:void 0}}),n.onDidOpen(l=>m(l.document)),n.onDidChangeContent(l=>m(l.document)),n.onDidSave(()=>d()),r.onDidChangeWatchedFiles(()=>d()),n.listen(r),r.listen()}import Tt from"node:fs";import ln from"node:path";var Et=/\{env:([A-Za-z_][A-Za-z0-9_]*)\}/g;function ie(t){return t.replace(Et,(e,r)=>{let n=process.env[r];if(n===void 0)throw new Error(`environment variable ${r} is not set (referenced as {env:${r}}) \u2014 export it or add it to .env`);return n})}function Ve(t){let e=new Set;for(let r of t)if(r)for(let n of r.matchAll(Et))e.add(n[1]);return e}function ze(t){let e=[];for(let r of t.steps){e.push(r.text,r.resolvedText),r.docString&&e.push(r.docString);for(let n of r.table??[])e.push(...n)}return e}function ae(t){let e=new Map;for(let r of Ve(ze(t))){let n=process.env[r];n!==void 0&&e.set(r,n)}return e}function te(t,e){let r=t;for(let[n,s]of[...e.entries()].sort((o,i)=>i[1].length-o[1].length))s.length<4||(r=r.split(s).join(`{env:${n}}`));return r}function At(t){let e=ln.join(t,".env");if(Tt.existsSync(e))for(let r of Tt.readFileSync(e,"utf8").split(`
|
|
24
|
+
`)){let n=r.trim();if(!n||n.startsWith("#"))continue;let s=n.indexOf("=");if(s<=0)continue;let o=n.slice(0,s).trim();if(!/^[A-Za-z_][A-Za-z0-9_]*$/.test(o))continue;let i=n.slice(s+1).trim();(i.startsWith('"')&&i.endsWith('"')||i.startsWith("'")&&i.endsWith("'"))&&(i=i.slice(1,-1)),process.env[o]===void 0&&(process.env[o]=i)}}import kn from"node:fs";import $n from"node:net";import qt from"node:path";import{chromium as Rn,firefox as Tn,webkit as En}from"playwright";import B from"node:fs";import Se from"node:path";function Ct(t,e){B.mkdirSync(Se.dirname(t),{recursive:!0}),B.writeFileSync(t,`${JSON.stringify(e,null,2)}
|
|
25
|
+
`)}function xe(t){return JSON.parse(B.readFileSync(t,"utf8"))}function ce(t){let e=Se.join(t,".saffron","proposals");if(!B.existsSync(e))return[];let r=[];for(let n of B.readdirSync(e)){let s=Se.join(e,n);if(B.statSync(s).isDirectory())for(let o of B.readdirSync(s)){if(!o.endsWith(".json"))continue;let i=Se.join(s,o);r.push({file:i,proposal:xe(i)})}}return r}function Pt(t,e){let r=xe(e),n=z(t,r.cache.feature,r.cache.scenario);return ee(n,r.cache),B.rmSync(e),n}function Ft(t){B.rmSync(t)}import jt from"node:path";var dn=/\{date([+-]\d+)?(?::([^}]+))?\}/g;function pn(t,e){let r=String(t.getFullYear()),n=String(t.getMonth()+1).padStart(2,"0"),s=String(t.getDate()).padStart(2,"0");return e.replaceAll("YYYY",r).replaceAll("MM",n).replaceAll("DD",s).replaceAll("M",String(t.getMonth()+1)).replaceAll("D",String(t.getDate()))}function qe(t,e=new Date){return t.replace(dn,(r,n,s)=>{let o=new Date(e);return o.setDate(o.getDate()+(n?parseInt(n,10):0)),pn(o,s??"YYYY-MM-DD")})}var un=/\b(\d{4})-(\d{2})-(\d{2})\b/g,fn=400;function It(t,e=new Date){let r=new Date(e.getFullYear(),e.getMonth(),e.getDate());return t.replace(un,(n,s,o,i)=>{let a=new Date(Number(s),Number(o)-1,Number(i));if(Number.isNaN(a.getTime()))return n;let d=Math.round((a.getTime()-r.getTime())/864e5);return Math.abs(d)>fn?n:d===0?"{date}":`{date${d>0?"+":""}${d}}`})}import{createHash as gn}from"node:crypto";async function J(t){let e;try{e=await t.locator("body").ariaSnapshot({timeout:3e3})}catch{return"sha256:unavailable"}let r=e.replace(/\d+/g,"#").replace(/\s+/g," ").trim();return`sha256:${gn("sha256").update(r).digest("hex")}`}var Ge=t=>new Promise(e=>setTimeout(e,t)),He=class{entries=[];seq=0;prevStepSeq=0;curStepSeq=0;listener;context;attach(e,r){this.context=e,this.listener=n=>{let s={seq:++this.seq,url:n.url(),method:n.request().method(),status:n.status()};if(this.entries.push(s),r){let o=n.headers()["content-type"]??"";/json|text|xml/i.test(o)&&n.text().then(i=>{s.body=i.length>262144?i.slice(0,262144):i}).catch(()=>{})}},e.on("response",this.listener)}detach(){this.context&&this.listener&&this.context.off("response",this.listener)}markStep(){this.prevStepSeq=this.curStepSeq,this.curStepSeq=this.seq}sincePreviousStep(){return this.entries.filter(e=>e.seq>this.prevStepSeq)}all(){return this.entries}};function mn(t,e){if(!e)return t>=200&&t<300;let r=/^([1-5])xx$/i.exec(e.trim());if(r){let n=Number(r[1])*100;return t>=n&&t<n+100}return t===Number(e.trim())}function _t(t,e,r,n,s){return!(!e.test(t.url)||r&&t.method.toUpperCase()!==r.toUpperCase()||!mn(t.status,n)||s&&!s.test(t.body??""))}var hn=(t,e)=>`response matching ${t.method??"any-method"} /${e}/ status ${t.status??"2xx"}${t.pattern?` body /${t.pattern}/`:""}`;function Dt(t,e,r,n=!1){let s=a=>ie(qe(q(a,r))),o=[],i=a=>{if(e.role){let d=e.nameRegex?new RegExp(s(e.nameRegex),"i"):e.name?s(e.name):void 0;o.push(a.getByRole(e.role,{name:d,exact:!1}))}e.label&&o.push(a.getByLabel(s(e.label))),e.placeholder&&o.push(a.getByPlaceholder(s(e.placeholder))),e.text&&o.push(a.getByText(s(e.text))),e.testId&&o.push(a.getByTestId(s(e.testId)));for(let d of e.fallbackSelectors??[])o.push(a.locator(d))};if(e.frame)i(t.frameLocator(s(e.frame)));else if(i(t),n)for(let a of t.frames())a!==t.mainFrame()&&i(a);if(o.length===0)throw new Error("Action target has neither role nor fallback selectors");return o}async function V(t,e,r){let n=Date.now()+e,s;for(;Date.now()<n;){try{if(await t())return}catch(o){s=o}await Ge(100)}throw new Error(`Timed out waiting for ${r}${s?` (${String(s)})`:""}`)}async function U(t,e){let r;for(let n of t)try{await e(n.first());return}catch(s){r=s}throw r}var We=t=>t.replace(/\s+/g," ").trim();async function Je(t,e,r,n,s,o=new Map,i){let a=u=>ie(qe(q(u,r))),d=e.value!==void 0?a(e.value):void 0,p=e.url!==void 0?a(e.url):void 0,m=e.action==="expectVisible"||e.action==="expectText"||e.action==="expectValue"||e.action==="expectMatches"||e.action==="expectAttribute"||e.action==="captureText",l=e.target?Dt(t,e.target,r,m):[],c={timeout:n},g={timeout:Math.min(1e3,n)},h=u=>async()=>{for(let f of l)try{if(await u(f.first()))return!0}catch{}return!1};switch(e.action){case"goto":{if(!p)throw new Error("goto requires url");let u=/^https?:\/\//.test(p)?p:new URL(p,s).toString();await t.goto(u,c);return}case"click":return U(l,u=>u.click(c));case"fill":return U(l,u=>u.fill(d??"",c));case"press":return l.length>0?U(l,u=>u.press(d??"",c)):t.keyboard.press(d??"");case"selectOption":return U(l,async u=>{await u.selectOption(d??"",c)});case"check":return U(l,u=>u.check(c));case"uncheck":return U(l,u=>u.uncheck(c));case"hover":return U(l,u=>u.hover(c));case"waitFor":return l.length>0?U(l,u=>u.waitFor({state:"visible",timeout:n})):Ge(Math.min(Number(d??"0"),n));case"expectVisible":return V(h(u=>u.isVisible()),n,"target to be visible");case"expectNotVisible":return U(l,u=>u.waitFor({state:"hidden",timeout:n}));case"expectText":return V(h(async u=>(await u.textContent(g)??"").includes(d??"")),n,`text "${d}"`);case"expectValue":return V(h(async u=>await u.inputValue(g)===(d??"")),n,`input value "${d}"`);case"expectUrl":{let u=p??d??"";return V(async()=>t.context().pages().some(f=>f.url().includes(u)),n,`URL containing "${u}" in any tab (current: ${t.context().pages().map(f=>f.url()).join(", ")})`)}case"handleDialog":{let u=(d??"accept")!=="dismiss",f=e.promptText?a(e.promptText):void 0;t.once("dialog",y=>{u?y.accept(f):y.dismiss()});return}case"uploadFile":{let u=(d??"").split(`
|
|
26
|
+
`).filter(Boolean).map(f=>jt.isAbsolute(f)?f:jt.resolve(f));if(u.length===0)throw new Error("uploadFile requires a path");if(l.length>0)return U(l,f=>f.setInputFiles(u,c));t.once("filechooser",f=>{f.setFiles(u)});return}case"dragTo":{if(!e.toTarget)throw new Error("dragTo requires toTarget");let u=Dt(t,e.toTarget,r);return U(l,f=>f.dragTo(u[0].first(),c))}case"captureText":{if(!e.saveAs)throw new Error("captureText requires saveAs");return V(h(async u=>{let f=We(await u.textContent(g)??"");return f===""?!1:(o.set(e.saveAs,f),!0)}),n,`non-empty text to capture as "${e.saveAs}"`)}case"expectDiffers":{if(!e.compareTo)throw new Error("expectDiffers requires compareTo");let u=o.get(e.compareTo);if(u===void 0)throw new Error(`expectDiffers: no captured value named "${e.compareTo}"`);if(l.length>0)return V(h(async x=>We(await x.textContent(g)??"")!==u),n,`target text to differ from "${e.compareTo}" (${u})`);let f=d??"",y=o.get(f);if(y===void 0)throw new Error(`expectDiffers: no captured value named "${f}"`);if(y===u)throw new Error(`expectDiffers: "${f}" (${y}) equals "${e.compareTo}" (${u})`);return}case"expectAttribute":{if(!e.attribute)throw new Error("expectAttribute requires attribute");if(!e.pattern)throw new Error("expectAttribute requires pattern");let u=new RegExp(e.pattern,"i");return V(h(async f=>u.test(await f.getAttribute(e.attribute,g)??"")),n,`attribute ${e.attribute} to match /${e.pattern}/i`)}case"expectMatches":{if(!e.pattern)throw new Error("expectMatches requires pattern");let u=new RegExp(e.pattern,"i");return V(h(async f=>u.test(We(await f.textContent(g)??""))),n,`target text to match /${e.pattern}/i`)}case"waitForResponse":case"expectResponse":{if(!e.urlPattern)throw new Error(`${e.action} requires urlPattern`);let u=new RegExp(a(e.urlPattern)),f=e.pattern?new RegExp(a(e.pattern)):void 0;if(!i){await t.waitForResponse(x=>_t({seq:0,url:x.url(),method:x.request().method(),status:x.status()},u,e.method,e.status,void 0),c);return}let y=()=>e.action==="expectResponse"?i.all():i.sincePreviousStep();return V(async()=>y().some(x=>_t(x,u,e.method,e.status,f)),n,hn(e,e.urlPattern))}default:throw new Error(`Unknown action type: ${e.action}`)}}function Ot(t,e){return t.actions.some(r=>[r.value,r.url,r.pattern,r.target?.name,r.target?.nameRegex,r.target?.label,r.target?.text,r.target?.placeholder].some(n=>n?.includes(e)))}function yn(t,e){return e.steps.length!==t.steps.length?!1:e.steps.every((r,n)=>{let s=t.steps[n];if(r.gherkin!==s.text)return!1;let o=r.table,i=s.table;if(!!o!=!!i)return!1;if(o&&i){let a=Object.keys(L(o)),d=Object.keys(L(i));if(a.length>0&&d.length>0){if(JSON.stringify(a)!==JSON.stringify(d))return!1}else if(JSON.stringify(o)!==JSON.stringify(i))return!1}if(r.docString!==void 0!=(s.docString!==void 0))return!1;if(r.docString!==void 0&&s.docString!==void 0&&r.docString!==s.docString){let a=e.steps.some(p=>Ot(p,"<docstring>")),d=e.steps.some(p=>Ot(p,r.docString));if(!a||d)return!1}return!0})}function Ye(t,e){let r=t.steps[e],n=r?.resolvedTable??r?.table;return{...t.params,...K(t.steps),...L(n),...W(r?.resolvedDocString??r?.docString)}}async function be(t,e,r,n={}){let s=n.actionTimeoutMs??5e3,o=n.retries??1,i=[],a=n.vars??new Map,d=()=>Object.fromEntries(a),p=r.steps.some(g=>g.actions.some(h=>(h.action==="waitForResponse"||h.action==="expectResponse")&&h.pattern!==void 0)),m=new He;m.attach(t.context(),p);let l=t,c=()=>{let g=t.context().pages(),h=g[g.length-1];h&&h!==l&&!h.isClosed()&&(l=h)};if(!yn(e,r))return m.detach(),{status:"stale",stepResults:i,captured:d()};try{for(let g=0;g<r.steps.length;g++){let h=r.steps[g],u=Ye(e,g);m.markStep();let f=!1,y=0;for(let x=0;x<h.actions.length;x++){let w=h.actions[x];c(),w.pageFingerprint&&await J(l)!==w.pageFingerprint&&(f=!0);let F,v=!1;for(let b=0;b<=o&&!v;b++){b>0&&(y++,await Ge(250*b),c());try{await Je(l,w,u,s,n.baseURL,a,m),v=!0}catch(j){F=j}}if(!v){let b=F instanceof Error?F.message:String(F);i.push({gherkin:h.gherkin,fromStepSet:h.fromStepSet,kind:h.kind,status:"failed",drift:f,...y?{retries:y}:{},error:b});for(let j=g+1;j<r.steps.length;j++)i.push({gherkin:r.steps[j].gherkin,kind:r.steps[j].kind,status:"skipped"});return{status:"failed",failure:{stepIndex:g,actionIndex:x,stepText:h.gherkin,kind:h.kind,error:b},stepResults:i,captured:d()}}}i.push({gherkin:h.gherkin,fromStepSet:h.fromStepSet,kind:h.kind,status:"passed",drift:f,...y?{retries:y}:{}})}return{status:"passed",stepResults:i,captured:d()}}finally{m.detach()}}var ve=class{steps;currentStepIndex=-1;done=!1;success=!1;narrative="";adaptations=[];suggestedFeatureEdit;featureEdits=[];announced=new Set;pendingFingerprint;constructor(e){this.steps=e.steps.map(r=>({gherkin:r.text,keyword:r.keyword,kind:r.kind,table:r.table,docString:r.docString,fromStepSet:r.source?.fromStepSet,actions:[]}))}addArmedAction(e){if(this.done)return;let r=Math.max(this.currentStepIndex,0),n=this.steps[r];if(!n||n.actions.some(i=>i.action===e.action&&i.value===e.value&&i.promptText===e.promptText))return;let o=Math.max(n.actions.length-1,0);n.actions.splice(o,0,e)}addActions(e){if(this.done||e.length===0)return;let r=Math.max(this.currentStepIndex,0),n=this.steps[r];n&&(this.pendingFingerprint&&e[0]&&(e[0].pageFingerprint=this.pendingFingerprint,this.pendingFingerprint=void 0),n.actions.push(...e))}},Y=t=>t.replace(/\\(['"\\])/g,"$1");function Sn(t){let e=t.match(/'((?:[^'\\]|\\.)*)'/);return e?Y(e[1]):void 0}function Qe(t){let e={},r=t.match(/(?:frameLocator\('((?:[^'\\]|\\.)*)'\)|locator\('((?:[^'\\]|\\.)*)'\)\.contentFrame\(\))/);r&&(e.frame=Y(r[1]??r[2]),t=t.slice((r.index??0)+r[0].length));let n=t.match(/getByRole\('([^']+)'(?:,\s*\{([^}]*)\})?\)/);if(n){e.role=n[1];let i=(n[2]??"").match(/name:\s*'((?:[^'\\]|\\.)*)'/);i&&(e.name=Y(i[1]))}let s=[[/getByLabel\('((?:[^'\\]|\\.)*)'\)/,"label"],[/getByText\('((?:[^'\\]|\\.)*)'\)/,"text"],[/getByTestId\('((?:[^'\\]|\\.)*)'\)/,"testId"],[/getByPlaceholder\('((?:[^'\\]|\\.)*)'\)/,"placeholder"]];for(let[i,a]of s){let d=t.match(i);d&&(e[a]=Y(d[1]))}let o=t.match(/(?:^|\.)locator\('((?:[^'\\]|\\.)*)'\)/);return o&&(e.fallbackSelectors=[Y(o[1])]),Object.keys(e).length>0?e:void 0}function xn(t,e){if(e&&t.startsWith(e)){let r=t.slice(e.length);return r.startsWith("/")?r:`/${r}`}return t}var bn={click:"click",dblclick:"click",fill:"fill",pressSequentially:"fill",type:"fill",press:"press",check:"check",uncheck:"uncheck",hover:"hover",selectOption:"selectOption"};function vn(t,e){let r=t.match(/page\.goto\('((?:[^'\\]|\\.)*)'\)/);if(r)return{action:"goto",url:xn(Y(r[1]),e)};let n=t.match(/page\.keyboard\.press\('((?:[^'\\]|\\.)*)'\)/);if(n)return{action:"press",value:Y(n[1])};let s=t.match(/await\s+page\.(.+)\.dragTo\(page\.(.+?)\);?\s*$/);if(s){let c=Qe(s[1]),g=Qe(s[2]);if(c&&g)return{action:"dragTo",target:c,toTarget:g}}let o=t.match(/await\s+page\.(.+)\.(click|dblclick|fill|pressSequentially|type|press|check|uncheck|hover|selectOption)\((.*)\);?\s*$/);if(!o)return;let[,i,a,d]=o,p=Qe(i);if(!p)return;let m={action:bn[a],target:p},l=Sn(d);return l!==void 0&&a!=="click"&&a!=="dblclick"&&(m.value=l),m}function Mt(t,e){let r=[],n=Array.isArray(t)?t:t?.content;if(Array.isArray(n))for(let o of n){let i=o?.text;typeof i=="string"&&r.push(i)}else typeof t=="string"&&r.push(t);let s=[];for(let o of r)for(let i of o.split(`
|
|
27
|
+
`)){let a=vn(i.trim(),e);a&&s.push(a)}return wn(s)}function wn(t){return t.filter((e,r)=>{let n=t[r+1];return!(e.action==="fill"&&(e.value??"")===""&&n?.action==="fill"&&JSON.stringify(n.target)===JSON.stringify(e.target))})}function Lt(t){let e=[];for(let r of t)if(/"[^"]+"|<[^<>]+>/.test(r.gherkin)){for(let s of r.actions)if(s.action==="fill"&&(s.value??"")===""){e.push(`Step "${r.gherkin}" recorded a fill with an empty value although the step passes one; the typed value was lost. Re-record with: saffron run --rerecord`);break}}return e}function Ke(t,e){let r=t;for(let[n,s]of Object.entries(e).filter(([,o])=>o.length>=3).sort((o,i)=>i[1].length-o[1].length))r=r.split(s).join(`<${n}>`);return r}function Ut(t,e){if(Object.entries(e).filter(([,s])=>s.length>=3).length===0)return t;let n=s=>s===void 0?void 0:Ke(s,e);return t.map(s=>({...s,actions:s.actions.map(o=>({...o,value:n(o.value),url:n(o.url),target:o.target?{...o.target,name:n(o.target.name),label:n(o.target.label),text:n(o.target.text),placeholder:n(o.target.placeholder)}:void 0}))}))}function Nt(t,e){let r=K(e.steps);return t.map((n,s)=>{let o=e.steps[s],i=o?.resolvedTable??o?.table,a=o?.resolvedDocString??o?.docString,d=Object.entries({...r,...L(i),...W(a)}).filter(([,m])=>m.length>=3).sort((m,l)=>l[1].length-m[1].length);if(d.length===0)return n;let p=m=>{if(m===void 0)return;let l=m;for(let[c,g]of d)l=l.split(g).join(`<${c}>`);return l};return{...n,actions:n.actions.map(m=>({...m,value:p(m.value),url:p(m.url),target:m.target?{...m.target,name:p(m.target.name),label:p(m.target.label),text:p(m.target.text),placeholder:p(m.target.placeholder)}:void 0}))}})}function Bt(t,e=new Date){let r=n=>n===void 0?void 0:It(n,e);return t.map(n=>({...n,actions:n.actions.map(s=>({...s,value:r(s.value),url:r(s.url),target:s.target?{...s.target,name:r(s.target.name),nameRegex:r(s.target.nameRegex),label:r(s.target.label),text:r(s.target.text)}:void 0}))}))}function Vt(t,e){if(e.size===0)return t;let r=n=>n===void 0?void 0:te(n,e);return t.map(n=>({...n,actions:n.actions.map(s=>({...s,value:r(s.value),url:r(s.url),pattern:r(s.pattern),target:s.target?{...s.target,name:r(s.target.name),nameRegex:r(s.target.nameRegex),label:r(s.target.label),text:r(s.target.text),placeholder:r(s.target.placeholder)}:void 0}))}))}function zt(t,e){let r=e??{};if(t==="mcp__playwright__browser_handle_dialog"){let n={action:"handleDialog",value:r.accept?"accept":"dismiss"};return typeof r.promptText=="string"&&r.promptText&&(n.promptText=r.promptText),n}if(t==="mcp__playwright__browser_file_upload"){let n=Array.isArray(r.paths)?r.paths:[];if(n.length===0)return;let s=process.cwd()+"/";return{action:"uploadFile",value:n.map(i=>typeof i=="string"&&i.startsWith(s)?i.slice(s.length):i).join(`
|
|
28
|
+
`)}}}var we={aiCalls:0,inputTokens:0,outputTokens:0,cacheReadTokens:0,cacheCreationTokens:0,costUsd:0};async function An(){return new Promise((t,e)=>{let r=$n.createServer();r.listen(0,"127.0.0.1",()=>{let{port:n}=r.address();r.close(()=>t(n))}),r.on("error",e)})}function le(t){let e=t.contexts().flatMap(r=>r.pages());return[...e].reverse().find(r=>r.url()!=="about:blank")??e.at(-1)}function Jt(t){let e=t.length;for(let r=t.length-1;r>=0&&t[r].kind==="assertion";r--)e=r;return e}function Wt(t,e){return t[e]?.kind==="assertion"&&e>=Jt(t)}function Ht(t,e,r=!1,n){let s=Jt(t);return t.map((o,i)=>{let a=e[i],d=a&&a.actions.length>0;return o.kind==="assertion"?r&&i<s&&d?a:o:d?n&&i===n.stepIndex&&n.actionIndex>0?{...a,actions:[...o.actions.slice(0,n.actionIndex),...a.actions]}:a:o})}function Cn(t,e){return t.map((r,n)=>{let s=e[n];return s&&s.actions.length>0?s:r})}function Gt(t,e){return{aiCalls:t.aiCalls+e.aiCalls,inputTokens:t.inputTokens+e.inputTokens,outputTokens:t.outputTokens+e.outputTokens,cacheReadTokens:t.cacheReadTokens+e.cacheReadTokens,cacheCreationTokens:t.cacheCreationTokens+e.cacheCreationTokens,costUsd:(t.costUsd??0)+(e.costUsd??0)}}var ke=class{constructor(e){this.options=e}options;browser;cdpEndpoint;stepIndex;getStepIndex(){if(!this.stepIndex&&(this.stepIndex=H(this.options.projectRoot),this.stepIndex.steps.size>0)){let e=this.stepIndex.divergent.length>0?` \xB7 ${this.stepIndex.divergent.length} step(s) have divergent recordings (Level-2 signal)`:"";console.log(`step index: ${this.stepIndex.steps.size} reusable steps from ${this.stepIndex.scannedCaches} cache(s)${e}`)}return this.stepIndex}async recordWithSeeding(e,r){let n=this.getStepIndex(),s=e.steps.length,o={...e.params,...K(e.steps)},i=e.steps.map(x=>{let w=n.steps.size>0?yt(n,x,o):void 0;return w?{gherkin:x.text,keyword:x.keyword,kind:x.kind,table:x.table,docString:x.docString,fromStepSet:x.source?.fromStepSet,actions:w.actions}:void 0}),a=e.steps.map(x=>({gherkin:x.text,keyword:x.keyword,kind:x.kind,table:x.table,docString:x.docString,fromStepSet:x.source?.fromStepSet,actions:[]})),d=new Map,p=[],m=[],l=[],c=[],g={...we},h=0,u=0,f=0,y=x=>({success:x,narrative:m.join(" ")||"Composed entirely from seeded recordings.",adaptations:p,suggestedFeatureEdit:c.length>0?c.join(`
|
|
29
|
+
`):void 0,featureEdits:l,recordedSteps:a,announcedSteps:[],usage:g,seededCount:h});for(;f<s;){if(i[f]){let b=f;for(;b<s&&i[b];)b++;let j=i.slice(f,b),S={...e,steps:e.steps.slice(f,b)},$=await be(r,S,this.buildCandidateCache(e,j),{baseURL:this.options.baseURL,actionTimeoutMs:this.options.actionTimeoutMs,retries:this.options.retries,vars:d}),T=$.status==="passed"?b-f:$.failure?.stepIndex??0;for(let _=0;_<T;_++)a[f+_]=j[_];if(h+=T,$.status==="passed"){f=b;continue}let P=f+T;p.push(`Seeded recording for "${e.steps[P].text}" did not replay in this scenario's context; the agent re-recorded it fresh. Divergence signal.`),i[P]=void 0,f=P;continue}if(!this.options.agent)return y(!1);u++;let x=f;for(;x<s&&!i[x];)x++;u>=3&&(x=s);let w=await this.withDialogsVisible(()=>this.options.agent.run({scenario:e,baseURL:this.options.baseURL,mode:"record",cdpEndpoint:this.cdpEndpoint,assertionPolicy:this.options.assertionPolicy,completedSteps:f>0?e.steps.slice(0,f).map(b=>b.resolvedText):void 0,recordOnly:x<s?{from:f,to:x-1}:void 0,takeFingerprint:async()=>J(le(this.browser)??r)}));if(g=Gt(g,w.usage),m.push(w.narrative),p.push(...w.adaptations),l.push(...w.featureEdits),w.suggestedFeatureEdit&&c.push(w.suggestedFeatureEdit),!w.success)return y(!1);let F=w.announcedSteps.filter(b=>b>=f&&b<s),v=Math.min(s-1,Math.max(F.length>0?Math.max(...F):x-1,f));for(let b=f;b<=v;b++)a[b]=w.recordedSteps[b];f=v+1}return y(!0)}async withDialogsVisible(e){let r=()=>{},n=[],s=[],o=i=>{i.on("dialog",r),n.push(i)};for(let i of this.browser?.contexts()??[]){i.on("page",o),s.push(i);for(let a of i.pages())o(a)}try{return await e()}finally{for(let i of s)i.off("page",o);for(let i of n)try{i.off("dialog",r)}catch{}}}async start(){let e=this.options.browser??"chromium";if(e==="chromium"){let r=await An();this.browser=await Rn.launch({headless:!this.options.headed,args:[`--remote-debugging-port=${r}`]}),this.cdpEndpoint=`http://127.0.0.1:${r}`}else this.browser=await(e==="firefox"?Tn:En).launch({headless:!this.options.headed}),this.cdpEndpoint=void 0}async stop(){await this.browser?.close(),this.browser=void 0}resolveStorageState(){if(!this.options.storageState)return;let e=qt.isAbsolute(this.options.storageState)?this.options.storageState:qt.join(this.options.projectRoot,this.options.storageState);if(!kn.existsSync(e))throw new Error(`storageState file not found: ${e}. Generate one with: npx playwright open --save-storage=${this.options.storageState} <url>`);return e}buildCandidateCache(e,r){return{version:1,feature:e.featurePath,scenario:e.name,recordedAt:new Date().toISOString(),recordedBy:"agent",steps:r}}async proofReplay(e,r){let n=await this.browser.newContext({storageState:this.resolveStorageState()}),s=await n.newPage();return{result:await be(s,e,r,{baseURL:this.options.baseURL,actionTimeoutMs:this.options.actionTimeoutMs,retries:this.options.retries}),context:n,page:s}}async runScenario(e,r={}){let n=await this.runScenarioInner(e,r),s=ae(e);if(s.size>0){let o=i=>i===void 0?i:te(i,s);n.error=o(n.error),n.narrative=o(n.narrative),n.proofError=o(n.proofError),n.suggestedFeatureEdit=o(n.suggestedFeatureEdit),n.adaptations=n.adaptations.map(i=>te(i,s));for(let i of n.stepResults)i.error=o(i.error)}return n}async runScenarioInner(e,r={}){if(!this.browser)throw new Error("Orchestrator not started");let n=Date.now(),{projectRoot:s,baseURL:o,actionTimeoutMs:i,retries:a}=this.options,d=z(s,e.featurePath,e.name),p=N(d),m=await this.browser.newContext({storageState:this.resolveStorageState()}),l=await m.newPage();try{let c;if(p&&(c=await be(l,e,p,{baseURL:o,actionTimeoutMs:i,retries:a}),c.status==="passed"))return{scenario:e,status:"green",source:"cache",stepResults:c.stepResults,adaptations:[],usage:we,durationMs:Date.now()-n};if(!(this.options.agent!==void 0&&this.cdpEndpoint!==void 0&&!r.agentDisabled)){let y=this.options.agent!==void 0&&this.cdpEndpoint===void 0&&!r.agentDisabled?`recording/healing requires Chromium (the agent attaches over CDP). Record with --browser chromium (the default), then replay with --browser ${this.options.browser}.`:"no agent is configured.";return{scenario:e,status:"red",source:"cache",stepResults:c?.stepResults??[],adaptations:[],usage:we,durationMs:Date.now()-n,error:p?c?.status==="stale"?`Cache is stale (feature file changed) and ${y}`:`Replay failed and ${y} ${c?.failure?.error??""}`.trimEnd():`No cache exists and ${y}`}}let h=p!==void 0&&c!==void 0&&c.status==="failed",u=0,f;try{if(h)f=await this.withDialogsVisible(()=>this.options.agent.run({scenario:e,baseURL:o,mode:"heal",model:this.options.healModel,cdpEndpoint:this.cdpEndpoint,assertionPolicy:this.options.assertionPolicy,completedSteps:c.stepResults.filter(y=>y.status==="passed").map(y=>y.gherkin),failedStep:{text:c.failure.stepText,kind:c.failure.kind,error:c.failure.error},takeFingerprint:async()=>{let y=le(this.browser)??l;return J(y)}}));else if(await l.goto("about:blank"),this.options.reuseSteps!==!1){let y=await this.recordWithSeeding(e,l);u=y.seededCount,f=y}else f=await this.withDialogsVisible(()=>this.options.agent.run({scenario:e,baseURL:o,mode:"record",cdpEndpoint:this.cdpEndpoint,assertionPolicy:this.options.assertionPolicy,takeFingerprint:async()=>{let y=le(this.browser)??l;return J(y)}}))}catch(y){let x=y instanceof Error?y.message:String(y);return{scenario:e,status:"red",source:h?"agent-heal":"agent-record",stepResults:c?.stepResults??[],adaptations:[],usage:we,durationMs:Date.now()-n,error:`Agent invocation failed: ${x.split(`
|
|
30
|
+
`)[0]}`}}if(f.success&&h&&c.failure.kind==="assertion"&&(this.options.assertionPolicy!=="adaptable-mid"||Wt(p.steps,c.failure.stepIndex))){let y=p.steps[c.failure.stepIndex],x=le(this.browser)??l,w=new Map(Object.entries(c.captured)),F=Ye(e,c.failure.stepIndex);try{for(let v of y.actions)await Je(x,v,F,this.options.actionTimeoutMs??5e3,o,w)}catch(v){let b=v instanceof Error?v.message:String(v);return{scenario:e,status:"red",source:"agent-heal",stepResults:c.stepResults,narrative:f.narrative,adaptations:f.adaptations,suggestedFeatureEdit:f.suggestedFeatureEdit,usage:f.usage,durationMs:Date.now()-n,error:`Assertion failed as written: "${y.gherkin}". Saffron never adapts assertions. If this change is intended, update the .feature file or re-record the scenario (saffron run --rerecord --filter @tag). If the recording itself looks wrong (an empty value, a wrong target), re-record it the same way. Verification error: ${b.split(`
|
|
31
|
+
`)[0]}`}}}return await this.finalizeAgentRun(e,f,h,p,n,u,h?c?.failure:void 0)}finally{await m.close()}}async finalizeAgentRun(e,r,n,s,o,i=0,a){let d=n?"agent-heal":"agent-record",p=r.recordedSteps.map(v=>({gherkin:v.gherkin,kind:v.kind,status:r.success?"passed":"failed"}));if(!r.success)return{scenario:e,status:"red",source:d,stepResults:p,narrative:r.narrative,adaptations:r.adaptations,usage:r.usage,durationMs:Date.now()-o,error:r.narrative};let m=n?Ht(s.steps,r.recordedSteps,this.options.assertionPolicy==="adaptable-mid",a?{stepIndex:a.stepIndex,actionIndex:a.actionIndex}:void 0):r.recordedSteps,l=[...r.adaptations],c={...r.usage},g,h;if(this.options.verifyProposals!==!1){let v=await this.proofReplay(e,this.buildCandidateCache(e,m));try{if(v.result.status!=="passed"&&v.result.failure){let b=v.result.failure;if(this.options.agent&&!(n&&b.kind==="assertion"&&(this.options.assertionPolicy!=="adaptable-mid"||Wt(s.steps,b.stepIndex))))try{let S=await this.withDialogsVisible(()=>this.options.agent.run({scenario:e,baseURL:this.options.baseURL,mode:"refine",model:n?this.options.healModel:void 0,cdpEndpoint:this.cdpEndpoint,assertionPolicy:this.options.assertionPolicy,completedSteps:v.result.stepResults.filter($=>$.status==="passed").map($=>$.gherkin),failedStep:{text:b.stepText,kind:b.kind,error:b.error},takeFingerprint:async()=>J(le(this.browser)??v.page)}));c=Gt(c,S.usage),S.success&&(l.push(...S.adaptations),m=n?Ht(m,S.recordedSteps,this.options.assertionPolicy==="adaptable-mid"):Cn(m,S.recordedSteps),await v.context.close(),v=await this.proofReplay(e,this.buildCandidateCache(e,m)))}catch{}}g=v.result.status==="passed",g||(h=v.result.failure?.error??`proof replay ${v.result.status}`)}finally{await v.context.close().catch(()=>{})}}let u=[...r.featureEdits];for(let v of St(m,H(this.options.projectRoot)))l.push(`duplicate wording: "${v.from}" recorded exactly the same actions as the existing step "${v.to}" (${v.feature} \u203A ${v.scenario}). Accept with --with-feature-edit to converge on the canonical wording.`),u.some(b=>b.index===v.index)||u.push({index:v.index,text:v.to});let f=ae(e),y=v=>f.size>0?te(v,f):v;l=l.map(y);let x={meta:{createdAt:new Date().toISOString(),mode:n?"heal":"record",narrative:y(r.narrative),adaptations:l,suggestedFeatureEdit:r.suggestedFeatureEdit?y(r.suggestedFeatureEdit):void 0,featureEdits:u.length>0?u:void 0,verified:g,proofError:h?y(h):void 0,seededSteps:i>0?i:void 0,usage:c},cache:{version:1,feature:e.featurePath,scenario:e.name,recordedAt:new Date().toISOString(),recordedBy:"agent",steps:m}},w=Lt(m);w.length>0&&(g=!1,h=[h,...w].filter(Boolean).join(" "));let F=ut(this.options.projectRoot,e.featurePath,e.name);return Ct(F,x),{scenario:e,status:"yellow",source:d,stepResults:p,narrative:r.narrative,adaptations:l,suggestedFeatureEdit:r.suggestedFeatureEdit,featureEdits:u.length>0?u:void 0,verified:g,proofError:h,seededSteps:i>0?i:void 0,proposalFile:F,usage:c,durationMs:Date.now()-o}}};import er from"node:path";import{createRequire as In}from"node:module";import{query as jn}from"@anthropic-ai/claude-agent-sdk";import{createSdkMcpServer as Pn,tool as re}from"@anthropic-ai/claude-agent-sdk";import{z as k}from"zod";var D=t=>({content:[{type:"text",text:t}]});function Yt(t){let e={};return t.role&&(e.role=t.role),t.name&&(e.name=t.name),t.nameRegex&&(e.nameRegex=t.nameRegex),t.label&&(e.label=t.label),t.text&&(e.text=t.text),t.selector&&(e.fallbackSelectors=[t.selector]),t.frame&&(e.frame=t.frame),Object.keys(e).length>0?e:void 0}var Qt={role:k.string().optional().describe("ARIA role of the element"),name:k.string().optional().describe("Accessible name \u2014 use the SHORTEST stable substring; never include volatile content like prices or dates"),nameRegex:k.string().optional().describe("Regex for the accessible name when it embeds volatile content, e.g. 'Select \\\\{date\\\\+1\\\\} as checkin' cannot work \u2014 instead use nameRegex 'Select .* as checkin'"),label:k.string().optional().describe("Form-label text for getByLabel \u2014 the literal <label> text only, never a free-text description"),text:k.string().optional().describe("getByText locator text"),selector:k.string().optional().describe("CSS fallback selector"),frame:k.string().optional().describe(`CSS selector of the iframe hosting the element, e.g. 'iframe[title="Feedback widget"]' \u2014 REQUIRED whenever the element lives inside an iframe; without it replay looks only at the top-level page`)};function Kt(t){return Pn({name:"saffron",version:"0.1.0",tools:[re("saffron_step","Announce the Gherkin step you are about to perform. MUST be called before executing browser actions for each step, with the step's zero-based index and exact text.",{index:k.number().int().min(0),text:k.string(),adaptation:k.string().optional().describe("If the written step does not match the real UI and you are deviating from it, describe the deviation in one sentence.")},async e=>(t.currentStepIndex=e.index,t.announced.add(e.index),e.adaptation&&t.adaptations.push(`Step ${e.index+1} ("${e.text}"): ${e.adaptation}`),D(`Recording step ${e.index}: ${e.text}`))),re("saffron_capture",`Record a captureText action for the current step: at replay time, the target element's text is read and stored under saveAs for later expectDiffers comparisons. Use for steps like 'I record the displayed price as "X"'. The target must select exactly the element whose text is the value.`,{saveAs:k.string().describe("Name to store the captured text under"),...Qt},async e=>{let r=Yt(e);return r?(t.addActions([{action:"captureText",target:r,saveAs:e.saveAs}]),D(`Capture recorded as "${e.saveAs}".`)):D("ERROR: capture needs a target (role/name, nameRegex, label, text, or selector).")}),re("saffron_dialog","Record how a native dialog (alert/confirm/prompt) must be handled at replay. Call this IMMEDIATELY whenever an action triggers a native dialog \u2014 whether you handled it via browser_handle_dialog or the browser reported it as already handled. At replay Saffron arms this handler BEFORE the triggering action, so without this call the replay will auto-dismiss the dialog and diverge from what you saw.",{accept:k.boolean().describe("true = accept/OK the dialog, false = dismiss/cancel"),promptText:k.string().optional().describe("Text typed into a prompt() before accepting")},async e=>(t.addArmedAction({action:"handleDialog",value:e.accept?"accept":"dismiss",...e.promptText?{promptText:e.promptText}:{}}),D(`Dialog handling recorded (${e.accept?"accept":"dismiss"}).`))),re("saffron_wait_api","Record a network wait for the current step: replay will block until a matching API response is observed (responses from the triggering step count). Call AFTER you verified via browser_network_requests that the response actually occurred with the expected status/body. Use a path-only urlPattern regex and never literal volatile IDs (/api/orders/\\d+, not /api/orders/8842).",{urlPattern:k.string().describe("Regex matched against the response URL \u2014 path-only for portability"),method:k.string().optional().describe("HTTP method to match (any if omitted)"),status:k.string().optional().describe("Expected status \u2014 exact ('201') or class ('2xx'); default any 2xx"),bodyPattern:k.string().optional().describe(`Regex the response body must match, e.g. '"status"\\s*:\\s*"READY"' for polling waits`)},async e=>(t.addActions([{action:"waitForResponse",urlPattern:e.urlPattern,...e.method?{method:e.method}:{},...e.status?{status:e.status}:{},...e.bodyPattern?{pattern:e.bodyPattern}:{}}]),D("Network wait recorded."))),re("saffron_assert","Record a verified assertion for the current Then step. Call ONLY after you have confirmed the condition holds via a page snapshot. If the condition does NOT hold, do not call this \u2014 call saffron_done with success=false instead. NEVER record volatile content (prices, dates, counters) as literal expected text \u2014 use kind=matches with a pattern, or kind=differs against captured values.",{kind:k.enum(["visible","notVisible","text","url","value","differs","matches","attribute","response"]),...Qt,expected:k.string().optional().describe("Expected text/value for kind=text|value \u2014 stable text only, never volatile values"),url:k.string().optional().describe("URL substring for kind=url (path only, no host)"),pattern:k.string().optional().describe("kind=matches|attribute: regex the target's text (or attribute) must match, e.g. 'NOK [\\\\d,]+' for a price"),attribute:k.string().optional().describe("kind=attribute: attribute name to check, e.g. 'href' for link-target assertions \u2014 use this for 'X should link to Y' instead of clicking through"),capture:k.string().optional().describe("kind=differs: name of the captured value on the LEFT of the comparison (omit when comparing the target element's live text)"),urlPattern:k.string().optional().describe("kind=response: regex matched against the response URL \u2014 path-only for portability, never literal volatile IDs (use /api/orders/\\d+)"),method:k.string().optional().describe("kind=response: HTTP method to match (any if omitted)"),status:k.string().optional().describe("kind=response: expected status \u2014 exact ('201') or class ('2xx'); default any 2xx"),bodyPattern:k.string().optional().describe("kind=response: regex the response body must match (verify via browser_network_requests first)"),compareTo:k.string().optional().describe("kind=differs: name of the captured value to compare against")},async e=>{let r=Yt(e),n;switch(e.kind){case"visible":n={action:"expectVisible",target:r};break;case"notVisible":n={action:"expectNotVisible",target:r};break;case"text":n={action:"expectText",target:r,value:e.expected??""};break;case"value":n={action:"expectValue",target:r,value:e.expected??""};break;case"url":n={action:"expectUrl",url:e.url??""};break;case"differs":if(!e.compareTo)return D("ERROR: kind=differs requires compareTo.");if(!e.capture&&!r)return D("ERROR: kind=differs needs either capture (a stored name) or a target element.");n={action:"expectDiffers",target:r,value:e.capture,compareTo:e.compareTo};break;case"matches":if(!e.pattern)return D("ERROR: kind=matches requires pattern.");n={action:"expectMatches",target:r,pattern:e.pattern};break;case"attribute":if(!e.attribute||!e.pattern)return D("ERROR: kind=attribute requires attribute and pattern.");n={action:"expectAttribute",target:r,attribute:e.attribute,pattern:e.pattern};break;case"response":if(!e.urlPattern)return D("ERROR: kind=response requires urlPattern.");n={action:"expectResponse",urlPattern:e.urlPattern,...e.method?{method:e.method}:{},...e.status?{status:e.status}:{},...e.bodyPattern?{pattern:e.bodyPattern}:{}};break}return!["url","differs","response"].includes(e.kind)&&!r?D("ERROR: assertion needs a target (role/name, nameRegex, label, text, or selector)."):(t.addActions([n]),D("Assertion recorded."))}),re("saffron_done","Finalize the run. Call exactly once, after the final step: success=true only if every step's intent was achieved and every assertion verified.",{success:k.boolean(),narrative:k.string().describe("2-4 sentence summary of the run: what happened, and why it failed if it did."),suggestedFeatureEdit:k.string().optional().describe("If you adapted any steps: the corrected Gherkin for those steps, as you would rewrite them in the .feature file (free text, for the human report)."),featureEdits:k.array(k.object({index:k.number().int().min(0).describe("zero-based step index"),text:k.string().describe("replacement step text WITHOUT the Given/When/Then keyword")})).optional().describe("Machine-applicable version of the suggested edits: one entry per adapted step. Provide this whenever you adapted a step.")},async e=>(t.done=!0,t.success=e.success,t.narrative=e.narrative,t.suggestedFeatureEdit=e.suggestedFeatureEdit,t.featureEdits=e.featureEdits??[],D("Run finalized.")))]})}function Xt(t={}){let e=t.deliberateSnapshots?`
|
|
32
32
|
8. Tool responses do NOT include a page snapshot. Take snapshots deliberately with browser_snapshot, and only when you actually need to see the page: after navigation, before locating an element you haven't seen yet, and when verifying an assertion. Do NOT snapshot after actions whose outcome is predictable (filling a field you just located, pressing Enter, a click whose result you will verify later anyway). Every snapshot is expensive.
|
|
33
33
|
9. Minimize round-trips: batch tool calls in a single response whenever they don't depend on each other's results \u2014 saffron_step together with that step's browser actions, several fills in one turn, bookkeeping calls (saffron_capture, saffron_assert) alongside the actions they belong to. Only stop and wait when you genuinely need a result before deciding your next call. Do not write narration between tool calls.`:"";return`You are Saffron's test execution agent. You execute Gherkin scenarios in a real browser using the playwright MCP tools, while Saffron records your actions into a deterministic replay cache.
|
|
34
34
|
|
|
@@ -44,7 +44,7 @@ Non-negotiable rules:
|
|
|
44
44
|
5c. Real-world page furniture is supported: elements inside IFRAMES record and replay fine (interact normally) \u2014 but when a saffron_assert targets an element INSIDE an iframe, you MUST pass the iframe's CSS selector as the 'frame' argument (e.g. frame='iframe[title="Feedback widget"]'); replay cannot see into frames otherwise. File uploads: use browser_file_upload with the file path. Drag-and-drop: use browser_drag. All of these are recorded and replay deterministically.
|
|
45
45
|
5d. NATIVE DIALOGS (alert/confirm/prompt): when an action opens one, the page blocks and the snapshot reports a modal state \u2014 resolve it with browser_handle_dialog (accept true/false; promptText for prompt()). That call is recorded automatically as an armed handler replay places BEFORE the triggering action, so do NOT also call saffron_dialog. After handling, verify the post-dialog state as usual. If a step's wording says to confirm/accept, accept; if it says cancel/dismiss, dismiss.
|
|
46
46
|
5e. NETWORK WAITS: when a step says to wait for an API call ("wait for the order API to succeed", "until the job API reports READY"), do it verify-first: perform the trigger, check via browser_network_requests that the matching response actually occurred with the expected status/body, then record it with saffron_wait_api. For Then steps about API contracts ("the request should have returned 201") verify the same way and record with saffron_assert kind=response. urlPattern is a path-only regex \u2014 NEVER a literal volatile ID (/api/orders/\\d+, not /api/orders/8842) and never the host.
|
|
47
|
-
7. Never ask the user questions; you are unattended.${e}`}function
|
|
47
|
+
7. Never ask the user questions; you are unattended.${e}`}function Fn(t,e){if(t[e].kind!=="assertion")return"action";let r=t.length;for(let n=t.length-1;n>=0&&t[n].kind==="assertion";n--)r=n;return e>=r?"assertion:final":"assertion"}function Zt(t){let{scenario:e}=t,r=e.steps.map((s,o)=>{let i=l=>ie(l),a=`${o}. [${Fn(e.steps,o)}] ${s.keyword} ${i(s.resolvedText)}`,d=s.resolvedTable??s.table,p=s.resolvedDocString??s.docString,m=a;return d&&(m+=`
|
|
48
48
|
${d.map(l=>` | ${l.map(i).join(" | ")} |`).join(`
|
|
49
49
|
`)}`),p!==void 0&&(m+=`
|
|
50
50
|
"""
|
|
@@ -52,49 +52,50 @@ ${i(p).split(`
|
|
|
52
52
|
`).map(l=>` ${l}`).join(`
|
|
53
53
|
`)}
|
|
54
54
|
"""`),m}).join(`
|
|
55
|
-
`),
|
|
55
|
+
`),n=[`Feature: ${e.featureName} (${e.featurePath})`,`Scenario: ${e.displayName}`,t.baseURL?`Base URL: ${t.baseURL}`:void 0,"","Steps (index. [kind] keyword text):",r,""];if(t.mode==="refine")n.push("MODE: REFINE. You (or a previous agent session) just recorded this scenario and it completed successfully \u2014 but the deterministic zero-AI replay of that recording FAILED at the step below. The application is working; the RECORDING is not replayable.","The browser tab shows the state where the proof replay stopped.","",`Steps that replayed fine:
|
|
56
56
|
${(t.completedSteps??[]).map(s=>` \u2713 ${s}`).join(`
|
|
57
|
-
`)||" (none)"}`,"",`Failed step: [${t.failedStep?.kind}] ${t.failedStep?.text}`,`Replay failure: ${t.failedStep?.error}`,"","Typical causes: a locator that only matched during your session (volatile accessible name \u2014 use nameRegex or a shorter stable name), a literal dynamic value baked into an assertion (use kind=matches/differs/captureText), or an assertion that cannot be checked without navigation (use kind=attribute for link targets).","Re-perform and re-record the failed step and every step after it in a REPLAYABLE form via saffron_step + browser tools / saffron_capture / saffron_assert. Express the SAME conditions the steps state \u2014 do not weaken them.","If a step genuinely cannot be expressed replayably, call saffron_done success=false and explain why.");else if(t.mode==="heal")
|
|
57
|
+
`)||" (none)"}`,"",`Failed step: [${t.failedStep?.kind}] ${t.failedStep?.text}`,`Replay failure: ${t.failedStep?.error}`,"","Typical causes: a locator that only matched during your session (volatile accessible name \u2014 use nameRegex or a shorter stable name), a literal dynamic value baked into an assertion (use kind=matches/differs/captureText), or an assertion that cannot be checked without navigation (use kind=attribute for link targets).","Re-perform and re-record the failed step and every step after it in a REPLAYABLE form via saffron_step + browser tools / saffron_capture / saffron_assert. Express the SAME conditions the steps state \u2014 do not weaken them.","If a step genuinely cannot be expressed replayably, call saffron_done success=false and explain why.");else if(t.mode==="heal")n.push("MODE: HEAL. A cached deterministic replay of this scenario failed mid-run.","The browser tab already holds the page state reached so far \u2014 do NOT restart from step 0; verify where you are with a snapshot and continue from the failed step.","",`Steps already completed by replay:
|
|
58
58
|
${(t.completedSteps??[]).map(s=>` \u2713 ${s}`).join(`
|
|
59
|
-
`)||" (none)"}`,"",`Failed step: [${t.failedStep?.kind}] ${t.failedStep?.text}`,`Failure: ${t.failedStep?.error}`,"","First diagnose: did the UI legitimately change (heal: adapt the actions, record via saffron_step + browser tools), or is the application broken (fail: saffron_done success=false with your diagnosis)?",t.assertionPolicy==="adaptable-mid"?"If the failed step is an [assertion:final]: you may re-verify the SAME condition once against the live page (timing), but if it does not hold AS WRITTEN, the run FAILS \u2014 call saffron_done success=false; never substitute it. If the failed step is a mid-scenario [assertion]: this project's policy allows adapting it \u2014 verify the equivalent condition on the legitimately-changed UI, record it via saffron_assert, and report the deviation via saffron_step's adaptation field plus featureEdits. An application defect still fails.":"If the failed step is an [assertion] or [assertion:final]: you may re-verify the SAME condition once against the live page (it may have been a timing issue). If the condition AS WRITTEN does not hold, the run FAILS \u2014 call saffron_done success=false. Do NOT substitute a different element, text, or URL that 'means the same thing'; suggesting a corrected assertion belongs in suggestedFeatureEdit of your failure report, never in saffron_assert.","Announce (saffron_step) and perform only the failed step and the steps after it.");else if(t.mode==="record"&&t.recordOnly){let{from:s,to:o}=t.recordOnly;
|
|
59
|
+
`)||" (none)"}`,"",`Failed step: [${t.failedStep?.kind}] ${t.failedStep?.text}`,`Failure: ${t.failedStep?.error}`,"","First diagnose: did the UI legitimately change (heal: adapt the actions, record via saffron_step + browser tools), or is the application broken (fail: saffron_done success=false with your diagnosis)?",t.assertionPolicy==="adaptable-mid"?"If the failed step is an [assertion:final]: you may re-verify the SAME condition once against the live page (timing), but if it does not hold AS WRITTEN, the run FAILS \u2014 call saffron_done success=false; never substitute it. If the failed step is a mid-scenario [assertion]: this project's policy allows adapting it \u2014 verify the equivalent condition on the legitimately-changed UI, record it via saffron_assert, and report the deviation via saffron_step's adaptation field plus featureEdits. An application defect still fails.":"If the failed step is an [assertion] or [assertion:final]: you may re-verify the SAME condition once against the live page (it may have been a timing issue). If the condition AS WRITTEN does not hold, the run FAILS \u2014 call saffron_done success=false. Do NOT substitute a different element, text, or URL that 'means the same thing'; suggesting a corrected assertion belongs in suggestedFeatureEdit of your failure report, never in saffron_assert.","Announce (saffron_step) and perform only the failed step and the steps after it.");else if(t.mode==="record"&&t.recordOnly){let{from:s,to:o}=t.recordOnly;n.push("MODE: RECORD (segment). This scenario is being composed from existing recordings plus your work.",t.completedSteps&&t.completedSteps.length>0?`Already executed against the live browser (do NOT redo):
|
|
60
60
|
${t.completedSteps.map(i=>` \u2713 ${i}`).join(`
|
|
61
|
-
`)}`:"You are starting from a blank page.","",`Record ONLY steps ${s} through ${o} (inclusive).`,`CRITICAL STOP RULE: the steps AFTER step ${o} already have recordings and will be replayed automatically the moment you stop. After completing step ${o}, immediately call saffron_done \u2014 do NOT perform, verify, or navigate into any later step. Executing a later step yourself would make the automatic replay run it TWICE.`)}else if(t.completedSteps&&t.completedSteps.length>0){let s=t.completedSteps.length;
|
|
62
|
-
`),"",`The browser currently shows the state after those ${s} step(s) (verify with a snapshot if unsure). Do NOT redo them and do NOT navigate away from the current state.`,`Start with saffron_step for step index ${s} and record from there to the end.`)}else
|
|
63
|
-
`)}var
|
|
64
|
-
`),a.set(l,f));let y=f[c-1],x=y?.match(/^(\s*)(\S+)\s+(.*)$/);if(!x||x[3]!==m.text){o.skipped.push({index:p.index,reason:`${l}:${c} no longer matches the recorded step text`});continue}let
|
|
65
|
-
`));return o}import
|
|
66
|
-
<path d="${
|
|
67
|
-
<path d="${
|
|
68
|
-
</svg>`}function
|
|
61
|
+
`)}`:"You are starting from a blank page.","",`Record ONLY steps ${s} through ${o} (inclusive).`,`CRITICAL STOP RULE: the steps AFTER step ${o} already have recordings and will be replayed automatically the moment you stop. After completing step ${o}, immediately call saffron_done \u2014 do NOT perform, verify, or navigate into any later step. Executing a later step yourself would make the automatic replay run it TWICE.`)}else if(t.completedSteps&&t.completedSteps.length>0){let s=t.completedSteps.length;n.push("MODE: RECORD (seeded). No cache exists for this scenario yet, but its first steps matched existing recordings and were ALREADY EXECUTED against the live browser by zero-AI replay:",t.completedSteps.map(o=>` \u2713 ${o}`).join(`
|
|
62
|
+
`),"",`The browser currently shows the state after those ${s} step(s) (verify with a snapshot if unsure). Do NOT redo them and do NOT navigate away from the current state.`,`Start with saffron_step for step index ${s} and record from there to the end.`)}else n.push("MODE: RECORD. No cache exists for this scenario yet.","Execute every step in order, starting at step 0. Navigate to the base URL first if the scenario implies a starting page.");return n.filter(s=>s!==void 0).join(`
|
|
63
|
+
`)}var _n=In(import.meta.url);function Dn(){let t=_n.resolve("@playwright/mcp/package.json");return er.join(er.dirname(t),"cli.js")}var On=new Set(["mcp__playwright__browser_navigate","mcp__playwright__browser_click","mcp__playwright__browser_type","mcp__playwright__browser_fill_form","mcp__playwright__browser_select_option","mcp__playwright__browser_press_key"]),$e=class{constructor(e={}){this.opts=e}opts;async run(e){let r=new ve(e.scenario),n={aiCalls:0,inputTokens:0,outputTokens:0,cacheReadTokens:0,cacheCreationTokens:0},s="",o=jn({prompt:Zt(e),options:{systemPrompt:Xt({deliberateSnapshots:(this.opts.snapshotMode??"none")==="none",adaptableMidAssertions:e.assertionPolicy==="adaptable-mid"}),model:e.model??this.opts.model,maxTurns:this.opts.maxTurns??100,tools:[],permissionMode:"bypassPermissions",allowDangerouslySkipPermissions:!0,persistSession:!1,settingSources:[],mcpServers:{playwright:{type:"stdio",command:process.execPath,args:[Dn(),"--cdp-endpoint",e.cdpEndpoint,"--snapshot-mode",this.opts.snapshotMode??"none","--image-responses","omit"]},saffron:Kt(r)},hooks:{PreToolUse:[{hooks:[async i=>{let a=i;if(On.has(a.tool_name)&&e.takeFingerprint&&!r.pendingFingerprint)try{r.pendingFingerprint=await e.takeFingerprint()}catch{}return{continue:!0}}]}],PostToolUse:[{hooks:[async i=>{let a=i,d=zt(a.tool_name,a.tool_input);return d&&r.addArmedAction(d),a.tool_name.startsWith("mcp__playwright__")&&(r.addActions(Mt(a.tool_response,e.baseURL)),r.pendingFingerprint=void 0),{continue:!0}}]}]}}});try{for await(let i of o)i.type==="result"&&(n.aiCalls=i.num_turns,n.costUsd=i.total_cost_usd,"usage"in i&&i.usage&&(n.inputTokens=i.usage.input_tokens??0,n.outputTokens=i.usage.output_tokens??0,n.cacheReadTokens=i.usage.cache_read_input_tokens??0,n.cacheCreationTokens=i.usage.cache_creation_input_tokens??0),i.subtype==="success"&&(s=i.result))}catch(i){let a=i instanceof Error?i.message:String(i);r.done=!0,r.success=!1,r.narrative=`Agent session error: ${a}`}return r.done||(r.success=!1,r.narrative=r.narrative||`Agent session ended without finalizing the run. Last output: ${s.slice(0,500)}`),{success:r.success,narrative:r.narrative,adaptations:r.adaptations,suggestedFeatureEdit:r.suggestedFeatureEdit,featureEdits:r.featureEdits.map(i=>({index:i.index,text:Ke(i.text,e.scenario.params)})),recordedSteps:Vt(Bt(Ut(Nt(r.steps,e.scenario),e.scenario.params)),ae(e.scenario)),announcedSteps:[...r.announced].sort((i,a)=>i-a),usage:n}}};import tr from"node:fs";import Re from"node:path";function Te(t){return JSON.stringify([t.role??null,t.name??null,t.nameRegex??null,t.label??null,t.text??null,t.testId??null,t.placeholder??null])}function rr(t,e){let r=new Map,n=Math.min(t.steps.length,e.steps.length);for(let s=0;s<n;s++){let o=t.steps[s].actions,i=e.steps[s].actions;if(o.length===i.length)for(let a=0;a<o.length;a++){let d=o[a],p=i[a];if(d.action!==p.action||!d.target||!p.target)continue;let m=Te(d.target);m!==Te(p.target)&&r.set(m,{from:d.target,to:p.target})}}return[...r.values()]}function Mn(t){let e=Re.join(t,".saffron","cache");if(!tr.existsSync(e))return[];let r=[];for(let n of tr.readdirSync(e,{recursive:!0})){let s=String(n);s.endsWith(".json")&&r.push(Re.join(e,s))}return r}function nr(t,e,r,n){if(e.length===0)return[];let s=new Map(e.map(i=>[Te(i.from),i])),o=[];for(let i of Mn(t)){if(Re.resolve(i)===Re.resolve(r))continue;let a=N(i);if(!a)continue;let d=[],p=!1;for(let m of a.steps){let l=0;for(let c of m.actions){if(!c.target)continue;let g=s.get(Te(c.target));g&&(l++,n||(c.target=structuredClone(g.to),p=!0))}l>0&&d.push({gherkin:m.gherkin,count:l})}d.length>0&&(p&&ee(i,a),o.push({file:i,scenario:a.scenario,feature:a.feature,steps:d}))}return o}function Xe(t){return t.role?`${t.role} "${t.name??t.nameRegex??""}"`:t.label?`label "${t.label}"`:t.text?`text "${t.text}"`:t.testId?`testId "${t.testId}"`:(t.fallbackSelectors??[]).join(",")||"(empty target)"}import sr from"node:fs";import Ze from"node:path";function or(t,e,r,n,s){let o={applied:[],skipped:[]},i=Z(Ze.join(t,e),t,s).find(p=>p.name===r);if(!i){for(let p of n)o.skipped.push({index:p.index,reason:`scenario "${r}" not found in ${e}`});return o}let a=new Map,d=new Set;for(let p of n){let m=i.steps[p.index];if(!m?.source){o.skipped.push({index:p.index,reason:`no step at index ${p.index}`});continue}let{file:l,line:c,fromStepSet:g,fromBackground:h}=m.source;if(h){o.skipped.push({index:p.index,reason:`"${m.text}" is a Background step shared across scenarios; edit it manually`});continue}let u=`${l}:${c}`;if(d.has(u))continue;let f=a.get(l);f||(f=sr.readFileSync(Ze.join(t,l),"utf8").split(`
|
|
64
|
+
`),a.set(l,f));let y=f[c-1],x=y?.match(/^(\s*)(\S+)\s+(.*)$/);if(!x||x[3]!==m.text){o.skipped.push({index:p.index,reason:`${l}:${c} no longer matches the recorded step text`});continue}let w=`${x[1]}${x[2]} ${p.text}`;f[c-1]=w,d.add(u),o.applied.push({file:l,line:c,from:y.trim(),to:w.trim(),fromStepSet:g})}for(let[p,m]of a)o.applied.some(l=>l.file===p)&&sr.writeFileSync(Ze.join(t,p),m.join(`
|
|
65
|
+
`));return o}import tt from"node:fs";import et from"node:fs";import Ee from"node:path";import{fileURLToPath as Ln}from"node:url";function Ae(){let t=Ee.dirname(Ln(import.meta.url));for(let e=0;e<6;e++){let r=Ee.join(t,"package.json");if(et.existsSync(r))try{if(JSON.parse(et.readFileSync(r,"utf8")).name==="saffron-ai")return t}catch{}t=Ee.dirname(t)}}function Ce(){let t=Ae();if(!t)return"unknown";try{return JSON.parse(et.readFileSync(Ee.join(t,"package.json"),"utf8")).version??"unknown"}catch{return"unknown"}}import rt from"node:path";var A=t=>t.replaceAll("&","&").replaceAll("<","<").replaceAll(">",">").replaceAll('"',"""),cr="M11 14 L10 4.5 L17.5 9.5 L24 2.5 L30.5 9.5 L38 4.5 L37 14 Z",Un="M24.085 45.646L24.085 45.646Q20.719 45.646 18.407 44.575Q16.095 43.504 14.905 41.651Q13.715 39.798 13.715 37.384L13.715 37.384L19.121 37.384Q19.121 38.336 19.648 39.152Q20.175 39.968 21.263 40.461Q22.351 40.954 24.085 40.954L24.085 40.954Q25.649 40.954 26.703 40.546Q27.757 40.138 28.301 39.407Q28.845 38.676 28.845 37.724L28.845 37.724Q28.845 36.534 27.825 35.837Q26.805 35.140 24.527 34.936L24.527 34.936L22.657 34.766Q18.951 34.460 16.741 32.454Q14.531 30.448 14.531 27.150L14.531 27.150Q14.531 24.770 15.670 23.036Q16.809 21.302 18.849 20.367Q20.889 19.432 23.677 19.432L23.677 19.432Q26.601 19.432 28.692 20.435Q30.783 21.438 31.905 23.257Q33.027 25.076 33.027 27.524L33.027 27.524L27.587 27.524Q27.587 26.606 27.145 25.841Q26.703 25.076 25.836 24.600Q24.969 24.124 23.677 24.124L23.677 24.124Q22.453 24.124 21.620 24.532Q20.787 24.940 20.379 25.637Q19.971 26.334 19.971 27.150L19.971 27.150Q19.971 28.204 20.719 28.986Q21.467 29.768 23.167 29.904L23.167 29.904L25.071 30.074Q27.791 30.312 29.865 31.247Q31.939 32.182 33.112 33.797Q34.285 35.412 34.285 37.724L34.285 37.724Q34.285 40.104 33.044 41.889Q31.803 43.674 29.525 44.660Q27.247 45.646 24.085 45.646Z";function Nn(t){return`<svg viewBox="0 0 48 48" width="${t}" height="${t}" aria-hidden="true">
|
|
66
|
+
<path d="${cr}" fill="#F4A300"/>
|
|
67
|
+
<path d="${Un}" fill="#E7E9EC"/>
|
|
68
|
+
</svg>`}function Bn(t){return`<svg viewBox="8 1 32 15" height="${t}" aria-hidden="true"><path d="${cr}" fill="#F4A300"/></svg>`}var Vn={green:"passed",yellow:"adapted",red:"failed"};function Pe(t){return t>=1e3?`${(t/1e3).toFixed(1)}s`:`${t}ms`}function de(t){return t>=1e6?`${(t/1e6).toFixed(1)}M`:t>=1e4?`${(t/1e3).toFixed(1)}k`:t.toLocaleString()}function Q(t,e,r,n="",s="",o=""){return`<div class="kpi">
|
|
69
69
|
<div class="kpi-label">${t}</div>
|
|
70
|
-
<div class="kpi-value ${
|
|
71
|
-
<div class="kpi-sub ${s}">${
|
|
72
|
-
</div>`}function
|
|
73
|
-
${Q("pass rate",`${s}<span class="unit">%</span>`,a,o,
|
|
74
|
-
${Q("passed",String(
|
|
75
|
-
${Q("adapted",String(
|
|
76
|
-
${Q("failed",String(
|
|
77
|
-
${Q("ai calls",String(
|
|
78
|
-
${Q("cache traffic",de(
|
|
79
|
-
${Q("ai cost",`<span class="unit">$</span>${
|
|
80
|
-
</section>`}function
|
|
81
|
-
`)}</section>`:""}function
|
|
82
|
-
${
|
|
70
|
+
<div class="kpi-value ${n}">${e}</div>
|
|
71
|
+
<div class="kpi-sub ${s}">${r}</div>${o}
|
|
72
|
+
</div>`}function ir(t,e){if(t.length<2)return"";let r=108,n=22,s=Math.min(...t),i=Math.max(...t)-s||1,a=t.map((d,p)=>`${(p/(t.length-1)*r).toFixed(1)},${(n-2-(d-s)/i*(n-4)).toFixed(1)}`).join(" ");return`<svg class="spark" viewBox="0 0 ${r} ${n}" width="${r}" height="${n}" aria-hidden="true"><polyline points="${a}" fill="none" stroke="${e}" stroke-width="1.8" stroke-linejoin="round" stroke-linecap="round"/></svg>`}var ar=(t,e="",r=0)=>`${t>=0?"+":"\u2212"}${Math.abs(t).toFixed(r)}${e} vs prev`;function zn(t,e){let r=t.totals,n=t.trends,s=r.scenarios===0?0:Math.round((r.green+r.yellow)/r.scenarios*100),o=r.red>0?"red":r.yellow>0?"gold":"green",i=r.inputTokens+r.outputTokens,a=n?.prev?`${ar(n.prev.passRatePp,"pp")} \xB7 ${r.green+r.yellow} of ${r.scenarios}`:`${r.green+r.yellow} of ${r.scenarios} scenarios`,d=n?.prev?`${ar(n.prev.costUsd,"",4).replace("+","+$").replace("\u2212","\u2212$")} \xB7 ${Pe(e)}`:`${Pe(e)} total runtime`;return`<section class="kpis">
|
|
73
|
+
${Q("pass rate",`${s}<span class="unit">%</span>`,a,o,n?.prev?n.prev.passRatePp>=0?"green":"red":"",ir(n?.series.passRate??[],"var(--green)"))}
|
|
74
|
+
${Q("passed",String(r.green),"deterministic replay","green",r.green>0?"green":"")}
|
|
75
|
+
${Q("adapted",String(r.yellow),r.yellow>0?"\u25B2 proposals pending review":"no adaptations",r.yellow>0?"gold":"",r.yellow>0?"gold":"")}
|
|
76
|
+
${Q("failed",String(r.red),r.red>0?"\u25BC needs attention":"all clear",r.red>0?"red":"",r.red>0?"red":"green")}
|
|
77
|
+
${Q("ai calls",String(r.aiCalls),`${de(i)} in+out tokens`,r.aiCalls>0?"":"green")}
|
|
78
|
+
${Q("cache traffic",de(r.cacheReadTokens+r.cacheCreationTokens),r.cacheReadTokens+r.cacheCreationTokens>0?`${de(r.cacheReadTokens)} read \xB7 ${de(r.cacheCreationTokens)} written`:"zero (fully cached run)","",r.cacheReadTokens+r.cacheCreationTokens>0?"":"green")}
|
|
79
|
+
${Q("ai cost",`<span class="unit">$</span>${r.costUsd.toFixed(r.costUsd>=1?2:4)}`,d,"","",ir(n?.series.costUsd??[],"var(--gold)"))}
|
|
80
|
+
</section>`}function qn(t){let e=t.trends;if(!e)return"";let r=[];return e.chronic.length>0&&r.push(`<div class="panel accent-gold"><div class="panel-title gold">chronic scenarios: stop paying for heals</div><ul>${e.chronic.map(n=>`<li>${A(n.feature)} \u203A ${A(n.scenario)}: ${n.heals} heal(s), ${n.reds} red(s) in the last ${n.runs} run(s). Re-record it or fix the feature text.</li>`).join("")}</ul></div>`),e.divergentSteps>0&&r.push(`<div class="panel"><div class="panel-title">step divergence (level-2 signal)</div><p>${e.divergentSteps} step text(s) currently have divergent recordings across caches.</p></div>`),e.topIssues.length>0&&r.push(`<div class="panel accent-red"><div class="panel-title red">recurring failure themes (last 20 runs)</div><ul>${e.topIssues.map(n=>`<li>${n.count}\xD7: ${A(n.message)}</li>`).join("")}</ul></div>`),r.length>0?`<section class="trends">${r.join(`
|
|
81
|
+
`)}</section>`:""}function Wn(t){return`<ol class="steps">${t.steps.map(r=>{let n=r.status==="passed"?'<span class="dot green"></span>':r.status==="failed"?'<span class="dot red"></span>':'<span class="dot idle"></span>';return`<li class="step ${r.status}">
|
|
82
|
+
${n}
|
|
83
83
|
<div class="step-body">
|
|
84
|
-
<span class="step-text">${A(
|
|
85
|
-
${
|
|
86
|
-
${
|
|
87
|
-
${
|
|
88
|
-
${
|
|
84
|
+
<span class="step-text">${A(r.gherkin)}</span>
|
|
85
|
+
${r.kind==="assertion"?'<span class="tag">assert</span>':""}
|
|
86
|
+
${r.fromStepSet?`<span class="tag indigo" title="inlined from a StepSet definition: edit once, every invoking scenario follows">from StepSet: ${A(r.fromStepSet)}</span>`:""}
|
|
87
|
+
${r.drift?'<span class="tag gold" title="page fingerprint no longer matches the recording">drift</span>':""}
|
|
88
|
+
${r.retries?`<span class="tag gold" title="an action failed and was re-attempted with backoff before the step settled">retried \xD7${r.retries}</span>`:""}
|
|
89
|
+
${r.error?`<div class="step-error">${A(r.error)}</div>`:""}
|
|
89
90
|
</div>
|
|
90
91
|
</li>`}).join(`
|
|
91
|
-
`)}</ol>`}function
|
|
92
|
+
`)}</ol>`}function Hn(t){return t.featureEdits&&t.featureEdits.length>0?`<div class="panel">
|
|
92
93
|
<div class="panel-title gold">suggested .feature edit</div>
|
|
93
|
-
<div class="diff">${t.featureEdits.map(
|
|
94
|
+
<div class="diff">${t.featureEdits.map(r=>{let n=t.steps[r.index]?.gherkin??`step ${r.index}`;return`<div class="diff-line del">\u2212 ${A(n)}</div><div class="diff-line add">+ ${A(r.text)}</div>`}).join("")}</div>
|
|
94
95
|
<div class="panel-hint">apply with <code>saffron accept --with-feature-edit</code></div>
|
|
95
|
-
</div>`:t.suggestedFeatureEdit?`<div class="panel"><div class="panel-title gold">suggested .feature edit</div><pre>${A(t.suggestedFeatureEdit)}</pre></div>`:""}function
|
|
96
|
+
</div>`:t.suggestedFeatureEdit?`<div class="panel"><div class="panel-title gold">suggested .feature edit</div><pre>${A(t.suggestedFeatureEdit)}</pre></div>`:""}function Gn(t){let e=t.usage.inputTokens+t.usage.outputTokens;if(t.usage.aiCalls===0)return'<span class="pill green">0 tokens</span>';let r=t.usage.costUsd?` \xB7 $${t.usage.costUsd.toFixed(2)}`:"";return`<span class="pill gold">${t.usage.aiCalls} ai \xB7 ${de(e)} tok${r}</span>`}function Jn(t){return`<details class="card" ${t.status!=="green"?"open":""}>
|
|
96
97
|
<summary>
|
|
97
|
-
<span class="pill ${t.status==="green"?"green":t.status==="yellow"?"gold":"red"}">${
|
|
98
|
+
<span class="pill ${t.status==="green"?"green":t.status==="yellow"?"gold":"red"}">${Vn[t.status]}</span>
|
|
98
99
|
<span class="card-title">${A(t.scenario)}</span>
|
|
99
100
|
<span class="card-meta">
|
|
100
101
|
<span class="m">${A(t.feature)}</span>
|
|
@@ -102,24 +103,24 @@ ${t.completedSteps.map(i=>` \u2713 ${i}`).join(`
|
|
|
102
103
|
<span class="m">${A(t.source)}</span>
|
|
103
104
|
<span class="m-sep">\xB7</span>
|
|
104
105
|
<span class="m">${Pe(t.durationMs)}</span>
|
|
105
|
-
${
|
|
106
|
+
${Gn(t)}
|
|
106
107
|
</span>
|
|
107
108
|
</summary>
|
|
108
109
|
<div class="card-body">
|
|
109
|
-
${
|
|
110
|
+
${Wn(t)}
|
|
110
111
|
${t.narrative?`<div class="panel"><div class="panel-title">agent narrative</div><p>${A(t.narrative)}</p></div>`:""}
|
|
111
112
|
${t.adaptations.length>0?`<div class="panel accent-gold"><div class="panel-title gold">adaptations</div><ul>${t.adaptations.map(e=>`<li>${A(e)}</li>`).join("")}</ul></div>`:""}
|
|
112
|
-
${
|
|
113
|
+
${Hn(t)}
|
|
113
114
|
${t.proposalFile?`<div class="panel"><div class="panel-title">cache proposal pending${t.verified===!0?' <span class="pill green">verified</span>':t.verified===!1?' <span class="pill red">unverified</span>':""}</div><p class="mono">${A(t.proposalFile)}</p>${t.verified===!1&&t.proofError?`<p class="mono" style="color:var(--red)">proof replay failed: ${A(t.proofError.split(`
|
|
114
115
|
`)[0])}</p>`:""}<div class="panel-hint">review with <code>saffron accept</code> / <code>saffron reject</code></div></div>`:""}
|
|
115
116
|
${t.error&&t.status==="red"?`<div class="panel accent-red"><div class="panel-title red">failure</div><p>${A(t.error)}</p></div>`:""}
|
|
116
117
|
</div>
|
|
117
|
-
</details>`}function
|
|
118
|
+
</details>`}function lr(t){let e=t.totals,r=new Date(t.startedAt),n=new Date(t.finishedAt).getTime()-r.getTime(),s=r.toLocaleString("en-GB",{day:"2-digit",month:"short",year:"numeric",hour:"2-digit",minute:"2-digit"});return`<!doctype html>
|
|
118
119
|
<html lang="en">
|
|
119
120
|
<head>
|
|
120
121
|
<meta charset="utf-8"/>
|
|
121
122
|
<meta name="viewport" content="width=device-width, initial-scale=1"/>
|
|
122
|
-
<title>Saffron
|
|
123
|
+
<title>Saffron run report</title>
|
|
123
124
|
<link rel="preconnect" href="https://fonts.googleapis.com">
|
|
124
125
|
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
|
125
126
|
<link href="https://fonts.googleapis.com/css2?family=Bricolage+Grotesque:opsz,wght@12..96,500;12..96,600;12..96,700&family=Geist:wght@400;500;600&family=JetBrains+Mono:wght@400;500;600&display=swap" rel="stylesheet">
|
|
@@ -234,7 +235,7 @@ ${t.completedSteps.map(i=>` \u2713 ${i}`).join(`
|
|
|
234
235
|
<div class="wrap">
|
|
235
236
|
<header class="top">
|
|
236
237
|
<div class="brand">
|
|
237
|
-
${
|
|
238
|
+
${Nn(38)}
|
|
238
239
|
<div>
|
|
239
240
|
<div class="brand-name">saffron</div>
|
|
240
241
|
<div class="brand-sub">test run report</div>
|
|
@@ -243,50 +244,51 @@ ${t.completedSteps.map(i=>` \u2713 ${i}`).join(`
|
|
|
243
244
|
<div class="run-meta">
|
|
244
245
|
<span>${A(s)}</span>
|
|
245
246
|
<span>\xB7</span>
|
|
246
|
-
<span>${Pe(
|
|
247
|
+
<span>${Pe(n)}</span>
|
|
247
248
|
${t.baseURL?`<span class="chip base">${A(t.baseURL)}</span>`:""}
|
|
248
249
|
</div>
|
|
249
250
|
</header>
|
|
250
251
|
|
|
251
|
-
${
|
|
252
|
-
${
|
|
252
|
+
${zn(t,n)}
|
|
253
|
+
${qn(t)}
|
|
253
254
|
|
|
254
255
|
<div class="section-head">
|
|
255
256
|
<span class="section-title">scenarios \xB7 ${e.scenarios}</span>
|
|
256
257
|
<span class="section-title">saffron v${A(t.version)}</span>
|
|
257
258
|
</div>
|
|
258
|
-
${t.scenarios.map(
|
|
259
|
+
${t.scenarios.map(Jn).join(`
|
|
259
260
|
`)}
|
|
260
261
|
|
|
261
|
-
<footer>${
|
|
262
|
+
<footer>${Bn(11)} generated by saffron v${A(t.version)} \xB7 ${A(new Date(t.finishedAt).toLocaleString("en-GB",{dateStyle:"medium",timeStyle:"medium"}))}</footer>
|
|
262
263
|
</div>
|
|
263
264
|
</body>
|
|
264
265
|
</html>
|
|
265
|
-
`}function
|
|
266
|
-
`),
|
|
267
|
-
`)[0]}));return{runId:e.toISOString(),startedAt:e.toISOString(),finishedAt:new Date().toISOString(),baseURL:
|
|
268
|
-
`)}function
|
|
269
|
-
`).filter(Boolean),s=[];for(let o of
|
|
270
|
-
`)}function
|
|
266
|
+
`}function dr(t,e,r){let n={scenarios:t.length,green:0,yellow:0,red:0,aiCalls:0,inputTokens:0,outputTokens:0,cacheReadTokens:0,cacheCreationTokens:0,costUsd:0};for(let s of t)n[s.status]+=1,n.aiCalls+=s.usage.aiCalls,n.inputTokens+=s.usage.inputTokens,n.outputTokens+=s.usage.outputTokens,n.cacheReadTokens+=s.usage.cacheReadTokens,n.cacheCreationTokens+=s.usage.cacheCreationTokens??0,n.costUsd+=s.usage.costUsd??0;return{tool:"saffron",version:Ce(),startedAt:e.toISOString(),finishedAt:new Date().toISOString(),baseURL:r,totals:n,scenarios:t.map(s=>({feature:s.scenario.featurePath,scenario:s.scenario.displayName,tags:[...s.scenario.tags],status:s.status,source:s.source,durationMs:s.durationMs,usage:s.usage,steps:s.stepResults,narrative:s.narrative,adaptations:s.adaptations,suggestedFeatureEdit:s.suggestedFeatureEdit,featureEdits:s.featureEdits,verified:s.verified,proofError:s.proofError,proposalFile:s.proposalFile,error:s.error}))}}function pr(t,e){let r=rt.join(t,".saffron","reports");tt.mkdirSync(r,{recursive:!0});let n=rt.join(r,"latest.json"),s=rt.join(r,"latest.html");return tt.writeFileSync(n,`${JSON.stringify(e,null,2)}
|
|
267
|
+
`),tt.writeFileSync(s,lr(e)),{json:n,html:s}}function ur(t){let e=t===void 0?[]:Array.isArray(t)?t:[t],r=new Set;for(let n of e)for(let s of n.split(/[\s,|]+/)){let o=s.trim();o&&r.add(o.startsWith("@")?o:`@${o}`)}return[...r]}function fr(t,e){return e.length===0?t:t.filter(r=>r.tags.some(n=>e.includes(n)))}function gr(t,e){return[...e??[],t]}import Fe from"node:fs";import mr from"node:path";function hr(t){return mr.join(t,".saffron","history.jsonl")}function yr(t,e,r,n){let s=t.map(o=>({feature:o.scenario.featurePath,scenario:o.scenario.displayName,tags:[...o.scenario.tags],status:o.status,source:o.source,durationMs:o.durationMs,verified:o.verified,seededSteps:o.seededSteps,adaptations:o.adaptations.length,driftSteps:o.stepResults.filter(i=>i.drift).length,aiCalls:o.usage.aiCalls,tokensInOut:o.usage.inputTokens+o.usage.outputTokens,cacheRead:o.usage.cacheReadTokens,cacheWrite:o.usage.cacheCreationTokens,costUsd:o.usage.costUsd??0,error:o.error?.split(`
|
|
268
|
+
`)[0]}));return{runId:e.toISOString(),startedAt:e.toISOString(),finishedAt:new Date().toISOString(),baseURL:n,green:s.filter(o=>o.status==="green").length,yellow:s.filter(o=>o.status==="yellow").length,red:s.filter(o=>o.status==="red").length,aiCalls:s.reduce((o,i)=>o+i.aiCalls,0),costUsd:s.reduce((o,i)=>o+i.costUsd,0),divergentSteps:r,scenarios:s}}function Sr(t,e){let r=hr(t);Fe.mkdirSync(mr.dirname(r),{recursive:!0}),Fe.appendFileSync(r,`${JSON.stringify(e)}
|
|
269
|
+
`)}function xr(t,e=50){let r=hr(t);if(!Fe.existsSync(r))return[];let n=Fe.readFileSync(r,"utf8").trim().split(`
|
|
270
|
+
`).filter(Boolean),s=[];for(let o of n.slice(-e))try{s.push(JSON.parse(o))}catch{}return s}var nt=t=>{let e=t.green+t.yellow+t.red;return e===0?0:Math.round((t.green+t.yellow)/e*100)};function br(t,e){let r=[...t,e],n=r.slice(-20),s=r.slice(-10),o=t.at(-1),i=o?{passRatePp:nt(e)-nt(o),costUsd:e.costUsd-o.costUsd,aiCalls:e.aiCalls-o.aiCalls,red:e.red-o.red}:void 0,a=new Map;for(let l of s)for(let c of l.scenarios){let g=`${c.feature}::${c.scenario}`,h=a.get(g)??{feature:c.feature,scenario:c.scenario,heals:0,reds:0,runs:0};h.runs++,c.source==="agent-heal"&&c.status==="yellow"&&h.heals++,c.status==="red"&&h.reds++,a.set(g,h)}let d=[...a.values()].filter(l=>l.heals>=2||l.reds>=3).sort((l,c)=>c.heals+c.reds-(l.heals+l.reds)),p=new Map;for(let l of n)for(let c of l.scenarios)c.status==="red"&&c.error&&p.set(c.error,(p.get(c.error)??0)+1);let m=[...p.entries()].sort((l,c)=>c[1]-l[1]).slice(0,3).map(([l,c])=>({message:l,count:c}));return{prev:i,series:{passRate:n.map(nt),costUsd:n.map(l=>Number(l.costUsd.toFixed(4))),aiCalls:n.map(l=>l.aiCalls)},chronic:d,divergentSteps:e.divergentSteps,topIssues:m}}import E from"node:fs";import O from"node:path";var pe={claude:".claude/skills",agents:".agents/skills",copilot:".github/skills",cursor:".cursor/skills"},Yn=["claude","agents"],kr="<!-- saffron:start -->",ot="<!-- saffron:end -->";function Qn(){return[kr,"## Saffron end-to-end tests","","UI tests are plain-Gherkin `.feature` / `.saffron` files under `features/`,","run by Saffron (`npx saffron run`). No step definitions exist: an AI agent","records each scenario once; later runs replay at zero tokens.","","- Load the `saffron` skill before writing or editing scenarios.","- Exact step text is the cache identity: run `npx saffron steps` and reuse"," recorded wordings instead of inventing synonyms.","- `Then` steps are never healed: write them as the verdict, last.","- Never edit `.saffron/cache/` or `.saffron/proposals/` by hand; change the"," scenario text and re-run. Review proposals with `npx saffron accept`.","- Secrets go in `{env:VAR}` tokens, never in files.",ot,""].join(`
|
|
271
|
+
`)}function vr(t,e){if(!E.existsSync(t))return E.writeFileSync(t,e),"created";let r=E.readFileSync(t,"utf8"),n=r.indexOf(kr),s=r.indexOf(ot);if(n!==-1&&s!==-1&&s>n){let i=r.slice(0,n)+e.trimEnd()+r.slice(s+ot.length);return i===r?"unchanged":(E.writeFileSync(t,i),"updated")}let o=r.endsWith(`
|
|
271
272
|
`)?`
|
|
272
273
|
`:`
|
|
273
274
|
|
|
274
|
-
`;return E.writeFileSync(t,
|
|
275
|
-
`),s?"updated":"created")}function
|
|
276
|
-
`),
|
|
275
|
+
`;return E.writeFileSync(t,r+o+e),"updated"}var st={command:"npx",args:["saffron","mcp"]},Kn={claude:{file:".mcp.json",key:"mcpServers",entry:st},cursor:{file:".cursor/mcp.json",key:"mcpServers",entry:st},copilot:{file:".vscode/mcp.json",key:"servers",entry:{type:"stdio",...st}}},wr={agents:"codex mcp add saffron -- npx saffron mcp (Codex; other hosts: see their MCP settings)"};function Xn(t,e){let r=O.join(t,e.file),n={},s=E.existsSync(r);if(s)try{n=JSON.parse(E.readFileSync(r,"utf8"))}catch{throw new Error(`${e.file} is not valid JSON: fix it or remove it, then re-run`)}let o=n[e.key]??{};return JSON.stringify(o.saffron)===JSON.stringify(e.entry)?"unchanged":(o.saffron=e.entry,n[e.key]=o,E.mkdirSync(O.dirname(r),{recursive:!0}),E.writeFileSync(r,JSON.stringify(n,null,2)+`
|
|
276
|
+
`),s?"updated":"created")}function $r(t,e){E.mkdirSync(e,{recursive:!0});for(let r of E.readdirSync(t,{withFileTypes:!0})){let n=O.join(t,r.name),s=O.join(e,r.name);r.isDirectory()?$r(n,s):E.copyFileSync(n,s)}}function Rr(t){let e=O.join(t.packageRoot,"skills","saffron");if(!E.existsSync(O.join(e,"SKILL.md")))throw new Error(`bundled skill not found at ${e}`);let r={skillDirs:[],instructionFiles:[],scaffolded:[],mcpFiles:[],mcpHints:[]},n=t.targets??Yn;for(let s of n){let o=O.join(t.projectRoot,pe[s],"saffron");E.existsSync(o)&&E.rmSync(o,{recursive:!0,force:!0}),$r(e,o),r.skillDirs.push(O.relative(t.projectRoot,o))}if(t.mcp!==!1)for(let s of n){let o=Kn[s];o?r.mcpFiles.push({file:o.file,outcome:Xn(t.projectRoot,o)}):wr[s]&&r.mcpHints.push(wr[s])}if(t.instructions!==!1){let s=Qn(),o=O.join(t.projectRoot,"AGENTS.md");r.instructionFiles.push({file:"AGENTS.md",outcome:vr(o,s)});let i=O.join(t.projectRoot,"CLAUDE.md");E.existsSync(i)&&r.instructionFiles.push({file:"CLAUDE.md",outcome:vr(i,s)})}if(t.scaffold!==!1){let s=O.join(t.projectRoot,"saffron.config.json");E.existsSync(s)||(E.writeFileSync(s,JSON.stringify({baseURL:"http://localhost:3000",features:"features"},null,2)+`
|
|
277
|
+
`),r.scaffolded.push("saffron.config.json"));let o=O.join(t.projectRoot,"features");E.existsSync(o)||(E.mkdirSync(o,{recursive:!0}),r.scaffolded.push("features/"));let i=O.join(t.projectRoot,".gitignore"),a=".saffron/reports/",d=E.existsSync(i)?E.readFileSync(i,"utf8"):"";d.split(`
|
|
277
278
|
`).some(p=>p.trim()===a)||(E.writeFileSync(i,d+(d&&!d.endsWith(`
|
|
278
279
|
`)?`
|
|
279
280
|
`:"")+a+`
|
|
280
|
-
`),
|
|
281
|
-
`)}function
|
|
282
|
-
`))}function Ie(t){console.error(`saffron: ${t.message}`),process.exit(2)}function
|
|
283
|
-
`)[0]}`),S.error&&console.log(` ${S.error}`)};await d.start();try{if(m>1&&i.length>1){let S=i.map((
|
|
284
|
-
${
|
|
285
|
-
${x}`);let
|
|
286
|
-
`)[0]})
|
|
287
|
-
${
|
|
288
|
-
|
|
289
|
-
`);
|
|
290
|
-
${r.meta.suggestedFeatureEdit.
|
|
281
|
+
`),r.scaffolded.push(".gitignore (+ .saffron/reports/)"))}return r}import Zn from"node:path";function Tr(t){return["saffron lsp is a language server: it speaks LSP over stdio and is meant","to be launched by your editor, not run by hand (nothing happens here).","","JetBrains (IntelliJ, WebStorm, PyCharm, Rider, incl. Community):"," 1. Completion, diagnostics, hover, go-to-definition: install the free"," LSP4IJ plugin, then Settings \u2192 Languages & Frameworks \u2192 Language"," Servers \u2192 +: command `npx saffron lsp`, working dir = project root,"," file patterns *.saffron and *.feature."," 2. Syntax highlighting comes from a TextMate bundle, not the LSP:"," Settings \u2192 Editor \u2192 TextMate Bundles \u2192 + and pick this folder:",` ${t?Zn.join(t,"textmate","saffron"):"node_modules/saffron-ai/textmate/saffron"}`,"","Neovim / other LSP editors: cmd = { 'npx', 'saffron', 'lsp' },","filetypes = { 'saffron', 'feature' }.","","VS Code: install the extension instead:"," code --install-extension ChathurangaJayasinghe.saffron-vscode"].join(`
|
|
282
|
+
`)}function ne(t){At(t);let e=R.join(t,"saffron.config.json");return I.existsSync(e)?JSON.parse(I.readFileSync(e,"utf8")):{}}function rs(t,e){let r=e.length>0?e:["features"],n=[];for(let s of r){let o=R.isAbsolute(s)?s:R.join(t,s);if(I.existsSync(o))if(I.statSync(o).isDirectory())for(let i of I.readdirSync(o,{recursive:!0})){let a=String(i);se(a)&&n.push(R.join(o,a))}else se(o)&&n.push(o)}return n.sort()}function Er(t,e,r){let n=R.isAbsolute(e.features??"features")?e.features:R.join(t,e.features??"features"),s=new Set(ue(n));for(let o of r)o.endsWith(".saffron")&&s.add(o);return X([...s].sort(),t)}function ns(){let t=process.env.ANTHROPIC_API_KEY||process.env.CLAUDE_CODE_OAUTH_TOKEN||process.env.CLAUDE_CODE_USE_BEDROCK||process.env.CLAUDE_CODE_USE_VERTEX,e=process.env.HOME??"",r=e&&(I.existsSync(R.join(e,".claude",".credentials.json"))||I.existsSync(R.join(e,".claude.json")));t||r||console.error(["note: no Claude credentials detected. Cached scenarios replay fine","without AI, but recording and healing need one of:"," \xB7 ANTHROPIC_API_KEY=sk-ant-... (console.anthropic.com \u2192 API keys)"," \xB7 a Claude Code login on this machine","Replay-only runs: add --no-agent to silence this note.",""].join(`
|
|
283
|
+
`))}function Ie(t){console.error(`saffron: ${t.message}`),process.exit(2)}function it(t){return t>=1e6?`${(t/1e6).toFixed(1)}M`:t>=1e4?`${(t/1e3).toFixed(1)}k`:t.toLocaleString()}var ss={green:"\x1B[32m",yellow:"\x1B[33m",red:"\x1B[31m"},os="\x1B[0m",C=new ts().name("saffron").description("Gherkin-native AI-fallback test runner: zero-token cached replay, runtime AI healing.").version(Ce()).option("-p, --project-root <dir>","project root",process.cwd());C.command("run").description("Run feature files (cached replay; AI records/heals on demand)").argument("[paths...]","feature files or directories (default: features/)").option("--base-url <url>","base URL of the app under test").option("--headed","run the browser headed").option("--filter <tags>",'only run scenarios carrying any of these @tags: comma-separated ("@smoke,@WA-TC1") or repeat the flag',gr).option("--no-agent","disable AI fallback (cache misses/failures go red)").option("--rerecord","discard the committed caches of the selected scenarios and let the agent record them fresh (the honest fix for a wrong recording)").option("--model <model>","model for the agent").option("--storage-state <file>","Playwright storageState JSON for authenticated sessions").option("--strict","treat passed-with-adaptation (yellow) as failure until its proposal is reviewed").option("--no-verify","skip the zero-AI proof replay of agent recordings before filing proposals").option("--no-reuse","disable Level-1 step reuse (seeding new recordings from existing step recordings)").option("--browser <name>",'replay browser: "chromium" (default), "firefox", or "webkit", recording/healing always use Chromium').option("--heal-model <model>","cheaper model for heal sessions").option("--workers <n>","parallel replay workers (agent recording/healing stays sequential)").option("--assertion-policy <policy>",'assertion adaptability: "strict" (default) or "adaptable-mid" (mid-scenario assertions may be adapted; the final assertion block never is)').action(async(t,e)=>{let r=R.resolve(C.opts().projectRoot),n=ne(r),s=e.baseUrl??n.baseURL,o=rs(r,t.length>0?t:n.features?[n.features]:[]);o.length===0&&(console.error("No .feature/.saffron files found."),process.exit(2)),e.agent!==!1&&ns();let i;try{let S=Er(r,n,o);i=o.flatMap($=>Z($,r,S))}catch(S){throw S instanceof M&&Ie(S),S}let a=ur(e.filter);if(a.length>0&&(i=fr(i,a),i.length===0&&(console.error(`No scenarios match --filter ${a.join(",")} in ${o.length} feature file(s).`),process.exit(2))),e.rerecord){e.agent===!1&&(console.error("saffron: --rerecord needs the agent; drop --no-agent"),process.exit(2));let S=0;for(let $ of i){let T=z(r,$.featurePath,$.name);I.existsSync(T)&&(I.rmSync(T),S++)}console.log(`rerecord: removed ${S} cache file(s) for ${i.length} scenario(s); the agent records them fresh (seeded from the other recordings). Restore with git if needed.`)}{let S=new Map;for(let $ of i){let T=ze($),P=N(z(r,$.featurePath,$.name));P&&T.push(JSON.stringify(P.steps));for(let _ of Ve(T))process.env[_]===void 0&&!S.has(_)&&S.set(_,$.displayName)}if(S.size>0){for(let[$,T]of S)console.error(`missing environment variable ${$} (referenced as {env:${$}} by "${T}")`);console.error("set the variable(s), or add them to a git-ignored .env in the project root"),process.exit(2)}}let d=new ke({projectRoot:r,baseURL:s,headed:e.headed,actionTimeoutMs:n.actionTimeoutMs,retries:n.retries,storageState:e.storageState??n.storageState,browser:(()=>{let S=e.browser??n.browser??"chromium";return S!=="chromium"&&S!=="firefox"&&S!=="webkit"&&(console.error(`invalid --browser "${S}": use "chromium", "firefox", or "webkit"`),process.exit(2)),S})(),healModel:e.healModel??n.healModel,verifyProposals:e.verify===!1?!1:n.verifyProposals,reuseSteps:e.reuse===!1?!1:n.reuseSteps,assertionPolicy:(()=>{let S=e.assertionPolicy??n.assertionPolicy??"strict";return S!=="strict"&&S!=="adaptable-mid"&&(console.error(`invalid --assertion-policy "${S}": use "strict" or "adaptable-mid"`),process.exit(2)),S})(),agent:e.agent?new $e({model:e.model??n.model,maxTurns:n.maxTurns,snapshotMode:n.snapshotMode}):void 0}),p=new Date,m=Math.max(1,Number.parseInt(String(e.workers??n.workers??1),10)||1),l=new Array(i.length),c=S=>{let $=S.scenario,T=ss[S.status],P=S.usage.inputTokens+S.usage.outputTokens,_=S.usage.cacheReadTokens+S.usage.cacheCreationTokens,Cr=_>0?`, ${it(_)} cache traffic`:"";console.log(`${T}${S.status.toUpperCase().padEnd(7)}${os}${$.displayName} (${S.source}, ${S.durationMs}ms, ${S.usage.aiCalls} AI calls, ${P} tokens${Cr})`);for(let _e of S.adaptations)console.log(` \u26A0 ${_e}`);S.seededSteps&&console.log(` \u26A1 ${S.seededSteps}/${S.stepResults.length} steps seeded from existing recordings (zero AI)`);let je=S.stepResults.reduce((_e,Pr)=>_e+(Pr.retries??0),0);je&&console.log(` \u21BB ${je} action ${je===1?"retry":"retries"} absorbed before settling (retries: ${n.retries??1} per action)`),S.verified===!0?console.log(" \u2713 proposal verified by zero-AI proof replay"):S.verified===!1&&console.log(` \u2717 proposal UNVERIFIED, proof replay failed: ${S.proofError?.split(`
|
|
284
|
+
`)[0]}`),S.error&&console.log(` ${S.error}`)};await d.start();try{if(m>1&&i.length>1){let S=i.map((T,P)=>({s:T,i:P})),$=[];await Promise.all(Array.from({length:Math.min(m,S.length)},async()=>{for(;;){let T=S.shift();if(!T)return;let P=await d.runScenario(T.s,{agentDisabled:!0});P.status==="red"?$.push(T):(l[T.i]=P,c(P))}}));for(let T of $.sort((P,_)=>P.i-_.i)){let P=await d.runScenario(T.s);l[T.i]=P,c(P)}}else for(let S=0;S<i.length;S++){let $=await d.runScenario(i[S]);l[S]=$,c($)}}finally{await d.stop()}let g=dr(l,p,s),h=H(r).divergent.length,u=yr(l,p,h,s),f=xr(r);g.trends=br(f,u),Sr(r,u);let{html:y,json:x}=pr(r,g),w=g.totals,F=w.cacheReadTokens+w.cacheCreationTokens;console.log(`
|
|
285
|
+
${w.green} passed, ${w.yellow} adapted, ${w.red} failed \xB7 ${w.aiCalls} AI calls \xB7 ${(w.inputTokens+w.outputTokens).toLocaleString()} tokens in+out${F>0?` \xB7 ${it(w.cacheReadTokens)} cache read \xB7 ${it(w.cacheCreationTokens)} cache written`:""} \xB7 $${w.costUsd.toFixed(4)}`);let v=g.trends;if(v?.prev){let S=v.prev.passRatePp,$=v.prev.costUsd;console.log(`vs previous run: pass rate ${S>=0?"+":""}${S}pp \xB7 cost ${$>=0?"+":"-"}$${Math.abs($).toFixed(4)} \xB7 AI calls ${v.prev.aiCalls>=0?"+":""}${v.prev.aiCalls}`)}for(let S of v?.chronic??[])console.log(`\u26A0 chronic: ${S.feature} \u203A ${S.scenario}: ${S.heals} heal(s), ${S.reds} red(s) in last ${S.runs} run(s); re-record or fix the feature text instead of paying for more heals`);console.log(`report: ${y}
|
|
286
|
+
${x}`);let b=ce(R.resolve(C.opts().projectRoot));b.length>0&&console.log(`${b.length} cache proposal(s) pending. Review with: saffron accept | saffron reject`);let j=e.strict??n.strict??!1;j&&w.yellow>0&&console.log(`strict mode: ${w.yellow} adapted scenario(s) treated as failure. Review proposals to go green.`),process.exit(w.red>0||j&&w.yellow>0?1:0)});C.command("accept").description("Promote pending cache proposals to committed caches").argument("[file]","proposal file (default with --all: every proposal)").option("--all","accept all pending proposals").option("--with-feature-edit","also apply the proposal's suggested .feature step rewrites").option("--propagate","apply the same locator fixes to every other cache using the same locator").option("--include-unverified","with --all: also accept proposals whose proof replay failed").action((t,e)=>{let r=R.resolve(C.opts().projectRoot),n=ne(r),s=t?[R.resolve(t)]:e.all?ce(r).map(o=>o.file):[];if(s.length===0){Ar(r);return}for(let o of s){let i=xe(o);if(i.meta.verified===!1){if(e.all&&!e.includeUnverified){console.log(`skipped UNVERIFIED ${R.basename(o)} (${i.meta.proofError?.split(`
|
|
287
|
+
`)[0]}). Re-record it (saffron run --rerecord), or accept it explicitly by file, or pass --include-unverified.`);continue}console.log(`\u26A0 accepting UNVERIFIED proposal (proof replay failed: ${i.meta.proofError?.split(`
|
|
288
|
+
`)[0]}): expect a heal or red on the next run.`)}let a=N(z(r,i.cache.feature,i.cache.scenario)),d=Pt(r,o);console.log(`accepted \u2192 ${d}`);let p=a?rr(a,i.cache):[];if(p.length>0){let c=nr(r,p,d,!e.propagate);for(let g of p)console.log(` locator fix: ${Xe(g.from)} \u2192 ${Xe(g.to)}`);if(c.length===0)console.log(" no other caches use this locator.");else if(e.propagate)for(let g of c)console.log(` propagated \u2192 ${g.feature} \u203A ${g.scenario} (${g.steps.map(h=>`"${h.gherkin}"`).join(", ")})`);else{console.log(` ${c.length} other cache(s) use the same locator: re-run with --propagate to fix them too:`);for(let g of c)console.log(` \xB7 ${g.feature} \u203A ${g.scenario}`)}}let m=i.meta.featureEdits??[];if(!e.withFeatureEdit||m.length===0)continue;let l;try{l=or(r,i.cache.feature,i.cache.scenario,m,Er(r,n,[]))}catch(c){throw c instanceof M&&Ie(c),c}for(let c of l.applied){let g=c.fromStepSet?`${c.file}:${c.line} (StepSet "${c.fromStepSet}", fixes every invoking scenario)`:`${c.file}:${c.line}`;console.log(` feature edit ${g}:`),console.log(` - ${c.from}`),console.log(` + ${c.to}`)}for(let c of l.skipped)console.log(` skipped edit [${c.index}]: ${c.reason}`);if(l.applied.length>0){let c=N(d);for(let g of m)c.steps[g.index]&&(c.steps[g.index].gherkin=g.text);ee(d,c)}}});C.command("steps").description("List/search the project's step vocabulary (files + caches): what exists, what's recorded, what to reuse").argument("[search]","case-insensitive substring filter").option("--json","machine-readable vocabulary").option("--snippets","write .vscode/saffron.code-snippets for native VS Code completion").action((t,e)=>{let r=R.resolve(C.opts().projectRoot),n=ne(r),s;try{s=G(r,n.features??"features")}catch(i){throw i instanceof M&&Ie(i),i}if(t&&(s=me(s,t)),e.json){console.log(JSON.stringify(s,null,2));return}if(e.snippets){let i=R.join(r,".vscode","saffron.code-snippets");I.mkdirSync(R.dirname(i),{recursive:!0}),I.writeFileSync(i,bt(s)),console.log(`wrote ${s.steps.length} step + ${s.stepSets.length} StepSet snippet(s) \u2192 ${i}`),console.log("re-run after recording sessions to keep completions current");return}let o={recorded:"\x1B[32m\u25CF\x1B[0m",divergent:"\x1B[33m\u25CF\x1B[0m",unrecorded:"\u25CB"};if(s.stepSets.length>0){console.log("step sets:");for(let i of s.stepSets)console.log(` ${i.name} (${i.steps.length} steps, ${i.usage} scenario(s)), ${i.file}:${i.line}`);console.log("")}for(let i of s.steps){let a=i.fromStepSet?` [${i.fromStepSet}]`:"";console.log(`${o[i.status]} ${i.keyword} ${i.text} \xD7${i.usage}${a}`)}console.log(`
|
|
289
|
+
${s.steps.length} step(s) across ${s.scannedFiles} file(s) \xB7 ${s.scannedCaches} cache(s) \xB7 \x1B[32m\u25CF\x1B[0m recorded (replays free) \xB7 \x1B[33m\u25CF\x1B[0m divergent \xB7 \u25CB unrecorded`)});C.command("author").description("Draft a feature file from plain-paragraph requirements, reusing the project's step vocabulary (AI)").argument("<prose-file>","text/markdown file with the requirements").option("-o, --out <file>","output path (default: features/<name>.saffron)").option("--model <model>","model for authoring").action(async(t,e)=>{let r=R.resolve(C.opts().projectRoot),n=ne(r),s=I.readFileSync(R.resolve(t),"utf8"),o;try{o=G(r,n.features??"features")}catch(p){throw p instanceof M&&Ie(p),p}console.log(`authoring with ${o.steps.length} known step(s) and ${o.stepSets.length} step set(s) as vocabulary\u2026`);let i=await wt(s,o,e.model??n.model),a=e.out?R.resolve(e.out):R.join(r,n.features??"features",`${R.basename(t).replace(/\.[^.]+$/,"")}.saffron`);I.existsSync(a)&&(console.error(`refusing to overwrite ${a}: pass --out for a new path`),process.exit(1)),I.mkdirSync(R.dirname(a),{recursive:!0}),I.writeFileSync(a,i.content);let d=i.content.split(`
|
|
290
|
+
`).map(p=>p.trim().replace(/^(Given|When|Then|And|But|\*)\s+/,"")).filter(p=>o.steps.some(m=>m.text===p)).length;console.log(`wrote ${a}`),console.log(`${d} step line(s) reuse the existing vocabulary (seedable) \xB7 $${(i.usage.costUsd??0).toFixed(4)}`),console.log("review the draft, then record it: saffron run "+R.relative(process.cwd(),a))});C.command("mcp").description("Serve the step vocabulary to AI assistants over stdio MCP (search_steps, list_step_sets)").action(async()=>{let t=R.resolve(C.opts().projectRoot),e=ne(t);await vt(t,e.features??"features")});C.command("lsp").description("Run the Saffron language server over stdio (completion, go-to-definition, hover, diagnostics). For JetBrains (LSP4IJ / Ultimate LSP API), Neovim, and any LSP-capable editor").action(()=>{process.stdin.isTTY&&(console.error(Tr(Ae())),process.exit(2));let t=R.resolve(C.opts().projectRoot),e=ne(t);Rt(t,e.features??"features")});C.command("reject").description("Discard pending cache proposals (AI will retry next run)").argument("[file]","proposal file").option("--all","reject all pending proposals").action((t,e)=>{let r=R.resolve(C.opts().projectRoot),n=t?[R.resolve(t)]:e.all?ce(r).map(s=>s.file):[];if(n.length===0){Ar(r);return}for(let s of n)Ft(s),console.log(`rejected ${s}`)});C.command("init").description("Make the project agent-ready: install the bundled Saffron skill into agent directories, add instructions to AGENTS.md, scaffold config").option("--agents <list>",`comma-separated targets: ${Object.keys(pe).join(", ")} (default: claude,agents)`).option("--no-instructions","do not touch AGENTS.md / CLAUDE.md").option("--no-scaffold","do not create saffron.config.json, features/, .gitignore entry").option("--no-mcp","do not register `saffron mcp` in project MCP configs (.mcp.json, .cursor/mcp.json, .vscode/mcp.json)").action(t=>{let e=R.resolve(C.opts().projectRoot),r=Ae();r||(console.error("saffron: cannot locate the installed package root"),process.exit(2));let n=t.agents?String(t.agents).split(",").map(o=>o.trim()).filter(Boolean):void 0;for(let o of n??[])o in pe||(console.error(`saffron: unknown agent target "${o}" (expected one of ${Object.keys(pe).join(", ")})`),process.exit(2));let s=Rr({projectRoot:e,packageRoot:r,targets:n,instructions:t.instructions,scaffold:t.scaffold,mcp:t.mcp});for(let o of s.skillDirs)console.log(`skill \u2192 ${o}/`);for(let o of s.mcpFiles)console.log(`${o.outcome.padEnd(9)} \u2192 ${o.file} (saffron mcp server)`);for(let o of s.mcpHints)console.log(`mcp hint \u2192 ${o}`);for(let o of s.instructionFiles)console.log(`${o.outcome.padEnd(9)} \u2192 ${o.file} (managed Saffron block)`);for(let o of s.scaffolded)console.log(`created \u2192 ${o}`);console.log("\nAgents now load the saffron skill when writing tests. Re-run after `npm update saffron-ai` to refresh it.")});C.command("report").description("Open the latest HTML report").option("--open","open in the default browser",!0).action(()=>{let t=R.resolve(C.opts().projectRoot),e=R.join(t,".saffron","reports","latest.html");I.existsSync(e)||(console.error("No report found. Run `saffron run` first."),process.exit(2)),console.log(e);let r=process.platform==="darwin"?"open":"xdg-open";es(r,[e],{detached:!0,stdio:"ignore"}).unref()});function Ar(t){let e=ce(t);if(e.length===0){console.log("No pending proposals.");return}console.log(`Pending proposals:
|
|
291
|
+
`);for(let{file:r,proposal:n}of e){console.log(` ${r}`);let s=n.meta.verified===!0?" \xB7 verified \u2713":n.meta.verified===!1?" \xB7 UNVERIFIED \u2717":"";console.log(` ${n.cache.feature} \u203A ${n.cache.scenario} (${n.meta.mode}, ${n.meta.createdAt}${s})`),console.log(` ${n.meta.narrative}`);for(let o of n.meta.adaptations)console.log(` \u26A0 ${o}`);n.meta.suggestedFeatureEdit&&console.log(` suggested .feature edit:
|
|
292
|
+
${n.meta.suggestedFeatureEdit.split(`
|
|
291
293
|
`).map(o=>` ${o}`).join(`
|
|
292
|
-
`)}`),console.log()}console.log("Accept with: saffron accept <file> | --all")}
|
|
294
|
+
`)}`),console.log()}console.log("Accept with: saffron accept <file> | --all")}C.parseAsync();
|
package/package.json
CHANGED
package/skills/saffron/SKILL.md
CHANGED
|
@@ -119,6 +119,7 @@ npx saffron run --no-agent # replay only (CI without AI access)
|
|
|
119
119
|
npx saffron accept # list pending proposals
|
|
120
120
|
npx saffron accept --all # promote verified proposals to caches
|
|
121
121
|
npx saffron report # open the HTML report
|
|
122
|
+
npx saffron run --rerecord --filter @t # a recording is wrong: record it fresh
|
|
122
123
|
```
|
|
123
124
|
|
|
124
125
|
Results: **green** = cached pass · **yellow** = passed with AI recording
|
|
@@ -142,7 +143,7 @@ re-record (mostly seeded from existing step recordings).
|
|
|
142
143
|
| Put a password literal in a step | `{env:VAR}` |
|
|
143
144
|
| Sleep/wait "for 3 seconds" | Wait for a UI signal or an API response |
|
|
144
145
|
| One 30-step scenario | Several 5–10 step scenarios + StepSets |
|
|
145
|
-
| Edit cache JSON to fix a test | Edit the scenario text
|
|
146
|
+
| Edit cache JSON to fix a test | Edit the scenario text and re-run, or `saffron run --rerecord` if the recording itself is wrong |
|
|
146
147
|
|
|
147
148
|
## References
|
|
148
149
|
|
|
@@ -39,6 +39,7 @@ your-project/
|
|
|
39
39
|
|---|---|
|
|
40
40
|
| `baseURL` | App under test; steps say "the login page", not full URLs |
|
|
41
41
|
| `storageState` | Playwright storage-state JSON so replays and the agent start authenticated (`npx playwright open --save-storage=.auth/state.json <url>`) |
|
|
42
|
+
| `retries` | Extra attempts per action with backoff before a step fails and the agent heals (default 1). Not a scenario re-run; reports show a `retried ×N` chip |
|
|
42
43
|
| `strict` | Yellow (passed-with-adaptation) exits 1 until reviewed: cached-green-only CI |
|
|
43
44
|
| `assertionPolicy` | `strict` (default) or `adaptable-mid`; the final assertion block is always strict |
|
|
44
45
|
| `verifyProposals` | Proof-replay every recording zero-AI before filing (default true) |
|