e2e-ai 1.4.2 → 1.4.3

package/README.md

@@ -598,6 +598,16 @@ If e2e-ai is installed globally or as a project dependency, you can use the bina
 
 ### Available Tools
 
+#### Orchestration (workflow automation)
+
+| Tool | Description | Input |
+|------|-------------|-------|
+| `e2e_ai_plan_workflow` | Plan an automation workflow — returns an ordered todo list of steps | `goal`, `key?`, `from?`, `skip?`, `voice?`, `trace?`, `scanDir?` |
+| `e2e_ai_execute_step` | Execute a single pipeline step | `step`, `key?`, `voice?`, `trace?`, `scanDir?`, `output?`, `extraArgs?` |
+| `e2e_ai_get_workflow_guide` | Get the full workflow guide explaining how the pipeline works | (none) |
+
+#### Project setup
+
 | Tool | Description | Input |
 |------|-------------|-------|
 | `e2e_ai_scan_codebase` | Scan project for test files, configs, fixtures, path aliases, and sample test content | `projectRoot?` (defaults to cwd) |
@@ -605,16 +615,35 @@ If e2e-ai is installed globally or as a project dependency, you can use the bina
 | `e2e_ai_read_agent` | Load an agent prompt by name — returns system prompt + config | `agentName` (e.g. `scenario-agent`) |
 | `e2e_ai_get_example` | Get the example context markdown template | (none) |
 
-### Usage with AI Assistants
-
-Once configured, an AI assistant can:
-
-1. **Scan your project** to understand its test structure, fixtures, and conventions
-2. **Read agent prompts** to understand how each pipeline step works
-3. **Validate context files** to ensure they have the right format before running commands
-4. **Get the example template** as a starting point for writing `e2e-ai.context.md`
-
-This enables AI assistants to help you set up e2e-ai, debug pipeline issues, and generate better project context files.
+### How AI Orchestration Works
+
+The MCP server includes built-in orchestration instructions that teach AI assistants (Claude Code, Cursor, etc.) how to run e2e-ai workflows autonomously. The protocol is:
+
+1. **Plan** — The AI calls `e2e_ai_plan_workflow` with your goal. It returns an ordered step list.
+2. **Approve** — The AI presents the plan to you for review. You can adjust steps before proceeding.
+3. **Execute** — The AI runs each step one at a time via `e2e_ai_execute_step`, reporting results between steps. If a step fails, it stops and asks you how to proceed.
+
+Each step is executed as a separate job (ideally a subagent) to keep context clean. The AI never runs multiple pipeline steps at once.
+
+**Example interaction:**
+
+> **You:** "Run the full test pipeline for PROJ-101"
+>
+> **AI:** *Calls `e2e_ai_plan_workflow`*, then presents:
+> 1. `record` — Launch browser codegen + voice recording
+> 2. `transcribe` — Transcribe voice via Whisper
+> 3. `scenario` — Generate YAML test scenario
+> 4. `generate` — Generate Playwright test
+> 5. `refine` — Refactor test with AI
+> 6. `test` — Run Playwright test
+> 7. `heal` — Self-heal if failing (can skip if test passes)
+> 8. `qa` — Generate QA documentation
+>
+> "Does this look right? Ready to start?"
+>
+> **You:** "Skip voice, go ahead"
+>
+> **AI:** *Removes transcribe, executes each step sequentially*
 
 ## Library API
 

package/dist/mcp.js

@@ -14856,8 +14856,9 @@ class StdioServerTransport {
 }
 
 // src/mcp.ts
-import { readFileSync as readFileSync2 } from "node:fs";
+import { readFileSync as readFileSync2, existsSync as existsSync2 } from "node:fs";
 import { join as join2 } from "node:path";
+import { execSync } from "node:child_process";
 
 // src/utils/scan.ts
 import { readdirSync, existsSync, readFileSync } from "node:fs";
@@ -14957,13 +14958,281 @@ function validateContext(content) {
 }
 
 // src/mcp.ts
-var server = new McpServer({
-  name: "e2e-ai",
-  version: "1.1.2"
-});
+var SERVER_INSTRUCTIONS = `
+# e2e-ai — Orchestration Guide
+
+You have access to e2e-ai, an AI-powered E2E test automation tool. Follow this protocol when the user asks you to perform any e2e-ai automation.
+
+## Core Principle: Plan → Approve → Execute Step-by-Step
+
+NEVER run multiple pipeline steps at once. Each step is a separate job with its own context.
+
+## Protocol
+
+1. **Plan first.** Call \`e2e_ai_plan_workflow\` with the user's goal. This returns a structured todo list of steps.
+2. **Present the plan.** Show the user the ordered step list with descriptions. Ask for confirmation or adjustments before proceeding.
+3. **Execute one step at a time.** For each step in the approved plan:
+   a. Tell the user which step you're about to run and why.
+   b. Call \`e2e_ai_execute_step\` with the step name and parameters.
+   c. Report the result to the user (success, key output, any warnings).
+   d. If the step fails, stop and discuss with the user before continuing.
+   e. Move to the next step only after the current one succeeds.
+4. **Use subagents when available.** If your AI platform supports subagents (e.g., Claude Code Agent tool), dispatch each step as a dedicated subagent to preserve context. Each subagent should:
+   - Receive only the context it needs (step name, key, relevant file paths)
+   - Call \`e2e_ai_execute_step\` to do its work
+   - Return the result to the orchestrator
+
+## Step Dependencies
+
+Steps produce artifacts that feed into later steps. The pipeline handles this automatically — each step picks up where the previous one left off. Do not skip steps unless the plan says a step can be skipped.
+
+## Interactive Steps
+
+The \`record\` step opens a browser and requires user interaction. When the plan includes \`record\`:
+- Tell the user they need to interact with the browser window
+- The step will block until they close the codegen window
+- After recording completes, proceed with the next step
+
+## When Things Fail
+
+- If \`test\` fails and \`heal\` is in the plan, that's expected — heal will attempt to fix it
+- If \`heal\` exhausts all retries, stop and show the user the last error output
+- For any other failure, stop and ask the user how to proceed
+
+## Available Workflows
+
+- **Full test pipeline**: record → transcribe → scenario → generate → refine → test → heal → qa
+- **From existing recording**: transcribe → scenario → generate → refine → test → heal → qa
+- **AI-only (no recording)**: scenario → generate → refine → test → heal → qa
+- **Generate from scenario**: generate → refine → test → heal → qa
+- **Test + heal loop**: test → heal
+- **Scanner pipeline**: scan → analyze → push
+- **Single step**: any individual command
+
+Always use \`e2e_ai_plan_workflow\` to determine the right steps — don't guess.
+`.trim();
+var TEST_PIPELINE_STEPS = [
+  {
+    name: "record",
+    description: "Launch Playwright codegen in the browser. Optionally records voice narration for richer test scenarios.",
+    produces: "codegen .ts file + optional .wav voice recording",
+    requires: "none",
+    interactive: true
+  },
+  {
+    name: "transcribe",
+    description: "Transcribe the voice recording via OpenAI Whisper. Merges timestamped voice comments into the codegen file.",
+    produces: "transcript JSON + annotated codegen file",
+    requires: "voice recording from record step",
+    interactive: false,
+    canSkip: "No voice recording exists or voice is disabled"
+  },
+  {
+    name: "scenario",
+    description: "AI analyzes the codegen + transcript and generates a structured YAML test scenario with semantic steps and expected results.",
+    produces: "YAML test scenario file",
+    requires: "codegen file (+ optional transcript)",
+    interactive: false
+  },
+  {
+    name: "generate",
+    description: "AI converts the YAML scenario into a complete Playwright .test.ts file using project conventions from context.md.",
+    produces: "Playwright .test.ts file",
+    requires: "YAML scenario file",
+    interactive: false
+  },
+  {
+    name: "refine",
+    description: "AI refactors the test: replaces raw selectors with semantic alternatives, adds proper timeouts, uses project helpers.",
+    produces: "improved .test.ts file (in-place)",
+    requires: "Playwright .test.ts file",
+    interactive: false
+  },
+  {
+    name: "test",
+    description: "Run the Playwright test with trace/video/screenshot capture. Reports pass/fail status.",
+    produces: "test results + trace files",
+    requires: "Playwright .test.ts file",
+    interactive: false
+  },
+  {
+    name: "heal",
+    description: "If the test failed, AI diagnoses the failure and patches the test. Retries up to 3 times with different strategies.",
+    produces: "patched .test.ts file (if test was failing)",
+    requires: "failing test + error output",
+    interactive: false,
+    canSkip: "Test already passes"
+  },
+  {
+    name: "qa",
+    description: "Generate formal QA documentation: markdown test case with preconditions, steps table, and optional Zephyr XML export.",
+    produces: "QA markdown + optional Zephyr XML",
+    requires: "Playwright .test.ts file + scenario",
+    interactive: false
+  }
+];
+var SCANNER_PIPELINE_STEPS = [
+  {
+    name: "scan",
+    description: "Scan the codebase AST: extract routes, components, hooks, imports, and dependency graph.",
+    produces: "ast-scan.json with full codebase structure",
+    requires: "none",
+    interactive: false
+  },
+  {
+    name: "analyze",
+    description: "AI analyzes the AST scan to identify features, workflows, components, and generate test scenarios.",
+    produces: "qa-map.json with features, workflows, scenarios",
+    requires: "ast-scan.json from scan step",
+    interactive: false
+  },
+  {
+    name: "push",
+    description: "Push the QA map to a remote API endpoint for integration with external tools.",
+    produces: "push confirmation with version info",
+    requires: "qa-map.json from analyze step + API config",
+    interactive: false
+  }
+];
+var ALL_STEPS = [...TEST_PIPELINE_STEPS, ...SCANNER_PIPELINE_STEPS];
+function planWorkflow(goal, options) {
+  const goalLower = goal.toLowerCase();
+  const notes = [];
+  const isScannerGoal = /\b(scan|analyze|qa.?map|feature.?analy|push.?qa|codebase.?scan)\b/.test(goalLower);
+  const isSingleStep = ALL_STEPS.some((s) => goalLower === s.name || goalLower === `run ${s.name}`);
+  let stepDefs;
+  if (isScannerGoal && !isSingleStep) {
+    stepDefs = [...SCANNER_PIPELINE_STEPS];
+    if (!/\bpush\b/.test(goalLower)) {
+      stepDefs = stepDefs.filter((s) => s.name !== "push");
+      notes.push("Push step excluded — add it if you want to upload the QA map to a remote API.");
+    }
+  } else if (isSingleStep) {
+    const stepName = ALL_STEPS.find((s) => goalLower.includes(s.name)).name;
+    stepDefs = ALL_STEPS.filter((s) => s.name === stepName);
+  } else {
+    stepDefs = [...TEST_PIPELINE_STEPS];
+    if (options.from) {
+      const fromIdx = stepDefs.findIndex((s) => s.name === options.from);
+      if (fromIdx > 0) {
+        const skipped = stepDefs.slice(0, fromIdx).map((s) => s.name);
+        stepDefs = stepDefs.slice(fromIdx);
+        notes.push(`Starting from "${options.from}" — skipping: ${skipped.join(", ")}`);
+      }
+    } else {
+      if (/\b(from recording|existing recording|already recorded)\b/.test(goalLower)) {
+        stepDefs = stepDefs.filter((s) => s.name !== "record");
+        notes.push("Skipping record — using existing recording files.");
+      }
+      if (/\b(from scenario|existing scenario|manual scenario|yaml)\b/.test(goalLower)) {
+        stepDefs = stepDefs.filter((s) => !["record", "transcribe", "scenario"].includes(s.name));
+        notes.push("Starting from generate — using existing scenario YAML.");
+      }
+      if (/\b(generate.?only|just.?generate|no.?record)\b/.test(goalLower)) {
+        stepDefs = stepDefs.filter((s) => !["record", "transcribe"].includes(s.name));
+      }
+      if (/\b(test.?and.?heal|test.?heal|heal.?loop|fix.?test|self.?heal)\b/.test(goalLower)) {
+        stepDefs = stepDefs.filter((s) => ["test", "heal"].includes(s.name));
+      }
+      if (/\b(refine|refactor)\b/.test(goalLower) && !/\brun\b/.test(goalLower)) {
+        stepDefs = stepDefs.filter((s) => s.name === "refine");
+      }
+      if (/\bqa\b/.test(goalLower) && /\b(doc|only|generate)\b/.test(goalLower)) {
+        stepDefs = stepDefs.filter((s) => s.name === "qa");
+      }
+    }
+  }
+  if (options.skip?.length) {
+    stepDefs = stepDefs.filter((s) => !options.skip.includes(s.name));
+    notes.push(`Skipping: ${options.skip.join(", ")}`);
+  }
+  if (options.voice === false) {
+    stepDefs = stepDefs.filter((s) => s.name !== "transcribe");
+    notes.push("Voice disabled — transcribe step removed.");
+  }
+  const cliBase = "e2e-ai";
+  const steps = stepDefs.map((s, i) => {
+    const args = [s.name];
+    if (options.key && !["scan", "analyze", "push"].includes(s.name)) {
+      args.push("--key", options.key);
+    }
+    if (s.name === "record") {
+      if (options.voice === false)
+        args.push("--no-voice");
+      if (options.trace === false)
+        args.push("--no-trace");
+    }
+    if (s.name === "scan" && options.scanDir) {
+      args.push("--scan-dir", options.scanDir);
+    }
+    return {
+      order: i + 1,
+      name: s.name,
+      description: s.description,
+      command: `${cliBase} ${args.join(" ")}`,
+      produces: s.produces,
+      interactive: s.interactive,
+      canSkip: s.canSkip
+    };
+  });
+  const pipeline2 = isScannerGoal ? "scanner" : isSingleStep ? "single" : "test";
+  if (!options.key && pipeline2 === "test" && steps.length > 1) {
+    notes.push("No --key provided. Use --key <ISSUE-KEY> to organize files by issue.");
+  }
+  return { goal, pipeline: pipeline2, steps, notes };
+}
+function executeStep(stepName, options) {
+  const args = [stepName];
+  if (options.key && !["scan", "analyze", "push"].includes(stepName)) {
+    args.push("--key", options.key);
+  }
+  if (stepName === "record") {
+    if (options.voice === false)
+      args.push("--no-voice");
+    if (options.trace === false)
+      args.push("--no-trace");
+  }
+  if (stepName === "scan" && options.scanDir) {
+    args.push("--scan-dir", options.scanDir);
+  }
+  if (options.output) {
+    args.push("--output", options.output);
+  }
+  if (options.extraArgs?.length) {
+    args.push(...options.extraArgs);
+  }
+  const pkgRoot = getPackageRoot();
+  const cliBin = join2(pkgRoot, "dist", "cli.js");
+  const command = `node ${cliBin} ${args.join(" ")}`;
+  try {
+    const stdout = execSync(command, {
+      cwd: process.cwd(),
+      encoding: "utf-8",
+      timeout: 300000,
+      env: { ...process.env },
+      stdio: ["pipe", "pipe", "pipe"]
+    });
+    return { success: true, output: stdout, command };
+  } catch (err) {
+    const stderr = err.stderr?.toString() ?? "";
+    const stdout = err.stdout?.toString() ?? "";
+    return {
+      success: false,
+      output: `EXIT CODE: ${err.status ?? "unknown"}
+
+STDOUT:
+${stdout}
+
+STDERR:
+${stderr}`,
+      command
+    };
+  }
+}
+var server = new McpServer({ name: "e2e-ai", version: "1.2.0" }, { instructions: SERVER_INSTRUCTIONS });
 server.registerTool("e2e_ai_scan_codebase", {
   title: "Scan Codebase",
-  description: "Scan a project directory for test files, configs, fixtures, path aliases, and sample test content",
+  description: "Scan a project directory for test files, configs, fixtures, path aliases, and sample test content. Use this during project setup or to understand test infrastructure.",
   inputSchema: exports_external.object({
     projectRoot: exports_external.string().optional().describe("Project root directory (defaults to cwd)")
   })
@@ -14976,7 +15245,7 @@ server.registerTool("e2e_ai_scan_codebase", {
 });
 server.registerTool("e2e_ai_validate_context", {
   title: "Validate Context",
-  description: "Validate that a context markdown file contains all required sections",
+  description: "Validate that a context markdown file contains all required sections (Application, Test Infrastructure, Feature Methods, Import Conventions, Selector Conventions, Test Structure Template, Utility Patterns).",
   inputSchema: exports_external.object({
     content: exports_external.string().describe("The markdown content of the context file to validate")
   })
@@ -14988,7 +15257,7 @@ server.registerTool("e2e_ai_validate_context", {
 });
 server.registerTool("e2e_ai_read_agent", {
   title: "Read Agent",
-  description: "Read an agent prompt definition by name. Returns the agent name, system prompt, and config (model, max_tokens, temperature).",
+  description: "Read an agent prompt definition by name. Returns the agent system prompt and config. Agents: transcript-agent, scenario-agent, playwright-generator-agent, refactor-agent, self-healing-agent, qa-testcase-agent, feature-analyzer-agent, scenario-planner-agent, init-agent.",
   inputSchema: exports_external.object({
     agentName: exports_external.string().describe("Agent name (e.g. scenario-agent, playwright-generator-agent)")
   })
@@ -15014,7 +15283,7 @@ server.registerTool("e2e_ai_read_agent", {
 });
 server.registerTool("e2e_ai_get_example", {
   title: "Get Example Context",
-  description: "Returns the full example context markdown file that shows the expected format for .e2e-ai/context.md",
+  description: "Returns the full example context markdown file that shows the expected format for .e2e-ai/context.md.",
   inputSchema: exports_external.object({})
 }, async () => {
   try {
@@ -15030,6 +15299,87 @@ server.registerTool("e2e_ai_get_example", {
     };
   }
 });
+server.registerTool("e2e_ai_plan_workflow", {
+  title: "Plan Workflow",
+  description: "Plan an e2e-ai automation workflow. Call this FIRST when the user asks to run any automation. " + "Returns an ordered list of steps (todo list) that should be executed one at a time. " + "Present the plan to the user for approval before executing any step.",
+  inputSchema: exports_external.object({
+    goal: exports_external.string().describe('What the user wants to achieve. Examples: "run full pipeline for PROJ-101", ' + '"generate test from existing recording", "scan codebase and analyze features", ' + '"heal failing test PROJ-101", "refactor test PROJ-101"'),
+    key: exports_external.string().optional().describe("Issue key (e.g. PROJ-101, LIN-42)"),
+    from: exports_external.string().optional().describe("Start from a specific step (skip all prior steps)"),
+    skip: exports_external.array(exports_external.string()).optional().describe('Steps to skip (e.g. ["transcribe", "heal"])'),
+    voice: exports_external.boolean().optional().describe("Enable voice recording (default: true)"),
+    trace: exports_external.boolean().optional().describe("Enable trace capture (default: true)"),
+    scanDir: exports_external.string().optional().describe("Directory to scan (for scanner pipeline)")
+  })
+}, async ({ goal, key, from, skip, voice, trace, scanDir }) => {
+  const plan = planWorkflow(goal, { key, from, skip, voice, trace, scanDir });
+  return {
+    content: [{
+      type: "text",
+      text: JSON.stringify(plan, null, 2)
+    }]
+  };
+});
+server.registerTool("e2e_ai_execute_step", {
+  title: "Execute Pipeline Step",
+  description: "Execute a single e2e-ai pipeline step. Call this ONE STEP AT A TIME from an approved plan. " + "Each step produces artifacts consumed by later steps. " + "If your AI platform supports subagents, run each step in a dedicated subagent to preserve context. " + 'The "record" step is interactive and will open a browser window — the user must interact with it.',
+  inputSchema: exports_external.object({
+    step: exports_external.string().describe("Step name: record, transcribe, scenario, generate, refine, test, heal, qa, scan, analyze, push"),
+    key: exports_external.string().optional().describe("Issue key (e.g. PROJ-101)"),
+    voice: exports_external.boolean().optional().describe("Enable voice recording (record step only)"),
+    trace: exports_external.boolean().optional().describe("Enable trace capture (record step only)"),
+    scanDir: exports_external.string().optional().describe("Directory to scan (scan step only)"),
+    output: exports_external.string().optional().describe("Custom output path (scan/analyze steps)"),
+    extraArgs: exports_external.array(exports_external.string()).optional().describe("Additional CLI arguments")
+  })
+}, async ({ step, key, voice, trace, scanDir, output, extraArgs }) => {
+  const validSteps = ALL_STEPS.map((s) => s.name);
+  if (!validSteps.includes(step)) {
+    return {
+      content: [{
+        type: "text",
+        text: `Error: Unknown step "${step}". Valid steps: ${validSteps.join(", ")}`
+      }],
+      isError: true
+    };
+  }
+  const result = executeStep(step, { key, voice, trace, scanDir, output, extraArgs });
+  return {
+    content: [{
+      type: "text",
+      text: JSON.stringify({
+        step,
+        success: result.success,
+        command: result.command,
+        output: result.output
+      }, null, 2)
+    }]
+  };
+});
+server.registerTool("e2e_ai_get_workflow_guide", {
+  title: "Get Workflow Guide",
+  description: "Returns the e2e-ai workflow guide explaining how the pipeline works, step by step. Useful for understanding what each step does and how they connect.",
+  inputSchema: exports_external.object({})
+}, async () => {
+  try {
+    const guidePath = join2(getPackageRoot(), "templates", "workflow.md");
+    if (!existsSync2(guidePath)) {
+      return {
+        content: [{ type: "text", text: "Error: workflow.md not found in templates" }],
+        isError: true
+      };
+    }
+    const content = readFileSync2(guidePath, "utf-8");
+    return {
+      content: [{ type: "text", text: content }]
+    };
+  } catch (err) {
+    return {
+      content: [{ type: "text", text: `Error: ${err.message}` }],
+      isError: true
+    };
+  }
+});
 async function main() {
   const transport = new StdioServerTransport;
   await server.connect(transport);

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "e2e-ai",
   "description": "AI-powered test automation pipeline — record, transcribe, generate, heal and ship Playwright tests from a single CLI",
-  "version": "1.4.2",
+  "version": "1.4.3",
   "private": false,
   "type": "module",
   "bin": {

package/templates/workflow.md

@@ -14,6 +14,10 @@ record → transcribe → scenario → generate → refine → test → heal →
 
 **In short:** You record yourself testing in the browser (optionally narrating what you're doing), and e2e-ai turns that into a production-ready Playwright test with QA documentation.
 
+**Two ways to run it:**
+- **CLI**: Run commands directly (`e2e-ai run --key PROJ-101`)
+- **AI assistant**: Ask your AI tool (Claude Code, Cursor, etc.) — the MCP server guides it through the pipeline step by step, asking for your approval before starting
+
 ---
 
 ## Setup
@@ -198,6 +202,27 @@ This is independent from the test pipeline — use it to get an overview of your
 
 ---
 
+## AI-Assisted Workflow (MCP)
+
+If you have the e2e-ai MCP server configured, you can ask your AI assistant to run the pipeline for you. The MCP server teaches the AI how to orchestrate the workflow:
+
+1. **You say:** "Run the full test pipeline for PROJ-101" (or any variation)
+2. **AI plans:** Calls `e2e_ai_plan_workflow` → gets an ordered step list
+3. **AI shows plan:** Presents the steps and asks for your approval
+4. **You adjust:** "Skip voice" / "Start from generate" / "Looks good, go"
+5. **AI executes:** Runs each step one at a time via `e2e_ai_execute_step`, reporting results between steps
+
+Each step runs as a separate subagent (when supported by the AI platform) to keep context clean and focused. If a step fails, the AI stops and asks you what to do.
+
+**Example prompts you can give your AI assistant:**
+- "Run the full pipeline for PROJ-101"
+- "Generate a test from the existing recording for PROJ-101, skip voice"
+- "Just run test and heal for PROJ-101"
+- "Scan the codebase and analyze features"
+- "Refactor the test for PROJ-101"
+
+---
+
 ## File Structure
 
 After running the pipeline for `PROJ-101`: