e2e-ai 1.2.0 → 1.3.0

package/README.md

@@ -2,6 +2,8 @@
 
 AI-powered E2E test automation pipeline. Takes you from a manual browser recording all the way to a stable, documented Playwright test — with optional Zephyr integration.
 
+Includes a **codebase scanner** that builds a QA map of your application's features, workflows, and test scenarios using AI analysis.
+
 ## Quick Start
 
 ```bash
@@ -17,26 +19,40 @@ npx e2e-ai --help
 # Full pipeline for an issue
 npx e2e-ai run --key PROJ-101
 
-# Individual commands
-npx e2e-ai scenario --key PROJ-101
-npx e2e-ai generate --key PROJ-101
-npx e2e-ai qa --key PROJ-101
+# Scan your codebase and generate a QA map
+npx e2e-ai scan
+npx e2e-ai analyze
 ```
 
 ## Architecture
 
+e2e-ai has two main workflows:
+
+### Test Pipeline
+
 ```
-                                    AI Agents (LLM)
-                                         |
-  record -> transcribe -> scenario -> generate -> refine -> test -> heal -> qa
-    |            |            |           |          |         |       |      |
-  codegen     whisper    transcript   scenario    refactor  playwright self-  QA doc
-  + audio     + merge    + scenario     agent      agent    runner   healing  + export
-                agent      agent                                     agent   agent
+record -> transcribe -> scenario -> generate -> refine -> test -> heal -> qa
+  |            |            |           |          |         |       |      |
+codegen     whisper    transcript   scenario    refactor  playwright self-  QA doc
++ audio     + merge    + scenario     agent      agent    runner   healing  + export
+              agent      agent                                     agent   agent
 ```
 
 Each step produces an artifact that feeds the next. You can run the full pipeline or any step individually.
 
+### Scanner Pipeline
+
+```
+scan -> analyze -> push
+  |        |         |
+ AST    feature    remote
+ scan   analyzer   API
+        + scenario  upload
+        planner
+```
+
+Scans your codebase structure (routes, components, hooks), uses AI to identify features and workflows, generates test scenarios, and optionally pushes the QA map to a remote API.
+
 ## Commands
 
 ### `init` - Project Setup
@@ -50,6 +66,8 @@ npx e2e-ai init
 npx e2e-ai init --non-interactive
 ```
 
+---
+
 ### `record` - Browser Recording
 
 Launches Playwright codegen with optional voice recording and trace capture.
@@ -198,6 +216,115 @@ npx e2e-ai run --key PROJ-101 --from scenario --skip test,heal
 
 Each step's output feeds the next via `PipelineContext`. If a step fails, the pipeline stops (unless the step is marked `nonBlocking`).
 
+---
+
+### `scan` - Codebase AST Scanner
+
+Scans your codebase and extracts structural information: routes, components, hooks, imports, and dependencies.
+
+```bash
+# Scan using config defaults (scans src/)
+npx e2e-ai scan
+
+# Scan a specific directory
+npx e2e-ai scan --scan-dir app
+
+# Write output to a custom path
+npx e2e-ai scan --output my-scan.json
+
+# Disable file-level caching
+npx e2e-ai scan --no-cache
+```
+
+**What it does**:
+1. Collects all `.ts`, `.tsx`, `.js`, `.jsx` files matching configured include/exclude patterns
+2. Parses each file with a regex-based TypeScript parser to extract imports, exports, components, and hooks
+3. Extracts routes from Next.js App Router (`app/`) and Pages Router (`pages/`) conventions
+4. Caches results per-file (by content hash) for fast incremental re-scans
+
+**Output**: `ASTScanResult` JSON containing:
+- `files` — all scanned files with imports, exports, line counts
+- `routes` — extracted routes with paths, methods, dynamic segments, layout files
+- `components` — React components with props, hook calls, export status
+- `hooks` — custom hook definitions with dependencies
+- `dependencies` — import graph edges between files
+
+Default output location: `.e2e-ai/ast-scan.json`
+
+**Options**:
+
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--output <file>` | Write output to specific file | `.e2e-ai/ast-scan.json` |
+| `--scan-dir <dir>` | Directory to scan | from config (`src`) |
+| `--no-cache` | Disable file-level caching | cache enabled |
+
+### `analyze` - AI QA Map Generation
+
+Analyzes an AST scan result using AI to identify features, workflows, components, and test scenarios.
+
+```bash
+# Analyze the default scan output
+npx e2e-ai analyze
+
+# Analyze a specific scan file
+npx e2e-ai analyze path/to/ast-scan.json
+
+# Only extract features (skip scenario generation)
+npx e2e-ai analyze --skip-scenarios
+
+# Write output to a custom path
+npx e2e-ai analyze --output my-qa-map.json
+```
+
+**What it does** — two-stage AI pipeline:
+
+1. **Stage 2 — Feature Analysis** (`feature-analyzer-agent`): Receives the AST scan and groups routes, components, and hooks into logical features and user-facing workflows. Identifies component roles (form, display, navigation, modal, layout, feedback) and maps API routes to workflow steps.
+
+2. **Stage 3 — Scenario Planning** (`scenario-planner-agent`): Receives the feature map and generates test scenarios for each workflow. Covers happy paths, validation, error handling, edge cases, and permission checks. Assigns priorities (critical/high/medium/low) and links scenarios to specific workflow steps.
+
+**Output**: `QAMapV2Payload` JSON containing:
+- `features` — logical application features with routes and source files
+- `workflows` — user journeys with ordered steps, API calls, conditional branches
+- `components` — UI components with types, props, workflow references
+- `scenarios` — test scenarios with steps, preconditions, expected outcomes, priorities
+
+Default output location: `.e2e-ai/qa-map.json`
+
+**Options**:
+
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--output <file>` | Write QA map to specific file | `.e2e-ai/qa-map.json` |
+| `--skip-scenarios` | Only run feature analysis | both stages |
+
+### `push` - Push QA Map to API
+
+Pushes a generated QA map to a remote API endpoint.
+
+```bash
+# Push the default QA map
+npx e2e-ai push
+
+# Push a specific file
+npx e2e-ai push path/to/qa-map.json
+
+# Associate with a git commit
+npx e2e-ai push --commit-sha abc123
+```
+
+**What it does**: Sends the `QAMapV2Payload` JSON as a POST request to the configured API endpoint with Bearer token authentication. The API responds with version info and auto-linking statistics.
+
+**Requires** `push.apiUrl` and `push.apiKey` in config (or `E2E_AI_API_URL` / `E2E_AI_API_KEY` env vars).
+
+**Options**:
+
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--commit-sha <sha>` | Git commit SHA to associate | - |
+
+---
+
 ## Typical Workflows
 
 ### Workflow 1: Full Recording Pipeline
@@ -254,6 +381,32 @@ npx e2e-ai scenario --key PROJ-101
 npx e2e-ai run --key PROJ-101 --from generate
 ```
 
+### Workflow 5: Codebase Scanner Pipeline
+
+Generate a full QA map from your codebase structure:
+
+```bash
+# Step 1: Scan the codebase AST
+npx e2e-ai scan
+
+# Step 2: AI analysis → features, workflows, scenarios
+npx e2e-ai analyze
+
+# Step 3: Push to remote API (optional)
+npx e2e-ai push
+```
+
+Or run stages selectively:
+
+```bash
+# Just extract features (no scenarios)
+npx e2e-ai analyze --skip-scenarios
+
+# Re-scan after code changes (cached, fast)
+npx e2e-ai scan
+npx e2e-ai analyze
+```
+
 ## Configuration
 
 Run `npx e2e-ai init` to generate `e2e-ai.config.ts`:
@@ -262,11 +415,56 @@ Run `npx e2e-ai init` to generate `e2e-ai.config.ts`:
 import { defineConfig } from 'e2e-ai/config';
 
 export default defineConfig({
+  // --- General ---
   inputSource: 'none',       // 'jira' | 'linear' | 'none'
   outputTarget: 'markdown',  // 'zephyr' | 'markdown' | 'both'
+
+  // --- LLM ---
+  llm: {
+    provider: 'openai',      // 'openai' | 'anthropic'
+    model: null,              // null = use agent defaults (gpt-4o)
+    agentModels: {},          // per-agent overrides: { 'scenario-agent': 'gpt-4o-mini' }
+  },
+
+  // --- Paths ---
+  paths: {
+    tests: 'e2e/tests',
+    scenarios: 'e2e/scenarios',
+    recordings: 'e2e/recordings',
+    transcripts: 'e2e/transcripts',
+    traces: 'e2e/traces',
+    workingDir: '.e2e-ai',
+    qaOutput: 'qa',
+  },
+
+  // --- Scanner ---
+  scanner: {
+    scanDir: 'src',                         // root directory to scan
+    include: ['**/*.ts', '**/*.tsx', '**/*.js', '**/*.jsx'],
+    exclude: [
+      '**/node_modules/**', '**/dist/**', '**/build/**',
+      '**/.next/**', '**/*.test.*', '**/*.spec.*',
+      '**/__tests__/**', '**/*.d.ts',
+    ],
+    cacheDir: '.e2e-ai/scan-cache',         // file-level parse cache
+  },
+
+  // --- Push (remote QA map API) ---
+  push: {
+    apiUrl: null,   // or set E2E_AI_API_URL env var
+    apiKey: null,   // or set E2E_AI_API_KEY env var
+  },
+
+  // --- Recording ---
   voice: { enabled: true },
-  llm: { provider: 'openai' },
-  contextFile: 'e2e-ai.context.md',
+  playwright: {
+    browser: 'chromium',
+    timeout: 120_000,
+    retries: 0,
+    traceMode: 'on',
+  },
+
+  contextFile: '.e2e-ai/context.md',
 });
 ```
 
@@ -298,11 +496,13 @@ AI_PROVIDER=openai              # openai | anthropic
 AI_MODEL=gpt-4o                 # Model override
 ANTHROPIC_API_KEY=sk-ant-...    # Required if AI_PROVIDER=anthropic
 BASE_URL=https://...            # Your application URL
+E2E_AI_API_URL=https://...      # Remote API for push command
+E2E_AI_API_KEY=key-...          # API key for push command
 ```
 
 ## AI Agents
 
-Six specialized agents live in `agents/*.md`. Each has:
+Eight specialized agents live in `agents/*.md`. Each has:
 - **YAML frontmatter**: model, max_tokens, temperature
 - **System prompt**: role + context
 - **Input/Output schemas**: what the agent receives and must produce
@@ -317,8 +517,10 @@ Six specialized agents live in `agents/*.md`. Each has:
 | `refactor-agent` | test + project context | Improved test file | `refine` |
 | `self-healing-agent` | failing test + error output | Diagnosis + patched test | `heal` |
 | `qa-testcase-agent` | test + scenario + issue data | QA markdown + test case JSON | `qa` |
+| `feature-analyzer-agent` | AST scan result | Features, workflows, components JSON | `analyze` |
+| `scenario-planner-agent` | Features + workflows | Complete QA map with scenarios JSON | `analyze` |
 
-You can customize agent behavior by editing the `.md` files directly. The frontmatter `model` field is the default model for that agent (overridable via `--model`).
+You can customize agent behavior by editing the `.md` files directly. The frontmatter `model` field is the default model for that agent (overridable via `--model` or `config.llm.agentModels`).
 
 ## Output Directory Structure
 
@@ -331,7 +533,30 @@ e2e/
 
 qa/                    # QA documentation .md files
 
-.e2e-ai/<KEY>/         # per-issue working dir: codegen, recordings/, intermediate files
+.e2e-ai/
+  <KEY>/               # per-issue working dir: codegen, recordings/, intermediate files
+  ast-scan.json        # scan command output
+  qa-map.json          # analyze command output
+  scan-cache/          # file-level parse cache (gitignored)
+```
+
+## Library API
+
+e2e-ai also exports types and config helpers for programmatic use:
+
+```typescript
+import { defineConfig, loadConfig, getProjectRoot } from 'e2e-ai';
+import type {
+  E2eAiConfig,
+  ResolvedConfig,
+  ASTScanResult,
+  QAMapV2Payload,
+  FeatureV2,
+  WorkflowV2,
+  ScenarioV2,
+  ComponentV2,
+  PushResult,
+} from 'e2e-ai';
 ```
 
 ## License

package/agents/feature-analyzer-agent.md

@@ -0,0 +1,81 @@
+---
+agent: feature-analyzer-agent
+version: "1.0"
+model: gpt-4o
+max_tokens: 8192
+temperature: 0.1
+---
+
+# System Prompt
+
+You are a QA feature analyst. You receive an AST scan result (routes, components, hooks, dependencies) of a web application and identify the logical features, user-facing workflows, and reusable components.
+
+Your job is to transform raw structural data into a high-level QA map that captures what the application does from a user's perspective.
+
+## Input Schema
+
+You receive a JSON object with:
+- `ast`: object - The ASTScanResult from a codebase scan (files, routes, components, hooks, dependencies)
+- `existingMap`: object (optional) - A previous QAMapV2 to update incrementally
+
+## Output Schema
+
+Respond with JSON only (no markdown fences, no extra text):
+
+```json
+{
+  "features": [
+    {
+      "id": "feat:<kebab-case>",
+      "name": "Human-readable feature name",
+      "description": "What this feature does from user perspective",
+      "routes": ["/path1", "/path2"],
+      "workflowIds": ["wf:<kebab>"],
+      "sourceFiles": ["src/path/file.ts"]
+    }
+  ],
+  "workflows": [
+    {
+      "id": "wf:<kebab-case>",
+      "name": "Human-readable workflow name",
+      "featureId": "feat:<parent>",
+      "type": "navigation|crud|multi-step|configuration|search-filter",
+      "preconditions": ["User is authenticated"],
+      "steps": [
+        {
+          "id": "step:<workflow>:<index>",
+          "order": 1,
+          "description": "What the user does",
+          "componentIds": ["comp:<kebab>"],
+          "apiCalls": ["POST /api/endpoint"],
+          "conditionalBranches": []
+        }
+      ],
+      "componentIds": ["comp:<kebab>"]
+    }
+  ],
+  "components": [
+    {
+      "id": "comp:<kebab-case>",
+      "name": "ComponentName",
+      "type": "form|display|navigation|modal|layout|feedback",
+      "sourceFiles": ["src/path/Component.tsx"],
+      "props": ["prop1", "prop2"],
+      "referencedByWorkflows": ["wf:<kebab>"]
+    }
+  ]
+}
+```
+
+## Rules
+
+1. Group routes and components into logical features based on shared URL paths, layouts, and data dependencies
+2. A feature represents a user-facing capability (e.g., "User Management", "Dashboard", "Settings")
+3. A workflow represents a specific user journey within a feature (e.g., "Create new user", "Filter dashboard by date")
+4. Workflow type must be one of: navigation, crud, multi-step, configuration, search-filter
+5. Identify components by their role: form (inputs), display (data rendering), navigation, modal, layout, feedback (toasts/alerts)
+6. Link components to workflows based on which route/page uses them (infer from imports and hook usage)
+7. API routes should be mapped as apiCalls within workflow steps
+8. Dynamic routes (containing `[param]`) indicate CRUD or detail-view workflows
+9. Prefer fewer, well-defined features over many granular ones. Aim for 3-15 features per app.
+10. Output valid JSON only, no markdown code fences or surrounding text

package/agents/scenario-planner-agent.md

@@ -0,0 +1,68 @@
+---
+agent: scenario-planner-agent
+version: "1.0"
+model: gpt-4o
+max_tokens: 8192
+temperature: 0.2
+---
+
+# System Prompt
+
+You are a QA scenario planner. You receive a QA map (features, workflows, components) and generate test scenarios for each workflow. Scenarios cover happy paths, edge cases, validation, error handling, and permission checks.
+
+Your output completes the QA map by adding the scenarios array, producing a full QAMapV2Payload ready for use.
+
+## Input Schema
+
+You receive a JSON object with:
+- `features`: array - Feature definitions from feature-analyzer-agent
+- `workflows`: array - Workflow definitions with steps
+- `components`: array - Component definitions
+
+## Output Schema
+
+Respond with JSON only (no markdown fences, no extra text). Return the complete payload including the original features/workflows/components plus a new `scenarios` array:
+
+```json
+{
+  "features": [...],
+  "workflows": [...],
+  "components": [...],
+  "scenarios": [
+    {
+      "id": "sc:<workflow-id>:<index>",
+      "workflowId": "wf:<kebab>",
+      "featureId": "feat:<kebab>",
+      "name": "Descriptive scenario name",
+      "description": "What this scenario verifies",
+      "category": "happy-path|permission|validation|error|edge-case|precondition",
+      "preconditions": ["User is authenticated", "Data exists"],
+      "steps": [
+        {
+          "order": 1,
+          "action": "What the user does",
+          "expectedResult": "What should happen"
+        }
+      ],
+      "expectedOutcome": "Final expected state",
+      "componentIds": ["comp:<kebab>"],
+      "workflowStepIds": ["step:<workflow>:<index>"],
+      "priority": "critical|high|medium|low"
+    }
+  ]
+}
+```
+
+## Rules
+
+1. Generate at least one happy-path scenario per workflow
+2. For CRUD workflows: test create, read, update, delete + validation failures
+3. For multi-step workflows: test complete flow + abandonment at each step
+4. For forms: test validation (empty fields, invalid input, boundary values)
+5. For workflows with conditionalBranches: generate one scenario per branch
+6. Priority mapping: happy-path critical flows = critical, validation = high, edge-cases = medium, precondition checks = low
+7. Each scenario should have 2-8 steps, each with a verifiable expectedResult
+8. Scenario names should be descriptive: "[Feature]: [what is being tested]"
+9. Link scenarios to workflow steps via workflowStepIds
+10. Aim for 3-8 scenarios per workflow depending on complexity
+11. Output valid JSON only, no markdown code fences or surrounding text

package/dist/cli-6c0wsk32.js

@@ -0,0 +1,165 @@
+import {
+  getPackageRoot,
+  getProjectRoot
+} from "./cli-cqabyzv3.js";
+
+// src/agents/loadAgent.ts
+import { readFileSync, existsSync } from "node:fs";
+import { join } from "node:path";
+function loadAgent(agentName, config) {
+  const localPath = join(getProjectRoot(), ".e2e-ai", "agents", `${agentName}.md`);
+  const packagePath = join(getPackageRoot(), "agents", `${agentName}.md`);
+  const filePath = existsSync(localPath) ? localPath : packagePath;
+  let content;
+  try {
+    content = readFileSync(filePath, "utf-8");
+  } catch {
+    throw new Error(`Agent file not found: ${filePath}`);
+  }
+  const { frontmatter, body } = parseFrontmatter(content);
+  const agentConfig = extractConfig(frontmatter);
+  let systemPrompt = body;
+  if (config) {
+    const contextPath = join(getProjectRoot(), ".e2e-ai", "context.md");
+    if (existsSync(contextPath)) {
+      const projectContext = readFileSync(contextPath, "utf-8").trim();
+      if (projectContext) {
+        systemPrompt = `${body}
+
+## Project Context
+
+${projectContext}`;
+      }
+    }
+    if (config.llm.agentModels[agentName]) {
+      agentConfig.model = config.llm.agentModels[agentName];
+    }
+  }
+  const sections = parseSections(body);
+  return {
+    name: frontmatter.agent ?? agentName,
+    systemPrompt,
+    inputSchema: sections["Input Schema"],
+    outputSchema: sections["Output Schema"],
+    rules: sections["Rules"],
+    example: sections["Example"],
+    config: agentConfig
+  };
+}
+function parseFrontmatter(content) {
+  const match = content.match(/^---\n([\s\S]*?)\n---\n([\s\S]*)$/);
+  if (!match)
+    return { frontmatter: {}, body: content };
+  const frontmatter = {};
+  for (const line of match[1].split(`
+`)) {
+    const colonIdx = line.indexOf(":");
+    if (colonIdx === -1)
+      continue;
+    const key = line.slice(0, colonIdx).trim();
+    let value = line.slice(colonIdx + 1).trim();
+    if (value.startsWith('"') && value.endsWith('"'))
+      value = value.slice(1, -1);
+    if (value === "true")
+      value = true;
+    if (value === "false")
+      value = false;
+    if (!isNaN(Number(value)) && value !== "")
+      value = Number(value);
+    frontmatter[key] = value;
+  }
+  return { frontmatter, body: match[2] };
+}
+function extractConfig(frontmatter) {
+  return {
+    model: frontmatter.model,
+    maxTokens: frontmatter.max_tokens ?? 4096,
+    temperature: frontmatter.temperature ?? 0.2
+  };
+}
+function parseSections(body) {
+  const sections = {};
+  const headingRegex = /^##\s+(.+)$/gm;
+  const headings = [];
+  let match;
+  while ((match = headingRegex.exec(body)) !== null) {
+    headings.push({ title: match[1].trim(), index: match.index });
+  }
+  const systemMatch = body.match(/^#\s+System Prompt\n([\s\S]*?)(?=\n##\s|$)/m);
+  if (systemMatch) {
+    sections["System Prompt"] = systemMatch[1].trim();
+  }
+  for (let i = 0;i < headings.length; i++) {
+    const start = headings[i].index + body.slice(headings[i].index).indexOf(`
+`) + 1;
+    const end = i + 1 < headings.length ? headings[i + 1].index : body.length;
+    sections[headings[i].title] = body.slice(start, end).trim();
+  }
+  return sections;
+}
+
+// src/utils/scan.ts
+import { readdirSync, existsSync as existsSync2, readFileSync as readFileSync2 } from "node:fs";
+import { join as join2, relative } from "node:path";
+async function scanCodebase(root) {
+  const scan = {
+    testFiles: [],
+    configFiles: [],
+    fixtureFiles: [],
+    featureFiles: [],
+    tsconfigPaths: {},
+    playwrightConfig: null,
+    sampleTestContent: null
+  };
+  function walk(dir, depth = 0) {
+    if (depth > 5)
+      return [];
+    const files = [];
+    try {
+      for (const entry of readdirSync(dir, { withFileTypes: true })) {
+        if (entry.name.startsWith(".") || entry.name === "node_modules" || entry.name === "dist")
+          continue;
+        const full = join2(dir, entry.name);
+        if (entry.isDirectory()) {
+          files.push(...walk(full, depth + 1));
+        } else {
+          files.push(full);
+        }
+      }
+    } catch {}
+    return files;
+  }
+  const allFiles = walk(root);
+  for (const file of allFiles) {
+    const rel = relative(root, file);
+    if (rel.endsWith(".test.ts") || rel.endsWith(".spec.ts")) {
+      scan.testFiles.push(rel);
+      if (!scan.sampleTestContent && scan.testFiles.length <= 3) {
+        try {
+          scan.sampleTestContent = readFileSync2(file, "utf-8").slice(0, 3000);
+        } catch {}
+      }
+    }
+    if (rel.endsWith(".feature.ts"))
+      scan.featureFiles.push(rel);
+    if (rel.includes("fixture") && rel.endsWith(".ts"))
+      scan.fixtureFiles.push(rel);
+    if (rel === "playwright.config.ts" || rel === "playwright.config.js")
+      scan.playwrightConfig = rel;
+    if (rel === "tsconfig.json" || rel.endsWith("/tsconfig.json")) {
+      try {
+        const tsconfig = JSON.parse(readFileSync2(file, "utf-8"));
+        if (tsconfig.compilerOptions?.paths) {
+          scan.tsconfigPaths = { ...scan.tsconfigPaths, ...tsconfig.compilerOptions.paths };
+        }
+      } catch {}
+    }
+  }
+  for (const name of ["playwright.config.ts", "vitest.config.ts", "jest.config.ts", "tsconfig.json", "package.json"]) {
+    if (existsSync2(join2(root, name)))
+      scan.configFiles.push(name);
+  }
+  return scan;
+}
+
+export { loadAgent, scanCodebase };