@yasserkhanorg/e2e-agents 1.10.1 → 1.10.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. package/README.md +116 -99
  2. package/dist/agent/config.d.ts +1 -1
  3. package/dist/agent/config.d.ts.map +1 -1
  4. package/dist/agent/config.js +6 -1
  5. package/dist/cache/cached_provider.d.ts +4 -0
  6. package/dist/cache/cached_provider.d.ts.map +1 -1
  7. package/dist/cache/cached_provider.js +3 -0
  8. package/dist/cli/commands/crew.d.ts.map +1 -1
  9. package/dist/cli/commands/crew.js +1 -0
  10. package/dist/cli/commands/init.js +5 -5
  11. package/dist/cli/commands/llm_health.d.ts.map +1 -1
  12. package/dist/cli/commands/llm_health.js +11 -67
  13. package/dist/cli/defaults.d.ts.map +1 -1
  14. package/dist/cli/defaults.js +47 -0
  15. package/dist/cli/parse_args.d.ts.map +1 -1
  16. package/dist/cli/parse_args.js +3 -1
  17. package/dist/cli/types.d.ts +1 -0
  18. package/dist/cli/types.d.ts.map +1 -1
  19. package/dist/cli/usage.d.ts.map +1 -1
  20. package/dist/cli/usage.js +28 -18
  21. package/dist/cli.js +5 -1
  22. package/dist/esm/agent/config.js +6 -1
  23. package/dist/esm/cache/cached_provider.js +3 -0
  24. package/dist/esm/cli/commands/crew.js +1 -0
  25. package/dist/esm/cli/commands/init.js +5 -5
  26. package/dist/esm/cli/commands/llm_health.js +11 -67
  27. package/dist/esm/cli/defaults.js +47 -0
  28. package/dist/esm/cli/parse_args.js +3 -1
  29. package/dist/esm/cli/usage.js +28 -18
  30. package/dist/esm/cli.js +5 -1
  31. package/dist/esm/prompts/coverage.js +2 -2
  32. package/dist/esm/prompts/cross-impact.js +1 -1
  33. package/dist/esm/prompts/generation.js +4 -4
  34. package/dist/esm/prompts/heal.js +4 -4
  35. package/dist/esm/prompts/impact.js +1 -1
  36. package/dist/esm/prompts/test-designer.js +1 -1
  37. package/dist/esm/provider_factory.js +17 -0
  38. package/dist/prompts/coverage.js +2 -2
  39. package/dist/prompts/cross-impact.js +1 -1
  40. package/dist/prompts/generation.js +4 -4
  41. package/dist/prompts/heal.js +4 -4
  42. package/dist/prompts/impact.js +1 -1
  43. package/dist/prompts/test-designer.js +1 -1
  44. package/dist/provider_factory.d.ts.map +1 -1
  45. package/dist/provider_factory.js +17 -0
  46. package/dist/provider_interface.d.ts +7 -0
  47. package/dist/provider_interface.d.ts.map +1 -1
  48. package/package.json +2 -2
package/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # @yasserkhanorg/e2e-agents
2
2
 
3
- AI-powered E2E test impact analysis, generation, healing, and autonomous QA for any project with route families — not just Mattermost.
3
+ Diff-aware E2E impact analysis and coverage gating for Playwright/Cypress teams. Optional AI features can suggest, generate, and heal tests once your project has a route-families.json manifest.
4
4
 
5
5
  [![npm](https://img.shields.io/npm/v/%40yasserkhanorg%2Fe2e-agents)](https://www.npmjs.com/package/@yasserkhanorg/e2e-agents)
6
6
  [![License](https://img.shields.io/badge/license-Apache%202.0-blue)](LICENSE)
@@ -8,30 +8,47 @@ AI-powered E2E test impact analysis, generation, healing, and autonomous QA for
8
8
 
9
9
  ## What It Does
10
10
 
11
- Given a git diff, `e2e-ai-agents` determines which E2E test flows are impacted, identifies coverage gaps, and can generate or heal tests for Playwright, Cypress, pytest (Python), or supertest/vitest (Node.js API) — all from the CLI. The tool is project-agnostic: any codebase with a `route-families.json` manifest works out of the box. The companion `e2e-qa-agent` goes further: it opens a real browser, explores your app autonomously, and produces a QA report with findings and a release-readiness verdict.
11
+ `e2e-agents` is built first for one painful CI job: given a git diff, tell us which E2E surface changed and whether the current suite already covers it.
12
12
 
13
- **Pipeline:** `impact` `plan` `generate` `heal` → `finalize`
14
- **Multi-Agent Crew:** `impact` + `cross-impact` + `regression-advisor` `strategist` `test-designer` `generator` `executor` `healer`
13
+ - **Primary**: diff-aware E2E impact analysis and coverage gating
14
+ - **Secondary**: optional AI features can suggest, generate, and heal tests once your project has a route-families.json manifest
15
+ - **Tertiary**: crew workflows, MCP integrations, plugins, and the autonomous QA agent
15
16
 
16
- > **How does this compare to other tools?** See [docs/comparison.md](docs/comparison.md) for a detailed analysis against Launchable, Codecov ATS, Qodo, Testsigma, mabl, GitHub Copilot, and others.
17
+ The clearest path today is a Playwright or Cypress repository with a maintained `route-families.json` manifest. That is the path this package is optimized to make unusually clear and useful.
18
+
19
+ ## Product Shape
20
+
21
+ | Level | Commands | What They Are For |
22
+ |------|----------|-------------------|
23
+ | Core CI Workflow | `impact`, `plan`, `gate` | Decide what changed, what is covered, and whether a PR should pass |
24
+ | Optional AI Workflow | `generate`, `heal`, `analyze`, `finalize-generated-tests` | Suggest, create, or repair tests after impact analysis |
25
+ | Setup and Calibration | `train`, `bootstrap`, `traceability-*`, `feedback`, `cost-report`, `llm-health` | Build the manifest, feed execution data back in, and inspect cost/provider health |
26
+ | Advanced / Experimental | `crew`, MCP mode, plugins, `e2e-qa-agent` | Deeper orchestration and browser-driven workflows beyond the core CI loop |
27
+
28
+ ## Known Limitations
29
+
30
+ - The clearest, most stable workflow is still **Playwright/Cypress impact analysis and gating**.
31
+ - AI generation and healing work best **after** the project has a good `route-families.json` manifest.
32
+ - Advanced features are improving, but they are **not** the best entry point if you only want dependable CI coverage decisions.
33
+ - The Mattermost profile is still the most opinionated path in the codebase. Generic flows are supported, but the package should be judged first on the core CI workflow above.
17
34
 
18
35
  ## Free Tier
19
36
 
20
- These commands work with **zero LLM cost** no API key required:
37
+ These commands work with **zero LLM cost** and do not require an API key:
21
38
 
22
39
  | Command | What It Does |
23
40
  |---------|-------------|
24
- | `impact` | Deterministic impact analysis from git diff |
25
- | `plan` | Coverage gap detection and test recommendations |
26
- | `train --no-enrich` | Build route-families manifest (scanner only) |
27
- | `bootstrap` | Generate route-families.json from a knowledge graph (deterministic) |
28
- | `gate` | CI coverage gate exit 1 if coverage is below threshold |
41
+ | `impact` | Deterministic impact analysis from a git diff |
42
+ | `plan` | Coverage-gap detection and recommended run set |
43
+ | `gate` | CI coverage gate that exits non-zero below a threshold |
44
+ | `train --no-enrich` | Build `route-families.json` with the scanner only |
45
+ | `bootstrap` | Generate `route-families.json` from a knowledge graph |
29
46
  | `traceability-capture` | Extract test-file relationships from Playwright JSON |
30
47
  | `traceability-ingest` | Merge traceability mappings into rolling manifest |
31
48
  | `feedback` | Ingest recommendation outcomes for calibration |
32
49
  | `cost-report` | View LLM cost breakdown from past runs |
33
50
 
34
- AI-powered features (crew workflows, test generation, healing) require an API key from [Anthropic](https://console.anthropic.com/), [OpenAI](https://platform.openai.com/), or a local [Ollama](https://ollama.ai/) instance (free).
51
+ Optional AI features use [Anthropic](https://console.anthropic.com/), [OpenAI](https://platform.openai.com/), or a local [Ollama](https://ollama.ai/) instance.
35
52
 
36
53
  ## Installation
37
54
 
@@ -41,103 +58,111 @@ npm install @yasserkhanorg/e2e-agents
41
58
 
42
59
  Requires Node.js >= 20. Ships both CommonJS and ESM builds.
43
60
 
44
- ## CLI Commands
61
+ Verify the CLI:
45
62
 
46
63
  ```bash
47
- # All-in-one: impact + plan + optional generate/heal
48
- npx e2e-ai-agents analyze --path /path/to/project [--generate] [--heal]
64
+ npx e2e-ai-agents --help
65
+ ```
49
66
 
50
- # Analyze which flows are impacted by code changes
51
- npx e2e-ai-agents impact --path /path/to/project
67
+ ## Core CI Commands
52
68
 
53
- # Generate a coverage plan with gap analysis
54
- npx e2e-ai-agents plan --path /path/to/project
69
+ Use these first. They are the heart of the package.
55
70
 
56
- # Generate tests for uncovered gaps (requires plan output)
57
- npx e2e-ai-agents generate --path /path/to/project
71
+ ```bash
72
+ # 1. See what changed
73
+ npx e2e-ai-agents impact --path /path/to/project --since origin/main
58
74
 
59
- # Bootstrap route-families.json from an Understand-Anything knowledge graph
60
- npx e2e-ai-agents bootstrap --path <project-root> [--kg-path <path>] [--test-mode ui|api|both] [--max-families <n>] [--dry-run]
75
+ # 2. Build a coverage plan and CI summary artifacts
76
+ npx e2e-ai-agents plan --path /path/to/project --since origin/main
61
77
 
62
- # CI coverage gate fails with exit code 1 if coverage is below threshold
63
- npx e2e-ai-agents gate --path <project-root> [--threshold <0-100>]
78
+ # 3. Fail the job if coverage is below a threshold
79
+ npx e2e-ai-agents gate --path /path/to/project --threshold 80
80
+ ```
64
81
 
65
- # Heal flaky/failing specs from a Playwright report
66
- npx e2e-ai-agents heal --path /path/to/project --traceability-report ./playwright-report.json
82
+ Notes:
67
83
 
68
- # Stage generated tests, commit, and open a PR
69
- npx e2e-ai-agents finalize-generated-tests --path /path/to/project --create-pr
84
+ - `impact` prints a deterministic summary to stdout.
85
+ - `plan` writes `.e2e-ai-agents/plan.json` and `.e2e-ai-agents/ci-summary.md`.
86
+ - `gate` expects a threshold in the range `0-100` and exits `1` when the threshold is missed.
70
87
 
71
- # Ingest test execution data for traceability
88
+ ## Setup and Calibration
89
+
90
+ These commands help the core CI workflow become accurate and project-aware.
91
+
92
+ ```bash
93
+ # Build the manifest from the repo structure
94
+ npx e2e-ai-agents train --path /path/to/project --no-enrich
95
+
96
+ # Or bootstrap it from an Understand-Anything knowledge graph
97
+ npx e2e-ai-agents bootstrap --path /path/to/project [--kg-path ./knowledge-graph.json]
98
+
99
+ # Feed execution data back into the manifest
72
100
  npx e2e-ai-agents traceability-capture --path /path/to/project --traceability-report ./playwright-report.json
73
101
  npx e2e-ai-agents traceability-ingest --path /path/to/project --traceability-input ./traceability-input.json
74
102
 
75
- # Ingest recommendation feedback for calibration
103
+ # Calibration and diagnostics
76
104
  npx e2e-ai-agents feedback --path /path/to/project --feedback-input ./feedback.json
77
-
78
- # Test LLM provider connectivity
105
+ npx e2e-ai-agents cost-report --path /path/to/project
79
106
  npx e2e-ai-agents llm-health
80
107
  ```
81
108
 
82
- `plan` and `suggest` are aliases. `analyze` is a convenience wrapper that runs impact + plan and optionally generation/healing in one invocation. Use `--help` for all available flags.
109
+ ## Optional AI Workflow
83
110
 
84
- ## Multi-Agent Crew
85
-
86
- The Crew orchestrates 10 specialized agents for deep test analysis. While the standard pipeline gives a fast pass/fail gate, the Crew produces structured test designs, cross-family impact maps, and prioritized test strategies.
111
+ Once impact analysis is useful and the manifest is in place, you can layer on AI assistance.
87
112
 
88
113
  ```bash
89
- # Quick strategy: impact + strategy recommendations (~$0.10, ~1 min)
90
- npx e2e-ai-agents crew --workflow quick-check --path /path/to/project --tests-root ./e2e-tests --since origin/master
114
+ # All-in-one wrapper: impact + coverage + optional generation/healing
115
+ npx e2e-ai-agents analyze --path /path/to/project [--generate] [--heal]
91
116
 
92
- # Full test design without generation (~$0.50-2.00, ~5-40 min)
93
- npx e2e-ai-agents crew --workflow design-only --path /path/to/project --tests-root ./e2e-tests --since origin/master
117
+ # Generate tests for uncovered gaps
118
+ npx e2e-ai-agents generate --path /path/to/project
94
119
 
95
- # End-to-end: design + generate + execute + heal (~$2-5, ~10-60 min)
96
- npx e2e-ai-agents crew --workflow full-qa --path /path/to/project --tests-root ./e2e-tests --since origin/master
120
+ # Heal flaky or failing specs from a Playwright report
121
+ npx e2e-ai-agents heal --path /path/to/project --traceability-report ./playwright-report.json
97
122
 
98
- # With budget cap and JSON output
99
- npx e2e-ai-agents crew --workflow design-only --budget-usd 2.00 --json --path /path/to/project --tests-root ./e2e-tests --since origin/master
123
+ # Stage generated tests, commit, and optionally open a PR
124
+ npx e2e-ai-agents finalize-generated-tests --path /path/to/project --create-pr
125
+ ```
100
126
 
101
- # Dry run: preview what would happen without LLM calls
102
- npx e2e-ai-agents crew --workflow design-only --dry-run --path /path/to/project --tests-root ./e2e-tests --since origin/master
127
+ `plan` and `suggest` are aliases. `analyze` is the convenience wrapper when you want the full path in one invocation.
103
128
 
104
- # View LLM cost breakdown
105
- npx e2e-ai-agents cost-report --path /path/to/project
106
- ```
129
+ ## Advanced / Experimental
107
130
 
108
- ### Budget Enforcement
131
+ These features are real, but they are not the clearest place to start if your goal is simple CI coverage decisions.
109
132
 
110
- The `--budget-usd` flag sets a hard cost limit for the entire crew run. Budget enforcement uses a **pre-reservation** model (like credit card authorization holds) to prevent parallel agents from overshooting:
133
+ ### Multi-Agent Crew
111
134
 
112
- 1. Before each LLM call, the provider **reserves** estimated cost in a shared ledger
113
- 2. Other parallel agents see the in-flight hold and stop if the budget would be exceeded
114
- 3. After the call completes, the reservation is **settled** to actual cost
135
+ The Crew orchestrates deeper multi-agent workflows on top of the same impact-analysis foundation. Use it when you want richer strategy output, structured test design, or end-to-end generation pipelines.
115
136
 
116
- This means 3 agents running in parallel against a $1.00 budget will not collectively spend $1.20. The overshoot is bounded by the estimation error of a single call (~$0.01).
137
+ ```bash
138
+ # Quick strategy recommendations
139
+ npx e2e-ai-agents crew --workflow quick-check --path /path/to/project --tests-root ./e2e-tests --since origin/master
117
140
 
118
- ### Resilience
141
+ # Full design-only workflow
142
+ npx e2e-ai-agents crew --workflow design-only --path /path/to/project --tests-root ./e2e-tests --since origin/master
119
143
 
120
- Provider calls are protected by a **circuit breaker** (3-failure threshold, 60s cooldown). If a provider goes down, calls fail fast instead of burning through retry timeouts. Circuit breakers are shared per provider type — if Anthropic is down, all agents discover it after 3 total failures.
144
+ # End-to-end workflow
145
+ npx e2e-ai-agents crew --workflow full-qa --path /path/to/project --tests-root ./e2e-tests --since origin/master
146
+ ```
121
147
 
122
- Only transient errors (429, 5xx, network) trip the circuit. Budget exceeded and auth errors do not.
148
+ Built-in safeguards include budget enforcement, provider circuit breaking, and structured output for downstream tooling.
123
149
 
124
150
  ### Plugins
125
151
 
126
152
  External agents can register into crew workflows via the `plugins` config:
127
153
 
128
154
  ```typescript
129
- // my-plugin.ts
130
- import type { AgentPlugin, AgentTask, AgentResult, CrewContext } from '@yasserkhanorg/e2e-agents';
155
+ import type {AgentPlugin, AgentTask, AgentResult, CrewContext} from '@yasserkhanorg/e2e-agents';
131
156
 
132
157
  const myPlugin: AgentPlugin = {
133
158
  role: 'my-custom-analyzer',
134
- phase: 'understand', // Run in the 'understand' phase
135
- runAfter: ['impact-analyst'], // After impact-analyst completes
159
+ phase: 'understand',
160
+ runAfter: ['impact-analyst'],
136
161
  async execute(task: AgentTask, ctx: CrewContext): Promise<AgentResult> {
137
- // Access ctx.impactedFlows, ctx.changedFiles, etc.
138
- return { role: 'my-custom-analyzer', status: 'success', output: null, warnings: [] };
162
+ return {role: 'my-custom-analyzer', status: 'success', output: null, warnings: []};
139
163
  },
140
164
  };
165
+
141
166
  export default myPlugin;
142
167
  ```
143
168
 
@@ -145,24 +170,19 @@ export default myPlugin;
145
170
  npx e2e-ai-agents crew --plugins ./my-plugin.ts --workflow full-qa --path ./app
146
171
  ```
147
172
 
148
- Plugins with `runAfter` dependencies run sequentially after their dependencies. Plugins without `runAfter` run in parallel with other agents in their phase. Plugin paths must be relative and cannot escape the workspace directory.
149
-
150
- See [docs/PLUGIN_API_STABILITY.md](docs/PLUGIN_API_STABILITY.md) for the full API contract and stability guarantees.
151
-
152
- ### What the Crew Adds Beyond the Pipeline
153
-
154
- | Capability | Pipeline | Crew |
155
- |-----------|---------|------|
156
- | Impact detection | Per-family, isolated | Same + cross-family ripple detection |
157
- | Test scenarios | Flat `scenariosToAdd` strings | Structured `TestCase[]` with type, preconditions, steps, expected outcome, rationale |
158
- | Test categories | None | 9: happy-path, edge-case, boundary, negative, state-transition, race-condition, permission, accessibility, performance |
159
- | Strategy | None | Per-flow approach (full-test / smoke-test / skip) with priority and rationale |
160
- | Regression awareness | None | Risk scoring from flaky history, calibration data, and file-pattern heuristics |
173
+ See [docs/PLUGIN_API_STABILITY.md](docs/PLUGIN_API_STABILITY.md) for the API contract and stability guarantees.
161
174
 
162
175
  ### Programmatic API
163
176
 
164
177
  ```typescript
165
- import { CrewOrchestrator, ImpactAnalystAgent, StrategistAgent, TestDesignerAgent, CrossImpactAgent, RegressionAdvisorAgent } from '@yasserkhanorg/e2e-agents';
178
+ import {
179
+ CrewOrchestrator,
180
+ ImpactAnalystAgent,
181
+ CrossImpactAgent,
182
+ RegressionAdvisorAgent,
183
+ StrategistAgent,
184
+ TestDesignerAgent,
185
+ } from '@yasserkhanorg/e2e-agents';
166
186
 
167
187
  const orchestrator = new CrewOrchestrator();
168
188
  orchestrator.registerAgent(new ImpactAnalystAgent());
@@ -178,11 +198,10 @@ const result = await orchestrator.run({
178
198
  workflow: 'design-only',
179
199
  });
180
200
 
181
- console.log(result.context.testDesigns); // Structured test cases
182
- console.log(result.context.crossImpacts); // Cross-family links
183
- console.log(result.context.strategyEntries); // Prioritized strategy
201
+ console.log(result.context.strategyEntries);
202
+ console.log(result.context.testDesigns);
203
+ console.log(result.context.crossImpacts);
184
204
  ```
185
-
186
205
  ## Route-Families Training
187
206
 
188
207
  ### What it produces
@@ -305,26 +324,24 @@ Create `e2e-ai-agents.config.json` in your project (auto-discovered):
305
324
  }
306
325
  ```
307
326
 
308
- ### Generation Profiles
327
+ ### Analysis Profiles
309
328
 
310
- The tool auto-detects your project type and generates tests following the appropriate conventions. Use the `--profile` flag (or the `profile` config key) to select a profile explicitly:
329
+ Profiles are not the same thing as frameworks. They control analysis strictness and project-specific conventions.
311
330
 
312
331
  | Profile | Description |
313
332
  |---------|-------------|
314
- | `mattermost` | Mattermost-specific conventions (strict mode, escalation for heuristic-only mappings) |
315
- | `generic` | Generic Playwright project |
316
- | `pytest` | Python projects using pytest + requests/httpx |
317
- | `supertest` | Node.js API projects using supertest/vitest |
333
+ | `default` | Standard analysis behavior for most repositories |
334
+ | `mattermost` | Mattermost-specific conventions and stricter handling of heuristic-only mappings |
318
335
 
319
- When `--profile` is omitted, the tool inspects `package.json`, `pyproject.toml`, and test directory structure to pick the best match automatically.
336
+ Framework detection is separate. The CLI can auto-detect Playwright, Cypress, pytest, supertest, and Selenium usage from the project structure and dependencies.
320
337
 
321
338
  ### Key options
322
339
 
323
340
  - **`testsRoot`** — path to tests when they live outside the app root
324
- - **`profile`** — `mattermost`, `generic`, `pytest`, or `supertest` (auto-detected when omitted)
341
+ - **`profile`** — `default` or `mattermost`
325
342
  - **`impact.dependencyGraph`** — static reverse dependency graph for transitive impact
326
343
  - **`impact.traceability`** — file-to-test mapping from CI execution data
327
- - **`impact.aiFlow`** — LLM-powered flow mapping (requires `ANTHROPIC_API_KEY`)
344
+ - **`impact.aiFlow`** — LLM-powered flow mapping through the configured provider
328
345
  - **`pipeline.mcp`** — use Playwright MCP server for browser-aware generation/healing
329
346
  - **`policy.enforcementMode`** — `advisory`, `warn`, or `block`
330
347
 
@@ -334,8 +351,6 @@ When `--profile` is omitted, the tool inspects `package.json`, `pyproject.toml`,
334
351
 
335
352
  ```yaml
336
353
  - name: Run E2E coverage check
337
- env:
338
- ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
339
354
  run: |
340
355
  npx e2e-ai-agents plan \
341
356
  --config ./e2e-ai-agents.config.json \
@@ -351,13 +366,15 @@ The `plan` command writes:
351
366
 
352
367
  Use `--fail-on-must-add-tests` to exit non-zero when uncovered P0/P1 gaps exist. Use `--github-output` to expose outputs to subsequent workflow steps.
353
368
 
369
+ If you want AI enrichment on top of the deterministic plan, add your provider environment variables to the workflow separately.
370
+
354
371
  See [examples/github-actions/pr-impact.yml](examples/github-actions/pr-impact.yml) for a complete workflow template.
355
372
 
356
373
  ## Pipeline Modes
357
374
 
358
375
  ### Package Native (default)
359
376
 
360
- Strategy-based test templates for Playwright, Cypress, pytest, or supertest/vitest with quality guardrails and iterative heal attempts.
377
+ Strategy-based test templates with quality guardrails and iterative heal attempts. The strongest path today is still a repo whose impact analysis and manifest quality are already in good shape.
361
378
 
362
379
  ### MCP Mode (`--pipeline-mcp`)
363
380
 
@@ -377,7 +394,7 @@ LLM-powered generate-run-fix loop: generates a spec, runs it, analyzes failures,
377
394
  Used internally for AI enrichment, test generation, and healing.
378
395
 
379
396
  ```bash
380
- # Anthropic (default)
397
+ # Anthropic
381
398
  export ANTHROPIC_API_KEY=sk-ant-...
382
399
 
383
400
  # OpenAI
@@ -401,7 +418,7 @@ const response = await claude.generateText('Analyze test failure');
401
418
 
402
419
  Factory pattern with auto-detection, hybrid mode (free local + premium fallback), and custom OpenAI-compatible endpoints are also supported. See the [provider API exports](src/index.ts) for full details.
403
420
 
404
- ## MCP Server
421
+ ## Advanced / Experimental: MCP Server
405
422
 
406
423
  Exposes 6 tools for test agents (Playwright v1.56+):
407
424
 
@@ -445,7 +462,7 @@ Schemas: [schemas/traceability-input.schema.json](schemas/traceability-input.sch
445
462
 
446
463
  All written under `<testsRoot>/.e2e-ai-agents/`.
447
464
 
448
- ## Autonomous QA Agent (`e2e-qa-agent`)
465
+ ## Advanced / Experimental: Autonomous QA Agent (`e2e-qa-agent`)
449
466
 
450
467
  An autonomous QA engineer that opens a real browser, navigates to changed features, tries edge cases, and produces a findings report — all unsupervised. Built on top of `agent-browser` and the Anthropic tool-use API.
451
468
 
@@ -487,7 +504,7 @@ Requires `agent-browser` CLI (`npm install -g agent-browser`) and `ANTHROPIC_API
487
504
 
488
505
  ## Production Usage
489
506
 
490
- The tool works with any project that has a `route-families.json` manifest frontend, backend, or full-stack. Used in production by [Mattermost](https://github.com/mattermost/mattermost) for CI-integrated E2E coverage gating, test generation, and spec healing. See the [Mattermost Playwright integration](https://github.com/mattermost/mattermost/tree/master/e2e-tests/playwright) for a real-world example.
507
+ The strongest production story today is a repo that maintains a good `route-families.json` manifest and uses the deterministic `impact -> plan -> gate` loop in CI. [Mattermost](https://github.com/mattermost/mattermost) is the most battle-tested public example for coverage gating, generation, and healing. See the [Mattermost Playwright integration](https://github.com/mattermost/mattermost/tree/master/e2e-tests/playwright) for a real-world setup.
491
508
 
492
509
  ## License
493
510
 
@@ -1,5 +1,5 @@
1
1
  export type AnalysisMode = 'impact' | 'gap';
2
- export type FrameworkType = 'auto' | 'playwright' | 'cypress' | 'selenium' | 'unknown';
2
+ export type FrameworkType = 'auto' | 'playwright' | 'cypress' | 'pytest' | 'supertest' | 'selenium' | 'unknown';
3
3
  export type ArtifactMode = 'commit' | 'keep-local' | 'none';
4
4
  export type AnalysisProfile = 'default' | 'mattermost' | string;
5
5
  export interface BudgetConfig {
@@ -1 +1 @@
1
- {"version":3,"file":"config.d.ts","sourceRoot":"","sources":["../../src/agent/config.ts"],"names":[],"mappings":"AAMA,MAAM,MAAM,YAAY,GAAG,QAAQ,GAAG,KAAK,CAAC;AAC5C,MAAM,MAAM,aAAa,GAAG,MAAM,GAAG,YAAY,GAAG,SAAS,GAAG,UAAU,GAAG,SAAS,CAAC;AACvF,MAAM,MAAM,YAAY,GAAG,QAAQ,GAAG,YAAY,GAAG,MAAM,CAAC;AAC5D,MAAM,MAAM,eAAe,GAAG,SAAS,GAAG,YAAY,GAAG,MAAM,CAAC;AAEhE,MAAM,WAAW,YAAY;IACzB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,SAAS,CAAC,EAAE,MAAM,CAAC;CACtB;AAED,MAAM,WAAW,cAAc;IAC3B,IAAI,EAAE,YAAY,CAAC;IACnB,QAAQ,EAAE,MAAM,CAAC;CACpB;AAED,MAAM,WAAW,cAAc;IAC3B,YAAY,EAAE,OAAO,CAAC;CACzB;AAED,MAAM,WAAW,mBAAmB;IAChC,QAAQ,CAAC,EAAE,MAAM,EAAE,CAAC;CACvB;AAED,MAAM,WAAW,mBAAmB;IAChC,QAAQ,CAAC,EAAE,MAAM,EAAE,CAAC;IACpB,OAAO,CAAC,EAAE,MAAM,EAAE,CAAC;CACtB;AAED,MAAM,WAAW,oBAAoB;IACjC,cAAc,EAAE;QACZ,EAAE,EAAE,MAAM,CAAC;QACX,EAAE,EAAE,MAAM,CAAC;QACX,EAAE,EAAE,MAAM,CAAC;KACd,CAAC;IACF,eAAe,EAAE,MAAM,CAAC;CAC3B;AAED,MAAM,MAAM,SAAS,GAAG,IAAI,GAAG,KAAK,GAAG,SAAS,CAAC;AAEjD,MAAM,MAAM,YAAY,GAClB,cAAc,GACd,YAAY,GACZ,eAAe,GACf,QAAQ,GACR,OAAO,GACP,aAAa,CAAC;AAEpB,MAAM,WAAW,UAAU;IACvB,YAAY,EAAE,SAAS,CAAC;CAC3B;AAED,MAAM,WAAW,cAAc;IAC3B,YAAY,EAAE,YAAY,EAAE,CAAC;CAChC;AAED,MAAM,WAAW,iBAAiB;IAC9B,WAAW,EAAE,MAAM,CAAC;IACpB,UAAU,EAAE,MAAM,CAAC;IACnB,gBAAgB,EAAE,MAAM,CAAC;IACzB,cAAc,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,WAAW,cAAc;IAC3B,OAAO,EAAE,OAAO,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;IAClB,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,EAAE,OAAO,CAAC;IACd,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,OAAO,CAAC,EAAE,QAAQ,GAAG,UAAU,GAAG,SAAS,GAAG,QAAQ,CAAC;IACvD,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,GAAG,CAAC,EAAE,OAAO,CAAC;IACd,gBAAgB,CAAC,EAAE,OAAO,CAAC;IAC3B,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,mBAAmB,CAAC,EAAE,MAAM,CAAC;IAC7B,UAAU,CAAC,EAAE,MAAM,CAAC;CACvB;AAED,MAAM,WAAW,SAAS;IACtB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,QAAQ,CAAC,EAAE,MAAM,CAAC;CACrB;AAED,MAAM,WAAW,UAAU;IACvB,WAAW,EAAE,MAAM,CAAC;IACpB,WAAW,EAAE,MAAM,CAAC;IACpB,gBAAgB,EAAE,MAAM,EAAE,CAAC;CAC9B;AAED,MAAM,WAAW,YAAY;IACzB,wBAAwB,EAAE,MAAM,CAAC;IACjC,sBAAsB,EAAE,MAAM,CAAC;IAC/B,4BAA4B,EAAE,MAAM,CAAC;IACrC,qBAAqB,EAAE,OAAO,CAAC;IAC/B,qBAAqB,EAAE,OAAO,CAAC;IAC/B,iBAAiB,EAAE,MAAM,EAAE,CAAC;IAC5B,eAAe,EAAE,UAAU,GAAG,MAAM,GAAG,OAAO,CAAC;IAC/C,cAAc,EAAE,KAAK,CAAC,SAAS,GAAG,gBAAgB,GAAG,eAAe,CAAC,CAAC;CACzE;AAED,MAAM,WAAW,2BAA2B;IACxC,OAAO,EAAE,OAAO,CAAC;IACjB,QAAQ,EAAE,MAAM,CAAC;IACjB,gBAAgB,EAAE,MAAM,CAAC;IACzB,YAAY,EAAE,MAAM,EAAE,CAAC;IACvB,eAAe,EAAE,MAAM,EAAE,CAAC;IAC1B,UAAU,EAAE,MAAM,EAAE,CAAC;IACrB,WAAW,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,EAAE,CAAC,CAAC;CACzC;AAED,MAAM,WAAW,wBAAwB;IACrC,OAAO,EAAE,OAAO,CAAC;IACjB,YAAY,EAAE,MAAM,CAAC;IACrB,iBAAiB,EAAE,MAAM,CAAC;CAC7B;AAED,MAAM,WAAW,yBAAyB;IACtC,OAAO,EAAE,OAAO,CAAC;IACjB,OAAO,EAAE,MAAM,CAAC;IAChB,eAAe,EAAE,MAAM,CAAC;CAC3B;AAED,MAAM,MAAM,iBAAiB,GAAG,WAAW,GAAG,QAAQ,GAAG,QAAQ,GAAG,MAAM,CAAC;AAE3E,MAAM,WAAW,qBAAqB;IAClC,OAAO,EAAE,OAAO,CAAC;IACjB,QAAQ,EAAE,iBAAiB,CAAC;IAC5B,YAAY,EAAE,MAAM,EAAE,CAAC;IACvB,kBAAkB,EAAE,MAAM,CAAC;IAC3B,iBAAiB,EAAE,MAAM,CAAC;IAC1B,SAAS,EAAE,MAAM,CAAC;IAClB,WAAW,EAAE,MAAM,CAAC;CACvB;AAED,MAAM,WAAW,kBAAkB;IAC/B,OAAO,EAAE,OAAO,CAAC;IACjB,MAAM,EAAE,OAAO,CAAC;IAChB,QAAQ,EAAE,iBAAiB,CAAC;IAC5B,YAAY,EAAE,MAAM,EAAE,CAAC;IACvB,kBAAkB,EAAE,MAAM,CAAC;IAC3B,kBAAkB,EAAE,MAAM,CAAC;IAC3B,SAAS,EAAE,MAAM,CAAC;IAClB,WAAW,EAAE,MAAM,CAAC;CACvB;AAED,MAAM,WAAW,SAAS;IACtB,KAAK,EAAE,MAAM,CAAC;IACd,kBAAkB,CAAC,EAAE,OAAO,CAAC;CAChC;AAED,MAAM,WAAW,mBAAmB;IAChC,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,MAAM,CAAC,EAAE,OAAO,CAAC;CACpB;AAED,MAAM,WAAW,gBAAgB;IAC7B,OAAO,EAAE,OAAO,CAAC;IACjB,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,SAAS,CAAC,EAAE,MAAM,CAAC;CACtB;AAED,MAAM,WAAW,WAAW;IACxB,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,eAAe,CAAC;IACzB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,IAAI,EAAE,YAAY,CAAC;IACnB,SAAS,EAAE,aAAa,CAAC;IACzB,gBAAgB,EAAE,MAAM,CAAC;IACzB,MAAM,EAAE,YAAY,CAAC;IACrB,SAAS,EAAE,cAAc,CAAC;IAC1B,SAAS,EAAE,cAAc,CAAC;IAC1B,aAAa,EAAE,mBAAmB,CAAC;IACnC,aAAa,EAAE,mBAAmB,CAAC;IACnC,cAAc,EAAE,oBAAoB,CAAC;IACrC,MAAM,EAAE;QACJ,aAAa,EAAE,OAAO,CAAC;QACvB,eAAe,EAAE,2BAA2B,CAAC;QAC7C,YAAY,EAAE,wBAAwB,CAAC;QACvC,aAAa,EAAE,yBAAyB,CAAC;QACzC,MAAM,EAAE,kBAAkB,CAAC;QAC3B,SAAS,EAAE,qBAAqB,CAAC;KACpC,CAAC;IACF,QAAQ,EAAE,cAAc,CAAC;IACzB,GAAG,EAAE,SAAS,CAAC;IACf,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,IAAI,EAAE,UAAU,CAAC;IACjB,MAAM,EAAE,YAAY,CAAC;IACrB,KAAK,EAAE,UAAU,CAAC;IAClB,QAAQ,EAAE,cAAc,CAAC;IACzB,WAAW,EAAE,iBAAiB,CAAC;IAC/B,GAAG,EAAE,SAAS,CAAC;IACf,aAAa,EAAE,mBAAmB,CAAC;IACnC,UAAU,EAAE,gBAAgB,CAAC;CAChC;AAED,MAAM,WAAW,cAAc;IAC3B,MAAM,EAAE,WAAW,CAAC;IACpB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,OAAO,EAAE,MAAM,CAAC;CACnB;AAmND,MAAM,WAAW,eAAe;IAC5B,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,OAAO,CAAC,EAAE,eAAe,CAAC;IAC1B,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,IAAI,CAAC,EAAE,YAAY,CAAC;IACpB,SAAS,CAAC,EAAE,aAAa,CAAC;IAC1B,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,MAAM,CAAC,EAAE,YAAY,CAAC;IACtB,YAAY,CAAC,EAAE,MAAM,EAAE,CAAC;IACxB,YAAY,CAAC,EAAE,MAAM,EAAE,CAAC;IACxB,WAAW,CAAC,EAAE,MAAM,EAAE,CAAC;IACvB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,QAAQ,CAAC,EAAE,OAAO,CAAC,cAAc,CAAC,CAAC;IACnC,MAAM,CAAC,EAAE,OAAO,CAAC,YAAY,CAAC,CAAC;IAC/B,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,WAAW,CAAC,EAAE,MAAM,CAAC;CACxB;AAuiBD,wBAAgB,aAAa,CAAC,GAAG,EAAE,MAAM,EAAE,UAAU,CAAC,EAAE,MAAM,EAAE,SAAS,CAAC,EAAE,eAAe,GAAG,cAAc,CA2L3G"}
1
+ {"version":3,"file":"config.d.ts","sourceRoot":"","sources":["../../src/agent/config.ts"],"names":[],"mappings":"AAMA,MAAM,MAAM,YAAY,GAAG,QAAQ,GAAG,KAAK,CAAC;AAC5C,MAAM,MAAM,aAAa,GAAG,MAAM,GAAG,YAAY,GAAG,SAAS,GAAG,QAAQ,GAAG,WAAW,GAAG,UAAU,GAAG,SAAS,CAAC;AAChH,MAAM,MAAM,YAAY,GAAG,QAAQ,GAAG,YAAY,GAAG,MAAM,CAAC;AAC5D,MAAM,MAAM,eAAe,GAAG,SAAS,GAAG,YAAY,GAAG,MAAM,CAAC;AAEhE,MAAM,WAAW,YAAY;IACzB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,SAAS,CAAC,EAAE,MAAM,CAAC;CACtB;AAED,MAAM,WAAW,cAAc;IAC3B,IAAI,EAAE,YAAY,CAAC;IACnB,QAAQ,EAAE,MAAM,CAAC;CACpB;AAED,MAAM,WAAW,cAAc;IAC3B,YAAY,EAAE,OAAO,CAAC;CACzB;AAED,MAAM,WAAW,mBAAmB;IAChC,QAAQ,CAAC,EAAE,MAAM,EAAE,CAAC;CACvB;AAED,MAAM,WAAW,mBAAmB;IAChC,QAAQ,CAAC,EAAE,MAAM,EAAE,CAAC;IACpB,OAAO,CAAC,EAAE,MAAM,EAAE,CAAC;CACtB;AAED,MAAM,WAAW,oBAAoB;IACjC,cAAc,EAAE;QACZ,EAAE,EAAE,MAAM,CAAC;QACX,EAAE,EAAE,MAAM,CAAC;QACX,EAAE,EAAE,MAAM,CAAC;KACd,CAAC;IACF,eAAe,EAAE,MAAM,CAAC;CAC3B;AAED,MAAM,MAAM,SAAS,GAAG,IAAI,GAAG,KAAK,GAAG,SAAS,CAAC;AAEjD,MAAM,MAAM,YAAY,GAClB,cAAc,GACd,YAAY,GACZ,eAAe,GACf,QAAQ,GACR,OAAO,GACP,aAAa,CAAC;AAEpB,MAAM,WAAW,UAAU;IACvB,YAAY,EAAE,SAAS,CAAC;CAC3B;AAED,MAAM,WAAW,cAAc;IAC3B,YAAY,EAAE,YAAY,EAAE,CAAC;CAChC;AAED,MAAM,WAAW,iBAAiB;IAC9B,WAAW,EAAE,MAAM,CAAC;IACpB,UAAU,EAAE,MAAM,CAAC;IACnB,gBAAgB,EAAE,MAAM,CAAC;IACzB,cAAc,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,WAAW,cAAc;IAC3B,OAAO,EAAE,OAAO,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;IAClB,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,EAAE,OAAO,CAAC;IACd,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,OAAO,CAAC,EAAE,QAAQ,GAAG,UAAU,GAAG,SAAS,GAAG,QAAQ,CAAC;IACvD,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,GAAG,CAAC,EAAE,OAAO,CAAC;IACd,gBAAgB,CAAC,EAAE,OAAO,CAAC;IAC3B,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,mBAAmB,CAAC,EAAE,MAAM,CAAC;IAC7B,UAAU,CAAC,EAAE,MAAM,CAAC;CACvB;AAED,MAAM,WAAW,SAAS;IACtB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,QAAQ,CAAC,EAAE,MAAM,CAAC;CACrB;AAED,MAAM,WAAW,UAAU;IACvB,WAAW,EAAE,MAAM,CAAC;IACpB,WAAW,EAAE,MAAM,CAAC;IACpB,gBAAgB,EAAE,MAAM,EAAE,CAAC;CAC9B;AAED,MAAM,WAAW,YAAY;IACzB,wBAAwB,EAAE,MAAM,CAAC;IACjC,sBAAsB,EAAE,MAAM,CAAC;IAC/B,4BAA4B,EAAE,MAAM,CAAC;IACrC,qBAAqB,EAAE,OAAO,CAAC;IAC/B,qBAAqB,EAAE,OAAO,CAAC;IAC/B,iBAAiB,EAAE,MAAM,EAAE,CAAC;IAC5B,eAAe,EAAE,UAAU,GAAG,MAAM,GAAG,OAAO,CAAC;IAC/C,cAAc,EAAE,KAAK,CAAC,SAAS,GAAG,gBAAgB,GAAG,eAAe,CAAC,CAAC;CACzE;AAED,MAAM,WAAW,2BAA2B;IACxC,OAAO,EAAE,OAAO,CAAC;IACjB,QAAQ,EAAE,MAAM,CAAC;IACjB,gBAAgB,EAAE,MAAM,CAAC;IACzB,YAAY,EAAE,MAAM,EAAE,CAAC;IACvB,eAAe,EAAE,MAAM,EAAE,CAAC;IAC1B,UAAU,EAAE,MAAM,EAAE,CAAC;IACrB,WAAW,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,EAAE,CAAC,CAAC;CACzC;AAED,MAAM,WAAW,wBAAwB;IACrC,OAAO,EAAE,OAAO,CAAC;IACjB,YAAY,EAAE,MAAM,CAAC;IACrB,iBAAiB,EAAE,MAAM,CAAC;CAC7B;AAED,MAAM,WAAW,yBAAyB;IACtC,OAAO,EAAE,OAAO,CAAC;IACjB,OAAO,EAAE,MAAM,CAAC;IAChB,eAAe,EAAE,MAAM,CAAC;CAC3B;AAED,MAAM,MAAM,iBAAiB,GAAG,WAAW,GAAG,QAAQ,GAAG,QAAQ,GAAG,MAAM,CAAC;AAE3E,MAAM,WAAW,qBAAqB;IAClC,OAAO,EAAE,OAAO,CAAC;IACjB,QAAQ,EAAE,iBAAiB,CAAC;IAC5B,YAAY,EAAE,MAAM,EAAE,CAAC;IACvB,kBAAkB,EAAE,MAAM,CAAC;IAC3B,iBAAiB,EAAE,MAAM,CAAC;IAC1B,SAAS,EAAE,MAAM,CAAC;IAClB,WAAW,EAAE,MAAM,CAAC;CACvB;AAED,MAAM,WAAW,kBAAkB;IAC/B,OAAO,EAAE,OAAO,CAAC;IACjB,MAAM,EAAE,OAAO,CAAC;IAChB,QAAQ,EAAE,iBAAiB,CAAC;IAC5B,YAAY,EAAE,MAAM,EAAE,CAAC;IACvB,kBAAkB,EAAE,MAAM,CAAC;IAC3B,kBAAkB,EAAE,MAAM,CAAC;IAC3B,SAAS,EAAE,MAAM,CAAC;IAClB,WAAW,EAAE,MAAM,CAAC;CACvB;AAED,MAAM,WAAW,SAAS;IACtB,KAAK,EAAE,MAAM,CAAC;IACd,kBAAkB,CAAC,EAAE,OAAO,CAAC;CAChC;AAED,MAAM,WAAW,mBAAmB;IAChC,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,MAAM,CAAC,EAAE,OAAO,CAAC;CACpB;AAED,MAAM,WAAW,gBAAgB;IAC7B,OAAO,EAAE,OAAO,CAAC;IACjB,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,SAAS,CAAC,EAAE,MAAM,CAAC;CACtB;AAED,MAAM,WAAW,WAAW;IACxB,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,eAAe,CAAC;IACzB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,IAAI,EAAE,YAAY,CAAC;IACnB,SAAS,EAAE,aAAa,CAAC;IACzB,gBAAgB,EAAE,MAAM,CAAC;IACzB,MAAM,EAAE,YAAY,CAAC;IACrB,SAAS,EAAE,cAAc,CAAC;IAC1B,SAAS,EAAE,cAAc,CAAC;IAC1B,aAAa,EAAE,mBAAmB,CAAC;IACnC,aAAa,EAAE,mBAAmB,CAAC;IACnC,cAAc,EAAE,oBAAoB,CAAC;IACrC,MAAM,EAAE;QACJ,aAAa,EAAE,OAAO,CAAC;QACvB,eAAe,EAAE,2BAA2B,CAAC;QAC7C,YAAY,EAAE,wBAAwB,CAAC;QACvC,aAAa,EAAE,yBAAyB,CAAC;QACzC,MAAM,EAAE,kBAAkB,CAAC;QAC3B,SAAS,EAAE,qBAAqB,CAAC;KACpC,CAAC;IACF,QAAQ,EAAE,cAAc,CAAC;IACzB,GAAG,EAAE,SAAS,CAAC;IACf,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,IAAI,EAAE,UAAU,CAAC;IACjB,MAAM,EAAE,YAAY,CAAC;IACrB,KAAK,EAAE,UAAU,CAAC;IAClB,QAAQ,EAAE,cAAc,CAAC;IACzB,WAAW,EAAE,iBAAiB,CAAC;IAC/B,GAAG,EAAE,SAAS,CAAC;IACf,aAAa,EAAE,mBAAmB,CAAC;IACnC,UAAU,EAAE,gBAAgB,CAAC;CAChC;AAED,MAAM,WAAW,cAAc;IAC3B,MAAM,EAAE,WAAW,CAAC;IACpB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,OAAO,EAAE,MAAM,CAAC;CACnB;AAmND,MAAM,WAAW,eAAe;IAC5B,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,OAAO,CAAC,EAAE,eAAe,CAAC;IAC1B,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,IAAI,CAAC,EAAE,YAAY,CAAC;IACpB,SAAS,CAAC,EAAE,aAAa,CAAC;IAC1B,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,MAAM,CAAC,EAAE,YAAY,CAAC;IACtB,YAAY,CAAC,EAAE,MAAM,EAAE,CAAC;IACxB,YAAY,CAAC,EAAE,MAAM,EAAE,CAAC;IACxB,WAAW,CAAC,EAAE,MAAM,EAAE,CAAC;IACvB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,QAAQ,CAAC,EAAE,OAAO,CAAC,cAAc,CAAC,CAAC;IACnC,MAAM,CAAC,EAAE,OAAO,CAAC,YAAY,CAAC,CAAC;IAC/B,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,WAAW,CAAC,EAAE,MAAM,CAAC;CACxB;AA8iBD,wBAAgB,aAAa,CAAC,GAAG,EAAE,MAAM,EAAE,UAAU,CAAC,EAAE,MAAM,EAAE,SAAS,CAAC,EAAE,eAAe,GAAG,cAAc,CA2L3G"}
@@ -338,7 +338,12 @@ function normalizeProfile(value) {
338
338
  return undefined;
339
339
  }
340
340
  function normalizeFramework(value) {
341
- if (value === 'auto' || value === 'playwright' || value === 'cypress' || value === 'selenium') {
341
+ if (value === 'auto' ||
342
+ value === 'playwright' ||
343
+ value === 'cypress' ||
344
+ value === 'pytest' ||
345
+ value === 'supertest' ||
346
+ value === 'selenium') {
342
347
  return value;
343
348
  }
344
349
  return undefined;
@@ -39,6 +39,10 @@ export declare class CachedProvider implements LLMProvider {
39
39
  generateText(prompt: string, options?: GenerateOptions): Promise<LLMResponse>;
40
40
  getUsageStats(): ProviderUsageStats;
41
41
  resetUsageStats(): void;
42
+ checkHealth(): Promise<{
43
+ healthy: boolean;
44
+ message: string;
45
+ }>;
42
46
  }
43
47
  export { ResponseCache, TTL } from './response_cache.js';
44
48
  export type { CacheEntry } from './response_cache.js';
@@ -1 +1 @@
1
- {"version":3,"file":"cached_provider.d.ts","sourceRoot":"","sources":["../../src/cache/cached_provider.ts"],"names":[],"mappings":"AAGA,OAAO,KAAK,EACR,WAAW,EACX,WAAW,EACX,eAAe,EACf,UAAU,EACV,oBAAoB,EACpB,kBAAkB,EACrB,MAAM,0BAA0B,CAAC;AAElC,OAAO,EAAC,aAAa,EAAM,MAAM,qBAAqB,CAAC;AAGvD;;GAEG;AACH,MAAM,WAAW,YAAY;IACzB,iFAAiF;IACjF,KAAK,EAAE,MAAM,CAAC;IACd,4CAA4C;IAC5C,MAAM,EAAE,MAAM,CAAC;IACf,4DAA4D;IAC5D,UAAU,EAAE,MAAM,EAAE,CAAC;CACxB;AAED;;;;;;;;;;GAUG;AACH,qBAAa,cAAe,YAAW,WAAW;IAC9C,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,YAAY,EAAE,oBAAoB,CAAC;IAE5C,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAc;IACpC,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAgB;IACtC,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAe;IACnC,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAS;IAG/B,YAAY,CAAC,EAAE,CAAC,MAAM,EAAE,UAAU,EAAE,EAAE,MAAM,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,eAAe,KAAK,OAAO,CAAC,WAAW,CAAC,CAAC;IACzG,UAAU,CAAC,EAAE,CAAC,MAAM,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,eAAe,KAAK,cAAc,CAAC,MAAM,EAAE,IAAI,EAAE,OAAO,CAAC,CAAC;gBAEtF,KAAK,EAAE,WAAW,EAAE,KAAK,EAAE,aAAa,EAAE,YAAY,EAAE,YAAY;IAqBhF;;;OAGG;IACG,YAAY,CAAC,MAAM,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,eAAe,GAAG,OAAO,CAAC,WAAW,CAAC;IAyCnF,aAAa,IAAI,kBAAkB;IAInC,eAAe,IAAI,IAAI;CAG1B;AAGD,OAAO,EAAC,aAAa,EAAE,GAAG,EAAC,MAAM,qBAAqB,CAAC;AACvD,YAAY,EAAC,UAAU,EAAC,MAAM,qBAAqB,CAAC"}
1
+ {"version":3,"file":"cached_provider.d.ts","sourceRoot":"","sources":["../../src/cache/cached_provider.ts"],"names":[],"mappings":"AAGA,OAAO,KAAK,EACR,WAAW,EACX,WAAW,EACX,eAAe,EACf,UAAU,EACV,oBAAoB,EACpB,kBAAkB,EACrB,MAAM,0BAA0B,CAAC;AAElC,OAAO,EAAC,aAAa,EAAM,MAAM,qBAAqB,CAAC;AAGvD;;GAEG;AACH,MAAM,WAAW,YAAY;IACzB,iFAAiF;IACjF,KAAK,EAAE,MAAM,CAAC;IACd,4CAA4C;IAC5C,MAAM,EAAE,MAAM,CAAC;IACf,4DAA4D;IAC5D,UAAU,EAAE,MAAM,EAAE,CAAC;CACxB;AAED;;;;;;;;;;GAUG;AACH,qBAAa,cAAe,YAAW,WAAW;IAC9C,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,YAAY,EAAE,oBAAoB,CAAC;IAE5C,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAc;IACpC,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAgB;IACtC,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAe;IACnC,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAS;IAG/B,YAAY,CAAC,EAAE,CAAC,MAAM,EAAE,UAAU,EAAE,EAAE,MAAM,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,eAAe,KAAK,OAAO,CAAC,WAAW,CAAC,CAAC;IACzG,UAAU,CAAC,EAAE,CAAC,MAAM,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,eAAe,KAAK,cAAc,CAAC,MAAM,EAAE,IAAI,EAAE,OAAO,CAAC,CAAC;gBAEtF,KAAK,EAAE,WAAW,EAAE,KAAK,EAAE,aAAa,EAAE,YAAY,EAAE,YAAY;IAqBhF;;;OAGG;IACG,YAAY,CAAC,MAAM,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,eAAe,GAAG,OAAO,CAAC,WAAW,CAAC;IAyCnF,aAAa,IAAI,kBAAkB;IAInC,eAAe,IAAI,IAAI;IAIjB,WAAW,IAAI,OAAO,CAAC;QAAC,OAAO,EAAE,OAAO,CAAC;QAAC,OAAO,EAAE,MAAM,CAAA;KAAC,CAAC;CAGpE;AAGD,OAAO,EAAC,aAAa,EAAE,GAAG,EAAC,MAAM,qBAAqB,CAAC;AACvD,YAAY,EAAC,UAAU,EAAC,MAAM,qBAAqB,CAAC"}
@@ -80,6 +80,9 @@ class CachedProvider {
80
80
  resetUsageStats() {
81
81
  this.inner.resetUsageStats();
82
82
  }
83
+ async checkHealth() {
84
+ return this.inner.checkHealth();
85
+ }
83
86
  }
84
87
  exports.CachedProvider = CachedProvider;
85
88
  // Re-export for convenience
@@ -1 +1 @@
1
- {"version":3,"file":"crew.d.ts","sourceRoot":"","sources":["../../../src/cli/commands/crew.ts"],"names":[],"mappings":"AAsBA,OAAO,KAAK,EAAC,UAAU,EAAC,MAAM,aAAa,CAAC;AAI5C,wBAAsB,cAAc,CAAC,IAAI,EAAE,UAAU,EAAE,UAAU,EAAE,MAAM,GAAG,SAAS,GAAG,OAAO,CAAC,IAAI,CAAC,CA8KpG"}
1
+ {"version":3,"file":"crew.d.ts","sourceRoot":"","sources":["../../../src/cli/commands/crew.ts"],"names":[],"mappings":"AAsBA,OAAO,KAAK,EAAC,UAAU,EAAC,MAAM,aAAa,CAAC;AAI5C,wBAAsB,cAAc,CAAC,IAAI,EAAE,UAAU,EAAE,UAAU,EAAE,MAAM,GAAG,SAAS,GAAG,OAAO,CAAC,IAAI,CAAC,CA+KpG"}
@@ -68,6 +68,7 @@ async function runCrewCommand(args, autoConfig) {
68
68
  providerOverride: args.llmProvider,
69
69
  budgetUSD: args.budgetUSD,
70
70
  dryRun: degraded || args.dryRun,
71
+ plugins: args.plugins,
71
72
  };
72
73
  // Create orchestrator and register all agents
73
74
  const orchestrator = new orchestrator_js_1.CrewOrchestrator();
@@ -81,10 +81,10 @@ function buildConfig(answers) {
81
81
  function printNextSteps() {
82
82
  console.log('');
83
83
  console.log(' Next steps:');
84
- console.log(' 1. Set your API key: export ANTHROPIC_API_KEY=sk-ant-...');
85
- console.log(' 2. Test connectivity: npx e2e-ai-agents llm-health');
86
- console.log(' 3. Run impact analysis: npx e2e-ai-agents impact');
87
- console.log(' 4. Add to CI: see examples/github-actions/pr-impact.yml');
84
+ console.log(' 1. Start with impact analysis: npx e2e-ai-agents impact --path .');
85
+ console.log(' 2. Build a coverage plan: npx e2e-ai-agents plan --path .');
86
+ console.log(' 3. Optional AI setup: export ANTHROPIC_API_KEY=sk-ant-...');
87
+ console.log(' 4. Verify provider health: npx e2e-ai-agents llm-health');
88
88
  console.log('');
89
89
  }
90
90
  async function runInitCommand(yes = false) {
@@ -124,7 +124,7 @@ async function runInitCommand(yes = false) {
124
124
  try {
125
125
  const appPath = await ask(rl, ' Path to your web app root', '.');
126
126
  const detectedFramework = (0, defaults_js_1.detectFramework)(appPath);
127
- const framework = await ask(rl, ' Test framework (auto | playwright | cypress | selenium)', detectedFramework);
127
+ const framework = await ask(rl, ' Test framework (auto | playwright | cypress | pytest | supertest | selenium)', detectedFramework);
128
128
  const detectedTestsRoot = (0, defaults_js_1.detectTestsRoot)(appPath);
129
129
  const testsRoot = await ask(rl, ' Path to tests root (relative to app root, "." if same)', detectedTestsRoot || '.');
130
130
  const detectedBranch = (0, defaults_js_1.detectGitDefaultBranch)(appPath);
@@ -1 +1 @@
1
- {"version":3,"file":"llm_health.d.ts","sourceRoot":"","sources":["../../../src/cli/commands/llm_health.ts"],"names":[],"mappings":"AA2DA,wBAAsB,YAAY,IAAI,OAAO,CAAC,IAAI,CAAC,CA4BlD"}
1
+ {"version":3,"file":"llm_health.d.ts","sourceRoot":"","sources":["../../../src/cli/commands/llm_health.ts"],"names":[],"mappings":"AAKA,wBAAsB,YAAY,IAAI,OAAO,CAAC,IAAI,CAAC,CAclD"}
@@ -3,76 +3,20 @@
3
3
  // See LICENSE.txt for license information.
4
4
  Object.defineProperty(exports, "__esModule", { value: true });
5
5
  exports.runLlmHealth = runLlmHealth;
6
- const anthropic_provider_js_1 = require("../../anthropic_provider.js");
7
- const openai_provider_js_1 = require("../../openai_provider.js");
8
- const ollama_provider_js_1 = require("../../ollama_provider.js");
9
- const provider_interface_js_1 = require("../../provider_interface.js");
10
- async function checkAnthropic() {
11
- if (!process.env.ANTHROPIC_API_KEY) {
12
- return null;
13
- }
14
- const model = process.env.ANTHROPIC_MODEL || 'claude-sonnet-4-5-20250929';
15
- try {
16
- const provider = new anthropic_provider_js_1.AnthropicProvider({ apiKey: process.env.ANTHROPIC_API_KEY, model });
17
- const response = await provider.generateText('Reply with OK.', { maxTokens: 8, timeout: 15000 });
18
- return { provider: 'Anthropic', model, ok: true, response: response.text.trim() };
19
- }
20
- catch (error) {
21
- const message = error instanceof provider_interface_js_1.LLMProviderError || error instanceof Error ? error.message : String(error);
22
- return { provider: 'Anthropic', model, ok: false, error: message };
23
- }
24
- }
25
- async function checkOpenAI() {
26
- if (!process.env.OPENAI_API_KEY) {
27
- return null;
28
- }
29
- const model = process.env.OPENAI_MODEL || 'gpt-4o';
30
- try {
31
- const provider = new openai_provider_js_1.OpenAIProvider({ apiKey: process.env.OPENAI_API_KEY, model });
32
- const response = await provider.generateText('Reply with OK.', { maxTokens: 8, timeout: 15000 });
33
- return { provider: 'OpenAI', model, ok: true, response: response.text.trim() };
34
- }
35
- catch (error) {
36
- const message = error instanceof provider_interface_js_1.LLMProviderError || error instanceof Error ? error.message : String(error);
37
- return { provider: 'OpenAI', model, ok: false, error: message };
38
- }
39
- }
40
- async function checkOllama() {
41
- const baseUrl = process.env.OLLAMA_HOST || 'http://localhost:11434';
42
- const model = process.env.OLLAMA_MODEL || 'llama3';
43
- try {
44
- const provider = new ollama_provider_js_1.OllamaProvider({ baseUrl, model });
45
- const response = await provider.generateText('Reply with OK.', { maxTokens: 8, timeout: 15000 });
46
- return { provider: 'Ollama', model, ok: true, response: response.text.trim() };
47
- }
48
- catch (error) {
49
- const message = error instanceof provider_interface_js_1.LLMProviderError || error instanceof Error ? error.message : String(error);
50
- return { provider: 'Ollama', model, ok: false, error: message };
51
- }
52
- }
6
+ const provider_factory_js_1 = require("../../provider_factory.js");
53
7
  async function runLlmHealth() {
54
- const checks = await Promise.allSettled([checkAnthropic(), checkOpenAI(), checkOllama()]);
55
- const results = [];
56
- for (const check of checks) {
57
- if (check.status === 'fulfilled' && check.value) {
58
- results.push(check.value);
59
- }
60
- }
61
- if (results.length === 0) {
62
- console.error('No LLM providers configured. Set ANTHROPIC_API_KEY, OPENAI_API_KEY, or OLLAMA_HOST.');
63
- process.exit(1);
64
- }
65
- let anyFailed = false;
66
- for (const r of results) {
67
- if (r.ok) {
68
- console.log(`${r.provider} OK (${r.model}) -> ${r.response}`);
69
- }
70
- else {
71
- console.error(`${r.provider} failed (${r.model}): ${r.error}`);
72
- anyFailed = true;
8
+ try {
9
+ const provider = await provider_factory_js_1.LLMProviderFactory.createFromEnv();
10
+ const health = await provider.checkHealth();
11
+ if (!health.healthy) {
12
+ console.error(`${provider.name} failed: ${health.message}`);
13
+ process.exit(1);
73
14
  }
15
+ console.log(`${provider.name} OK -> ${health.message}`);
74
16
  }
75
- if (anyFailed) {
17
+ catch (error) {
18
+ const message = error instanceof Error ? error.message : String(error);
19
+ console.error(`LLM health check failed: ${message}`);
76
20
  process.exit(1);
77
21
  }
78
22
  }
@@ -1 +1 @@
1
- {"version":3,"file":"defaults.d.ts","sourceRoot":"","sources":["../../src/cli/defaults.ts"],"names":[],"mappings":"AAOA,OAAO,KAAK,EAAC,aAAa,EAAC,MAAM,oBAAoB,CAAC;AAEtD,MAAM,WAAW,gBAAgB;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb,SAAS,EAAE,MAAM,CAAC;IAClB,SAAS,EAAE,aAAa,CAAC;IACzB,KAAK,EAAE,MAAM,CAAC;CACjB;AAED;;GAEG;AACH,wBAAgB,eAAe,CAAC,OAAO,EAAE,MAAM,GAAG,aAAa,CAsB9D;AAED;;GAEG;AACH,wBAAgB,eAAe,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAoBnE;AAED;;;GAGG;AACH,wBAAgB,sBAAsB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,CA4B9D;AAED;;GAEG;AACH,wBAAgB,iBAAiB,CAAC,QAAQ,EAAE,MAAM,GAAG,MAAM,CAa1D;AAED;;;GAGG;AACH,wBAAgB,eAAe,CAAC,QAAQ,EAAE;IACtC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,SAAS,CAAC,EAAE,aAAa,CAAC;IAC1B,QAAQ,CAAC,EAAE,MAAM,CAAC;CACrB,GAAG,gBAAgB,CAOnB"}
1
+ {"version":3,"file":"defaults.d.ts","sourceRoot":"","sources":["../../src/cli/defaults.ts"],"names":[],"mappings":"AAOA,OAAO,KAAK,EAAC,aAAa,EAAC,MAAM,oBAAoB,CAAC;AAEtD,MAAM,WAAW,gBAAgB;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb,SAAS,EAAE,MAAM,CAAC;IAClB,SAAS,EAAE,aAAa,CAAC;IACzB,KAAK,EAAE,MAAM,CAAC;CACjB;AAyCD;;GAEG;AACH,wBAAgB,eAAe,CAAC,OAAO,EAAE,MAAM,GAAG,aAAa,CAkC9D;AAED;;GAEG;AACH,wBAAgB,eAAe,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAoBnE;AAED;;;GAGG;AACH,wBAAgB,sBAAsB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,CA4B9D;AAED;;GAEG;AACH,wBAAgB,iBAAiB,CAAC,QAAQ,EAAE,MAAM,GAAG,MAAM,CAa1D;AAED;;;GAGG;AACH,wBAAgB,eAAe,CAAC,QAAQ,EAAE;IACtC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,SAAS,CAAC,EAAE,aAAa,CAAC;IAC1B,QAAQ,CAAC,EAAE,MAAM,CAAC;CACrB,GAAG,gBAAgB,CAOnB"}