bmad-method-test-architecture-enterprise 1.2.6 → 1.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +65 -10
- package/docs/explanation/tea-overview.md +41 -2
- package/docs/explanation/testing-as-engineering.md +39 -35
- package/docs/how-to/customization/configure-browser-automation.md +67 -2
- package/docs/reference/configuration.md +149 -4
- package/docs/reference/knowledge-base.md +22 -6
- package/docs/reference/troubleshooting.md +8 -6
- package/package.json +1 -1
- package/release_notes.md +10 -4
- package/src/module.yaml +49 -1
- package/src/testarch/knowledge/contract-testing.md +24 -2
- package/src/testarch/knowledge/pact-mcp.md +202 -0
- package/src/testarch/knowledge/pactjs-utils-consumer-helpers.md +211 -0
- package/src/testarch/knowledge/pactjs-utils-overview.md +210 -0
- package/src/testarch/knowledge/pactjs-utils-provider-verifier.md +315 -0
- package/src/testarch/knowledge/pactjs-utils-request-filter.md +224 -0
- package/src/testarch/tea-index.csv +5 -0
- package/src/workflows/testarch/atdd/steps-c/step-01-preflight-and-context.md +30 -0
- package/src/workflows/testarch/atdd/workflow.yaml +1 -0
- package/src/workflows/testarch/automate/steps-c/step-01-preflight-and-context.md +32 -0
- package/src/workflows/testarch/automate/steps-c/step-03-generate-tests.md +15 -0
- package/src/workflows/testarch/automate/steps-c/step-03a-subprocess-api.md +11 -1
- package/src/workflows/testarch/automate/workflow.yaml +1 -0
- package/src/workflows/testarch/ci/steps-c/step-02-generate-pipeline.md +46 -0
- package/src/workflows/testarch/ci/steps-c/step-03-configure-quality-gates.md +7 -0
- package/src/workflows/testarch/ci/workflow.yaml +1 -0
- package/src/workflows/testarch/framework/steps-c/step-03-scaffold-framework.md +32 -0
- package/src/workflows/testarch/framework/workflow.yaml +1 -0
- package/src/workflows/testarch/nfr-assess/workflow.yaml +1 -0
- package/src/workflows/testarch/test-design/steps-c/step-02-load-context.md +30 -0
- package/src/workflows/testarch/test-design/workflow.yaml +1 -0
- package/src/workflows/testarch/test-review/steps-c/step-01-load-context.md +29 -1
- package/src/workflows/testarch/test-review/workflow.yaml +1 -1
- package/src/workflows/testarch/trace/workflow.yaml +1 -0
- package/test/test-installation-components.js +1 -1
- package/test/test-knowledge-base.js +1 -1
- package/tools/validate-doc-links.js +2 -2
- package/docs/how-to/customization/enable-tea-mcp-enhancements.md +0 -23
package/README.md
CHANGED
|
@@ -21,26 +21,79 @@ TEA plugs into BMad the same way a specialist plugs into a team. It uses the sam
|
|
|
21
21
|
|
|
22
22
|
## Architecture & Flow
|
|
23
23
|
|
|
24
|
-
BMad is a small **agent + workflow engine
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
24
|
+
BMad is a small **agent + workflow engine**. There is no external orchestrator — everything runs inside the LLM context window through structured instructions.
|
|
25
|
+
|
|
26
|
+
### Building Blocks
|
|
27
|
+
|
|
28
|
+
Each workflow directory contains these files, and each has a specific job:
|
|
29
|
+
|
|
30
|
+
| File | What it does | When it loads |
|
|
31
|
+
| ----------------- | ------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
|
|
32
|
+
| `tea.agent.yaml` | Expert persona — identity, principles, critical actions, menu of triggers | First — always in context |
|
|
33
|
+
| `workflow.yaml` | Machine-readable metadata — config variables, required tools, tags | Second — resolves `{project-root}`, `{config_source}`, `{test_artifacts}` |
|
|
34
|
+
| `workflow.md` | Human-readable entry point — goals, mode menu (Create/Edit/Validate), routes to first step | Second — presents mode choice |
|
|
35
|
+
| `instructions.md` | Workflow-specific rules and context (optional, supplements workflow.md) | On demand |
|
|
36
|
+
| `steps-c/*.md` | **Create** steps — primary execution, 5-9 sequential files | One at a time (just-in-time) |
|
|
37
|
+
| `steps-e/*.md` | **Edit** steps — always 2 files: assess target, apply edit | One at a time |
|
|
38
|
+
| `steps-v/*.md` | **Validate** steps — always 1 file: evaluate against checklist | On demand |
|
|
39
|
+
| `checklist.md` | Validation criteria — what "done" looks like for this workflow | Read by steps-v |
|
|
40
|
+
| `*-template.md` | Output skeleton with `{PLACEHOLDER}` vars — steps fill these in to produce the final artifact | Read by steps-c when generating output |
|
|
41
|
+
| `tea-index.csv` | Knowledge fragment index — id, name, tags, tier (core/extended/specialized), file path | Read by step-01 to decide which fragments to load |
|
|
42
|
+
| `knowledge/*.md` | 40 reusable fragments — standards, patterns, API references (e.g., `data-factories.md`, `pactjs-utils-overview.md`) | Selectively read into context based on tier + config flags |
|
|
32
43
|
|
|
33
44
|
```mermaid
|
|
34
45
|
flowchart LR
|
|
35
46
|
U[User] --> A[Agent Persona]
|
|
36
47
|
A --> W[Workflow Entry: workflow.md]
|
|
37
48
|
W --> S[Step Files: steps-c / steps-e / steps-v]
|
|
38
|
-
S --> K[Knowledge Fragments<br/>
|
|
39
|
-
S --> T[Templates & Checklists<br
|
|
49
|
+
S --> K[Knowledge Fragments<br/>tea-index.csv → knowledge/*.md]
|
|
50
|
+
S --> T[Templates & Checklists<br/>*-template.md, checklist.md]
|
|
40
51
|
S --> O[Outputs: docs/tests/reports<br/>when a step writes output]
|
|
41
52
|
O --> V[Validation: checklist + report]
|
|
42
53
|
```
|
|
43
54
|
|
|
55
|
+
### How It Works at Runtime
|
|
56
|
+
|
|
57
|
+
1. **Trigger** — User types `/bmad:tea:automate` (or shorthand `TA`). The agent menu in `tea.agent.yaml` maps the trigger to `automate/workflow.yaml`.
|
|
58
|
+
2. **Agent loads** — `tea.agent.yaml` injects the persona (identity, principles, critical actions) into the context window.
|
|
59
|
+
3. **Workflow loads** — `workflow.yaml` resolves config variables and `workflow.md` presents the mode menu (Create / Edit / Validate), then routes to the first step file.
|
|
60
|
+
4. **Step-by-step execution** — Only the current step file is in context (just-in-time loading). Each step explicitly names the next one (`nextStepFile: './step-02-...'`). The LLM reads, executes, saves output, then loads the next step. No future steps are ever preloaded.
|
|
61
|
+
5. **Knowledge injection** — Step-01 reads `tea-index.csv` and selectively loads fragments by **tier** (core = always, extended = on-demand, specialized = only when relevant) and **config flags** (e.g., `tea_use_pactjs_utils`). This is deliberate context engineering: a backend project loads ~1,800 lines of fragments; a fullstack project loads ~4,500 lines. Conditional loading cuts context usage by 40-50%.
|
|
62
|
+
6. **Templates** — When a step produces output (e.g., a traceability matrix or test review report), it reads the `*-template.md` file and fills in the `{PLACEHOLDER}` values with computed results. The template provides consistent structure; the step provides the content.
|
|
63
|
+
7. **Subprocess isolation** — Heavy workflows (e.g., `automate`) spawn parallel subprocesses that each run in an isolated context. Subprocesses write structured JSON to temp files. An aggregation step reads the JSON outputs — only the results enter the main context, not the full subprocess history.
|
|
64
|
+
8. **Progress tracking** — Each step appends to an output file with YAML frontmatter (`stepsCompleted`, `lastStep`, `lastSaved`). Resume mode reads this frontmatter and routes to the next incomplete step.
|
|
65
|
+
9. **Validation** — The `steps-v/` mode reads `checklist.md` and evaluates the workflow's output against its criteria, producing a pass/fail validation report.
|
|
66
|
+
|
|
67
|
+
### Workflows vs Skills
|
|
68
|
+
|
|
69
|
+
BMad workflows and Claude Code Skills solve different problems at different scales:
|
|
70
|
+
|
|
71
|
+
| Capability | Claude Code Skills | BMad Workflows |
|
|
72
|
+
| ----------------- | --------------------------- | ---------------------------------------------------------------------------- |
|
|
73
|
+
| **Execution** | Single prompt, one shot | 5-9 sequential steps with explicit handoffs |
|
|
74
|
+
| **State** | Stateless | YAML frontmatter tracking (`stepsCompleted`, `lastStep`) with resume |
|
|
75
|
+
| **Knowledge** | Whatever fits in one prompt | Tiered index (40 fragments), conditional loading by config + stack detection |
|
|
76
|
+
| **Context mgmt** | Everything in one shot | Just-in-time step loading, subprocess isolation (separate contexts) |
|
|
77
|
+
| **Output** | Freeform | Templates with `{PLACEHOLDER}` vars filled by specific steps |
|
|
78
|
+
| **Validation** | None | Dedicated mode (`steps-v/`) evaluating against checklists |
|
|
79
|
+
| **Configuration** | None | `module.yaml` with prompted config flags driving conditional behavior |
|
|
80
|
+
| **Modes** | None | Create / Edit / Validate — three separate step chains per workflow |
|
|
81
|
+
|
|
82
|
+
The key insight is that there is **no external runtime engine** — the LLM _is_ the engine. BMad workflows are structured markdown that the LLM follows as instructions: "read this file, execute it completely, save your output, load the next file." Skills are a single tool in a toolbox; BMad workflows are a workshop with a process manual.
|
|
83
|
+
|
|
84
|
+
**How workflows become commands.** When you run `npx bmad-method install`, the installer converts every workflow and agent into a Claude Code command in `.claude/commands/`. For example, `bmad-tea-testarch-automate.md` tells the LLM: "load the core workflow engine (`workflow.xml`), pass it this workflow config (`automate/workflow.yaml`), follow the instructions exactly." That single command file is the bridge — it triggers the workflow entry point; the multi-step engine takes over from there.
|
|
85
|
+
|
|
86
|
+
```
|
|
87
|
+
.claude/commands/ # Generated by installer
|
|
88
|
+
├── bmad-agent-tea-tea.md # /tea → loads agent persona + menu
|
|
89
|
+
├── bmad-tea-testarch-automate.md # /automate → loads workflow.xml + workflow.yaml
|
|
90
|
+
├── bmad-tea-testarch-test-design.md # /test-design → ...
|
|
91
|
+
├── bmad-bmm-create-prd.md # /create-prd → BMM workflow
|
|
92
|
+
└── ... (61 commands total across all installed modules)
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
The BMAD-METHOD source repo also has standalone `.claude/skills/` (e.g., `bmad-os-release-module`, `bmad-os-gh-triage`) for its own maintenance workflows. External tools can register skills too (e.g., `playwright-cli install --skills`). The installer supports 10+ platforms: Claude Code, Cursor, GitHub Copilot, Codex, Gemini, Windsurf, Cline, and more.
|
|
96
|
+
|
|
44
97
|
## Install
|
|
45
98
|
|
|
46
99
|
```bash
|
|
@@ -86,6 +139,8 @@ TEA variables are defined in `src/module.yaml` and prompted during install:
|
|
|
86
139
|
|
|
87
140
|
- `test_artifacts` — base output folder for test artifacts
|
|
88
141
|
- `tea_use_playwright_utils` — enable Playwright Utils integration (boolean)
|
|
142
|
+
- `tea_use_pactjs_utils` — enable Pact.js Utils integration for contract testing (boolean)
|
|
143
|
+
- `tea_pact_mcp` — SmartBear MCP for PactFlow/Broker interaction: mcp, none (string)
|
|
89
144
|
- `tea_browser_automation` — browser automation mode: auto, cli, mcp, none (string)
|
|
90
145
|
- `test_framework` — detected or configured test framework (Playwright, Cypress, Jest, Vitest, pytest, JUnit, Go test, dotnet test, RSpec)
|
|
91
146
|
- `test_stack_type` — detected or configured stack type (frontend, backend, fullstack)
|
|
@@ -338,6 +338,16 @@ Production-ready fixtures and utilities that enhance TEA workflows.
|
|
|
338
338
|
- Impacts: `framework`, `atdd`, `automate`, `test-review`, `ci`
|
|
339
339
|
- Utilities include: api-request, auth-session, network-recorder, intercept-network-call, recurse, log, file-utils, burn-in, network-error-monitor, fixtures-composition
|
|
340
340
|
|
|
341
|
+
### Pact.js Utils (`@seontechnologies/pactjs-utils`)
|
|
342
|
+
|
|
343
|
+
Production-ready contract testing utilities that reduce raw Pact.js boilerplate and standardize provider verification patterns.
|
|
344
|
+
|
|
345
|
+
- Install: `npm install -D @seontechnologies/pactjs-utils @pact-foundation/pact`
|
|
346
|
+
- Config: `tea_use_pactjs_utils: true` (default is `true`)
|
|
347
|
+
- Impacts: `framework`, `atdd`, `automate`, `test-design`, `test-review`, `ci`
|
|
348
|
+
- Utilities include: createProviderState, toJsonMap, buildVerifierOptions, buildMessageVerifierOptions, createRequestFilter, noOpRequestFilter, handlePactBrokerUrlAndSelectors, getProviderVersionTags
|
|
349
|
+
- Supports local monorepo flow (`pactUrls`) and remote broker flow (`PACT_BROKER_BASE_URL`, `PACT_BROKER_TOKEN`)
|
|
350
|
+
|
|
341
351
|
### Browser Automation (Playwright CLI + MCP)
|
|
342
352
|
|
|
343
353
|
**CLI and MCP are complementary tools, not competitors.** Auto mode uses each where it shines — CLI for token-efficient stateless snapshots, MCP for rich stateful automation — while giving users full control to override when they know better.
|
|
@@ -370,6 +380,34 @@ Production-ready fixtures and utilities that enhance TEA workflows.
|
|
|
370
380
|
|
|
371
381
|
**To disable**: set `tea_browser_automation: "none"` in config or skip both CLI and MCP installation.
|
|
372
382
|
|
|
383
|
+
### Pact MCP (SmartBear MCP for PactFlow/Pact Broker)
|
|
384
|
+
|
|
385
|
+
Optional MCP integration for design-time broker interaction in contract testing workflows.
|
|
386
|
+
|
|
387
|
+
**Configuration** (`_bmad/tea/config.yaml`):
|
|
388
|
+
|
|
389
|
+
tea_pact_mcp: "mcp" # mcp | none
|
|
390
|
+
|
|
391
|
+
| Mode | What happens |
|
|
392
|
+
| ------ | ------------------------------------------------------------------------------------------------------------------- |
|
|
393
|
+
| `mcp` | TEA can use SmartBear MCP tools for provider-state discovery, test review support, can-i-deploy, and matrix checks. |
|
|
394
|
+
| `none` | TEA uses CLI/script guidance only for broker interactions. |
|
|
395
|
+
|
|
396
|
+
**Setup:**
|
|
397
|
+
|
|
398
|
+
- Install: `npm install -g @smartbear/mcp` (or use `npx -y @smartbear/mcp@latest`)
|
|
399
|
+
- Claude Code example: `claude mcp add --transport stdio smartbear -- npx -y @smartbear/mcp@latest`
|
|
400
|
+
- Required broker env vars: `PACT_BROKER_BASE_URL` and token/basic-auth credentials
|
|
401
|
+
|
|
402
|
+
**Which workflows benefit:**
|
|
403
|
+
|
|
404
|
+
- `test-design` — Fetch provider states and broker landscape
|
|
405
|
+
- `automate` — Assist pact test generation with broker context
|
|
406
|
+
- `test-review` — Review pact tests against broker-informed best practices
|
|
407
|
+
- `ci` — Reference can-i-deploy and matrix checks
|
|
408
|
+
|
|
409
|
+
**Note:** Pact MCP complements `pactjs-utils`. MCP helps at planning/review time; `pactjs-utils` is used in runtime test code.
|
|
410
|
+
|
|
373
411
|
---
|
|
374
412
|
|
|
375
413
|
## Complete TEA Documentation Navigation
|
|
@@ -400,6 +438,7 @@ Production-ready fixtures and utilities that enhance TEA workflows.
|
|
|
400
438
|
|
|
401
439
|
- [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md) - Production-ready fixtures and 9 utilities
|
|
402
440
|
- [Configure Browser Automation](/docs/how-to/customization/configure-browser-automation.md) - Playwright CLI + MCP setup, auto mode
|
|
441
|
+
- [TEA Configuration Reference (Pact.js Utils + Pact MCP)](/docs/reference/configuration.md#tea_use_pactjs_utils) - Contract-testing integrations and setup
|
|
403
442
|
|
|
404
443
|
### Use-Case Guides
|
|
405
444
|
|
|
@@ -416,7 +455,7 @@ Production-ready fixtures and utilities that enhance TEA workflows.
|
|
|
416
455
|
- [Test Quality Standards](/docs/explanation/test-quality-standards.md) - Definition of Done, determinism, isolation, explicit assertions
|
|
417
456
|
- [Fixture Architecture](/docs/explanation/fixture-architecture.md) - Pure function → fixture → composition pattern
|
|
418
457
|
- [Network-First Patterns](/docs/explanation/network-first-patterns.md) - Intercept-before-navigate, eliminating flakiness
|
|
419
|
-
- [Knowledge Base System](/docs/explanation/knowledge-base-system.md) - Context engineering with tea-index.csv,
|
|
458
|
+
- [Knowledge Base System](/docs/explanation/knowledge-base-system.md) - Context engineering with tea-index.csv, 40 fragments
|
|
420
459
|
- [Engagement Models](/docs/explanation/engagement-models.md) - TEA Lite, TEA Solo, TEA Integrated (5 models explained)
|
|
421
460
|
|
|
422
461
|
### Philosophy & Design
|
|
@@ -431,5 +470,5 @@ Production-ready fixtures and utilities that enhance TEA workflows.
|
|
|
431
470
|
|
|
432
471
|
- [TEA Command Reference](/docs/reference/commands.md) - All 9 workflows: inputs, outputs, phases, frequency
|
|
433
472
|
- [TEA Configuration Reference](/docs/reference/configuration.md) - Config options, file locations, setup examples
|
|
434
|
-
- [Knowledge Base Index](/docs/reference/knowledge-base.md) -
|
|
473
|
+
- [Knowledge Base Index](/docs/reference/knowledge-base.md) - 40 fragments categorized and explained
|
|
435
474
|
- [Glossary - TEA Section](/docs/glossary/index.md#test-architect-tea-concepts) - 20 TEA-specific terms defined
|
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: 'AI-Generated Testing: Why Most Approaches Fail'
|
|
3
|
-
description: How Playwright-Utils, TEA workflows, and
|
|
3
|
+
description: How Playwright-Utils, pactjs-utils, TEA workflows, Playwright CLI, and MCPs solve AI test quality problems
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
AI-generated tests frequently fail in production because they lack systematic quality standards. This document explains the problem and presents a solution combining
|
|
6
|
+
AI-generated tests frequently fail in production because they lack systematic quality standards. This document explains the problem and presents a solution combining utility standards, TEA workflows, and automation interfaces across both UI, API and contract testing.
|
|
7
7
|
|
|
8
8
|
:::note[Source]
|
|
9
9
|
This article is adapted from [The Testing Meta Most Teams Have Not Caught Up To Yet](https://dev.to/muratkeremozcan/the-testing-meta-most-teams-have-not-caught-up-to-yet-5765) by Murat K Ozcan.
|
|
@@ -30,25 +30,22 @@ AI excels at generating code quickly, but testing requires precision and consist
|
|
|
30
30
|
|
|
31
31
|
The solution combines three components that work together to enforce quality:
|
|
32
32
|
|
|
33
|
-
### Playwright-Utils
|
|
33
|
+
### 1. Utilities: Playwright-Utils + Pact.js Utils
|
|
34
34
|
|
|
35
|
-
|
|
35
|
+
`@seontechnologies/playwright-utils` standardizes commonly reinvented testing primitives across UI, API, web, and non-web flows. `@seontechnologies/pactjs-utils` standardizes Pact.js contract-testing primitives for provider state setup, request filtering, and provider/message verifier configuration.
|
|
36
36
|
|
|
37
|
-
| Utility
|
|
38
|
-
|
|
|
39
|
-
|
|
|
40
|
-
|
|
|
41
|
-
| intercept-network-call | Network mocking and interception |
|
|
42
|
-
| recurse | Retry logic and polling |
|
|
43
|
-
| log | Structured logging |
|
|
44
|
-
| network-recorder | Record and replay network traffic |
|
|
45
|
-
| burn-in | Smart test selection for CI |
|
|
46
|
-
| network-error-monitor | HTTP error detection |
|
|
47
|
-
| file-utils | CSV/PDF handling |
|
|
37
|
+
| Track | Utility Layer | Purpose |
|
|
38
|
+
| ------------------ | ------------------------------------ | -------------------------------------------------------- |
|
|
39
|
+
| UI/API/Web/Non-web | `@seontechnologies/playwright-utils` | Reusable testing primitives and fixtures |
|
|
40
|
+
| Contract | `@seontechnologies/pactjs-utils` | Reusable Pact consumer/provider helpers and verification |
|
|
48
41
|
|
|
49
|
-
|
|
42
|
+
**Playwright-Utils examples:** `api-request`, `auth-session`, `intercept-network-call`, `recurse`, `log`, `network-recorder`, `burn-in`, `network-error-monitor`, `file-utils`.
|
|
50
43
|
|
|
51
|
-
|
|
44
|
+
**pactjs-utils examples:** `createProviderState`, `toJsonMap`, `createRequestFilter`, `noOpRequestFilter`, `buildVerifierOptions`, `buildMessageVerifierOptions`.
|
|
45
|
+
|
|
46
|
+
Together, these utility libraries eliminate the need to reinvent core testing primitives across UI, API, web, non-web, and contract testing.
|
|
47
|
+
|
|
48
|
+
### 2. Process: TEA (Test Architect Agent)
|
|
52
49
|
|
|
53
50
|
A quality operating model packaged as eight executable workflows spanning test design, CI/CD gates, and release readiness. TEA encodes test architecture expertise into repeatable processes.
|
|
54
51
|
|
|
@@ -67,28 +64,35 @@ A quality operating model packaged as eight executable workflows spanning test d
|
|
|
67
64
|
TEA doesn't just generate tests—it provides a complete quality operating model with workflows for planning, execution, and release gates.
|
|
68
65
|
:::
|
|
69
66
|
|
|
70
|
-
### Playwright MCPs
|
|
67
|
+
### 3. Automation Interfaces: Playwright CLI + MCPs
|
|
68
|
+
|
|
69
|
+
Automation interfaces enable real-time verification during test generation and review across browser and contract tracks:
|
|
70
|
+
|
|
71
|
+
- **Playwright CLI**: token-efficient browser automation for stateless execution and fast checks in workflows.
|
|
72
|
+
- **Playwright MCP**: stateful browser automation with richer context for interactive exploration and DOM validation.
|
|
73
|
+
- **Pact MCP**: broker-aware contract automation for verification matrix queries, provider-state discovery, compatibility analysis, and `can-i-deploy` deployment decisions.
|
|
71
74
|
|
|
72
|
-
|
|
75
|
+
Instead of inferring behavior from documentation alone, these interfaces allow agents to:
|
|
73
76
|
|
|
74
|
-
- Run flows and confirm the DOM against the accessibility tree
|
|
75
|
-
- Validate network
|
|
76
|
-
-
|
|
77
|
-
-
|
|
77
|
+
- Run browser flows and confirm the DOM against the accessibility tree
|
|
78
|
+
- Validate UI/API network behavior in real-time
|
|
79
|
+
- Query Pact verification matrix results across consumer/provider versions
|
|
80
|
+
- Check provider states and contract compatibility before release
|
|
81
|
+
- Execute `can-i-deploy` checks against target environments
|
|
78
82
|
|
|
79
83
|
## How They Work Together
|
|
80
84
|
|
|
81
85
|
The three components form a quality pipeline:
|
|
82
86
|
|
|
83
|
-
| Stage | Component
|
|
84
|
-
| ------------ |
|
|
85
|
-
| Standards | Playwright-Utils | Provides production-ready patterns and
|
|
86
|
-
| Process | TEA Workflows
|
|
87
|
-
| Verification | Playwright
|
|
87
|
+
| Stage | Component | Action |
|
|
88
|
+
| ------------ | ------------------------------------------ | ------------------------------------------------------------ |
|
|
89
|
+
| Standards | Playwright-Utils + pactjs-utils | Provides production-ready patterns for UI and contract tests |
|
|
90
|
+
| Process | TEA Workflows | Enforces systematic test planning and review |
|
|
91
|
+
| Verification | Playwright CLI + Playwright MCP + Pact MCP | Validates tests and contracts against live systems |
|
|
88
92
|
|
|
89
93
|
**Before (AI-only):** 20 tests with redundant coverage, incorrect assertions, and flaky behavior.
|
|
90
94
|
|
|
91
|
-
**After (Full Stack):** Risk-based selection, verified selectors, validated behavior, reviewable code.
|
|
95
|
+
**After (Full Stack):** Risk-based selection, verified selectors, validated behavior, contract compatibility checks, reviewable code.
|
|
92
96
|
|
|
93
97
|
## Why This Matters
|
|
94
98
|
|
|
@@ -101,11 +105,11 @@ Traditional AI testing approaches fail because they:
|
|
|
101
105
|
|
|
102
106
|
The three-part stack addresses each gap:
|
|
103
107
|
|
|
104
|
-
| Gap | Solution
|
|
105
|
-
| --------------- |
|
|
106
|
-
| No standards | Playwright-Utils
|
|
107
|
-
| No planning | TEA `test-design` creates risk-based test plans
|
|
108
|
-
| No verification | Playwright
|
|
109
|
-
| No review | TEA `test-review` audits quality with scoring
|
|
108
|
+
| Gap | Solution |
|
|
109
|
+
| --------------- | ------------------------------------------------------------------------ |
|
|
110
|
+
| No standards | Playwright-Utils + pactjs-utils provide production-ready patterns |
|
|
111
|
+
| No planning | TEA `test-design` creates risk-based test plans |
|
|
112
|
+
| No verification | Playwright CLI + Playwright MCP + Pact MCP validate against live systems |
|
|
113
|
+
| No review | TEA `test-review` audits quality with scoring |
|
|
110
114
|
|
|
111
115
|
This approach is sometimes called _context engineering_—loading domain-specific standards into AI context automatically rather than relying on prompts alone. TEA's `tea-index.csv` manifest loads relevant knowledge fragments so the AI doesn't relearn testing patterns each session.
|
|
@@ -37,7 +37,7 @@ The global npm install is one-time. The skills install (`playwright-cli install
|
|
|
37
37
|
|
|
38
38
|
### For MCP (`mcp` or `auto` mode)
|
|
39
39
|
|
|
40
|
-
|
|
40
|
+
Add these MCP server entries to your tool's configuration file:
|
|
41
41
|
|
|
42
42
|
```json
|
|
43
43
|
{
|
|
@@ -49,12 +49,77 @@ Configure MCP servers in your IDE:
|
|
|
49
49
|
"playwright-test": {
|
|
50
50
|
"command": "npx",
|
|
51
51
|
"args": ["playwright", "run-test-mcp-server"]
|
|
52
|
+
},
|
|
53
|
+
"smartbear": {
|
|
54
|
+
"command": "npx",
|
|
55
|
+
"args": ["-y", "@smartbear/mcp@latest"],
|
|
56
|
+
"env": {
|
|
57
|
+
"PACT_BROKER_BASE_URL": "https://{tenant}.pactflow.io",
|
|
58
|
+
"PACT_BROKER_TOKEN": "<your-api-token>"
|
|
59
|
+
}
|
|
52
60
|
}
|
|
53
61
|
}
|
|
54
62
|
}
|
|
55
63
|
```
|
|
56
64
|
|
|
57
|
-
|
|
65
|
+
The `smartbear` server is optional — only needed if you use the [Pact MCP integration](/docs/reference/configuration.md#tea_pact_mcp) for contract testing workflows. See the [pact-mcp knowledge fragment](/docs/reference/knowledge-base.md#pact-contract-testing-integration) for details.
|
|
66
|
+
|
|
67
|
+
#### Where to put the config
|
|
68
|
+
|
|
69
|
+
| Tool | Config File | Format |
|
|
70
|
+
| ----------- | ------------------------------------- | ---------------------- |
|
|
71
|
+
| Claude Code | `~/.claude.json` | JSON (`mcpServers`) |
|
|
72
|
+
| Codex | `~/.codex/config.toml` | TOML (`[mcp_servers]`) |
|
|
73
|
+
| Gemini CLI | `~/.gemini/settings.json` | JSON (`mcpServers`) |
|
|
74
|
+
| Cursor | `~/.cursor/mcp.json` | JSON (`mcpServers`) |
|
|
75
|
+
| Windsurf | `~/.codeium/windsurf/mcp_config.json` | JSON (`mcpServers`) |
|
|
76
|
+
|
|
77
|
+
#### CLI shortcuts
|
|
78
|
+
|
|
79
|
+
Claude Code and Codex support adding MCP servers from the command line:
|
|
80
|
+
|
|
81
|
+
```bash
|
|
82
|
+
# Claude Code — Playwright
|
|
83
|
+
claude mcp add playwright -- npx @playwright/mcp@latest
|
|
84
|
+
claude mcp add playwright-test -- npx playwright run-test-mcp-server
|
|
85
|
+
|
|
86
|
+
# Claude Code — SmartBear (Pact)
|
|
87
|
+
claude mcp add smartbear --scope user \
|
|
88
|
+
-e PACT_BROKER_BASE_URL=https://{tenant}.pactflow.io \
|
|
89
|
+
-e PACT_BROKER_TOKEN=<your-token> \
|
|
90
|
+
-- npx -y @smartbear/mcp@latest
|
|
91
|
+
|
|
92
|
+
# Codex — Playwright
|
|
93
|
+
codex mcp add playwright -- npx @playwright/mcp@latest
|
|
94
|
+
codex mcp add playwright-test -- npx playwright run-test-mcp-server
|
|
95
|
+
|
|
96
|
+
# Codex — SmartBear (Pact)
|
|
97
|
+
codex mcp add smartbear -- npx -y @smartbear/mcp@latest
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
#### Codex TOML format
|
|
101
|
+
|
|
102
|
+
Codex uses TOML instead of JSON. If editing the config file manually:
|
|
103
|
+
|
|
104
|
+
```toml
|
|
105
|
+
[mcp_servers.playwright]
|
|
106
|
+
command = "npx"
|
|
107
|
+
args = ["@playwright/mcp@latest"]
|
|
108
|
+
|
|
109
|
+
[mcp_servers.playwright-test]
|
|
110
|
+
command = "npx"
|
|
111
|
+
args = ["playwright", "run-test-mcp-server"]
|
|
112
|
+
|
|
113
|
+
[mcp_servers.smartbear]
|
|
114
|
+
command = "npx"
|
|
115
|
+
args = ["-y", "@smartbear/mcp@latest"]
|
|
116
|
+
|
|
117
|
+
[mcp_servers.smartbear.env]
|
|
118
|
+
PACT_BROKER_BASE_URL = "https://{tenant}.pactflow.io"
|
|
119
|
+
PACT_BROKER_TOKEN = "<your-api-token>"
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
Note the key is `mcp_servers` (underscored), not `mcpServers`.
|
|
58
123
|
|
|
59
124
|
## How Auto Mode Works
|
|
60
125
|
|
|
@@ -30,6 +30,8 @@ user_skill_level: intermediate
|
|
|
30
30
|
output_folder: _bmad-output
|
|
31
31
|
test_artifacts: _bmad-output/test-artifacts
|
|
32
32
|
tea_use_playwright_utils: true
|
|
33
|
+
tea_use_pactjs_utils: true
|
|
34
|
+
tea_pact_mcp: 'mcp'
|
|
33
35
|
tea_browser_automation: 'auto'
|
|
34
36
|
```
|
|
35
37
|
|
|
@@ -75,19 +77,18 @@ test_artifacts: docs/testing-artifacts
|
|
|
75
77
|
|
|
76
78
|
Enable Playwright Utils integration for production-ready fixtures and utilities.
|
|
77
79
|
|
|
78
|
-
**Schema Location:** `src/
|
|
80
|
+
**Schema Location:** `src/module.yaml` (TEA module config)
|
|
79
81
|
|
|
80
82
|
**User Config:** `_bmad/tea/config.yaml`
|
|
81
83
|
|
|
82
84
|
**Type:** `boolean`
|
|
83
85
|
|
|
84
|
-
**Default:** `
|
|
86
|
+
**Default:** `true`
|
|
85
87
|
|
|
86
88
|
**Installer Prompt:**
|
|
87
89
|
|
|
88
90
|
```
|
|
89
|
-
|
|
90
|
-
You must install packages yourself, or use test architect's `framework` command.
|
|
91
|
+
Enable Playwright Utils integration?
|
|
91
92
|
```
|
|
92
93
|
|
|
93
94
|
**Purpose:** Enables TEA to:
|
|
@@ -130,6 +131,130 @@ npm install -D @seontechnologies/playwright-utils
|
|
|
130
131
|
|
|
131
132
|
---
|
|
132
133
|
|
|
134
|
+
### tea_use_pactjs_utils
|
|
135
|
+
|
|
136
|
+
Enable Pact.js Utils integration for production-ready contract testing utilities.
|
|
137
|
+
|
|
138
|
+
**Schema Location:** `src/module.yaml` (TEA module config)
|
|
139
|
+
|
|
140
|
+
**User Config:** `_bmad/tea/config.yaml`
|
|
141
|
+
|
|
142
|
+
**Type:** `boolean`
|
|
143
|
+
|
|
144
|
+
**Default:** `true`
|
|
145
|
+
|
|
146
|
+
**Installer Prompt:**
|
|
147
|
+
|
|
148
|
+
```
|
|
149
|
+
Enable Pact.js Utils integration for contract testing?
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
**Purpose:** Enables TEA to:
|
|
153
|
+
|
|
154
|
+
- Load Pact.js Utils knowledge fragments during context loading
|
|
155
|
+
- Scaffold contract test structure and examples in `framework`
|
|
156
|
+
- Generate contract testing guidance in `atdd` and `automate`
|
|
157
|
+
- Review provider verification patterns in `test-review`
|
|
158
|
+
- Add contract pipeline stage and gates in `ci`
|
|
159
|
+
|
|
160
|
+
**Affects Workflows:**
|
|
161
|
+
|
|
162
|
+
- `framework` - Creates pact test folders and pactjs-utils sample patterns
|
|
163
|
+
- `atdd` - Loads pactjs-utils fragments for contract-aware generation context
|
|
164
|
+
- `automate` - Loads pactjs-utils fragments and passes pact config to subprocesses
|
|
165
|
+
- `test-design` - Loads pactjs-utils fragments for system/epic planning
|
|
166
|
+
- `test-review` - Uses pactjs-utils provider/review patterns
|
|
167
|
+
- `ci` - Adds contract-test stage and quality gates
|
|
168
|
+
|
|
169
|
+
**Example (Enable):**
|
|
170
|
+
|
|
171
|
+
```yaml
|
|
172
|
+
tea_use_pactjs_utils: true
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
**Example (Disable):**
|
|
176
|
+
|
|
177
|
+
```yaml
|
|
178
|
+
tea_use_pactjs_utils: false
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
**Prerequisites:**
|
|
182
|
+
|
|
183
|
+
```bash
|
|
184
|
+
npm install -D @seontechnologies/pactjs-utils @pact-foundation/pact
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
**Related:**
|
|
188
|
+
|
|
189
|
+
- [Pact.js Utils docs](https://seontechnologies.github.io/pactjs-utils/)
|
|
190
|
+
- [TEA Overview - Optional Integrations](/docs/explanation/tea-overview.md#optional-integrations)
|
|
191
|
+
|
|
192
|
+
---
|
|
193
|
+
|
|
194
|
+
### tea_pact_mcp
|
|
195
|
+
|
|
196
|
+
Pact MCP strategy for broker interaction during contract testing workflows.
|
|
197
|
+
|
|
198
|
+
**Schema Location:** `src/module.yaml` (TEA module config)
|
|
199
|
+
|
|
200
|
+
**User Config:** `_bmad/tea/config.yaml`
|
|
201
|
+
|
|
202
|
+
**Type:** `string`
|
|
203
|
+
|
|
204
|
+
**Default:** `"mcp"`
|
|
205
|
+
|
|
206
|
+
**Options:** `"mcp"` | `"none"`
|
|
207
|
+
|
|
208
|
+
**Installer Prompt:**
|
|
209
|
+
|
|
210
|
+
```
|
|
211
|
+
Enable SmartBear MCP for PactFlow/Pact Broker interaction?
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
**Purpose:** Controls whether TEA can use SmartBear MCP tools for:
|
|
215
|
+
|
|
216
|
+
- Provider-state discovery in design/generation workflows
|
|
217
|
+
- Pact test review assistance
|
|
218
|
+
- Can-i-deploy and matrix guidance in CI workflows
|
|
219
|
+
|
|
220
|
+
**Affects Workflows:**
|
|
221
|
+
|
|
222
|
+
- `test-design` - Broker-aware contract context loading
|
|
223
|
+
- `automate` - Broker-assisted pact generation context
|
|
224
|
+
- `test-review` - MCP-assisted pact review context
|
|
225
|
+
- `ci` - MCP can-i-deploy/matrix guidance references
|
|
226
|
+
|
|
227
|
+
**Prerequisites:**
|
|
228
|
+
|
|
229
|
+
```bash
|
|
230
|
+
npm install -g @smartbear/mcp
|
|
231
|
+
# or: npx -y @smartbear/mcp@latest
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
**Required broker env vars (for Pact features):**
|
|
235
|
+
|
|
236
|
+
- `PACT_BROKER_BASE_URL`
|
|
237
|
+
- `PACT_BROKER_TOKEN` (or username/password for basic auth)
|
|
238
|
+
|
|
239
|
+
**Example (Enable MCP):**
|
|
240
|
+
|
|
241
|
+
```yaml
|
|
242
|
+
tea_pact_mcp: 'mcp'
|
|
243
|
+
```
|
|
244
|
+
|
|
245
|
+
**Example (Disable MCP):**
|
|
246
|
+
|
|
247
|
+
```yaml
|
|
248
|
+
tea_pact_mcp: 'none'
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
**Related:**
|
|
252
|
+
|
|
253
|
+
- [Configure Browser Automation Guide](/docs/how-to/customization/configure-browser-automation.md)
|
|
254
|
+
- [SmartBear MCP docs](https://developer.smartbear.com/smartbear-mcp/docs/getting-started)
|
|
255
|
+
|
|
256
|
+
---
|
|
257
|
+
|
|
133
258
|
### tea_browser_automation
|
|
134
259
|
|
|
135
260
|
Browser automation strategy for TEA workflows. Controls how TEA interacts with live browsers during test generation.
|
|
@@ -711,12 +836,15 @@ project_name: my-project
|
|
|
711
836
|
user_skill_level: beginner # or intermediate/expert
|
|
712
837
|
output_folder: _bmad-output
|
|
713
838
|
tea_use_playwright_utils: true # Recommended
|
|
839
|
+
tea_use_pactjs_utils: true # Recommended - production-ready contract testing utilities
|
|
840
|
+
tea_pact_mcp: 'mcp' # Recommended - SmartBear MCP for PactFlow/Broker interaction
|
|
714
841
|
tea_browser_automation: 'auto' # Recommended
|
|
715
842
|
```
|
|
716
843
|
|
|
717
844
|
**Why recommended:**
|
|
718
845
|
|
|
719
846
|
- Playwright Utils: Production-ready fixtures and utilities
|
|
847
|
+
- Pact.js Utils: Optional contract-testing acceleration for backend/fullstack services
|
|
720
848
|
- Browser automation (auto): Smart CLI/MCP selection with fallback
|
|
721
849
|
- Together: The three-part stack (see [Testing as Engineering](/docs/explanation/testing-as-engineering.md))
|
|
722
850
|
|
|
@@ -724,6 +852,9 @@ tea_browser_automation: 'auto' # Recommended
|
|
|
724
852
|
|
|
725
853
|
```bash
|
|
726
854
|
npm install -D @seontechnologies/playwright-utils
|
|
855
|
+
# Optional contract-testing stack:
|
|
856
|
+
# npm install -D @seontechnologies/pactjs-utils @pact-foundation/pact
|
|
857
|
+
# npm install -g @smartbear/mcp
|
|
727
858
|
npm install -g @playwright/cli@latest # For CLI mode
|
|
728
859
|
# Configure MCP servers in IDE (see Configure Browser Automation guide)
|
|
729
860
|
```
|
|
@@ -739,6 +870,8 @@ npm install -g @playwright/cli@latest # For CLI mode
|
|
|
739
870
|
project_name: my-project
|
|
740
871
|
output_folder: _bmad-output
|
|
741
872
|
tea_use_playwright_utils: false
|
|
873
|
+
tea_use_pactjs_utils: false
|
|
874
|
+
tea_pact_mcp: 'none'
|
|
742
875
|
tea_browser_automation: 'none'
|
|
743
876
|
```
|
|
744
877
|
|
|
@@ -761,6 +894,8 @@ tea_browser_automation: 'none'
|
|
|
761
894
|
project_name: monorepo
|
|
762
895
|
output_folder: _bmad-output
|
|
763
896
|
tea_use_playwright_utils: true
|
|
897
|
+
tea_use_pactjs_utils: true
|
|
898
|
+
tea_pact_mcp: 'mcp'
|
|
764
899
|
```
|
|
765
900
|
|
|
766
901
|
**Package configs:**
|
|
@@ -774,6 +909,8 @@ output_folder: ../../_bmad-output/web
|
|
|
774
909
|
project_name: api-service
|
|
775
910
|
output_folder: ../../_bmad-output/api
|
|
776
911
|
tea_use_playwright_utils: false # Using vanilla Playwright only
|
|
912
|
+
tea_use_pactjs_utils: true # API service uses contract testing utilities
|
|
913
|
+
tea_pact_mcp: 'mcp' # Optional broker interaction via MCP
|
|
777
914
|
```
|
|
778
915
|
|
|
779
916
|
---
|
|
@@ -796,6 +933,8 @@ project_knowledge: docs
|
|
|
796
933
|
|
|
797
934
|
# TEA Configuration (Recommended: Enable both for full stack)
|
|
798
935
|
tea_use_playwright_utils: true # Recommended - production-ready utilities
|
|
936
|
+
tea_use_pactjs_utils: true # Recommended - production-ready contract testing utilities
|
|
937
|
+
tea_pact_mcp: 'mcp' # Recommended - SmartBear MCP for broker integration
|
|
799
938
|
tea_browser_automation: 'auto' # Recommended - smart CLI/MCP selection
|
|
800
939
|
|
|
801
940
|
# Languages
|
|
@@ -817,6 +956,12 @@ document_output_language: english
|
|
|
817
956
|
5. (Optional) Enable playwright-utils:
|
|
818
957
|
npm install -D @seontechnologies/playwright-utils
|
|
819
958
|
Set tea_use_playwright_utils: true
|
|
959
|
+
6. (Optional) Enable pactjs-utils:
|
|
960
|
+
npm install -D @seontechnologies/pactjs-utils @pact-foundation/pact
|
|
961
|
+
Set tea_use_pactjs_utils: true
|
|
962
|
+
7. (Optional) Enable Pact MCP:
|
|
963
|
+
npm install -g @smartbear/mcp
|
|
964
|
+
Set tea_pact_mcp: 'mcp'
|
|
820
965
|
```
|
|
821
966
|
|
|
822
967
|
---
|