bmad-method-test-architecture-enterprise 1.3.0 β†’ 1.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,9 +1,9 @@
1
1
  ---
2
2
  title: 'AI-Generated Testing: Why Most Approaches Fail'
3
- description: How Playwright-Utils, TEA workflows, and Playwright MCPs solve AI test quality problems
3
+ description: How Playwright-Utils, pactjs-utils, TEA workflows, Playwright CLI, and MCPs solve AI test quality problems
4
4
  ---
5
5
 
6
- AI-generated tests frequently fail in production because they lack systematic quality standards. This document explains the problem and presents a solution combining three components: Playwright-Utils, TEA (Test Engineering Architect), and Playwright MCPs.
6
+ AI-generated tests frequently fail in production because they lack systematic quality standards. This document explains the problem and presents a solution combining utility standards, TEA workflows, and automation interfaces across both UI, API and contract testing.
7
7
 
8
8
  :::note[Source]
9
9
  This article is adapted from [The Testing Meta Most Teams Have Not Caught Up To Yet](https://dev.to/muratkeremozcan/the-testing-meta-most-teams-have-not-caught-up-to-yet-5765) by Murat K Ozcan.
@@ -30,25 +30,22 @@ AI excels at generating code quickly, but testing requires precision and consist
30
30
 
31
31
  The solution combines three components that work together to enforce quality:
32
32
 
33
- ### Playwright-Utils
33
+ ### 1. Utilities: Playwright-Utils + Pact.js Utils
34
34
 
35
- Bridges the gap between Cypress ergonomics and Playwright's capabilities by standardizing commonly reinvented primitives through utility functions.
35
+ `@seontechnologies/playwright-utils` standardizes commonly reinvented testing primitives across UI, API, web, and non-web flows. `@seontechnologies/pactjs-utils` standardizes Pact.js contract-testing primitives for provider state setup, request filtering, and provider/message verifier configuration.
36
36
 
37
- | Utility | Purpose |
38
- | ---------------------- | --------------------------------- |
39
- | api-request | API calls with schema validation |
40
- | auth-session | Authentication handling |
41
- | intercept-network-call | Network mocking and interception |
42
- | recurse | Retry logic and polling |
43
- | log | Structured logging |
44
- | network-recorder | Record and replay network traffic |
45
- | burn-in | Smart test selection for CI |
46
- | network-error-monitor | HTTP error detection |
47
- | file-utils | CSV/PDF handling |
37
+ | Track | Utility Layer | Purpose |
38
+ | ------------------ | ------------------------------------ | -------------------------------------------------------- |
39
+ | UI/API/Web/Non-web | `@seontechnologies/playwright-utils` | Reusable testing primitives and fixtures |
40
+ | Contract | `@seontechnologies/pactjs-utils` | Reusable Pact consumer/provider helpers and verification |
48
41
 
49
- These utilities eliminate the need to reinvent authentication, API calls, retries, and logging for every project.
42
+ **Playwright-Utils examples:** `api-request`, `auth-session`, `intercept-network-call`, `recurse`, `log`, `network-recorder`, `burn-in`, `network-error-monitor`, `file-utils`.
50
43
 
51
- ### TEA (Test Architect Agent)
44
+ **pactjs-utils examples:** `createProviderState`, `toJsonMap`, `createRequestFilter`, `noOpRequestFilter`, `buildVerifierOptions`, `buildMessageVerifierOptions`.
45
+
46
+ Together, these utility libraries eliminate the need to reinvent core testing primitives across UI, API, web, non-web, and contract testing.
47
+
48
+ ### 2. Process: TEA (Test Architect Agent)
52
49
 
53
50
  A quality operating model packaged as eight executable workflows spanning test design, CI/CD gates, and release readiness. TEA encodes test architecture expertise into repeatable processes.
54
51
 
@@ -67,28 +64,35 @@ A quality operating model packaged as eight executable workflows spanning test d
67
64
  TEA doesn't just generate testsβ€”it provides a complete quality operating model with workflows for planning, execution, and release gates.
68
65
  :::
69
66
 
70
- ### Playwright MCPs
67
+ ### 3. Automation Interfaces: Playwright CLI + MCPs
68
+
69
+ Automation interfaces enable real-time verification during test generation and review across browser and contract tracks:
70
+
71
+ - **Playwright CLI**: token-efficient browser automation for stateless execution and fast checks in workflows.
72
+ - **Playwright MCP**: stateful browser automation with richer context for interactive exploration and DOM validation.
73
+ - **Pact MCP**: broker-aware contract automation for verification matrix queries, provider-state discovery, compatibility analysis, and `can-i-deploy` deployment decisions.
71
74
 
72
- Model Context Protocols enable real-time verification during test generation. Instead of inferring selectors and behavior from documentation, MCPs allow agents to:
75
+ Instead of inferring behavior from documentation alone, these interfaces allow agents to:
73
76
 
74
- - Run flows and confirm the DOM against the accessibility tree
75
- - Validate network responses in real-time
76
- - Discover actual functionality through interactive exploration
77
- - Verify generated tests against live applications
77
+ - Run browser flows and confirm the DOM against the accessibility tree
78
+ - Validate UI/API network behavior in real-time
79
+ - Query Pact verification matrix results across consumer/provider versions
80
+ - Check provider states and contract compatibility before release
81
+ - Execute `can-i-deploy` checks against target environments
78
82
 
79
83
  ## How They Work Together
80
84
 
81
85
  The three components form a quality pipeline:
82
86
 
83
- | Stage | Component | Action |
84
- | ------------ | ---------------- | --------------------------------------------------- |
85
- | Standards | Playwright-Utils | Provides production-ready patterns and utilities |
86
- | Process | TEA Workflows | Enforces systematic test planning and review |
87
- | Verification | Playwright MCPs | Validates generated tests against live applications |
87
+ | Stage | Component | Action |
88
+ | ------------ | ------------------------------------------ | ------------------------------------------------------------ |
89
+ | Standards | Playwright-Utils + pactjs-utils | Provides production-ready patterns for UI and contract tests |
90
+ | Process | TEA Workflows | Enforces systematic test planning and review |
91
+ | Verification | Playwright CLI + Playwright MCP + Pact MCP | Validates tests and contracts against live systems |
88
92
 
89
93
  **Before (AI-only):** 20 tests with redundant coverage, incorrect assertions, and flaky behavior.
90
94
 
91
- **After (Full Stack):** Risk-based selection, verified selectors, validated behavior, reviewable code.
95
+ **After (Full Stack):** Risk-based selection, verified selectors, validated behavior, contract compatibility checks, reviewable code.
92
96
 
93
97
  ## Why This Matters
94
98
 
@@ -101,11 +105,11 @@ Traditional AI testing approaches fail because they:
101
105
 
102
106
  The three-part stack addresses each gap:
103
107
 
104
- | Gap | Solution |
105
- | --------------- | --------------------------------------------------- |
106
- | No standards | Playwright-Utils provides production-ready patterns |
107
- | No planning | TEA `test-design` creates risk-based test plans |
108
- | No verification | Playwright MCPs validate against live applications |
109
- | No review | TEA `test-review` audits quality with scoring |
108
+ | Gap | Solution |
109
+ | --------------- | ------------------------------------------------------------------------ |
110
+ | No standards | Playwright-Utils + pactjs-utils provide production-ready patterns |
111
+ | No planning | TEA `test-design` creates risk-based test plans |
112
+ | No verification | Playwright CLI + Playwright MCP + Pact MCP validate against live systems |
113
+ | No review | TEA `test-review` audits quality with scoring |
110
114
 
111
115
  This approach is sometimes called _context engineering_β€”loading domain-specific standards into AI context automatically rather than relying on prompts alone. TEA's `tea-index.csv` manifest loads relevant knowledge fragments so the AI doesn't relearn testing patterns each session.
@@ -62,7 +62,7 @@ Add these MCP server entries to your tool's configuration file:
62
62
  }
63
63
  ```
64
64
 
65
- The `smartbear` server is optional β€” only needed if you use the [Pact MCP integration](/docs/how-to/customization/configure-browser-automation.md#related) for contract testing workflows. See the [pact-mcp knowledge fragment](../../src/testarch/knowledge/pact-mcp.md) for details.
65
+ The `smartbear` server is optional β€” only needed if you use the [Pact MCP integration](/docs/reference/configuration.md#tea_pact_mcp) for contract testing workflows. See the [pact-mcp knowledge fragment](/docs/reference/knowledge-base.md#pact-contract-testing-integration) for details.
66
66
 
67
67
  #### Where to put the config
68
68
 
@@ -141,7 +141,7 @@ Enable Pact.js Utils integration for production-ready contract testing utilities
141
141
 
142
142
  **Type:** `boolean`
143
143
 
144
- **Default:** `false`
144
+ **Default:** `true`
145
145
 
146
146
  **Installer Prompt:**
147
147
 
@@ -201,7 +201,7 @@ Pact MCP strategy for broker interaction during contract testing workflows.
201
201
 
202
202
  **Type:** `string`
203
203
 
204
- **Default:** `"none"`
204
+ **Default:** `"mcp"`
205
205
 
206
206
  **Options:** `"mcp"` | `"none"`
207
207
 
@@ -593,7 +593,7 @@ If the BMAD installer can run but cannot fetch the Test Architect module from Gi
593
593
 
594
594
  **Causes**:
595
595
 
596
- - All 40 fragments loading at once
596
+ - Up to 40 fragments loading at once (depends on workflow and enabled integrations)
597
597
  - Large fragment file sizes
598
598
  - Disk I/O bottleneck
599
599
 
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "$schema": "https://json.schemastore.org/package.json",
3
3
  "name": "bmad-method-test-architecture-enterprise",
4
- "version": "1.3.0",
4
+ "version": "1.3.1",
5
5
  "description": "Master Test Architect for quality strategy, test automation, and release gates",
6
6
  "keywords": [
7
7
  "bmad",
package/release_notes.md CHANGED
@@ -1,10 +1,13 @@
1
- ## πŸš€ What's New in v1.3.0
2
-
3
- ### ✨ New Features
4
- - feat: pactjs utils support
1
+ ## πŸš€ What's New in v1.3.1
5
2
 
6
3
  ### πŸ“¦ Other Changes
7
- - Merge pull request #41 from bmad-code-org/feat/pactjs-utils-support
4
+ - docs: update pactjs utils
5
+ - addressed review commments
6
+ - Merge pull request #42 from bmad-code-org/docs/update-pactjs-utils
7
+ - Merge pull request #43 from bmad-code-org/chore/refresh-lock
8
+
9
+ ### πŸ”§ Maintenance
10
+ - chore: refresh lock
8
11
 
9
12
 
10
13
  ## πŸ“¦ Installation
@@ -14,4 +17,4 @@ npx bmad-method install
14
17
  # Select "Test Architect" from module menu
15
18
  ```
16
19
 
17
- **Full Changelog**: https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/compare/v1.2.6...v1.3.0
20
+ **Full Changelog**: https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/compare/v1.3.0...v1.3.1
@@ -154,6 +154,7 @@ await provider
154
154
  - **Null handling**: `null` becomes `"null"` string in JsonMap (Pact requirement)
155
155
  - **Date handling**: Date objects become ISO 8601 strings
156
156
  - **No nested objects in JsonMap**: Nested objects are JSON-stringified β€” provider state handlers must parse them
157
+ - **Array serialization is lossy**: Arrays are converted via `String()` (e.g., `[1,2,3]` β†’ `"1,2,3"`) β€” prefer passing arrays as JSON-stringified objects for round-trip safety
157
158
  - **Message pacts**: Works identically with `MessageConsumerPact` β€” same `.given()` API
158
159
 
159
160
  ## Related Fragments
@@ -265,8 +265,8 @@ const opts: VerifierOptions = {
265
265
  providerVersionBranch: process.env.GITHUB_HEAD_REF || process.env.GITHUB_REF_NAME,
266
266
  consumerVersionSelectors:
267
267
  process.env.PACT_BREAKING_CHANGE === 'true'
268
- ? [{ mainBranch: true }, { deployedOrReleased: true }]
269
- : [{ mainBranch: true }, { deployedOrReleased: true }, { matchingBranch: true }],
268
+ ? [{ matchingBranch: true }]
269
+ : [{ matchingBranch: true }, { mainBranch: true }, { deployedOrReleased: true }],
270
270
  pactUrls: process.env.PACT_PAYLOAD_URL ? [process.env.PACT_PAYLOAD_URL] : undefined,
271
271
  stateHandlers: {
272
272
  /* ... */
@@ -198,20 +198,27 @@ const opts: VerifierOptions = {
198
198
  ### Right: Separate auth into createRequestFilter
199
199
 
200
200
  ```typescript
201
- // βœ… Clean separation
202
- const requestFilter = createRequestFilter({
203
- tokenGenerator: () => fetchAuthToken(),
204
- });
201
+ // βœ… Clean separation β€” async setup wraps token fetch (CommonJS-safe)
202
+ async function setupVerifierOptions() {
203
+ const token = await fetchAuthToken(); // Resolve async token BEFORE creating filter
205
204
 
206
- const opts = buildVerifierOptions({
207
- provider: 'my-api',
208
- port: '3001',
209
- includeMainAndDeployed: true,
210
- requestFilter,
211
- stateHandlers: {
212
- /* ... */
213
- },
214
- });
205
+ const requestFilter = createRequestFilter({
206
+ tokenGenerator: () => token, // Synchronous β€” returns pre-fetched value
207
+ });
208
+
209
+ return buildVerifierOptions({
210
+ provider: 'my-api',
211
+ port: '3001',
212
+ includeMainAndDeployed: true,
213
+ requestFilter,
214
+ stateHandlers: {
215
+ /* ... */
216
+ },
217
+ });
218
+ }
219
+
220
+ // In tests/hooks, callers can await setupVerifierOptions():
221
+ // const opts = await setupVerifierOptions();
215
222
  ```
216
223
 
217
224
  _Source: @seontechnologies/pactjs-utils request-filter module, pact-js-example-provider verification tests_
@@ -59,7 +59,7 @@ Create the idiomatic test directory for the detected language:
59
59
  - **Ruby (RSpec)**: `spec/` with `spec/unit/`, `spec/integration/`, `spec/api/`, `spec/support/`
60
60
  - **Rust**: `tests/` for integration tests, inline `#[cfg(test)]` modules for unit tests
61
61
 
62
- **If `tea_use_pactjs_utils` is enabled** (and `{detected_stack}` is `backend` or `fullstack`):
62
+ **If `config.tea_use_pactjs_utils` is enabled** (and `{detected_stack}` is `backend` or `fullstack`):
63
63
 
64
64
  Create contract testing directory structure:
65
65
 
@@ -180,7 +180,7 @@ Create helpers for:
180
180
  - Auth helpers
181
181
  - Test data factories (language-idiomatic patterns)
182
182
 
183
- **If `tea_use_pactjs_utils` is enabled** (and `{detected_stack}` is `backend` or `fullstack`):
183
+ **If `config.tea_use_pactjs_utils` is enabled** (and `{detected_stack}` is `backend` or `fullstack`):
184
184
 
185
185
  Create contract test samples in `pact/` directory:
186
186
 
@@ -279,7 +279,7 @@ function applyFixes(content, issues) {
279
279
  if (issue.status === 'auto-fixable' && issue.suggestedFix) {
280
280
  const oldLink = `[${issue.linkText}](${issue.href})`;
281
281
  const newLink = `[${issue.linkText}](${issue.suggestedFix})`;
282
- updated = updated.replace(oldLink, newLink);
282
+ updated = updated.replaceAll(oldLink, newLink);
283
283
  }
284
284
  }
285
285
 
@@ -1,23 +0,0 @@
1
- ---
2
- title: 'Enable TEA MCP Enhancements (Deprecated)'
3
- description: This guide has been superseded by Configure Browser Automation
4
- ---
5
-
6
- # Enable TEA MCP Enhancements
7
-
8
- > **Deprecated:** This guide has been replaced by [Configure Browser Automation](/docs/how-to/customization/configure-browser-automation.md).
9
-
10
- The `tea_use_mcp_enhancements` boolean flag has been replaced by `tea_browser_automation`, which supports four modes: `auto`, `cli`, `mcp`, and `none`.
11
-
12
- **Migration:**
13
-
14
- | Old Setting | New Equivalent |
15
- | --------------------------------- | -------------------------------- |
16
- | `tea_use_mcp_enhancements: true` | `tea_browser_automation: "auto"` |
17
- | `tea_use_mcp_enhancements: false` | `tea_browser_automation: "none"` |
18
-
19
- Please see [Configure Browser Automation](/docs/how-to/customization/configure-browser-automation.md) for full setup instructions.
20
-
21
- ---
22
-
23
- Generated with [BMad Method](https://bmad-method.org) - TEA (Test Engineering Architect)