bmad-method-test-architecture-enterprise 1.0.1 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/release_notes.md +5 -8
- package/src/module-help.csv +3 -3
- package/src/module.yaml +25 -0
- package/src/testarch/knowledge/api-request.md +121 -0
- package/src/testarch/knowledge/api-testing-patterns.md +64 -0
- package/src/testarch/knowledge/overview.md +11 -11
- package/src/testarch/tea-index.csv +1 -1
- package/src/workflows/testarch/atdd/atdd-checklist-template.md +6 -0
- package/src/workflows/testarch/atdd/instructions.md +7 -0
- package/src/workflows/testarch/atdd/steps-c/step-01-preflight-and-context.md +25 -0
- package/src/workflows/testarch/atdd/steps-c/step-01b-resume.md +96 -0
- package/src/workflows/testarch/atdd/steps-c/step-02-generation-mode.md +25 -0
- package/src/workflows/testarch/atdd/steps-c/step-03-test-strategy.md +25 -0
- package/src/workflows/testarch/atdd/steps-c/step-04c-aggregate.md +25 -0
- package/src/workflows/testarch/atdd/steps-c/step-05-validate-and-complete.md +25 -0
- package/src/workflows/testarch/atdd/workflow.md +2 -0
- package/src/workflows/testarch/automate/instructions.md +7 -0
- package/src/workflows/testarch/automate/steps-c/step-01-preflight-and-context.md +25 -0
- package/src/workflows/testarch/automate/steps-c/step-01b-resume.md +94 -0
- package/src/workflows/testarch/automate/steps-c/step-02-identify-targets.md +25 -0
- package/src/workflows/testarch/automate/steps-c/step-03c-aggregate.md +25 -0
- package/src/workflows/testarch/automate/steps-c/step-04-validate-and-summarize.md +24 -0
- package/src/workflows/testarch/automate/workflow.md +2 -0
- package/src/workflows/testarch/ci/instructions.md +7 -0
- package/src/workflows/testarch/ci/steps-c/step-01-preflight.md +25 -0
- package/src/workflows/testarch/ci/steps-c/step-01b-resume.md +110 -0
- package/src/workflows/testarch/ci/steps-c/step-02-generate-pipeline.md +27 -2
- package/src/workflows/testarch/ci/steps-c/step-03-configure-quality-gates.md +25 -0
- package/src/workflows/testarch/ci/steps-c/step-04-validate-and-summary.md +25 -0
- package/src/workflows/testarch/ci/workflow.md +2 -0
- package/src/workflows/testarch/framework/instructions.md +7 -0
- package/src/workflows/testarch/framework/steps-c/step-01-preflight.md +25 -0
- package/src/workflows/testarch/framework/steps-c/step-01b-resume.md +116 -0
- package/src/workflows/testarch/framework/steps-c/step-02-select-framework.md +25 -0
- package/src/workflows/testarch/framework/steps-c/step-03-scaffold-framework.md +25 -0
- package/src/workflows/testarch/framework/steps-c/step-04-docs-and-scripts.md +23 -0
- package/src/workflows/testarch/framework/steps-c/step-05-validate-and-summary.md +25 -0
- package/src/workflows/testarch/framework/workflow.md +2 -0
- package/src/workflows/testarch/nfr-assess/instructions.md +7 -0
- package/src/workflows/testarch/nfr-assess/nfr-report-template.md +6 -0
- package/src/workflows/testarch/nfr-assess/steps-c/step-01-load-context.md +25 -0
- package/src/workflows/testarch/nfr-assess/steps-c/step-01b-resume.md +106 -0
- package/src/workflows/testarch/nfr-assess/steps-c/step-02-define-thresholds.md +25 -0
- package/src/workflows/testarch/nfr-assess/steps-c/step-03-gather-evidence.md +25 -0
- package/src/workflows/testarch/nfr-assess/steps-c/step-04e-aggregate-nfr.md +28 -0
- package/src/workflows/testarch/nfr-assess/steps-c/step-05-generate-report.md +25 -1
- package/src/workflows/testarch/nfr-assess/workflow.md +2 -0
- package/src/workflows/testarch/test-design/instructions.md +7 -0
- package/src/workflows/testarch/test-design/steps-c/step-01-detect-mode.md +25 -0
- package/src/workflows/testarch/test-design/steps-c/step-01b-resume.md +102 -0
- package/src/workflows/testarch/test-design/steps-c/step-02-load-context.md +25 -0
- package/src/workflows/testarch/test-design/steps-c/step-03-risk-and-testability.md +25 -0
- package/src/workflows/testarch/test-design/steps-c/step-04-coverage-plan.md +25 -0
- package/src/workflows/testarch/test-design/steps-c/step-05-generate-output.md +25 -0
- package/src/workflows/testarch/test-design/test-design-architecture-template.md +6 -0
- package/src/workflows/testarch/test-design/test-design-qa-template.md +6 -0
- package/src/workflows/testarch/test-design/test-design-template.md +6 -0
- package/src/workflows/testarch/test-design/workflow.md +2 -0
- package/src/workflows/testarch/test-review/instructions.md +7 -0
- package/src/workflows/testarch/test-review/steps-c/step-01-load-context.md +25 -0
- package/src/workflows/testarch/test-review/steps-c/step-01b-resume.md +104 -0
- package/src/workflows/testarch/test-review/steps-c/step-02-discover-tests.md +25 -0
- package/src/workflows/testarch/test-review/steps-c/step-03f-aggregate-scores.md +28 -0
- package/src/workflows/testarch/test-review/steps-c/step-04-generate-report.md +25 -1
- package/src/workflows/testarch/test-review/test-review-template.md +6 -0
- package/src/workflows/testarch/test-review/workflow.md +2 -0
- package/src/workflows/testarch/trace/instructions.md +7 -0
- package/src/workflows/testarch/trace/steps-c/step-01-load-context.md +25 -0
- package/src/workflows/testarch/trace/steps-c/step-01b-resume.md +102 -0
- package/src/workflows/testarch/trace/steps-c/step-02-discover-tests.md +25 -0
- package/src/workflows/testarch/trace/steps-c/step-03-map-criteria.md +25 -0
- package/src/workflows/testarch/trace/steps-c/step-04-analyze-gaps.md +27 -2
- package/src/workflows/testarch/trace/steps-c/step-05-gate-decision.md +14 -0
- package/src/workflows/testarch/trace/trace-template.md +6 -0
- package/src/workflows/testarch/trace/workflow.md +2 -0
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "https://json.schemastore.org/package.json",
|
|
3
3
|
"name": "bmad-method-test-architecture-enterprise",
|
|
4
|
-
"version": "1.0
|
|
4
|
+
"version": "1.2.0",
|
|
5
5
|
"description": "Master Test Architect for quality strategy, test automation, and release gates",
|
|
6
6
|
"keywords": [
|
|
7
7
|
"bmad",
|
package/release_notes.md
CHANGED
|
@@ -1,13 +1,10 @@
|
|
|
1
|
-
## 🚀 What's New in v1.0
|
|
1
|
+
## 🚀 What's New in v1.2.0
|
|
2
2
|
|
|
3
|
-
###
|
|
4
|
-
-
|
|
5
|
-
- fix: CI failures
|
|
3
|
+
### ✨ New Features
|
|
4
|
+
- feat:enhance api-request with operation-based overload and update documentation
|
|
6
5
|
|
|
7
6
|
### 📦 Other Changes
|
|
8
|
-
-
|
|
9
|
-
- re-run checks
|
|
10
|
-
- Merge pull request #24 from bmad-code-org/fix/playwright-cli-docs-alignment
|
|
7
|
+
- Merge pull request #28 from bmad-code-org/feat/knowledge-update-pw-utils-3-14-0
|
|
11
8
|
|
|
12
9
|
|
|
13
10
|
## 📦 Installation
|
|
@@ -17,4 +14,4 @@ npx bmad-method install
|
|
|
17
14
|
# Select "Test Architect" from module menu
|
|
18
15
|
```
|
|
19
16
|
|
|
20
|
-
**Full Changelog**: https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/compare/v1.
|
|
17
|
+
**Full Changelog**: https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/compare/v1.1.0...v1.2.0
|
package/src/module-help.csv
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
module,phase,name,code,sequence,workflow-file,command,required,agent,options,description,output-location,outputs,
|
|
2
2
|
tea,0-learning,Teach Me Testing,TMT,10,_bmad/tea/workflows/testarch/teach-me-testing/workflow.md,bmad_tea_teach-me-testing,false,tea,Create Mode,"Teach testing fundamentals through 7 sessions (TEA Academy)",test_artifacts,"progress file|session notes|certificate",
|
|
3
|
-
tea,3-solutioning,Test
|
|
4
|
-
tea,3-solutioning,
|
|
5
|
-
tea,3-solutioning,
|
|
3
|
+
tea,3-solutioning,Test Design,TD,10,_bmad/tea/workflows/testarch/test-design/workflow.yaml,bmad_tea_test-design,false,tea,Create Mode,"Risk-based test planning",test_artifacts,"test design document",
|
|
4
|
+
tea,3-solutioning,Test Framework,TF,20,_bmad/tea/workflows/testarch/framework/workflow.yaml,bmad_tea_framework,false,tea,Create Mode,"Initialize production-ready test framework",test_artifacts,"framework scaffold",
|
|
5
|
+
tea,3-solutioning,CI Setup,CI,30,_bmad/tea/workflows/testarch/ci/workflow.yaml,bmad_tea_ci,false,tea,Create Mode,"Configure CI/CD quality pipeline",test_artifacts,"ci config",
|
|
6
6
|
tea,4-implementation,ATDD,AT,10,_bmad/tea/workflows/testarch/atdd/workflow.yaml,bmad_tea_atdd,false,tea,Create Mode,"Generate failing tests (TDD red phase)",test_artifacts,"atdd tests",
|
|
7
7
|
tea,4-implementation,Test Automation,TA,20,_bmad/tea/workflows/testarch/automate/workflow.yaml,bmad_tea_automate,false,tea,Create Mode,"Expand test coverage",test_artifacts,"test suite",
|
|
8
8
|
tea,4-implementation,Test Review,RV,30,_bmad/tea/workflows/testarch/test-review/workflow.yaml,bmad_tea_test-review,false,tea,Validate Mode,"Quality audit (0-100 scoring)",test_artifacts,"review report",
|
package/src/module.yaml
CHANGED
|
@@ -95,6 +95,31 @@ trace_output:
|
|
|
95
95
|
default: "traceability"
|
|
96
96
|
result: "{test_artifacts}/{value}"
|
|
97
97
|
|
|
98
|
+
post-install-notes:
|
|
99
|
+
tea_browser_automation:
|
|
100
|
+
cli: |
|
|
101
|
+
Playwright CLI Setup:
|
|
102
|
+
npm install -g @playwright/cli@latest
|
|
103
|
+
playwright-cli install --skills # Run from project root
|
|
104
|
+
Node.js 18+ required.
|
|
105
|
+
mcp: |
|
|
106
|
+
Playwright MCP Setup (two servers):
|
|
107
|
+
1. playwright — npx @playwright/mcp@latest
|
|
108
|
+
2. playwright-test — npx playwright run-test-mcp-server
|
|
109
|
+
Configure both MCP servers in your IDE settings.
|
|
110
|
+
See: https://github.com/microsoft/playwright-mcp
|
|
111
|
+
auto: |
|
|
112
|
+
Playwright CLI Setup:
|
|
113
|
+
npm install -g @playwright/cli@latest
|
|
114
|
+
playwright-cli install --skills # Run from project root
|
|
115
|
+
Node.js 18+ required.
|
|
116
|
+
|
|
117
|
+
Playwright MCP Setup (two servers):
|
|
118
|
+
1. playwright — npx @playwright/mcp@latest
|
|
119
|
+
2. playwright-test — npx playwright run-test-mcp-server
|
|
120
|
+
Configure both MCP servers in your IDE settings.
|
|
121
|
+
See: https://github.com/microsoft/playwright-mcp
|
|
122
|
+
|
|
98
123
|
# Directories to create during installation (declarative, no code execution)
|
|
99
124
|
directories:
|
|
100
125
|
- "{test_artifacts}"
|
|
@@ -371,6 +371,95 @@ test.describe('GraphQL API', () => {
|
|
|
371
371
|
- Check `body.errors` for GraphQL errors (not status code)
|
|
372
372
|
- Works for queries and mutations
|
|
373
373
|
|
|
374
|
+
### Example 8: Operation-Based Overload (OpenAPI / Code Generators)
|
|
375
|
+
|
|
376
|
+
**Context**: When using a code generator (orval, openapi-generator, custom scripts) that produces typed operation definitions from an OpenAPI spec, pass the operation object directly to `apiRequest`. This eliminates manual `method`/`path` extraction and `typeof` assertions while preserving full type inference for request body, response, and query parameters. Available since v3.14.0.
|
|
377
|
+
|
|
378
|
+
**Implementation**:
|
|
379
|
+
|
|
380
|
+
```typescript
|
|
381
|
+
// Generated operation definition — structural typing, no import from playwright-utils needed
|
|
382
|
+
// type OperationShape = { path: string; method: 'POST'|'GET'|'PUT'|'DELETE'|'PATCH'|'HEAD'; response: unknown; request: unknown; query?: unknown }
|
|
383
|
+
|
|
384
|
+
import { test, expect } from '@seontechnologies/playwright-utils/api-request/fixtures';
|
|
385
|
+
|
|
386
|
+
// --- Basic usage: operation replaces method + path ---
|
|
387
|
+
test('should upsert person via operation overload', async ({ apiRequest }) => {
|
|
388
|
+
const { status, body } = await apiRequest({
|
|
389
|
+
operation: upsertPersonv2({ customerId }),
|
|
390
|
+
headers: getHeaders(customerId),
|
|
391
|
+
body: personInput, // compile-time typed as Schemas.PersonInput
|
|
392
|
+
});
|
|
393
|
+
|
|
394
|
+
expect(status).toBe(200);
|
|
395
|
+
expect(body.id).toBeDefined(); // body typed as Schemas.Person
|
|
396
|
+
});
|
|
397
|
+
|
|
398
|
+
// --- Typed query parameters (replaces string concatenation) ---
|
|
399
|
+
test('should list people with typed query', async ({ apiRequest }) => {
|
|
400
|
+
const { body } = await apiRequest({
|
|
401
|
+
operation: getPeoplev2({ customerId }),
|
|
402
|
+
headers: getHeaders(customerId),
|
|
403
|
+
query: { page: 0, page_size: 5 }, // typed from operation's query definition
|
|
404
|
+
});
|
|
405
|
+
|
|
406
|
+
expect(body.items).toHaveLength(5);
|
|
407
|
+
});
|
|
408
|
+
|
|
409
|
+
// --- Params escape hatch (pre-formatted query strings) ---
|
|
410
|
+
test('should fetch billing history with raw params', async ({ apiRequest }) => {
|
|
411
|
+
const { body } = await apiRequest({
|
|
412
|
+
operation: getBillingHistoryv2({ customerId }),
|
|
413
|
+
headers: getHeaders(customerId),
|
|
414
|
+
params: {
|
|
415
|
+
'filters[start_date]': getThisMonthTimestamp(),
|
|
416
|
+
'filters[date_type]': 'MONTH',
|
|
417
|
+
},
|
|
418
|
+
});
|
|
419
|
+
|
|
420
|
+
expect(body.entries.length).toBeGreaterThan(0);
|
|
421
|
+
});
|
|
422
|
+
|
|
423
|
+
// --- Works with recurse (polling) ---
|
|
424
|
+
test('should poll until person is reviewed', async ({ apiRequest, recurse }) => {
|
|
425
|
+
await recurse(
|
|
426
|
+
async () =>
|
|
427
|
+
apiRequest({
|
|
428
|
+
operation: getPersonv2({ customerId, hash }),
|
|
429
|
+
headers: getHeaders(customerId),
|
|
430
|
+
}),
|
|
431
|
+
(res) => {
|
|
432
|
+
expect(res.status).toBe(200);
|
|
433
|
+
expect(res.body.status).toBe('REVIEWED');
|
|
434
|
+
},
|
|
435
|
+
{ timeout: 30000, interval: 1000 },
|
|
436
|
+
);
|
|
437
|
+
});
|
|
438
|
+
|
|
439
|
+
// --- Schema validation chains work identically ---
|
|
440
|
+
test('should create movie with schema validation', async ({ apiRequest }) => {
|
|
441
|
+
const { body } = await apiRequest({
|
|
442
|
+
operation: createMovieOp,
|
|
443
|
+
headers: commonHeaders(authToken),
|
|
444
|
+
body: movie,
|
|
445
|
+
}).validateSchema(CreateMovieResponseSchema, {
|
|
446
|
+
shape: { status: 200, data: { name: movie.name } },
|
|
447
|
+
});
|
|
448
|
+
|
|
449
|
+
expect(body.data.id).toBeDefined();
|
|
450
|
+
});
|
|
451
|
+
```
|
|
452
|
+
|
|
453
|
+
**Key Points**:
|
|
454
|
+
|
|
455
|
+
- Pass `operation` instead of `method` + `path` — mutually exclusive at compile time
|
|
456
|
+
- Response body, request body, and query types inferred from operation definition
|
|
457
|
+
- Uses structural typing (duck typing) — works with any code generator producing `{ path, method, response, request, query? }`
|
|
458
|
+
- `query` field auto-serializes to bracket notation (`filters[type]=pep`, `ids[0]=10`)
|
|
459
|
+
- `params` escape hatch for pre-formatted strings — wins over `query` on conflict
|
|
460
|
+
- Fully composable with `recurse`, `validateSchema`, and all existing features
|
|
461
|
+
- `response`/`request`/`query` on the operation are type-level only — runtime never reads their values
|
|
462
|
+
|
|
374
463
|
## Comparison with Vanilla Playwright
|
|
375
464
|
|
|
376
465
|
| Vanilla Playwright | playwright-utils apiRequest |
|
|
@@ -393,6 +482,7 @@ test.describe('GraphQL API', () => {
|
|
|
393
482
|
- ✅ Tests requiring retry logic
|
|
394
483
|
- ✅ Background API calls in UI tests
|
|
395
484
|
- ✅ Contract testing support
|
|
485
|
+
- ✅ Type-safe API testing with OpenAPI-generated operations (v3.14.0+)
|
|
396
486
|
|
|
397
487
|
**Stick with vanilla Playwright for:**
|
|
398
488
|
|
|
@@ -440,3 +530,34 @@ const response: any = await apiRequest({ method: 'GET', path: '/users' });
|
|
|
440
530
|
const { body } = await apiRequest<User[]>({ method: 'GET', path: '/users' });
|
|
441
531
|
// body is typed as User[]
|
|
442
532
|
```
|
|
533
|
+
|
|
534
|
+
**❌ Mixing operation overload with explicit generics:**
|
|
535
|
+
|
|
536
|
+
```typescript
|
|
537
|
+
// Don't pass a generic when using operation — types are inferred from the operation
|
|
538
|
+
const { body } = await apiRequest<MyType>({
|
|
539
|
+
operation: getPersonv2({ customerId }),
|
|
540
|
+
headers: getHeaders(customerId),
|
|
541
|
+
});
|
|
542
|
+
```
|
|
543
|
+
|
|
544
|
+
**✅ Let the operation infer the types:**
|
|
545
|
+
|
|
546
|
+
```typescript
|
|
547
|
+
const { body } = await apiRequest({
|
|
548
|
+
operation: getPersonv2({ customerId }),
|
|
549
|
+
headers: getHeaders(customerId),
|
|
550
|
+
});
|
|
551
|
+
// body type inferred from operation.response
|
|
552
|
+
```
|
|
553
|
+
|
|
554
|
+
**❌ Mixing operation with method/path:**
|
|
555
|
+
|
|
556
|
+
```typescript
|
|
557
|
+
// Compile error — operation and method/path are mutually exclusive
|
|
558
|
+
await apiRequest({
|
|
559
|
+
operation: getPersonv2({ customerId }),
|
|
560
|
+
method: 'GET', // Error: method?: never
|
|
561
|
+
path: '/api/person', // Error: path?: never
|
|
562
|
+
});
|
|
563
|
+
```
|
|
@@ -197,6 +197,7 @@ test.describe('Orders API', () => {
|
|
|
197
197
|
- `validateSchema` throws if response doesn't match
|
|
198
198
|
- Built-in retry for transient failures
|
|
199
199
|
- Type-safe `body` access
|
|
200
|
+
- **Note**: If your project uses code-generated operations from an OpenAPI spec, see [Example 8](#example-8-operation-based-api-testing-openapi--code-generators) for the preferred `operation`-based overload (v3.14.0+)
|
|
200
201
|
|
|
201
202
|
### Example 3: Microservice-to-Microservice Testing
|
|
202
203
|
|
|
@@ -713,6 +714,69 @@ test.describe('Authenticated API Tests', () => {
|
|
|
713
714
|
- Test auth, expired tokens, and RBAC
|
|
714
715
|
- Pure API testing without UI
|
|
715
716
|
|
|
717
|
+
### Example 8: Operation-Based API Testing (OpenAPI / Code Generators)
|
|
718
|
+
|
|
719
|
+
**Context**: When your project uses code-generated operation definitions from an OpenAPI spec, leverage the operation-based overload of `apiRequest` (v3.14.0+) instead of manual `method`/`path` extraction. This eliminates `typeof` assertions and provides full type inference for request body, response, and query parameters.
|
|
720
|
+
|
|
721
|
+
**Implementation**:
|
|
722
|
+
|
|
723
|
+
```typescript
|
|
724
|
+
// tests/api/operations.spec.ts
|
|
725
|
+
import { test, expect } from '@seontechnologies/playwright-utils/api-request/fixtures';
|
|
726
|
+
|
|
727
|
+
test.describe('API Tests with Generated Operations', () => {
|
|
728
|
+
test('should create entity with full type safety', async ({ apiRequest }) => {
|
|
729
|
+
// Operation object from code generator — contains path, method, and type info
|
|
730
|
+
const { status, body } = await apiRequest({
|
|
731
|
+
operation: createEntityOp({ workspaceId }),
|
|
732
|
+
headers: getHeaders(workspaceId),
|
|
733
|
+
body: entityInput, // Compile-time typed from operation.request
|
|
734
|
+
});
|
|
735
|
+
|
|
736
|
+
expect(status).toBe(201);
|
|
737
|
+
expect(body.id).toBeDefined(); // body typed from operation.response
|
|
738
|
+
});
|
|
739
|
+
|
|
740
|
+
test('should list with typed query parameters', async ({ apiRequest }) => {
|
|
741
|
+
// query field replaces manual string concatenation
|
|
742
|
+
const { body } = await apiRequest({
|
|
743
|
+
operation: listEntitiesOp({ workspaceId }),
|
|
744
|
+
headers: getHeaders(workspaceId),
|
|
745
|
+
query: { page: 0, page_size: 10, status: 'active' },
|
|
746
|
+
});
|
|
747
|
+
|
|
748
|
+
expect(body.items).toHaveLength(10);
|
|
749
|
+
expect(body.total).toBeGreaterThan(10);
|
|
750
|
+
});
|
|
751
|
+
|
|
752
|
+
test('should poll async operation until complete', async ({ apiRequest, recurse }) => {
|
|
753
|
+
const { body: job } = await apiRequest({
|
|
754
|
+
operation: startJobOp({ workspaceId }),
|
|
755
|
+
headers: getHeaders(workspaceId),
|
|
756
|
+
body: { type: 'export' },
|
|
757
|
+
});
|
|
758
|
+
|
|
759
|
+
await recurse(
|
|
760
|
+
async () =>
|
|
761
|
+
apiRequest({
|
|
762
|
+
operation: getJobOp({ workspaceId, jobId: job.id }),
|
|
763
|
+
headers: getHeaders(workspaceId),
|
|
764
|
+
}),
|
|
765
|
+
(res) => res.body.status === 'completed',
|
|
766
|
+
{ timeout: 60000, interval: 2000 },
|
|
767
|
+
);
|
|
768
|
+
});
|
|
769
|
+
});
|
|
770
|
+
```
|
|
771
|
+
|
|
772
|
+
**Key Points**:
|
|
773
|
+
|
|
774
|
+
- `operation` replaces `method` + `path` — mutually exclusive at compile time
|
|
775
|
+
- Types for body, response, and query all inferred from the operation definition
|
|
776
|
+
- Works with any code generator using structural typing (no imports from playwright-utils needed in generator)
|
|
777
|
+
- Composable with `recurse`, `validateSchema`, and all existing `apiRequest` features
|
|
778
|
+
- Preferred approach over `typeof operation.response` for generated operations
|
|
779
|
+
|
|
716
780
|
## API Test Configuration
|
|
717
781
|
|
|
718
782
|
### Playwright Config for API-Only Tests
|
|
@@ -38,17 +38,17 @@ npm install -D @seontechnologies/playwright-utils
|
|
|
38
38
|
|
|
39
39
|
### Core Testing Utilities
|
|
40
40
|
|
|
41
|
-
| Utility | Purpose
|
|
42
|
-
| -------------------------- |
|
|
43
|
-
| **api-request** | Typed HTTP client with schema validation and
|
|
44
|
-
| **recurse** | Polling for async operations, background jobs
|
|
45
|
-
| **auth-session** | Token persistence, multi-user, service-to-service
|
|
46
|
-
| **log** | Playwright report-integrated logging
|
|
47
|
-
| **file-utils** | CSV/XLSX/PDF/ZIP reading & validation
|
|
48
|
-
| **burn-in** | Smart test selection with git diff
|
|
49
|
-
| **network-recorder** | HAR record/playback for offline testing
|
|
50
|
-
| **intercept-network-call** | Network spy/stub with auto JSON parsing
|
|
51
|
-
| **network-error-monitor** | Automatic HTTP 4xx/5xx detection
|
|
41
|
+
| Utility | Purpose | Test Context |
|
|
42
|
+
| -------------------------- | ----------------------------------------------------------------------------- | ------------------ |
|
|
43
|
+
| **api-request** | Typed HTTP client with schema validation, retry, and operation-based overload | **API/Backend** |
|
|
44
|
+
| **recurse** | Polling for async operations, background jobs | **API/Backend** |
|
|
45
|
+
| **auth-session** | Token persistence, multi-user, service-to-service | **API/Backend/UI** |
|
|
46
|
+
| **log** | Playwright report-integrated logging | **API/Backend/UI** |
|
|
47
|
+
| **file-utils** | CSV/XLSX/PDF/ZIP reading & validation | **API/Backend/UI** |
|
|
48
|
+
| **burn-in** | Smart test selection with git diff | **CI/CD** |
|
|
49
|
+
| **network-recorder** | HAR record/playback for offline testing | UI only |
|
|
50
|
+
| **intercept-network-call** | Network spy/stub with auto JSON parsing | UI only |
|
|
51
|
+
| **network-error-monitor** | Automatic HTTP 4xx/5xx detection | UI only |
|
|
52
52
|
|
|
53
53
|
**Note**: 6 of 9 utilities work without a browser. Only 3 are UI-specific (network-recorder, intercept-network-call, network-error-monitor).
|
|
54
54
|
|
|
@@ -21,7 +21,7 @@ test-healing-patterns,Test Healing Patterns,"Common failure patterns and automat
|
|
|
21
21
|
selector-resilience,Selector Resilience,"Robust selector strategies and debugging techniques","selectors,locators,debugging,ui",knowledge/selector-resilience.md
|
|
22
22
|
timing-debugging,Timing Debugging,"Race condition identification and deterministic wait fixes","timing,async,debugging",knowledge/timing-debugging.md
|
|
23
23
|
overview,Playwright Utils Overview,"Installation, design principles, fixture patterns for API and UI testing","playwright-utils,fixtures,api,backend,ui",knowledge/overview.md
|
|
24
|
-
api-request,API Request,"Typed HTTP client, schema validation, retry logic for API and service testing","api,backend,service-testing,api-testing,playwright-utils",knowledge/api-request.md
|
|
24
|
+
api-request,API Request,"Typed HTTP client, schema validation, retry logic, operation-based overload for API and service testing","api,backend,service-testing,api-testing,playwright-utils,openapi,codegen,operation",knowledge/api-request.md
|
|
25
25
|
network-recorder,Network Recorder,"HAR record/playback, CRUD detection for offline UI testing","network,playwright-utils,ui,har",knowledge/network-recorder.md
|
|
26
26
|
auth-session,Auth Session,"Token persistence, multi-user, API and browser authentication","auth,playwright-utils,api,backend,jwt,token",knowledge/auth-session.md
|
|
27
27
|
intercept-network-call,Intercept Network Call,"Network spy/stub, JSON parsing for UI tests","network,playwright-utils,ui",knowledge/intercept-network-call.md
|
|
@@ -36,3 +36,10 @@ From `workflow.yaml`, resolve:
|
|
|
36
36
|
|
|
37
37
|
Load, read completely, and execute:
|
|
38
38
|
`{project-root}/_bmad/tea/workflows/testarch/atdd/steps-c/step-01-preflight-and-context.md`
|
|
39
|
+
|
|
40
|
+
### 3. Resume Support
|
|
41
|
+
|
|
42
|
+
If the user selects **Resume** mode, load, read completely, and execute:
|
|
43
|
+
`{project-root}/_bmad/tea/workflows/testarch/atdd/steps-c/step-01b-resume.md`
|
|
44
|
+
|
|
45
|
+
This checks the output document for progress tracking frontmatter and routes to the next incomplete step.
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: 'step-01-preflight-and-context'
|
|
3
3
|
description: 'Verify prerequisites and load story, framework, and knowledge base'
|
|
4
|
+
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
|
|
4
5
|
nextStepFile: './step-02-generation-mode.md'
|
|
5
6
|
knowledgeIndex: '{project-root}/_bmad/tea/testarch/tea-index.csv'
|
|
6
7
|
---
|
|
@@ -104,6 +105,30 @@ Use `{knowledgeIndex}` to load:
|
|
|
104
105
|
|
|
105
106
|
Summarize loaded inputs and confirm with the user. Then proceed.
|
|
106
107
|
|
|
108
|
+
---
|
|
109
|
+
|
|
110
|
+
## 6. Save Progress
|
|
111
|
+
|
|
112
|
+
**Save this step's accumulated work to `{outputFile}`.**
|
|
113
|
+
|
|
114
|
+
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
|
|
115
|
+
|
|
116
|
+
```yaml
|
|
117
|
+
---
|
|
118
|
+
stepsCompleted: ['step-01-preflight-and-context']
|
|
119
|
+
lastStep: 'step-01-preflight-and-context'
|
|
120
|
+
lastSaved: '{date}'
|
|
121
|
+
---
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Then write this step's output below the frontmatter.
|
|
125
|
+
|
|
126
|
+
- **If `{outputFile}` already exists**, update:
|
|
127
|
+
- Add `'step-01-preflight-and-context'` to `stepsCompleted` array (only if not already present)
|
|
128
|
+
- Set `lastStep: 'step-01-preflight-and-context'`
|
|
129
|
+
- Set `lastSaved: '{date}'`
|
|
130
|
+
- Append this step's output to the appropriate section.
|
|
131
|
+
|
|
107
132
|
Load next step: `{nextStepFile}`
|
|
108
133
|
|
|
109
134
|
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: 'step-01b-resume'
|
|
3
|
+
description: 'Resume interrupted workflow from last completed step'
|
|
4
|
+
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Step 1b: Resume Workflow
|
|
8
|
+
|
|
9
|
+
## STEP GOAL
|
|
10
|
+
|
|
11
|
+
Resume an interrupted workflow by loading the existing output document, displaying progress, and routing to the next incomplete step.
|
|
12
|
+
|
|
13
|
+
## MANDATORY EXECUTION RULES
|
|
14
|
+
|
|
15
|
+
- 📖 Read the entire step file before acting
|
|
16
|
+
- ✅ Speak in `{communication_language}`
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## EXECUTION PROTOCOLS:
|
|
21
|
+
|
|
22
|
+
- 🎯 Follow the MANDATORY SEQUENCE exactly
|
|
23
|
+
- 📖 Load the next step only when instructed
|
|
24
|
+
|
|
25
|
+
## CONTEXT BOUNDARIES:
|
|
26
|
+
|
|
27
|
+
- Available context: Output document with progress frontmatter
|
|
28
|
+
- Focus: Load progress and route to next step
|
|
29
|
+
- Limits: Do not re-execute completed steps
|
|
30
|
+
- Dependencies: Output document must exist from a previous run
|
|
31
|
+
|
|
32
|
+
## MANDATORY SEQUENCE
|
|
33
|
+
|
|
34
|
+
**CRITICAL:** Follow this sequence exactly.
|
|
35
|
+
|
|
36
|
+
### 1. Load Output Document
|
|
37
|
+
|
|
38
|
+
Read `{outputFile}` and parse YAML frontmatter for:
|
|
39
|
+
|
|
40
|
+
- `stepsCompleted` — array of completed step names
|
|
41
|
+
- `lastStep` — last completed step name
|
|
42
|
+
- `lastSaved` — timestamp of last save
|
|
43
|
+
|
|
44
|
+
**If `{outputFile}` does not exist**, display:
|
|
45
|
+
|
|
46
|
+
"⚠️ **No previous progress found.** There is no output document to resume from. Please use **[C] Create** to start a fresh workflow run."
|
|
47
|
+
|
|
48
|
+
**THEN:** Halt. Do not proceed.
|
|
49
|
+
|
|
50
|
+
---
|
|
51
|
+
|
|
52
|
+
### 2. Display Progress Dashboard
|
|
53
|
+
|
|
54
|
+
Display progress with ✅/⬜ indicators:
|
|
55
|
+
|
|
56
|
+
1. ✅/⬜ Preflight & Context (step-01-preflight-and-context)
|
|
57
|
+
2. ✅/⬜ Generation Mode (step-02-generation-mode)
|
|
58
|
+
3. ✅/⬜ Test Strategy (step-03-test-strategy)
|
|
59
|
+
4. ✅/⬜ Generate Tests + Aggregate (step-04c-aggregate)
|
|
60
|
+
5. ✅/⬜ Validate & Complete (step-05-validate-and-complete)
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
### 3. Route to Next Step
|
|
65
|
+
|
|
66
|
+
Based on `lastStep`, load the next incomplete step:
|
|
67
|
+
|
|
68
|
+
- `'step-01-preflight-and-context'` → load `./step-02-generation-mode.md`
|
|
69
|
+
- `'step-02-generation-mode'` → load `./step-03-test-strategy.md`
|
|
70
|
+
- `'step-03-test-strategy'` → load `./step-04-generate-tests.md`
|
|
71
|
+
- `'step-04c-aggregate'` → load `./step-05-validate-and-complete.md`
|
|
72
|
+
- `'step-05-validate-and-complete'` → **Workflow already complete.** Display: "✅ **All steps completed.** Use **[V] Validate** to review outputs or **[E] Edit** to make revisions." Then halt.
|
|
73
|
+
|
|
74
|
+
**If `lastStep` does not match any value above**, display: "⚠️ **Unknown progress state** (`lastStep`: {lastStep}). Please use **[C] Create** to start fresh." Then halt.
|
|
75
|
+
|
|
76
|
+
**Otherwise**, load the identified step file, read completely, and execute.
|
|
77
|
+
|
|
78
|
+
The existing content in `{outputFile}` provides context from previously completed steps.
|
|
79
|
+
|
|
80
|
+
---
|
|
81
|
+
|
|
82
|
+
## 🚨 SYSTEM SUCCESS/FAILURE METRICS
|
|
83
|
+
|
|
84
|
+
### ✅ SUCCESS:
|
|
85
|
+
|
|
86
|
+
- Output document loaded and parsed correctly
|
|
87
|
+
- Progress dashboard displayed accurately
|
|
88
|
+
- Routed to correct next step
|
|
89
|
+
|
|
90
|
+
### ❌ SYSTEM FAILURE:
|
|
91
|
+
|
|
92
|
+
- Not loading output document
|
|
93
|
+
- Incorrect progress display
|
|
94
|
+
- Routing to wrong step
|
|
95
|
+
|
|
96
|
+
**Master Rule:** Resume MUST route to the exact next incomplete step. Never re-execute completed steps.
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: 'step-02-generation-mode'
|
|
3
3
|
description: 'Choose AI generation or recording mode'
|
|
4
|
+
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
|
|
4
5
|
nextStepFile: './step-03-test-strategy.md'
|
|
5
6
|
---
|
|
6
7
|
|
|
@@ -81,6 +82,30 @@ If `none`:
|
|
|
81
82
|
|
|
82
83
|
State the chosen mode and why. Then proceed.
|
|
83
84
|
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
## 4. Save Progress
|
|
88
|
+
|
|
89
|
+
**Save this step's accumulated work to `{outputFile}`.**
|
|
90
|
+
|
|
91
|
+
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
|
|
92
|
+
|
|
93
|
+
```yaml
|
|
94
|
+
---
|
|
95
|
+
stepsCompleted: ['step-02-generation-mode']
|
|
96
|
+
lastStep: 'step-02-generation-mode'
|
|
97
|
+
lastSaved: '{date}'
|
|
98
|
+
---
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
Then write this step's output below the frontmatter.
|
|
102
|
+
|
|
103
|
+
- **If `{outputFile}` already exists**, update:
|
|
104
|
+
- Add `'step-02-generation-mode'` to `stepsCompleted` array (only if not already present)
|
|
105
|
+
- Set `lastStep: 'step-02-generation-mode'`
|
|
106
|
+
- Set `lastSaved: '{date}'`
|
|
107
|
+
- Append this step's output to the appropriate section.
|
|
108
|
+
|
|
84
109
|
Load next step: `{nextStepFile}`
|
|
85
110
|
|
|
86
111
|
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: 'step-03-test-strategy'
|
|
3
3
|
description: 'Map acceptance criteria to test levels and priorities'
|
|
4
|
+
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
|
|
4
5
|
nextStepFile: './step-04-generate-tests.md'
|
|
5
6
|
---
|
|
6
7
|
|
|
@@ -62,6 +63,30 @@ Assign P0–P3 priorities using risk and business impact.
|
|
|
62
63
|
|
|
63
64
|
Ensure all tests are designed to **fail before implementation** (TDD red phase).
|
|
64
65
|
|
|
66
|
+
---
|
|
67
|
+
|
|
68
|
+
## 5. Save Progress
|
|
69
|
+
|
|
70
|
+
**Save this step's accumulated work to `{outputFile}`.**
|
|
71
|
+
|
|
72
|
+
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
|
|
73
|
+
|
|
74
|
+
```yaml
|
|
75
|
+
---
|
|
76
|
+
stepsCompleted: ['step-03-test-strategy']
|
|
77
|
+
lastStep: 'step-03-test-strategy'
|
|
78
|
+
lastSaved: '{date}'
|
|
79
|
+
---
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
Then write this step's output below the frontmatter.
|
|
83
|
+
|
|
84
|
+
- **If `{outputFile}` already exists**, update:
|
|
85
|
+
- Add `'step-03-test-strategy'` to `stepsCompleted` array (only if not already present)
|
|
86
|
+
- Set `lastStep: 'step-03-test-strategy'`
|
|
87
|
+
- Set `lastSaved: '{date}'`
|
|
88
|
+
- Append this step's output to the appropriate section.
|
|
89
|
+
|
|
65
90
|
Load next step: `{nextStepFile}`
|
|
66
91
|
|
|
67
92
|
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: 'step-04c-aggregate'
|
|
3
3
|
description: 'Aggregate subprocess outputs and complete ATDD test infrastructure'
|
|
4
|
+
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
|
|
4
5
|
nextStepFile: './step-05-validate-and-complete.md'
|
|
5
6
|
---
|
|
6
7
|
|
|
@@ -304,6 +305,30 @@ Proceed to Step 5 when:
|
|
|
304
305
|
- ✅ Summary statistics calculated and saved
|
|
305
306
|
- ✅ Output displayed to user
|
|
306
307
|
|
|
308
|
+
---
|
|
309
|
+
|
|
310
|
+
### 8. Save Progress
|
|
311
|
+
|
|
312
|
+
**Save this step's accumulated work to `{outputFile}`.**
|
|
313
|
+
|
|
314
|
+
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
|
|
315
|
+
|
|
316
|
+
```yaml
|
|
317
|
+
---
|
|
318
|
+
stepsCompleted: ['step-04c-aggregate']
|
|
319
|
+
lastStep: 'step-04c-aggregate'
|
|
320
|
+
lastSaved: '{date}'
|
|
321
|
+
---
|
|
322
|
+
```
|
|
323
|
+
|
|
324
|
+
Then write this step's output below the frontmatter.
|
|
325
|
+
|
|
326
|
+
- **If `{outputFile}` already exists**, update:
|
|
327
|
+
- Add `'step-04c-aggregate'` to `stepsCompleted` array (only if not already present)
|
|
328
|
+
- Set `lastStep: 'step-04c-aggregate'`
|
|
329
|
+
- Set `lastSaved: '{date}'`
|
|
330
|
+
- Append this step's output to the appropriate section.
|
|
331
|
+
|
|
307
332
|
Load next step: `{nextStepFile}`
|
|
308
333
|
|
|
309
334
|
---
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: 'step-05-validate-and-complete'
|
|
3
3
|
description: 'Validate ATDD outputs and summarize'
|
|
4
|
+
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
# Step 5: Validate & Complete
|
|
@@ -58,6 +59,30 @@ Report:
|
|
|
58
59
|
- Key risks or assumptions
|
|
59
60
|
- Next recommended workflow (e.g., implementation or `automate`)
|
|
60
61
|
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## 3. Save Progress
|
|
65
|
+
|
|
66
|
+
**Save this step's accumulated work to `{outputFile}`.**
|
|
67
|
+
|
|
68
|
+
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
|
|
69
|
+
|
|
70
|
+
```yaml
|
|
71
|
+
---
|
|
72
|
+
stepsCompleted: ['step-05-validate-and-complete']
|
|
73
|
+
lastStep: 'step-05-validate-and-complete'
|
|
74
|
+
lastSaved: '{date}'
|
|
75
|
+
---
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
Then write this step's output below the frontmatter.
|
|
79
|
+
|
|
80
|
+
- **If `{outputFile}` already exists**, update:
|
|
81
|
+
- Add `'step-05-validate-and-complete'` to `stepsCompleted` array (only if not already present)
|
|
82
|
+
- Set `lastStep: 'step-05-validate-and-complete'`
|
|
83
|
+
- Set `lastSaved: '{date}'`
|
|
84
|
+
- Append this step's output to the appropriate section.
|
|
85
|
+
|
|
61
86
|
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|
|
62
87
|
|
|
63
88
|
### ✅ SUCCESS:
|
|
@@ -29,11 +29,13 @@ This workflow uses **tri-modal step-file architecture**:
|
|
|
29
29
|
"Welcome to the workflow. What would you like to do?"
|
|
30
30
|
|
|
31
31
|
- **[C] Create** — Run the workflow
|
|
32
|
+
- **[R] Resume** — Resume an interrupted workflow
|
|
32
33
|
- **[V] Validate** — Validate existing outputs
|
|
33
34
|
- **[E] Edit** — Edit existing outputs
|
|
34
35
|
|
|
35
36
|
### 2. Route to First Step
|
|
36
37
|
|
|
37
38
|
- **If C:** Load `steps-c/step-01-preflight-and-context.md`
|
|
39
|
+
- **If R:** Load `steps-c/step-01b-resume.md`
|
|
38
40
|
- **If V:** Load `steps-v/step-01-validate.md`
|
|
39
41
|
- **If E:** Load `steps-e/step-01-assess.md`
|