@ivorycanvas/qamap 0.3.5 → 0.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +49 -0
- package/README.md +37 -23
- package/dist/agent-init.js +2 -2
- package/dist/agent-init.js.map +1 -1
- package/dist/behavior-intent.d.ts +6 -0
- package/dist/behavior-intent.js +183 -0
- package/dist/behavior-intent.js.map +1 -0
- package/dist/behavior-manifest.d.ts +6 -0
- package/dist/behavior-manifest.js +221 -0
- package/dist/behavior-manifest.js.map +1 -0
- package/dist/behavior.d.ts +151 -0
- package/dist/behavior.js +458 -0
- package/dist/behavior.js.map +1 -0
- package/dist/change-intent.d.ts +81 -0
- package/dist/change-intent.js +884 -0
- package/dist/change-intent.js.map +1 -0
- package/dist/cli.js +13 -8
- package/dist/cli.js.map +1 -1
- package/dist/e2e.d.ts +16 -1
- package/dist/e2e.js +290 -25
- package/dist/e2e.js.map +1 -1
- package/dist/index.d.ts +11 -3
- package/dist/index.js +5 -1
- package/dist/index.js.map +1 -1
- package/dist/qa.d.ts +1 -0
- package/dist/qa.js +293 -88
- package/dist/qa.js.map +1 -1
- package/dist/terminal.js +1 -1
- package/dist/terminal.js.map +1 -1
- package/dist/test-plan.d.ts +18 -0
- package/dist/test-plan.js +123 -13
- package/dist/test-plan.js.map +1 -1
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/docs/adoption.md +11 -10
- package/docs/agent-format.md +16 -10
- package/docs/agent-skill.md +8 -5
- package/docs/architecture.md +131 -0
- package/docs/benchmarking.md +23 -4
- package/docs/commands.md +13 -7
- package/docs/e2e-output-examples.md +29 -28
- package/docs/quickstart-demo.md +29 -17
- package/docs/release-validation.md +49 -6
- package/docs/releasing.md +26 -3
- package/docs/roadmap.md +20 -10
- package/package.json +4 -2
- package/schema/qamap-agent.schema.json +122 -3
- package/schema/qamap-behavior.schema.json +339 -0
- package/skills/qamap-pr-qa/SKILL.md +20 -14
package/docs/agent-format.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Agent Format Contract
|
|
2
2
|
|
|
3
|
-
`qamap qa --format agent` prints one line of JSON
|
|
3
|
+
`qamap qa --format agent` prints one compact line of JSON designed to be pasted into a coding agent's context instead of the full markdown report. The complete line is capped at 8KB. When the uncapped result would be larger, QAMap preserves the strongest evidence and reports total and omitted counts instead of silently overflowing the context budget. This page is the contract for that output: what the fields mean, what an agent may rely on, and how the format is allowed to change.
|
|
4
4
|
|
|
5
5
|
```sh
|
|
6
6
|
qamap qa . --base origin/main --head HEAD --format agent
|
|
@@ -19,10 +19,11 @@ qamap qa . --base origin/main --head HEAD --format agent
|
|
|
19
19
|
The intended loop for a coding agent:
|
|
20
20
|
|
|
21
21
|
1. Run the command above and parse stdout as JSON.
|
|
22
|
-
2.
|
|
23
|
-
3.
|
|
24
|
-
4.
|
|
25
|
-
5.
|
|
22
|
+
2. Read `intents` first. Confirm its confidence, lifecycle, and QA scenarios. Inspect `scenarios[].sources` before accepting a recommendation: diff sources identify the base/head file, line, symbol, hunk, and relation that caused the scenario to be proposed. `direct` is scenario-specific evidence, `supporting` completes the lifecycle, and `contextual` explains intent but cannot independently promote a scenario to critical. When `reviewRequired` is true, ask a human to confirm it.
|
|
23
|
+
3. Treat `readiness` as automation readiness, not as a verdict on the PR. `requiredBootstrap` contains non-runner repository context only; runner adoption lives under the opt-in `automation` object.
|
|
24
|
+
4. Use `flows[].changedFiles`, `flows[].evidence`, and `flows[].reviewQuestion` to understand why the flow was selected. Use `steps`, `selectors`, and `successSignal` to write or review tests; `runnable` says how much to trust the generated draft.
|
|
25
|
+
5. Surface `requiredEvidence` in the PR description, and paste `prChecklist` items into the PR body.
|
|
26
|
+
6. Run `commands` to validate.
|
|
26
27
|
|
|
27
28
|
## Fields
|
|
28
29
|
|
|
@@ -31,15 +32,20 @@ The intended loop for a coding agent:
|
|
|
31
32
|
| `schema` | object | `{ name: "qamap.qa", version: 1 }` — check before parsing. |
|
|
32
33
|
| `base`, `head` | string | Git refs the diff was computed from. |
|
|
33
34
|
| `project` | string | Detected project type (for example `web`, `react-native`, `node`, `unknown`). |
|
|
34
|
-
| `runner` | string |
|
|
35
|
+
| `runner` | string | Automation output adapter selected after QA intent analysis: `maestro`, `playwright`, or `manual`. |
|
|
35
36
|
| `manifest` | string \| null | Verification manifest path in use, or `null` when the run used repo signals and the PR diff only. |
|
|
36
37
|
| `readiness` | object | `score` (0–100) and `level` (`ready` \| `near-runnable` \| `needs-work` \| `blocked`). Human reports render the same value as a four-stage journey; the machine value is stable. |
|
|
37
38
|
| `testSuite` | object | `present` (boolean) and `files` (number of detected test files). |
|
|
38
|
-
| `
|
|
39
|
-
| `
|
|
39
|
+
| `intentCount`, `omittedIntentCount` | number | Total inferred intents and the count omitted from the compact payload. |
|
|
40
|
+
| `intents` | array | Evidence-backed change intents (capped). Each includes `title`, `confidence`, `reviewRequired`, backward-compatible string `evidence`, structured `sources`, ordered `lifecycle` phases, and runner-independent QA `scenarios`. Every compact scenario carries `confidence`, `reviewRequired`, structured `sources`, and assertions; `scenarioCount` and `omittedScenarioCount` disclose capping. Empty when commit and diff evidence cannot support a behavior intent. |
|
|
41
|
+
| `firstDraftCommand` | string? | Deprecated v1 compatibility field. New output omits it so runner setup is not promoted as the default QA action. |
|
|
42
|
+
| `automation` | object? | Explicitly optional adapter handoff: `optIn`, `adapter`, `setupStatus`, `draftCommand`, and optional `setupCommand`. Use it only after the QA scenario is accepted. |
|
|
43
|
+
| `flowCount`, `omittedFlowCount` | number | Total affected flows and the count omitted from the compact payload. |
|
|
44
|
+
| `flows` | array | Affected user flows, most relevant first (capped). Each has `title`, `source`, backward-compatible `draft`, optional `runnable`, `entry`, and `verificationMode`, plus `changedFiles`, `reviewQuestion`, `successSignal`, `steps`, `selectors`, and short `evidence` reasons. Test-only changes expose `existingEvidence`; configuration, docs, generated artifacts, and changed tests use `verificationMode`. |
|
|
45
|
+
| `compaction` | object | Present only when lower-priority detail was reduced to keep the complete line within 8KB. Carries `maxBytes` and the uncapped `originalBytes`. |
|
|
40
46
|
| `requiredEvidence` | array | Required-priority QA evidence still missing, capped at 8: `flow`, `kind`, `title`. |
|
|
41
47
|
| `recommendedEvidenceCount` | number | How many recommended-priority items were omitted; run without `--format agent` to see them. |
|
|
42
|
-
| `requiredBootstrap` | array |
|
|
48
|
+
| `requiredBootstrap` | array | Non-runner repository context steps (capped at 3): `title`, `action`. Runner setup is represented only under `automation`. |
|
|
43
49
|
| `prChecklist` | array of string | Ready-to-paste PR checklist lines (capped). |
|
|
44
50
|
| `commands` | array of string | Suggested next commands, most useful first (capped at 4). |
|
|
45
51
|
|
|
@@ -48,5 +54,5 @@ List fields are capped to keep the payload small; caps may grow within version 1
|
|
|
48
54
|
## Example
|
|
49
55
|
|
|
50
56
|
```json
|
|
51
|
-
{"schema":{"name":"qamap.qa","version":1},"base":"main","head":"HEAD","project":"web","runner":"playwright","manifest":null,"readiness":{"score":51,"level":"needs-work"},"testSuite":{"present":false,"files":0},"
|
|
57
|
+
{"schema":{"name":"qamap.qa","version":1},"base":"main","head":"HEAD","project":"web","runner":"playwright","manifest":null,"readiness":{"score":51,"level":"needs-work"},"testSuite":{"present":false,"files":0},"intentCount":1,"omittedIntentCount":0,"intents":[{"title":"Submit notification preferences","confidence":"high","reviewRequired":false,"evidence":["feat: submit notification preferences"],"sources":[{"kind":"diff","reason":"Invoke fetch.","file":"src/pages/preferences.tsx","symbol":"fetch","relation":"supporting","side":"head","startLine":7,"endLine":7,"hunk":"@@ -1,5 +1,19 @@"}],"scenarioCount":1,"omittedScenarioCount":0,"lifecycle":[{"phase":"trigger","label":"Submit notification preferences."},{"phase":"side-effect","label":"Invoke fetch."},{"phase":"observable-outcome","label":"Show the saved state."}],"scenarios":[{"priority":"critical","kind":"primary","title":"Submit notification preferences","confidence":"high","reviewRequired":false,"sources":[{"kind":"diff","reason":"Invoke fetch.","file":"src/pages/preferences.tsx","symbol":"fetch","relation":"supporting","side":"head","startLine":7,"endLine":7,"hunk":"@@ -1,5 +1,19 @@"}],"assertions":["Verify the saved state becomes observable."]}]}],"automation":{"optIn":true,"adapter":"playwright","setupStatus":"proposed","draftCommand":"qamap e2e draft . --base main --head HEAD","setupCommand":"qamap e2e setup . --runner playwright"},"flowCount":1,"omittedFlowCount":0,"flows":[{"title":"Submit notification preferences","source":"commit-and-diff-intent","draft":"tests/e2e/submit-notification-preferences.spec.ts","runnable":"near-runnable","entry":"route: /preferences (high)","changedFiles":["src/pages/preferences.tsx"],"reviewQuestion":"Does the changed preference lifecycle produce the saved state?","successSignal":"visible text Preferences saved appears","steps":["Submit preferences.","Invoke fetch.","Verify the saved state."],"selectors":["web-test-id: preferences-save"],"evidence":["Commit and diff evidence support this change intent."]}],"requiredEvidence":[],"recommendedEvidenceCount":1,"requiredBootstrap":[],"prChecklist":["Review the proposed QA scenario and its diff source."],"commands":["npm run build"]}
|
|
52
58
|
```
|
package/docs/agent-skill.md
CHANGED
|
@@ -32,7 +32,7 @@ Run this before writing a PR body or asking for review. Agents should prefer the
|
|
|
32
32
|
pnpm dlx @ivorycanvas/qamap qa . --base origin/main --head HEAD --format agent
|
|
33
33
|
```
|
|
34
34
|
|
|
35
|
-
The result carries `flows[]` (
|
|
35
|
+
The result carries `intents[]` with scenario-level structured diff `sources`, `flows[]` (affected behavior, entry route, steps, selectors), `requiredEvidence[]`, optional `automation`, `prChecklist[]`, and `commands[]` under `schema: qamap.qa`.
|
|
36
36
|
|
|
37
37
|
For a human-readable report, drop the flag; for installed projects write it to a file:
|
|
38
38
|
|
|
@@ -68,12 +68,15 @@ If your agent supports symlinked skills, point its skill directory at `skills/qa
|
|
|
68
68
|
|
|
69
69
|
## What The Agent Should Do With The Output
|
|
70
70
|
|
|
71
|
-
Use the `PR Comment Draft`
|
|
71
|
+
Use `Change Intent Evidence` and the `PR Comment Draft` as review context:
|
|
72
72
|
|
|
73
|
+
- commit-backed intent, confidence, and whether human review is required
|
|
74
|
+
- ordered behavior lifecycle
|
|
75
|
+
- primary, failure, boundary, and state-transition QA scenarios
|
|
76
|
+
- the strongest commit or `file:line` source for every proposed scenario
|
|
73
77
|
- affected flow
|
|
74
|
-
-
|
|
75
|
-
-
|
|
76
|
-
- missing fixture, selector, assertion, runner, or validation evidence
|
|
78
|
+
- missing fixture, selector, or assertion evidence
|
|
79
|
+
- optional automation adapter selected only after QA design
|
|
77
80
|
- PR checklist items
|
|
78
81
|
|
|
79
82
|
If the command says a generated recommendation is wrong, do not keep re-prompting the agent with the same context. Update the repo-local manifest after human review:
|
|
@@ -0,0 +1,131 @@
|
|
|
1
|
+
# Architecture
|
|
2
|
+
|
|
3
|
+
QAMap is moving from deterministic PR-to-draft heuristics toward a local, change-aware QA engine. The core must remain useful without cloud services, source upload, or LLM calls.
|
|
4
|
+
|
|
5
|
+
The target pipeline is:
|
|
6
|
+
|
|
7
|
+
```txt
|
|
8
|
+
commit range + base/head diff
|
|
9
|
+
-> change intent analysis
|
|
10
|
+
-> analyzer adapters
|
|
11
|
+
-> behavior graph
|
|
12
|
+
-> behavior lifecycle + impact selection
|
|
13
|
+
-> runner-independent QA scenarios
|
|
14
|
+
-> Playwright / Maestro / manual adapter
|
|
15
|
+
-> explicit local execution
|
|
16
|
+
-> normalized evidence and verdict
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
The repository, not a model session, is the source of truth. Commit subjects and bodies provide author intent, source code supplies observable structure, and `.qamap/manifest.yaml` supplies reviewed product intent that commits and code alone cannot prove.
|
|
20
|
+
|
|
21
|
+
## Change Intent
|
|
22
|
+
|
|
23
|
+
`src/change-intent.ts` reads behavior-bearing commits in the selected base/head range and joins related `feat`, `fix`, `hotfix`, `perf`, and supporting `refactor` commits through normalized domain terms. Added diff symbols provide independent evidence for triggers, conditions, state changes, side effects, and observable outcomes.
|
|
24
|
+
|
|
25
|
+
Each intent contains:
|
|
26
|
+
|
|
27
|
+
- the original commit evidence and source scope;
|
|
28
|
+
- an explicit confidence and `reviewRequired` flag;
|
|
29
|
+
- an ordered behavior lifecycle;
|
|
30
|
+
- runner-independent primary, failure, boundary, and state-transition QA scenarios.
|
|
31
|
+
|
|
32
|
+
One richly evidenced squash commit can reach high confidence. A title without connected diff evidence cannot. Working-tree-only inference is always low confidence and review-required. Release, docs, style, CI, and test-only commits do not become product intents.
|
|
33
|
+
|
|
34
|
+
The analysis is deterministic and local. It does not execute repository code, contact GitHub, upload source, or call an LLM.
|
|
35
|
+
|
|
36
|
+
## Behavior Graph
|
|
37
|
+
|
|
38
|
+
`src/behavior.ts` defines the framework-neutral intermediate representation. A graph contains stable nodes, typed edges, confidence, evidence, and direct or propagated change impact.
|
|
39
|
+
|
|
40
|
+
Generated graphs identify the shipped contract through `schema/qamap-behavior.schema.json` and `schemaVersion: 1`. The schema URL and node, edge, surface, and evidence enums are exported from the package so adapters and consumers can reject unsupported shapes deterministically.
|
|
41
|
+
|
|
42
|
+
Initial node kinds cover:
|
|
43
|
+
|
|
44
|
+
- domains and flows;
|
|
45
|
+
- routes, screens, endpoints, commands, and artifact surfaces;
|
|
46
|
+
- actions, states, effects, contracts, and assertions;
|
|
47
|
+
- fixtures, locators, and source files.
|
|
48
|
+
|
|
49
|
+
Initial edges describe containment, entrypoints, ordering, expected outcomes, fixture use, locators, implementation sources, and diff impact.
|
|
50
|
+
|
|
51
|
+
Every inferred node must retain provenance. A node without a commit, source, diff, selector, fixture, test, manifest, or named inference reason should not influence a QA verdict.
|
|
52
|
+
|
|
53
|
+
Node and edge ids are content-derived and stable. Re-running analysis against the same repository state must produce the same graph identity even when report timestamps differ.
|
|
54
|
+
|
|
55
|
+
## Compatibility Adapter
|
|
56
|
+
|
|
57
|
+
The first graph integration uses `qamap.inferred-flow-compat`. It translates the existing E2E flow observations into the new graph so the IR can be introduced without changing existing CLI recommendations.
|
|
58
|
+
|
|
59
|
+
This adapter is a migration bridge, not the final analysis architecture. Framework adapters should gradually emit graph fragments directly, after which draft generation will consume the graph instead of the graph consuming completed drafts.
|
|
60
|
+
|
|
61
|
+
`qamap.change-intent` is the first direct product adapter. It emits intent contracts, lifecycle actions/states/effects, scenario assertions, source links, and commit provenance before an automation runner is selected.
|
|
62
|
+
|
|
63
|
+
## Analyzer Adapters
|
|
64
|
+
|
|
65
|
+
An analyzer adapter has two operations:
|
|
66
|
+
|
|
67
|
+
```ts
|
|
68
|
+
interface BehaviorAnalyzerAdapter {
|
|
69
|
+
id: string;
|
|
70
|
+
version: string;
|
|
71
|
+
detect(context): Detection;
|
|
72
|
+
analyze(context): BehaviorGraphFragment;
|
|
73
|
+
}
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Detection must be evidence-based and may decline a repository. Analysis failures are isolated and reported as diagnostics so one optional adapter cannot erase useful output from the others.
|
|
77
|
+
|
|
78
|
+
Adapters should be layered:
|
|
79
|
+
|
|
80
|
+
1. language adapters provide symbols, imports, calls, and schemas;
|
|
81
|
+
2. framework adapters provide routes, screens, handlers, and lifecycle conventions;
|
|
82
|
+
3. repository adapters provide manifests, tests, fixtures, and local policy;
|
|
83
|
+
4. executor adapters compile selected scenarios for an existing runner.
|
|
84
|
+
|
|
85
|
+
Support is reported by capability rather than a single yes/no framework badge:
|
|
86
|
+
|
|
87
|
+
| Level | Contract |
|
|
88
|
+
| --- | --- |
|
|
89
|
+
| Deep | Behavior impact, deterministic scenarios, and local execution are supported. |
|
|
90
|
+
| Structural | Routes, contracts, existing tests, and validation commands are understood. |
|
|
91
|
+
| Generic | Diff and dependency evidence are available; QAMap does not invent a product journey. |
|
|
92
|
+
|
|
93
|
+
TypeScript-based web stacks are the first deep-analysis target because the existing import, route, selector, and fixture signals already provide a useful base. Vue, Nuxt, and SvelteKit should reuse the common web behavior model rather than fork the QA pipeline. Mobile, API, Python, Go, and JVM support should enter through the same adapter contract.
|
|
94
|
+
|
|
95
|
+
## Manifest Boundary
|
|
96
|
+
|
|
97
|
+
The generated graph is local cache material and should not be committed in full. The verification manifest stores only durable, reviewed knowledge:
|
|
98
|
+
|
|
99
|
+
- important flows and criticality;
|
|
100
|
+
- expected and forbidden outcomes;
|
|
101
|
+
- auth, permission, fixture, and environment requirements;
|
|
102
|
+
- stable anchors and invariants;
|
|
103
|
+
- accepted corrections and suppressions.
|
|
104
|
+
|
|
105
|
+
This keeps first-run setup light while allowing one human correction to improve later PRs deterministically.
|
|
106
|
+
|
|
107
|
+
## Execution Boundary
|
|
108
|
+
|
|
109
|
+
`qamap qa` remains static and read-only. It must not execute scanned project code.
|
|
110
|
+
|
|
111
|
+
Future execution belongs to an explicit `qamap verify` mode with a visible execution plan. The intended safeguards are:
|
|
112
|
+
|
|
113
|
+
- generated tests live in an operating-system temporary directory by default;
|
|
114
|
+
- target repository files are not modified unless a separate write command is requested;
|
|
115
|
+
- commands, environment variables, network access, and time limits are policy controlled;
|
|
116
|
+
- existing mocks and fixtures are preferred over fabricated data;
|
|
117
|
+
- missing setup becomes `not verifiable`, never a false pass;
|
|
118
|
+
- source code and evidence remain local and no LLM is called.
|
|
119
|
+
|
|
120
|
+
Playwright, Maestro, and other tools are executor implementations. They are not the product-level recommendation shown first to users.
|
|
121
|
+
|
|
122
|
+
## Migration Order
|
|
123
|
+
|
|
124
|
+
1. Derive change intent and runner-independent QA scenarios from commit and diff evidence.
|
|
125
|
+
2. Move route, screen, endpoint, selector, fixture, and contract discovery into graph-producing adapters.
|
|
126
|
+
3. Compare base and head graphs to select affected behavior rather than relying on file categories alone.
|
|
127
|
+
4. Compile selected graph paths through Playwright, Maestro, or manual adapters.
|
|
128
|
+
5. Add explicit, temporary execution and normalized evidence.
|
|
129
|
+
6. Add manifest accept, reject, and repair commands so reviewed outcomes improve later analysis.
|
|
130
|
+
|
|
131
|
+
Version `0.4.0` establishes the commit-to-intent-to-scenario slice for synthetic web and mobile lifecycle changes. The next minor is reserved for a policy-controlled scenario execution and normalized evidence slice; compatible analyzer and adapter improvements remain patch releases during `0.x` development.
|
package/docs/benchmarking.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
QAMap's unit tests prove that the implementation behaves as coded. The benchmark contract checks a different question: does a representative PR receive a useful QA answer?
|
|
4
4
|
|
|
5
|
-
`bench.config.json` is committed and runs in CI. Each target under `test/benchmarks/` contains a `base/` repository snapshot and a `head/` overlay. The runner materializes them as a temporary Git repository with a `main` baseline and one feature commit. QAMap reads those repositories but never installs dependencies or executes their code.
|
|
5
|
+
`bench.config.json` is committed and runs in CI. Each target under `test/benchmarks/` contains a `base/` repository snapshot and a `head/` overlay. The runner materializes them as a temporary Git repository with a `main` baseline and one feature commit. Intent fixtures set a synthetic `commitMessage` so commit-to-lifecycle behavior is part of the contract. QAMap reads those repositories but never installs dependencies or executes their code.
|
|
6
6
|
|
|
7
7
|
## Run the public contract
|
|
8
8
|
|
|
@@ -14,6 +14,9 @@ The command fails when any target violates its declared expectations. The initia
|
|
|
14
14
|
|
|
15
15
|
- a web app with no tests;
|
|
16
16
|
- a web app with Playwright and an existing mock handler;
|
|
17
|
+
- Vue and SvelteKit web changes with framework-native route files;
|
|
18
|
+
- a web preferences change that must become submit, persistence, request-failure, and re-entry QA instead of a generic journey;
|
|
19
|
+
- a mobile reminder change that must become scheduling, calendar, duplicate, resynchronization, and entry-routing QA;
|
|
17
20
|
- an Expo app with Maestro;
|
|
18
21
|
- an API service that should produce a contract checklist instead of a browser journey;
|
|
19
22
|
- a design-token repository that should stay on artifact verification;
|
|
@@ -27,13 +30,26 @@ Each target can declare:
|
|
|
27
30
|
|
|
28
31
|
| Field | Meaning |
|
|
29
32
|
| --- | --- |
|
|
30
|
-
| `
|
|
33
|
+
| `commitMessage` | Synthetic feature-commit message used when materializing a fixture. |
|
|
34
|
+
| `runner` | Expected `playwright`, `maestro`, or `manual` output adapter. Runner correctness alone is not a useful intent benchmark. |
|
|
31
35
|
| `minFlows` | Minimum number of affected flows. |
|
|
36
|
+
| `minChangeIntents` | Minimum evidence-backed Change Intents. |
|
|
37
|
+
| `minHighConfidenceIntents` | Minimum intents supported strongly enough by commit and diff evidence to avoid mandatory review. |
|
|
38
|
+
| `minCommitBehaviorNodes` | Minimum Behavior Graph nodes carrying commit provenance. |
|
|
32
39
|
| `minImportPropagatedFlows` | Minimum flows discovered through reverse imports. |
|
|
33
40
|
| `minDiffAnchoredFlows` | Minimum flows using selector evidence introduced by the diff. |
|
|
34
41
|
| `minManifestMatches` | Minimum domain, flow, and check matches from an external base manifest. |
|
|
35
42
|
| `minManifestFlowMatches` | Minimum flow-level matches from the external base manifest. |
|
|
36
43
|
| `minManifestBackedFlows` | Minimum QA flows that preserve manifest provenance. |
|
|
44
|
+
| `minManifestBehaviorNodes` | Minimum Behavior Graph nodes carrying verification-manifest evidence. |
|
|
45
|
+
| `mustHaveBehaviorKinds` | Behavior Graph node kinds that must be present, such as `flow`, `surface`, `source`, `assertion`, or `locator`. |
|
|
46
|
+
| `mustNameIntents` | Concrete terms that must appear in the inferred intent title. |
|
|
47
|
+
| `mustNotNameIntents` | Misleading terms that must not appear in inferred intent titles. |
|
|
48
|
+
| `mustIncludeLifecycle` | Trigger, condition, action, state, effect, or outcome terms that must survive in the ordered lifecycle. |
|
|
49
|
+
| `mustIncludeQaScenarios` | Failure, boundary, state-transition, or primary QA terms that must be proposed before runner compilation. |
|
|
50
|
+
| `mustFindIntentEvidence` | Commit or diff terms that must remain attached to intent provenance. |
|
|
51
|
+
| `mustTraceScenarioFiles` | Changed files that must appear in at least one scenario's exact direct/supporting base- or head-side diff source. |
|
|
52
|
+
| `maxUntracedCriticalScenarios` | Maximum critical scenarios without a direct/supporting diff source carrying a file and line number. Contextual commit evidence cannot satisfy this contract; lifecycle fixtures keep it at zero. |
|
|
37
53
|
| `mustReachFiles` | Files that the selected flows must reach. |
|
|
38
54
|
| `mustNameFlows` | Product terms that must appear in a user-facing flow title. |
|
|
39
55
|
| `mustNotNameFlows` | Misleading flow-title terms that must not be emitted. |
|
|
@@ -41,12 +57,13 @@ Each target can declare:
|
|
|
41
57
|
| `mustIncludeSteps` | Behavior terms that must appear in draft steps. |
|
|
42
58
|
| `mustFindSelectors` | Stable selector evidence that must be recovered from the repository. |
|
|
43
59
|
| `mustFindSuccessSignals` | Observable outcome text that must appear in the flow's success criteria. |
|
|
60
|
+
| `mustFindEntrypoints` | Route, screen, or command entrypoints that affected flows must recover. |
|
|
44
61
|
| `mustFindEvidence` | Required evidence or fixture terms that must be reported. |
|
|
45
62
|
| `mustNotFindEvidence` | Evidence terms that would be false positives for this change. |
|
|
46
63
|
| `mustRecommendCommands` | Commands the setup or validation path must expose. |
|
|
47
64
|
| `maxBlankActions` | Maximum malformed or empty draft steps; public fixtures keep this at zero. |
|
|
48
65
|
| `maxGenericTitles` | Maximum titles ending in generic `primary journey` or `smoke flow` wording. |
|
|
49
|
-
| `maxAgentBytes` | Maximum UTF-8 payload size for `qa --format agent`. |
|
|
66
|
+
| `maxAgentBytes` | Maximum UTF-8 payload size for `qa --format agent`. Production output also has a global 8KB ceiling and discloses omitted intent/flow counts. |
|
|
50
67
|
|
|
51
68
|
Set `manifestBaseline: true` on a committed fixture to generate its manifest from the base snapshot into the benchmark temp directory, then pass that external manifest to analysis of the head commit. The fixture repository is never modified by this step. This protects the feedback loop itself: a baseline must affect the next PR, not merely serialize valid YAML.
|
|
52
69
|
|
|
@@ -62,7 +79,9 @@ node scripts/bench.mjs --baseline bench-results/<file>.json
|
|
|
62
79
|
|
|
63
80
|
When both files exist, `pnpm bench` prefers the gitignored local config. CI always passes `--config bench.config.json --assert`, so private paths cannot affect the public quality gate.
|
|
64
81
|
|
|
65
|
-
Saved results include flow titles, draft paths, recall gaps, readiness, agent payload size, and timing. Use a saved baseline to see heuristic movement, but treat the committed expectation contract as the merge gate.
|
|
82
|
+
Saved results include intent titles, lifecycle and scenario terms, scenario trace coverage, flow titles, draft paths, recall gaps, readiness, agent payload size, and timing. Use a saved baseline to see heuristic movement, but treat the committed expectation contract as the merge gate.
|
|
83
|
+
|
|
84
|
+
Every benchmark target also enforces the Behavior Graph base contract: graph schema version 1, at least one graph flow for every planned flow, at least one impacted node for a non-empty diff, and no edge whose endpoint is missing. The table reports `graph n/i` as total nodes versus impacted nodes. These checks keep the graph connected to real PR analysis while framework-specific adapters are introduced incrementally.
|
|
66
85
|
|
|
67
86
|
## Adding a regression
|
|
68
87
|
|
package/docs/commands.md
CHANGED
|
@@ -21,7 +21,7 @@ Use `pnpm dlx @ivorycanvas/qamap ...` for one-off runs without installing QAMap
|
|
|
21
21
|
|
|
22
22
|
## Reading The Output
|
|
23
23
|
|
|
24
|
-
Human-facing reports (`text` and `markdown` formats) open with an **At a Glance** section: affected behavior, the reviewer question to answer before merge, concrete repository evidence, the proposed draft path,
|
|
24
|
+
Human-facing reports (`text` and `markdown` formats) open with an **At a Glance** section: evidence-backed change intent, behavior lifecycle, affected behavior, the reviewer question to answer before merge, concrete repository evidence, the proposed draft path, and missing trust requirements. Runner information appears later as an automation adapter. When printed to an interactive terminal the report is colorized (headings, statuses, priority tags, inline commands); files written with `--output`, pipes, CI logs, and the machine formats (`json`, `agent`, `sarif`) are always plain. The standard `NO_COLOR` and `FORCE_COLOR` environment variables are honored.
|
|
25
25
|
|
|
26
26
|
Draft readiness is reported as a **stage on a fixed four-step journey**, for example `Stage: setup needed (1 of 4) — readiness 0/100`. A fresh repository usually starts at stage 1 — that is the expected starting point, not a failure. Each stage maps to a stable `readiness.level` value in the `json` and `agent` formats, which keeps machine output unchanged:
|
|
27
27
|
|
|
@@ -37,7 +37,9 @@ Draft readiness is reported as a **stage on a fixed four-step journey**, for exa
|
|
|
37
37
|
|
|
38
38
|
On a changed branch, QAMap tries to produce reviewable verification artifacts instead of only saying "write more tests":
|
|
39
39
|
|
|
40
|
-
- a
|
|
40
|
+
- a commit-and-diff-backed change intent with confidence, review requirements, and source evidence
|
|
41
|
+
- an ordered trigger, condition, action, state-change, side-effect, and observable-outcome lifecycle
|
|
42
|
+
- runner-independent primary, failure, boundary, and state-transition QA scenarios
|
|
41
43
|
- draft Playwright, Maestro, CLI command, or manual checklist files when the repository shape supports them
|
|
42
44
|
- a repo-level verification manifest loop where humans correct durable flows once and later PRs get sharper route/check/test draft suggestions
|
|
43
45
|
- a runner setup proposal that explains why Playwright or Maestro fits the changed surface and which files/commands would be created if the team accepts it
|
|
@@ -61,9 +63,9 @@ That means QAMap is most valuable when it becomes the team's verification base:
|
|
|
61
63
|
| `qamap eval . --base origin/main --head HEAD --pr-body-file pr-body.md` | Score change readiness across intent, risk, tests, and review size. |
|
|
62
64
|
| `qamap github-action . --mode review --base origin/main --head HEAD` | Generate GitHub Action annotations, step summary, and PR comment body. |
|
|
63
65
|
| `qamap test-plan . --base origin/main --head HEAD --include-working-tree` | Suggest domain test scenarios for changed files. |
|
|
64
|
-
| `qamap qa . --base origin/main --head HEAD` | One-command PR QA:
|
|
65
|
-
| `qamap qa . --base origin/main --head HEAD --format agent` | The same decision content as one
|
|
66
|
-
| `qamap e2e plan . --base origin/main --head HEAD` |
|
|
66
|
+
| `qamap qa . --base origin/main --head HEAD` | One-command PR QA: change intent, behavior lifecycle, QA scenarios, affected flows, missing evidence, and optional automation drafts. |
|
|
67
|
+
| `qamap qa . --base origin/main --head HEAD --format agent` | The same decision content as one compact JSON line for coding agents — a versioned contract documented in [docs/agent-format.md](agent-format.md). |
|
|
68
|
+
| `qamap e2e plan . --base origin/main --head HEAD` | Derive change intent and QA scenarios, then map them to coverage, test evidence, testability gaps, and an automation adapter. |
|
|
67
69
|
| `qamap e2e plan . --base origin/main --head HEAD --record-history` | Save a compact local run snapshot under `.qamap/runs/` while keeping JSON/Markdown output usable. |
|
|
68
70
|
| `qamap e2e setup . --runner playwright` | Explicitly apply the accepted runner setup and create the first changed-flow E2E draft without overwriting existing files. |
|
|
69
71
|
| `qamap e2e draft . --base origin/main --head HEAD --dry-run` | Preview generated Maestro, Playwright, or manual E2E drafts without writing files. |
|
|
@@ -90,7 +92,9 @@ For monorepos, pass `--workspace-root` when scanning a package. Package-local ch
|
|
|
90
92
|
|
|
91
93
|
`qamap test-plan` turns changed file paths into a review-ready domain test checklist. It also discovers common validation commands from `package.json`, `pyproject.toml`, `go.mod`, `Cargo.toml`, Gradle files, and Maven `pom.xml`. Add `--include-working-tree` for local, uncommitted changes while iterating.
|
|
92
94
|
|
|
93
|
-
`qamap e2e plan`
|
|
95
|
+
`qamap e2e plan` first reads behavior-bearing commits in the selected base/head range. Related `feat`, `fix`, `hotfix`, `perf`, and supporting `refactor` commits are grouped into change intents, then connected to added diff symbols. The result is an evidence-backed lifecycle and a set of runner-independent primary, failure, boundary, and state-transition scenarios. Low-confidence or working-tree-only intent remains review-required instead of becoming trusted regression policy.
|
|
96
|
+
|
|
97
|
+
After that judgment, QAMap detects whether the target looks like Expo/React Native, web, API/service, CLI, or another repository shape and selects a Maestro, Playwright, or manual output adapter. The plan compares scenario targets with existing test evidence, identifies required mocks and fixtures, and reports missing stable selectors such as `testID` or `data-testid`. Runner detection is an implementation detail, not the first value shown to the user.
|
|
94
98
|
|
|
95
99
|
The plan also includes an execution profile: detected start command, test command, Playwright `baseURL`, mobile app id, runner config files, env fixture files, confidence, and blockers. This keeps generated E2E drafts honest about whether they are runnable candidates or still review-only scaffolds.
|
|
96
100
|
|
|
@@ -100,7 +104,7 @@ When a repository does not already have the selected E2E runner, the plan includ
|
|
|
100
104
|
|
|
101
105
|
When run at a monorepo root, the E2E plan also reports changed app/package targets. This helps a maintainer move from a broad workspace diff to scoped commands such as `qamap e2e plan services/listing --workspace-root . --base origin/main --head HEAD`, where package-specific runner detection and flow naming are usually sharper.
|
|
102
106
|
|
|
103
|
-
Each candidate flow also includes a flow language brief: actor, trigger, goal, success signal, reviewer question, and edge cases.
|
|
107
|
+
Each candidate flow also includes a flow language brief: actor, trigger, goal, success signal, reviewer question, and edge cases. Intent-backed flows additionally retain the original commits, ordered lifecycle, QA scenarios, confidence, and review requirement so generated tests can be traced to evidence rather than only changed file names.
|
|
104
108
|
|
|
105
109
|
The bootstrap section answers what must happen before generated drafts can be treated as real regression coverage. For example, a testless web project can get required steps for Playwright setup, first draft generation, stable selector work, fixture/mock data, and missing validation evidence, plus recommended steps for `.qamap/manifest.yaml`, `.qamap/domains.yml`, `.qamap/flows.yml`, and local history recording.
|
|
106
110
|
|
|
@@ -207,6 +211,8 @@ Pass `--record-history` when you want QAMap to keep a compact local snapshot of
|
|
|
207
211
|
|
|
208
212
|
The draft result is meant to be useful as a PR artifact, not only as generated files. Markdown and JSON output include:
|
|
209
213
|
|
|
214
|
+
- `changeAnalysis`: commit-backed intents, confidence, review requirements, source evidence, lifecycle stages, and runner-independent QA scenarios
|
|
215
|
+
- intent-backed draft files retain `intentId`, `intentConfidence`, `lifecycle`, and `qaScenarios`; generated files include the same evidence as comments
|
|
210
216
|
- `languageBrief`: actor, trigger, goal, success signal, reviewer question, and edge cases for each draft file
|
|
211
217
|
- `promotionStatus`: whether the draft is a `commit-candidate`, `needs-review`, or `low-signal`
|
|
212
218
|
- `runnableStatus`: whether the draft is a `runnable-candidate`, `near-runnable`, or `review-only`
|
|
@@ -15,20 +15,31 @@ The output should be specific enough to paste into a PR comment:
|
|
|
15
15
|
```txt
|
|
16
16
|
# QAMap QA Draft
|
|
17
17
|
|
|
18
|
+
At a Glance
|
|
19
|
+
- Change intent: Submit checkout and persist the confirmed order [high]
|
|
20
|
+
- Behavior lifecycle: trigger: Submit checkout -> side-effect: Create order -> observable-outcome: Show order confirmation
|
|
21
|
+
- Affected behavior: Submit checkout and persist the confirmed order
|
|
22
|
+
|
|
23
|
+
Change Intent Evidence
|
|
24
|
+
- Commit: feat: submit checkout and persist the confirmed order
|
|
25
|
+
- Critical scenario: Submit checkout and persist the confirmed order
|
|
26
|
+
- Assert: order confirmation is visible and persisted
|
|
27
|
+
- Recommended scenario: Failure, timeout, and retry handling
|
|
28
|
+
|
|
18
29
|
Summary
|
|
19
30
|
- Project: Web
|
|
20
|
-
-
|
|
31
|
+
- Automation adapter: Playwright
|
|
21
32
|
- Manifest: not found; using repo signals and PR diff only
|
|
22
33
|
- Stage: almost runnable (3 of 4)
|
|
23
34
|
|
|
24
35
|
PR Comment Draft
|
|
25
|
-
- Affected flow:
|
|
36
|
+
- Affected flow: Submit checkout and persist the confirmed order
|
|
26
37
|
- User journey: Customer -> Open route /checkout -> Complete checkout with realistic form data
|
|
27
38
|
- Success signal: confirmation state is visible after submit
|
|
28
39
|
- Changed files: src/pages/checkout/index.tsx
|
|
29
40
|
|
|
30
41
|
Suggested E2E / QA Draft
|
|
31
|
-
- tests/e2e/checkout-
|
|
42
|
+
- tests/e2e/submit-checkout-and-persist-the-confirmed-order.spec.ts: near runnable
|
|
32
43
|
- Open route /checkout.
|
|
33
44
|
- Fill checkout email.
|
|
34
45
|
- Submit checkout.
|
|
@@ -139,7 +150,7 @@ flows:
|
|
|
139
150
|
|
|
140
151
|
```txt
|
|
141
152
|
Project: Web
|
|
142
|
-
|
|
153
|
+
Automation adapter: Playwright
|
|
143
154
|
Execution profile: high
|
|
144
155
|
Start command: pnpm run dev
|
|
145
156
|
Test command: pnpm run test:e2e
|
|
@@ -224,7 +235,7 @@ For an Expo or React Native change, QAMap should recommend Maestro and carry mob
|
|
|
224
235
|
|
|
225
236
|
```txt
|
|
226
237
|
Project: Expo / React Native
|
|
227
|
-
|
|
238
|
+
Automation adapter: Maestro
|
|
228
239
|
|
|
229
240
|
Flow: Ink Drawing UI smoke flow
|
|
230
241
|
Actor: User
|
|
@@ -269,7 +280,7 @@ For backend changes such as `src/v1/listing/utils.ts`, QAMap should not invent a
|
|
|
269
280
|
|
|
270
281
|
```txt
|
|
271
282
|
Project: API / service
|
|
272
|
-
|
|
283
|
+
Automation adapter: Manual
|
|
273
284
|
|
|
274
285
|
Flow: Listing API contract smoke checklist
|
|
275
286
|
Actor: API consumer or upstream service
|
|
@@ -296,7 +307,7 @@ For an npm package that exposes `package.json` bin entries, QAMap should not inv
|
|
|
296
307
|
|
|
297
308
|
```txt
|
|
298
309
|
Project: CLI
|
|
299
|
-
|
|
310
|
+
Automation adapter: Manual
|
|
300
311
|
|
|
301
312
|
Flow: CLI command verification checklist
|
|
302
313
|
Actor: CLI user or maintainer
|
|
@@ -319,30 +330,17 @@ The manual draft should name concrete command evidence:
|
|
|
319
330
|
|
|
320
331
|
## Test-Light Project
|
|
321
332
|
|
|
322
|
-
When a project has little or no E2E setup,
|
|
333
|
+
When a project has little or no E2E setup, `qamap qa` still returns the same runner-independent QA judgment. Runner setup remains an explicit team choice:
|
|
323
334
|
|
|
324
335
|
```txt
|
|
325
|
-
##
|
|
326
|
-
|
|
327
|
-
QAMap did not find committed test files for this target. The next step is to create the first runnable starter draft, not to stop at a checklist.
|
|
328
|
-
|
|
329
|
-
- Recommended first runner: Playwright
|
|
330
|
-
- Create command: `qamap e2e setup . --runner playwright`
|
|
331
|
-
- Install command: `pnpm add -D @playwright/test`
|
|
332
|
-
- Draft files QAMap can create:
|
|
333
|
-
- `playwright.config.ts`
|
|
334
|
-
- `tests/e2e/`
|
|
335
|
-
- `tests/e2e/checkout-primary-journey.spec.ts`
|
|
336
|
-
- Files to update: `package.json`
|
|
336
|
+
## Optional Automation
|
|
337
337
|
|
|
338
|
-
|
|
339
|
-
4 required bootstrap steps must be resolved before generated E2E drafts should be treated as regression coverage.
|
|
338
|
+
The QA judgment above does not require adopting this adapter.
|
|
340
339
|
|
|
341
|
-
|
|
342
|
-
-
|
|
343
|
-
-
|
|
344
|
-
-
|
|
345
|
-
- Add stable selectors for changed UI surfaces
|
|
340
|
+
- Adapter candidate: Playwright
|
|
341
|
+
- Draft target: `tests/e2e/checkout-primary-journey.spec.ts`
|
|
342
|
+
- Preview or create a draft: `qamap e2e draft . --base main --head HEAD`
|
|
343
|
+
- If the team accepts this adapter, inspect its setup proposal: `qamap e2e setup . --runner playwright`
|
|
346
344
|
```
|
|
347
345
|
|
|
348
346
|
Draft action items should make the next developer action explicit:
|
|
@@ -460,9 +458,12 @@ Related Changed Files
|
|
|
460
458
|
Good QAMap output should answer these questions quickly:
|
|
461
459
|
|
|
462
460
|
- What behavior did this branch probably change?
|
|
461
|
+
- Which commits and diff symbols support that intent, and how confident is the inference?
|
|
462
|
+
- What trigger, condition, state change, side effect, and observable outcome form its lifecycle?
|
|
463
|
+
- Which primary, failure, boundary, and state-transition checks follow from that lifecycle?
|
|
463
464
|
- What does the team call that behavior?
|
|
464
465
|
- Who or what exercises it?
|
|
465
|
-
- Which
|
|
466
|
+
- Which existing automation adapter can compile the selected QA scenario?
|
|
466
467
|
- Which setup, selector, fixture, or validation gap blocks this from becoming real regression coverage?
|
|
467
468
|
|
|
468
469
|
If the output only says "make an E2E test" without these answers, it is not ready for the current release bar.
|
package/docs/quickstart-demo.md
CHANGED
|
@@ -15,6 +15,7 @@ src/features/checkout/submitCheckout.ts
|
|
|
15
15
|
The reviewer wants to know:
|
|
16
16
|
|
|
17
17
|
- Which user flow is affected?
|
|
18
|
+
- Which diff hunk caused each QA scenario to be proposed?
|
|
18
19
|
- Should this become a Playwright test, a manual checklist, or only a review note?
|
|
19
20
|
- What blocks the generated draft from being trusted as regression coverage?
|
|
20
21
|
|
|
@@ -26,9 +27,9 @@ Run QAMap on the branch:
|
|
|
26
27
|
pnpm dlx @ivorycanvas/qamap qa . --base origin/main --head HEAD
|
|
27
28
|
```
|
|
28
29
|
|
|
29
|
-
`qa` is important for first contact. It lets maintainers
|
|
30
|
+
`qa` is important for first contact. It lets maintainers review change intent, behavior lifecycle, scenario confidence, exact diff sources, and a PR checklist without writing files or selecting a test runner.
|
|
30
31
|
|
|
31
|
-
|
|
32
|
+
After the team accepts a scenario and wants executable coverage:
|
|
32
33
|
|
|
33
34
|
```sh
|
|
34
35
|
pnpm dlx @ivorycanvas/qamap e2e draft . --base origin/main --head HEAD --dry-run
|
|
@@ -145,11 +146,15 @@ Input
|
|
|
145
146
|
```txt
|
|
146
147
|
Output
|
|
147
148
|
- PR comment/checklist draft
|
|
149
|
+
- commit-backed change intent and confidence
|
|
150
|
+
- ordered behavior lifecycle
|
|
151
|
+
- primary, failure, boundary, and state-transition QA scenarios
|
|
152
|
+
- scenario-level confidence, review status, and exact commit or head-side `file:line` sources
|
|
148
153
|
- changed domain language
|
|
149
154
|
- candidate user flow
|
|
150
155
|
- manifest evidence when a repo-local flow/check matches
|
|
151
|
-
-
|
|
152
|
-
- draft file path
|
|
156
|
+
- optional automation adapter selected after QA design
|
|
157
|
+
- optional draft file path
|
|
153
158
|
- flow language brief
|
|
154
159
|
- runnable status
|
|
155
160
|
- self-check status
|
|
@@ -157,33 +162,40 @@ Output
|
|
|
157
162
|
- blockers that explain why a draft is not ready yet
|
|
158
163
|
```
|
|
159
164
|
|
|
160
|
-
The result should not stop at "selector needed" or "fixture needed." A useful QAMap
|
|
165
|
+
The result should not stop at "selector needed" or "fixture needed." A useful QAMap report should say which behavior changed, which scenarios follow, which exact hunk supports each scenario, what remains uncertain, and where to update the manifest if the recommendation is wrong. Automation comes after that review.
|
|
161
166
|
|
|
162
167
|
## Markdown Preview
|
|
163
168
|
|
|
164
169
|
```txt
|
|
165
170
|
# QAMap QA Draft
|
|
166
171
|
|
|
172
|
+
At a Glance
|
|
173
|
+
- Change intent: Submit notification preferences and show the saved state [high]
|
|
174
|
+
- Behavior lifecycle: trigger -> state-change -> side-effect -> observable-outcome
|
|
175
|
+
|
|
167
176
|
Summary
|
|
168
177
|
- Project: Web
|
|
169
|
-
- Recommended runner: Playwright
|
|
170
178
|
- Manifest: .qamap/manifest.yaml
|
|
171
|
-
- Stage: almost runnable (3 of 4)
|
|
172
179
|
|
|
173
|
-
|
|
174
|
-
-
|
|
175
|
-
-
|
|
176
|
-
-
|
|
177
|
-
-
|
|
180
|
+
QA scenarios
|
|
181
|
+
- [critical] changed preference lifecycle [high]
|
|
182
|
+
- Source: src/pages/preferences.tsx:17, symbol onClick
|
|
183
|
+
- Assert: the saved state is visible
|
|
184
|
+
- [recommended] failure, timeout, and retry handling [medium; review required]
|
|
185
|
+
- Source: src/pages/preferences.tsx:7, symbol fetch
|
|
186
|
+
- Assert: retries do not duplicate requests or side effects
|
|
178
187
|
|
|
179
|
-
|
|
180
|
-
- [required] fixture: Add deterministic
|
|
181
|
-
- [recommended] selector: Confirm stable selectors for changed checkout controls.
|
|
188
|
+
Evidence gaps in this QA proposal
|
|
189
|
+
- [required] fixture: Add deterministic response data for /api/preferences.
|
|
182
190
|
|
|
183
191
|
PR checklist
|
|
184
|
-
- [ ] Review
|
|
185
|
-
- [ ] Confirm success and failed-response assertions.
|
|
192
|
+
- [ ] Review each QA scenario and its diff source.
|
|
193
|
+
- [ ] Confirm success and failed-response assertions for /api/preferences.
|
|
186
194
|
- [ ] Run pnpm run test:e2e.
|
|
195
|
+
|
|
196
|
+
Optional automation
|
|
197
|
+
- Adapter candidate: Playwright
|
|
198
|
+
- qamap e2e draft . --base origin/main --head HEAD
|
|
187
199
|
```
|
|
188
200
|
|
|
189
201
|
## Draft Shape
|