qaa-agent 1.9.2 → 1.9.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (43) hide show
  1. package/CHANGELOG.md +230 -179
  2. package/CLAUDE.md +720 -557
  3. package/README.md +384 -357
  4. package/VERSION +1 -1
  5. package/agents/qa-pipeline-orchestrator.md +1739 -1425
  6. package/agents/qaa-analyzer.md +0 -1
  7. package/agents/qaa-bug-detective.md +792 -655
  8. package/agents/qaa-codebase-mapper.md +54 -1
  9. package/agents/qaa-discovery.md +431 -384
  10. package/agents/qaa-e2e-runner.md +717 -577
  11. package/agents/qaa-executor.md +966 -830
  12. package/agents/qaa-planner.md +14 -1
  13. package/agents/qaa-project-researcher.md +539 -400
  14. package/agents/qaa-scanner.md +86 -1
  15. package/agents/qaa-testid-injector.md +0 -1
  16. package/agents/qaa-validator.md +86 -1
  17. package/bin/install.cjs +374 -252
  18. package/bin/lib/context7-cache.cjs +299 -0
  19. package/bin/lib/intent-detector.cjs +490 -0
  20. package/commands/qa-audit.md +269 -126
  21. package/commands/qa-create-test-ado.md +439 -404
  22. package/commands/qa-create-test.md +672 -365
  23. package/commands/qa-fix.md +691 -513
  24. package/commands/qa-map.md +319 -139
  25. package/commands/qa-pr.md +63 -0
  26. package/commands/qa-research.md +212 -157
  27. package/commands/qa-start.md +62 -6
  28. package/commands/qa-test-report.md +254 -219
  29. package/commands/qa-testid.md +31 -0
  30. package/frameworks/cypress.json +54 -0
  31. package/frameworks/jest.json +59 -0
  32. package/frameworks/playwright.json +58 -0
  33. package/frameworks/pytest.json +59 -0
  34. package/frameworks/robot-framework.json +54 -0
  35. package/frameworks/selenium.json +65 -0
  36. package/frameworks/vitest.json +57 -0
  37. package/package.json +41 -38
  38. package/workflows/qa-analyze.md +100 -4
  39. package/workflows/qa-from-ticket.md +50 -2
  40. package/workflows/qa-gap.md +50 -2
  41. package/workflows/qa-start.md +2048 -1405
  42. package/workflows/qa-testid.md +50 -2
  43. package/workflows/qa-validate.md +50 -2
@@ -1,219 +1,254 @@
1
- # QA Test Report
2
-
3
- Generate a per-ticket QA execution summary and append it to the Azure DevOps work item's `Custom.QATestCasesReport` field. For each test case linked to the work item, pull the execution status from ADO Test Runs — or prompt the user when ADO has no run result — then render a bulleted "Tested Scenarios" list (with a separate "Failures" section when anything failed).
4
-
5
- ## Usage
6
-
7
- ```
8
- /qa-test-report <work-item-id>
9
- ```
10
-
11
- ### Arguments
12
-
13
- | Parameter | Purpose | Default |
14
- |-----------|---------|---------|
15
- | `<work-item-id>` | Azure DevOps work item ID (Bug, Feature, User Story, Ticket) whose linked test cases you want reported | Required — prompt the user if missing |
16
-
17
- ## What It Produces
18
-
19
- - Markdown report printed to chat (for user review)
20
- - HTML report **appended** to the work item's `Custom.QATestCasesReport` field, separated from prior content by a blank line
21
- - No local file is written
22
-
23
- ---
24
-
25
- ## Process
26
-
27
- ### Phase 1 Resolve the Work Item
28
-
29
- 1. If `$ARGUMENTS` is empty or missing a work item ID, ask the user: *"Which Azure DevOps work item ID should I build the QA Test Cases report for?"* Wait for the answer before proceeding.
30
- 2. Call `wit_get_work_item` with `expand: "relations"` for the resolved ID.
31
- - Capture: **title**, **type** (`Bug`, `Feature`, `User Story`, `Ticket`), **state**, **area path**, **iteration path**.
32
- - For context used in Phase 4, capture type-appropriate content fields:
33
- - `Bug` / `Ticket`: **Repro Steps** (`Microsoft.VSTS.TCM.ReproSteps`), **System Info** (`Microsoft.VSTS.TCM.SystemInfo`), **Description**, **QA Notes** (`CIIScrum.QANotes`), **What is expected to happen** (`Custom.Whatisexpectedtohappen`), **What is actually happening** (`Custom.Whatisactuallyhappening`).
34
- - `User Story`: **Acceptance Criteria** (`Microsoft.VSTS.Common.AcceptanceCriteria`), **Description**.
35
- - `Feature`: **Description**, **Acceptance Criteria** if present.
36
- - Note the project for all subsequent ADO calls.
37
- 3. Also call `wit_list_work_item_comments` comments often contain fail reasons or tester observations referenced in Phase 3 fallbacks.
38
-
39
- ---
40
-
41
- ### Phase 2 — Resolve the Test Case List
42
-
43
- Source-resolution order:
44
-
45
- 1. **Check for a local file** at `ai-tasks/ticket-{id}/test-cases.md` (produced by `/qa-create-test --ado`). If it exists, parse the TC IDs it contains — this is a hint list only.
46
- 2. **Always query ADO** — inspect the relations returned in Phase 1, filter for link type `"Microsoft.VSTS.Common.TestedBy-Forward"` (*Tested By*), and build the authoritative list of linked TC IDs.
47
- 3. **If both exist and disagree**: trust **ADO**. Log the discrepancy in chat as an FYI (e.g., `"TC#4521 was in local file but not linked in ADO — skipped"` or `"TC#4530 is linked in ADO but missing from local file — included"`) but proceed with the ADO list.
48
- 4. **If no TCs are found in ADO** (and the local file has none either): print `"No test cases linked to work item #{id} — nothing to report."` and stop.
49
-
50
- For every TC ID in the final list, call `wit_get_work_item` with `expand: "relations"` to capture: **title**, **state**, **steps** (`Microsoft.VSTS.TCM.Steps`), **priority**, **tags**, and any linked runs.
51
-
52
- ---
53
-
54
- ### Phase 3 Resolve Execution Status Per Test Case
55
-
56
- For each TC, determine a status of **Passed**, **Failed**, or **Blocked** using this order.
57
-
58
- **Step 1 — Fetch Test Point outcomes via ADO REST (formal path).**
59
-
60
- The ADO MCP surface does not expose Test Point outcomes, but the REST API does. Use the same auth pattern as `/qa-create-test --ado`: the `ADO_MCP_AUTH_TOKEN` env var as a PAT, sent via Basic auth.
61
-
62
- 1. Derive **org** and **project** from the work item context captured in Phase 1 (not hardcoded). The work-item URL / project name on the `wit_get_work_item` response gives you both.
63
- 2. If `ADO_MCP_AUTH_TOKEN` is **not set**, skip directly to Step 2 (and note in the final report: *"Auto-status skipped — ADO_MCP_AUTH_TOKEN not set."*).
64
- 3. Otherwise, call the Test Points REST endpoint once per run with the list of linked TC IDs:
65
-
66
- ```bash
67
- curl -s \
68
- --header "Authorization: Basic $(echo -n :${ADO_MCP_AUTH_TOKEN} | base64)" \
69
- --header "Content-Type: application/json" \
70
- --request POST \
71
- --data '{"pointsFilter":{"testcaseIds":[<id1>,<id2>,...]}}' \
72
- "https://dev.azure.com/{org}/{project}/_apis/test/points?api-version=7.1"
73
- ```
74
-
75
- 4. Parse the response. For each returned point you get: `testCase.id`, `testPlan.id`, `suite.id`, `outcome`, `lastTestRun.id`, `lastResultDetails.dateCompleted`, `lastResultDetails.runBy`.
76
- 5. **If a TC has multiple points** (e.g., it lives in several suites or runs on multiple configurations), pick the point with the **most recent `lastResultDetails.dateCompleted`**.
77
- 6. Map `outcome` (case-insensitive) to our statuses:
78
- - `passed` → **Passed**
79
- - `failed` → **Failed**
80
- - `blocked` **Blocked**
81
- - `notExecuted` / `none` / `notApplicable` / `paused` / `inProgress` / `warning` / `error` treat as missing, fall through to Step 2 for this TC
82
- 7. **If the call fails** (non-2xx, network error, token rejected): log the failure briefly in chat, skip to Step 2 for all TCs, and note in the final report that auto-status was unavailable.
83
-
84
- **Step 2 — Ask the user** (per TC that didn't resolve in Step 1). Show the TC ID and its synthesized scenario description (from Phase 4) for context:
85
-
86
- ```
87
- TC #{id} — {scenario description}
88
- Status? [Passed / Failed / Blocked]
89
- ```
90
-
91
- Accept case-insensitive input. Re-prompt if the answer is not one of the three.
92
-
93
- **Step 3If the resolved status is Failed**, capture a reason:
94
-
95
- 1. Check the Test Run data first when Step 1 returned an outcome, also fetch the Test Result for `lastTestRun.id` to read `errorMessage`/`comment`:
96
- ```bash
97
- curl -s \
98
- --header "Authorization: Basic $(echo -n :${ADO_MCP_AUTH_TOKEN} | base64)" \
99
- "https://dev.azure.com/{org}/{project}/_apis/test/Runs/{runId}/results?api-version=7.1"
100
- ```
101
- If the result has a non-empty `errorMessage` or `comment`, use that as the reason.
102
- 2. Also check the TC's own comments (`wit_list_work_item_comments` for the TC ID) and the parent work item's comments for any line mentioning this TC.
103
- 3. If no reason is found, ask the user:
104
- ```
105
- TC #{id} failed — what was the reason?
106
- ```
107
- Accept a free-text one-liner.
108
-
109
- Keep the per-TC answers (status, reason, and auto/manual source) in memory; do not write them anywhere intermediate.
110
-
111
- ---
112
-
113
- ### Phase 4 — Synthesize Scenario Descriptions
114
-
115
- For each TC, produce a **readable one-line scenario description** for the bullet list. Do **not** copy the raw TC title when it is terse or cryptic — TC titles are often shorthand.
116
-
117
- Build the description from:
118
- - TC **title** (starting point)
119
- - TC **steps** (each step's action + expected result gives you what was actually verified)
120
- - Parent work item **description / repro steps / acceptance criteria** (gives you the user-facing phrasing of *what should be true after the fix*)
121
-
122
- Write the description in the voice of the tested outcome, not the test action. Examples:
123
-
124
- | Raw TC title | Good scenario description |
125
- |---|---|
126
- | `Verify PH row display logic` | `$0 Previous Balance row no longer appears in Payment History` |
127
- | `Check login 401 scenario` | `Invalid credentials return a 401 with an error banner` |
128
- | `Regression: limit 100` | `Entry limit enforced at 100 the 101st entry is rejected` |
129
-
130
- Keep bullets shortone line, past-tense or assertive present-tense, user-visible wording.
131
-
132
- ---
133
-
134
- ### Phase 5 — Build the Report
135
-
136
- **Markdown version (for chat output):**
137
-
138
- ```markdown
139
- Tested Scenarios
140
-
141
- - ✅ {scenario}
142
- - ✅ {scenario}
143
- - ❌ {scenario}
144
- - ⚠️ {scenario}
145
-
146
- Failures
147
-
148
- - {scenario}: {reason}
149
- ```
150
-
151
- Rules:
152
- - Status emoji: **Passed → ✅**, **Failed → ❌**, **Blocked → ⚠️**
153
- - Include the `Failures` section **only if at least one TC failed**. One bullet per failure with the captured reason appended after `:`.
154
- - Do **not** include a "Result" section.
155
- - Do **not** include TC IDs, priority, tags, or counts in the bullets.
156
- - Keep order stable — preserve the order TCs appeared in the ADO linked list.
157
-
158
- **HTML version (for the ADO field):**
159
-
160
- Render with the minimum markup needed for ADO's rich-text field to display bullets and line breaks:
161
-
162
- ```html
163
- <b>Tested Scenarios</b><br>
164
- <ul>
165
- <li>✅ {scenario}</li>
166
- <li>✅ {scenario}</li>
167
- <li>❌ {scenario}</li>
168
- <li>⚠️ {scenario}</li>
169
- </ul>
170
- <b>Failures</b><br>
171
- <ul>
172
- <li>❌ {scenario}: {reason}</li>
173
- </ul>
174
- ```
175
-
176
- Rules:
177
- - Escape scenario text for HTML (`&` → `&amp;`, `<` → `&lt;`, `>` → `&gt;`).
178
- - Emit the `<b>Failures</b>…` block **only if at least one TC failed**.
179
- - No inline styles, no classes, no wrapping `<div>`.
180
-
181
- ---
182
-
183
- ### Phase 6 — Append to the ADO Field
184
-
185
- 1. Re-read the current `Custom.QATestCasesReport` value for the work item (it may contain prior runs you must preserve).
186
- 2. Build the new field value:
187
- - If the existing value is empty or whitespace: new value = the HTML report from Phase 5.
188
- - Otherwise: new value = existing content + `<br><br>` + the HTML report from Phase 5.
189
- 3. Call `wit_update_work_item` setting `Custom.QATestCasesReport` to the new value. Pass only this field — do not include any others in the update.
190
-
191
- Do **not** overwrite other fields. Do **not** create a local file.
192
-
193
- ---
194
-
195
- ### Phase 7 Present to User
196
-
197
- Print the **markdown** version of the report to the user in chat. After the report, print a one-line confirmation:
198
-
199
- ```
200
- Appended to work item #{id} → Custom.QATestCasesReport.
201
- ```
202
-
203
- If any scenarios required a manual status answer or a manual failure reason during Phase 3, append a short note listing how many:
204
-
205
- ```
206
- {n} status(es) entered manually · {m} failure reason(s) entered manually.
207
- ```
208
-
209
- If the TC list had any local-vs-ADO discrepancies (Phase 2), append a short note listing them.
210
-
211
- ---
212
-
213
- ## Notes
214
-
215
- - This skill does **not** create test cases. Use `/qa-create-test --ado <id>` for that.
216
- - This skill is read-mostly on the ADO side — it only writes to `Custom.QATestCasesReport`. It does not change TC state, does not create Test Runs, and does not modify links.
217
- - Scenario descriptions are synthesized, not verbatim TC titles — review the chat output before trusting the field content.
218
-
219
- $ARGUMENTS
1
+ # QA Test Report
2
+
3
+ Generate a per-ticket QA execution summary and append it to the Azure DevOps work item's `Custom.QATestCasesReport` field. For each test case linked to the work item, pull the execution status from ADO Test Runs — or prompt the user when ADO has no run result — then render a bulleted "Tested Scenarios" list (with a separate "Failures" section when anything failed).
4
+
5
+ ## ⚠ MANDATORY: How to Execute This Command
6
+
7
+ Execute these steps IN ORDER. Every step is mandatory. Do NOT improvise,
8
+ reorder, skip, or invent steps. This command reads Azure DevOps and writes ONLY
9
+ the `Custom.QATestCasesReport` field; it spawns no sub-agents and creates no
10
+ test cases.
11
+
12
+ 1. PHASE 1 — Resolve the work item: if `$ARGUMENTS` has no work item ID, ask the
13
+ user before proceeding. Fetch it with `wit_get_work_item` (relations) plus
14
+ its comments.
15
+ 2. PHASE 2 Resolve the test-case list. ADO is authoritative: build the list
16
+ from the *Tested By* links; a local `test-cases.md` is only a hint. If no TCs
17
+ are linked, print "nothing to report" and STOP.
18
+ 3. PHASE 3 — Resolve each TC's status (Passed/Failed/Blocked): try the ADO Test
19
+ Points REST endpoint first; for any TC that doesn't resolve, ask the user. For
20
+ Failed TCs, capture a reason (run data comments ask the user).
21
+ 4. PHASE 4 — Synthesize a readable one-line scenario description per TC (not the
22
+ raw TC title).
23
+ 5. PHASE 5 — Build BOTH the markdown (for chat) and HTML (for the field) reports.
24
+ Include the Failures section only if something failed.
25
+ 6. PHASE 6 — APPEND the HTML report to `Custom.QATestCasesReport`: re-read the
26
+ current value and preserve it (existing + `<br><br>` + new). Update ONLY that
27
+ fieldnever overwrite others.
28
+ 7. PHASE 7 — Print the markdown report + the append confirmation to the user.
29
+
30
+ ### DO NOT
31
+ - OVERWRITE `Custom.QATestCasesReport` always append, preserving prior content
32
+ - Write to any field other than `Custom.QATestCasesReport`
33
+ - Create test cases, change TC state, or create Test Runs (this command is read-mostly)
34
+ - Copy raw TC titles as scenario descriptions when they are terse
35
+ - Invent steps not in this command, or reorder the phases
36
+
37
+ If you find yourself overwriting the field, touching other fields, or creating or
38
+ modifying test cases — STOP and restart from step 1.
39
+
40
+ ## Usage
41
+
42
+ ```
43
+ /qa-test-report <work-item-id>
44
+ ```
45
+
46
+ ### Arguments
47
+
48
+ | Parameter | Purpose | Default |
49
+ |-----------|---------|---------|
50
+ | `<work-item-id>` | Azure DevOps work item ID (Bug, Feature, User Story, Ticket) whose linked test cases you want reported | Required prompt the user if missing |
51
+
52
+ ## What It Produces
53
+
54
+ - Markdown report printed to chat (for user review)
55
+ - HTML report **appended** to the work item's `Custom.QATestCasesReport` field, separated from prior content by a blank line
56
+ - No local file is written
57
+
58
+ ---
59
+
60
+ ## Process
61
+
62
+ ### Phase 1 Resolve the Work Item
63
+
64
+ 1. If `$ARGUMENTS` is empty or missing a work item ID, ask the user: *"Which Azure DevOps work item ID should I build the QA Test Cases report for?"* Wait for the answer before proceeding.
65
+ 2. Call `wit_get_work_item` with `expand: "relations"` for the resolved ID.
66
+ - Capture: **title**, **type** (`Bug`, `Feature`, `User Story`, `Ticket`), **state**, **area path**, **iteration path**.
67
+ - For context used in Phase 4, capture type-appropriate content fields:
68
+ - `Bug` / `Ticket`: **Repro Steps** (`Microsoft.VSTS.TCM.ReproSteps`), **System Info** (`Microsoft.VSTS.TCM.SystemInfo`), **Description**, **QA Notes** (`CIIScrum.QANotes`), **What is expected to happen** (`Custom.Whatisexpectedtohappen`), **What is actually happening** (`Custom.Whatisactuallyhappening`).
69
+ - `User Story`: **Acceptance Criteria** (`Microsoft.VSTS.Common.AcceptanceCriteria`), **Description**.
70
+ - `Feature`: **Description**, **Acceptance Criteria** if present.
71
+ - Note the project for all subsequent ADO calls.
72
+ 3. Also call `wit_list_work_item_comments` — comments often contain fail reasons or tester observations referenced in Phase 3 fallbacks.
73
+
74
+ ---
75
+
76
+ ### Phase 2 Resolve the Test Case List
77
+
78
+ Source-resolution order:
79
+
80
+ 1. **Check for a local file** at `ai-tasks/ticket-{id}/test-cases.md` (produced by `/qa-create-test --ado`). If it exists, parse the TC IDs it contains — this is a hint list only.
81
+ 2. **Always query ADO** inspect the relations returned in Phase 1, filter for link type `"Microsoft.VSTS.Common.TestedBy-Forward"` (*Tested By*), and build the authoritative list of linked TC IDs.
82
+ 3. **If both exist and disagree**: trust **ADO**. Log the discrepancy in chat as an FYI (e.g., `"TC#4521 was in local file but not linked in ADO skipped"` or `"TC#4530 is linked in ADO but missing from local file — included"`) but proceed with the ADO list.
83
+ 4. **If no TCs are found in ADO** (and the local file has none either): print `"No test cases linked to work item #{id} — nothing to report."` and stop.
84
+
85
+ For every TC ID in the final list, call `wit_get_work_item` with `expand: "relations"` to capture: **title**, **state**, **steps** (`Microsoft.VSTS.TCM.Steps`), **priority**, **tags**, and any linked runs.
86
+
87
+ ---
88
+
89
+ ### Phase 3 — Resolve Execution Status Per Test Case
90
+
91
+ For each TC, determine a status of **Passed**, **Failed**, or **Blocked** using this order.
92
+
93
+ **Step 1Fetch Test Point outcomes via ADO REST (formal path).**
94
+
95
+ The ADO MCP surface does not expose Test Point outcomes, but the REST API does. Use the same auth pattern as `/qa-create-test --ado`: the `ADO_MCP_AUTH_TOKEN` env var as a PAT, sent via Basic auth.
96
+
97
+ 1. Derive **org** and **project** from the work item context captured in Phase 1 (not hardcoded). The work-item URL / project name on the `wit_get_work_item` response gives you both.
98
+ 2. If `ADO_MCP_AUTH_TOKEN` is **not set**, skip directly to Step 2 (and note in the final report: *"Auto-status skipped — ADO_MCP_AUTH_TOKEN not set."*).
99
+ 3. Otherwise, call the Test Points REST endpoint once per run with the list of linked TC IDs:
100
+
101
+ ```bash
102
+ curl -s \
103
+ --header "Authorization: Basic $(echo -n :${ADO_MCP_AUTH_TOKEN} | base64)" \
104
+ --header "Content-Type: application/json" \
105
+ --request POST \
106
+ --data '{"pointsFilter":{"testcaseIds":[<id1>,<id2>,...]}}' \
107
+ "https://dev.azure.com/{org}/{project}/_apis/test/points?api-version=7.1"
108
+ ```
109
+
110
+ 4. Parse the response. For each returned point you get: `testCase.id`, `testPlan.id`, `suite.id`, `outcome`, `lastTestRun.id`, `lastResultDetails.dateCompleted`, `lastResultDetails.runBy`.
111
+ 5. **If a TC has multiple points** (e.g., it lives in several suites or runs on multiple configurations), pick the point with the **most recent `lastResultDetails.dateCompleted`**.
112
+ 6. Map `outcome` (case-insensitive) to our statuses:
113
+ - `passed` **Passed**
114
+ - `failed` → **Failed**
115
+ - `blocked` **Blocked**
116
+ - `notExecuted` / `none` / `notApplicable` / `paused` / `inProgress` / `warning` / `error` → treat as missing, fall through to Step 2 for this TC
117
+ 7. **If the call fails** (non-2xx, network error, token rejected): log the failure briefly in chat, skip to Step 2 for all TCs, and note in the final report that auto-status was unavailable.
118
+
119
+ **Step 2 — Ask the user** (per TC that didn't resolve in Step 1). Show the TC ID and its synthesized scenario description (from Phase 4) for context:
120
+
121
+ ```
122
+ TC #{id} {scenario description}
123
+ Status? [Passed / Failed / Blocked]
124
+ ```
125
+
126
+ Accept case-insensitive input. Re-prompt if the answer is not one of the three.
127
+
128
+ **Step 3 If the resolved status is Failed**, capture a reason:
129
+
130
+ 1. Check the Test Run data first when Step 1 returned an outcome, also fetch the Test Result for `lastTestRun.id` to read `errorMessage`/`comment`:
131
+ ```bash
132
+ curl -s \
133
+ --header "Authorization: Basic $(echo -n :${ADO_MCP_AUTH_TOKEN} | base64)" \
134
+ "https://dev.azure.com/{org}/{project}/_apis/test/Runs/{runId}/results?api-version=7.1"
135
+ ```
136
+ If the result has a non-empty `errorMessage` or `comment`, use that as the reason.
137
+ 2. Also check the TC's own comments (`wit_list_work_item_comments` for the TC ID) and the parent work item's comments for any line mentioning this TC.
138
+ 3. If no reason is found, ask the user:
139
+ ```
140
+ TC #{id} failed — what was the reason?
141
+ ```
142
+ Accept a free-text one-liner.
143
+
144
+ Keep the per-TC answers (status, reason, and auto/manual source) in memory; do not write them anywhere intermediate.
145
+
146
+ ---
147
+
148
+ ### Phase 4 — Synthesize Scenario Descriptions
149
+
150
+ For each TC, produce a **readable one-line scenario description** for the bullet list. Do **not** copy the raw TC title when it is terse or cryptic — TC titles are often shorthand.
151
+
152
+ Build the description from:
153
+ - TC **title** (starting point)
154
+ - TC **steps** (each step's action + expected result — gives you what was actually verified)
155
+ - Parent work item **description / repro steps / acceptance criteria** (gives you the user-facing phrasing of *what should be true after the fix*)
156
+
157
+ Write the description in the voice of the tested outcome, not the test action. Examples:
158
+
159
+ | Raw TC title | Good scenario description |
160
+ |---|---|
161
+ | `Verify PH row display logic` | `$0 Previous Balance row no longer appears in Payment History` |
162
+ | `Check login 401 scenario` | `Invalid credentials return a 401 with an error banner` |
163
+ | `Regression: limit 100` | `Entry limit enforced at 100 — the 101st entry is rejected` |
164
+
165
+ Keep bullets short — one line, past-tense or assertive present-tense, user-visible wording.
166
+
167
+ ---
168
+
169
+ ### Phase 5 — Build the Report
170
+
171
+ **Markdown version (for chat output):**
172
+
173
+ ```markdown
174
+ Tested Scenarios
175
+
176
+ - ✅ {scenario}
177
+ - {scenario}
178
+ - {scenario}
179
+ - ⚠️ {scenario}
180
+
181
+ Failures
182
+
183
+ - {scenario}: {reason}
184
+ ```
185
+
186
+ Rules:
187
+ - Status emoji: **Passed ✅**, **Failed ❌**, **Blocked ⚠️**
188
+ - Include the `Failures` section **only if at least one TC failed**. One bullet per failure with the captured reason appended after `:`.
189
+ - Do **not** include a "Result" section.
190
+ - Do **not** include TC IDs, priority, tags, or counts in the bullets.
191
+ - Keep order stable preserve the order TCs appeared in the ADO linked list.
192
+
193
+ **HTML version (for the ADO field):**
194
+
195
+ Render with the minimum markup needed for ADO's rich-text field to display bullets and line breaks:
196
+
197
+ ```html
198
+ <b>Tested Scenarios</b><br>
199
+ <ul>
200
+ <li>✅ {scenario}</li>
201
+ <li>✅ {scenario}</li>
202
+ <li>❌ {scenario}</li>
203
+ <li>⚠️ {scenario}</li>
204
+ </ul>
205
+ <b>Failures</b><br>
206
+ <ul>
207
+ <li>❌ {scenario}: {reason}</li>
208
+ </ul>
209
+ ```
210
+
211
+ Rules:
212
+ - Escape scenario text for HTML (`&` → `&amp;`, `<` → `&lt;`, `>` → `&gt;`).
213
+ - Emit the `<b>Failures</b>…` block **only if at least one TC failed**.
214
+ - No inline styles, no classes, no wrapping `<div>`.
215
+
216
+ ---
217
+
218
+ ### Phase 6 — Append to the ADO Field
219
+
220
+ 1. Re-read the current `Custom.QATestCasesReport` value for the work item (it may contain prior runs you must preserve).
221
+ 2. Build the new field value:
222
+ - If the existing value is empty or whitespace: new value = the HTML report from Phase 5.
223
+ - Otherwise: new value = existing content + `<br><br>` + the HTML report from Phase 5.
224
+ 3. Call `wit_update_work_item` setting `Custom.QATestCasesReport` to the new value. Pass only this field — do not include any others in the update.
225
+
226
+ Do **not** overwrite other fields. Do **not** create a local file.
227
+
228
+ ---
229
+
230
+ ### Phase 7 — Present to User
231
+
232
+ Print the **markdown** version of the report to the user in chat. After the report, print a one-line confirmation:
233
+
234
+ ```
235
+ Appended to work item #{id} → Custom.QATestCasesReport.
236
+ ```
237
+
238
+ If any scenarios required a manual status answer or a manual failure reason during Phase 3, append a short note listing how many:
239
+
240
+ ```
241
+ {n} status(es) entered manually · {m} failure reason(s) entered manually.
242
+ ```
243
+
244
+ If the TC list had any local-vs-ADO discrepancies (Phase 2), append a short note listing them.
245
+
246
+ ---
247
+
248
+ ## Notes
249
+
250
+ - This skill does **not** create test cases. Use `/qa-create-test --ado <id>` for that.
251
+ - This skill is read-mostly on the ADO side — it only writes to `Custom.QATestCasesReport`. It does not change TC state, does not create Test Runs, and does not modify links.
252
+ - Scenario descriptions are synthesized, not verbatim TC titles — review the chat output before trusting the field content.
253
+
254
+ $ARGUMENTS
@@ -2,6 +2,37 @@
2
2
 
3
3
  Scan frontend source code, audit missing data-testid attributes, and inject them following the naming convention in CLAUDE.md. Creates a separate branch for the changes.
4
4
 
5
+ ## ⚠ MANDATORY: How to Execute This Command
6
+
7
+ Execute these steps IN ORDER. Every step is mandatory. Do NOT improvise,
8
+ reorder, skip, or invent steps.
9
+
10
+ 1. Read `CLAUDE.md` — the data-testid Convention section — before anything else.
11
+ 2. Execute `workflows/qa-testid.md` end-to-end, in order, without skipping steps.
12
+ 3. Preserve ALL workflow gates. In particular, do NOT bypass:
13
+ - **Framework detection** (workflow Step 2, `detect_frontend`): if no frontend
14
+ directory, no framework, or no component files are found, STOP with the
15
+ workflow's error — do NOT proceed to injection.
16
+ - **Audit checkpoint / user approval** (workflow Step 4, CHECKPOINT phase):
17
+ the testid-injector MUST present TESTID_AUDIT_REPORT.md and wait for the
18
+ user to approve proposed values BEFORE injecting. Do NOT inject without it.
19
+ - **Branch creation** (workflow Step 4 INJECT phase / Step 5,
20
+ `create_branch_and_commit`): injection happens ONLY on the separate branch
21
+ `qa/testid-inject-{YYYY-MM-DD}` — never on the working branch.
22
+ 4. Spawn the scanner and testid-injector via Task() with their
23
+ `<critical_directive>` blocks, as the workflow specifies — do NOT do the
24
+ scan/audit/injection inline.
25
+
26
+ ### DO NOT
27
+ - Inject any data-testid before the user approves the audit report
28
+ - Modify source files on the working branch (injection is branch-isolated)
29
+ - Change application logic — only add data-testid attributes
30
+ - Skip framework detection or proceed when it reports no frontend
31
+ - Invent steps not in the workflow, or reorder them
32
+
33
+ If you find yourself injecting before approval, working off-branch, or skipping
34
+ the framework-detection gate — STOP and restart from step 1.
35
+
5
36
  ## Usage
6
37
 
7
38
  /qa-testid [<source-directory>]
@@ -0,0 +1,54 @@
1
+ {
2
+ "name": "cypress",
3
+ "display_name": "Cypress",
4
+ "languages": [
5
+ "typescript",
6
+ "javascript"
7
+ ],
8
+ "extensions": [
9
+ ".cy.ts",
10
+ ".cy.js"
11
+ ],
12
+ "detection": {
13
+ "manifest_files": [
14
+ "package.json"
15
+ ],
16
+ "manifest_grep": "\"cypress\"",
17
+ "config_files": [
18
+ "cypress.config.ts",
19
+ "cypress.config.js"
20
+ ],
21
+ "file_patterns": [
22
+ "cypress/**/*.cy.ts",
23
+ "cypress/**/*.cy.js"
24
+ ]
25
+ },
26
+ "validation_commands": {
27
+ "syntax_check_ts": "tsc --noEmit",
28
+ "syntax_check_js": "node --check {file}",
29
+ "dry_run": "npx cypress verify",
30
+ "lint": "eslint {file}"
31
+ },
32
+ "conventions": {
33
+ "test_dirs": [
34
+ "cypress/e2e/",
35
+ "cypress/integration/"
36
+ ],
37
+ "pom_dirs": [
38
+ "cypress/support/page-objects/",
39
+ "cypress/pages/"
40
+ ],
41
+ "fixture_dirs": [
42
+ "cypress/fixtures/"
43
+ ],
44
+ "config_filename": "cypress.config.ts"
45
+ },
46
+ "context7_keys": [
47
+ "cypress",
48
+ "cypress-io"
49
+ ],
50
+ "test_kinds": [
51
+ "e2e",
52
+ "component"
53
+ ]
54
+ }
@@ -0,0 +1,59 @@
1
+ {
2
+ "name": "jest",
3
+ "display_name": "Jest",
4
+ "languages": [
5
+ "javascript",
6
+ "typescript"
7
+ ],
8
+ "extensions": [
9
+ ".test.js",
10
+ ".test.ts",
11
+ ".spec.js",
12
+ ".spec.ts"
13
+ ],
14
+ "detection": {
15
+ "manifest_files": [
16
+ "package.json"
17
+ ],
18
+ "manifest_grep": "\"jest\"",
19
+ "config_files": [
20
+ "jest.config.ts",
21
+ "jest.config.js",
22
+ "jest.config.cjs",
23
+ "jest.config.mjs"
24
+ ],
25
+ "file_patterns": [
26
+ "**/*.test.js",
27
+ "**/*.test.ts",
28
+ "__tests__/**/*",
29
+ "tests/**/*.test.*"
30
+ ]
31
+ },
32
+ "validation_commands": {
33
+ "syntax_check_ts": "tsc --noEmit",
34
+ "syntax_check_js": "node --check {file}",
35
+ "dry_run": "jest --listTests",
36
+ "lint": "eslint {file}"
37
+ },
38
+ "conventions": {
39
+ "test_dirs": [
40
+ "__tests__/",
41
+ "tests/",
42
+ "src/__tests__/"
43
+ ],
44
+ "pom_dirs": [],
45
+ "fixture_dirs": [
46
+ "__fixtures__/",
47
+ "tests/fixtures/"
48
+ ],
49
+ "config_filename": "jest.config.ts"
50
+ },
51
+ "context7_keys": [
52
+ "jest",
53
+ "facebook/jest"
54
+ ],
55
+ "test_kinds": [
56
+ "unit",
57
+ "integration"
58
+ ]
59
+ }