@qa-gentic/agents 1.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +203 -0
- package/bin/postinstall.js +75 -0
- package/bin/qa-stlc.js +76 -0
- package/package.json +48 -0
- package/skills/qa-stlc/AGENT-BEHAVIOR.md +373 -0
- package/skills/qa-stlc/deduplication-protocol.md +303 -0
- package/skills/qa-stlc/generate-gherkin.md +550 -0
- package/skills/qa-stlc/generate-playwright-code.md +439 -0
- package/skills/qa-stlc/generate-test-cases.md +176 -0
- package/skills/qa-stlc/write-helix-files.md +349 -0
- package/src/cmd-init.js +84 -0
- package/src/cmd-mcp-config.js +177 -0
- package/src/cmd-skills.js +124 -0
- package/src/cmd-verify.js +129 -0
- package/src/qa_stlc_agents/__init__.py +0 -0
- package/src/qa_stlc_agents/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/__pycache__/server.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/server.py +502 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/tools/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/tools/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/tools/__pycache__/ado_gherkin.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/tools/ado_gherkin.py +854 -0
- package/src/qa_stlc_agents/agent_helix_writer/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_helix_writer/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_helix_writer/__pycache__/server.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_helix_writer/server.py +529 -0
- package/src/qa_stlc_agents/agent_helix_writer/tools/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_helix_writer/tools/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_helix_writer/tools/__pycache__/helix_write.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_helix_writer/tools/helix_write.py +622 -0
- package/src/qa_stlc_agents/agent_playwright_generator/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_playwright_generator/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_playwright_generator/__pycache__/server.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_playwright_generator/server.py +2771 -0
- package/src/qa_stlc_agents/agent_playwright_generator/tools/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_playwright_generator/tools/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_playwright_generator/tools/__pycache__/ado_attach.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_playwright_generator/tools/ado_attach.py +62 -0
- package/src/qa_stlc_agents/agent_test_case_manager/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_test_case_manager/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_test_case_manager/__pycache__/server.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_test_case_manager/server.py +483 -0
- package/src/qa_stlc_agents/agent_test_case_manager/tools/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_test_case_manager/tools/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_test_case_manager/tools/__pycache__/ado_workitem.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_test_case_manager/tools/ado_workitem.py +302 -0
- package/src/qa_stlc_agents/shared/__init__.py +0 -0
- package/src/qa_stlc_agents/shared/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/shared/__pycache__/auth.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/shared/auth.py +119 -0
|
@@ -0,0 +1,373 @@
|
|
|
1
|
+
# Agent Behavior Contract — QA STLC Agents
|
|
2
|
+
|
|
3
|
+
> **This file is authoritative for all coding agents (Claude Code, GitHub Copilot, Cursor,
|
|
4
|
+
> Windsurf, and any LLM operating on this repo).**
|
|
5
|
+
> It defines what agents are and are not permitted to do. Every rule here overrides any
|
|
6
|
+
> inference, "reasonable assumption", or pattern learned from prior context.
|
|
7
|
+
>
|
|
8
|
+
> Version: 1.1 — all rules are explicit; none are inferred.
|
|
9
|
+
> Change log: see bottom of this file.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## 1. Scope Delimitation — What This Repo Does and Does Not Do
|
|
14
|
+
|
|
15
|
+
### What the agents in this repo do
|
|
16
|
+
|
|
17
|
+
| Agent | What it does | What it produces |
|
|
18
|
+
|---|---|---|
|
|
19
|
+
| `qa-test-case-manager` | Fetches ADO work items; creates and links manual test cases | ADO test case work items linked via `TestedBy-Forward` |
|
|
20
|
+
| `qa-gherkin-generator` | Fetches ADO work item hierarchies; validates and attaches `.feature` files | `.feature` files attached to ADO work items |
|
|
21
|
+
| `qa-playwright-generator` | Generates TypeScript Playwright code from Gherkin; attaches files to ADO | `locators.ts`, `*Page.ts`, `*.steps.ts` attached to ADO work items |
|
|
22
|
+
| `qa-helix-writer` | Writes already-generated files to disk at Helix-QA paths | Files on disk — no ADO writes |
|
|
23
|
+
|
|
24
|
+
### What the agents do NOT do
|
|
25
|
+
|
|
26
|
+
- Do NOT make any product decisions (scope, priority, what to test).
|
|
27
|
+
- Do NOT infer missing acceptance criteria, test data, or screen names.
|
|
28
|
+
- Do NOT carry forward any decision from one artifact to the next.
|
|
29
|
+
- Do NOT create local files unless the user explicitly requests a download.
|
|
30
|
+
- Do NOT attach anything to ADO unless the explicit rule for that artifact type is satisfied.
|
|
31
|
+
- Do NOT proceed past a gap or ambiguity — they stop and ask.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## 2. Zero-Inference Rule
|
|
36
|
+
|
|
37
|
+
**An agent must never substitute inference for an explicit instruction.**
|
|
38
|
+
|
|
39
|
+
This applies specifically to:
|
|
40
|
+
|
|
41
|
+
| Situation | Prohibited inference | Required behaviour |
|
|
42
|
+
|---|---|---|
|
|
43
|
+
| User provides a work item ID but no type | Guessing whether it is Epic / Feature / PBI / Bug | Call the fetch tool; let the response `work_item_type` field determine routing |
|
|
44
|
+
| User says "generate everything" | Assuming delivery destinations for each artifact type | Ask per artifact: "Attach Gherkin to ADO? Attach Playwright to ADO? Save locally?" |
|
|
45
|
+
| User confirms Gherkin ADO attachment | Inferring Playwright code should also be attached | Wait for explicit confirmation before calling `attach_code_to_work_item` |
|
|
46
|
+
| User declines creating test cases in ADO | Inferring they also decline Gherkin or Playwright ADO attachment | Treat each artifact delivery as a completely independent decision |
|
|
47
|
+
| User says "yes" to one step | Carrying that "yes" forward to the next step | Seek fresh confirmation for each distinct action |
|
|
48
|
+
| Navigation path not in work item | Inventing a path | Write placeholder `<!-- TODO: confirm screen name -->` and surface to user |
|
|
49
|
+
| Test data not provided | Inventing emails, IDs, file contents | Stop and ask; never hardcode invented data |
|
|
50
|
+
| Work item appears new | Skipping the deduplication protocol | Always run Phase 1 of deduplication-protocol.md before creating anything |
|
|
51
|
+
|
|
52
|
+
---
|
|
53
|
+
|
|
54
|
+
## 3. Explicit Artifact Delivery Rules
|
|
55
|
+
|
|
56
|
+
Every artifact type has exactly one delivery rule. Completing one artifact does **not** trigger delivery of another.
|
|
57
|
+
|
|
58
|
+
### 3A — Gherkin `.feature` files
|
|
59
|
+
|
|
60
|
+
| Condition | Action |
|
|
61
|
+
|---|---|
|
|
62
|
+
| User asks to generate Gherkin | Generate the file; show content to user; **do not attach yet** |
|
|
63
|
+
| `validate_gherkin_content` returns `valid: false` | Fix errors; do not attach |
|
|
64
|
+
| `validate_gherkin_content` returns `valid: true` | **Ask the user**: "Attach this `.feature` file to ADO work item #{id}?" |
|
|
65
|
+
| User confirms attachment | Call `attach_gherkin_to_feature` (Feature path) or `attach_gherkin_to_work_item` (PBI/Bug path) |
|
|
66
|
+
| User declines | Save locally only if user explicitly requests; otherwise present content inline |
|
|
67
|
+
| Work item type is Epic | **HARD STOP** — do not attach to Epic; tell user to specify a child Feature ID |
|
|
68
|
+
|
|
69
|
+
### 3B — Playwright TypeScript files
|
|
70
|
+
|
|
71
|
+
| Condition | Action |
|
|
72
|
+
|---|---|
|
|
73
|
+
| User asks to generate Playwright code | Generate files; show output summary; **do not attach yet** |
|
|
74
|
+
| User has not explicitly requested ADO attachment | Do NOT call `attach_code_to_work_item` |
|
|
75
|
+
| User explicitly requests ADO attachment | Call `attach_code_to_work_item` with net-new delta files only |
|
|
76
|
+
| User previously declined test case creation | This has **no bearing** on Playwright attachment — ask separately |
|
|
77
|
+
| User previously confirmed Gherkin attachment | This has **no bearing** on Playwright attachment — ask separately |
|
|
78
|
+
| Work item type is Epic | **HARD STOP** — return `epic_not_supported`; do not attach |
|
|
79
|
+
|
|
80
|
+
### 3C — Manual test cases (ADO)
|
|
81
|
+
|
|
82
|
+
| Condition | Action |
|
|
83
|
+
|---|---|
|
|
84
|
+
| Work item type is Epic | **HARD STOP** — `fetch_work_item` returns `epic_not_supported`; inform user; do not call `create_and_link_test_cases` |
|
|
85
|
+
| Work item type is Feature | Server returns `confirmation_required: true`; show user the proposed count + Feature title; wait for "yes" |
|
|
86
|
+
| User says "yes" to Feature confirmation | Retry `create_and_link_test_cases` with the **identical arguments** plus `confirmed=true` — do NOT change `work_item_id` or any other parameter |
|
|
87
|
+
| User says "no" or "cancel" to Feature confirmation | Abort entirely; report "No test cases were created." Do NOT retry with a different work item type. |
|
|
88
|
+
| Work item type is PBI or Bug | Run deduplication protocol; create only net-new test cases |
|
|
89
|
+
| `existing_test_cases_count > 0` | Always call `get_linked_test_cases` first to get real titles before generating |
|
|
90
|
+
|
|
91
|
+
### 3D — Local file creation
|
|
92
|
+
|
|
93
|
+
| Condition | Action |
|
|
94
|
+
|---|---|
|
|
95
|
+
| User says "save", "download", or "export to file" | Create local file as requested |
|
|
96
|
+
| User has not requested a local file | Do NOT create one |
|
|
97
|
+
| ADO attachment succeeded | Do NOT also create a local copy unless user asks |
|
|
98
|
+
| ADO attachment failed | Offer the content inline; ask if user wants a local file |
|
|
99
|
+
|
|
100
|
+
### 3E — Helix-QA disk writes
|
|
101
|
+
|
|
102
|
+
| Condition | Action |
|
|
103
|
+
|---|---|
|
|
104
|
+
| User asks to write files to disk | Follow `write-helix-files.md` in full — never skip any step |
|
|
105
|
+
| `write_helix_files` succeeds | Report the deduplication summary to the user |
|
|
106
|
+
| `write_helix_files` fails | **See failure recovery rules below** — do NOT fall back to `create_file` |
|
|
107
|
+
| A locator file already exists on disk (`list_helix_tree` returns it) | Extend that file via `write_helix_files` — **NEVER create a new file** |
|
|
108
|
+
| A step file already exists on disk | Merge into that file via `write_helix_files` — **NEVER create a new file** |
|
|
109
|
+
| User has not explicitly asked for local files | Do NOT call `create_file` or any filesystem tool |
|
|
110
|
+
|
|
111
|
+
**Failure recovery for `write_helix_files`:**
|
|
112
|
+
|
|
113
|
+
| Error type | Root cause | Recovery action |
|
|
114
|
+
|---|---|---|
|
|
115
|
+
| Parenthesis / bracket balance error in a `*.steps.ts` file | Step definitions use regex patterns (`/^I click ([^"]*)/`) — the regex preprocessor strips string literals before counting parens, so capture groups cause a nonzero delta | Call `pre_validate_cucumber_steps` on the steps file — it identifies every offending pattern and provides ready-to-paste Cucumber expression replacements. Fix all flagged patterns, then retry `write_helix_files`. |
|
|
116
|
+
| `OSError` on a specific path | Path does not exist or permission denied | Report path to user; do not fall back to `create_file` |
|
|
117
|
+
| Any other tool error | Unknown | Report the raw error to user verbatim; ask how to proceed |
|
|
118
|
+
|
|
119
|
+
**The `create_file` tool is PROHIBITED for Helix-QA artifacts.** Every file in `src/` that
|
|
120
|
+
belongs to the QA test suite — locators, page objects, step definitions, feature files,
|
|
121
|
+
cucumber config — must be written exclusively via `qa-helix-writer:write_helix_files`.
|
|
122
|
+
Using `create_file` bypasses deduplication, interface-adaptation rewrites, and file routing.
|
|
123
|
+
If `write_helix_files` cannot be made to succeed, surface the blocker to the user rather
|
|
124
|
+
than silently routing around it.
|
|
125
|
+
|
|
126
|
+
| Condition | Action |
|
|
127
|
+
|---|---|
|
|
128
|
+
| Files to write | Always call `list_helix_tree` first to see what already exists |
|
|
129
|
+
| File already exists at target path | Read it first; check for overlap; write only net-new content |
|
|
130
|
+
| `generate_playwright_code` output available | Pass the `files` dict directly to `write_helix_files` — do not reformat |
|
|
131
|
+
|
|
132
|
+
---
|
|
133
|
+
|
|
134
|
+
## 4. Strict Input → Tool → Output Chains
|
|
135
|
+
|
|
136
|
+
Each user request type has one and only one correct tool chain. Do not improvise.
|
|
137
|
+
|
|
138
|
+
### Chain A — Manual test cases from PBI or Bug
|
|
139
|
+
|
|
140
|
+
```
|
|
141
|
+
INPUT: PBI or Bug ID + org URL + project name
|
|
142
|
+
│
|
|
143
|
+
├─ fetch_work_item(id)
|
|
144
|
+
│ └─ if epic_not_supported → STOP, inform user
|
|
145
|
+
│ └─ if confirmation_required → STOP, ask user; proceed only on "yes"
|
|
146
|
+
│
|
|
147
|
+
├─ get_linked_test_cases(id) ← always if count > 0
|
|
148
|
+
│
|
|
149
|
+
├─ [deduplication-protocol Phase 2]
|
|
150
|
+
│
|
|
151
|
+
└─ create_and_link_test_cases(net-new only)
|
|
152
|
+
|
|
153
|
+
OUTPUT: ADO test case IDs + deduplication report
|
|
154
|
+
NO local file, NO Gherkin, NO Playwright — unless user explicitly requests them
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
### Chain B — Gherkin from Feature
|
|
158
|
+
|
|
159
|
+
```
|
|
160
|
+
INPUT: Feature ID + org URL + project name
|
|
161
|
+
│
|
|
162
|
+
├─ fetch_feature_hierarchy(id)
|
|
163
|
+
│
|
|
164
|
+
├─ [deduplication-protocol Phase 1+2]
|
|
165
|
+
│
|
|
166
|
+
├─ [gap check — stop and ask if any G1–G7 gap found]
|
|
167
|
+
│
|
|
168
|
+
├─ [live Playwright MCP snapshots of every screen]
|
|
169
|
+
│
|
|
170
|
+
├─ validate_gherkin_content(content, scope="feature")
|
|
171
|
+
│ └─ if valid: false → fix; re-validate; do NOT attach
|
|
172
|
+
│
|
|
173
|
+
├─ validate_gherkin_steps(content)
|
|
174
|
+
│
|
|
175
|
+
├─ [show content to user]
|
|
176
|
+
│
|
|
177
|
+
├─ [ASK: "Attach to ADO Feature #{id}?"]
|
|
178
|
+
│ └─ yes → attach_gherkin_to_feature(id)
|
|
179
|
+
│ └─ no → stop; offer local download only if user asks
|
|
180
|
+
│
|
|
181
|
+
└─ [STOP — do not proceed to Playwright unless user explicitly asks]
|
|
182
|
+
|
|
183
|
+
OUTPUT: .feature file attached to Feature work item (or shown inline)
|
|
184
|
+
NO Playwright code unless user explicitly requests it
|
|
185
|
+
NO test case creation unless user explicitly requests it
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
### Chain C — Gherkin from PBI or Bug
|
|
189
|
+
|
|
190
|
+
```
|
|
191
|
+
INPUT: PBI or Bug ID + org URL + project name
|
|
192
|
+
│
|
|
193
|
+
├─ fetch_work_item_for_gherkin(id)
|
|
194
|
+
│
|
|
195
|
+
├─ get_linked_test_cases(id) ← context only
|
|
196
|
+
│
|
|
197
|
+
├─ [deduplication-protocol Phase 1+2]
|
|
198
|
+
│
|
|
199
|
+
├─ [gap check]
|
|
200
|
+
│
|
|
201
|
+
├─ [live Playwright MCP snapshots]
|
|
202
|
+
│
|
|
203
|
+
├─ validate_gherkin_content(content, scope="work_item")
|
|
204
|
+
│
|
|
205
|
+
├─ [show content to user]
|
|
206
|
+
│
|
|
207
|
+
├─ [ASK: "Attach to ADO work item #{id}?"]
|
|
208
|
+
│ └─ yes → attach_gherkin_to_work_item(id)
|
|
209
|
+
│ └─ no → stop
|
|
210
|
+
│
|
|
211
|
+
└─ [STOP — do not proceed to Playwright unless user explicitly asks]
|
|
212
|
+
|
|
213
|
+
OUTPUT: .feature file attached to PBI/Bug work item (or shown inline)
|
|
214
|
+
```
|
|
215
|
+
|
|
216
|
+
### Chain D — Gherkin from Epic
|
|
217
|
+
|
|
218
|
+
```
|
|
219
|
+
INPUT: Epic ID + org URL + project name
|
|
220
|
+
│
|
|
221
|
+
├─ fetch_epic_hierarchy(id)
|
|
222
|
+
│ └─ returns: features[] each with child_work_items[] and existing_test_cases[]
|
|
223
|
+
│
|
|
224
|
+
├─ [read ALL child Features and ALL child PBIs/Bugs before writing any Gherkin]
|
|
225
|
+
│
|
|
226
|
+
├─ for each child Feature:
|
|
227
|
+
│ ├─ [deduplication-protocol Phase 1+2 for that Feature ID]
|
|
228
|
+
│ ├─ [gap check for that Feature]
|
|
229
|
+
│ ├─ [live snapshots for that Feature's screens]
|
|
230
|
+
│ ├─ generate .feature file (5–10 scenarios)
|
|
231
|
+
│ ├─ validate_gherkin_content(scope="feature")
|
|
232
|
+
│ ├─ [show content to user]
|
|
233
|
+
│ ├─ [ASK: "Attach .feature to Feature #{child_id}?"]
|
|
234
|
+
│ └─ yes → attach_gherkin_to_feature(child_id)
|
|
235
|
+
│
|
|
236
|
+
└─ [NEVER call create_and_link_test_cases on the Epic ID]
|
|
237
|
+
|
|
238
|
+
OUTPUT: one .feature file per child Feature
|
|
239
|
+
test cases NOT created unless user explicitly asks for child Feature/PBI coverage
|
|
240
|
+
```
|
|
241
|
+
|
|
242
|
+
### Chain E — Playwright TypeScript from Gherkin
|
|
243
|
+
|
|
244
|
+
```
|
|
245
|
+
INPUT: validated .feature content + work item ID + org URL + project name
|
|
246
|
+
│
|
|
247
|
+
├─ [deduplication-protocol Phase 1 — read existing locators, page objects, steps]
|
|
248
|
+
│
|
|
249
|
+
├─ [gap check — identify state-dependent screens; stop and ask if blockers found]
|
|
250
|
+
│
|
|
251
|
+
├─ [live Playwright MCP snapshots through full flow]
|
|
252
|
+
│
|
|
253
|
+
├─ generate_playwright_code(gherkin, context_map)
|
|
254
|
+
│
|
|
255
|
+
├─ [apply delta filter — remove keys / methods / steps already in existing files]
|
|
256
|
+
│
|
|
257
|
+
├─ [show delta summary to user]
|
|
258
|
+
│
|
|
259
|
+
├─ [ASK: "Attach these Playwright files to ADO work item #{id}?"]
|
|
260
|
+
│ └─ yes → attach_code_to_work_item(delta files only)
|
|
261
|
+
│ └─ no → stop; offer local download only if user asks
|
|
262
|
+
│
|
|
263
|
+
└─ [STOP — do not trigger Helix write unless user explicitly asks]
|
|
264
|
+
|
|
265
|
+
OUTPUT: delta Playwright files attached to work item (or shown inline)
|
|
266
|
+
NO Helix disk write unless user explicitly requests it
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
---
|
|
270
|
+
|
|
271
|
+
## 5. Ambiguity Resolution — Stop, Don't Guess
|
|
272
|
+
|
|
273
|
+
When any of the following is unclear, **stop and ask the user**. Do not proceed.
|
|
274
|
+
|
|
275
|
+
| Ambiguity | What to ask |
|
|
276
|
+
|---|---|
|
|
277
|
+
| Work item ID given but type is unknown | Call fetch tool and surface the `work_item_type` field; do not guess |
|
|
278
|
+
| "Generate everything for #123" | "Which artifacts do you want? (A) Manual test cases in ADO (B) Gherkin .feature file (C) Playwright TypeScript (D) All of the above — and which should be attached to ADO vs saved locally?" |
|
|
279
|
+
| User says "attach it" after multiple artifacts generated | "Which artifact are you referring to — the Gherkin .feature file, the Playwright TypeScript files, or both?" |
|
|
280
|
+
| No org URL or project name provided | "Please provide the ADO organisation URL (e.g. `https://dev.azure.com/myorg`) and project name." |
|
|
281
|
+
| No screen name or navigation path available | Use `<!-- TODO: confirm screen name -->` placeholder; surface to user; do not invent |
|
|
282
|
+
| No test data for a state-dependent screen | Stop; ask what file / email / record to use; do not invent |
|
|
283
|
+
| User says "yes" ambiguously after a confirmation prompt | Re-read the most recent prompt; if still unclear, re-ask with the specific action being confirmed |
|
|
284
|
+
|
|
285
|
+
---
|
|
286
|
+
|
|
287
|
+
## 6. No Carry-Over Between Decisions
|
|
288
|
+
|
|
289
|
+
Each of these decisions is **completely independent**. A prior answer to one does not answer another.
|
|
290
|
+
|
|
291
|
+
```
|
|
292
|
+
[1] Generate Gherkin? → answered
|
|
293
|
+
[2] Attach Gherkin to ADO? → must ask SEPARATELY after [1]
|
|
294
|
+
[3] Generate Playwright code? → must ask SEPARATELY
|
|
295
|
+
[4] Attach Playwright to ADO? → must ask SEPARATELY after [3]
|
|
296
|
+
[5] Create test cases in ADO? → must ask SEPARATELY
|
|
297
|
+
[6] Save any artifact locally? → must ask SEPARATELY
|
|
298
|
+
[7] Write to Helix-QA on disk? → must ask SEPARATELY
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
**Declining [5] does not affect [2], [4], [6], or [7].**
|
|
302
|
+
**Confirming [2] does not pre-answer [4].**
|
|
303
|
+
**Saying "yes" at step [1] does not mean "yes" at any other step.**
|
|
304
|
+
|
|
305
|
+
---
|
|
306
|
+
|
|
307
|
+
## 7. Decision Tree — Work Item Type Routing
|
|
308
|
+
|
|
309
|
+
```
|
|
310
|
+
User provides work item ID
|
|
311
|
+
│
|
|
312
|
+
▼
|
|
313
|
+
Call the fetch tool for the intended operation
|
|
314
|
+
(fetch_work_item, fetch_work_item_for_gherkin, or fetch_epic_hierarchy)
|
|
315
|
+
│
|
|
316
|
+
▼
|
|
317
|
+
Read work_item_type from response
|
|
318
|
+
│
|
|
319
|
+
┌────┴─────────────────────────────────────────────┐
|
|
320
|
+
│ │
|
|
321
|
+
"Epic" not "Epic"
|
|
322
|
+
│ │
|
|
323
|
+
▼ ┌─────┴────────┐
|
|
324
|
+
HARD STOP for test cases "Feature" "PBI"/"Bug"
|
|
325
|
+
Use fetch_epic_hierarchy for Gherkin │ │
|
|
326
|
+
Never call create_and_link_test_cases │ │
|
|
327
|
+
Never call attach_code_to_work_item │ │
|
|
328
|
+
on the Epic ID Feature path PBI/Bug path
|
|
329
|
+
(5–10 scenarios) (3–9 scenarios)
|
|
330
|
+
attach_gherkin_to_feature
|
|
331
|
+
attach_gherkin_to_work_item
|
|
332
|
+
```
|
|
333
|
+
|
|
334
|
+
```
|
|
335
|
+
Feature path for test cases
|
|
336
|
+
│
|
|
337
|
+
▼
|
|
338
|
+
create_and_link_test_cases returns confirmation_required: true
|
|
339
|
+
│
|
|
340
|
+
▼
|
|
341
|
+
STOP — show user: Feature title + proposed TC count
|
|
342
|
+
│
|
|
343
|
+
┌────┴────┐
|
|
344
|
+
"yes" "no"/"cancel"
|
|
345
|
+
│ │
|
|
346
|
+
▼ ABORT — report "No test cases created"
|
|
347
|
+
retry create_and_link_test_cases
|
|
348
|
+
with identical args + confirmed=true
|
|
349
|
+
(do NOT change work_item_id or pivot to a child PBI)
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
---
|
|
353
|
+
|
|
354
|
+
## 8. Versioning — Explicit vs Inferred
|
|
355
|
+
|
|
356
|
+
All rules in this file are **explicitly stated**. None are inferred from code patterns,
|
|
357
|
+
prior conversation, or training data.
|
|
358
|
+
|
|
359
|
+
When a rule is changed:
|
|
360
|
+
|
|
361
|
+
1. Update the rule in this file with a dated entry in the change log below.
|
|
362
|
+
2. Update the corresponding skill file in `skills/` to match.
|
|
363
|
+
3. Run `./scripts/install-skills.sh vscode` to sync `.github/copilot-instructions/`.
|
|
364
|
+
4. Update `CLAUDE.md` if the rule affects a key rule number.
|
|
365
|
+
|
|
366
|
+
### Change log
|
|
367
|
+
|
|
368
|
+
| Date | Version | Change | Author |
|
|
369
|
+
|---|---|---|---|
|
|
370
|
+
| 2025-01-01 | 1.0 | Initial explicit rule set — all 8 sections | repo init |
|
|
371
|
+
| 2026-04-03 | 1.1 | 3C: added `confirmed=true` retry protocol for Feature confirmation gate | repo update |
|
|
372
|
+
| 2026-04-03 | 1.1 | 3E: paren error recovery now references `pre_validate_cucumber_steps` as first diagnostic step | repo update |
|
|
373
|
+
| 2026-04-03 | 1.1 | §7: Feature decision tree clarifies retry mechanism (`confirmed=true`, no pivot to child PBI) | repo update |
|
|
@@ -0,0 +1,303 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: deduplication-protocol
|
|
3
|
+
description: >
|
|
4
|
+
MANDATORY pre-flight protocol for every QA STLC agent run. Must be read and applied BEFORE
|
|
5
|
+
creating any test cases, Gherkin feature files, Playwright locators, page objects, or step
|
|
6
|
+
definitions in Azure DevOps. Prevents duplicate test cases, duplicate feature file attachments,
|
|
7
|
+
and duplicate Playwright code being created across multiple runs or agents on the same work item.
|
|
8
|
+
Triggers on any task involving: ADO work items, test cases, feature files, Gherkin BDD, Playwright
|
|
9
|
+
automation, locators, page objects, step definitions, qa-gherkin-generator, qa-test-case-manager,
|
|
10
|
+
qa-playwright-generator tools.
|
|
11
|
+
compatibility:
|
|
12
|
+
tools:
|
|
13
|
+
- qa-test-case-manager:get_linked_test_cases
|
|
14
|
+
- qa-test-case-manager:fetch_work_item
|
|
15
|
+
- qa-test-case-manager:create_and_link_test_cases
|
|
16
|
+
- qa-gherkin-generator:fetch_feature_hierarchy
|
|
17
|
+
- qa-gherkin-generator:attach_gherkin_to_feature
|
|
18
|
+
- qa-playwright-generator:generate_playwright_code
|
|
19
|
+
- qa-playwright-generator:attach_code_to_work_item
|
|
20
|
+
- qa-playwright-generator:validate_gherkin_steps
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
# QA Deduplication Protocol
|
|
24
|
+
|
|
25
|
+
> **Read `AGENT-BEHAVIOR.md` before this protocol.**
|
|
26
|
+
> This protocol is a pre-flight gate only — it does not authorise creating anything.
|
|
27
|
+
> All creation decisions require separate explicit user confirmation per artifact type.
|
|
28
|
+
|
|
29
|
+
> **This protocol is mandatory. No artifact may be created or attached to ADO without completing
|
|
30
|
+
> the READ → DIFF → CREATE-ONLY-WHAT-IS-MISSING pipeline below.**
|
|
31
|
+
>
|
|
32
|
+
> Any agent that skips this protocol and creates duplicates has violated the QA STLC workflow.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## Why This Exists
|
|
37
|
+
|
|
38
|
+
When QA STLC agents run more than once on the same work item — or when multiple agents run in
|
|
39
|
+
sequence on the same item — they have no memory of what previous runs created. Without this
|
|
40
|
+
protocol every run blindly creates new test cases, attaches new feature files, and generates new
|
|
41
|
+
Playwright code that duplicates existing artifacts. This produces:
|
|
42
|
+
|
|
43
|
+
- 2–5× the intended number of ADO test cases per work item
|
|
44
|
+
- Multiple `.feature` files on the same work item covering identical scenarios
|
|
45
|
+
- Multiple versions of `locators.ts` / `PageObject.ts` / `steps.ts` that diverge silently
|
|
46
|
+
- Test suites that fail because the same step definition is registered twice
|
|
47
|
+
|
|
48
|
+
This skill codifies the fix as a durable, enforceable protocol that any agent can read and follow.
|
|
49
|
+
|
|
50
|
+
---
|
|
51
|
+
|
|
52
|
+
## The Three-Phase Mandatory Workflow
|
|
53
|
+
|
|
54
|
+
```
|
|
55
|
+
┌───────────────────────────────────────────────────────────────┐
|
|
56
|
+
│ PHASE 1 — READ (always first, no exceptions) │
|
|
57
|
+
│ PHASE 2 — DIFF (semantic matching, not just string equality) │
|
|
58
|
+
│ PHASE 3 — CREATE only what has no existing coverage │
|
|
59
|
+
└───────────────────────────────────────────────────────────────┘
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## PHASE 1 — READ Everything That Already Exists
|
|
65
|
+
|
|
66
|
+
Before touching any artifact, make all of the following calls **in this order**.
|
|
67
|
+
|
|
68
|
+
This protocol handles **any work item type** — PBI, Bug, or Feature — as follows:
|
|
69
|
+
|
|
70
|
+
| Work item type passed | Step 1A | Step 1B |
|
|
71
|
+
|---|---|---|
|
|
72
|
+
| **PBI or Bug** | `fetch_work_item(id)` → get parent Feature id | `fetch_feature_hierarchy(parent_feature_id)` |
|
|
73
|
+
| **Feature** | Skip `fetch_work_item` | `fetch_feature_hierarchy(id)` directly |
|
|
74
|
+
|
|
75
|
+
### 1A — Fetch the work item (PBI or Bug only)
|
|
76
|
+
|
|
77
|
+
```
|
|
78
|
+
qa-test-case-manager:fetch_work_item(
|
|
79
|
+
organization_url, project_name, work_item_id
|
|
80
|
+
)
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
Extract and store:
|
|
84
|
+
- Title, description, acceptance_criteria
|
|
85
|
+
- **Parent Feature ID** — used in Step 1B
|
|
86
|
+
- Story points, priority, state
|
|
87
|
+
|
|
88
|
+
Skip this step if the ID passed is already a Feature.
|
|
89
|
+
|
|
90
|
+
### 1B — Fetch the parent Feature hierarchy
|
|
91
|
+
|
|
92
|
+
```
|
|
93
|
+
qa-gherkin-generator:fetch_feature_hierarchy(
|
|
94
|
+
organization_url, project_name, feature_id ← parent Feature ID from 1A, or the ID itself if Feature
|
|
95
|
+
)
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Extract and store:
|
|
99
|
+
- Feature title, description, acceptance criteria
|
|
100
|
+
- **All child PBIs and Bugs** with their titles + acceptance criteria
|
|
101
|
+
- All existing test cases across the whole feature
|
|
102
|
+
- All `.feature` file attachments already on the feature
|
|
103
|
+
- Build the **flow map** from sibling work items (see generate-gherkin.md Step 1D)
|
|
104
|
+
|
|
105
|
+
> **Why sibling PBIs matter:** A PBI is one slice of a larger flow defined across multiple
|
|
106
|
+
> work items. Siblings often represent prerequisite or downstream steps. Without reading all
|
|
107
|
+
> of them the agent invents navigation steps and test data already defined elsewhere.
|
|
108
|
+
|
|
109
|
+
### 1C — get_linked_test_cases on the specific work item
|
|
110
|
+
|
|
111
|
+
```
|
|
112
|
+
qa-test-case-manager:get_linked_test_cases(
|
|
113
|
+
organization_url, project_name, work_item_id ← the original ID (PBI, Bug, or Feature)
|
|
114
|
+
)
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
Extract and store:
|
|
118
|
+
- All existing test case IDs, titles (normalised), priority values
|
|
119
|
+
|
|
120
|
+
### 1D — Check for existing Playwright attachments
|
|
121
|
+
|
|
122
|
+
From the feature hierarchy response, check for:
|
|
123
|
+
- `locators.ts` — extract all locator keys already defined
|
|
124
|
+
- `*Page.ts` — extract all method names already defined
|
|
125
|
+
- `*.steps.ts` — extract all step definition strings already registered
|
|
126
|
+
|
|
127
|
+
> If you cannot read an attachment's content, treat it as fully covering its domain and
|
|
128
|
+
> produce NO new file of that type unless you can prove a gap.
|
|
129
|
+
|
|
130
|
+
---
|
|
131
|
+
|
|
132
|
+
## PHASE 2 — DIFF Using Semantic Matching
|
|
133
|
+
|
|
134
|
+
String equality is not enough. Two test cases are **semantically equivalent** (duplicates)
|
|
135
|
+
if they test the same condition on the same subject, even when worded differently.
|
|
136
|
+
|
|
137
|
+
### Normalisation algorithm (apply before comparing)
|
|
138
|
+
|
|
139
|
+
1. Lowercase the full title
|
|
140
|
+
2. Strip tag prefixes: `[smoke]`, `[regression]`, `[a11y]`, `[negative]`, `[boundary]`, etc.
|
|
141
|
+
3. Strip filler words: `the`, `a`, `an`, `is`, `are`, `should`, `will`, `when`, `after`, `during`
|
|
142
|
+
4. Extract the **subject noun** (what is being tested: button, CSV, backend, upload, template…)
|
|
143
|
+
5. Extract the **condition** (visible, absent, downloaded, cleared, rejected, correct…)
|
|
144
|
+
6. If subject + condition match an existing case → **DUPLICATE**, do not create
|
|
145
|
+
|
|
146
|
+
### Semantic coverage matrix
|
|
147
|
+
|
|
148
|
+
```
|
|
149
|
+
For each proposed test case:
|
|
150
|
+
normalise(proposed.title) → (subject, condition)
|
|
151
|
+
for each existing test case:
|
|
152
|
+
normalise(existing.title) → (subject, condition)
|
|
153
|
+
if subject matches AND condition matches → DUPLICATE → skip
|
|
154
|
+
if no match found → NET-NEW → add to creation list
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
Only pass the **net-new** list to `create_and_link_test_cases`.
|
|
158
|
+
|
|
159
|
+
### Gherkin scenario deduplication
|
|
160
|
+
|
|
161
|
+
For each proposed Gherkin scenario:
|
|
162
|
+
1. Extract the scenario title
|
|
163
|
+
2. Apply the same normalisation algorithm
|
|
164
|
+
3. Compare against all scenario titles in existing `.feature` attachments AND existing ADO test case titles
|
|
165
|
+
4. Semantic match → skip; zero net-new → do not attach; some net-new → attach delta file only
|
|
166
|
+
|
|
167
|
+
### Playwright code deduplication
|
|
168
|
+
|
|
169
|
+
- `locators.ts`: only emit keys not already in existing file; if delta empty → skip; if non-empty → attach as `locators.delta.ts`
|
|
170
|
+
- `*Page.ts`: only emit methods not already present; if delta empty → skip; if non-empty → attach as `*Page.delta.ts`
|
|
171
|
+
- `*.steps.ts`: only emit step strings not already registered; NEVER re-register an existing step (causes `Ambiguous step definition` runtime error)
|
|
172
|
+
|
|
173
|
+
---
|
|
174
|
+
|
|
175
|
+
## PHASE 3 — CREATE Only What Is Missing
|
|
176
|
+
|
|
177
|
+
| Diff result | Action |
|
|
178
|
+
|---|---|
|
|
179
|
+
| Zero net-new | **Skip.** Log: `✅ Already fully covered — nothing to create.` |
|
|
180
|
+
| Some net-new, some duplicates | **Create only net-new.** Log which were skipped and why. |
|
|
181
|
+
| All net-new (first run) | **Create all.** Normal flow. |
|
|
182
|
+
|
|
183
|
+
### Mandatory deduplication report
|
|
184
|
+
|
|
185
|
+
Output after every run regardless of whether anything was created:
|
|
186
|
+
|
|
187
|
+
```
|
|
188
|
+
## Deduplication Report — Work Item #<id>
|
|
189
|
+
|
|
190
|
+
### Test Cases
|
|
191
|
+
- Existing: <count> linked
|
|
192
|
+
- Proposed: <count>
|
|
193
|
+
- Duplicates skipped: <count> (<titles>)
|
|
194
|
+
- Net-new created: <count> (<IDs if created>)
|
|
195
|
+
|
|
196
|
+
### Gherkin Feature File
|
|
197
|
+
- Existing attachments: <filenames or "none">
|
|
198
|
+
- Proposed scenarios: <count>
|
|
199
|
+
- Duplicate scenarios skipped: <count> (<titles>)
|
|
200
|
+
- Net-new scenarios: <count>
|
|
201
|
+
- Action: <"Skipped — fully covered" | "Attached delta" | "Attached full (first run)">
|
|
202
|
+
|
|
203
|
+
### Playwright Code
|
|
204
|
+
- Existing locators.ts: <"found — N keys" | "not found">
|
|
205
|
+
- Net-new locator keys: <count> (<list or "none">)
|
|
206
|
+
- Existing page object: <"found — N methods" | "not found">
|
|
207
|
+
- Net-new page methods: <count> (<list or "none">)
|
|
208
|
+
- Existing steps file: <"found — N steps" | "not found">
|
|
209
|
+
- Net-new step definitions: <count> (<list or "none">)
|
|
210
|
+
- Action per file: <"Skipped" | "Attached delta" | "Attached full (first run)">
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
---
|
|
214
|
+
|
|
215
|
+
## Hard Rules — Never Violate These
|
|
216
|
+
|
|
217
|
+
1. **NEVER call `create_and_link_test_cases` without first calling `get_linked_test_cases`** on the same work item.
|
|
218
|
+
2. **NEVER attach a `.feature` file without first checking for existing feature file attachments.**
|
|
219
|
+
3. **NEVER attach a `steps.ts` that re-registers an existing step string** — causes `Ambiguous step definition` at runtime.
|
|
220
|
+
4. **NEVER treat tag differences as meaningful.** `[REGRESSION] Export List downloads CSV` is a duplicate of `[SMOKE] Clicking Export List downloads a valid CSV file`.
|
|
221
|
+
5. **NEVER create more than one test case covering the same (subject, condition) pair.**
|
|
222
|
+
6. **NEVER skip this protocol because the work item appears new.** `existing_test_cases_count: 0` in the hierarchy can still have cases via `get_linked_test_cases` — always call both.
|
|
223
|
+
|
|
224
|
+
---
|
|
225
|
+
|
|
226
|
+
## Integration With Other QA Skills
|
|
227
|
+
|
|
228
|
+
This protocol is a **work-item-scoped pre-flight gate**.
|
|
229
|
+
|
|
230
|
+
- **Each unique `work_item_id` gets exactly one PHASE 1 run.** Findings are cached and reused by every subsequent agent operating on the same item.
|
|
231
|
+
- **A different `work_item_id` always triggers a fresh PHASE 1.** Cache from item A must never be used for item B.
|
|
232
|
+
|
|
233
|
+
```
|
|
234
|
+
work_item_id = 111 → PHASE 1 runs in full → CACHE[111] populated
|
|
235
|
+
generate-gherkin on 111 → reads CACHE[111], skips PHASE 1
|
|
236
|
+
generate-playwright on 111 → reads CACHE[111], skips PHASE 1
|
|
237
|
+
|
|
238
|
+
work_item_id = 222 → PHASE 1 runs in full → CACHE[222] populated (CACHE[111] untouched)
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
Skills that delegate here:
|
|
242
|
+
```
|
|
243
|
+
skills/generate-gherkin.md → delegates here; reads work-item cache
|
|
244
|
+
skills/generate-playwright-code.md → delegates here; reads work-item cache
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
### Work-Item Cache Schema
|
|
248
|
+
|
|
249
|
+
```
|
|
250
|
+
CACHE[work_item_id] = {
|
|
251
|
+
work_item: { id, type, title, acceptance_criteria }, # PBI/Bug/Feature
|
|
252
|
+
parent_feature: { id, title, description },
|
|
253
|
+
sibling_pbis: [{ id, type, title, acceptance_criteria, state }],
|
|
254
|
+
flow_map: <assembled string describing the full user journey>,
|
|
255
|
+
existing_test_cases: [{ id, title, priority }],
|
|
256
|
+
existing_attachments: {
|
|
257
|
+
feature_files: [{ name, content }],
|
|
258
|
+
locators_ts: { found: bool, keys: [] },
|
|
259
|
+
page_objects: [{ name, methods: [] }],
|
|
260
|
+
steps_files: [{ name, step_strings: [] }],
|
|
261
|
+
},
|
|
262
|
+
gap_check_completed: bool, # set true after generate-gherkin Step 2 passes
|
|
263
|
+
phase1_completed: true,
|
|
264
|
+
}
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
**Before running PHASE 1**, check `CACHE[work_item_id].phase1_completed`:
|
|
268
|
+
- If **true** → skip PHASE 1 entirely; use cached data for PHASE 2.
|
|
269
|
+
- If **false / not set** → run PHASE 1 in full and populate the cache.
|
|
270
|
+
|
|
271
|
+
---
|
|
272
|
+
|
|
273
|
+
## Examples
|
|
274
|
+
|
|
275
|
+
### Multiple work items in one session
|
|
276
|
+
|
|
277
|
+
```
|
|
278
|
+
► PBI #111 processed:
|
|
279
|
+
CACHE[111] not found → PHASE 1 in full → CACHE[111].phase1_completed = true
|
|
280
|
+
generate-gherkin on 111 → reads CACHE[111]
|
|
281
|
+
generate-playwright on 111 → reads CACHE[111], skips all ADO reads
|
|
282
|
+
|
|
283
|
+
► Bug #222 processed:
|
|
284
|
+
CACHE[222] not found → PHASE 1 in full → CACHE[222].phase1_completed = true
|
|
285
|
+
(CACHE[111] untouched — completely independent)
|
|
286
|
+
```
|
|
287
|
+
|
|
288
|
+
### Second run on a fully covered item
|
|
289
|
+
|
|
290
|
+
```
|
|
291
|
+
PHASE 1: get_linked_test_cases(273440) → 35 cases; fetch_feature_hierarchy → .feature attached
|
|
292
|
+
PHASE 2: 14 proposed → 14/14 duplicates; 0 net-new locators, methods, steps
|
|
293
|
+
PHASE 3: Skip all
|
|
294
|
+
REPORT: ✅ Work item #273440 fully covered. Nothing to create.
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
### Partial gap run
|
|
298
|
+
|
|
299
|
+
```
|
|
300
|
+
PHASE 1: 3 cases found, no .feature, no Playwright attachments
|
|
301
|
+
PHASE 2: 5 proposed → 3 duplicates → 2 net-new
|
|
302
|
+
PHASE 3: create 2 test cases; attach full .feature (first run); attach full Playwright files
|
|
303
|
+
```
|