@qa-gentic/agents 1.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +203 -0
- package/bin/postinstall.js +75 -0
- package/bin/qa-stlc.js +76 -0
- package/package.json +48 -0
- package/skills/qa-stlc/AGENT-BEHAVIOR.md +373 -0
- package/skills/qa-stlc/deduplication-protocol.md +303 -0
- package/skills/qa-stlc/generate-gherkin.md +550 -0
- package/skills/qa-stlc/generate-playwright-code.md +439 -0
- package/skills/qa-stlc/generate-test-cases.md +176 -0
- package/skills/qa-stlc/write-helix-files.md +349 -0
- package/src/cmd-init.js +84 -0
- package/src/cmd-mcp-config.js +177 -0
- package/src/cmd-skills.js +124 -0
- package/src/cmd-verify.js +129 -0
- package/src/qa_stlc_agents/__init__.py +0 -0
- package/src/qa_stlc_agents/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/__pycache__/server.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/server.py +502 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/tools/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/tools/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/tools/__pycache__/ado_gherkin.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_gherkin_generator/tools/ado_gherkin.py +854 -0
- package/src/qa_stlc_agents/agent_helix_writer/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_helix_writer/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_helix_writer/__pycache__/server.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_helix_writer/server.py +529 -0
- package/src/qa_stlc_agents/agent_helix_writer/tools/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_helix_writer/tools/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_helix_writer/tools/__pycache__/helix_write.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_helix_writer/tools/helix_write.py +622 -0
- package/src/qa_stlc_agents/agent_playwright_generator/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_playwright_generator/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_playwright_generator/__pycache__/server.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_playwright_generator/server.py +2771 -0
- package/src/qa_stlc_agents/agent_playwright_generator/tools/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_playwright_generator/tools/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_playwright_generator/tools/__pycache__/ado_attach.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_playwright_generator/tools/ado_attach.py +62 -0
- package/src/qa_stlc_agents/agent_test_case_manager/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_test_case_manager/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_test_case_manager/__pycache__/server.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_test_case_manager/server.py +483 -0
- package/src/qa_stlc_agents/agent_test_case_manager/tools/__init__.py +0 -0
- package/src/qa_stlc_agents/agent_test_case_manager/tools/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_test_case_manager/tools/__pycache__/ado_workitem.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/agent_test_case_manager/tools/ado_workitem.py +302 -0
- package/src/qa_stlc_agents/shared/__init__.py +0 -0
- package/src/qa_stlc_agents/shared/__pycache__/__init__.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/shared/__pycache__/auth.cpython-310.pyc +0 -0
- package/src/qa_stlc_agents/shared/auth.py +119 -0
|
@@ -0,0 +1,439 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: generate-playwright-code
|
|
3
|
+
description: >
|
|
4
|
+
Use this skill whenever the user asks to generate, create, or update Playwright TypeScript
|
|
5
|
+
automation code for a Feature, PBI, or Bug. Triggers when the user mentions "Playwright",
|
|
6
|
+
"page object", "locators", "step definitions", "automation", "self-healing", "locator
|
|
7
|
+
repository", "TimingHealer", "VisualIntentChecker", or asks to automate a Gherkin feature
|
|
8
|
+
file. Always reads existing attached Playwright files from the ADO work item before generating
|
|
9
|
+
anything, to avoid duplicate locators, page methods, and step definitions. Works with any ADO
|
|
10
|
+
work item type: PBI, Bug, or Feature ID.
|
|
11
|
+
compatibility:
|
|
12
|
+
tools:
|
|
13
|
+
- qa-playwright-generator:generate_playwright_code
|
|
14
|
+
- qa-playwright-generator:attach_code_to_work_item
|
|
15
|
+
- qa-playwright-generator:validate_gherkin_steps
|
|
16
|
+
- qa-test-case-manager:get_linked_test_cases
|
|
17
|
+
- qa-gherkin-generator:fetch_feature_hierarchy
|
|
18
|
+
- playwright:browser_navigate
|
|
19
|
+
- playwright:browser_snapshot
|
|
20
|
+
- playwright:browser_fill_form
|
|
21
|
+
- playwright:browser_click
|
|
22
|
+
- playwright:browser_file_upload
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
# Generate Playwright Code Skill
|
|
26
|
+
|
|
27
|
+
> **Read `AGENT-BEHAVIOR.md` before this skill.** The behavior rules there override any
|
|
28
|
+
> inference. This skill provides the step-by-step workflow only.
|
|
29
|
+
|
|
30
|
+
Generate production-quality, three-layer self-healing Playwright TypeScript automation from a
|
|
31
|
+
validated Gherkin feature file, using live Playwright MCP snapshots for verified locators.
|
|
32
|
+
Works with any ADO work item type: PBI, Bug, or Feature ID.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## Three Healing Layers
|
|
37
|
+
|
|
38
|
+
Every generated page object implements three layers of self-healing:
|
|
39
|
+
|
|
40
|
+
| Layer | Class | What it heals |
|
|
41
|
+
|---|---|---|
|
|
42
|
+
| 1 — Locator | `LocatorHealer` | primary selector → role → label → text → AI Vision → CDPSession AX tree |
|
|
43
|
+
| 2 — Timing | `TimingHealer` | network-trace drift → auto-adjusted timeouts → HealingDashboard |
|
|
44
|
+
| 3 — Visual | `VisualIntentChecker` | element screenshot diff at assertions → HealingDashboard |
|
|
45
|
+
|
|
46
|
+
Healed selectors are persisted in `LocatorRepository` — zero overhead on repeat runs.
|
|
47
|
+
HealingDashboard runs at `http://localhost:7890` during test execution for approve/reject.
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## ⛔ Mandatory Pre-Flight: Deduplication Protocol
|
|
52
|
+
|
|
53
|
+
Before generating or attaching any file, run the deduplication protocol:
|
|
54
|
+
`skills/deduplication-protocol.md`
|
|
55
|
+
|
|
56
|
+
The protocol is **work-item-scoped**: PHASE 1 runs once per `work_item_id` and results are
|
|
57
|
+
cached. If `generate-gherkin` already ran for this `work_item_id`, skip PHASE 1 and read from
|
|
58
|
+
`CACHE[work_item_id]` directly. A different `work_item_id` always triggers a fresh PHASE 1.
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
## Step-by-Step Workflow
|
|
63
|
+
|
|
64
|
+
### Step 1 — Load existing Playwright artifacts from cache
|
|
65
|
+
|
|
66
|
+
Read from `CACHE[work_item_id]` (populated by the deduplication protocol pre-flight):
|
|
67
|
+
|
|
68
|
+
```
|
|
69
|
+
CACHE[work_item_id].existing_attachments.locators_ts → existing locator keys
|
|
70
|
+
CACHE[work_item_id].existing_attachments.page_objects → existing method names
|
|
71
|
+
CACHE[work_item_id].existing_attachments.steps_files → existing step strings
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
If a file exists but content could not be read, treat it as fully covering its domain and
|
|
75
|
+
produce no new file of that type unless you can prove a gap.
|
|
76
|
+
|
|
77
|
+
---
|
|
78
|
+
|
|
79
|
+
### Step 2 — GAP CHECK: identify screens you cannot reach and ASK before proceeding
|
|
80
|
+
|
|
81
|
+
Before navigating the live app, scan the Gherkin feature file for every `Given` step that
|
|
82
|
+
requires a specific app state — a state only reachable by performing a real action (uploading
|
|
83
|
+
a file, submitting a form, triggering an error, completing a multi-step flow).
|
|
84
|
+
|
|
85
|
+
For each such step, ask: **can I reach this screen right now with the information I have?**
|
|
86
|
+
|
|
87
|
+
#### Blockers that require a user question before proceeding
|
|
88
|
+
|
|
89
|
+
| Blocker | Example | What to ask |
|
|
90
|
+
|---|---|---|
|
|
91
|
+
| File upload required to reach a screen | "Given I am on the failure summary screen" — only after uploading a CSV with bad rows | "Please provide the CSV file or tell me which emails to use as known duplicates" |
|
|
92
|
+
| Specific record / ID required | "Given work item #12345 has linked test cases" | "Which org / record should I test against?" |
|
|
93
|
+
| Error state requires specific bad data | "Given the upload is rejected for unsupported format" | "Should I use .txt or .pdf to test the rejection?" |
|
|
94
|
+
| Multi-step prerequisite from a sibling PBI | "Given the mass add has completed" | "Should I replay the upload step, or is there a seed/shortcut?" |
|
|
95
|
+
| Screen only exists after a backend event | "Given the processing is complete" | "Is there a test endpoint I can hit to seed this state?" |
|
|
96
|
+
|
|
97
|
+
**If any blocker is found, output a question block and wait for answers.**
|
|
98
|
+
|
|
99
|
+
#### Question format
|
|
100
|
+
|
|
101
|
+
```
|
|
102
|
+
Before I navigate the live app to capture locators for "<feature title>", I need:
|
|
103
|
+
|
|
104
|
+
**[Blocker type]: [screen or element name]**
|
|
105
|
+
[One sentence describing what the Gherkin step requires]
|
|
106
|
+
→ [Specific question — what file, what email, what record, what sequence?]
|
|
107
|
+
|
|
108
|
+
I will not proceed until these are answered, to avoid inventing selectors
|
|
109
|
+
(stability: 0) for screens I cannot reach.
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
**Only proceed to Step 3 once every blocker is resolved, OR the user explicitly grants
|
|
113
|
+
sign-off to proceed with known `stability: 0` limitations documented.**
|
|
114
|
+
|
|
115
|
+
---
|
|
116
|
+
|
|
117
|
+
### Step 3 — Live locator verification through the FULL flow
|
|
118
|
+
|
|
119
|
+
Navigate through **every screen in the Gherkin feature** — including state-dependent screens —
|
|
120
|
+
using the real test data provided in Step 2. Never snapshot only the landing page and infer.
|
|
121
|
+
|
|
122
|
+
```
|
|
123
|
+
playwright:browser_navigate(APP_BASE_URL)
|
|
124
|
+
playwright:browser_snapshot() ← login screen
|
|
125
|
+
|
|
126
|
+
playwright:browser_fill_form(fields: [
|
|
127
|
+
{ name: "Email", type: "textbox", ref: "<ref>", value: "<email>" },
|
|
128
|
+
{ name: "Password", type: "textbox", ref: "<ref>", value: "<password>" }
|
|
129
|
+
])
|
|
130
|
+
playwright:browser_click(ref: "<login-button-ref>")
|
|
131
|
+
playwright:browser_snapshot() ← dashboard — verify logged in
|
|
132
|
+
|
|
133
|
+
← Navigate through EVERY prerequisite step to reach state-dependent screens
|
|
134
|
+
playwright:browser_click(...) ← e.g. "Add Users" button
|
|
135
|
+
playwright:browser_click(...) ← e.g. "Add Multiple" tab
|
|
136
|
+
playwright:browser_snapshot() ← capture screen locators
|
|
137
|
+
|
|
138
|
+
← Upload real test file (from user's Step 2 answer)
|
|
139
|
+
playwright:browser_file_upload(
|
|
140
|
+
files: ["<real file path>"],
|
|
141
|
+
element: "file upload input",
|
|
142
|
+
ref: "<ref from snapshot>"
|
|
143
|
+
)
|
|
144
|
+
playwright:browser_snapshot() ← verify file attached, Next enabled
|
|
145
|
+
|
|
146
|
+
playwright:browser_click(...) ← "Next" / Submit
|
|
147
|
+
playwright:browser_snapshot() ← capture processing screen (if any)
|
|
148
|
+
playwright:browser_snapshot() ← capture state-dependent result screen
|
|
149
|
+
← capture ALL interactive elements here
|
|
150
|
+
|
|
151
|
+
playwright:browser_click(...) ← any further actions (Export, Accept, etc.)
|
|
152
|
+
playwright:browser_snapshot() ← capture post-action state
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
**Rule:** Every locator in the generated `locators.ts` must appear in one of these snapshots.
|
|
156
|
+
If a screen cannot be reached because test data is missing, go back to Step 2 and ask — do
|
|
157
|
+
not assign `stability: 0` and continue silently.
|
|
158
|
+
|
|
159
|
+
---
|
|
160
|
+
|
|
161
|
+
### Step 4 — Build the context_map
|
|
162
|
+
|
|
163
|
+
From each snapshot, map AX tree nodes to locator entries using this stability rank:
|
|
164
|
+
|
|
165
|
+
| Selector source | Stability |
|
|
166
|
+
|---|---|
|
|
167
|
+
| `data-testid` attribute | 100 |
|
|
168
|
+
| `aria-role` + accessible name | 90 |
|
|
169
|
+
| `id` attribute | 80 |
|
|
170
|
+
| `aria-label` | 70 |
|
|
171
|
+
| `placeholder` text | 60 |
|
|
172
|
+
| Gherkin-inferred (no live verification) | 0 — `⚠ NOT VERIFIED` comment required |
|
|
173
|
+
|
|
174
|
+
Only include keys NOT already present in the existing `locators.ts` (from Step 1).
|
|
175
|
+
|
|
176
|
+
**Example context_map (built from Playwright MCP AX snapshots):**
|
|
177
|
+
```json
|
|
178
|
+
{
|
|
179
|
+
"emailInput": { "selector": "[data-testid='auth0_login-input-email']", "intent": "Auth0 login email field", "stability": 100 },
|
|
180
|
+
"loginButton": { "selector": "[data-testid='auth0_login-button-login']", "intent": "Submit login credentials", "stability": 100 },
|
|
181
|
+
"addUsersButton": { "selector": "[data-testid='person_header-button-add_users']", "intent": "Add Users button, opens slide-out", "stability": 100 },
|
|
182
|
+
"addMultipleTab": { "selector": "[data-testid='person_slider-tab-add_multiple']", "intent": "Add Multiple tab in Add Users panel", "stability": 100 },
|
|
183
|
+
"fileInput": { "selector": "input[type='file']", "intent": "Hidden file input for CSV upload", "stability": 70 },
|
|
184
|
+
"nextButton": { "selector": "button:has-text('Next')", "intent": "Proceed after file upload", "stability": 90 },
|
|
185
|
+
"exportListButton": { "selector": "button:has-text('Export List')", "intent": "Export failed users CSV", "stability": 90, "visualIntent": true },
|
|
186
|
+
"failureSummaryTable":{ "selector": "table", "intent": "Failed user rows table", "stability": 80, "visualIntent": true }
|
|
187
|
+
}
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
---
|
|
191
|
+
|
|
192
|
+
### Step 5 — Generate the code
|
|
193
|
+
|
|
194
|
+
```
|
|
195
|
+
qa-playwright-generator:generate_playwright_code(
|
|
196
|
+
gherkin_content : <validated .feature file content>,
|
|
197
|
+
page_class_name : "<ScreenName>Page",
|
|
198
|
+
app_name : "<AppName>",
|
|
199
|
+
context_map : <context_map from Step 4>,
|
|
200
|
+
healing_strategy : "role-label-text-ai",
|
|
201
|
+
enable_timing_healing : true,
|
|
202
|
+
enable_visual_regression : true
|
|
203
|
+
)
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
The tool produces four files:
|
|
207
|
+
- `locators.ts` — selector registry with intent, stability score, visualIntent flag
|
|
208
|
+
- `{ScreenName}Page.ts` — three-layer self-healing page object
|
|
209
|
+
- `{feature}.steps.ts` — Cucumber step definitions
|
|
210
|
+
- `cucumber-profile.js` snippet — profile config to add to `config/cucumber.js`
|
|
211
|
+
|
|
212
|
+
---
|
|
213
|
+
|
|
214
|
+
### Step 6 — Apply the deduplication delta filter
|
|
215
|
+
|
|
216
|
+
Diff generated output against `CACHE[work_item_id].existing_attachments`:
|
|
217
|
+
|
|
218
|
+
**locators.ts:** Remove keys already in existing file. Empty delta → skip. Non-empty → attach as `locators.delta.ts`:
|
|
219
|
+
```typescript
|
|
220
|
+
// DELTA — merge these keys into the existing locators.ts
|
|
221
|
+
// Generated: <date> | Work item: #<id>
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
**{ScreenName}Page.ts:** Remove methods already in existing page object. Empty delta → skip. Non-empty → attach as `{ScreenName}Page.delta.ts` with merge header.
|
|
225
|
+
|
|
226
|
+
**{feature}.steps.ts:** Remove any `Given(` / `When(` / `Then(` whose step string already exists. Empty delta → skip. Non-empty → attach only net-new steps.
|
|
227
|
+
**CRITICAL: Never re-register an existing step string** — causes `Ambiguous step definition` at runtime.
|
|
228
|
+
|
|
229
|
+
---
|
|
230
|
+
|
|
231
|
+
### Step 7 — Delivery gate (mandatory before attaching to ADO)
|
|
232
|
+
|
|
233
|
+
Before calling `attach_code_to_work_item`, present the following to the user and wait:
|
|
234
|
+
|
|
235
|
+
```
|
|
236
|
+
I've generated the following Playwright TypeScript files for work item #{id}:
|
|
237
|
+
|
|
238
|
+
📄 locators.delta.ts — {N} new locator keys
|
|
239
|
+
📄 {ScreenName}Page.delta.ts — {N} new page methods
|
|
240
|
+
📄 {feature}.steps.delta.ts — {N} new step definitions
|
|
241
|
+
|
|
242
|
+
**Do you want me to attach these to ADO work item #{id}? (yes / no)**
|
|
243
|
+
|
|
244
|
+
Note:
|
|
245
|
+
- Saying yes here only attaches the Playwright files.
|
|
246
|
+
- It does NOT create manual test cases or attach Gherkin files.
|
|
247
|
+
- If you also want those, say so separately.
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
**Do not call `attach_code_to_work_item` until the user replies "yes".**
|
|
251
|
+
|
|
252
|
+
If user says "yes", call with delta files only:
|
|
253
|
+
|
|
254
|
+
```
|
|
255
|
+
qa-playwright-generator:attach_code_to_work_item(
|
|
256
|
+
organization_url : "<org_url>",
|
|
257
|
+
project_name : "<project>",
|
|
258
|
+
work_item_id : <id>,
|
|
259
|
+
files : [
|
|
260
|
+
{ file_name: "locators.delta.ts", content: "..." },
|
|
261
|
+
{ file_name: "{ScreenName}Page.delta.ts", content: "..." },
|
|
262
|
+
{ file_name: "{feature}.steps.delta.ts", content: "..." }
|
|
263
|
+
]
|
|
264
|
+
)
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
If user says "no": present the files inline. Offer local download only if user says
|
|
268
|
+
"save" or "download". Do not create a local file automatically.
|
|
269
|
+
|
|
270
|
+
**After attach — STOP. Do not trigger Helix write or any other action automatically.**
|
|
271
|
+
Helix-QA disk writes are a separate decision requiring a separate user request.
|
|
272
|
+
|
|
273
|
+
---
|
|
274
|
+
|
|
275
|
+
## Generated File Structures
|
|
276
|
+
|
|
277
|
+
### locators.ts
|
|
278
|
+
|
|
279
|
+
```typescript
|
|
280
|
+
/**
|
|
281
|
+
* Locators: <ScreenName>Page
|
|
282
|
+
* SOURCE: Playwright MCP AX tree snapshot — <app-url>/<route>
|
|
283
|
+
* Stability: data-testid(100) > role+name(90) > id(80) > aria-label(70) > placeholder(60)
|
|
284
|
+
* ⚠ stability:0 entries were NOT reached in the live session — get user sign-off before using.
|
|
285
|
+
*/
|
|
286
|
+
export const <ScreenName>Locators = {
|
|
287
|
+
// stability: 100 — data-testid, survives refactors
|
|
288
|
+
addUsersButton: {
|
|
289
|
+
selector : "[data-testid='person_header-button-add_users']",
|
|
290
|
+
intent : "Add Users button that opens the slide-out panel",
|
|
291
|
+
stability : 100,
|
|
292
|
+
},
|
|
293
|
+
// stability: 90 — aria-role+name, visualIntent: screenshot at assertions
|
|
294
|
+
exportListButton: {
|
|
295
|
+
selector : "button:has-text('Export List')",
|
|
296
|
+
intent : "Export List button on the failure summary screen",
|
|
297
|
+
stability : 90,
|
|
298
|
+
visualIntent : true,
|
|
299
|
+
},
|
|
300
|
+
// ⚠ stability: 0 — NOT VERIFIED: could not reach this screen (reason: <why>)
|
|
301
|
+
// Get user sign-off before using in automation.
|
|
302
|
+
someUnreachedElement: {
|
|
303
|
+
selector : "[data-testid='TODO']",
|
|
304
|
+
intent : "...",
|
|
305
|
+
stability : 0,
|
|
306
|
+
},
|
|
307
|
+
} as const;
|
|
308
|
+
|
|
309
|
+
export type LocatorKey = keyof typeof <ScreenName>Locators;
|
|
310
|
+
```
|
|
311
|
+
|
|
312
|
+
### {ScreenName}Page.ts
|
|
313
|
+
|
|
314
|
+
```typescript
|
|
315
|
+
import { Page, expect, Download } from '@playwright/test';
|
|
316
|
+
import { fixture } from '@hooks/pageFixture';
|
|
317
|
+
import { <ScreenName>Locators } from './locators';
|
|
318
|
+
import { LocatorHealer } from '@utils/locators/LocatorHealer';
|
|
319
|
+
import { LocatorRepository } from '@utils/locators/LocatorRepository';
|
|
320
|
+
import { VisualIntentChecker } from '@utils/locators/VisualIntentChecker';
|
|
321
|
+
import { TimingHealer } from '@utils/locators/TimingHealer';
|
|
322
|
+
|
|
323
|
+
/**
|
|
324
|
+
* <ScreenName>Page — Three-Layer Self-Healing Page Object
|
|
325
|
+
* Layer 1 (LocatorHealer): primary → role → label → text → AI Vision → AX tree
|
|
326
|
+
* Layer 2 (TimingHealer): network-trace drift → auto-adjusted timeouts
|
|
327
|
+
* Layer 3 (VisualIntentChecker): element screenshot diff at key assertions
|
|
328
|
+
* HealingDashboard: http://localhost:7890
|
|
329
|
+
* RULE: Never use page.click / page.fill / page.locator directly.
|
|
330
|
+
*/
|
|
331
|
+
export default class <ScreenName>Page {
|
|
332
|
+
private readonly loc = <ScreenName>Locators;
|
|
333
|
+
private readonly page: Page;
|
|
334
|
+
private readonly healer: LocatorHealer;
|
|
335
|
+
private readonly repo: LocatorRepository;
|
|
336
|
+
private readonly visual: VisualIntentChecker;
|
|
337
|
+
private readonly timing: TimingHealer;
|
|
338
|
+
|
|
339
|
+
constructor(page?: Page) {
|
|
340
|
+
this.page = page ?? fixture().page;
|
|
341
|
+
this.repo = fixture().locatorRepository ?? new LocatorRepository();
|
|
342
|
+
this.healer = new LocatorHealer(this.page, fixture().logger, this.repo);
|
|
343
|
+
this.visual = new VisualIntentChecker(this.page, fixture().logger, this.repo);
|
|
344
|
+
this.timing = new TimingHealer(this.page, fixture().logger, this.repo);
|
|
345
|
+
Object.entries(this.loc).forEach(([k, v]) => this.repo.register(k, v.selector, v.intent));
|
|
346
|
+
}
|
|
347
|
+
|
|
348
|
+
async login(email: string, password: string): Promise<void> {
|
|
349
|
+
await this.page.goto(process.env.APP_BASE_URL!, { waitUntil: 'networkidle' });
|
|
350
|
+
await this.healer.fillWithHealing('emailInput', this.loc.emailInput.selector, email, this.loc.emailInput.intent);
|
|
351
|
+
await this.healer.fillWithHealing('passwordInput', this.loc.passwordInput.selector, password, this.loc.passwordInput.intent);
|
|
352
|
+
await this.healer.clickWithHealing('loginButton', this.loc.loginButton.selector, this.loc.loginButton.intent);
|
|
353
|
+
await this.timing.waitForNetworkIdle('login');
|
|
354
|
+
}
|
|
355
|
+
|
|
356
|
+
// All interactions go through LocatorHealer — never raw page.click / page.fill
|
|
357
|
+
}
|
|
358
|
+
```
|
|
359
|
+
|
|
360
|
+
### {feature}.steps.ts
|
|
361
|
+
|
|
362
|
+
```typescript
|
|
363
|
+
import { Given, When, Then, Before } from '@cucumber/cucumber';
|
|
364
|
+
import { expect } from '@playwright/test';
|
|
365
|
+
import { fixture } from '@hooks/pageFixture';
|
|
366
|
+
import <ScreenName>Page from '@pages/<screenName>/<ScreenName>Page';
|
|
367
|
+
|
|
368
|
+
let page_: <ScreenName>Page;
|
|
369
|
+
|
|
370
|
+
Before(async () => { page_ = new <ScreenName>Page(fixture().page); });
|
|
371
|
+
|
|
372
|
+
Given('I am logged in to {word} as {string}', async (_app: string, email: string) => {
|
|
373
|
+
await page_.login(email, process.env.APP_PASSWORD!);
|
|
374
|
+
});
|
|
375
|
+
|
|
376
|
+
// Only net-new steps — never re-register steps already in existing files
|
|
377
|
+
```
|
|
378
|
+
|
|
379
|
+
---
|
|
380
|
+
|
|
381
|
+
## Run Commands
|
|
382
|
+
|
|
383
|
+
```bash
|
|
384
|
+
ENABLE_SELF_HEALING=true \
|
|
385
|
+
HEALING_DASHBOARD_PORT=7890 \
|
|
386
|
+
APP_BASE_URL=https://account.myamplify.io \
|
|
387
|
+
APP_EMAIL=<email> \
|
|
388
|
+
APP_PASSWORD=<password> \
|
|
389
|
+
cucumber-js --config=config/cucumber.js -p <feature_profile>
|
|
390
|
+
|
|
391
|
+
# Smoke only:
|
|
392
|
+
cucumber-js --config=config/cucumber.js -p <feature_profile> --tags "@smoke"
|
|
393
|
+
```
|
|
394
|
+
|
|
395
|
+
---
|
|
396
|
+
|
|
397
|
+
## Quality Gates — Do Not Attach Without These
|
|
398
|
+
|
|
399
|
+
- [ ] Deduplication protocol run — existing locator keys, methods, step strings extracted
|
|
400
|
+
- [ ] **Gap check (Step 2) completed** — every state-dependent screen identified
|
|
401
|
+
- [ ] **All blockers answered by user before any navigation began**
|
|
402
|
+
- [ ] **Live snapshots taken of every screen in the full flow** using real test data
|
|
403
|
+
- [ ] **Zero `stability: 0` locators** for screens reachable with provided test data
|
|
404
|
+
- [ ] Every `stability: 0` locator has `⚠ NOT VERIFIED` comment with explicit user sign-off
|
|
405
|
+
- [ ] No step string re-registers an already-registered Given/When/Then
|
|
406
|
+
- [ ] Page object methods use `LocatorHealer` — never raw `page.click` / `page.fill`
|
|
407
|
+
- [ ] Credentials sourced from `process.env`, not hardcoded
|
|
408
|
+
- [ ] Delta files have merge-instruction comment at top
|
|
409
|
+
|
|
410
|
+
---
|
|
411
|
+
|
|
412
|
+
## Locator Stability Reference
|
|
413
|
+
|
|
414
|
+
| Pattern | Stability | When to use |
|
|
415
|
+
|---|---|---|
|
|
416
|
+
| `[data-testid='...']` | 100 | Always prefer — survives refactors |
|
|
417
|
+
| `role=button[name='...']` | 90 | When data-testid absent |
|
|
418
|
+
| `#id` | 80 | When id is stable and meaningful |
|
|
419
|
+
| `[aria-label='...']` | 70 | Accessibility attributes |
|
|
420
|
+
| `[placeholder='...']` | 60 | Input fields only |
|
|
421
|
+
| Text / position selectors | 0 | Last resort — flag for upgrade |
|
|
422
|
+
|
|
423
|
+
---
|
|
424
|
+
|
|
425
|
+
## Explicit Delivery Rules — No Inference
|
|
426
|
+
|
|
427
|
+
| Decision | Rule |
|
|
428
|
+
|---|---|
|
|
429
|
+
| Generate Playwright code | Always: show file summary and delta counts first |
|
|
430
|
+
| Attach to ADO | Requires explicit "yes" at Step 7 gate |
|
|
431
|
+
| User says "no" at Step 7 | Present files inline; offer local download only if user asks |
|
|
432
|
+
| Local file creation | Only if user says "save" or "download" |
|
|
433
|
+
| Helix-QA disk write | Must be a separate user request — not triggered by ADO attachment |
|
|
434
|
+
| Test case creation | Must be a separate user request — not triggered by this skill |
|
|
435
|
+
| Gherkin attachment | Must be a separate user request — not triggered by this skill |
|
|
436
|
+
| Prior Gherkin "yes" | Has NO bearing on Playwright attachment decision |
|
|
437
|
+
| Prior test case "no" | Has NO bearing on Playwright attachment decision |
|
|
438
|
+
|
|
439
|
+
**Each artifact delivery is independent. A yes/no for one does not answer any other.**
|
|
@@ -0,0 +1,176 @@
|
|
|
1
|
+
# Skill: Generate Test Cases from Azure DevOps Work Item
|
|
2
|
+
|
|
3
|
+
> **Read `AGENT-BEHAVIOR.md` before this skill.** The behavior rules there override any
|
|
4
|
+
> inference. This skill provides the step-by-step workflow only.
|
|
5
|
+
|
|
6
|
+
Use this skill when asked to generate, create, or write test cases for an Azure DevOps
|
|
7
|
+
PBI, Bug, User Story, or Feature.
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## 🚨 HARD STOP RULES — Read Before Anything Else
|
|
12
|
+
|
|
13
|
+
**HARD STOP 0 — Never infer delivery destination.**
|
|
14
|
+
Generating test cases ≠ creating them in ADO. Show the user what you plan to create.
|
|
15
|
+
Get explicit confirmation. Do not auto-submit.
|
|
16
|
+
|
|
17
|
+
**HARD STOP 1 — Never create test cases for an Epic.**
|
|
18
|
+
Test cases in ADO are always scoped to PBIs, Bugs, or Features — never to Epics.
|
|
19
|
+
If the user provides an Epic ID and asks for test cases:
|
|
20
|
+
- Call `fetch_work_item` with the Epic ID — the tool will return `"error": "epic_not_supported"`.
|
|
21
|
+
- Inform the user explicitly:
|
|
22
|
+
> "Manual test cases cannot be created or linked to an Epic in ADO.
|
|
23
|
+
> Test cases must be attached to individual Features, PBIs, or Bugs.
|
|
24
|
+
> Would you like me to use `fetch_epic_hierarchy` to identify the child Features
|
|
25
|
+
> and PBIs/Bugs, then create test cases for those instead?"
|
|
26
|
+
- Do NOT call `create_and_link_test_cases` with an Epic ID under any circumstances.
|
|
27
|
+
|
|
28
|
+
**HARD STOP 2 — Always ask for user confirmation before creating test cases for a Feature.**
|
|
29
|
+
When the target work item is a Feature, the server will return `"confirmation_required": true`.
|
|
30
|
+
At that point, **stop and ask the user**:
|
|
31
|
+
> "I'm about to create {N} manual test case(s) linked to Feature #{id} — "{title}".
|
|
32
|
+
> Please confirm (yes / no) before I proceed."
|
|
33
|
+
Do not call `create_and_link_test_cases` again until the user replies affirmatively.
|
|
34
|
+
If the user replies 'no' or 'cancel', abort and inform them no test cases were created.
|
|
35
|
+
|
|
36
|
+
## ⛔ Mandatory Pre-Flight: Deduplication Protocol
|
|
37
|
+
|
|
38
|
+
Before executing any step below, run the deduplication protocol:
|
|
39
|
+
`skills/deduplication-protocol.md`
|
|
40
|
+
|
|
41
|
+
The protocol is **work-item-scoped**: PHASE 1 runs once per `work_item_id` and its results
|
|
42
|
+
are cached. If this skill is invoked for the same `work_item_id` that another agent already
|
|
43
|
+
processed, skip PHASE 1 and read from `CACHE[work_item_id]` directly.
|
|
44
|
+
A different `work_item_id` always triggers a fresh PHASE 1.
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
## Steps
|
|
49
|
+
|
|
50
|
+
### Step 1 — Fetch the work item and existing test cases
|
|
51
|
+
|
|
52
|
+
**1A — Fetch the work item:**
|
|
53
|
+
Call `fetch_work_item` with the work item ID, organization URL, and project name.
|
|
54
|
+
|
|
55
|
+
The response contains:
|
|
56
|
+
- `work_item` — title, acceptance_criteria, description, repro_steps (for Bugs), story_points, priority, area_path, iteration_path
|
|
57
|
+
- `parent_feature` — parent Feature if present
|
|
58
|
+
- `existing_test_cases_count` — count of already-linked test cases (use as a signal only)
|
|
59
|
+
- `coverage_hints` — keys derived from AC text flagging areas that need coverage
|
|
60
|
+
|
|
61
|
+
**1B — Fetch existing test case titles (always, before generating):**
|
|
62
|
+
`existing_test_cases_count` is a count, not the actual titles. Two concurrent runs against the
|
|
63
|
+
same work item will duplicate test cases unless you fetch the real titles first.
|
|
64
|
+
|
|
65
|
+
```
|
|
66
|
+
qa-test-case-manager:get_linked_test_cases(
|
|
67
|
+
organization_url : ...
|
|
68
|
+
project_name : ...
|
|
69
|
+
work_item_id : <same ID>
|
|
70
|
+
)
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
The response contains a `linked_test_cases` list with `id`, `title`, `state`, `priority`.
|
|
74
|
+
**Skip any test case whose title (or a near-identical paraphrase) already exists in this list.**
|
|
75
|
+
If `existing_test_cases_count` is 0 you may skip the call.
|
|
76
|
+
|
|
77
|
+
### Step 2 — Determine complexity
|
|
78
|
+
Use `story_points` and the count of distinct acceptance criteria:
|
|
79
|
+
- **Simple** (3–6 TCs): ≤ 2 points or ≤ 2 AC items
|
|
80
|
+
- **Medium** (6–12 TCs): 3–5 points or 3–5 AC items
|
|
81
|
+
- **Complex** (12–18 TCs): > 5 points or > 5 AC items
|
|
82
|
+
|
|
83
|
+
Add +1 TC for each `coverage_hint` not already addressed.
|
|
84
|
+
|
|
85
|
+
### Step 3 — Apply the granularity rule
|
|
86
|
+
Ask: "Does this step naturally flow into the next as part of one user journey?" If yes,
|
|
87
|
+
merge into ONE test case with multiple steps. Avoid micro-interaction test cases.
|
|
88
|
+
|
|
89
|
+
- ✗ BAD: TC="Slide-out opens", TC="File picker appears"
|
|
90
|
+
- ✓ GOOD: TC="User opens profile and initiates photo update" (multi-step)
|
|
91
|
+
|
|
92
|
+
### Step 4 — Design test cases covering all applicable categories
|
|
93
|
+
|
|
94
|
+
| Category | Description |
|
|
95
|
+
|---|---|
|
|
96
|
+
| Happy paths | Successful end-to-end flows |
|
|
97
|
+
| Success feedback | Toast/confirmation messages after success |
|
|
98
|
+
| Post-action state | UI reflects the change (photo displayed, count updated) |
|
|
99
|
+
| Validation errors | Invalid input, wrong format, missing required fields |
|
|
100
|
+
| Boundary conditions | Test AT the exact limit stated in ACs (exactly 5 MB, not 4 MB or 6 MB) |
|
|
101
|
+
| Edge cases | Empty state, default state, fallback display (initials avatar) |
|
|
102
|
+
| Negative scenarios | Failure states, error toasts, state unchanged after failure |
|
|
103
|
+
| Data persistence | Data survives page refresh or re-login |
|
|
104
|
+
| Computed/derived state | Values derived from other fields update correctly |
|
|
105
|
+
| Platform-specific | Mobile/webview scenarios when mentioned in ACs |
|
|
106
|
+
| Cancel flows | Cancel/discard discards changes without saving |
|
|
107
|
+
|
|
108
|
+
Apply every hint in `coverage_hints`. Skip titles already in the `linked_test_cases` list from Step 1B.
|
|
109
|
+
|
|
110
|
+
### Step 4B — Delivery gate (mandatory — do not skip)
|
|
111
|
+
|
|
112
|
+
Before calling `create_and_link_test_cases`, present the following to the user and wait:
|
|
113
|
+
|
|
114
|
+
```
|
|
115
|
+
I've designed {N} test cases for work item #{id} — "{title}".
|
|
116
|
+
|
|
117
|
+
Here is the list:
|
|
118
|
+
1. {TC title}
|
|
119
|
+
2. {TC title}
|
|
120
|
+
...
|
|
121
|
+
|
|
122
|
+
**Do you want me to create these in ADO now? (yes / no)**
|
|
123
|
+
|
|
124
|
+
Note:
|
|
125
|
+
- Saying yes here only creates test cases — it does not generate Gherkin or Playwright code.
|
|
126
|
+
- If you also want Gherkin or Playwright code, say so separately.
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
**Do not call `create_and_link_test_cases` until the user replies "yes".**
|
|
130
|
+
**Do not carry this confirmation forward to any other artifact.**
|
|
131
|
+
|
|
132
|
+
### Step 5 — Create and link
|
|
133
|
+
Call `create_and_link_test_cases` with your test cases. Do NOT specify `area_path` —
|
|
134
|
+
it is automatically inherited from the parent work item.
|
|
135
|
+
|
|
136
|
+
Each test case:
|
|
137
|
+
- `title` — clear, specific, describes what is being tested
|
|
138
|
+
- `steps` — array of `{ action, expected_result }` — at least 2 steps per TC
|
|
139
|
+
- `priority` — 1=Critical, 2=High (default), 3=Medium, 4=Low
|
|
140
|
+
|
|
141
|
+
## Example tool call
|
|
142
|
+
|
|
143
|
+
```json
|
|
144
|
+
{
|
|
145
|
+
"work_item_id": 12345,
|
|
146
|
+
"organization_url": "https://dev.azure.com/myorg",
|
|
147
|
+
"project_name": "MyProject",
|
|
148
|
+
"test_cases": [
|
|
149
|
+
{
|
|
150
|
+
"title": "TC_12345_01 — User uploads valid profile photo and sees success confirmation",
|
|
151
|
+
"priority": 2,
|
|
152
|
+
"steps": [
|
|
153
|
+
{"action": "Navigate to My Profile slide-out", "expected_result": "Profile section opens showing current photo or initials"},
|
|
154
|
+
{"action": "Click Update Photo and select a valid JPG file under 5 MB", "expected_result": "Image preview is shown in the crop editor"},
|
|
155
|
+
{"action": "Adjust crop area and click Save", "expected_result": "Success toast 'Profile photo updated' is displayed and photo appears in the header"}
|
|
156
|
+
]
|
|
157
|
+
}
|
|
158
|
+
]
|
|
159
|
+
}
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
---
|
|
163
|
+
|
|
164
|
+
## Explicit Delivery Rules — No Inference
|
|
165
|
+
|
|
166
|
+
| Decision | Rule |
|
|
167
|
+
|---|---|
|
|
168
|
+
| Generate test cases | Always: show the proposed list first |
|
|
169
|
+
| Create in ADO | Requires explicit "yes" at Step 4B gate |
|
|
170
|
+
| User says "no" at gate | Abort; report "No test cases created" |
|
|
171
|
+
| User confirms Feature test cases | Separate confirmation at HARD STOP 2 — not carried from Step 4B |
|
|
172
|
+
| Generate Gherkin next | Must be a separate user request — not triggered by this skill |
|
|
173
|
+
| Generate Playwright next | Must be a separate user request — not triggered by this skill |
|
|
174
|
+
| Save locally | Only if user explicitly says "save" or "download" |
|
|
175
|
+
|
|
176
|
+
**Each artifact delivery is independent. A yes/no for one does not answer any other.**
|