@qa-gentic/stlc-agents 1.0.16 → 1.0.18

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. package/ORCHESTRATION_RULES.md +283 -0
  2. package/README.md +246 -321
  3. package/bin/postinstall.js +26 -2
  4. package/bin/qa-stlc.js +23 -0
  5. package/package.json +15 -2
  6. package/skills/write-helix-files/SKILL.md +6 -0
  7. package/src/cli/cmd-cost.js +253 -0
  8. package/src/cli/cmd-init.js +19 -2
  9. package/src/cli/cmd-mcp-config.js +123 -62
  10. package/src/cli/cmd-skills.js +21 -4
  11. package/src/stlc_agents/agent_gherkin_generator/server.py +88 -4
  12. package/src/stlc_agents/agent_helix_writer/tools/helix_write.py +60 -28
  13. package/src/stlc_agents/agent_jira_manager/server.py +209 -2
  14. package/src/stlc_agents/agent_jira_manager/tools/jira_workitem.py +36 -0
  15. package/src/stlc_agents/agent_playwright_generator/server.py +968 -105
  16. package/src/stlc_agents/agent_test_case_manager/server.py +121 -2
  17. package/src/stlc_agents/shared/cost_tracker.py +395 -0
  18. package/src/stlc_agents/shared/install_hook.py +154 -0
  19. package/src/stlc_agents/shared/pricing.py +72 -0
  20. package/src/stlc_agents/__pycache__/__init__.cpython-310.pyc +0 -0
  21. package/src/stlc_agents/agent_gherkin_generator/__pycache__/__init__.cpython-310.pyc +0 -0
  22. package/src/stlc_agents/agent_gherkin_generator/__pycache__/server.cpython-310.pyc +0 -0
  23. package/src/stlc_agents/agent_gherkin_generator/tools/__pycache__/__init__.cpython-310.pyc +0 -0
  24. package/src/stlc_agents/agent_gherkin_generator/tools/__pycache__/ado_gherkin.cpython-310.pyc +0 -0
  25. package/src/stlc_agents/agent_helix_writer/__pycache__/__init__.cpython-310.pyc +0 -0
  26. package/src/stlc_agents/agent_helix_writer/__pycache__/server.cpython-310.pyc +0 -0
  27. package/src/stlc_agents/agent_helix_writer/tools/__pycache__/__init__.cpython-310.pyc +0 -0
  28. package/src/stlc_agents/agent_helix_writer/tools/__pycache__/boilerplate.cpython-310.pyc +0 -0
  29. package/src/stlc_agents/agent_helix_writer/tools/__pycache__/helix_write.cpython-310.pyc +0 -0
  30. package/src/stlc_agents/agent_jira_manager/__pycache__/__init__.cpython-310.pyc +0 -0
  31. package/src/stlc_agents/agent_jira_manager/__pycache__/server.cpython-310.pyc +0 -0
  32. package/src/stlc_agents/agent_jira_manager/tools/__pycache__/__init__.cpython-310.pyc +0 -0
  33. package/src/stlc_agents/agent_jira_manager/tools/__pycache__/jira_workitem.cpython-310.pyc +0 -0
  34. package/src/stlc_agents/agent_playwright_generator/__pycache__/__init__.cpython-310.pyc +0 -0
  35. package/src/stlc_agents/agent_playwright_generator/__pycache__/server.cpython-310.pyc +0 -0
  36. package/src/stlc_agents/agent_playwright_generator/tools/__pycache__/__init__.cpython-310.pyc +0 -0
  37. package/src/stlc_agents/agent_playwright_generator/tools/__pycache__/ado_attach.cpython-310.pyc +0 -0
  38. package/src/stlc_agents/agent_test_case_manager/__pycache__/__init__.cpython-310.pyc +0 -0
  39. package/src/stlc_agents/agent_test_case_manager/__pycache__/server.cpython-310.pyc +0 -0
  40. package/src/stlc_agents/agent_test_case_manager/tools/__pycache__/__init__.cpython-310.pyc +0 -0
  41. package/src/stlc_agents/agent_test_case_manager/tools/__pycache__/ado_workitem.cpython-310.pyc +0 -0
  42. package/src/stlc_agents/shared/__pycache__/__init__.cpython-310.pyc +0 -0
  43. package/src/stlc_agents/shared/__pycache__/auth.cpython-310.pyc +0 -0
  44. package/src/stlc_agents/shared_jira/__pycache__/__init__.cpython-310.pyc +0 -0
  45. package/src/stlc_agents/shared_jira/__pycache__/auth.cpython-310.pyc +0 -0
package/README.md CHANGED
@@ -1,11 +1,12 @@
1
1
  # @qa-gentic/stlc-agents
2
2
 
3
- > AI-powered QA STLC automation — from Azure DevOps **or Jira Cloud** work item to self-healing Playwright TypeScript in a Helix-QA project.
3
+ > AI-powered QA STLC automation — from Azure DevOps **or Jira Cloud** work item state change to self-healing Playwright TypeScript in a Helix-QA project.
4
4
 
5
- Works with **GitHub Copilot** (VS Code Agent mode), **Claude Code**, **Cursor**, and **Windsurf**.
5
+ Works with **GitHub Copilot** (VS Code Agent mode), **Claude Code**, **Cursor**, and **Windsurf**.
6
+ Also runs fully headless via the **webhook bridge** — no human in the loop required.
6
7
 
7
- ![npm version](https://img.shields.io/badge/npm-v1.0.10-blue)
8
- ![PyPI version](https://img.shields.io/badge/pypi-v1.0.10-blue)
8
+ ![npm version](https://img.shields.io/badge/npm-v1.0.17-blue)
9
+ ![PyPI version](https://img.shields.io/badge/pypi-v1.0.17-blue)
9
10
  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
10
11
  [![Node.js >=18](https://img.shields.io/badge/node-%3E%3D18-brightgreen)](https://nodejs.org)
11
12
  [![Python >=3.10](https://img.shields.io/badge/python-%3E%3D3.10-blue)](https://python.org)
@@ -14,350 +15,357 @@ Works with **GitHub Copilot** (VS Code Agent mode), **Claude Code**, **Cursor**,
14
15
 
15
16
  ## What It Does
16
17
 
17
- Five Python MCP servers cover the full QA Software Test Life Cycle:
18
+ Five Python MCP servers cover the full QA Software Test Life Cycle. A sixth server (Playwright MCP) drives a real browser during code generation.
18
19
 
19
- | Agent | Input | Output |
20
- |---|---|---|
21
- | `qa-test-case-manager` | ADO PBI / Bug / Feature ID | Manual test cases created & linked via TestedBy-Forward |
22
- | `qa-gherkin-generator` | ADO / Jira Epic / Feature / PBI / Bug ID | `.feature` file attached to ADO work item or saved locally |
23
- | `qa-playwright-generator` | Gherkin + live browser | `locators.ts` + page objects + step defs attached to ADO or saved locally |
24
- | `qa-helix-writer` | Generated `.ts` files + `helix_root` | Files written to Helix-QA directory layout on disk |
25
- | `qa-jira-manager` | **Jira** Story / Bug / Task / Epic | Test cases created in Jira Gherkin Playwright TypeScript Helix-QA files |
20
+ | Agent | Server name | Input | Output |
21
+ |---|---|---|---|
22
+ | Agent 1 | `qa-test-case-manager` | ADO PBI / Bug / Feature | Manual test cases created & linked (TestedBy-Forward), deduped on re-trigger |
23
+ | Agent 2 | `qa-gherkin-generator` | ADO Feature / PBI / Bug | `.feature` file validated and attached to work item |
24
+ | Agent 3 | `qa-playwright-generator` | Gherkin + optional AX-tree `context_map` | `locators.ts` + page objects + step defs (cached, retrieved via `get_generated_files`) |
25
+ | Agent 4 | `qa-helix-writer` | Generated `.ts` files + `helix_root` | Files written to Helix-QA directory layout, never overwrites |
26
+ | Agent 5 | `qa-jira-manager` | Jira Story / Bug / Task | Test cases created & linked in Jira, `.feature` attached to issue, deduped on re-trigger |
27
+
28
+ ---
29
+
30
+ ## End-to-End Flow
26
31
 
27
- A sixth server — **Playwright MCP** (`http://localhost:8931/mcp`) — drives a real browser during code generation, replacing hand-authored locators with accessibility-tree-derived, zero-hallucination selectors.
32
+ ### Webhook-triggered (headless)
33
+
34
+ A work item state change in ADO or Jira fires a webhook POST to the bridge. The bridge normalises the payload and routes it through the pipeline automatically.
35
+
36
+ ```
37
+ ADO Service Hook / Jira Webhook
38
+ │ POST
39
+
40
+ webhook_bridge/server.py (FastAPI — qa-stlc-serve)
41
+
42
+ ├─ parsers.py raw payload → normalised event dict
43
+ ├─ state_router.py event → STAGE_TEST_CASES | STAGE_FULL_PIPELINE | SKIP
44
+
45
+ └─ ci_runner/pipeline.py
46
+ ├─ run_test_cases() → Agent 1 (ADO) or Agent 5 (Jira)
47
+ └─ run_post_done() → Agent 2 → Agent 3 → Agent 4
48
+ ```
49
+
50
+ ### State → stage mapping
51
+
52
+ | Platform | Work item state | Stage | Agents called |
53
+ |---|---|---|---|
54
+ | ADO | `Approved` / `Committed` | `STAGE_TEST_CASES` | Agent 1 only |
55
+ | ADO | `Done` | `STAGE_FULL_PIPELINE` | Agents 2 → 3 → 4 |
56
+ | Jira | `In Progress` / `Selected for Development` | `STAGE_TEST_CASES` | Agent 5 only |
57
+ | Jira | `Done` | `STAGE_FULL_PIPELINE` | Agents 2 → 3 → 4 |
58
+
59
+ Any other state is silently dropped. State names are configurable in `state_router.py`.
60
+
61
+ ### STAGE_TEST_CASES — Agent 1 / Agent 5
62
+
63
+ ```
64
+ fetch_work_item / fetch_jira_issue
65
+
66
+ [LLM] generate_test_cases() ← pipeline bridges fetch → LLM → create
67
+
68
+ create_deduped_test_cases() ← internally: get_linked → filter dupes → create_and_link
69
+ ```
28
70
 
29
- > **ADO or Jira?** Both pipelines now run the full STLC: fetch → analyse → test cases → Gherkin → Playwright → Helix-QA. ADO uses `qa-test-case-manager` as the entry point; Jira uses `qa-jira-manager`. Agents 2–4 (`qa-gherkin-generator`, `qa-playwright-generator`, `qa-helix-writer`) are shared by both pipelines.
71
+ On re-trigger, titles already linked are skipped (case-insensitive, stop-word normalised). Only net-new test cases are created.
72
+
73
+ ### STAGE_FULL_PIPELINE — Agents 2 → 3 → 4
74
+
75
+ First error stops the chain and surfaces in the result dict. Subsequent agents are not called.
76
+
77
+ **Agent 2 — Gherkin**
78
+
79
+ ```
80
+ ADO Feature : fetch_feature_hierarchy → [LLM] generate_gherkin
81
+ ADO PBI/Bug : fetch_work_item_for_gherkin → [LLM] generate_gherkin
82
+ Jira : fetch_jira_issue (via Agent 5) → [LLM] generate_gherkin
83
+
84
+ validate_gherkin_content() ← structural check before attach
85
+ ↓ (if invalid → pipeline stops, returns validation errors)
86
+ ADO Feature : attach_gherkin_to_feature()
87
+ ADO PBI/Bug : attach_gherkin_to_work_item()
88
+ Jira : attach_gherkin_to_issue() ← uploads via Jira attachment API
89
+ ```
90
+
91
+ The `generate_and_attach_gherkin` composite tool wraps validate + attach for CI/headless callers.
92
+
93
+ **Agent 3 — Playwright**
94
+
95
+ ```
96
+ generate_playwright_code(gherkin_content, page_class_name)
97
+ ↓ returns cache_key (files held in-memory)
98
+ get_generated_files(cache_key)
99
+ ↓ returns { "path/to/file.ts": "content", ... }
100
+ ```
101
+
102
+ `page_class_name` is derived from `event["title"]` (ADO) or `event["summary"]` (Jira) — camel-cased, max 4 words. Without a `context_map`, locators are Gherkin-inferred (stability=0). Pass `app_url` to embed a snapshot hint comment in `locators.ts`.
103
+
104
+ **Agent 4 — Helix writer**
105
+
106
+ ```
107
+ inspect_helix_project(helix_root)
108
+ ↓ framework_state: absent | partial | present
109
+ write_helix_files(helix_root, files, mode)
110
+ mode = "scaffold_and_tests" if absent or partial
111
+ mode = "tests_only" if present
112
+ ```
113
+
114
+ Agent 4 handles all deduplication and conflict renaming internally. No file is ever overwritten.
30
115
 
31
116
  ---
32
117
 
33
118
  ## Install
34
119
 
35
-
36
120
  ```bash
37
121
  # 1. Install the CLI + npm package globally
38
- # You will be prompted to choose your integration: ado / jira / both
39
- # The choice is saved to ~/.qa-stlc/integration and reused by all subsequent commands.
122
+ # Prompted to choose integration: ado / jira / both
40
123
  npm install -g @qa-gentic/stlc-agents
41
- ```
42
-
43
- > **Note:** On most systems, the installer will prompt you to select your integration (ADO, Jira, or both) and then automatically run `qa-stlc init` and offer to scaffold a new project. If you do **not** see the prompt (for example, if npm skips postinstall for an already-installed package, or in some non-interactive environments), simply run:
44
- >
45
- > ```bash
46
- > qa-stlc init
47
- > qa-stlc scaffold
48
- > ```
49
- >
50
- > This will perform the same setup steps as the postinstall script.
51
124
 
52
- ```bash
53
- # 2. Bootstrap your project (installs Python agents, copies skills, writes MCP config)
54
- # --integration overrides the saved preference; omit to reuse the choice from step 1.
55
- qa-stlc init --vscode --integration ado # GitHub Copilot / VS Code — ADO pipeline
56
- qa-stlc init --vscode --integration jira # GitHub Copilot / VS Code — Jira pipeline
57
- qa-stlc init --vscode --integration both # GitHub Copilot / VS Code — both pipelines
58
- # or for Claude Code (no --vscode flag):
125
+ # 2. Bootstrap your project
126
+ qa-stlc init --vscode --integration ado # GitHub Copilot / VS Code ADO
127
+ qa-stlc init --vscode --integration jira # GitHub Copilot / VS Code Jira
128
+ qa-stlc init --vscode --integration both # GitHub Copilot / VS Code — both
129
+ # or for Claude Code:
59
130
  qa-stlc init --integration ado
60
131
 
61
132
  # 3. Scaffold a new Playwright + Cucumber + TypeScript QA project
62
133
  qa-stlc scaffold --name my-qa-project
63
- # or into an existing directory
64
- qa-stlc scaffold --name acme-e2e --dir /path/to/acme-e2e
65
134
 
66
- # 4. Start the Playwright browser server (required for code generation)
135
+ # 4. Start the Playwright browser server (required for live-locator generation)
67
136
  npx @playwright/mcp@latest --port 8931
68
137
  ```
69
138
 
70
- `qa-stlc init` does four things:
71
- 1. `pip install qa-gentic-stlc-agents` — installs all five Python MCP servers
72
- 2. Copies skill files to `.github/copilot-instructions/` (and `.claude/` if not `--vscode`)
73
- 3. Copies custom agent files to `.github/agents/` (used by GitHub Copilot and Claude's agent picker)
74
- 4. Writes `.vscode/mcp.json` with all six servers configured
139
+ `qa-stlc init` does five things:
75
140
 
76
- > **Integration flag:** `--integration` controls which agents and skill files are installed.
77
- > If omitted, the value saved during `npm install -g` is used.
78
- > You can change it at any time by re-running `qa-stlc init --integration <ado|jira|both>`.
141
+ 1. `pip install qa-gentic-stlc-agents` installs all five Python MCP servers + rules files
142
+ 2. Copies skill files to `.github/copilot-instructions/` (and `.claude/` if not `--vscode`)
143
+ 3. Copies custom agent files to `.github/agents/`
144
+ 4. Copies `ORCHESTRATION_RULES.md` to project root for reference during multi-step tasks
145
+ 5. Writes `.vscode/mcp.json` (or `.mcp.json`) with all six servers configured
79
146
 
80
147
  ---
81
148
 
82
149
  ## Requirements
83
150
 
84
- - Node.js 18
85
- - Python ≥ 3.10, < 3.14
86
- - For **Azure DevOps** (Agents 1–4) — a `.env` containing:
151
+ | Requirement | Value |
152
+ |---|---|
153
+ | Node.js | >= 18 |
154
+ | Python | >= 3.10, < 3.14 |
155
+ | ADO PAT | Required for Agents 1–4 (ADO pipeline) |
156
+ | Jira OAuth 2.0 (3LO) | Required for Agent 5 (Jira pipeline) |
157
+ | Playwright MCP | `npx @playwright/mcp@latest --port 8931` |
158
+
159
+ **ADO `.env` vars:**
87
160
 
88
161
  ```env
89
162
  ADO_ORGANIZATION_URL=https://dev.azure.com/your-org
90
163
  ADO_PROJECT_NAME=YourProject
91
164
  ADO_PAT=your-personal-access-token
92
- APP_BASE_URL=your-app-base-url
93
- APP_EMAIL=your-test-email@example.com
94
- APP_PASSWORD=your-test-password
165
+ APP_BASE_URL=https://your-app.example.com
166
+
167
+ # LLM — pick one provider:
168
+ AI_HEALING_PROVIDER=anthropic
169
+ AI_HEALING_API_KEY=sk-ant-...
170
+ # or: OPENAI_API_KEY / AZURE_OPENAI_API_KEY / GITHUB_TOKEN (Copilot)
95
171
  ```
96
172
 
97
- - For **Jira Cloud** (Agent 5) — additional `.env` vars:
173
+ **Jira additional `.env` vars:**
98
174
 
99
175
  ```env
100
176
  JIRA_CLIENT_ID=your-atlassian-oauth-client-id
101
177
  JIRA_CLIENT_SECRET=your-atlassian-oauth-client-secret
102
178
  JIRA_CLOUD_ID=your-atlassian-cloud-id
103
- # Optional — defaults to http://localhost:8765/callback
104
- # JIRA_REDIRECT_URI=http://localhost:8765/callback
105
179
  ```
106
180
 
107
- See [Jira Authentication Setup](#jira-authentication-setup) for how to obtain these values.
108
-
109
- ---
110
-
111
- ## Commands
112
-
113
- ```bash
114
- qa-stlc init [--vscode] [--python <path>] [--integration ado|jira|both]
115
- # Full bootstrap: pip install + skills + agent files + MCP config
116
- # Integration defaults to the value saved during npm install -g (prompted at that time)
117
-
118
- qa-stlc scaffold [--name <name>] [--dir <path>] [--no-install]
119
- # Copy full Playwright + Cucumber + TypeScript boilerplate to a new project directory
181
+ **Webhook bridge additional `.env` vars:**
120
182
 
121
- qa-stlc skills [--target claude|vscode|cursor|windsurf|both|print]
122
- # Copy skill files to the correct AI coding agent directory
123
-
124
- qa-stlc mcp-config [--vscode] [--print] [--python <path>] [--playwright-port <n>]
125
- # Write .vscode/mcp.json (--vscode) or .mcp.json (Claude Code)
126
-
127
- qa-stlc verify
128
- # Check that all six MCP servers are reachable
183
+ ```env
184
+ WEBHOOK_SECRET=your-shared-secret
185
+ HELIX_PROJECT_ROOT=/path/to/helix-qa # where Agent 4 writes files
186
+ PLAYWRIGHT_MCP_URL=http://localhost:8931/mcp # leave blank to skip Agent 3
129
187
  ```
130
188
 
131
189
  ---
132
190
 
133
- ## Usage
191
+ ## CLI Commands
134
192
 
135
- Open your AI coding agent and use natural language:
193
+ | Command | Description |
194
+ |---|---|
195
+ | `qa-stlc init [--vscode] [--integration ado\|jira\|both]` | Full bootstrap: pip install + skills + agent files + MCP config |
196
+ | `qa-stlc scaffold [--name n] [--dir path]` | Copy full Playwright + Cucumber + TypeScript boilerplate to a new project |
197
+ | `qa-stlc skills [--target claude\|vscode\|cursor\|windsurf]` | Copy skill files to the correct AI coding agent directory |
198
+ | `qa-stlc mcp-config [--vscode] [--print]` | Write `.vscode/mcp.json` or `.mcp.json` with all servers configured |
199
+ | `qa-stlc verify` | Check that all MCP servers are reachable and auth is cached |
200
+ | `qa-stlc-serve [--host] [--port] [--reload]` | Start the webhook bridge (FastAPI) |
136
201
 
137
- ```
138
- # Azure DevOps — Generate test cases
139
- Generate test cases for work item #12345 in MyProject at https://dev.azure.com/myorg
202
+ ---
140
203
 
141
- # Azure DevOps — Generate Gherkin BDD
142
- Generate a Gherkin regression suite for Feature #11000 in MyProject at https://dev.azure.com/myorg
204
+ ## Orchestration Rules
143
205
 
144
- # Jira Fetch issue and generate test cases
145
- Generate test cases for Jira issue PROJ-123
206
+ When `qa-stlc init` is run, `ORCHESTRATION_RULES.md` is installed to your project root and placed in both npm (`node_modules/@qa-gentic/stlc-agents/`) and pip (`site-packages/stlc_agents/`) installations.
146
207
 
147
- # Jira Fetch an Epic hierarchy
148
- Fetch the full Epic hierarchy for PROJ-10 in Jira
208
+ Refer to this file in multi-step QA workflows to ensure:
209
+ - **Step breakdown:** every task is decomposed into named steps with explicit inputs/outputs
210
+ - **Pre-flight gates:** output validation runs *before* generation, not after
211
+ - **No skipped steps:** intermediate data is structured and handed off explicitly, never inferred
212
+ - **Countable verification:** code generation steps state expected item counts before implementation
213
+ - **Zero stubs:** no partial outputs with TODOs or empty bodies passed downstream
149
214
 
150
- # Generate Playwright code (requires Playwright MCP running — works with both ADO and Jira flows)
151
- Generate Playwright TypeScript for the Gherkin I just created. The login page is at /auth/login
215
+ Integration per AI coding agent:
152
216
 
153
- # Write to Helix-QA project
154
- Write the generated files to my Helix-QA project at /workspace/my-qa-project
155
- ```
217
+ - **GitHub Copilot (VS Code):** Add to `.github/copilot-instructions.md` or reference in your workspace prompt
218
+ - **Claude Code:** Reference in `.claude/instructions.md` or as a project document
219
+ - **Cursor:** Add to `.cursor/rules/orchestration.mdc` (scope: always)
220
+ - **Windsurf:** Reference in `.windsurf/rules.md` or global rules panel
221
+
222
+ See section 8 of `ORCHESTRATION_RULES.md` for agent-specific integration notes.
156
223
 
157
224
  ---
158
225
 
159
226
  ## Tool Reference
160
227
 
161
- ### qa-test-case-manager _(Azure DevOps)_
228
+ ### Agent 1 — `qa-test-case-manager` _(Azure DevOps)_
162
229
 
163
230
  | Tool | Description |
164
231
  |---|---|
165
- | `fetch_work_item` | Fetch a PBI, Bug, or Feature with acceptance criteria, linked test cases, and coverage hints. Returns `epic_not_supported` for Epics. Returns `confirmation_required: true` for Features. |
166
- | `get_linked_test_cases` | List all test cases already linked to a work item. Call before generating to power the deduplication diff. |
167
- | `create_and_link_test_cases` | Create structured manual test cases in ADO and link them via TestedBy-Forward. |
232
+ | `fetch_work_item` | Fetch a PBI, Bug, or Feature with acceptance criteria, coverage hints, and existing TC count. Returns `epic_not_supported` for Epics. |
233
+ | `get_linked_test_cases` | List all test cases linked via TestedBy-Forward (used by dedup). |
234
+ | `create_and_link_test_cases` | Create structured manual test cases and link to work item. Feature confirmation gate fires unless `confirmed=true`. |
235
+ | `create_deduped_test_cases` | **Headless/webhook tool.** Internally calls `get_linked_test_cases`, filters duplicates (normalised title match), then calls `create_and_link_test_cases` on net-new only. Safe to call on every re-trigger. |
168
236
 
169
- ### qa-gherkin-generator _(Azure DevOps)_
237
+ ### Agent 2 — `qa-gherkin-generator` _(Azure DevOps)_
170
238
 
171
239
  | Tool | Description |
172
240
  |---|---|
173
- | `fetch_feature_hierarchy` | Fetch a Feature and all child PBIs/Bugs with acceptance criteria and existing attachments. |
174
- | `fetch_work_item_for_gherkin` | Fetch a PBI or Bug with parent Feature context, suggested file name, and linked test case steps. |
175
- | `attach_gherkin_to_feature` | Validate and attach a `.feature` file to a Feature work item. |
176
- | `attach_gherkin_to_work_item` | Validate and attach a `.feature` file to a PBI or Bug work item. |
177
- | `validate_gherkin_content` | Structural validation tags, scenario count, navigation steps. Returns `valid: bool` + `errors` + `warnings`. |
241
+ | `fetch_feature_hierarchy` | Fetch a Feature + all child PBIs/Bugs + coverage hints. |
242
+ | `fetch_work_item_for_gherkin` | Fetch a single PBI or Bug with parent Feature context. |
243
+ | `validate_gherkin_content` | Structural check: @smoke/@regression tags, scenario count (5–10 feature / 3–9 work_item), every scenario has `When`, no duplicate titles. |
244
+ | `attach_gherkin_to_feature` | Validate + upload + link `.feature` to a Feature work item. |
245
+ | `attach_gherkin_to_work_item` | Validate + upload + link `.feature` to a PBI or Bug. |
246
+ | `generate_and_attach_gherkin` | **Headless/webhook composite.** Accepts pre-generated `gherkin_content`, validates, attaches. Returns `status: validation_failed` with errors if invalid — pipeline must re-generate. |
178
247
 
179
- ### qa-playwright-generator _(ADO + Jira)_
248
+ ### Agent 3 — `qa-playwright-generator` _(ADO + Jira)_
180
249
 
181
250
  | Tool | Description |
182
251
  |---|---|
183
- | `generate_playwright_code` | Generate `locators.ts`, `*Page.ts`, `*.steps.ts`, and `cucumber-profile.js` from validated Gherkin + live AX-tree `context_map`. Hard-blocks if `context_map` is absent to prevent hallucinated locators. |
184
- | `scaffold_locator_repository` | Generate the six Helix-QA healing infrastructure files. |
252
+ | `generate_playwright_code` | Generate `locators.ts`, `*Page.ts`, `*.steps.ts`, `cucumber-profile.js`. Returns `cache_key`. Optional `context_map` for AX-tree-verified locators; optional `app_url` embeds snapshot hint. |
253
+ | `get_generated_files` | Retrieve full file content by `cache_key` from Agent 3's in-memory cache. |
254
+ | `scaffold_locator_repository` | Generate the five Helix-QA healing infrastructure files (`LocatorHealer`, `TimingHealer`, `VisualIntentChecker`, `LocatorRepository`, `HealingDashboard`). Call once per project. |
185
255
  | `validate_gherkin_steps` | Check for duplicate step strings and missing `When` steps. |
186
- | `attach_code_to_work_item` | Attach delta Playwright TypeScript files to an ADO work item. Pass only net-new delta files. |
187
-
188
- ### qa-helix-writer _(ADO + Jira)_
189
-
190
- | Tool | Description |
191
- |---|---|
192
- | `inspect_helix_project` | Returns `framework_state` (`present` / `partial` / `absent`) and `recommendation`. |
193
- | `list_helix_tree` | List the full directory tree of a Helix-QA project. |
194
- | `read_helix_file` | Read an existing file from the Helix-QA project for overlap detection. |
195
- | `write_helix_files` | Write generated files to the correct Helix-QA paths. |
256
+ | `attach_code_to_work_item` | Attach generated TypeScript delta files to an ADO work item. |
196
257
 
197
- ### qa-jira-manager _(Jira Cloud)_
258
+ ### Agent 4 — `qa-helix-writer` _(ADO + Jira)_
198
259
 
199
260
  | Tool | Description |
200
261
  |---|---|
201
- | `fetch_jira_issue` | Fetch a Story, Bug, or Task with acceptance criteria, coverage hints, and existing linked test case count. Returns `epic_use_hierarchy` for Epics. |
202
- | `fetch_jira_epic_hierarchy` | Fetch an Epic with all child issues (Stories, Tasks, Bugs, Sub-tasks). |
203
- | `create_and_link_test_cases` | Create test case issues in Jira (type `Test`, falls back to `Task`) and link them via `is tested by`. Steps stored as an ADF table — no Xray required. Returns `confirmation_required: true` for Epics. |
204
- | `get_linked_test_cases` | List all issues linked to a Jira issue via `is tested by` / `Test` link type. Call before generating to avoid duplicates. |
262
+ | `inspect_helix_project` | Returns `framework_state`: `present` / `partial` / `absent`. Drives `mode` selection in pipeline. |
263
+ | `write_helix_files` | Write files to Helix-QA layout. `mode=scaffold_and_tests` for new/partial projects; `mode=tests_only` for existing. Never overwrites; deduplicates and conflict-renames. |
264
+ | `list_helix_tree` | Full directory listing of a Helix-QA project. |
265
+ | `read_helix_file` | Read an existing file for overlap detection. |
205
266
 
206
- ### playwright _(external)_
267
+ ### Agent 5 — `qa-jira-manager` _(Jira Cloud)_
207
268
 
208
269
  | Tool | Description |
209
270
  |---|---|
210
- | `browser_navigate` | Navigate to a URL in the live browser. |
211
- | `browser_snapshot` | Capture the full AX tree source for zero-hallucination locators. |
212
- | `browser_fill_form` | Fill multiple form fields by ref. |
213
- | `browser_click` | Click an element by ref. |
214
- | `browser_file_upload` | Upload a file to a file input by ref. |
271
+ | `fetch_jira_issue` | Fetch a Story, Bug, or Task with acceptance criteria and coverage hints. Returns `epic_use_hierarchy` for Epics. |
272
+ | `get_linked_test_cases` | List all issues linked via `is tested by` / `Test` link type (used by dedup). |
273
+ | `create_and_link_test_cases` | Create test case issues in Jira (type `Test`, falls back to `Task`) and link. Steps stored as ADF table — no Xray required. Epic confirmation gate fires unless `confirmed=true`. |
274
+ | `create_deduped_test_cases` | **Headless/webhook tool.** Internally calls `get_linked_test_cases`, filters duplicates (normalised summary match), then calls `create_and_link_test_cases` on net-new only. Safe on every re-trigger. |
275
+ | `attach_gherkin_to_issue` | Upload a `.feature` file as an attachment to a Jira issue via the Jira attachment API. File is named `{issue_key}_{summary_kebab}_regression.feature`. |
215
276
 
216
277
  ---
217
278
 
218
- ## MCP Config
219
-
220
- ### VS Code / GitHub Copilot (`.vscode/mcp.json`)
221
-
222
- ```json
223
- {
224
- "servers": {
225
- "qa-test-case-manager": { "command": "/path/to/.venv/bin/qa-test-case-manager" },
226
- "qa-gherkin-generator": { "command": "/path/to/.venv/bin/qa-gherkin-generator" },
227
- "qa-playwright-generator": { "command": "/path/to/.venv/bin/qa-playwright-generator" },
228
- "qa-helix-writer": { "command": "/path/to/.venv/bin/qa-helix-writer" },
229
- "qa-jira-manager": {
230
- "command": "/path/to/.venv/bin/qa-jira-manager",
231
- "env": {
232
- "JIRA_CLIENT_ID": "${env:JIRA_CLIENT_ID}",
233
- "JIRA_CLIENT_SECRET": "${env:JIRA_CLIENT_SECRET}",
234
- "JIRA_CLOUD_ID": "${env:JIRA_CLOUD_ID}"
235
- }
236
- },
237
- "playwright": { "type": "http", "url": "http://localhost:8931/mcp" }
238
- }
239
- }
240
- ```
279
+ ## Three-Layer Self-Healing Architecture
241
280
 
242
- ### Claude Code (`.mcp.json`)
243
-
244
- ```json
245
- {
246
- "mcpServers": {
247
- "qa-test-case-manager": { "command": "/path/to/.venv/bin/qa-test-case-manager" },
248
- "qa-gherkin-generator": { "command": "/path/to/.venv/bin/qa-gherkin-generator" },
249
- "qa-playwright-generator": { "command": "/path/to/.venv/bin/qa-playwright-generator" },
250
- "qa-helix-writer": { "command": "/path/to/.venv/bin/qa-helix-writer" },
251
- "qa-jira-manager": {
252
- "command": "/path/to/.venv/bin/qa-jira-manager",
253
- "env": {
254
- "JIRA_CLIENT_ID": "${env:JIRA_CLIENT_ID}",
255
- "JIRA_CLIENT_SECRET": "${env:JIRA_CLIENT_SECRET}",
256
- "JIRA_CLOUD_ID": "${env:JIRA_CLOUD_ID}"
257
- }
258
- },
259
- "playwright": { "type": "url", "url": "ws://localhost:8931" }
260
- }
261
- }
262
- ```
281
+ | Layer | Class | What It Heals |
282
+ |---|---|---|
283
+ | 1 — Locator | `LocatorHealer` | primary selector → role → label → text → AI Vision → CDPSession AX tree |
284
+ | 2 — Timing | `TimingHealer` | network-trace drift → auto-adjusted timeouts → HealingDashboard |
285
+ | 3 — Visual | `VisualIntentChecker` | element screenshot diff at assertions → HealingDashboard |
263
286
 
264
- > VS Code requires `http://localhost:8931/mcp` (with `/mcp` suffix).
265
- > Claude Code requires `ws://localhost:8931` (WebSocket, no `/mcp`).
266
- > `qa-stlc mcp-config` writes the correct format automatically.
287
+ Healed selectors persist in `LocatorRepository`. All AI suggestions require human approval at `http://localhost:7890` before entering version control.
267
288
 
268
289
  ---
269
290
 
270
- ## Skills Installed
291
+ ## Webhook / Auto-Trigger Setup
271
292
 
272
- Skills follow the [Agent Skills specification](https://agentskills.io/specification) and live under `skills/<n>/SKILL.md`.
293
+ ### Local dev
273
294
 
274
- | Skill | Purpose |
275
- |---|---|
276
- | `AGENT-BEHAVIOR.md` | Zero-inference contract. Read first. Always. |
277
- | `generate-test-cases/SKILL.md` | PBI / Bug / Feature → ADO manual test cases |
278
- | `generate-gherkin/SKILL.md` | Epic / Feature / PBI / Bug → validated `.feature` + ADO attach |
279
- | `generate-playwright-code/SKILL.md` | Gherkin + live browser → three-layer self-healing Playwright TypeScript |
280
- | `write-helix-files/SKILL.md` | Generated files → Helix-QA disk layout, scaffold or merge |
281
- | `deduplication-protocol/SKILL.md` | READ → DIFF → CREATE mandatory pre-flight gate |
282
- | `qa-jira-manager/SKILL.md` | Jira issue → test case issues + `is tested by` links |
283
-
284
- > **Installed copies** (`.claude/skills/`, `.github/copilot-instructions/`) are generated by
285
- > `qa-stlc init` / `qa-stlc skills` at project bootstrap time and are gitignored.
286
- > The canonical source for all skills is the `skills/` directory.
295
+ ```bash
296
+ # 1. Install webhook extras
297
+ pip install "qa-gentic-stlc-agents[webhook]"
287
298
 
288
- ---
299
+ # 2. Start the bridge
300
+ make serve
289
301
 
290
- ## Jira Authentication Setup
302
+ # 3. Expose via ngrok
303
+ ngrok http 8080
291
304
 
292
- `qa-jira-manager` uses **Jira OAuth 2.0 (3LO)** with a persistent file-based token cache
293
- at `~/.jira-cache/jira-token.json` — the same silent-cache / browser-fallback pattern
294
- used by the ADO MSAL auth module.
305
+ # 4. Register hooks
306
+ make register-ado-hooks BRIDGE_URL=https://xxxx.ngrok.io
307
+ make register-jira-hooks BRIDGE_URL=https://xxxx.ngrok.io
308
+ ```
295
309
 
296
- ### Step 1 — Create an Atlassian OAuth 2.0 (3LO) app
310
+ ### Azure Functions deploy
297
311
 
298
- 1. Go to <https://developer.atlassian.com/console/myapps/>
299
- 2. Click **Create** → **OAuth 2.0 integration**
300
- 3. Add callback URL: `http://localhost:8765/callback`
301
- 4. Under **Permissions**, enable: `read:jira-user`, `read:jira-work`, `write:jira-work`, `offline_access`
302
- 5. Copy the **Client ID** and **Secret**
312
+ ```bash
313
+ make deploy-azure FUNC_APP=stlc-webhook-bridge
314
+ make register-ado-hooks BRIDGE_URL=https://stlc-webhook-bridge.azurewebsites.net/api
315
+ make register-jira-hooks BRIDGE_URL=https://stlc-webhook-bridge.azurewebsites.net/api
316
+ ```
303
317
 
304
- ### Step 2 — Find your Cloud ID
318
+ ### Docker
305
319
 
306
320
  ```bash
307
- curl -H "Authorization: Bearer <any-token>" \
308
- https://api.atlassian.com/oauth/token/accessible-resources
321
+ make docker-build-webhook
322
+ make docker-run-webhook
309
323
  ```
310
324
 
311
- Copy the `id` field for your Jira site.
325
+ ### LLM provider selection
312
326
 
313
- ### Step 3 Add to `.env`
327
+ | Provider | Env var(s) | Default model |
328
+ |---|---|---|
329
+ | `anthropic` | `AI_HEALING_API_KEY` or `ANTHROPIC_API_KEY` | `claude-haiku-4-5-20251001` |
330
+ | `copilot` | `GITHUB_TOKEN` | `gpt-4o` |
331
+ | `openai` | `OPENAI_API_KEY` | `gpt-4o-mini` |
332
+ | `azure-openai` | `AZURE_OPENAI_API_KEY` + `AZURE_OPENAI_ENDPOINT` | `gpt-4o-mini` |
333
+ | `ollama` | `OLLAMA_HOST` (optional) | `llama3.2` |
314
334
 
315
- ```env
316
- JIRA_CLIENT_ID=your-client-id
317
- JIRA_CLIENT_SECRET=your-client-secret
318
- JIRA_CLOUD_ID=your-cloud-id
319
- ```
335
+ Override model at any time: `LLM_MODEL=claude-sonnet-4-6`
320
336
 
321
- ### Step 4 First run
337
+ ### Token cost (headless, Haiku default)
322
338
 
323
- The browser opens once on startup. Sign in to Jira. The token is cached silently
324
- and refreshed automatically for ~60 days — no re-prompting.
339
+ | Stage | Tokens / work item | Approx cost |
340
+ |---|---|---|
341
+ | Test case generation (state → Approved) | ~8,000–15,000 | $0.01–$0.03 |
342
+ | Gherkin generation (state → Done) | ~10,000–18,000 | $0.01–$0.04 |
343
+ | Playwright TS generation (state → Done) | ~20,000–35,000 | $0.03–$0.07 |
325
344
 
326
345
  ---
327
346
 
328
- ## Workflow Comparison: ADO vs Jira
347
+ ## ADO vs Jira — Side by Side
329
348
 
330
349
  | Step | Azure DevOps | Jira Cloud |
331
350
  |---|---|---|
351
+ | Trigger | Service Hook: `workitem.updated` | Webhook: `jira:issue_updated` |
332
352
  | Fetch issue | `fetch_work_item` | `fetch_jira_issue` |
333
- | Fetch Epic hierarchy | Not supported — warn user | Not supported — warn user |
334
- | Check for duplicates | `get_linked_test_cases` | `get_linked_test_cases` |
335
- | Create & link test cases | `create_and_link_test_cases` | `create_and_link_test_cases` |
336
- | Generate Gherkin | `fetch_work_item_for_gherkin` `attach_gherkin_to_work_item` | `qa-gherkin-generator` with Jira issue AC (save locally or attach to Jira) |
337
- | Generate Playwright | `generate_playwright_code` (shared) | `generate_playwright_code` (shared) |
353
+ | Check duplicates | `get_linked_test_cases` | `get_linked_test_cases` |
354
+ | Create test cases (interactive) | `create_and_link_test_cases` | `create_and_link_test_cases` |
355
+ | Create test cases (headless) | `create_deduped_test_cases` | `create_deduped_test_cases` |
356
+ | Gherkin attach | `attach_gherkin_to_feature` / `attach_gherkin_to_work_item` | `attach_gherkin_to_issue` |
357
+ | Headless Gherkin composite | `generate_and_attach_gherkin` | `generate_and_attach_gherkin` + `attach_gherkin_to_issue` |
358
+ | Playwright generation | `generate_playwright_code` (shared) | `generate_playwright_code` (shared) |
338
359
  | Write to disk | `write_helix_files` (shared) | `write_helix_files` (shared) |
339
- | Authentication | MSAL silent + browser (`~/.msal-cache/`) | OAuth 2.0 (3LO) + browser (`~/.jira-cache/`) |
340
- | Link relation | `TestedBy-Forward` | `is tested by` / `Test` link type |
341
-
342
- ---
343
-
344
- ## Three-Layer Self-Healing Architecture
345
-
346
- | Layer | Class | What it heals |
347
- |---|---|---|
348
- | 1 — Locator | `LocatorHealer` | primary selector → role → label → text → AI Vision → CDPSession AX tree |
349
- | 2 — Timing | `TimingHealer` | network-trace drift → auto-adjusted timeouts → HealingDashboard |
350
- | 3 — Visual | `VisualIntentChecker` | element screenshot diff at assertions → HealingDashboard |
351
-
352
- Healed selectors persist in `LocatorRepository`. HealingDashboard: `http://localhost:7890`.
353
-
354
- All suggestions require human approval before entering version control.
360
+ | Link relation | `TestedBy-Forward` | `is tested by` / `Test` |
361
+ | Auth | MSAL silent + browser (`~/.msal-cache/`) | OAuth 2.0 3LO + browser (`~/.jira-cache/`) |
355
362
 
356
363
  ---
357
364
 
358
365
  ## Run Tests
359
366
 
360
367
  ```bash
368
+ # Full suite
361
369
  ENABLE_SELF_HEALING=true \
362
370
  HEALING_DASHBOARD_PORT=7890 \
363
371
  APP_BASE_URL=<your-app-base-url> \
@@ -371,96 +379,13 @@ cucumber-js --config=config/cucumber.js -p <feature_profile> --tags "@smoke"
371
379
 
372
380
  ---
373
381
 
374
- ## Scaffold a New QA Project
375
-
376
- The `scaffold` command copies the full Playwright + Cucumber + TypeScript boilerplate framework — including three-layer AI self-healing, BDD templates, and CI/CD integration — into a new project directory.
377
-
378
- ```bash
379
- # Scaffold into ./my-qa-project (default name)
380
- qa-stlc scaffold
381
-
382
- # Custom project name and location
383
- qa-stlc scaffold --name acme-e2e --dir /path/to/workspace/acme-e2e
384
-
385
- # Skip npm install (e.g. to inspect files first)
386
- qa-stlc scaffold --name acme-e2e --no-install
387
- ```
388
-
389
- After scaffolding:
390
-
391
- ```bash
392
- cd my-qa-project
393
- npx playwright install chromium # install browser binaries
394
- cp .env.example .env # already done by scaffold
395
- # Edit .env — set BASE_URL and optionally AI_API_KEY
396
- npm test # run the example scenario
397
- ```
398
-
399
- ### What Gets Scaffolded
400
-
401
- | Layer | Contents |
402
- |---|---|
403
- | **Config** | `package.json`, `tsconfig.json`, `cucumber.js`, `.env.example`, `.gitignore` |
404
- | **CI/CD** | `Dockerfile`, `docker-compose.yml`, `azure-pipelines.yml` |
405
- | **Framework** | `src/hooks/`, `src/pages/BasePage.ts`, `src/locators/`, `src/config/` |
406
- | **Self-healing** | All healing utilities: `LocatorHealer`, `TimingHealer`, `VisualIntentChecker`, `HealingDashboard`, `review-server`, `healix-ci-apply`, etc. |
407
- | **AI utilities** | `AILocatorGenerator`, `AISelfHealing`, `AITestGenerator` |
408
- | **Auth** | `AuthManager`, `AuthSetup` |
409
- | **Templates** | `_template.feature`, `_template.locators.ts`, `_template.steps.ts`, `_TemplatePage.ts` |
410
- | **Example test** | `src/test/features/example.feature` + steps |
411
-
412
- ### Healing Dashboard
413
-
414
- During a test run the live dashboard is available at **http://localhost:7890**. After the run use the review server for promote-to-source + PR creation:
415
-
416
- ```bash
417
- npm run healix:review # http://localhost:7891
418
- # or in CI:
419
- HEALIX_CI_AUTO_APPROVE=true npm run healix:apply-ci
420
- ```
421
-
422
- ---
423
-
424
- ## Development
425
-
426
- ```bash
427
- # Install (all five agents from one pip package)
428
- make install-ado # install + ADO next steps
429
- make install-jira # install + Jira next steps
430
- make install-both # install + Both next steps
431
-
432
- # Verify
433
- make verify-ado # ADO agents + MSAL auth cache
434
- make verify-jira # Jira agents + OAuth token cache
435
- make verify-both # all agents + both auth caches
436
-
437
- # Skills
438
- make skills-ado # ADO skill files → .claude/skills/ + copilot-instructions/
439
- make skills-jira # Jira skill files
440
- make skills-both # all skill files
441
-
442
- # MCP config
443
- make mcp-config-ado # write .mcp.json for ADO pipeline
444
- make mcp-config-jira # write .mcp.json for Jira pipeline
445
- make mcp-config-both # write .mcp.json for both pipelines
446
-
447
- # Test + clean
448
- make test # run pytest tests/ -v
449
- make clean # remove .venv, __pycache__, dist
450
- ```
451
-
452
- ---
453
-
454
382
  ## Documentation
455
383
 
456
- - [Quick Start: Azure DevOps](QUICKSTART-ADO.md)
457
- - [Quick Start: Jira Cloud](QUICKSTART-JIRA.md)
458
- - [Architecture: ADO](ARCHITECTURE-ADO.md)
459
- - [Architecture: Jira](ARCHITECTURE-JIRA.md)
460
- - [Walkthrough: ADO](WALKTHROUGH-ADO.md)
461
- - [Walkthrough: Jira](WALKTHROUGH-JIRA.md)
462
- - [Architecture Reference](docs/architecture.md)
463
- - [End-to-End Walkthrough](docs/walkthrough.md)
384
+ - [ARCHITECTURE-ADO.md](ARCHITECTURE-ADO.md) — Full technical architecture, ADO pipeline
385
+ - [ARCHITECTURE-JIRA.md](ARCHITECTURE-JIRA.md) — Full technical architecture, Jira pipeline
386
+ - [WALKTHROUGH-ADO.md](WALKTHROUGH-ADO.md) — End-to-end walkthrough, ADO pipeline
387
+ - [WALKTHROUGH-JIRA.md](WALKTHROUGH-JIRA.md) — End-to-end walkthrough, Jira pipeline
388
+ - [WEBHOOK.md](WEBHOOK.md) — Webhook bridge setup, deployment, and state trigger customisation
464
389
 
465
390
  ---
466
391