@chris1807/claude-kit 2.1.55 → 2.1.57
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +20 -10
- package/bin/cli.js +89 -4
- package/package.json +1 -1
- package/templates/agents/global/qa.md +136 -0
- package/templates/commands/cherry-pick.md +125 -14
- package/templates/commands/create-release.md +1 -1
- package/templates/commands/deploy-release.md +134 -27
- package/templates/commands/promote.md +120 -11
- package/templates/commands/qa.md +353 -0
- package/templates/hooks/sensitive-data-output-blocker.sh +56 -18
- package/templates/hooks/uat-reminder.sh +1 -0
- package/templates/infrastructure/CLAUDE-WORKFLOW.md +86 -13
- package/templates/infrastructure/settings.json +9 -0
package/README.md
CHANGED
|
@@ -181,6 +181,7 @@ The installer automatically **skips global agents** that are already installed (
|
|
|
181
181
|
├── mockup.md # HTML mockup designer
|
|
182
182
|
├── reviewer.md # Code reviewer (read-only)
|
|
183
183
|
├── test-runner.md # Test executor — xUnit, Vitest, Playwright (read-only)
|
|
184
|
+
├── qa.md # Browser QA on a deployed environment (read-only)
|
|
184
185
|
├── build-validator.md # Build checker (read-only)
|
|
185
186
|
├── lint-checker.md # ESLint + dotnet format checker
|
|
186
187
|
├── azure-ops.md # Azure infrastructure management
|
|
@@ -329,7 +330,8 @@ Claude automatically:
|
|
|
329
330
|
2. Cherry-picks their commits into `release/23-to-staging`
|
|
330
331
|
3. Creates a PR targeting the `staging` branch — merging it triggers the Staging pipeline
|
|
331
332
|
4. Links all work items to the PR
|
|
332
|
-
5.
|
|
333
|
+
5. Asks who verifies each product group on Staging (`COM`, `PAY`, …) — candidates are the non-developer names already on those work items
|
|
334
|
+
6. Watches the PR and the Staging pipeline; once it's green, moves the work items to `Staging` and assigns them to the approved verifiers
|
|
333
335
|
|
|
334
336
|
The same command walks a release through `test` → `staging` → `prod`; with no environment given it picks the next one.
|
|
335
337
|
|
|
@@ -339,7 +341,7 @@ The same command walks a release through `test` → `staging` → `prod`; with n
|
|
|
339
341
|
/cherry-pick AB#1234 AB#1235 prod
|
|
340
342
|
```
|
|
341
343
|
|
|
342
|
-
Cherry-picks specific work items to an environment without a formal release. Same gate check and
|
|
344
|
+
Cherry-picks specific work items to an environment without a formal release. Same gate check, same verifier question, and the same automatic state advance + assignment once the pipeline is green as `/deploy-release`. `main` is also a valid target — that's how a hot fix that went straight to `prod` is brought back into the compare branch.
|
|
343
345
|
|
|
344
346
|
### Promote an Environment
|
|
345
347
|
|
|
@@ -349,7 +351,7 @@ Cherry-picks specific work items to an environment without a formal release. Sam
|
|
|
349
351
|
/promote ← auto-detects from the current branch
|
|
350
352
|
```
|
|
351
353
|
|
|
352
|
-
Creates a PR to promote all code from one branch to the next in the chain (`main → dev → test → staging → prod`). Shows every commit and work item before confirming, gate-checks the work items, and advances
|
|
354
|
+
Creates a PR to promote all code from one branch to the next in the chain (`main → dev → test → staging → prod`). Shows every commit and work item before confirming, gate-checks the work items, asks who verifies each product group, and advances and assigns them once the merge's pipeline is green. `main → dev` is the usual first hop after feature PRs merge — nothing has deployed before that.
|
|
353
355
|
|
|
354
356
|
### Rollback a Deployment
|
|
355
357
|
|
|
@@ -439,7 +441,7 @@ Every environment has a work item state, and the promotion commands keep them in
|
|
|
439
441
|
| `test → staging` | `Ready for Staging` (QA sign-off) | `Staging` |
|
|
440
442
|
| `staging → prod` | `Ready to Deploy` (stakeholder sign-off) | `Deployed` |
|
|
441
443
|
|
|
442
|
-
`/deploy-release`, `/cherry-pick`, and `/promote` check the gate before creating the PR,
|
|
444
|
+
`/deploy-release`, `/cherry-pick`, and `/promote` check the gate before creating the PR, ask who verifies each product group while the PR is in review, then watch the PR and the environment's CD pipeline — on a green pipeline they set the state and the assignee together. A failed pipeline changes nothing. `/status` flags anything whose state lags the branch it's on.
|
|
443
445
|
|
|
444
446
|
### Branch Naming
|
|
445
447
|
|
|
@@ -507,7 +509,7 @@ The PR merges into `main`. **That merge deploys nothing** — `main` is the comp
|
|
|
507
509
|
/promote main dev
|
|
508
510
|
```
|
|
509
511
|
|
|
510
|
-
Merge that PR,
|
|
512
|
+
Merge that PR. Claude asks who verifies each product group on Dev, watches the Dev pipeline, and once it's green moves the work items to `Ready for Testing` and assigns them to those people.
|
|
511
513
|
|
|
512
514
|
### Step 2: Deploy Changes (Quick Commits)
|
|
513
515
|
|
|
@@ -550,7 +552,8 @@ Claude will:
|
|
|
550
552
|
4. Cherry-pick all commits for each work item
|
|
551
553
|
5. Create a PR from `release/23-to-test` → `test`
|
|
552
554
|
6. Link all work items to the PR
|
|
553
|
-
7.
|
|
555
|
+
7. Ask who verifies each product group on Test
|
|
556
|
+
8. Watch the PR and the Test pipeline; on a green pipeline, move the work items to `Testing` and assign them to the approved verifiers
|
|
554
557
|
|
|
555
558
|
After the PR is reviewed and merged, the Test CD pipeline triggers automatically. QA tests there and sets each passing item to `Ready for Staging`; then `/deploy-release 23 staging` repeats the process (gate `Ready for Staging`, state after merge `Staging`).
|
|
556
559
|
|
|
@@ -566,7 +569,7 @@ When stakeholders have verified on Staging and set the items to `Ready to Deploy
|
|
|
566
569
|
/deploy-release 23 prod
|
|
567
570
|
```
|
|
568
571
|
|
|
569
|
-
Same process — gate `Ready to Deploy`, cherry-picks from `staging` into a PR targeting `prod`. After merge
|
|
572
|
+
Same process — gate `Ready to Deploy`, cherry-picks from `staging` into a PR targeting `prod`. After merge the Production CD pipeline triggers; when it comes back green the items move to `Deployed` and are assigned to whoever you named for each product group.
|
|
570
573
|
|
|
571
574
|
### Selective Deployment
|
|
572
575
|
|
|
@@ -671,12 +674,13 @@ Claude reviews for:
|
|
|
671
674
|
| `/fix-review` | `/fix-review 142` | Fix everything flagged on a PR — human reviewer comments and automated `/review` findings alike: implement in severity order, validate, push, resolve threads |
|
|
672
675
|
| `/deploy` | `/deploy "message"` | Commit, push, trigger pipeline if on an environment branch (`dev`/`test`/`staging`/`prod`) — never on `main` |
|
|
673
676
|
| `/create-release` | `/create-release 23` | Group work items into Release #23 iteration with tags |
|
|
674
|
-
| `/deploy-release` | `/deploy-release 23 staging` | Gate-check → cherry-pick release work items to environment via PR → advance
|
|
677
|
+
| `/deploy-release` | `/deploy-release 23 staging` | Gate-check → cherry-pick release work items to environment via PR → ask who verifies → advance + assign on a green pipeline |
|
|
675
678
|
| `/add-to-release` | `/add-to-release 24 AB#4599` | Add work items to an existing release |
|
|
676
|
-
| `/cherry-pick` | `/cherry-pick AB#1234 AB#1235 prod` | Gate-check → cherry-pick specific work items to environment via PR → advance
|
|
677
|
-
| `/promote` | `/promote main dev` | PR to promote all code to the next branch in the chain (`main → dev → test → staging → prod`) → advance
|
|
679
|
+
| `/cherry-pick` | `/cherry-pick AB#1234 AB#1235 prod` | Gate-check → cherry-pick specific work items to environment via PR → ask who verifies → advance + assign on a green pipeline |
|
|
680
|
+
| `/promote` | `/promote main dev` | PR to promote all code to the next branch in the chain (`main → dev → test → staging → prod`) → ask who verifies → advance + assign on a green pipeline |
|
|
678
681
|
| `/rollback` | `/rollback AB#1234 prod` | Revert specific commits on an environment via PR |
|
|
679
682
|
| `/status` | `/status release 24` | Check status of a release, pipeline, work item, or environment; flags work items whose state lags their environment |
|
|
683
|
+
| `/qa` | `/qa AB#1234 [env]` | Verify the item is fully deployed with a green pipeline → open the app in a real browser → sign in as a test account → full regression of the screens the story touched → pass/fail comment on the work item (never a state change). Environments: `local`, `dev`, `test`, `staging` |
|
|
680
684
|
| `/plan-backlog` | `/plan-backlog [project]` | Sweep backlog for Dev Ready stories with points and no tasks → propose one child task with hours per story |
|
|
681
685
|
| `/plan-sprint` | `/plan-sprint [project]` | Sweep the current sprint for stories/bugs with no child tasks → propose one child task with hours per item |
|
|
682
686
|
| `/quote-backlog` | `/quote-backlog [project]` | Sweep backlog for unpointed items ready to estimate — stories in `Design Approved`, bugs in `New` (bugs have no design states) → review completeness, check for duplicates, suggest rewrites, propose points + creator comments (10 at a time, approval-gated). Also audits the drop-out queue — tagged bugs and stories bounced to Design Review — for items whose creator answered but which nobody returned to the sweep |
|
|
@@ -729,6 +733,12 @@ export STRIPE_SECRET_KEY="sk_test_..."
|
|
|
729
733
|
# Azure CLI (no env var needed — just log in)
|
|
730
734
|
az login
|
|
731
735
|
|
|
736
|
+
# QA test accounts (required for /qa — browser testing on a deployed environment).
|
|
737
|
+
# One pair per role. The variable NAMES go in your project's `## Environment URLs`
|
|
738
|
+
# section; the VALUES live only here. Use throwaway least-privilege QA accounts.
|
|
739
|
+
export COMPASS_QA_ADMIN_USERNAME="qa-admin@example.com"
|
|
740
|
+
export COMPASS_QA_ADMIN_PASSWORD="..."
|
|
741
|
+
|
|
732
742
|
# Microsoft Teams (no env vars needed — uses Microsoft Graph device-code auth;
|
|
733
743
|
# first invocation prints a code + URL to sign in)
|
|
734
744
|
```
|
package/bin/cli.js
CHANGED
|
@@ -161,6 +161,74 @@ function mergeHooks(existing, template) {
|
|
|
161
161
|
// don't linger. manager/uat-generator/api-tester were superseded by the
|
|
162
162
|
// Workflow tool and inline command steps; devops-tracker by direct Azure
|
|
163
163
|
// DevOps MCP access in the main loop.
|
|
164
|
+
// The workflow section written into a target CLAUDE.md is delimited by this marker so
|
|
165
|
+
// a re-install can replace it without touching anything the project added afterwards.
|
|
166
|
+
// Installs from before the marker existed are recovered heuristically — see
|
|
167
|
+
// splitWorkflowSection.
|
|
168
|
+
const WORKFLOW_END_MARKER = '<!-- claude-kit:workflow:end -->';
|
|
169
|
+
|
|
170
|
+
const WORKFLOW_HEADING_RE = /\n## (?:Claude Kit Workflow|Care Solutions AI Workflow)/;
|
|
171
|
+
|
|
172
|
+
/**
|
|
173
|
+
* Split a target CLAUDE.md around the kit's workflow section.
|
|
174
|
+
*
|
|
175
|
+
* Returns { head, preserved } where `head` is everything before the workflow
|
|
176
|
+
* heading and `preserved` is project content that followed the kit's block —
|
|
177
|
+
* or null when there is no workflow section at all.
|
|
178
|
+
*
|
|
179
|
+
* The old implementation replaced /heading[\s\S]*$/, i.e. everything to end of
|
|
180
|
+
* file, which silently deleted any `## Pipeline Configuration` or
|
|
181
|
+
* `## Environment URLs` a project had placed below the heading. With the marker
|
|
182
|
+
* present the boundary is exact. Without it (a pre-marker install) we fall back
|
|
183
|
+
* to a heading scan: the template's only top-level heading outside a fenced code
|
|
184
|
+
* block is the workflow heading itself, so any other non-fenced `## ` heading in
|
|
185
|
+
* the tail is project content and is kept. Fenced headings are skipped because
|
|
186
|
+
* the template quotes `## Pipeline Configuration` inside ```markdown examples.
|
|
187
|
+
*/
|
|
188
|
+
function splitWorkflowSection(existing) {
|
|
189
|
+
const match = existing.match(WORKFLOW_HEADING_RE);
|
|
190
|
+
if (!match) return null;
|
|
191
|
+
|
|
192
|
+
const head = existing.slice(0, match.index).trimEnd();
|
|
193
|
+
const tail = existing.slice(match.index);
|
|
194
|
+
|
|
195
|
+
const markerIdx = tail.indexOf(WORKFLOW_END_MARKER);
|
|
196
|
+
if (markerIdx !== -1) {
|
|
197
|
+
return {
|
|
198
|
+
head,
|
|
199
|
+
preserved: tail.slice(markerIdx + WORKFLOW_END_MARKER.length).trim(),
|
|
200
|
+
recovered: false,
|
|
201
|
+
};
|
|
202
|
+
}
|
|
203
|
+
|
|
204
|
+
const lines = tail.split('\n');
|
|
205
|
+
// `tail` begins with the newline preceding the heading, so the heading is not at
|
|
206
|
+
// index 0. Find it explicitly and scan from the line after it — starting at a
|
|
207
|
+
// fixed offset would treat the heading itself as project content.
|
|
208
|
+
const headingIdx = lines.findIndex((l) =>
|
|
209
|
+
/^## (?:Claude Kit Workflow|Care Solutions AI Workflow)/.test(l)
|
|
210
|
+
);
|
|
211
|
+
let inFence = false;
|
|
212
|
+
let cut = -1;
|
|
213
|
+
for (let i = headingIdx + 1; i < lines.length; i++) {
|
|
214
|
+
if (/^\s*```/.test(lines[i])) {
|
|
215
|
+
inFence = !inFence;
|
|
216
|
+
continue;
|
|
217
|
+
}
|
|
218
|
+
if (inFence) continue;
|
|
219
|
+
if (/^## /.test(lines[i])) {
|
|
220
|
+
cut = i;
|
|
221
|
+
break;
|
|
222
|
+
}
|
|
223
|
+
}
|
|
224
|
+
|
|
225
|
+
return {
|
|
226
|
+
head,
|
|
227
|
+
preserved: cut === -1 ? '' : lines.slice(cut).join('\n').trim(),
|
|
228
|
+
recovered: cut !== -1,
|
|
229
|
+
};
|
|
230
|
+
}
|
|
231
|
+
|
|
164
232
|
const RETIRED_GLOBAL_AGENTS = ['manager.md', 'uat-generator.md', 'api-tester.md'];
|
|
165
233
|
const RETIRED_PROJECT_AGENTS = ['devops-tracker.md'];
|
|
166
234
|
|
|
@@ -515,13 +583,16 @@ async function main() {
|
|
|
515
583
|
'utf8'
|
|
516
584
|
);
|
|
517
585
|
|
|
586
|
+
// Delimited so a re-install replaces only the kit's own section.
|
|
587
|
+
const workflowBlock = `${workflowContent.trimEnd()}\n\n${WORKFLOW_END_MARKER}\n`;
|
|
588
|
+
|
|
518
589
|
const sensitiveDataPolicy = `## SENSITIVE DATA — MANDATORY RULE
|
|
519
590
|
|
|
520
591
|
**NEVER query, display, read, grep, or expose sensitive PII fields from the database or codebase — even if the values are encrypted.** Blocked fields: TIN, SSN, EIN, TaxId, BankAccountNumber, RoutingNumber, and any \`Encrypted*\` variants. Always use explicit inclusion projections listing only non-sensitive fields. Direct users to the application UI for sensitive data access.
|
|
521
592
|
`;
|
|
522
593
|
|
|
523
594
|
if (!await fs.pathExists(claudeMdPath)) {
|
|
524
|
-
await fs.writeFile(claudeMdPath, `# ${basename(targetDir)}\n\n${sensitiveDataPolicy}\n${
|
|
595
|
+
await fs.writeFile(claudeMdPath, `# ${basename(targetDir)}\n\n${sensitiveDataPolicy}\n${workflowBlock}`);
|
|
525
596
|
console.log(chalk.green(' ✓ Created CLAUDE.md with sensitive data policy and workflow'));
|
|
526
597
|
} else {
|
|
527
598
|
let existing = await fs.readFile(claudeMdPath, 'utf8');
|
|
@@ -547,8 +618,22 @@ async function main() {
|
|
|
547
618
|
if (existing.includes('Claude Kit Workflow') || existing.includes('Care Solutions AI Workflow')) {
|
|
548
619
|
// Check whether the existing workflow section matches the current template.
|
|
549
620
|
// If it does, nothing to do. If not, replace it (auto in --all, prompt otherwise).
|
|
550
|
-
const
|
|
551
|
-
const
|
|
621
|
+
const split = splitWorkflowSection(existing);
|
|
622
|
+
const { head, preserved, recovered } = split;
|
|
623
|
+
const expected = preserved
|
|
624
|
+
? `${head}\n\n${workflowBlock}\n${preserved}\n`
|
|
625
|
+
: `${head}\n\n${workflowBlock}`;
|
|
626
|
+
if (recovered && preserved) {
|
|
627
|
+
const names = (preserved.match(/^## .*$/gm) || []).map((h) => h.replace(/^##\s*/, ''));
|
|
628
|
+
console.log(
|
|
629
|
+
chalk.yellow(
|
|
630
|
+
` ! Kept project content found below the workflow section: ${names.join(', ') || 'unlabelled section'}`
|
|
631
|
+
)
|
|
632
|
+
);
|
|
633
|
+
console.log(
|
|
634
|
+
chalk.gray(' It now sits after the end marker, so future updates will leave it alone.')
|
|
635
|
+
);
|
|
636
|
+
}
|
|
552
637
|
if (existing.trim() === expected.trim()) {
|
|
553
638
|
console.log(chalk.gray(' = Workflow section up to date'));
|
|
554
639
|
} else if (installAll) {
|
|
@@ -569,7 +654,7 @@ async function main() {
|
|
|
569
654
|
}
|
|
570
655
|
}
|
|
571
656
|
} else {
|
|
572
|
-
await fs.appendFile(claudeMdPath, `\n\n${
|
|
657
|
+
await fs.appendFile(claudeMdPath, `\n\n${workflowBlock}`);
|
|
573
658
|
console.log(chalk.green(' ✓ Workflow appended to CLAUDE.md'));
|
|
574
659
|
}
|
|
575
660
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@chris1807/claude-kit",
|
|
3
|
-
"version": "2.1.
|
|
3
|
+
"version": "2.1.57",
|
|
4
4
|
"description": "Claude Code starter kit for Azure DevOps teams — agents, hooks, MCP servers, slash commands, and end-to-end work item → PR → release → deploy workflow automation",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: qa
|
|
3
|
+
description: Regression-tests a work item in a real browser via Playwright MCP. Signs in with test-account credentials from environment variables, exercises the affected screens, and reports pass/fail with evidence. Read-only against the codebase and against Azure DevOps.
|
|
4
|
+
tools: Read, Glob, Grep, Bash, mcp__playwright__browser_navigate, mcp__playwright__browser_navigate_back, mcp__playwright__browser_snapshot, mcp__playwright__browser_find, mcp__playwright__browser_click, mcp__playwright__browser_type, mcp__playwright__browser_fill_form, mcp__playwright__browser_select_option, mcp__playwright__browser_press_key, mcp__playwright__browser_hover, mcp__playwright__browser_wait_for, mcp__playwright__browser_resize, mcp__playwright__browser_handle_dialog, mcp__playwright__browser_console_messages, mcp__playwright__browser_network_requests, mcp__playwright__browser_take_screenshot, mcp__playwright__browser_tabs, mcp__playwright__browser_close
|
|
5
|
+
disallowedTools: Write, Edit
|
|
6
|
+
model: sonnet
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# QA Agent
|
|
10
|
+
|
|
11
|
+
You test a work item in a real browser against a running environment, then report what you found. You are **read-only** against the codebase and you have **no Azure DevOps tools** — you never post a comment, never change a work item state, and never ask the caller a question. You return one report; the main loop does the writing.
|
|
12
|
+
|
|
13
|
+
## Tools you deliberately do not have
|
|
14
|
+
|
|
15
|
+
Do not work around these. If a check genuinely needs one, report the check as `NOT TESTABLE` and say which capability was missing.
|
|
16
|
+
|
|
17
|
+
| Missing | Why it was withheld |
|
|
18
|
+
|---|---|
|
|
19
|
+
| `browser_evaluate`, `browser_run_code_unsafe` | Arbitrary page JS could read any DOM value and defeat every rule below. The omission is the enforcement. |
|
|
20
|
+
| `browser_network_request` (singular) | It returns full request and response bodies — the likeliest route for a TIN or SSN into a transcript. Use `browser_network_requests` (plural), which gives method, URL, and status. |
|
|
21
|
+
| `browser_file_upload`, `browser_drag`, `browser_drop` | Not needed by the checklist below. |
|
|
22
|
+
| Any `mcp__azure-devops__*` tool | You must be structurally unable to advance a work item. |
|
|
23
|
+
| `Write`, `Edit` | You test the app; you never change the code. |
|
|
24
|
+
|
|
25
|
+
## What you are given
|
|
26
|
+
|
|
27
|
+
The caller passes a brief containing: the work item id, title, type and state; the target environment name and its App URL (and API URL if known); a **baseline** environment URL for differential re-checks; the sign-in descriptor; the **names** of the environment variables holding the test account credentials; the ordered list of screens with each screen's changed files; and the acceptance criteria.
|
|
28
|
+
|
|
29
|
+
You are never given a password. You read it yourself.
|
|
30
|
+
|
|
31
|
+
## Signing in
|
|
32
|
+
|
|
33
|
+
1. Read the credentials with Bash — `printenv <USERNAME_VAR>` and `printenv <PASSWORD_VAR>`. If either is empty, stop immediately and report `BLOCKED — <VAR> is not set`.
|
|
34
|
+
2. `browser_navigate` to the App URL, then to the login route from the sign-in descriptor if you are not redirected there.
|
|
35
|
+
3. Fill both fields in **one** `browser_fill_form` call, then submit. One call, not two `browser_type` calls — the credential should appear once.
|
|
36
|
+
4. Confirm you are signed in by finding an element that only exists when authenticated (the descriptor names the landing route). If you land back on the login form, stop and report `BLOCKED — sign-in failed`. Do not retry more than once; a second failure on a shared environment risks locking the account.
|
|
37
|
+
5. If the browser is redirected to an external identity provider (`login.microsoftonline.com` or similar) or an MFA prompt appears, stop and report `BLOCKED — sign-in requires MFA/SSO, cannot run unattended`.
|
|
38
|
+
|
|
39
|
+
**Never** echo, log, repeat, screenshot, or include either credential value in your report. Never write one to a file.
|
|
40
|
+
|
|
41
|
+
## Regression Method
|
|
42
|
+
|
|
43
|
+
Work the screens in the order given. For each one, run **every** row of the checklist — the acceptance criteria are additional, not a substitute. Record a result per row before moving on.
|
|
44
|
+
|
|
45
|
+
| # | Test | How | Expected |
|
|
46
|
+
|---|------|-----|----------|
|
|
47
|
+
| 1 | Route loads | `browser_navigate` to the route | Renders. No error boundary, no blank region |
|
|
48
|
+
| 2 | Deep link | Navigate straight to the URL rather than clicking through | Route rehydrates; no bounce to login, no 404 |
|
|
49
|
+
| 3 | Console clean | `browser_console_messages` | No uncaught errors. React key/prop warnings are noted, not failed |
|
|
50
|
+
| 4 | Network clean | `browser_network_requests` | No 4xx/5xx to the API. Record `METHOD path → status` only |
|
|
51
|
+
| 5 | Data or empty state | `browser_snapshot` | Rows render, **or** an explicit empty-state message. A blank region is a **fail** |
|
|
52
|
+
| 6 | Filters, search, sort | Exercise each control once | Result set changes, no error, control reflects its state |
|
|
53
|
+
| 7 | Pagination | Next, then previous | Content changes; the indicator tracks |
|
|
54
|
+
| 8 | Form — invalid | Submit each form empty | **Inline** validation; nothing submitted |
|
|
55
|
+
| 9 | Form — valid | Submit valid data | Success indication; the list or detail updates |
|
|
56
|
+
| 10 | Form — cancel | Fill, then cancel | No mutation; returns to the prior view |
|
|
57
|
+
| 11 | Dialogs | Open and close each modal and row action | Opens, closes, focus returns. A native `window.confirm`/`alert`/`prompt` is a **fail** |
|
|
58
|
+
| 12 | Back button | Browser back after a mutation | Lands somewhere coherent; no stale or duplicated data |
|
|
59
|
+
| 13 | Responsive | `browser_resize` to 1440, then 768 | No overlap, no clipped controls, no horizontal scroll |
|
|
60
|
+
| 14 | Role gating | If a second role's credentials are named in the brief, sign in as it and revisit | Restricted controls are **absent**, not present-and-erroring |
|
|
61
|
+
|
|
62
|
+
Then one explicit test per acceptance criterion: `PASS`, `FAIL`, or `NOT TESTABLE (reason)`.
|
|
63
|
+
|
|
64
|
+
**Retry once before recording a failure**, using `browser_wait_for` on the element you expect rather than a fixed delay. A cold-start 502 on a Dev App Service is the most common false failure. If it passes on the retry, record `FLAKY`, not `PASS`.
|
|
65
|
+
|
|
66
|
+
**Differential re-check.** For each failure, if the brief gave you a baseline URL, re-run *only* the failing steps there. Fails on the baseline too → the issue predates this story. Passes on the baseline → this story introduced it. Report the outcome per failure; do not draw the final conclusion, the caller attributes it. Cap at 5 re-checks and say if you hit the cap.
|
|
67
|
+
|
|
68
|
+
## Write Safety
|
|
69
|
+
|
|
70
|
+
You are operating on a shared environment with real integrations. A wrong click here sends real money or real email.
|
|
71
|
+
|
|
72
|
+
- Prefer creating a new record over editing an existing one.
|
|
73
|
+
- Prefix everything you create with `QA AB#<id>` so a human can find it.
|
|
74
|
+
- **Never delete a pre-existing record.**
|
|
75
|
+
- **Never** fire a bulk or destructive action, a payment submission or settlement, a batch approval, a 1099 or tax-document generation, an export-all, or anything that emails or notifies a real person.
|
|
76
|
+
- If a screen can only be exercised by a destructive action, record `NOT TESTABLE — destructive` and move on.
|
|
77
|
+
- List everything you created in the report.
|
|
78
|
+
|
|
79
|
+
## Sensitive Data
|
|
80
|
+
|
|
81
|
+
The project's `CLAUDE.md` forbids exposing TIN, SSN, EIN, TaxId, BankAccountNumber, RoutingNumber, and any `Encrypted*` variant — including encrypted values. On a real screen these are rendered, so:
|
|
82
|
+
|
|
83
|
+
- **Never transcribe a field value** from a snapshot. Report only *present*, *populated*, *empty*, *masked*, or *validation shown*.
|
|
84
|
+
- Prefer `browser_find` over a full `browser_snapshot` when you only need to check one element on a screen that displays protected fields.
|
|
85
|
+
- **Never screenshot a screen showing a protected value.** Screenshots are failure evidence only, saved outside the repo, and referenced by local path — they are never attached to a work item.
|
|
86
|
+
- If a screen exists to display protected data, test its structure, validation, and navigation, and record the content checks as `NOT TESTABLE — sensitive screen`.
|
|
87
|
+
- If a tool result comes back blocked by the sensitive-data hook, do not retry it. Record `BLOCKED — protected fields on screen, manual QA required` and move to the next screen.
|
|
88
|
+
|
|
89
|
+
## Report Format
|
|
90
|
+
|
|
91
|
+
```markdown
|
|
92
|
+
## QA — AB#{id}: {title}
|
|
93
|
+
{type} · {state} · {environment} — {app-url}
|
|
94
|
+
Signed in as: {role} (from {USERNAME_VAR}) — value not shown
|
|
95
|
+
Screens tested: {n} · {route list}
|
|
96
|
+
|
|
97
|
+
### Acceptance Criteria
|
|
98
|
+
| # | Criterion | Result | Evidence |
|
|
99
|
+
|---|-----------|--------|----------|
|
|
100
|
+
| 1 | {criterion} | PASS | {route} — {what you observed} |
|
|
101
|
+
|
|
102
|
+
### Screen Regression — {route}
|
|
103
|
+
| # | Test | Steps | Expected Result | Result |
|
|
104
|
+
|---|------|-------|-----------------|--------|
|
|
105
|
+
| 1 | Route loads | Navigate to {route} | Renders, no error boundary | PASS |
|
|
106
|
+
|
|
107
|
+
### Failures
|
|
108
|
+
#### F1 — {short title}
|
|
109
|
+
- Screen: {route}
|
|
110
|
+
- Steps: 1. … 2. … 3. …
|
|
111
|
+
- Expected: {what should happen}
|
|
112
|
+
- Actual: {what happened}
|
|
113
|
+
- Evidence: console `{first line}` | network `POST /api/x → 500`
|
|
114
|
+
- Differential: {passes on baseline | fails on baseline too | not checked — {reason}}
|
|
115
|
+
- Touches this story's changed files: {yes — <file> | no}
|
|
116
|
+
- Screenshot: {local path}
|
|
117
|
+
|
|
118
|
+
### Not testable
|
|
119
|
+
- {check} — {why}
|
|
120
|
+
|
|
121
|
+
### Test data created
|
|
122
|
+
- {entity} "QA AB#{id} — …" (id {x})
|
|
123
|
+
|
|
124
|
+
**Sweep status:** COMPLETE | INCOMPLETE — {what was not reached}
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
## Rules
|
|
128
|
+
|
|
129
|
+
- Never modify any file. Never post anything anywhere. Never ask the caller a question — report and stop.
|
|
130
|
+
- Never change a work item state. You have no tool that can, and that is deliberate.
|
|
131
|
+
- Never echo a credential value, and never include one in the report.
|
|
132
|
+
- Never transcribe a protected field's value. See Sensitive Data.
|
|
133
|
+
- A control you could not find is a **FAIL**, never a PASS. "I didn't see it" is not "it works."
|
|
134
|
+
- If you run out of room and cannot finish the screens, report `INCOMPLETE` and name what you did not reach. A truncated sweep must never read as a pass.
|
|
135
|
+
- Report what you observed, not what you expected the code to do. You are testing the deployed build, not the source.
|
|
136
|
+
- Close the browser (`browser_close`) when you are done.
|
|
@@ -73,7 +73,7 @@ Options:
|
|
|
73
73
|
Which option? (1 / 2 / 3)
|
|
74
74
|
```
|
|
75
75
|
|
|
76
|
-
- **Option 1:** Run `git cherry-pick --skip` and continue with remaining work items. Note the skipped item in the summary — it does **not** get its state advanced in Step
|
|
76
|
+
- **Option 1:** Run `git cherry-pick --skip` and continue with remaining work items. Note the skipped item in the summary — it does **not** get its state advanced or its assignee changed in Step 8.
|
|
77
77
|
- **Option 2:** Run `git cherry-pick --abort`, delete the cherry-pick branch, and switch back to the original branch.
|
|
78
78
|
- **Option 3:** Wait for the user to resolve conflicts and run `git cherry-pick --continue`, then proceed.
|
|
79
79
|
|
|
@@ -94,32 +94,143 @@ Cherry-picking:
|
|
|
94
94
|
- **description**: List all work items with IDs and titles
|
|
95
95
|
3. Link all work items to the PR via `wit_link_work_item_to_pull_request`
|
|
96
96
|
|
|
97
|
-
## Step 6:
|
|
97
|
+
## Step 6: Pick Who Verifies Each Group (before the merge)
|
|
98
|
+
|
|
99
|
+
When the target is `main` there is no pipeline, no state change, and **no assignment** — say so and skip to Step 8's summary.
|
|
100
|
+
|
|
101
|
+
Otherwise the items are about to land in {environment} for **someone** to verify. Decide who now, while the PR is still in review — nothing is written yet; the assignment is applied in Step 8 together with the state change, once the pipeline is green.
|
|
102
|
+
|
|
103
|
+
### 6a. Group by title prefix
|
|
104
|
+
|
|
105
|
+
Work item titles in the CSI Development project are `PREFIX - Title` (`COM`, `PAY`, `CDA`, `AUD`, `SER`, `PSSF`, `TPS`, `ILP`, `RBWO`, `MTG`, `CSI`). Group the cherry-picked **User Stories, Bugs, and Hot Fixes** by that prefix. Titles with a missing or unrecognized prefix go in a `(no prefix)` group. Items skipped on a conflict (Step 4) or dropped at the gate (Step 1) are not on the branch — leave them out. Features and Tasks are never assigned by this command.
|
|
106
|
+
|
|
107
|
+
One question per group. Different products usually have different verifiers, but the same person may take several groups — that is the normal case, not an error.
|
|
108
|
+
|
|
109
|
+
### 6b. Build each group's candidate list from the items themselves
|
|
110
|
+
|
|
111
|
+
For every item in the group, collect the distinct identities already on it:
|
|
112
|
+
|
|
113
|
+
- **whoever set the gate state** — `wit_work_item` `action: list_revisions`, then the `System.ChangedBy` on the revision where `System.State` became the target's gate state (`Ready for Staging`, `Ready to Deploy`, …). That is the QA person or stakeholder who signed the item off, and it is usually the right verifier.
|
|
114
|
+
- `System.CreatedBy` — who asked for the work
|
|
115
|
+
- comment authors — `wit_work_item` `action: list_comments`
|
|
116
|
+
- `System.ChangedBy` — the most recent editor
|
|
117
|
+
|
|
118
|
+
Then **drop the developers**, so what remains is the non-developer names on the item:
|
|
119
|
+
|
|
120
|
+
- each item's current `System.AssignedTo` (the dev who implemented it)
|
|
121
|
+
- the PR author
|
|
122
|
+
- the authors of the cherry-picked commits — `git log --format='%an <%ae>' <target-branch>..HEAD | sort -u`
|
|
123
|
+
|
|
124
|
+
Rank the survivors by how many items in the group they appear on — the gate-state signer first, then creators. **Do not invent names**, do not pull from the org directory or a team list, and do not carry a name over from another group's items — every candidate must come from the items in that group.
|
|
125
|
+
|
|
126
|
+
If the developer filter empties a group's list, say so and show the unfiltered names marked `(also a developer here)`, plus the option to type a name.
|
|
127
|
+
|
|
128
|
+
### 6c. Ask
|
|
129
|
+
|
|
130
|
+
```
|
|
131
|
+
## Who verifies on {environment}?
|
|
132
|
+
|
|
133
|
+
**COM** — 2 items (AB#1234, AB#1236)
|
|
134
|
+
1. Jane Doe (jane@caresolutions.com) — set Ready to Deploy on both
|
|
135
|
+
2. Sam Lee (sam@caresolutions.com) — commented on AB#1234
|
|
136
|
+
3. Someone else — give me a name or email
|
|
137
|
+
4. Leave the current assignee alone
|
|
138
|
+
|
|
139
|
+
**PAY** — 2 items (AB#1240, AB#1241)
|
|
140
|
+
1. Pat Ruiz (pat@caresolutions.com) — created both
|
|
141
|
+
2. Jane Doe (jane@caresolutions.com) — commented on AB#1240
|
|
142
|
+
3. Someone else
|
|
143
|
+
4. Leave the current assignee alone
|
|
144
|
+
|
|
145
|
+
Reply per group — `COM 1, PAY 1` — or `all 1` to give every group to the same person.
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
### 6d. Confirm the mapping
|
|
149
|
+
|
|
150
|
+
Echo the resolved plan and wait for an explicit `yes`:
|
|
151
|
+
|
|
152
|
+
```
|
|
153
|
+
When the {environment} pipeline goes green I will apply:
|
|
154
|
+
|
|
155
|
+
| ID | Title | Prefix | Assign to | State |
|
|
156
|
+
|----|-------|--------|-----------|-------|
|
|
157
|
+
| AB#1234 | COM - Add payment export | COM | Jane Doe | Deployed |
|
|
158
|
+
| AB#1240 | PAY - Fix login redirect | PAY | Pat Ruiz | Deployed |
|
|
159
|
+
|
|
160
|
+
Approve? (yes / change)
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
`change` re-asks 6c. Only after `yes` do you move on — if the PR merges before the user answers, keep waiting for the answer, then run Step 8.
|
|
164
|
+
|
|
165
|
+
## Step 7: Watch the Merge, Then the Pipeline
|
|
166
|
+
|
|
167
|
+
Do NOT merge the PR yourself — a reviewer approves and completes it. Report the PR, then watch for the deployment:
|
|
98
168
|
|
|
99
169
|
```
|
|
100
170
|
Cherry-pick PR created for {environment}.
|
|
101
171
|
|
|
102
172
|
PR: {pr-url}
|
|
103
173
|
Work items:
|
|
104
|
-
- AB#1234: Add payment export
|
|
105
|
-
- AB#
|
|
174
|
+
- AB#1234: COM - Add payment export
|
|
175
|
+
- AB#1240: PAY - Fix login redirect
|
|
176
|
+
|
|
177
|
+
Merging the PR triggers the CD pipeline for {environment}. I'm watching the PR
|
|
178
|
+
and that pipeline — when it comes back green I'll set these {count} items to
|
|
179
|
+
`{state after merge}` and assign them as approved above. Nothing is written
|
|
180
|
+
until then. Say `stop watching` to leave the states alone.
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
1. **Wait for the PR to complete.** Poll `repo_pull_request` until `status` is `completed`; note the merge commit and completion time. `abandoned` → stop, change nothing, report it.
|
|
184
|
+
2. **Find the CD run.** Look up the target branch's pipeline ID(s) in the Pipeline Configuration table, then poll `pipelines_build` for runs on `refs/heads/<target-branch>` queued at or after the merge time — match the merge commit when the run exposes it — and **capture each run's id**. A run that was already completed before the merge is not this deployment; ignore it. A branch with more than one pipeline (API + Client) must have **all** of them green.
|
|
185
|
+
3. **Wait for a terminal result** on each run: `succeeded`, `partiallySucceeded`, `failed`, or `canceled`.
|
|
186
|
+
|
|
187
|
+
**How to wait.** Never block on a foreground `sleep`. If the Azure CLI with the `azure-devops` extension is available, arm a background watch that notifies you on a terminal state:
|
|
106
188
|
|
|
107
|
-
|
|
108
|
-
|
|
189
|
+
```bash
|
|
190
|
+
# <run-id> is the run identified in step 2 — never "the latest run on the branch",
|
|
191
|
+
# which can be a run queued before the merge and already completed.
|
|
192
|
+
until az pipelines runs show --org <org-url> --project <project> \
|
|
193
|
+
--id <run-id> --query status -o tsv | grep -qx completed; do
|
|
194
|
+
sleep 60
|
|
195
|
+
done
|
|
196
|
+
az pipelines runs show --org <org-url> --project <project> \
|
|
197
|
+
--id <run-id> --query "[buildNumber,status,result]" -o tsv
|
|
109
198
|
```
|
|
110
199
|
|
|
111
|
-
|
|
200
|
+
Run it with `Monitor` (or Bash `run_in_background`) so the session stays usable. Watch the **run id**, and print `result` when it exits — a guard that only greps for success is silent through a failed or canceled deploy, which looks identical to one still running. One watch per pipeline when the branch has more than one.
|
|
201
|
+
|
|
202
|
+
Without the CLI, re-poll through the Azure DevOps MCP on roughly a 60-second cadence and report progress as you go. If the deploy outlasts the session's attention, say where it stands and that `check` will re-poll immediately — never claim a pipeline succeeded that you have not seen succeed.
|
|
203
|
+
|
|
204
|
+
## Step 8: Assign and Advance States (automatic, on a green pipeline)
|
|
112
205
|
|
|
113
|
-
|
|
206
|
+
This step runs **by itself** the moment the pipeline reports success — do not ask again; the approval in Step 6d covers it. All three preconditions must hold: the PR is `completed`, **every** CD run for {environment} is `succeeded`, and a Step 6d mapping was approved.
|
|
114
207
|
|
|
115
|
-
|
|
208
|
+
- `partiallySucceeded` → not green. Report which stage failed and ask `proceed` / `hold`.
|
|
209
|
+
- `failed` / `canceled` → change nothing. Report the run, the failing stage, and the log link (`pipelines_build_log`), and offer `/rollback` if the environment is broken.
|
|
210
|
+
- No Step 6d approval yet (user hasn't answered) → do the state changes only when they answer; don't guess an assignee.
|
|
116
211
|
|
|
117
|
-
|
|
118
|
-
|
|
212
|
+
For every cherry-picked work item of type **User Story**, **Bug**, or **Hot Fix**, one `wit_work_item_write` update per item setting both fields:
|
|
213
|
+
|
|
214
|
+
1. `System.State` → the target's state (`Ready for Testing` / `Testing` / `Staging` / `Deployed`).
|
|
119
215
|
- **Never move backward** — an item already past the target state keeps it; note it.
|
|
120
|
-
- **Never touch a Feature or Task.**
|
|
121
216
|
- Skipped (conflict) or dropped items are not on the branch — leave them alone.
|
|
122
217
|
- An invalid-state error means this project's template differs — confirm with `get_type`, report the item, continue. Don't silently swallow it.
|
|
123
|
-
|
|
218
|
+
2. `System.AssignedTo` → the person approved for that item's prefix group. Pass the identity's `uniqueName` / email, not the display name; if the update rejects it, resolve with `core_get_identity_ids` and retry once. A `Leave the current assignee alone` group keeps its assignee — never clear a field the user didn't ask you to clear.
|
|
219
|
+
|
|
220
|
+
**Never touch a Feature or Task** — neither state nor assignee.
|
|
221
|
+
|
|
222
|
+
Report both fields, so a wrong assignment is easy to put back:
|
|
223
|
+
|
|
224
|
+
```
|
|
225
|
+
Cherry-pick is deployed to {environment} (pipeline {build-number}, succeeded).
|
|
226
|
+
|
|
227
|
+
| ID | State: was → now | Assignee: was → now |
|
|
228
|
+
|----|------------------|---------------------|
|
|
229
|
+
| AB#1234 | Ready to Deploy → Deployed | Chris Waters → Jane Doe |
|
|
230
|
+
| AB#1240 | Ready to Deploy → Deployed | Chris Waters → Pat Ruiz |
|
|
231
|
+
| AB#1250 | Deployed (unchanged — already ahead) | unchanged |
|
|
232
|
+
|
|
233
|
+
Next: {the human step — "QA tests on Test and sets Ready for Staging" / "stakeholders verify on Staging and set Ready to Deploy" / "verify in production, then Close"}.
|
|
234
|
+
```
|
|
124
235
|
|
|
125
|
-
If the user
|
|
236
|
+
If the user said `stop watching`, or an update fails, leave the rest alone, say exactly which items were and were not updated, and note that `/status` will flag the lag later.
|
|
@@ -60,7 +60,7 @@ Work items assigned:
|
|
|
60
60
|
- AB#1235: Fix login redirect
|
|
61
61
|
- AB#1236: View history
|
|
62
62
|
|
|
63
|
-
Deploy it one environment at a time — each merge deploys, and the command advances the work items' states
|
|
63
|
+
Deploy it one environment at a time — each merge deploys, and once the pipeline is green the command advances the work items' states and assigns them to the verifiers you pick:
|
|
64
64
|
/deploy-release {N} test → items go to Testing
|
|
65
65
|
/deploy-release {N} staging → items go to Staging (after QA sets Ready for Staging)
|
|
66
66
|
/deploy-release {N} prod → items go to Deployed (after stakeholders set Ready to Deploy)
|