qa-skills 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +168 -0
- package/bin/cli.js +42 -0
- package/dist/agents/registry.d.ts +5 -0
- package/dist/agents/registry.d.ts.map +1 -0
- package/dist/agents/registry.js +101 -0
- package/dist/agents/registry.js.map +1 -0
- package/dist/agents/types.d.ts +9 -0
- package/dist/agents/types.d.ts.map +1 -0
- package/dist/agents/types.js +2 -0
- package/dist/agents/types.js.map +1 -0
- package/dist/dependencies.d.ts +21 -0
- package/dist/dependencies.d.ts.map +1 -0
- package/dist/dependencies.js +125 -0
- package/dist/dependencies.js.map +1 -0
- package/dist/installer.d.ts +25 -0
- package/dist/installer.d.ts.map +1 -0
- package/dist/installer.js +437 -0
- package/dist/installer.js.map +1 -0
- package/dist/scaffold.d.ts +27 -0
- package/dist/scaffold.d.ts.map +1 -0
- package/dist/scaffold.js +182 -0
- package/dist/scaffold.js.map +1 -0
- package/package.json +40 -0
- package/skills/qa-accessibility-test-writer/SKILL.md +127 -0
- package/skills/qa-accessibility-test-writer/references/axe-core-patterns.md +349 -0
- package/skills/qa-accessibility-test-writer/references/best-practices.md +184 -0
- package/skills/qa-accessibility-test-writer/references/wcag-tests.md +331 -0
- package/skills/qa-api-contract-curator/SKILL.md +104 -0
- package/skills/qa-api-contract-curator/references/breaking-changes.md +363 -0
- package/skills/qa-api-contract-curator/references/openapi-structure.md +404 -0
- package/skills/qa-browser-data-collector/SKILL.md +132 -0
- package/skills/qa-browser-data-collector/references/data-collection-checklist.md +91 -0
- package/skills/qa-browser-data-collector/references/playwright-mcp-patterns.md +113 -0
- package/skills/qa-bug-ticket-creator/SKILL.md +148 -0
- package/skills/qa-bug-ticket-creator/references/bug-report-format.md +149 -0
- package/skills/qa-bug-ticket-creator/references/severity-guide.md +81 -0
- package/skills/qa-bug-ticket-creator/templates/bug-ticket-template.md +39 -0
- package/skills/qa-changelog-analyzer/SKILL.md +134 -0
- package/skills/qa-changelog-analyzer/references/git-analysis-patterns.md +138 -0
- package/skills/qa-changelog-analyzer/references/impact-mapping.md +120 -0
- package/skills/qa-clickup-integration/SKILL.md +166 -0
- package/skills/qa-clickup-integration/references/api-patterns.md +102 -0
- package/skills/qa-clickup-integration/references/field-mapping.md +71 -0
- package/skills/qa-codeceptjs-writer/SKILL.md +136 -0
- package/skills/qa-codeceptjs-writer/references/best-practices.md +207 -0
- package/skills/qa-codeceptjs-writer/references/config.md +255 -0
- package/skills/qa-codeceptjs-writer/references/patterns.md +285 -0
- package/skills/qa-coverage-analyzer/SKILL.md +166 -0
- package/skills/qa-coverage-analyzer/references/best-practices.md +142 -0
- package/skills/qa-coverage-analyzer/references/coverage-dimensions.md +155 -0
- package/skills/qa-coverage-analyzer/references/tools.md +204 -0
- package/skills/qa-cypress-writer/SKILL.md +134 -0
- package/skills/qa-cypress-writer/references/assertions.md +121 -0
- package/skills/qa-cypress-writer/references/best-practices.md +82 -0
- package/skills/qa-cypress-writer/references/config.md +121 -0
- package/skills/qa-cypress-writer/references/patterns.md +170 -0
- package/skills/qa-data-factory/SKILL.md +126 -0
- package/skills/qa-data-factory/references/factory-patterns.md +164 -0
- package/skills/qa-data-factory/references/faker-guide.md +131 -0
- package/skills/qa-diagram-generator/SKILL.md +125 -0
- package/skills/qa-diagram-generator/references/c4-model.md +53 -0
- package/skills/qa-diagram-generator/references/charts.md +58 -0
- package/skills/qa-diagram-generator/references/class-diagram.md +85 -0
- package/skills/qa-diagram-generator/references/er-diagram.md +69 -0
- package/skills/qa-diagram-generator/references/flowchart.md +92 -0
- package/skills/qa-diagram-generator/references/from-screenshot.md +45 -0
- package/skills/qa-diagram-generator/references/gantt.md +49 -0
- package/skills/qa-diagram-generator/references/journey.md +50 -0
- package/skills/qa-diagram-generator/references/mindmap.md +75 -0
- package/skills/qa-diagram-generator/references/sequence.md +69 -0
- package/skills/qa-diagram-generator/references/state-diagram.md +56 -0
- package/skills/qa-discovery-interview/SKILL.md +182 -0
- package/skills/qa-discovery-interview/references/completeness-checklist.md +53 -0
- package/skills/qa-discovery-interview/references/conflict-patterns.md +101 -0
- package/skills/qa-discovery-interview/references/qa-categories.md +147 -0
- package/skills/qa-discovery-interview/templates/qa-brief-template.md +168 -0
- package/skills/qa-environment-checker/SKILL.md +142 -0
- package/skills/qa-environment-checker/references/dependency-matrix.md +101 -0
- package/skills/qa-environment-checker/references/health-checks.md +209 -0
- package/skills/qa-environment-checker/templates/env-readiness-template.md +64 -0
- package/skills/qa-flaky-detector/SKILL.md +153 -0
- package/skills/qa-flaky-detector/references/ci-analysis.md +140 -0
- package/skills/qa-flaky-detector/references/flaky-patterns.md +247 -0
- package/skills/qa-github-issues-enhanced/SKILL.md +175 -0
- package/skills/qa-github-issues-enhanced/references/issue-templates.md +425 -0
- package/skills/qa-github-issues-enhanced/references/label-taxonomy.md +130 -0
- package/skills/qa-github-issues-enhanced/references/workflow-patterns.md +188 -0
- package/skills/qa-httpx-writer/SKILL.md +138 -0
- package/skills/qa-httpx-writer/references/assertions.md +195 -0
- package/skills/qa-httpx-writer/references/best-practices.md +140 -0
- package/skills/qa-httpx-writer/references/config.md +212 -0
- package/skills/qa-httpx-writer/references/patterns.md +262 -0
- package/skills/qa-jest-writer/SKILL.md +131 -0
- package/skills/qa-jest-writer/references/assertions.md +125 -0
- package/skills/qa-jest-writer/references/best-practices.md +136 -0
- package/skills/qa-jest-writer/references/config.md +134 -0
- package/skills/qa-jest-writer/references/patterns.md +172 -0
- package/skills/qa-jira-integration/SKILL.md +135 -0
- package/skills/qa-jira-integration/references/api-patterns.md +143 -0
- package/skills/qa-jira-integration/references/field-mapping.md +79 -0
- package/skills/qa-jira-integration/references/xray-integration.md +85 -0
- package/skills/qa-jmeter-writer/SKILL.md +171 -0
- package/skills/qa-jmeter-writer/references/best-practices.md +157 -0
- package/skills/qa-jmeter-writer/references/config.md +204 -0
- package/skills/qa-jmeter-writer/references/patterns.md +242 -0
- package/skills/qa-junit5-writer/SKILL.md +157 -0
- package/skills/qa-junit5-writer/references/assertions.md +118 -0
- package/skills/qa-junit5-writer/references/config.md +97 -0
- package/skills/qa-junit5-writer/references/patterns.md +162 -0
- package/skills/qa-k6-writer/SKILL.md +155 -0
- package/skills/qa-k6-writer/references/best-practices.md +236 -0
- package/skills/qa-k6-writer/references/config.md +219 -0
- package/skills/qa-k6-writer/references/patterns.md +304 -0
- package/skills/qa-linear-integration/SKILL.md +137 -0
- package/skills/qa-linear-integration/references/api-patterns.md +249 -0
- package/skills/qa-linear-integration/references/field-mapping.md +121 -0
- package/skills/qa-locust-writer/SKILL.md +151 -0
- package/skills/qa-locust-writer/references/best-practices.md +126 -0
- package/skills/qa-locust-writer/references/config.md +170 -0
- package/skills/qa-locust-writer/references/patterns.md +235 -0
- package/skills/qa-manual-test-designer/SKILL.md +145 -0
- package/skills/qa-manual-test-designer/references/exploratory-charters.md +138 -0
- package/skills/qa-manual-test-designer/references/personas.md +146 -0
- package/skills/qa-manual-test-designer/templates/exploratory-charter-template.md +47 -0
- package/skills/qa-manual-test-designer/templates/test-case-template.md +31 -0
- package/skills/qa-mobile-test-writer/SKILL.md +144 -0
- package/skills/qa-mobile-test-writer/references/best-practices.md +214 -0
- package/skills/qa-mobile-test-writer/references/config.md +309 -0
- package/skills/qa-mobile-test-writer/references/patterns.md +304 -0
- package/skills/qa-nfr-analyst/SKILL.md +177 -0
- package/skills/qa-nfr-analyst/references/iso-25010-model.md +159 -0
- package/skills/qa-nfr-analyst/references/owasp-wstg-baseline.md +202 -0
- package/skills/qa-nfr-analyst/references/wcag-checklist.md +184 -0
- package/skills/qa-nfr-analyst/templates/owasp-checklist-template.md +89 -0
- package/skills/qa-nfr-analyst/templates/wcag-checklist-template.md +48 -0
- package/skills/qa-orchestrator/SKILL.md +132 -0
- package/skills/qa-orchestrator/references/handoff-chains.md +105 -0
- package/skills/qa-orchestrator/references/pipeline-modes.md +115 -0
- package/skills/qa-orchestrator/references/scheduler-rules.md +84 -0
- package/skills/qa-pact-writer/SKILL.md +133 -0
- package/skills/qa-pact-writer/references/best-practices.md +100 -0
- package/skills/qa-pact-writer/references/config.md +135 -0
- package/skills/qa-pact-writer/references/patterns.md +161 -0
- package/skills/qa-plan-creator/SKILL.md +139 -0
- package/skills/qa-plan-creator/references/introduction-plan.md +43 -0
- package/skills/qa-plan-creator/references/migration-plan.md +44 -0
- package/skills/qa-plan-creator/references/onboarding-plan.md +46 -0
- package/skills/qa-plan-creator/references/performance-plan.md +44 -0
- package/skills/qa-plan-creator/references/regression-plan.md +45 -0
- package/skills/qa-plan-creator/references/release-plan.md +45 -0
- package/skills/qa-plan-creator/references/sprint-plan.md +44 -0
- package/skills/qa-plan-creator/references/test-plan.md +59 -0
- package/skills/qa-plan-creator/references/uat-plan.md +43 -0
- package/skills/qa-plan-creator/templates/checklist-template.md +36 -0
- package/skills/qa-plan-creator/templates/regression-checklist-template.md +49 -0
- package/skills/qa-plan-creator/templates/release-checklist-template.md +46 -0
- package/skills/qa-plan-creator/templates/test-plan-template.md +74 -0
- package/skills/qa-playwright-py-writer/SKILL.md +156 -0
- package/skills/qa-playwright-py-writer/references/best-practices.md +194 -0
- package/skills/qa-playwright-py-writer/references/config.md +195 -0
- package/skills/qa-playwright-py-writer/references/patterns.md +212 -0
- package/skills/qa-playwright-ts-writer/SKILL.md +151 -0
- package/skills/qa-playwright-ts-writer/references/assertions.md +109 -0
- package/skills/qa-playwright-ts-writer/references/best-practices.md +191 -0
- package/skills/qa-playwright-ts-writer/references/config.md +144 -0
- package/skills/qa-playwright-ts-writer/references/patterns.md +171 -0
- package/skills/qa-pytest-writer/SKILL.md +145 -0
- package/skills/qa-pytest-writer/references/assertions.md +149 -0
- package/skills/qa-pytest-writer/references/best-practices.md +97 -0
- package/skills/qa-pytest-writer/references/config.md +176 -0
- package/skills/qa-pytest-writer/references/patterns.md +251 -0
- package/skills/qa-qase-integration/SKILL.md +149 -0
- package/skills/qa-qase-integration/references/api-reference.md +354 -0
- package/skills/qa-qase-integration/references/ci-integration.md +196 -0
- package/skills/qa-qase-integration/references/field-mapping.md +157 -0
- package/skills/qa-requirements-generator/SKILL.md +152 -0
- package/skills/qa-requirements-generator/references/iso-29148-structure.md +153 -0
- package/skills/qa-requirements-generator/references/requirement-patterns.md +278 -0
- package/skills/qa-rest-assured-writer/SKILL.md +137 -0
- package/skills/qa-rest-assured-writer/references/best-practices.md +50 -0
- package/skills/qa-rest-assured-writer/references/config.md +124 -0
- package/skills/qa-rest-assured-writer/references/patterns.md +192 -0
- package/skills/qa-risk-analyzer/SKILL.md +158 -0
- package/skills/qa-risk-analyzer/references/impact-analysis.md +133 -0
- package/skills/qa-risk-analyzer/references/risk-factors.md +123 -0
- package/skills/qa-robot-framework-writer/SKILL.md +147 -0
- package/skills/qa-robot-framework-writer/references/best-practices.md +249 -0
- package/skills/qa-robot-framework-writer/references/config.md +204 -0
- package/skills/qa-robot-framework-writer/references/libraries.md +273 -0
- package/skills/qa-robot-framework-writer/references/patterns.md +216 -0
- package/skills/qa-security-test-writer/SKILL.md +123 -0
- package/skills/qa-security-test-writer/references/best-practices.md +155 -0
- package/skills/qa-security-test-writer/references/owasp-top10.md +331 -0
- package/skills/qa-security-test-writer/references/zap-config.md +258 -0
- package/skills/qa-selenium-java-writer/SKILL.md +143 -0
- package/skills/qa-selenium-java-writer/references/best-practices.md +59 -0
- package/skills/qa-selenium-java-writer/references/config.md +143 -0
- package/skills/qa-selenium-java-writer/references/patterns.md +170 -0
- package/skills/qa-selenium-py-writer/SKILL.md +150 -0
- package/skills/qa-selenium-py-writer/references/best-practices.md +175 -0
- package/skills/qa-selenium-py-writer/references/config.md +224 -0
- package/skills/qa-selenium-py-writer/references/patterns.md +255 -0
- package/skills/qa-shortcut-integration/SKILL.md +143 -0
- package/skills/qa-shortcut-integration/references/api-patterns.md +126 -0
- package/skills/qa-shortcut-integration/references/field-mapping.md +66 -0
- package/skills/qa-spec-auditor/SKILL.md +162 -0
- package/skills/qa-spec-auditor/references/audit-checklist.md +144 -0
- package/skills/qa-spec-auditor/references/drift-patterns.md +207 -0
- package/skills/qa-spec-writer/SKILL.md +143 -0
- package/skills/qa-spec-writer/references/gherkin-guide.md +253 -0
- package/skills/qa-spec-writer/references/specification-patterns.md +274 -0
- package/skills/qa-spring-test-writer/SKILL.md +170 -0
- package/skills/qa-spring-test-writer/references/best-practices.md +57 -0
- package/skills/qa-spring-test-writer/references/config.md +179 -0
- package/skills/qa-spring-test-writer/references/patterns.md +235 -0
- package/skills/qa-supertest-writer/SKILL.md +150 -0
- package/skills/qa-supertest-writer/references/assertions.md +192 -0
- package/skills/qa-supertest-writer/references/best-practices.md +102 -0
- package/skills/qa-supertest-writer/references/config.md +166 -0
- package/skills/qa-supertest-writer/references/patterns.md +242 -0
- package/skills/qa-task-creator/SKILL.md +142 -0
- package/skills/qa-task-creator/references/linking-patterns.md +127 -0
- package/skills/qa-task-creator/references/task-types.md +169 -0
- package/skills/qa-task-creator/templates/task-template.md +24 -0
- package/skills/qa-test-doc-compiler/SKILL.md +114 -0
- package/skills/qa-test-doc-compiler/references/agile-tailoring.md +220 -0
- package/skills/qa-test-doc-compiler/references/iso-29119-3-documents.md +302 -0
- package/skills/qa-test-healer/SKILL.md +101 -0
- package/skills/qa-test-healer/references/diagnosis-patterns.md +142 -0
- package/skills/qa-test-healer/references/fix-strategies.md +177 -0
- package/skills/qa-test-reporter/SKILL.md +130 -0
- package/skills/qa-test-reporter/references/best-practices.md +162 -0
- package/skills/qa-test-reporter/references/iso-29119-reports.md +236 -0
- package/skills/qa-test-reporter/references/report-formats.md +287 -0
- package/skills/qa-test-reviewer/SKILL.md +142 -0
- package/skills/qa-test-reviewer/references/anti-patterns.md +268 -0
- package/skills/qa-test-reviewer/references/review-checklist.md +93 -0
- package/skills/qa-test-strategy/SKILL.md +133 -0
- package/skills/qa-test-strategy/references/entry-exit-criteria.md +176 -0
- package/skills/qa-test-strategy/references/risk-matrix.md +102 -0
- package/skills/qa-test-strategy/references/testing-types.md +143 -0
- package/skills/qa-testcase-from-docs/SKILL.md +161 -0
- package/skills/qa-testcase-from-docs/references/test-case-format.md +196 -0
- package/skills/qa-testcase-from-docs/references/test-design-techniques.md +126 -0
- package/skills/qa-testcase-from-docs/templates/test-case-template.md +31 -0
- package/skills/qa-testcase-from-ui/SKILL.md +109 -0
- package/skills/qa-testcase-from-ui/references/ui-element-patterns.md +126 -0
- package/skills/qa-testcase-from-ui/references/visual-analysis-guide.md +146 -0
- package/skills/qa-testcase-from-ui/templates/test-case-template.md +31 -0
- package/skills/qa-visual-regression-writer/SKILL.md +175 -0
- package/skills/qa-visual-regression-writer/references/best-practices.md +154 -0
- package/skills/qa-visual-regression-writer/references/config.md +220 -0
- package/skills/qa-visual-regression-writer/references/patterns.md +213 -0
- package/skills/qa-vitest-writer/SKILL.md +141 -0
- package/skills/qa-vitest-writer/references/assertions.md +105 -0
- package/skills/qa-vitest-writer/references/best-practices.md +62 -0
- package/skills/qa-vitest-writer/references/config.md +127 -0
- package/skills/qa-vitest-writer/references/patterns.md +141 -0
- package/skills/qa-webdriverio-writer/SKILL.md +145 -0
- package/skills/qa-webdriverio-writer/references/best-practices.md +176 -0
- package/skills/qa-webdriverio-writer/references/config.md +240 -0
- package/skills/qa-webdriverio-writer/references/patterns.md +269 -0
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
# Shortcut API Patterns
|
|
2
|
+
|
|
3
|
+
## Base URL and Headers
|
|
4
|
+
|
|
5
|
+
```
|
|
6
|
+
Base URL: https://api.app.shortcut.com/api/v3
|
|
7
|
+
Headers:
|
|
8
|
+
Shortcut-Token: <SHORTCUT_API_TOKEN>
|
|
9
|
+
Content-Type: application/json
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
## Stories
|
|
13
|
+
|
|
14
|
+
### Create Story
|
|
15
|
+
|
|
16
|
+
```
|
|
17
|
+
POST /api/v3/stories
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
**Request body:**
|
|
21
|
+
```json
|
|
22
|
+
{
|
|
23
|
+
"name": "Login fails when password contains special chars",
|
|
24
|
+
"description": "**Expected:** User can log in with special chars in password.\n**Actual:** 500 error.\n**Steps:** 1. Enter password with @#$ 2. Click Login",
|
|
25
|
+
"story_type": "bug",
|
|
26
|
+
"project_id": 123,
|
|
27
|
+
"labels": [{"name": "auth"}, {"name": "severity-critical"}],
|
|
28
|
+
"workflow_state_id": 50000001,
|
|
29
|
+
"external_links": [
|
|
30
|
+
{"url": "https://example.com/test-case/TC-001", "title": "Test Case TC-001"}
|
|
31
|
+
]
|
|
32
|
+
}
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
**Story types:** `bug`, `feature`, `chore`
|
|
36
|
+
|
|
37
|
+
### Get Story
|
|
38
|
+
|
|
39
|
+
```
|
|
40
|
+
GET /api/v3/stories/{story_id}
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
### Update Story
|
|
44
|
+
|
|
45
|
+
```
|
|
46
|
+
PUT /api/v3/stories/{story_id}
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
**Partial update example:**
|
|
50
|
+
```json
|
|
51
|
+
{
|
|
52
|
+
"workflow_state_id": 50000002,
|
|
53
|
+
"labels": [{"name": "verified"}],
|
|
54
|
+
"description": "Original description + verification notes"
|
|
55
|
+
}
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
### Search Stories
|
|
59
|
+
|
|
60
|
+
```
|
|
61
|
+
GET /api/v3/search/stories?query=login&project_id=123
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
Query supports text search, filters by project, epic, iteration, label.
|
|
65
|
+
|
|
66
|
+
## Epics
|
|
67
|
+
|
|
68
|
+
### List Epics
|
|
69
|
+
|
|
70
|
+
```
|
|
71
|
+
GET /api/v3/epics
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Returns epics with `id`, `name`, `state`. Use to resolve epic by name for story creation.
|
|
75
|
+
|
|
76
|
+
### Link Story to Epic
|
|
77
|
+
|
|
78
|
+
Include `epic_id` in create/update story request.
|
|
79
|
+
|
|
80
|
+
## Iterations
|
|
81
|
+
|
|
82
|
+
### List Iterations
|
|
83
|
+
|
|
84
|
+
```
|
|
85
|
+
GET /api/v3/iterations
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Returns iterations (sprints) with `id`, `name`, `status`. Use to assign stories to current sprint.
|
|
89
|
+
|
|
90
|
+
### Assign Story to Iteration
|
|
91
|
+
|
|
92
|
+
Include `iteration_id` in create/update story request.
|
|
93
|
+
|
|
94
|
+
## Workflows
|
|
95
|
+
|
|
96
|
+
### List Workflow States
|
|
97
|
+
|
|
98
|
+
```
|
|
99
|
+
GET /api/v3/workflows
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Returns workflow states per project. Map QA status (e.g., "Open", "In Progress", "Done") to `workflow_state_id`.
|
|
103
|
+
|
|
104
|
+
## Labels
|
|
105
|
+
|
|
106
|
+
Labels are created in Shortcut UI or via API. Use exact names when adding to stories:
|
|
107
|
+
|
|
108
|
+
```json
|
|
109
|
+
"labels": [{"name": "auth"}, {"name": "severity-critical"}, {"name": "test-type-e2e"}]
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
## External Links
|
|
113
|
+
|
|
114
|
+
Add links to test cases, requirements, evidence:
|
|
115
|
+
|
|
116
|
+
```json
|
|
117
|
+
"external_links": [
|
|
118
|
+
{"url": "https://example.com/test/TC-001", "title": "Test Case TC-001"},
|
|
119
|
+
{"url": "https://example.com/req/REQ-42", "title": "Requirement REQ-42"}
|
|
120
|
+
]
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
## Rate Limits
|
|
124
|
+
|
|
125
|
+
- 200 requests per minute
|
|
126
|
+
- Throttle bulk operations; use batch endpoints when available
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
# QA Field → Shortcut Field Mapping
|
|
2
|
+
|
|
3
|
+
## Bug Report (from qa-bug-ticket-creator)
|
|
4
|
+
|
|
5
|
+
| QA Field | Shortcut Field | Notes |
|
|
6
|
+
| -------- | -------------- | ----- |
|
|
7
|
+
| Title | `name` | Direct mapping |
|
|
8
|
+
| Description (expected/actual/steps) | `description` | Markdown; use headers for structure |
|
|
9
|
+
| Severity | `labels` | e.g., `severity-critical`, `severity-major` |
|
|
10
|
+
| Priority | `labels` | e.g., `priority-p1`, `priority-p2` |
|
|
11
|
+
| Component/Module | `labels` | e.g., `auth`, `frontend`, `api` |
|
|
12
|
+
| Environment | `description` | Append to description or comment |
|
|
13
|
+
| Evidence (screenshots, logs) | `external_links` | Link to artifact URLs |
|
|
14
|
+
| — | `story_type` | Always `bug` for bug reports |
|
|
15
|
+
|
|
16
|
+
## QA Task (from qa-task-creator)
|
|
17
|
+
|
|
18
|
+
| QA Field | Shortcut Field | Notes |
|
|
19
|
+
| -------- | -------------- | ----- |
|
|
20
|
+
| Title | `name` | Direct mapping |
|
|
21
|
+
| Description + acceptance criteria | `description` | Markdown |
|
|
22
|
+
| Task type (Dev/QA/Docs/Enhancement) | `story_type` | Dev→feature, QA→chore, Docs→chore, Enhancement→feature |
|
|
23
|
+
| Component | `labels` | Component label |
|
|
24
|
+
| Links to bug/requirement | `external_links` | Add URL + title |
|
|
25
|
+
| — | `story_type` | `feature` or `chore` per task type |
|
|
26
|
+
|
|
27
|
+
## Coverage Gap (from qa-coverage-analyzer)
|
|
28
|
+
|
|
29
|
+
| QA Field | Shortcut Field | Notes |
|
|
30
|
+
| -------- | -------------- | ----- |
|
|
31
|
+
| Gap description (e.g., "Add tests for AuthService") | `name` | Concise |
|
|
32
|
+
| Module/file, coverage % | `description` | Context |
|
|
33
|
+
| Component | `labels` | e.g., `test-coverage-gap`, `auth` |
|
|
34
|
+
| — | `story_type` | `chore` |
|
|
35
|
+
|
|
36
|
+
## Flaky Test (from qa-flaky-detector)
|
|
37
|
+
|
|
38
|
+
| QA Field | Shortcut Field | Notes |
|
|
39
|
+
| -------- | -------------- | ----- |
|
|
40
|
+
| Test name + "flaky" | `name` | e.g., "Login E2E test flaky" |
|
|
41
|
+
| Pattern, frequency | `description` | Flaky pattern details |
|
|
42
|
+
| Component | `labels` | e.g., `flaky-test`, `auth` |
|
|
43
|
+
| — | `story_type` | `bug` |
|
|
44
|
+
|
|
45
|
+
## Workflow State Mapping
|
|
46
|
+
|
|
47
|
+
| QA Status | Shortcut workflow_state | Typical name |
|
|
48
|
+
| --------- | ----------------------- | ------------ |
|
|
49
|
+
| New/Open | Initial state | "To Do", "Backlog" |
|
|
50
|
+
| In Progress | Mid-state | "In Progress", "Development" |
|
|
51
|
+
| Ready for Test | Pre-done | "Ready for QA", "Review" |
|
|
52
|
+
| Verified/Done | Done state | "Done", "Verified" |
|
|
53
|
+
|
|
54
|
+
Resolve `workflow_state_id` from `GET /api/v3/workflows`; names vary by workspace.
|
|
55
|
+
|
|
56
|
+
## Label Conventions
|
|
57
|
+
|
|
58
|
+
| Category | Examples |
|
|
59
|
+
| -------- | -------- |
|
|
60
|
+
| Severity | `severity-critical`, `severity-major`, `severity-minor` |
|
|
61
|
+
| Priority | `priority-p1`, `priority-p2`, `priority-p3` |
|
|
62
|
+
| Component | `auth`, `frontend`, `api`, `checkout` |
|
|
63
|
+
| Test type | `test-type-unit`, `test-type-e2e`, `test-type-api` |
|
|
64
|
+
| QA-specific | `test-coverage-gap`, `flaky-test`, `docs-update` |
|
|
65
|
+
|
|
66
|
+
Create these labels in Shortcut if they don't exist.
|
|
@@ -0,0 +1,162 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: qa-spec-auditor
|
|
3
|
+
description: Living documentation auditor that compares requirements vs UI/behavior vs API contract vs test results to detect spec drift, outdated docs, and undocumented features.
|
|
4
|
+
output_dir: reports/audit
|
|
5
|
+
dependencies:
|
|
6
|
+
recommended:
|
|
7
|
+
- qa-requirements-generator
|
|
8
|
+
- qa-spec-writer
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
# QA Spec Auditor
|
|
12
|
+
|
|
13
|
+
## Purpose
|
|
14
|
+
|
|
15
|
+
Compare "how it should be" vs "how it is" after each release or sprint. Detect spec drift, outdated documentation, undocumented features, and regression set optimization opportunities. Produce actionable drift reports with task suggestions for qa-task-creator.
|
|
16
|
+
|
|
17
|
+
## Trigger Phrases
|
|
18
|
+
|
|
19
|
+
- "Spec audit" / "Documentation audit for [release/sprint]"
|
|
20
|
+
- "Spec drift" / "Detect drift between docs and implementation"
|
|
21
|
+
- "Outdated documentation" / "Docs vs implementation comparison"
|
|
22
|
+
- "Undocumented features" / "Find features not in specs"
|
|
23
|
+
- "Requirements vs UI" / "Requirements vs API contract"
|
|
24
|
+
- "Living documentation audit" / "Spec compliance check"
|
|
25
|
+
- "Regression set optimization" / "Worn out tests" / "Test rotation"
|
|
26
|
+
|
|
27
|
+
## Comparison Dimensions
|
|
28
|
+
|
|
29
|
+
| Dimension | Source A | Source B | Detects |
|
|
30
|
+
|-----------|----------|----------|---------|
|
|
31
|
+
| **Requirements ↔ UI/behavior** | Requirements, specs | Live UI (Playwright MCP) | UI doesn't match spec; missing flows |
|
|
32
|
+
| **Requirements ↔ API contract** | Requirements, specs | OpenAPI, actual endpoints | API drift; undocumented endpoints |
|
|
33
|
+
| **Requirements ↔ Test results** | Requirements | Test cases, RTM, execution | Untested requirements; orphan tests |
|
|
34
|
+
| **Documentation ↔ Implementation** | Docs, README, wikis | Code, UI, API | Outdated docs; undocumented behavior |
|
|
35
|
+
|
|
36
|
+
See `references/audit-checklist.md` for checklists per dimension.
|
|
37
|
+
|
|
38
|
+
## Detection Categories
|
|
39
|
+
|
|
40
|
+
| Category | Description | Output Action |
|
|
41
|
+
|----------|-------------|---------------|
|
|
42
|
+
| **Outdated documentation** | Docs describe old behavior | Update docs |
|
|
43
|
+
| **Undocumented features** | Implementation has features not in specs | Add to specs or create tests |
|
|
44
|
+
| **Spec drift** | Behavior changed but docs not updated | Update docs or fix implementation |
|
|
45
|
+
| **Regression set optimization** | "Worn out" tests (always pass, low value); suggest rotation | Suggest test rotation; prioritize high-value tests |
|
|
46
|
+
|
|
47
|
+
See `references/drift-patterns.md` for common patterns and remediation.
|
|
48
|
+
|
|
49
|
+
## Output Deliverables
|
|
50
|
+
|
|
51
|
+
1. **Drift Report** — Per dimension: findings, severity, affected artifacts
|
|
52
|
+
2. **Action Items** — Update docs / fix implementation / create tests
|
|
53
|
+
3. **Task Suggestions** — Structured for qa-task-creator (doc updates, test creation, spec fixes)
|
|
54
|
+
4. **Regression Set Recommendations** — Tests to rotate, prioritize, or deprioritize
|
|
55
|
+
|
|
56
|
+
### Drift Report Template
|
|
57
|
+
|
|
58
|
+
```markdown
|
|
59
|
+
# Spec Audit — [Release/Sprint]
|
|
60
|
+
|
|
61
|
+
## Summary
|
|
62
|
+
| Dimension | Findings | Critical | Action Items |
|
|
63
|
+
|-----------|----------|----------|--------------|
|
|
64
|
+
| Req ↔ UI | N | X | [Count] |
|
|
65
|
+
| Req ↔ API | N | X | [Count] |
|
|
66
|
+
| Req ↔ Tests | N | X | [Count] |
|
|
67
|
+
| Docs ↔ Impl | N | X | [Count] |
|
|
68
|
+
|
|
69
|
+
## Findings by Dimension
|
|
70
|
+
### Requirements ↔ UI
|
|
71
|
+
- [Finding] — [Severity] — [Action]
|
|
72
|
+
### Requirements ↔ API
|
|
73
|
+
- [Finding] — [Severity] — [Action]
|
|
74
|
+
### Requirements ↔ Tests
|
|
75
|
+
- [Finding] — [Severity] — [Action]
|
|
76
|
+
### Documentation ↔ Implementation
|
|
77
|
+
- [Finding] — [Severity] — [Action]
|
|
78
|
+
|
|
79
|
+
## Task Suggestions (for qa-task-creator)
|
|
80
|
+
1. [Task type] — [Description] — [Owner]
|
|
81
|
+
2. ...
|
|
82
|
+
|
|
83
|
+
## Regression Set Recommendations
|
|
84
|
+
- Rotate: [Tests that are "worn out"]
|
|
85
|
+
- Prioritize: [High-value tests]
|
|
86
|
+
- Deprioritize: [Low-value, stable tests]
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
## Workflow
|
|
90
|
+
|
|
91
|
+
1. **Input:** Requirements, specs, OpenAPI/API docs, test cases/RTM, optional UI URL
|
|
92
|
+
2. **Requirements ↔ UI:** Use Playwright MCP to explore UI; compare flows, elements, validation to spec
|
|
93
|
+
3. **Requirements ↔ API:** Compare OpenAPI/spec to requirements; detect undocumented endpoints, schema drift
|
|
94
|
+
4. **Requirements ↔ Tests:** Validate RTM; find untested requirements, orphan tests
|
|
95
|
+
5. **Documentation ↔ Implementation:** Compare docs to code/UI/API; flag outdated sections
|
|
96
|
+
6. **Regression optimization:** Analyze test history; identify always-pass, low-value tests; suggest rotation
|
|
97
|
+
7. **Output:** Drift report, action items, task suggestions for qa-task-creator
|
|
98
|
+
|
|
99
|
+
## MCP Tools Used
|
|
100
|
+
|
|
101
|
+
- **Playwright MCP (cursor-ide-browser):** UI exploration, compare live behavior to spec
|
|
102
|
+
- **GitHub MCP:** Change tracking, PR/commit history for docs and code
|
|
103
|
+
|
|
104
|
+
## Integration with Other Skills
|
|
105
|
+
|
|
106
|
+
| Need | Skill | Usage |
|
|
107
|
+
|------|-------|-------|
|
|
108
|
+
| Task creation | qa-task-creator | Convert action items to structured tasks |
|
|
109
|
+
| Requirements source | qa-requirements-generator | Normalized requirements for comparison |
|
|
110
|
+
| API contract | qa-api-contract-curator | OpenAPI for API dimension |
|
|
111
|
+
| Test cases/RTM | qa-testcase-from-docs, qa-coverage-analyzer | Test mapping, coverage |
|
|
112
|
+
| Spec writing | qa-spec-writer | Update specs when drift found |
|
|
113
|
+
|
|
114
|
+
## Scope
|
|
115
|
+
|
|
116
|
+
**Can do (autonomous):**
|
|
117
|
+
- Compare requirements vs UI (via Playwright MCP)
|
|
118
|
+
- Compare requirements vs API contract (OpenAPI)
|
|
119
|
+
- Compare requirements vs test results (RTM, coverage)
|
|
120
|
+
- Compare documentation vs implementation
|
|
121
|
+
- Detect outdated docs, undocumented features, spec drift
|
|
122
|
+
- Produce drift report with action items
|
|
123
|
+
- Suggest tasks for qa-task-creator
|
|
124
|
+
- Recommend regression set optimization (test rotation)
|
|
125
|
+
- Use GitHub MCP for change tracking
|
|
126
|
+
|
|
127
|
+
**Cannot do (requires confirmation):**
|
|
128
|
+
- Modify documentation or code directly
|
|
129
|
+
- Change requirements without stakeholder approval
|
|
130
|
+
- Override organizational doc/spec policies
|
|
131
|
+
|
|
132
|
+
**Will not do (out of scope):**
|
|
133
|
+
- Execute tests (consume results only)
|
|
134
|
+
- Write or modify test automation
|
|
135
|
+
- Deploy or change production systems
|
|
136
|
+
|
|
137
|
+
## Quality Checklist
|
|
138
|
+
|
|
139
|
+
- [ ] All four comparison dimensions addressed
|
|
140
|
+
- [ ] Drift report includes severity per finding
|
|
141
|
+
- [ ] Action items are specific (update X, fix Y, create test Z)
|
|
142
|
+
- [ ] Task suggestions formatted for qa-task-creator
|
|
143
|
+
- [ ] Regression recommendations include rationale
|
|
144
|
+
- [ ] No hardcoded secrets; URLs/config referenced
|
|
145
|
+
|
|
146
|
+
## Troubleshooting
|
|
147
|
+
|
|
148
|
+
| Symptom | Likely Cause | Fix |
|
|
149
|
+
|---------|--------------|-----|
|
|
150
|
+
| UI comparison fails | Playwright MCP not available; auth required | Use API/docs comparison only; note "UI audit skipped" |
|
|
151
|
+
| API contract missing | No OpenAPI; endpoints undocumented | Use qa-api-contract-curator first; manual endpoint list |
|
|
152
|
+
| RTM incomplete | Requirements lack IDs; tests not traced | Use qa-requirements-generator; qa-testcase-from-docs for traceability |
|
|
153
|
+
| Too many findings | Broad scope; noisy comparison | Focus on critical modules; prioritize by business impact |
|
|
154
|
+
| Regression data missing | No test history | Use execution logs; flag "insufficient data" |
|
|
155
|
+
| GitHub MCP errors | Token, permissions | Verify GitHub token; check repo access |
|
|
156
|
+
|
|
157
|
+
## Reference Files
|
|
158
|
+
|
|
159
|
+
| Topic | Reference |
|
|
160
|
+
|-------|-----------|
|
|
161
|
+
| Audit checklist per dimension | `references/audit-checklist.md` |
|
|
162
|
+
| Drift patterns and remediation | `references/drift-patterns.md` |
|
|
@@ -0,0 +1,144 @@
|
|
|
1
|
+
# Spec Audit Checklist by Dimension
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
|
|
5
|
+
Use these checklists when auditing each comparison dimension. Tick items as verified; flag any mismatches as findings.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## 1. Requirements ↔ UI/Behavior
|
|
10
|
+
|
|
11
|
+
### Prerequisites
|
|
12
|
+
|
|
13
|
+
- Requirements document with IDs and acceptance criteria
|
|
14
|
+
- Live UI URL (or Playwright MCP target)
|
|
15
|
+
- Optional: Screenshots, Figma links
|
|
16
|
+
|
|
17
|
+
### Checklist
|
|
18
|
+
|
|
19
|
+
| # | Check | How to Verify | Finding Type |
|
|
20
|
+
|---|-------|---------------|--------------|
|
|
21
|
+
| 1 | All required flows exist in UI | Navigate each flow; verify steps | Missing flow |
|
|
22
|
+
| 2 | UI elements match spec (labels, placeholders) | Compare text, ARIA, structure | Spec drift |
|
|
23
|
+
| 3 | Validation rules match spec | Trigger invalid input; compare messages | Spec drift |
|
|
24
|
+
| 4 | Required fields enforced | Omit required fields; verify blocking | Spec drift |
|
|
25
|
+
| 5 | Error states documented and implemented | Trigger errors; compare to spec | Undocumented / Drift |
|
|
26
|
+
| 6 | Navigation/structure matches spec | Compare menu, breadcrumbs, routes | Spec drift |
|
|
27
|
+
| 7 | No extra flows not in spec | Explore UI; flag unknown flows | Undocumented feature |
|
|
28
|
+
| 8 | Accessibility (if in spec) | axe-core, keyboard nav per WCAG | Spec drift |
|
|
29
|
+
|
|
30
|
+
### Output
|
|
31
|
+
|
|
32
|
+
- List of requirements with UI mismatches
|
|
33
|
+
- Severity: Critical (blocking), High (wrong behavior), Medium (cosmetic), Low (minor)
|
|
34
|
+
|
|
35
|
+
---
|
|
36
|
+
|
|
37
|
+
## 2. Requirements ↔ API Contract
|
|
38
|
+
|
|
39
|
+
### Prerequisites
|
|
40
|
+
|
|
41
|
+
- Requirements with API expectations
|
|
42
|
+
- OpenAPI spec or equivalent
|
|
43
|
+
- Optional: Actual endpoint responses (from network traffic)
|
|
44
|
+
|
|
45
|
+
### Checklist
|
|
46
|
+
|
|
47
|
+
| # | Check | How to Verify | Finding Type |
|
|
48
|
+
|---|-------|---------------|--------------|
|
|
49
|
+
| 1 | All required endpoints exist | Compare req IDs to OpenAPI paths | Missing endpoint |
|
|
50
|
+
| 2 | Request/response schema matches spec | Compare JSON schema to requirements | Spec drift |
|
|
51
|
+
| 3 | Status codes match spec | Document expected 200/400/404/etc. | Spec drift |
|
|
52
|
+
| 4 | Authentication/authorization per spec | Verify auth headers, scopes | Spec drift |
|
|
53
|
+
| 5 | No undocumented endpoints | List all OpenAPI paths; flag untraced | Undocumented feature |
|
|
54
|
+
| 6 | Versioning matches (if applicable) | Check /v1/, /v2/ vs spec | Spec drift |
|
|
55
|
+
| 7 | Error response format per spec | Trigger errors; compare structure | Spec drift |
|
|
56
|
+
| 8 | Deprecated endpoints flagged | Check OpenAPI deprecation vs spec | Outdated doc |
|
|
57
|
+
|
|
58
|
+
### Output
|
|
59
|
+
|
|
60
|
+
- Endpoints with mismatches
|
|
61
|
+
- Undocumented endpoints
|
|
62
|
+
- Schema drift details
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
## 3. Requirements ↔ Test Results
|
|
67
|
+
|
|
68
|
+
### Prerequisites
|
|
69
|
+
|
|
70
|
+
- Requirements with IDs
|
|
71
|
+
- RTM (Requirement → Test Case)
|
|
72
|
+
- Test execution results (pass/fail, coverage)
|
|
73
|
+
|
|
74
|
+
### Checklist
|
|
75
|
+
|
|
76
|
+
| # | Check | How to Verify | Finding Type |
|
|
77
|
+
|---|-------|---------------|--------------|
|
|
78
|
+
| 1 | Every requirement has ≥1 test | RTM lookup | Untested requirement |
|
|
79
|
+
| 2 | Every test traces to ≥1 requirement | RTM reverse lookup | Orphan test |
|
|
80
|
+
| 3 | Failed tests map to requirements | Cross-ref results to RTM | Coverage gap |
|
|
81
|
+
| 4 | Test coverage matches risk | High-risk reqs have adequate coverage | Gap |
|
|
82
|
+
| 5 | Execution status current | Last run date; stale tests | Outdated |
|
|
83
|
+
| 6 | Automation status accurate | Manual vs automated flag | Outdated doc |
|
|
84
|
+
| 7 | Test design technique diversity | Per qa-coverage-analyzer | Technique gap |
|
|
85
|
+
|
|
86
|
+
### Output
|
|
87
|
+
|
|
88
|
+
- Untested requirements
|
|
89
|
+
- Orphan tests
|
|
90
|
+
- Stale or low-value tests
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## 4. Documentation ↔ Implementation
|
|
95
|
+
|
|
96
|
+
### Prerequisites
|
|
97
|
+
|
|
98
|
+
- README, wikis, internal docs
|
|
99
|
+
- Codebase, UI, API (as implementation source)
|
|
100
|
+
|
|
101
|
+
### Checklist
|
|
102
|
+
|
|
103
|
+
| # | Check | How to Verify | Finding Type |
|
|
104
|
+
|---|-------|---------------|--------------|
|
|
105
|
+
| 1 | Setup/install steps work | Follow README; verify success | Outdated doc |
|
|
106
|
+
| 2 | API examples run | Execute doc examples; compare output | Outdated doc |
|
|
107
|
+
| 3 | Config options match code | Compare doc to actual config/env | Outdated doc |
|
|
108
|
+
| 4 | Architecture diagram matches code | Compare to structure, imports | Outdated doc |
|
|
109
|
+
| 5 | Changelog/release notes current | Compare to git history | Outdated doc |
|
|
110
|
+
| 6 | Deprecation notices accurate | Check code for deprecated flags | Outdated doc |
|
|
111
|
+
| 7 | No significant features missing from docs | Code review vs doc scope | Undocumented feature |
|
|
112
|
+
| 8 | Screenshots/UI examples current | Compare to live UI | Outdated doc |
|
|
113
|
+
|
|
114
|
+
### Output
|
|
115
|
+
|
|
116
|
+
- Doc sections to update
|
|
117
|
+
- Undocumented implementation details
|
|
118
|
+
- Broken examples or steps
|
|
119
|
+
|
|
120
|
+
---
|
|
121
|
+
|
|
122
|
+
## 5. Regression Set Optimization (Bonus)
|
|
123
|
+
|
|
124
|
+
### Prerequisites
|
|
125
|
+
|
|
126
|
+
- Test execution history (pass/fail over time)
|
|
127
|
+
- Test-to-requirement mapping
|
|
128
|
+
- Risk data (from qa-risk-analyzer)
|
|
129
|
+
|
|
130
|
+
### Checklist
|
|
131
|
+
|
|
132
|
+
| # | Check | How to Verify | Recommendation |
|
|
133
|
+
|---|-------|---------------|----------------|
|
|
134
|
+
| 1 | "Worn out" tests | Always pass, never fail, many runs | Rotate out or reduce frequency |
|
|
135
|
+
| 2 | High-value tests | Cover critical reqs, catch regressions | Prioritize; run first |
|
|
136
|
+
| 3 | Low-value, stable tests | Low risk, always pass | Deprioritize |
|
|
137
|
+
| 4 | Flaky tests | Intermittent failures | Fix or quarantine |
|
|
138
|
+
| 5 | Missing coverage for changed code | Per qa-risk-analyzer impact | Add tests |
|
|
139
|
+
|
|
140
|
+
### Output
|
|
141
|
+
|
|
142
|
+
- Tests to rotate
|
|
143
|
+
- Tests to prioritize
|
|
144
|
+
- Tests to deprioritize or remove
|
|
@@ -0,0 +1,207 @@
|
|
|
1
|
+
# Common Spec Drift Patterns and Remediation
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
|
|
5
|
+
Spec drift occurs when documentation, requirements, or specs diverge from actual implementation. This reference catalogs common patterns and recommended remediation.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Pattern 1: Behavior Changed, Docs Not Updated
|
|
10
|
+
|
|
11
|
+
### Description
|
|
12
|
+
|
|
13
|
+
Implementation was modified (bug fix, enhancement, refactor) but requirements, specs, or user docs were not updated.
|
|
14
|
+
|
|
15
|
+
### Detection
|
|
16
|
+
|
|
17
|
+
- Git history: code changed, doc files unchanged
|
|
18
|
+
- Compare current behavior to doc description
|
|
19
|
+
- User reports: "The docs say X but it does Y"
|
|
20
|
+
|
|
21
|
+
### Remediation
|
|
22
|
+
|
|
23
|
+
| Owner | Action |
|
|
24
|
+
|-------|--------|
|
|
25
|
+
| **Tech writer / QA** | Update requirements, spec, or user docs to match implementation |
|
|
26
|
+
| **Product** | Confirm if behavior change was intentional; if not, file bug |
|
|
27
|
+
| **Dev** | Add "update docs" to DoD for behavior-changing PRs |
|
|
28
|
+
|
|
29
|
+
### Prevention
|
|
30
|
+
|
|
31
|
+
- Include doc updates in PR checklist
|
|
32
|
+
- Link requirements to implementation in traceability
|
|
33
|
+
- Periodic spec audits (this skill)
|
|
34
|
+
|
|
35
|
+
---
|
|
36
|
+
|
|
37
|
+
## Pattern 2: Undocumented Feature
|
|
38
|
+
|
|
39
|
+
### Description
|
|
40
|
+
|
|
41
|
+
A feature exists in the product (UI, API, behavior) but has no corresponding requirement, spec, or documentation.
|
|
42
|
+
|
|
43
|
+
### Detection
|
|
44
|
+
|
|
45
|
+
- UI exploration: flows or elements not in requirements
|
|
46
|
+
- API: endpoints not in OpenAPI or requirements
|
|
47
|
+
- Code: features without doc references
|
|
48
|
+
|
|
49
|
+
### Remediation
|
|
50
|
+
|
|
51
|
+
| Owner | Action |
|
|
52
|
+
|-------|--------|
|
|
53
|
+
| **Product / BA** | Add requirement; assess if intentional or technical debt |
|
|
54
|
+
| **QA** | Create test cases; add to RTM |
|
|
55
|
+
| **Tech writer** | Document in user docs, API docs |
|
|
56
|
+
| **qa-task-creator** | Create tasks: "Add requirement for [feature]", "Document [endpoint]" |
|
|
57
|
+
|
|
58
|
+
### Prevention
|
|
59
|
+
|
|
60
|
+
- Feature flags with doc requirement
|
|
61
|
+
- API-first design: OpenAPI before implementation
|
|
62
|
+
- Exploratory testing to catch gaps
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
## Pattern 3: Outdated Documentation
|
|
67
|
+
|
|
68
|
+
### Description
|
|
69
|
+
|
|
70
|
+
Documentation describes old behavior, deprecated options, or incorrect steps. Implementation has moved on.
|
|
71
|
+
|
|
72
|
+
### Detection
|
|
73
|
+
|
|
74
|
+
- Setup steps fail
|
|
75
|
+
- API examples return different structure
|
|
76
|
+
- Screenshots don't match UI
|
|
77
|
+
- Changelog missing recent releases
|
|
78
|
+
|
|
79
|
+
### Remediation
|
|
80
|
+
|
|
81
|
+
| Owner | Action |
|
|
82
|
+
|-------|--------|
|
|
83
|
+
| **Tech writer** | Update affected sections; verify against current implementation |
|
|
84
|
+
| **Dev** | Update code comments, README, inline docs |
|
|
85
|
+
| **QA** | Re-verify doc examples; report inaccuracies |
|
|
86
|
+
|
|
87
|
+
### Prevention
|
|
88
|
+
|
|
89
|
+
- Doc review in release process
|
|
90
|
+
- Automated checks: run doc examples in CI
|
|
91
|
+
- Single source of truth (e.g., OpenAPI for API docs)
|
|
92
|
+
|
|
93
|
+
---
|
|
94
|
+
|
|
95
|
+
## Pattern 4: Requirements ↔ Test Gap
|
|
96
|
+
|
|
97
|
+
### Description
|
|
98
|
+
|
|
99
|
+
- **Untested requirement:** Requirement has no test
|
|
100
|
+
- **Orphan test:** Test has no requirement traceability
|
|
101
|
+
- **Stale test:** Test passes but doesn't validate current behavior
|
|
102
|
+
|
|
103
|
+
### Detection
|
|
104
|
+
|
|
105
|
+
- RTM gaps (qa-coverage-analyzer)
|
|
106
|
+
- Test name/description doesn't match any requirement
|
|
107
|
+
- Test steps outdated after behavior change
|
|
108
|
+
|
|
109
|
+
### Remediation
|
|
110
|
+
|
|
111
|
+
| Gap Type | Action |
|
|
112
|
+
|----------|--------|
|
|
113
|
+
| **Untested requirement** | Create test (qa-testcase-from-docs); add to RTM |
|
|
114
|
+
| **Orphan test** | Map to requirement or deprecate if redundant |
|
|
115
|
+
| **Stale test** | Update test steps; re-validate against current behavior |
|
|
116
|
+
|
|
117
|
+
### Prevention
|
|
118
|
+
|
|
119
|
+
- RTM maintained in same workflow as requirements
|
|
120
|
+
- Test review when requirements change
|
|
121
|
+
- Traceability in test case format
|
|
122
|
+
|
|
123
|
+
---
|
|
124
|
+
|
|
125
|
+
## Pattern 5: API Contract Drift
|
|
126
|
+
|
|
127
|
+
### Description
|
|
128
|
+
|
|
129
|
+
- OpenAPI/spec doesn't match actual API responses
|
|
130
|
+
- New parameters, fields, or endpoints not in spec
|
|
131
|
+
- Deprecated fields still in spec but removed from API
|
|
132
|
+
|
|
133
|
+
### Detection
|
|
134
|
+
|
|
135
|
+
- Contract testing (Pact, OpenAPI validator)
|
|
136
|
+
- Manual comparison: actual response vs schema
|
|
137
|
+
- Consumer integration failures
|
|
138
|
+
|
|
139
|
+
### Remediation
|
|
140
|
+
|
|
141
|
+
| Owner | Action |
|
|
142
|
+
|-------|--------|
|
|
143
|
+
| **API owner** | Update OpenAPI to match implementation |
|
|
144
|
+
| **qa-api-contract-curator** | Formalize contract from traffic or code |
|
|
145
|
+
| **Consumer teams** | Update clients if breaking change |
|
|
146
|
+
|
|
147
|
+
### Prevention
|
|
148
|
+
|
|
149
|
+
- OpenAPI generated from code (or vice versa)
|
|
150
|
+
- Contract tests in CI
|
|
151
|
+
- Versioning and deprecation policy
|
|
152
|
+
|
|
153
|
+
---
|
|
154
|
+
|
|
155
|
+
## Pattern 6: "Worn Out" Regression Tests
|
|
156
|
+
|
|
157
|
+
### Description
|
|
158
|
+
|
|
159
|
+
Tests that always pass, run frequently, but rarely catch bugs. They consume time without adding value. Often cover stable, low-risk areas.
|
|
160
|
+
|
|
161
|
+
### Detection
|
|
162
|
+
|
|
163
|
+
- Execution history: 100% pass over many runs
|
|
164
|
+
- Low risk score (qa-risk-analyzer)
|
|
165
|
+
- Redundant with other tests
|
|
166
|
+
|
|
167
|
+
### Remediation
|
|
168
|
+
|
|
169
|
+
| Action | When |
|
|
170
|
+
|--------|------|
|
|
171
|
+
| **Rotate out** | Run less frequently (e.g., weekly vs every PR) |
|
|
172
|
+
| **Consolidate** | Merge with similar tests |
|
|
173
|
+
| **Deprioritize** | Run after high-risk tests |
|
|
174
|
+
| **Remove** | If truly redundant and low value |
|
|
175
|
+
|
|
176
|
+
### Prevention
|
|
177
|
+
|
|
178
|
+
- Risk-based test prioritization
|
|
179
|
+
- Periodic test value assessment
|
|
180
|
+
- Replace static tests with parameterized or generated tests for coverage
|
|
181
|
+
|
|
182
|
+
---
|
|
183
|
+
|
|
184
|
+
## Severity Guidelines
|
|
185
|
+
|
|
186
|
+
| Severity | Criteria | Example |
|
|
187
|
+
|----------|----------|---------|
|
|
188
|
+
| **Critical** | Blocking; wrong behavior; security | Auth flow doesn't match spec; API returns wrong data |
|
|
189
|
+
| **High** | Major discrepancy; user impact | Required validation missing; doc steps fail |
|
|
190
|
+
| **Medium** | Noticeable drift; moderate impact | Extra field in API; outdated screenshot |
|
|
191
|
+
| **Low** | Minor; cosmetic | Typo in doc; deprecated option still listed |
|
|
192
|
+
|
|
193
|
+
---
|
|
194
|
+
|
|
195
|
+
## Task Suggestion Format for qa-task-creator
|
|
196
|
+
|
|
197
|
+
When producing task suggestions, use this structure:
|
|
198
|
+
|
|
199
|
+
```markdown
|
|
200
|
+
## Task: [Title]
|
|
201
|
+
- **Type:** doc_update | spec_fix | test_creation | implementation_fix
|
|
202
|
+
- **Dimension:** req_ui | req_api | req_tests | docs_impl
|
|
203
|
+
- **Finding:** [Brief description]
|
|
204
|
+
- **Action:** [Specific action]
|
|
205
|
+
- **Artifacts:** [Files, IDs, URLs]
|
|
206
|
+
- **Priority:** critical | high | medium | low
|
|
207
|
+
```
|