gaia-framework 1.65.1 → 1.83.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/commands/gaia-create-stakeholder.md +20 -0
- package/.claude/commands/gaia-test-gap-analysis.md +17 -0
- package/CLAUDE.md +102 -1
- package/README.md +2 -2
- package/_gaia/_config/global.yaml +5 -1
- package/_gaia/_config/lifecycle-sequence.yaml +20 -0
- package/_gaia/_config/skill-manifest.csv +2 -0
- package/_gaia/_config/workflow-manifest.csv +3 -1
- package/_gaia/core/engine/workflow.xml +11 -1
- package/_gaia/core/protocols/review-gate-check.xml +29 -1
- package/_gaia/core/workflows/party-mode/steps/step-01-agent-loading.md +60 -9
- package/_gaia/creative/workflows/problem-solving/checklist.md +64 -14
- package/_gaia/creative/workflows/problem-solving/instructions.xml +367 -22
- package/_gaia/creative/workflows/problem-solving/workflow.yaml +31 -1
- package/_gaia/dev/agents/_base-dev.md +7 -1
- package/_gaia/dev/skills/_skill-index.yaml +9 -0
- package/_gaia/dev/skills/figma-integration.md +296 -0
- package/_gaia/lifecycle/knowledge/brownfield/config-contradiction-scan.md +137 -0
- package/_gaia/lifecycle/knowledge/brownfield/dead-code-scan.md +179 -0
- package/_gaia/lifecycle/knowledge/brownfield/test-execution-scan.md +209 -0
- package/_gaia/lifecycle/skills/document-rulesets.md +91 -6
- package/_gaia/lifecycle/templates/brownfield-scan-doc-code-prompt.md +219 -0
- package/_gaia/lifecycle/templates/brownfield-scan-hardcoded-prompt.md +169 -0
- package/_gaia/lifecycle/templates/brownfield-scan-integration-seam-prompt.md +127 -0
- package/_gaia/lifecycle/templates/brownfield-scan-runtime-behavior-prompt.md +141 -0
- package/_gaia/lifecycle/templates/brownfield-scan-security-prompt.md +440 -0
- package/_gaia/lifecycle/templates/gap-entry-schema.md +282 -0
- package/_gaia/lifecycle/templates/infra-prd-template.md +356 -0
- package/_gaia/lifecycle/templates/platform-prd-template.md +431 -0
- package/_gaia/lifecycle/templates/prd-template.md +70 -0
- package/_gaia/lifecycle/templates/story-template.md +22 -1
- package/_gaia/lifecycle/workflows/2-planning/create-ux-design/instructions.xml +52 -3
- package/_gaia/lifecycle/workflows/4-implementation/add-feature/checklist.md +1 -1
- package/_gaia/lifecycle/workflows/4-implementation/add-feature/instructions.xml +2 -3
- package/_gaia/lifecycle/workflows/4-implementation/add-stories/checklist.md +5 -0
- package/_gaia/lifecycle/workflows/4-implementation/add-stories/instructions.xml +73 -1
- package/_gaia/lifecycle/workflows/4-implementation/create-stakeholder/checklist.md +25 -0
- package/_gaia/lifecycle/workflows/4-implementation/create-stakeholder/instructions.xml +79 -0
- package/_gaia/lifecycle/workflows/4-implementation/create-stakeholder/workflow.yaml +22 -0
- package/_gaia/lifecycle/workflows/4-implementation/create-story/instructions.xml +11 -1
- package/_gaia/lifecycle/workflows/4-implementation/retrospective/instructions.xml +21 -1
- package/_gaia/lifecycle/workflows/4-implementation/retrospective/workflow.yaml +1 -1
- package/_gaia/lifecycle/workflows/4-implementation/validate-story/instructions.xml +11 -0
- package/_gaia/lifecycle/workflows/anytime/brownfield-onboarding/checklist.md +12 -0
- package/_gaia/lifecycle/workflows/anytime/brownfield-onboarding/instructions.xml +248 -4
- package/_gaia/lifecycle/workflows/anytime/brownfield-onboarding/workflow.yaml +1 -0
- package/_gaia/testing/workflows/test-gap-analysis/checklist.md +8 -0
- package/_gaia/testing/workflows/test-gap-analysis/instructions.xml +53 -0
- package/_gaia/testing/workflows/test-gap-analysis/workflow.yaml +38 -0
- package/bin/gaia-framework.js +44 -8
- package/bin/helpers/derive-bump-label.js +41 -0
- package/bin/helpers/validate-bump-labels.js +38 -0
- package/gaia-install.sh +96 -21
- package/package.json +1 -1
- package/_gaia/_memory/tier2-results/.gitkeep +0 -0
- package/_gaia/_memory/tier2-results/checkpoint-resume-2026-03-24.yaml +0 -6
- package/_gaia/_memory/tier2-results/engine-scenarios-2026-03-22.yaml +0 -14
|
@@ -0,0 +1,209 @@
|
|
|
1
|
+
# Test Execution Scan — Brownfield Subagent Prompt
|
|
2
|
+
|
|
3
|
+
> **Version:** 1.0.0
|
|
4
|
+
> **Story:** E11-S9
|
|
5
|
+
> **Traces to:** FR-110, US-37, ADR-021
|
|
6
|
+
> **Category:** runtime-behavior (test failures map to runtime-behavior per gap-entry-schema.md)
|
|
7
|
+
> **Output format:** Standardized gap entry schema (`_gaia/lifecycle/templates/gap-entry-schema.md`)
|
|
8
|
+
|
|
9
|
+
## Objective
|
|
10
|
+
|
|
11
|
+
Run the existing test suite at `{project-path}` during brownfield discovery. Capture test failures as gap entries conforming to the gap schema. This scan is **non-blocking** — failures do not halt the brownfield workflow.
|
|
12
|
+
|
|
13
|
+
## Test Runner Auto-Detection
|
|
14
|
+
|
|
15
|
+
Detect the test runner by checking for the following files at `{project-path}`. Use the **priority order** below — select the first matching runner. For monorepo/polyglot projects, detect **all** matching runners and execute them sequentially.
|
|
16
|
+
|
|
17
|
+
### Detection Priority Order
|
|
18
|
+
|
|
19
|
+
| Priority | File Check | Condition | Runner Command |
|
|
20
|
+
|----------|-----------|-----------|----------------|
|
|
21
|
+
| 1 | `package.json` | Has `scripts.test` defined AND value is not `"echo \"Error: no test specified\""` | `npm test` |
|
|
22
|
+
| 2 | `pytest.ini` / `pyproject.toml` / `setup.cfg` | `pytest.ini` exists, OR `pyproject.toml` contains `[tool.pytest]`, OR `setup.cfg` contains `[tool:pytest]` | `pytest` |
|
|
23
|
+
| 3 | `pom.xml` | File exists | `mvn test` |
|
|
24
|
+
| 4 | `build.gradle` / `build.gradle.kts` | Either file exists | `gradle test` |
|
|
25
|
+
| 5 | `go.mod` | File exists | `go test ./...` |
|
|
26
|
+
| 6 | `pubspec.yaml` | File exists | `flutter test` |
|
|
27
|
+
|
|
28
|
+
### No Test Suite Detected (AC6)
|
|
29
|
+
|
|
30
|
+
If no test runner is detected at `{project-path}`, produce a single info-level gap entry:
|
|
31
|
+
|
|
32
|
+
```yaml
|
|
33
|
+
id: "GAP-TEST-INFO-001"
|
|
34
|
+
category: "runtime-behavior"
|
|
35
|
+
severity: "info"
|
|
36
|
+
title: "No test suite detected"
|
|
37
|
+
description: "No recognized test runner configuration found at {project-path}. The project has no automated tests or uses an unsupported test framework."
|
|
38
|
+
evidence:
|
|
39
|
+
file: "{project-path}"
|
|
40
|
+
line: 0
|
|
41
|
+
recommendation: "Add a test framework (Jest, pytest, JUnit, etc.) and write initial unit tests for critical paths."
|
|
42
|
+
verified_by: "machine-detected"
|
|
43
|
+
confidence: "high"
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
Proceed without error after logging this gap.
|
|
47
|
+
|
|
48
|
+
## Test Execution with Timeout (AC3)
|
|
49
|
+
|
|
50
|
+
Execute each detected runner with a configurable timeout (default **5 minutes** / 300 seconds).
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
timeout 300 npm test 2>&1
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
### Timeout Behavior
|
|
57
|
+
|
|
58
|
+
- If the timeout is exceeded, terminate the process gracefully
|
|
59
|
+
- Capture partial results from stdout/stderr up to the timeout point
|
|
60
|
+
- Log a warning-level gap entry noting the timeout:
|
|
61
|
+
|
|
62
|
+
```yaml
|
|
63
|
+
id: "GAP-TEST-{seq}"
|
|
64
|
+
category: "runtime-behavior"
|
|
65
|
+
severity: "medium"
|
|
66
|
+
title: "Test suite timed out after 5 minutes"
|
|
67
|
+
description: "Test execution exceeded the 300s timeout. Partial results captured: {N} tests ran before timeout."
|
|
68
|
+
evidence:
|
|
69
|
+
file: "{test-config-file}"
|
|
70
|
+
line: 0
|
|
71
|
+
recommendation: "Investigate slow tests. Consider splitting the test suite or increasing the timeout for CI."
|
|
72
|
+
verified_by: "machine-detected"
|
|
73
|
+
confidence: "medium"
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### Sequential Execution for Multiple Runners (AC9)
|
|
77
|
+
|
|
78
|
+
For monorepo or polyglot projects with multiple test runners detected:
|
|
79
|
+
1. Execute each detected runner sequentially (not in parallel)
|
|
80
|
+
2. Aggregate results across all runners
|
|
81
|
+
3. Include the runner name in the `description` field of each gap entry (e.g., "npm test: ...")
|
|
82
|
+
4. Use a shared sequence counter across all runners for gap entry IDs
|
|
83
|
+
|
|
84
|
+
## Output Parsing (AC4)
|
|
85
|
+
|
|
86
|
+
After each test run completes (or times out), parse the output to extract metrics:
|
|
87
|
+
- **Total** test count
|
|
88
|
+
- **Passing** count
|
|
89
|
+
- **Failing** count
|
|
90
|
+
- **Skipped** count
|
|
91
|
+
- **Error messages** for each failing test
|
|
92
|
+
|
|
93
|
+
### Parsing Patterns by Runner
|
|
94
|
+
|
|
95
|
+
**Jest/Mocha/Vitest:**
|
|
96
|
+
- Summary line: `Tests: N passed, N failed, N total` (Jest) or `Tests N passed | N failed` (Vitest)
|
|
97
|
+
- Individual: `FAIL src/path/to/test.js` / `PASS src/path/to/test.js`
|
|
98
|
+
- Exit code 1 = failures present
|
|
99
|
+
|
|
100
|
+
**pytest:**
|
|
101
|
+
- Summary line: `N passed, N failed, N error`
|
|
102
|
+
- Individual: `FAILED test_file.py::test_name`
|
|
103
|
+
|
|
104
|
+
**Maven Surefire:**
|
|
105
|
+
- Summary in `target/surefire-reports/` XML files
|
|
106
|
+
- Console: `Tests run: N, Failures: N, Errors: N, Skipped: N`
|
|
107
|
+
|
|
108
|
+
**Go test:**
|
|
109
|
+
- Per-test: `--- FAIL: TestName` / `--- PASS: TestName`
|
|
110
|
+
- Summary: `FAIL` or `ok` per package
|
|
111
|
+
|
|
112
|
+
**Flutter test:**
|
|
113
|
+
- Summary: `All tests passed!` or `N tests failed`
|
|
114
|
+
- Per-test: `FAILED: test description`
|
|
115
|
+
|
|
116
|
+
## Infrastructure Error Detection (AC8)
|
|
117
|
+
|
|
118
|
+
Before converting failures to gap entries, check if the error is an infrastructure dependency failure rather than an actual test failure.
|
|
119
|
+
|
|
120
|
+
**Infrastructure error heuristics:**
|
|
121
|
+
- Pattern match stderr/stdout for: `ECONNREFUSED`, `connection refused`, `missing environment variable`, `ENOENT`, `docker`, `database connection`, `redis`, `ETIMEDOUT`, `EHOSTUNREACH`
|
|
122
|
+
- Exit codes indicating non-test errors (e.g., process crash, missing binary)
|
|
123
|
+
|
|
124
|
+
If an infrastructure error is detected:
|
|
125
|
+
- Do NOT convert to test failure gap entries
|
|
126
|
+
- Instead, log a **warning-level** gap entry:
|
|
127
|
+
|
|
128
|
+
```yaml
|
|
129
|
+
id: "GAP-TEST-{seq}"
|
|
130
|
+
category: "runtime-behavior"
|
|
131
|
+
severity: "medium"
|
|
132
|
+
title: "Test infrastructure dependency unavailable"
|
|
133
|
+
description: "Test execution failed due to infrastructure dependency: {detected_pattern}. This is not a test logic failure."
|
|
134
|
+
evidence:
|
|
135
|
+
file: "{test-config-file}"
|
|
136
|
+
line: 0
|
|
137
|
+
recommendation: "Ensure required infrastructure (databases, caches, external services) is available before running tests. Consider using test doubles for external dependencies."
|
|
138
|
+
verified_by: "machine-detected"
|
|
139
|
+
confidence: "medium"
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
## Gap Entry Conversion (AC5)
|
|
143
|
+
|
|
144
|
+
For each failing test, produce a gap entry conforming to the standardized gap schema:
|
|
145
|
+
|
|
146
|
+
### ID Format
|
|
147
|
+
|
|
148
|
+
- `GAP-TEST-{seq}` where `{seq}` is a zero-padded 3-digit sequence (001, 002, ...)
|
|
149
|
+
- Example: `GAP-TEST-001`, `GAP-TEST-002`
|
|
150
|
+
|
|
151
|
+
### Severity Mapping by Test Type
|
|
152
|
+
|
|
153
|
+
Infer test type from file path patterns:
|
|
154
|
+
|
|
155
|
+
| File Path Pattern | Test Type | Severity |
|
|
156
|
+
|-------------------|-----------|----------|
|
|
157
|
+
| `test/unit/`, `tests/unit/`, `__tests__/`, `*.unit.test.*` | unit | medium |
|
|
158
|
+
| `test/integration/`, `tests/integration/`, `*.integration.test.*` | integration | high |
|
|
159
|
+
| `test/e2e/`, `tests/e2e/`, `test/end-to-end/`, `*.e2e.test.*` | e2e | critical |
|
|
160
|
+
| Cannot be determined | default | medium |
|
|
161
|
+
|
|
162
|
+
### Gap Entry Template
|
|
163
|
+
|
|
164
|
+
```yaml
|
|
165
|
+
id: "GAP-TEST-{seq}"
|
|
166
|
+
category: "runtime-behavior"
|
|
167
|
+
severity: "{severity_from_test_type}"
|
|
168
|
+
title: "{test_name} — failing"
|
|
169
|
+
description: "{runner_name}: {error_message}"
|
|
170
|
+
evidence:
|
|
171
|
+
file: "{test_file_path}"
|
|
172
|
+
line: "{line_number_if_available}"
|
|
173
|
+
recommendation: "Fix the failing test or update the test to match current behavior."
|
|
174
|
+
verified_by: "machine-detected"
|
|
175
|
+
confidence: "high"
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
## Token Budget Control (AC7)
|
|
179
|
+
|
|
180
|
+
Per NFR-024, the total output must stay within the 40K token framework budget.
|
|
181
|
+
|
|
182
|
+
- Each gap entry averages ~100 tokens
|
|
183
|
+
- If test output produces more than 70 gap entries, truncate:
|
|
184
|
+
- Keep the first 70 gap entries (prioritized by severity: critical > high > medium > low)
|
|
185
|
+
- Add a summary line: `<!-- TRUNCATED: {N} additional test failures omitted to stay within NFR-024 token budget -->`
|
|
186
|
+
- If raw test output exceeds budget before parsing, truncate the raw output and parse what is available
|
|
187
|
+
|
|
188
|
+
## Output File
|
|
189
|
+
|
|
190
|
+
Write all gap entries to: `{planning_artifacts}/brownfield-scan-test-execution.md`
|
|
191
|
+
|
|
192
|
+
Format:
|
|
193
|
+
```markdown
|
|
194
|
+
# Brownfield Scan: Test Execution
|
|
195
|
+
|
|
196
|
+
> Scan type: test-execution
|
|
197
|
+
> Runner(s): {detected_runners}
|
|
198
|
+
> Date: {date}
|
|
199
|
+
|
|
200
|
+
## Test Metrics
|
|
201
|
+
|
|
202
|
+
| Runner | Total | Passed | Failed | Skipped |
|
|
203
|
+
|--------|-------|--------|--------|---------|
|
|
204
|
+
| {runner} | {n} | {n} | {n} | {n} |
|
|
205
|
+
|
|
206
|
+
## Gap Entries
|
|
207
|
+
|
|
208
|
+
{YAML gap entries here}
|
|
209
|
+
```
|
|
@@ -1,17 +1,35 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: document-rulesets
|
|
3
|
-
version: '1.
|
|
3
|
+
version: '1.1'
|
|
4
4
|
applicable_agents: [validator]
|
|
5
|
-
description: 'Document-specific validation rulesets for artifact type detection, structural quality checks per artifact type, and two-pass validation logic.'
|
|
6
|
-
sections: [type-detection, prd-rules, arch-rules, ux-rules, test-plan-rules, epics-rules, two-pass-logic]
|
|
5
|
+
description: 'Document-specific validation rulesets for artifact type detection (path and frontmatter), structural quality checks per artifact type (application, infrastructure, platform PRDs), and two-pass validation logic.'
|
|
6
|
+
sections: [type-detection, prd-rules, infra-prd-rules, platform-prd-rules, arch-rules, ux-rules, test-plan-rules, epics-rules, two-pass-logic]
|
|
7
7
|
---
|
|
8
8
|
|
|
9
9
|
<!-- SECTION: type-detection -->
|
|
10
10
|
## Artifact Type Detection
|
|
11
11
|
|
|
12
|
-
###
|
|
12
|
+
### Frontmatter-Based Detection (Higher Priority)
|
|
13
13
|
|
|
14
|
-
|
|
14
|
+
Check frontmatter first before falling back to filename detection. If the artifact has YAML frontmatter with a `template` field, use the frontmatter mapping table below. Frontmatter detection takes higher priority than filename detection because it is explicit and unambiguous.
|
|
15
|
+
|
|
16
|
+
| Frontmatter `template` Value | Ruleset ID(s) | Description |
|
|
17
|
+
|------------------------------|---------------|-------------|
|
|
18
|
+
| `'prd'` | prd-rules | Application Product Requirements Document |
|
|
19
|
+
| `'infra-prd'` | infra-prd-rules | Infrastructure PRD |
|
|
20
|
+
| `'platform-prd'` | prd-rules + infra-prd-rules | Platform PRD (both rulesets applied) |
|
|
21
|
+
|
|
22
|
+
### Frontmatter Detection Algorithm
|
|
23
|
+
|
|
24
|
+
1. Parse the artifact's YAML frontmatter (content between opening `---` and closing `---`)
|
|
25
|
+
2. Check for a `template` field in the frontmatter
|
|
26
|
+
3. If `template` field exists, match against the frontmatter mapping table above
|
|
27
|
+
4. If a match is found, return the corresponding ruleset ID(s). For `'platform-prd'`, return both `prd-rules` and `infra-prd-rules` — both rulesets are applied sequentially
|
|
28
|
+
5. If `template` field is absent or does not match, fall through to path-based detection below
|
|
29
|
+
|
|
30
|
+
### Path-to-Ruleset Mapping (Fallback)
|
|
31
|
+
|
|
32
|
+
If no frontmatter match is found, detect the artifact type from the file path basename. Match against these patterns:
|
|
15
33
|
|
|
16
34
|
| File Pattern | Ruleset ID | Description |
|
|
17
35
|
|-------------|------------|-------------|
|
|
@@ -21,7 +39,7 @@ Detect the artifact type from the file path basename. Match against these patter
|
|
|
21
39
|
| `test-plan.md` | test-plan-rules | Test Plan |
|
|
22
40
|
| `epics-and-stories.md` | epics-rules | Epics and Stories |
|
|
23
41
|
|
|
24
|
-
### Detection Algorithm
|
|
42
|
+
### Path-Based Detection Algorithm
|
|
25
43
|
|
|
26
44
|
1. Extract the basename from the artifact path (e.g., `docs/planning-artifacts/prd.md` -> `prd.md`)
|
|
27
45
|
2. Match basename against the mapping table above (case-insensitive)
|
|
@@ -30,6 +48,7 @@ Detect the artifact type from the file path basename. Match against these patter
|
|
|
30
48
|
|
|
31
49
|
### Edge Cases and Fallback
|
|
32
50
|
|
|
51
|
+
- **Frontmatter takes priority**: If both frontmatter and filename match, frontmatter wins
|
|
33
52
|
- **Nested directories**: Only the basename matters — `any/nested/path/prd.md` still matches prd-rules
|
|
34
53
|
- **Custom paths**: Files with non-standard names (e.g., `custom-doc.md`) fall back to unknown/unrecognized type
|
|
35
54
|
- **Case sensitivity**: Match is case-insensitive (`PRD.md` and `prd.md` both match prd-rules)
|
|
@@ -66,6 +85,72 @@ Verify user personas referenced in requirements match personas defined in the pe
|
|
|
66
85
|
Verify terminology is used consistently across sections. Cross-reference all sections for contradictions. Flag inconsistencies as WARNING.
|
|
67
86
|
<!-- END SECTION -->
|
|
68
87
|
|
|
88
|
+
<!-- SECTION: infra-prd-rules -->
|
|
89
|
+
## Infra PRD Validation Rules
|
|
90
|
+
|
|
91
|
+
Structural quality checks for Infrastructure Product Requirements Documents. This ruleset validates infra PRDs per ADR-022 section 10.16.7 (FR-126).
|
|
92
|
+
|
|
93
|
+
### Section Presence Check
|
|
94
|
+
|
|
95
|
+
Verify all 13 required infrastructure PRD sections exist and are non-empty. Flag missing sections as CRITICAL.
|
|
96
|
+
|
|
97
|
+
Required sections:
|
|
98
|
+
1. Overview & Scope
|
|
99
|
+
2. Goals and Non-Goals
|
|
100
|
+
3. Platform Capabilities
|
|
101
|
+
4. Resource Specifications
|
|
102
|
+
5. Operational SLOs
|
|
103
|
+
6. Security Posture
|
|
104
|
+
7. Environment Strategy & Developer Experience
|
|
105
|
+
8. Dependencies & Provider Constraints
|
|
106
|
+
9. Cost Model
|
|
107
|
+
10. Verification Strategy
|
|
108
|
+
11. Operational Runbooks
|
|
109
|
+
12. Requirements Summary
|
|
110
|
+
13. Open Questions
|
|
111
|
+
|
|
112
|
+
### IR/OR/SR ID Uniqueness
|
|
113
|
+
|
|
114
|
+
Verify requirement IDs use the infrastructure ID scheme (IR-###, OR-###, SR-###). Check ID uniqueness within each prefix family — no duplicate IR-001 entries, no duplicate OR-001 entries, no duplicate SR-001 entries. Flag duplicate IDs as WARNING.
|
|
115
|
+
|
|
116
|
+
### Security Posture Non-Empty
|
|
117
|
+
|
|
118
|
+
The Security Posture section is mandatory and must be non-empty for all infrastructure PRDs. An empty or placeholder-only Security Posture section is a CRITICAL finding. This section must contain substantive content covering IAM/RBAC, network segmentation, secrets management, or compliance mapping.
|
|
119
|
+
|
|
120
|
+
### Cost Model Per-Environment Estimates
|
|
121
|
+
|
|
122
|
+
The Cost Model section must include per-environment cost estimates. At minimum, dev, staging, and prod environments must have cost projections. Flag missing per-environment estimates as WARNING.
|
|
123
|
+
|
|
124
|
+
### Verification Strategy Policy-as-Code Reference
|
|
125
|
+
|
|
126
|
+
The Verification Strategy section must reference at least one policy-as-code tool. Recognized tools include: OPA, Rego, Checkov, tfsec, Sentinel, or equivalent. Flag missing policy-as-code references as WARNING.
|
|
127
|
+
|
|
128
|
+
### Platform Capabilities Format Validation
|
|
129
|
+
|
|
130
|
+
Each entry in the Platform Capabilities section must follow the format: "Enable {team/service} to {capability} with {SLO}". Entries that do not match this pattern should be flagged as WARNING with a suggestion to reformat.
|
|
131
|
+
<!-- END SECTION -->
|
|
132
|
+
|
|
133
|
+
<!-- SECTION: platform-prd-rules -->
|
|
134
|
+
## Platform PRD Validation Rules
|
|
135
|
+
|
|
136
|
+
Platform PRDs represent hybrid projects that combine application and infrastructure concerns. A platform PRD must pass both the `prd-rules` (application) and `infra-prd-rules` (infrastructure) validation rulesets.
|
|
137
|
+
|
|
138
|
+
### Composite Validation
|
|
139
|
+
|
|
140
|
+
When validating a platform PRD (detected via `template: 'platform-prd'` in frontmatter):
|
|
141
|
+
1. Run all checks from `prd-rules` — application structural rules apply
|
|
142
|
+
2. Run all checks from `infra-prd-rules` — infrastructure structural rules apply
|
|
143
|
+
3. Merge findings from both rulesets into a single report
|
|
144
|
+
|
|
145
|
+
### Dual ID Scheme Validation
|
|
146
|
+
|
|
147
|
+
Platform PRDs use both application and infrastructure requirement ID schemes:
|
|
148
|
+
- Application requirements: FR-### (functional) and NFR-### (non-functional)
|
|
149
|
+
- Infrastructure requirements: IR-### (infrastructure), OR-### (operational), SR-### (security)
|
|
150
|
+
|
|
151
|
+
Both ID namespaces must be validated for uniqueness within their respective prefix families. No collisions between FR-001 and IR-001 because the prefix disambiguates.
|
|
152
|
+
<!-- END SECTION -->
|
|
153
|
+
|
|
69
154
|
<!-- SECTION: arch-rules -->
|
|
70
155
|
## Architecture Validation Rules
|
|
71
156
|
|
|
@@ -0,0 +1,219 @@
|
|
|
1
|
+
# Documentation-Code Mismatch Scanner — Subagent Prompt
|
|
2
|
+
|
|
3
|
+
> Brownfield deep analysis scan subagent for detecting documentation-code drift: stale docs, undocumented features, version mismatches, and config option drift.
|
|
4
|
+
> Reference: Architecture ADR-021, Section 10.15.2, 10.15.3, 10.15.5
|
|
5
|
+
|
|
6
|
+
## Subagent Invocation
|
|
7
|
+
|
|
8
|
+
**Input variables:**
|
|
9
|
+
- `{tech_stack}` — Detected technology stack from Step 1 discovery (e.g., "Java/Spring", "Node/Express", "Python/Django", "Go/Gin")
|
|
10
|
+
- `{project-path}` — Absolute path to the project source code directory
|
|
11
|
+
|
|
12
|
+
**Output file:** `{planning_artifacts}/brownfield-scan-doc-code.md`
|
|
13
|
+
|
|
14
|
+
**Invocation model:** Spawned via Agent tool in a single message alongside the other deep analysis scan subagents (parallel execution per architecture 10.15.2).
|
|
15
|
+
|
|
16
|
+
## Subagent Prompt
|
|
17
|
+
|
|
18
|
+
```
|
|
19
|
+
You are a Documentation-Code Mismatch Scanner for brownfield project analysis. Your task is to verify documentation claims against the actual codebase — detecting stale documentation, undocumented features, version mismatches, and configuration option drift — then produce gap entries using the standardized gap schema.
|
|
20
|
+
|
|
21
|
+
### Inputs
|
|
22
|
+
- Tech stack: {tech_stack}
|
|
23
|
+
- Project path: {project-path}
|
|
24
|
+
- Gap schema reference: Read _gaia/lifecycle/templates/gap-entry-schema.md for the output format
|
|
25
|
+
|
|
26
|
+
### Phase 1: Documentation File Discovery
|
|
27
|
+
|
|
28
|
+
Scan the project for all discoverable documentation files. Apply both generic and stack-specific discovery patterns.
|
|
29
|
+
|
|
30
|
+
**Generic documentation files (all stacks):**
|
|
31
|
+
|
|
32
|
+
| File/Pattern | Location | Description |
|
|
33
|
+
|-------------|----------|-------------|
|
|
34
|
+
| `README.md` | Project root | Primary project documentation |
|
|
35
|
+
| `CONTRIBUTING.md` | Project root | Contributor guidelines |
|
|
36
|
+
| `CHANGELOG.md` | Project root | Version history |
|
|
37
|
+
| `docs/` directory | Project root | Dedicated documentation directory |
|
|
38
|
+
| `*.md` in project root | Project root | Any markdown files at top level |
|
|
39
|
+
| `openapi.yaml`, `openapi.json` | Project root, `api/` | OpenAPI 3.x specification |
|
|
40
|
+
| `swagger.yaml`, `swagger.json` | Project root, `api/` | Swagger 2.x specification |
|
|
41
|
+
|
|
42
|
+
**Stack-Aware Documentation Patterns:**
|
|
43
|
+
|
|
44
|
+
#### Java/Spring
|
|
45
|
+
|
|
46
|
+
| File/Pattern | Location | Description |
|
|
47
|
+
|-------------|----------|-------------|
|
|
48
|
+
| Javadoc comments (`/** ... */`) | Source files | Inline API documentation and docstrings |
|
|
49
|
+
| `application.yml` / `application.properties` comments | `src/main/resources/` | Configuration documentation |
|
|
50
|
+
| `pom.xml` `<description>` elements | Project root | Maven project metadata |
|
|
51
|
+
| Spring REST Docs output | `build/generated-snippets/`, `target/generated-snippets/` | Auto-generated API documentation |
|
|
52
|
+
| `src/main/resources/static/docs/` | Resources | Bundled documentation |
|
|
53
|
+
|
|
54
|
+
#### Node/Express
|
|
55
|
+
|
|
56
|
+
| File/Pattern | Location | Description |
|
|
57
|
+
|-------------|----------|-------------|
|
|
58
|
+
| JSDoc comments (`/** ... */`) | Source files | Inline API documentation and docstrings |
|
|
59
|
+
| `package.json` `description`, `scripts` | Project root | Package metadata and available commands |
|
|
60
|
+
| `typedoc.json` or JSDoc config | Project root | Documentation generator config |
|
|
61
|
+
| `.env.example` | Project root | Documented environment variables |
|
|
62
|
+
| `docs/api/` | Documentation dir | API documentation files |
|
|
63
|
+
|
|
64
|
+
#### Python/Django
|
|
65
|
+
|
|
66
|
+
| File/Pattern | Location | Description |
|
|
67
|
+
|-------------|----------|-------------|
|
|
68
|
+
| Docstrings (`"""..."""`) | Source files | Inline documentation and docstrings |
|
|
69
|
+
| `pyproject.toml` `[project]` section | Project root | Package metadata |
|
|
70
|
+
| `requirements.txt` | Project root | Dependency documentation |
|
|
71
|
+
| Sphinx `conf.py` | `docs/` | Documentation generator config |
|
|
72
|
+
| Django `urls.py` docstrings | App directories | Route-level documentation |
|
|
73
|
+
| `setup.py` / `setup.cfg` metadata | Project root | Legacy package metadata |
|
|
74
|
+
|
|
75
|
+
#### Go/Gin
|
|
76
|
+
|
|
77
|
+
| File/Pattern | Location | Description |
|
|
78
|
+
|-------------|----------|-------------|
|
|
79
|
+
| Go doc comments (`// Package ...`) | Source files | Package-level documentation and docstrings |
|
|
80
|
+
| `go.mod` module declaration | Project root | Module metadata |
|
|
81
|
+
| `Makefile` targets and comments | Project root | Build command documentation |
|
|
82
|
+
| `cmd/` directory README files | `cmd/` subdirs | CLI command documentation |
|
|
83
|
+
| `internal/` package docs | `internal/` | Internal package documentation |
|
|
84
|
+
|
|
85
|
+
**Edge Case: Empty/Stub Documentation**
|
|
86
|
+
Skip documentation files with fewer than 2 non-empty lines after the header (title line). These are stub files that provide no actionable claims to verify. Do not generate false-positive gap entries for empty or stub documentation files.
|
|
87
|
+
|
|
88
|
+
**Edge Case: Non-UTF-8 Encoding**
|
|
89
|
+
Attempt to read each documentation file as UTF-8. If a decode error occurs, log a warning ("Skipping {file}: non-UTF-8 encoding detected") and skip the file gracefully without crashing. Do not generate gap entries for files that cannot be decoded.
|
|
90
|
+
|
|
91
|
+
### Phase 2: Claim Extraction
|
|
92
|
+
|
|
93
|
+
For each discovered documentation file, extract verifiable claims organized by type:
|
|
94
|
+
|
|
95
|
+
**Claim Type 1: Endpoint Claims**
|
|
96
|
+
Extract documented API endpoints — method, path, description. Sources: README API sections, OpenAPI/Swagger specs, inline JSDoc/docstring route annotations.
|
|
97
|
+
|
|
98
|
+
**Claim Type 2: Configuration Option Claims**
|
|
99
|
+
Extract documented configuration options — environment variables, config keys, default values. Sources: README configuration sections, `.env.example` files, config documentation.
|
|
100
|
+
|
|
101
|
+
**Claim Type 3: Dependency Claims**
|
|
102
|
+
Extract documented dependencies — package names, version constraints, runtime requirements. Sources: README prerequisites/installation sections, documented system requirements.
|
|
103
|
+
|
|
104
|
+
**Claim Type 4: Build/Run Command Claims**
|
|
105
|
+
Extract documented build, test, and run commands — script names, CLI invocations, make targets. Sources: README getting started sections, CONTRIBUTING.md, Makefile documentation.
|
|
106
|
+
|
|
107
|
+
### Phase 3: Code Verification
|
|
108
|
+
|
|
109
|
+
For each extracted claim, verify it against the actual codebase:
|
|
110
|
+
|
|
111
|
+
**Endpoint Verification:**
|
|
112
|
+
- Grep for route definitions matching documented paths (e.g., `app.get('/api/v1/users')`, `@GetMapping("/api/v1/users")`, `path('api/v1/users/')`, `router.GET("/api/v1/users")`)
|
|
113
|
+
- Match HTTP method + path pattern
|
|
114
|
+
- Flag documented endpoints not found in code as stale documentation
|
|
115
|
+
- Flag route definitions not found in documentation as undocumented features that are not documented
|
|
116
|
+
|
|
117
|
+
**Configuration Option Verification:**
|
|
118
|
+
- Grep for documented config key usage in source files (e.g., `process.env.MAX_RETRIES`, `@Value("${max.retries}")`, `os.environ.get('MAX_RETRIES')`, `os.Getenv("MAX_RETRIES")`)
|
|
119
|
+
- Check if documented defaults match actual defaults in code
|
|
120
|
+
- Flag documented config options not referenced anywhere in source as stale documentation
|
|
121
|
+
|
|
122
|
+
**Dependency Verification:**
|
|
123
|
+
- Compare documented dependencies against entries in package manifests: `package.json`, `pom.xml`, `go.mod`, `requirements.txt`, `pyproject.toml`
|
|
124
|
+
- Check version constraints: if README says "requires Node 16" but `engines` field says `>=20`, flag as version mismatch
|
|
125
|
+
- Flag documented dependencies not in manifests as stale documentation
|
|
126
|
+
- Flag manifest dependencies not mentioned in docs as missing documentation (low severity)
|
|
127
|
+
|
|
128
|
+
**Build Command Verification:**
|
|
129
|
+
- Verify documented build/run commands exist: check `scripts` in `package.json`, `Makefile` targets, `Dockerfile` commands, management commands in Django
|
|
130
|
+
- Flag documented commands that do not exist as stale documentation
|
|
131
|
+
- Flag undocumented scripts/targets as missing documentation (low severity)
|
|
132
|
+
|
|
133
|
+
### Phase 4: OpenAPI/Swagger Auto-Generated Spec Detection
|
|
134
|
+
|
|
135
|
+
When an OpenAPI or Swagger spec file is found, determine whether it is auto-generated or hand-written:
|
|
136
|
+
|
|
137
|
+
**Auto-generated spec indicators:**
|
|
138
|
+
- `x-generator` field in spec root (e.g., `x-generator: swagger-codegen`)
|
|
139
|
+
- `info.x-generated-by` field in spec info section
|
|
140
|
+
- Known generator tool signatures in comments or metadata: `swagger-codegen`, `openapi-generator`, `tsoa`, `springdoc-openapi`, `drf-spectacular`, `swag` (Go)
|
|
141
|
+
- Presence of `@Generated` or similar annotations in companion files
|
|
142
|
+
|
|
143
|
+
**Treatment of auto-generated specs:**
|
|
144
|
+
- Flag auto-generated specs with lower confidence findings (INFO severity instead of WARNING/MEDIUM)
|
|
145
|
+
- Add a note in the gap entry description: "Source is auto-generated spec — lower confidence for drift detection"
|
|
146
|
+
- Still verify claims from auto-generated specs, but do not treat mismatches as high severity since the spec may be stale due to regeneration lag rather than intentional drift
|
|
147
|
+
|
|
148
|
+
### Phase 5: Mismatch Detection and Classification
|
|
149
|
+
|
|
150
|
+
Classify each verified claim into one of three categories:
|
|
151
|
+
|
|
152
|
+
**Category A: Stale Documentation (documented but not in code)**
|
|
153
|
+
Features, endpoints, config options, or commands that appear in documentation but no longer exist in the codebase. These represent documentation that was not updated when code changed.
|
|
154
|
+
- **Default severity: medium**
|
|
155
|
+
|
|
156
|
+
**Category B: Missing Documentation (in code but not documented)**
|
|
157
|
+
Features, endpoints, config options, or commands that exist in the codebase but are not mentioned in any documentation file. These represent undocumented functionality.
|
|
158
|
+
- **Default severity: low** (unless the undocumented item is a public API endpoint, then medium)
|
|
159
|
+
|
|
160
|
+
**Category C: Version Mismatches**
|
|
161
|
+
Version numbers, runtime requirements, or dependency constraints in documentation that conflict with actual values in package files or code.
|
|
162
|
+
- **Default severity: medium**
|
|
163
|
+
|
|
164
|
+
### Phase 6: Gap Entry Generation
|
|
165
|
+
|
|
166
|
+
For each mismatch, produce a gap entry following the standardized gap schema:
|
|
167
|
+
|
|
168
|
+
- **id:** `GAP-DOC-CODE-{seq}` where seq is zero-padded 3-digit (e.g., GAP-DOC-CODE-001, GAP-DOC-CODE-002)
|
|
169
|
+
- **category:** `doc-code-drift`
|
|
170
|
+
- **severity:** See severity mapping in Phase 5 (default medium for stale docs and version mismatches, low for missing docs, INFO for auto-generated spec findings)
|
|
171
|
+
- **title:** Short summary (max 80 characters)
|
|
172
|
+
- **description:** Include the claim type, source documentation file, and what specifically mismatches
|
|
173
|
+
- **evidence:** `file` (relative path to the documentation or code file) and `line` (line number or range)
|
|
174
|
+
- **recommendation:** Actionable guidance — update docs, remove stale references, add missing documentation
|
|
175
|
+
- **verified_by:** `machine-detected`
|
|
176
|
+
- **confidence:** `high` (exact match/mismatch confirmed), `medium` (partial match, needs human review), `low` (heuristic detection, may be false positive)
|
|
177
|
+
|
|
178
|
+
### Token Budget Compliance (NFR-024)
|
|
179
|
+
|
|
180
|
+
Each gap entry must average approximately 100 tokens in structured YAML format:
|
|
181
|
+
- Use structured YAML, not prose paragraphs
|
|
182
|
+
- Keep `title` under 80 characters
|
|
183
|
+
- Keep `description` to 1-2 sentences
|
|
184
|
+
- Keep `recommendation` to 1-2 sentences
|
|
185
|
+
- Reference source via `evidence` instead of embedding code snippets
|
|
186
|
+
|
|
187
|
+
**Maximum:** 70 gap entries per scan output file.
|
|
188
|
+
|
|
189
|
+
**Truncation logic:** If total gap entries exceed 70, retain highest-severity entries first (critical > high > medium > low > info). Truncate the lowest-severity entries. Append a summary at the end of the output file:
|
|
190
|
+
"Truncated {N} entries of severity {severity} — {total} total doc-code mismatches found, {kept} entries retained."
|
|
191
|
+
|
|
192
|
+
### Output Format
|
|
193
|
+
|
|
194
|
+
Write all gap entries to `{planning_artifacts}/brownfield-scan-doc-code.md` using this format:
|
|
195
|
+
|
|
196
|
+
```markdown
|
|
197
|
+
# Brownfield Scan: Documentation-Code Mismatch Analysis
|
|
198
|
+
|
|
199
|
+
> Generated by: Documentation-Code Mismatch Scanner
|
|
200
|
+
> Tech stack: {tech_stack}
|
|
201
|
+
> Date: {date}
|
|
202
|
+
> Total findings: {count}
|
|
203
|
+
|
|
204
|
+
## Gap Entries
|
|
205
|
+
|
|
206
|
+
\`\`\`yaml
|
|
207
|
+
- id: "GAP-DOC-CODE-001"
|
|
208
|
+
category: "doc-code-drift"
|
|
209
|
+
severity: "medium"
|
|
210
|
+
title: "README documents /api/v1/legacy endpoint that does not exist"
|
|
211
|
+
description: "README.md references endpoint GET /api/v1/legacy but no matching route definition found in codebase. Stale documentation."
|
|
212
|
+
evidence:
|
|
213
|
+
file: "README.md"
|
|
214
|
+
line: 47
|
|
215
|
+
recommendation: "Remove /api/v1/legacy reference from README.md or restore the endpoint if removal was unintentional."
|
|
216
|
+
verified_by: "machine-detected"
|
|
217
|
+
confidence: "high"
|
|
218
|
+
\`\`\`
|
|
219
|
+
```
|