@metasession.co/devaudit-cli 0.1.0 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +23 -11
- package/dist/index.js +21 -5
- package/dist/index.js.map +1 -1
- package/package.json +9 -5
- package/scripts/upload-evidence.sh +225 -0
- package/sdlc/.claude/settings.local.json +11 -0
- package/sdlc/CLAUDE.md +73 -0
- package/sdlc/HOST_ADAPTER.md +127 -0
- package/sdlc/SKILLS.md +137 -0
- package/sdlc/STACK_ADAPTER.md +130 -0
- package/sdlc/ai-rules/INSTRUCTIONS-SDLC.md +172 -0
- package/sdlc/ai-rules/README.md +103 -0
- package/sdlc/ai-rules/SDLC_RULES.md +584 -0
- package/sdlc/ai-rules/claude/CLAUDE.md +192 -0
- package/sdlc/ai-rules/cursor/.cursorrules +167 -0
- package/sdlc/ai-rules/windsurf/.windsurfrules +167 -0
- package/sdlc/article.md +219 -0
- package/sdlc/files/_common/0-project-setup.md +410 -0
- package/sdlc/files/_common/1-plan-requirement.md +381 -0
- package/sdlc/files/_common/2-implement-and-test.md +276 -0
- package/sdlc/files/_common/3-compile-evidence.md +603 -0
- package/sdlc/files/_common/4-submit-for-review.md +362 -0
- package/sdlc/files/_common/5-deploy-main.md +251 -0
- package/sdlc/files/_common/Periodic_Security_Review_Schedule.md +169 -0
- package/sdlc/files/_common/README_TEMPLATE.md +441 -0
- package/sdlc/files/_common/Test_Architecture.md +461 -0
- package/sdlc/files/_common/Test_Plan_TEMPLATE.md +311 -0
- package/sdlc/files/_common/Test_Policy.md +277 -0
- package/sdlc/files/_common/Test_Strategy.md +359 -0
- package/sdlc/files/_common/github/ISSUE_TEMPLATE/bug.yml +75 -0
- package/sdlc/files/_common/github/ISSUE_TEMPLATE/config.yml +11 -0
- package/sdlc/files/_common/github/ISSUE_TEMPLATE/requirement.yml +75 -0
- package/sdlc/files/_common/github/ISSUE_TEMPLATE/task.yml +48 -0
- package/sdlc/files/_common/github/pull_request_template.md +69 -0
- package/sdlc/files/_common/implementing-an-sdlc-issue.md +413 -0
- package/sdlc/files/_common/scripts/derive-release-version.sh +40 -0
- package/sdlc/files/_common/scripts/derive-release-version.test.sh +98 -0
- package/sdlc/files/_common/scripts/submit-for-uat-review.sh +162 -0
- package/sdlc/files/_common/scripts/validate-commits.sh +83 -0
- package/sdlc/files/_common/scripts/validate-compliance-artifacts.sh +202 -0
- package/sdlc/files/_common/scripts/validate-compliance-artifacts.test.sh +202 -0
- package/sdlc/files/_common/skills/_schema/skill.schema.json +36 -0
- package/sdlc/files/_common/skills/e2e-test-engineer/SKILL.md +254 -0
- package/sdlc/files/_common/skills/e2e-test-engineer/references/bootstrap.md +244 -0
- package/sdlc/files/_common/skills/e2e-test-engineer/references/evidence.ts +40 -0
- package/sdlc/files/_common/skills/sdlc-implementer/SKILL.md +189 -0
- package/sdlc/files/_common/skills/sdlc-implementer/references/call-graph.md +64 -0
- package/sdlc/files/_common/skills/sdlc-implementer/references/change-request-loop.md +192 -0
- package/sdlc/files/_common/skills/sdlc-implementer/references/compliance-constraints.md +81 -0
- package/sdlc/files/ci/check-release-approval.yml.template +201 -0
- package/sdlc/files/ci/ci-status-fallback.yml.template +41 -0
- package/sdlc/files/ci/ci.yml.template +390 -0
- package/sdlc/files/ci/compliance-evidence.yml.template +161 -0
- package/sdlc/files/ci/compliance-validation.yml.template +34 -0
- package/sdlc/files/ci/post-deploy-prod.yml.template +159 -0
- package/sdlc/files/ci/python/ci.yml.template +335 -0
- package/sdlc/files/hosts/_schema/adapter.schema.json +103 -0
- package/sdlc/files/hosts/railway/adapter.json +32 -0
- package/sdlc/files/sdlc-config.example.json +74 -0
- package/sdlc/files/stacks/_schema/adapter.schema.json +151 -0
- package/sdlc/files/stacks/node/adapter.json +54 -0
- package/sdlc/files/stacks/node/hooks/.prettierrc.json +9 -0
- package/sdlc/files/stacks/node/hooks/commit-msg +7 -0
- package/sdlc/files/stacks/node/hooks/commitlint.config.mjs +64 -0
- package/sdlc/files/stacks/node/hooks/lint-staged.config.mjs +16 -0
- package/sdlc/files/stacks/node/hooks/pre-commit +13 -0
- package/sdlc/files/stacks/node/hooks/pre-push +15 -0
- package/sdlc/files/stacks/node/scripts/check-requirement-jsdoc.sh +54 -0
- package/sdlc/files/stacks/python/adapter.json +36 -0
- package/sdlc/files/stacks/python/hooks/.pre-commit-config.yaml +51 -0
|
@@ -0,0 +1,603 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Compile test, security, and AI evidence, update RTM, create release ticket for review
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
# Compile Evidence
|
|
6
|
+
|
|
7
|
+
**Pipeline Stage:** 3 of 5
|
|
8
|
+
**Previous:** `2-implement-and-test.md`
|
|
9
|
+
**Next:** `4-submit-for-review.md`
|
|
10
|
+
**References:** Test Strategy (`sdlc/files/Test_Strategy.md` in DevAudit) (evidence requirements), Test Architecture (tooling), Test Plan (artifact structure)
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## When to Use
|
|
15
|
+
|
|
16
|
+
- After implementing a tracked requirement (REQ-XXX)
|
|
17
|
+
- After significant changes needing audit trail
|
|
18
|
+
- Before creating a PR to `main`
|
|
19
|
+
|
|
20
|
+
**Skip** for trivial changes — go straight to `4-submit-for-review.md`.
|
|
21
|
+
|
|
22
|
+
## Evidence Storage Rule
|
|
23
|
+
|
|
24
|
+
**Markdown stays in git. Binary and JSON evidence goes to DevAudit.**
|
|
25
|
+
|
|
26
|
+
| Artifact | Store in | Why |
|
|
27
|
+
|----------|----------|-----|
|
|
28
|
+
| `compliance/RTM.md` | Git | Source of truth, version history, PR-reviewable |
|
|
29
|
+
| `compliance/evidence/REQ-XXX/test-scope.md` | Git | Planning artifact, reviewed in PRs |
|
|
30
|
+
| `compliance/evidence/REQ-XXX/implementation-plan.md` | Git | Design decisions artifact (MEDIUM/HIGH risk), reviewed in PRs |
|
|
31
|
+
| `compliance/evidence/REQ-XXX/test-plan.md` | Git | Test strategy — tests to add/update/remove, mapped to criteria |
|
|
32
|
+
| `compliance/evidence/REQ-XXX/test-execution-summary.md` | Git | Gate results, test changes, coverage against test plan |
|
|
33
|
+
| `compliance/evidence/REQ-XXX/ai-use-note.md` | Git | Small markdown, needs PR review |
|
|
34
|
+
| `compliance/evidence/REQ-XXX/ai-prompts.md` | Git | Small markdown, needs PR review |
|
|
35
|
+
| `compliance/evidence/REQ-XXX/security-summary.md` | Git | Small markdown, needs PR review |
|
|
36
|
+
| `compliance/pending-releases/RELEASE-TICKET-*.md` | Git | Reviewed and moved to approved-releases |
|
|
37
|
+
| E2E results (JSON) | DevAudit | Large, bloats git history |
|
|
38
|
+
| Screenshots (PNG/JPG) | DevAudit | Binary, bloats git history |
|
|
39
|
+
| SAST results (JSON) | DevAudit | Large JSON, bloats git history |
|
|
40
|
+
| Dependency audit (JSON) | DevAudit | Large JSON, bloats git history |
|
|
41
|
+
| Unit test output (TXT) | DevAudit | Verbose output, bloats git history |
|
|
42
|
+
| Test reports (HTML) | DevAudit | Binary, bloats git history |
|
|
43
|
+
|
|
44
|
+
## Release Identity
|
|
45
|
+
|
|
46
|
+
CI uploads are scoped to a **release record** in DevAudit, keyed by `(project, version)`. Every gate result, test artifact, and compliance document for one logical feature must land in the same release record — that is how the portal evaluates the release-completeness checklist and the four-gate panel.
|
|
47
|
+
|
|
48
|
+
The templated workflows derive the release version from the **latest commit on the branch** via `scripts/derive-release-version.sh`:
|
|
49
|
+
|
|
50
|
+
| Commit shape | Release version |
|
|
51
|
+
|---|---|
|
|
52
|
+
| Subject `[REQ-037] feat(...)` | `REQ-037` |
|
|
53
|
+
| Subject `feat(...)` + body `Ref: REQ-037` | `REQ-037` |
|
|
54
|
+
| Subject contains multiple tags (e.g. `[REQ-037][REQ-038]`) | First match wins → `REQ-037` |
|
|
55
|
+
| No REQ tag (housekeeping, dep bumps) | Bare date → `v2026.05.17` |
|
|
56
|
+
|
|
57
|
+
Both `ci.yml` and `compliance-evidence.yml` call the same helper, so a feature spanning many commits and a mix of code/docs pushes converges on **one** release record. Subject takes priority over body when both are present.
|
|
58
|
+
|
|
59
|
+
If you need a separate release container for a sub-piece of work — e.g. carving REQ-038 out of an in-flight feature — give it its own REQ-ID and tag the commits accordingly.
|
|
60
|
+
|
|
61
|
+
## Steps
|
|
62
|
+
|
|
63
|
+
### Step 0: Confirm CI Is Green
|
|
64
|
+
|
|
65
|
+
Before compiling evidence, verify the latest CI run on `develop` passed:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
gh run list --branch develop --limit 1
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
**Do NOT proceed** if CI is failing or was cancelled. Evidence must reflect a green pipeline. If CI failed, return to `2-implement-and-test.md` and fix the issue first.
|
|
72
|
+
|
|
73
|
+
For MEDIUM/HIGH risk with AI involvement, also verify:
|
|
74
|
+
```bash
|
|
75
|
+
# AI prompt log must exist and be non-empty
|
|
76
|
+
test -s compliance/evidence/REQ-XXX/ai-prompts.md && echo "OK" || echo "MISSING — create ai-prompts.md before proceeding"
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
### Step 1: Verify All Local Gates Pass
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
npx tsc --noEmit
|
|
85
|
+
semgrep scan --config auto [SOURCE_DIR]/ --severity ERROR --severity WARNING
|
|
86
|
+
npm audit --audit-level=high
|
|
87
|
+
npx playwright test
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
All must pass. Evidence must reflect a green suite.
|
|
91
|
+
|
|
92
|
+
> **Skill available:** for the E2E gate specifically, invoke the **`e2e-test-engineer`** skill (at `.claude/skills/e2e-test-engineer/SKILL.md`). It runs the suite, triages failures into flake / test-bug / app-defect / intended-visual-diff / unintended-visual-diff before taking any action, checks each acceptance criterion has a passing test, and files defects via whatever tracker the project uses (GitHub, GitLab, Jira, Linear, Azure DevOps; markdown report as fallback). See [`sdlc/SKILLS.md`](../sdlc/SKILLS.md) for the full list of available skills.
|
|
93
|
+
|
|
94
|
+
### Step 1a: Generate Test Execution Summary
|
|
95
|
+
|
|
96
|
+
After confirming all gates pass, generate the test execution summary. This documents what ran, the results, and maps back to the test plan.
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
cat > compliance/evidence/REQ-XXX/test-execution-summary.md << 'EOF'
|
|
100
|
+
# Test Execution Summary — REQ-XXX
|
|
101
|
+
|
|
102
|
+
**Date:** [YYYY-MM-DD]
|
|
103
|
+
**Git SHA:** [short SHA]
|
|
104
|
+
**CI Run:** [run ID or "local"]
|
|
105
|
+
|
|
106
|
+
## Gate Results
|
|
107
|
+
|
|
108
|
+
| Gate | Result | Details |
|
|
109
|
+
|------|--------|---------|
|
|
110
|
+
| TypeScript | PASS | 0 errors |
|
|
111
|
+
| SAST | PASS | [N] findings ([N] baseline) |
|
|
112
|
+
| Dependency Audit | PASS | [N] unaccepted high/critical |
|
|
113
|
+
| E2E Tests | PASS | [N]/[N] passed |
|
|
114
|
+
| Build | PASS | Production build succeeded |
|
|
115
|
+
|
|
116
|
+
## Test Changes in This Release
|
|
117
|
+
|
|
118
|
+
**Added:**
|
|
119
|
+
- `e2e/[spec-file].spec.ts` — [N] tests ([description])
|
|
120
|
+
|
|
121
|
+
**Updated:**
|
|
122
|
+
- `e2e/[spec-file].spec.ts` — [what changed]
|
|
123
|
+
|
|
124
|
+
**Removed:**
|
|
125
|
+
- [none, or file + justification]
|
|
126
|
+
|
|
127
|
+
## Test Plan Coverage
|
|
128
|
+
|
|
129
|
+
| Acceptance Criterion | Status | Test |
|
|
130
|
+
|---------------------|--------|------|
|
|
131
|
+
| [From test-plan.md] | PASS | `[spec-file]::[test-name]` |
|
|
132
|
+
|
|
133
|
+
## Evidence Locations
|
|
134
|
+
|
|
135
|
+
| Evidence | Location |
|
|
136
|
+
|----------|----------|
|
|
137
|
+
| E2E results | DevAudit: [project]/REQ-XXX/e2e-results.json |
|
|
138
|
+
| SAST results | DevAudit: [project]/REQ-XXX/sast-results.json |
|
|
139
|
+
| Dependency audit | DevAudit: [project]/REQ-XXX/dependency-audit.json |
|
|
140
|
+
| Playwright report | CI artifact: playwright-report/ |
|
|
141
|
+
EOF
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
This summary is committed to git (small markdown) and uploaded to DevAudit where reviewers can see it inline on the release dashboard.
|
|
145
|
+
|
|
146
|
+
---
|
|
147
|
+
|
|
148
|
+
### Step 2: Verify JSDoc Headers
|
|
149
|
+
|
|
150
|
+
```bash
|
|
151
|
+
git diff --name-only origin/main...HEAD -- '*.ts' '*.tsx' | head -20
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
Each modified file should have `@requirement REQ-XXX` header.
|
|
155
|
+
|
|
156
|
+
### Step 3: Upload Test Evidence to DevAudit
|
|
157
|
+
|
|
158
|
+
Upload evidence to DevAudit so reviewers can access full test results (Playwright reports, SAST scans, dependency audits) without needing GitHub Checks tab access. This is the primary way reviewers verify test evidence.
|
|
159
|
+
|
|
160
|
+
The upload script is available in the DevAudit repository at `scripts/upload-evidence.sh`.
|
|
161
|
+
|
|
162
|
+
```bash
|
|
163
|
+
# Ensure DevAudit environment variables are set
|
|
164
|
+
# SUPABASE_URL and SUPABASE_SERVICE_ROLE_KEY
|
|
165
|
+
|
|
166
|
+
# Upload E2E results
|
|
167
|
+
./scripts/upload-evidence.sh [PROJECT_SLUG] REQ-XXX e2e_result [E2E_RESULTS_PATH] \
|
|
168
|
+
--git-sha "$(git rev-parse HEAD)" \
|
|
169
|
+
--branch "$(git branch --show-current)"
|
|
170
|
+
|
|
171
|
+
# Upload unit test results
|
|
172
|
+
npm test -- --verbose 2>&1 | tee /tmp/unit-test-results.txt
|
|
173
|
+
./scripts/upload-evidence.sh [PROJECT_SLUG] REQ-XXX test_report /tmp/unit-test-results.txt \
|
|
174
|
+
--git-sha "$(git rev-parse HEAD)"
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
**Alternative (git-based):** If not using DevAudit, save evidence locally:
|
|
178
|
+
```bash
|
|
179
|
+
cp [E2E_RESULTS_PATH] compliance/evidence/REQ-XXX/
|
|
180
|
+
npm test -- --verbose 2>&1 | tee compliance/evidence/REQ-XXX/unit-test-results.txt
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
### Step 4: Upload Security Evidence
|
|
184
|
+
|
|
185
|
+
```bash
|
|
186
|
+
# Generate evidence files
|
|
187
|
+
semgrep scan --config auto [SOURCE_DIR]/ --json > /tmp/sast-results.json 2>&1
|
|
188
|
+
npm audit --json > /tmp/dependency-audit.json 2>&1
|
|
189
|
+
|
|
190
|
+
# Upload to DevAudit
|
|
191
|
+
./scripts/upload-evidence.sh [PROJECT_SLUG] REQ-XXX audit_log /tmp/sast-results.json \
|
|
192
|
+
--git-sha "$(git rev-parse HEAD)"
|
|
193
|
+
./scripts/upload-evidence.sh [PROJECT_SLUG] REQ-XXX audit_log /tmp/dependency-audit.json \
|
|
194
|
+
--git-sha "$(git rev-parse HEAD)"
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
Create a security summary (keep in git — this is a compliance document, not binary evidence):
|
|
198
|
+
```bash
|
|
199
|
+
cat > compliance/evidence/REQ-XXX/security-summary.md << EOF
|
|
200
|
+
## Security Evidence Summary — REQ-XXX
|
|
201
|
+
**Date:** $(date -I)
|
|
202
|
+
**SAST Tool:** Semgrep (auto config)
|
|
203
|
+
**SAST High/Critical Findings:** 0
|
|
204
|
+
**Dependency Audit High/Critical:** 0
|
|
205
|
+
Evidence uploaded to DevAudit project: [PROJECT_SLUG]
|
|
206
|
+
EOF
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
For Medium/High risk, add access control and audit log verification to the security summary.
|
|
210
|
+
|
|
211
|
+
### Step 5: Save AI Evidence (If Applicable)
|
|
212
|
+
|
|
213
|
+
Verify `ai-use-note.md` and `ai-prompts.md` exist (if AI was used). If missing:
|
|
214
|
+
|
|
215
|
+
```bash
|
|
216
|
+
cat > compliance/evidence/REQ-XXX/ai-use-note.md << 'EOF'
|
|
217
|
+
# AI Use Record — REQ-XXX
|
|
218
|
+
**AI Tool:** [tool name]
|
|
219
|
+
**Code Generated By AI:** [list files]
|
|
220
|
+
**Human Reviewer:** [name]
|
|
221
|
+
**Review Date:** [date]
|
|
222
|
+
**Regenerations:** [none / list]
|
|
223
|
+
EOF
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
### Step 6: Verify Test Scope and Implementation Plan
|
|
227
|
+
|
|
228
|
+
Review `compliance/evidence/REQ-XXX/test-scope.md` (created during PLAN stage). Confirm all testing items have been addressed:
|
|
229
|
+
|
|
230
|
+
```bash
|
|
231
|
+
cat compliance/evidence/REQ-XXX/test-scope.md
|
|
232
|
+
# Check: are all [ ] items now [x]?
|
|
233
|
+
# If not, complete the outstanding items before proceeding
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
For MEDIUM/HIGH risk, verify the implementation plan exists and matches what was built:
|
|
237
|
+
|
|
238
|
+
```bash
|
|
239
|
+
cat compliance/evidence/REQ-XXX/implementation-plan.md
|
|
240
|
+
# Check: does the plan match the actual implementation?
|
|
241
|
+
# If the approach changed during development, update the plan to reflect what was actually built.
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
### Step 7: Update RTM
|
|
245
|
+
|
|
246
|
+
Open `compliance/RTM.md`, Part B. Update status:
|
|
247
|
+
|
|
248
|
+
```markdown
|
|
249
|
+
| REQ-XXX | Description | [RISK] | implementation-files | evidence | TESTED - PENDING SIGN-OFF | Pending | -- |
|
|
250
|
+
```
|
|
251
|
+
|
|
252
|
+
### Step 8: Generate Release Ticket
|
|
253
|
+
|
|
254
|
+
Create `compliance/pending-releases/RELEASE-TICKET-REQ-XXX.md`:
|
|
255
|
+
|
|
256
|
+
```markdown
|
|
257
|
+
# Release Ticket: REQ-XXX — [Title]
|
|
258
|
+
|
|
259
|
+
**Status:** TESTED - PENDING SIGN-OFF
|
|
260
|
+
**Date:** [YYYY-MM-DD]
|
|
261
|
+
**Requirement ID:** REQ-XXX
|
|
262
|
+
**Risk Level:** [LOW / MEDIUM / HIGH]
|
|
263
|
+
**PR:** [Will be linked when PR is created]
|
|
264
|
+
|
|
265
|
+
---
|
|
266
|
+
|
|
267
|
+
## Summary
|
|
268
|
+
[1-3 sentences]
|
|
269
|
+
|
|
270
|
+
## AI Involvement
|
|
271
|
+
- **AI Tool Used:** [tool / none]
|
|
272
|
+
- **AI-Generated Files:** [list, or "none"]
|
|
273
|
+
- **Human Reviewer of AI Code:** [name]
|
|
274
|
+
- **Components Regenerated:** [none / list]
|
|
275
|
+
|
|
276
|
+
## Implementation Details
|
|
277
|
+
**Files Modified:**
|
|
278
|
+
- `path/to/file1.ts` — [what changed]
|
|
279
|
+
|
|
280
|
+
**Dependencies Added/Changed:**
|
|
281
|
+
- [package@version — purpose — vulnerability status]
|
|
282
|
+
- [or "No dependency changes"]
|
|
283
|
+
|
|
284
|
+
## Test Evidence
|
|
285
|
+
| Test Type | Count | Passed | Failed | Evidence Location |
|
|
286
|
+
|-----------|-------|--------|--------|-------------------|
|
|
287
|
+
| E2E (Playwright) | [N] | [N] | 0 | DevAudit portal: [PROJECT_SLUG]/REQ-XXX |
|
|
288
|
+
| Unit | [N] | [N] | 0 | DevAudit portal: [PROJECT_SLUG]/REQ-XXX |
|
|
289
|
+
|
|
290
|
+
## Security Evidence
|
|
291
|
+
| Check | Result | Evidence Location |
|
|
292
|
+
|-------|--------|-------------------|
|
|
293
|
+
| SAST | 0 high/critical | DevAudit portal: [PROJECT_SLUG]/REQ-XXX |
|
|
294
|
+
| Dependency Audit | 0 high/critical | DevAudit portal: [PROJECT_SLUG]/REQ-XXX |
|
|
295
|
+
| Access Control | [PASS/N/A] | Git: `compliance/evidence/REQ-XXX/security-summary.md` |
|
|
296
|
+
| Audit Log | [PASS/N/A] | Git: `compliance/evidence/REQ-XXX/security-summary.md` |
|
|
297
|
+
|
|
298
|
+
## Acceptance Criteria
|
|
299
|
+
- [x] [From test-scope.md]
|
|
300
|
+
- [x] All E2E tests passing
|
|
301
|
+
- [x] TypeScript clean
|
|
302
|
+
- [x] SAST clean
|
|
303
|
+
- [x] Dependencies clean
|
|
304
|
+
- [x] AI use documented (if applicable)
|
|
305
|
+
|
|
306
|
+
## Risk Assessment
|
|
307
|
+
- [Any risks introduced]
|
|
308
|
+
- [New dependencies and trust assessment]
|
|
309
|
+
|
|
310
|
+
## Post-Deploy Actions
|
|
311
|
+
|
|
312
|
+
| Type | Script / Command | Target | Required | Notes |
|
|
313
|
+
|------|-----------------|--------|----------|-------|
|
|
314
|
+
| — | None | — | — | No post-deploy actions required |
|
|
315
|
+
|
|
316
|
+
<!-- Replace the "None" row above with actual actions if this release requires them:
|
|
317
|
+
| Data migration | `npx tsx scripts/backfill-x.ts "[CONN_STRING]"` | Prod DB | Yes | [description] |
|
|
318
|
+
| Schema migration | `npx prisma migrate deploy` | Prod DB | Yes | [description] |
|
|
319
|
+
-->
|
|
320
|
+
|
|
321
|
+
**Run these after deployment, before production verification.**
|
|
322
|
+
|
|
323
|
+
---
|
|
324
|
+
|
|
325
|
+
## Reviewer Checklist
|
|
326
|
+
- [ ] Code matches requirement
|
|
327
|
+
- [ ] Test evidence present and all-pass
|
|
328
|
+
- [ ] Security evidence present and clean
|
|
329
|
+
- [ ] Test scope fully addressed
|
|
330
|
+
- [ ] RTM correct status and risk
|
|
331
|
+
- [ ] No sensitive data committed
|
|
332
|
+
- [ ] No regressions
|
|
333
|
+
- [ ] AI code reviewed (if applicable)
|
|
334
|
+
- [ ] No hallucinated dependencies
|
|
335
|
+
- [ ] Post-deploy actions documented (or confirmed none required)
|
|
336
|
+
|
|
337
|
+
---
|
|
338
|
+
|
|
339
|
+
## Audit Trail
|
|
340
|
+
| Date | Action | Actor | Notes |
|
|
341
|
+
|------|--------|-------|-------|
|
|
342
|
+
| [date] | Requirement created | [who] | Risk: [level] |
|
|
343
|
+
| [date] | Implementation completed | [who] | [details] |
|
|
344
|
+
| [date] | AI code reviewed | [reviewer] | [files] |
|
|
345
|
+
| [date] | Tests passed | [who] | E2E + SAST: clean |
|
|
346
|
+
| [date] | UAT verification passed | [who] | Health + smoke + feature verified |
|
|
347
|
+
| [date] | Post-deploy actions | [who] | [Completed / None required] |
|
|
348
|
+
| [date] | Submitted for review | [who] | PR #[number] |
|
|
349
|
+
```
|
|
350
|
+
|
|
351
|
+
### Step 9: Commit Compliance Docs and Push
|
|
352
|
+
|
|
353
|
+
Commit compliance documents and push immediately. Heavy CI gates (E2E, TypeScript, build) skip markdown-only pushes via `paths-ignore`, so this push is cheap — but the Compliance Evidence Upload workflow fires and pushes the new artefacts to DevAudit. Pushing immediately means destination breakage (dead alias URL, revoked API key, schema drift) surfaces in seconds, not at the end of the stage.
|
|
354
|
+
|
|
355
|
+
> **What changed in sdlc-v1.22.0:** Earlier versions of this stage held the commit locally and batched the push at the end. We learned the hard way that this hides destination integration bugs — a stale `devaudit.base_url` was invisible until the batched push, by which point the dev had already done UAT verification against the assumption that evidence had been uploaded. Push-early surfaces those problems within seconds.
|
|
356
|
+
|
|
357
|
+
If using DevAudit, commit only compliance documents (RTM, release ticket, test scope, AI notes, security summary). Binary evidence (JSON results, screenshots) is stored in DevAudit, not git.
|
|
358
|
+
|
|
359
|
+
**Before committing, verify all required artifacts exist:**
|
|
360
|
+
- [ ] `compliance/evidence/REQ-XXX/test-scope.md`
|
|
361
|
+
- [ ] `compliance/evidence/REQ-XXX/test-plan.md`
|
|
362
|
+
- [ ] `compliance/evidence/REQ-XXX/test-execution-summary.md`
|
|
363
|
+
- [ ] `compliance/evidence/REQ-XXX/implementation-plan.md` (MEDIUM/HIGH risk only)
|
|
364
|
+
- [ ] `compliance/evidence/REQ-XXX/ai-prompts.md` (if AI used on MEDIUM/HIGH risk)
|
|
365
|
+
- [ ] `compliance/evidence/REQ-XXX/security-summary.md`
|
|
366
|
+
- [ ] `compliance/pending-releases/RELEASE-TICKET-REQ-XXX.md`
|
|
367
|
+
|
|
368
|
+
```bash
|
|
369
|
+
# DevAudit projects — commit + push compliance docs
|
|
370
|
+
git add compliance/RTM.md compliance/pending-releases/RELEASE-TICKET-REQ-XXX.md \
|
|
371
|
+
compliance/evidence/REQ-XXX/test-scope.md \
|
|
372
|
+
compliance/evidence/REQ-XXX/test-plan.md \
|
|
373
|
+
compliance/evidence/REQ-XXX/implementation-plan.md \
|
|
374
|
+
compliance/evidence/REQ-XXX/test-execution-summary.md \
|
|
375
|
+
compliance/evidence/REQ-XXX/ai-use-note.md \
|
|
376
|
+
compliance/evidence/REQ-XXX/ai-prompts.md \
|
|
377
|
+
compliance/evidence/REQ-XXX/security-summary.md
|
|
378
|
+
git commit -m "compliance: [REQ-XXX] evidence compiled - awaiting review"
|
|
379
|
+
git push origin develop
|
|
380
|
+
```
|
|
381
|
+
|
|
382
|
+
If NOT using DevAudit (git-based evidence):
|
|
383
|
+
```bash
|
|
384
|
+
git add compliance/RTM.md compliance/pending-releases/RELEASE-TICKET-REQ-XXX.md compliance/evidence/REQ-XXX/
|
|
385
|
+
git commit -m "compliance: [REQ-XXX] evidence compiled - awaiting review"
|
|
386
|
+
git push origin develop
|
|
387
|
+
```
|
|
388
|
+
|
|
389
|
+
#### Wait for the Compliance Evidence Upload workflow
|
|
390
|
+
|
|
391
|
+
```bash
|
|
392
|
+
gh run watch --workflow "Compliance Evidence Upload"
|
|
393
|
+
```
|
|
394
|
+
|
|
395
|
+
If it fails — typically a stale `devaudit.base_url`, a revoked `DEVAUDIT_API_KEY`, or schema drift — fix the configuration and re-push. Resolving this here is fast and recoverable; the same failure caught at the end of the stage would mean a long detour through UAT verification before discovering the upload never happened.
|
|
396
|
+
|
|
397
|
+
### Step 10: UAT-Environment Verification (CONDITIONAL)
|
|
398
|
+
|
|
399
|
+
**Skip this step entirely if any of these are true:**
|
|
400
|
+
|
|
401
|
+
- Project's `sdlc-config.json` has `uat.enabled: false` — meaning the project has no deployed UAT environment configured (internal services, retroactive-compliance pickups, etc.).
|
|
402
|
+
- Requirement's risk class is **not** listed in project's `uat.required_risk_classes` (defaults: `payment`, `destructive_migration`, `realtime`, `physical_ux`). Text-only fixes, internal refactors, low-risk UI tweaks carry none of these and skip UAT-env verification.
|
|
403
|
+
|
|
404
|
+
When skipped, proceed directly to Step 11.
|
|
405
|
+
|
|
406
|
+
> **Why opt-in by risk class?** UAT-env verification has two functions: (a) catching environment-specific issues (env vars, DB differences, build behaviour) and (b) recording that a human exercised the deployed system before approval. (a) is only valuable when there are environment-specific failure modes — a text-label change can't carry one. (b) is the four-eyes record, which Step 11 captures independently. Running UAT-env verification on every requirement adds ceremony without value; running it on risky requirements adds confidence where it matters. See sdlc-v1.22.0 release notes for the full rationale.
|
|
407
|
+
|
|
408
|
+
When this step DOES apply, the develop branch's auto-deploy to UAT must complete first. **Wait for the deployment to complete**, then verify the change works in the UAT environment.
|
|
409
|
+
|
|
410
|
+
#### WAIT CHECKPOINT: Confirm CI + Deployment Complete
|
|
411
|
+
|
|
412
|
+
Before UAT verification, confirm BOTH CI and deployment are complete:
|
|
413
|
+
|
|
414
|
+
```bash
|
|
415
|
+
# Confirm CI passed
|
|
416
|
+
gh run list --branch develop --limit 1
|
|
417
|
+
|
|
418
|
+
# Confirm UAT deployment is live (check hosting platform dashboard)
|
|
419
|
+
curl -s [UAT_URL]/[HEALTH_ENDPOINT]
|
|
420
|
+
# Expected: success response
|
|
421
|
+
```
|
|
422
|
+
|
|
423
|
+
**Do NOT proceed** with UAT verification until both CI is green and the deployment is live. Testing against a stale deployment produces invalid evidence.
|
|
424
|
+
|
|
425
|
+
#### 10a. Wait for UAT deployment
|
|
426
|
+
|
|
427
|
+
Monitor in the hosting platform dashboard or wait for the build to complete.
|
|
428
|
+
|
|
429
|
+
#### 10b. Health check
|
|
430
|
+
|
|
431
|
+
```bash
|
|
432
|
+
curl -s [UAT_URL]/[HEALTH_ENDPOINT]
|
|
433
|
+
# Expected: success response
|
|
434
|
+
```
|
|
435
|
+
|
|
436
|
+
#### 10c. Smoke test
|
|
437
|
+
|
|
438
|
+
```bash
|
|
439
|
+
# Homepage loads
|
|
440
|
+
curl -s -o /dev/null -w "%{http_code}" [UAT_URL]/
|
|
441
|
+
# Expected: 200
|
|
442
|
+
|
|
443
|
+
# A key endpoint responds correctly
|
|
444
|
+
curl -s -o /dev/null -w "%{http_code}" [UAT_URL]/[PUBLIC_ENDPOINT]
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
#### 10d. Feature-specific verification
|
|
448
|
+
|
|
449
|
+
Manually verify the specific feature or fix you implemented works on UAT. This catches environment-specific issues (env vars, database differences, build behavior) that local testing cannot.
|
|
450
|
+
|
|
451
|
+
#### 10e. Record UAT results
|
|
452
|
+
|
|
453
|
+
```bash
|
|
454
|
+
cat >> compliance/evidence/REQ-XXX/security-summary.md << EOF
|
|
455
|
+
|
|
456
|
+
## UAT Verification — $(date -I)
|
|
457
|
+
- UAT Health check: PASS
|
|
458
|
+
- UAT Smoke test: PASS
|
|
459
|
+
- Feature verification: PASS — [brief description of what was verified]
|
|
460
|
+
- UAT URL: [UAT_URL]
|
|
461
|
+
EOF
|
|
462
|
+
|
|
463
|
+
git add compliance/evidence/REQ-XXX/security-summary.md
|
|
464
|
+
git commit -m "compliance: [REQ-XXX] UAT-environment verification passed"
|
|
465
|
+
git push origin develop
|
|
466
|
+
```
|
|
467
|
+
|
|
468
|
+
**If UAT-env verification fails:** Fix the issue on `develop`, re-run local gates, push, and repeat UAT-env verification. Do NOT proceed to Step 11 until UAT-env is green.
|
|
469
|
+
|
|
470
|
+
### Step 11: Submit for Review + Approve Release in DevAudit (MANDATORY)
|
|
471
|
+
|
|
472
|
+
This is the **four-eyes release approval gate**. It has two transitions:
|
|
473
|
+
|
|
474
|
+
- **Step 11a. Submit for UAT review** (`draft → uat_review`) — the dev (or AI agent) signals the release is ready for review.
|
|
475
|
+
- **Step 11b. Approve** (`uat_review → uat_approved`) — an authorised reviewer (a different person under `approval.mode: dual_actor`) reviews the evidence and clicks Approve.
|
|
476
|
+
|
|
477
|
+
#### Step 11a — Submit for UAT review
|
|
478
|
+
|
|
479
|
+
Two paths; do whichever fits the project's workflow:
|
|
480
|
+
|
|
481
|
+
**Manual.** Open the release in DevAudit and click **Submit for UAT Review**. URL shape: `https://[DEVAUDIT_BASE_URL]/projects/[PROJECT_SLUG]/releases/[releaseId]` — posted as a comment on the develop branch by CI (look at the latest run of `Release Approval Gate` or `Compliance Evidence Upload`).
|
|
482
|
+
|
|
483
|
+
**Scripted.** Run the bundled script (synced from META-COMPLY into every consuming project under `scripts/`):
|
|
484
|
+
|
|
485
|
+
```bash
|
|
486
|
+
./scripts/submit-for-uat-review.sh [PROJECT_SLUG] v2026.MM.DD
|
|
487
|
+
```
|
|
488
|
+
|
|
489
|
+
The script:
|
|
490
|
+
|
|
491
|
+
1. Checks the working tree is clean and develop is up-to-date with origin.
|
|
492
|
+
2. Checks a `RELEASE-TICKET-*.md` exists in `compliance/pending-releases/`.
|
|
493
|
+
3. Checks CI gates are green on the current develop HEAD (via `gh run list`).
|
|
494
|
+
4. Resolves the release id from DevAudit using `DEVAUDIT_API_KEY` (existing).
|
|
495
|
+
5. Submits with `DEVAUDIT_USER_TOKEN` (Personal Access Token issued from `/settings/tokens` in DevAudit). The submission carries the issuing user's identity, so `isOwnRelease` keeps holding for Step 11b under `dual_actor`.
|
|
496
|
+
6. Idempotent — if the release is already in `uat_review` (or later), exits 0 with a note rather than failing.
|
|
497
|
+
|
|
498
|
+
Required environment variables for the scripted path:
|
|
499
|
+
|
|
500
|
+
| Var | What it is | Where to set |
|
|
501
|
+
|---|---|---|
|
|
502
|
+
| `DEVAUDIT_USER_TOKEN` | Personal Access Token (`mctok_…`) attributed to the running user | Issue at `/settings/tokens`; store as a repo secret for CI or `.env` for local |
|
|
503
|
+
| `DEVAUDIT_API_KEY` | Project-scoped API key (existing) | Already set for evidence uploads |
|
|
504
|
+
| `DEVAUDIT_BASE_URL` | DevAudit URL | Resolved by CI templates; locally read from `sdlc-config.json devaudit.base_url` |
|
|
505
|
+
|
|
506
|
+
#### Step 11b — Approve
|
|
507
|
+
|
|
508
|
+
After Step 11a completes (status: `uat_review`), an authorised reviewer:
|
|
509
|
+
|
|
510
|
+
1. Opens the release in DevAudit (same URL as above).
|
|
511
|
+
2. Reviews:
|
|
512
|
+
- **Quality gate results** (TypeScript, SAST, dependency audit, E2E, coverage) — uploaded by CI on the Stage 2 implementation push.
|
|
513
|
+
- **Compliance Markdowns** (RTM, release ticket, test-scope, test-execution-summary, security-summary, ai-prompts) — uploaded by Compliance Evidence Upload on the Step 9 push.
|
|
514
|
+
- **UAT-environment verification record** (if Step 10 ran) — in `security-summary.md`.
|
|
515
|
+
3. Clicks **Approve**. The release status transitions to `release_approved` (backend enum still `uat_approved` in v1.22.x for backwards-compat; renamed in v1.23.0).
|
|
516
|
+
|
|
517
|
+
If something looks wrong, click **Reject** and add a comment. Return to Stage 2 to fix, then re-walk Stage 3 from Step 1.
|
|
518
|
+
|
|
519
|
+
#### Approver mode
|
|
520
|
+
|
|
521
|
+
Project's `sdlc-config.json` `approval.mode` setting:
|
|
522
|
+
|
|
523
|
+
- `dual_actor` (recommended) — DevAudit enforces `approver_user_id ≠ release_creator_user_id`. If you're the release creator, you cannot approve your own release; delegate to another authorised reviewer.
|
|
524
|
+
- `solo_with_gap` — DevAudit allows self-approval. This is a documented control gap; the gap must be recorded in `compliance/risk-register.md` with explicit acknowledgement of which compliance clauses it diverges from (SOC2 CC8.1, ISO 29119 §5.4).
|
|
525
|
+
- `auto_low_risk` — LOW-risk requirements auto-approve once Compliance Evidence Upload completes (audited as a system event). MEDIUM/HIGH requirements always require a human click.
|
|
526
|
+
|
|
527
|
+
The `Release Approval Gate` workflow on Stage 4's PR enforces this — it polls DevAudit's API for `release.status` and fails the PR if approval isn't recorded. Wait for the next CI run on develop to confirm the gate sees the approval before proceeding to Stage 4.
|
|
528
|
+
|
|
529
|
+
## DevAudit CI Integration
|
|
530
|
+
|
|
531
|
+
Projects using DevAudit can automate evidence upload via the reusable GitHub Actions workflow. After CI tests pass, evidence is uploaded to the centralized portal where auditors can browse it.
|
|
532
|
+
|
|
533
|
+
### Versioning Convention
|
|
534
|
+
|
|
535
|
+
Releases use **date-based versioning** by default:
|
|
536
|
+
|
|
537
|
+
```
|
|
538
|
+
v{YYYY}.{MM}.{DD} — e.g. v2026.03.27
|
|
539
|
+
v{YYYY}.{MM}.{DD}.{N} — e.g. v2026.03.27.2 (second release same day)
|
|
540
|
+
```
|
|
541
|
+
|
|
542
|
+
The version is auto-generated by CI from the current date. Projects may override with semantic versioning (v1.2.0) if preferred by setting a `RELEASE_VERSION` variable.
|
|
543
|
+
|
|
544
|
+
### How Releases Are Created
|
|
545
|
+
|
|
546
|
+
CI **auto-creates releases** in DevAudit when uploading evidence. The workflow passes `--create-release-if-missing` to the upload script, which creates a `draft` release if one doesn't exist for the given version. This means:
|
|
547
|
+
|
|
548
|
+
- You don't need to manually create releases in DevAudit
|
|
549
|
+
- Evidence is always linked to a release (never orphaned)
|
|
550
|
+
- The release dashboard shows evidence immediately after CI runs
|
|
551
|
+
- Requirements are auto-synced from `compliance/RTM.md` to enable completeness tracking
|
|
552
|
+
|
|
553
|
+
### CI Workflow Setup
|
|
554
|
+
|
|
555
|
+
Add this job to your CI pipeline (after E2E tests pass):
|
|
556
|
+
|
|
557
|
+
```yaml
|
|
558
|
+
upload-evidence:
|
|
559
|
+
needs: [e2e-tests]
|
|
560
|
+
uses: metasession-dev/devaudit/.github/workflows/upload-evidence.yml@main
|
|
561
|
+
with:
|
|
562
|
+
project-slug: your-project-slug
|
|
563
|
+
release-version: v2026.03.27 # or use date-based auto-generation
|
|
564
|
+
environment: uat
|
|
565
|
+
secrets:
|
|
566
|
+
SUPABASE_URL: ${{ secrets.META_COMPLY_SUPABASE_URL }}
|
|
567
|
+
SUPABASE_SERVICE_ROLE_KEY: ${{ secrets.META_COMPLY_SERVICE_ROLE_KEY }}
|
|
568
|
+
```
|
|
569
|
+
|
|
570
|
+
This automatically:
|
|
571
|
+
- Creates the release in DevAudit if it doesn't exist (status: `draft`)
|
|
572
|
+
- Uploads compliance source documents (RTM, test plan, test cases, test summary report)
|
|
573
|
+
- Syncs `known_requirements` from RTM.md for completeness tracking
|
|
574
|
+
- Tags each upload with git SHA and CI run ID for traceability
|
|
575
|
+
|
|
576
|
+
### Additional Template Workflows
|
|
577
|
+
|
|
578
|
+
Copy these from `sdlc/files/ci/` into your project's `.github/workflows/`:
|
|
579
|
+
|
|
580
|
+
**`check-release-approval.yml`** (renamed from `check-uat-approval.yml` in sdlc-v1.22.0) — Release Approval Gate on PRs to main:
|
|
581
|
+
- Blocks merge until the release is approved in DevAudit (`uat_approved` / `release_approved` / downstream statuses)
|
|
582
|
+
- Add as a required status check on the `main` branch protection rule
|
|
583
|
+
|
|
584
|
+
**`post-deploy-prod.yml`** — Production evidence capture after merge to main:
|
|
585
|
+
- Runs production smoke tests
|
|
586
|
+
- Uploads production evidence to DevAudit (environment: production)
|
|
587
|
+
- Marks the release as `released`
|
|
588
|
+
|
|
589
|
+
The source of truth for compliance documents remains in git. DevAudit holds read-only snapshots so auditors see the full compliance picture in one place.
|
|
590
|
+
|
|
591
|
+
## Output
|
|
592
|
+
|
|
593
|
+
- RTM: `TESTED - PENDING SIGN-OFF`
|
|
594
|
+
- Release ticket in `compliance/pending-releases/`
|
|
595
|
+
- Test + security + AI evidence uploaded to DevAudit (or in `compliance/evidence/REQ-XXX/` if git-based)
|
|
596
|
+
- Compliance documents (test scope, AI notes, security summary) committed to git and pushed
|
|
597
|
+
- Test scope fully addressed
|
|
598
|
+
- UAT-environment verification passed and recorded (only if Step 10 applied — opt-in by risk class)
|
|
599
|
+
- **Release approved in DevAudit** (status: `uat_approved` / `release_approved`) — always required before PR to main
|
|
600
|
+
|
|
601
|
+
## Next Step
|
|
602
|
+
|
|
603
|
+
Proceed to `4-submit-for-review.md`.
|