@metasession.co/devaudit-cli 0.1.1 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (70) hide show
  1. package/README.md +13 -10
  2. package/dist/index.js +17 -5
  3. package/dist/index.js.map +1 -1
  4. package/package.json +9 -5
  5. package/scripts/upload-evidence.sh +225 -0
  6. package/sdlc/.claude/settings.local.json +11 -0
  7. package/sdlc/CLAUDE.md +73 -0
  8. package/sdlc/HOST_ADAPTER.md +127 -0
  9. package/sdlc/SKILLS.md +137 -0
  10. package/sdlc/STACK_ADAPTER.md +130 -0
  11. package/sdlc/ai-rules/INSTRUCTIONS-SDLC.md +172 -0
  12. package/sdlc/ai-rules/README.md +103 -0
  13. package/sdlc/ai-rules/SDLC_RULES.md +584 -0
  14. package/sdlc/ai-rules/claude/CLAUDE.md +192 -0
  15. package/sdlc/ai-rules/cursor/.cursorrules +167 -0
  16. package/sdlc/ai-rules/windsurf/.windsurfrules +167 -0
  17. package/sdlc/article.md +219 -0
  18. package/sdlc/files/_common/0-project-setup.md +410 -0
  19. package/sdlc/files/_common/1-plan-requirement.md +381 -0
  20. package/sdlc/files/_common/2-implement-and-test.md +276 -0
  21. package/sdlc/files/_common/3-compile-evidence.md +603 -0
  22. package/sdlc/files/_common/4-submit-for-review.md +362 -0
  23. package/sdlc/files/_common/5-deploy-main.md +251 -0
  24. package/sdlc/files/_common/Periodic_Security_Review_Schedule.md +169 -0
  25. package/sdlc/files/_common/README_TEMPLATE.md +441 -0
  26. package/sdlc/files/_common/Test_Architecture.md +461 -0
  27. package/sdlc/files/_common/Test_Plan_TEMPLATE.md +311 -0
  28. package/sdlc/files/_common/Test_Policy.md +277 -0
  29. package/sdlc/files/_common/Test_Strategy.md +359 -0
  30. package/sdlc/files/_common/github/ISSUE_TEMPLATE/bug.yml +75 -0
  31. package/sdlc/files/_common/github/ISSUE_TEMPLATE/config.yml +11 -0
  32. package/sdlc/files/_common/github/ISSUE_TEMPLATE/requirement.yml +75 -0
  33. package/sdlc/files/_common/github/ISSUE_TEMPLATE/task.yml +48 -0
  34. package/sdlc/files/_common/github/pull_request_template.md +69 -0
  35. package/sdlc/files/_common/implementing-an-sdlc-issue.md +413 -0
  36. package/sdlc/files/_common/scripts/derive-release-version.sh +40 -0
  37. package/sdlc/files/_common/scripts/derive-release-version.test.sh +98 -0
  38. package/sdlc/files/_common/scripts/submit-for-uat-review.sh +162 -0
  39. package/sdlc/files/_common/scripts/validate-commits.sh +83 -0
  40. package/sdlc/files/_common/scripts/validate-compliance-artifacts.sh +202 -0
  41. package/sdlc/files/_common/scripts/validate-compliance-artifacts.test.sh +202 -0
  42. package/sdlc/files/_common/skills/_schema/skill.schema.json +36 -0
  43. package/sdlc/files/_common/skills/e2e-test-engineer/SKILL.md +254 -0
  44. package/sdlc/files/_common/skills/e2e-test-engineer/references/bootstrap.md +244 -0
  45. package/sdlc/files/_common/skills/e2e-test-engineer/references/evidence.ts +40 -0
  46. package/sdlc/files/_common/skills/sdlc-implementer/SKILL.md +189 -0
  47. package/sdlc/files/_common/skills/sdlc-implementer/references/call-graph.md +64 -0
  48. package/sdlc/files/_common/skills/sdlc-implementer/references/change-request-loop.md +192 -0
  49. package/sdlc/files/_common/skills/sdlc-implementer/references/compliance-constraints.md +81 -0
  50. package/sdlc/files/ci/check-release-approval.yml.template +201 -0
  51. package/sdlc/files/ci/ci-status-fallback.yml.template +41 -0
  52. package/sdlc/files/ci/ci.yml.template +390 -0
  53. package/sdlc/files/ci/compliance-evidence.yml.template +161 -0
  54. package/sdlc/files/ci/compliance-validation.yml.template +34 -0
  55. package/sdlc/files/ci/post-deploy-prod.yml.template +159 -0
  56. package/sdlc/files/ci/python/ci.yml.template +335 -0
  57. package/sdlc/files/hosts/_schema/adapter.schema.json +103 -0
  58. package/sdlc/files/hosts/railway/adapter.json +32 -0
  59. package/sdlc/files/sdlc-config.example.json +74 -0
  60. package/sdlc/files/stacks/_schema/adapter.schema.json +151 -0
  61. package/sdlc/files/stacks/node/adapter.json +54 -0
  62. package/sdlc/files/stacks/node/hooks/.prettierrc.json +9 -0
  63. package/sdlc/files/stacks/node/hooks/commit-msg +7 -0
  64. package/sdlc/files/stacks/node/hooks/commitlint.config.mjs +64 -0
  65. package/sdlc/files/stacks/node/hooks/lint-staged.config.mjs +16 -0
  66. package/sdlc/files/stacks/node/hooks/pre-commit +13 -0
  67. package/sdlc/files/stacks/node/hooks/pre-push +15 -0
  68. package/sdlc/files/stacks/node/scripts/check-requirement-jsdoc.sh +54 -0
  69. package/sdlc/files/stacks/python/adapter.json +36 -0
  70. package/sdlc/files/stacks/python/hooks/.pre-commit-config.yaml +51 -0
@@ -0,0 +1,219 @@
1
+ # Building a Compliance-Grade Software Development Pipeline for the Age of AI-Assisted Coding
2
+
3
+ _How a small team built a tiered documentation system, automated security gates, and workflow-driven compliance pipeline that satisfies ISO 29119, ISO 27001, and emerging AI governance requirements — without drowning in paperwork._
4
+
5
+ ---
6
+
7
+ ## The Problem No One Has Solved Cleanly Yet
8
+
9
+ Software compliance has always been about proving that what you shipped was built correctly, tested thoroughly, and reviewed by a human who understood what they were approving. The documentation burden is real but manageable: write a test plan, maintain a traceability matrix, keep your evidence, get someone to sign off.
10
+
11
+ Then AI started writing the code.
12
+
13
+ Not all of it. Not even most of it, in many teams. But enough that the fundamental assumptions behind compliance frameworks started to crack. When an AI generates a function, who is accountable for its correctness? When it suggests a dependency, how do you know the package actually exists? When you reprompt and get a different implementation of the same feature, how do you prove the second version is equivalent to the first?
14
+
15
+ These aren't hypothetical concerns. Regulatory bodies are beginning to provide guidance, and the core message is blunt: you are responsible for the code, regardless of who or what wrote it. The EU AI Act is adding disclosure requirements for AI-built systems in high-risk categories. ISO 27001 auditors want to see that your SDLC explicitly addresses AI tooling. And the informal, iterative, low-documentation nature of how most developers use AI is the opposite of what compliance requires.
16
+
17
+ The question isn't whether you can use AI and still be compliant. You can. The question is what the framework looks like that makes it work without turning every feature into a bureaucratic ordeal.
18
+
19
+ This article describes one answer. It's not the only answer, but it's a complete, working system built from first principles and tested against real compliance requirements.
20
+
21
+ ---
22
+
23
+ ## What We Built
24
+
25
+ Over a series of iterations, we developed a two-tier documentation system with a workflow-driven development pipeline that handles everything from requirement planning through production deployment, with AI governance built into every stage.
26
+
27
+ The system consists of 12 documents across two tiers, a GitHub Actions CI pipeline for independent verification, and five operational workflows that a developer (or an AI assistant working alongside a developer) follows for every change.
28
+
29
+ ### The Tiered Structure
30
+
31
+ The first insight was that compliance documentation has two distinct layers that most teams accidentally merge into one: universal standards that apply regardless of what you're building, and project-specific details that change with every product.
32
+
33
+ **Tier 1** contains four documents that never reference a specific project:
34
+
35
+ The **Test Policy** defines why the organization tests, what it commits to, and who is accountable for what. This is where AI governance lives at the policy level — the permitted tools, the mandatory controls, the accountability chain. It states plainly that the human who commits AI-generated code is responsible for it, and that "the AI wrote it" is not an acceptable answer to an auditor.
36
+
37
+ The **Test Strategy** defines how testing is approached methodically. Testing levels (unit through acceptance), security testing methodology (per-commit SAST and dependency scanning, per-release access control and audit log verification, periodic penetration testing), AI-assisted development testing protocol (risk classification, human review requirements, the regeneration retest rule), verification vs. validation, defect management, traceability, and evidence requirements. It references tools by category ("SAST tool") but never names a specific product.
38
+
39
+ The **Test Architecture** defines what tests are built with. Specific tools and versions (Playwright, Jest, Semgrep, Snyk), directory structures, design patterns (Page Object Model, fixtures, factories), ESLint and Prettier configurations, CI pipeline stage specifications, artifact storage and retention policies, code style and naming conventions. This is the document an engineer opens when setting up test infrastructure.
40
+
41
+ The **Periodic Security Review Schedule** defines the recurring security activities that happen on a calendar rather than per-feature: quarterly SAST reviews, dependency deep audits, access control reviews, and audit log integrity checks; annual penetration testing, disaster recovery testing, and third-party security assessments.
42
+
43
+ The critical design decision was giving each document a single, non-overlapping responsibility. The Policy says "we will run SAST on every commit." The Strategy says "SAST is a mandatory gate that blocks merge on high/critical findings." The Architecture says "use Semgrep with auto config." No content appears in two places. When something changes, you update one document, not three.
44
+
45
+ **Tier 2** contains the project-specific documents: a Test Plan with concrete environment details, test suite counts, and exit criteria; five operational workflow files; a project setup guide; and a deployment reference. When you start a new project, you copy Tier 2 and customize it. Tier 1 stays untouched.
46
+
47
+ ---
48
+
49
+ ## The Six Workflows
50
+
51
+ The operational heart of the system is six documents that the developer follows sequentially. These aren't guidelines or reference material — they're executable procedures with specific commands, file paths, and decision points.
52
+
53
+ ### Workflow 0: Project Setup (Run Once)
54
+
55
+ Before any development work begins, this workflow configures the repository, branch protection, CI pipeline, compliance directories, and local tooling. The key output is the GitHub Actions workflow file that implements the independent verification gate.
56
+
57
+ This solved a problem that emerged during development: the five per-feature workflows assume CI is already configured. Without a setup workflow, teams would either skip CI configuration (creating a compliance gap) or configure it inconsistently across projects. Making it workflow 0 — an explicit, documented, one-time step — ensures every project starts with the same infrastructure.
58
+
59
+ The setup guide includes the complete GitHub Actions YAML with `# UPDATE` markers for project-specific values (Node version, source directory, database service, seed script, environment variables). A developer customizes the markers and has a working CI pipeline.
60
+
61
+ ### Workflow 1: Plan Requirement
62
+
63
+ Before writing any code, the developer defines the requirement in the RTM, classifies its risk level, and generates a test scope document.
64
+
65
+ The risk classification is the decision that governs everything downstream. Low risk (internal tools, no regulated data) gets standard gates and minimal documentation. Medium risk (PII, user-facing features, API changes) adds targeted security testing and validation. High risk (authentication, payments, RBAC) requires full verification and validation, independent review, and penetration testing consideration.
66
+
67
+ AI involvement raises risk by one level when the code touches Medium or High categories. This isn't punitive — it reflects the empirical reality that AI-generated code introduces injection vulnerabilities, insecure defaults, and hallucinated dependencies at higher rates than careful human development.
68
+
69
+ The test scope document is generated proportionally to risk. For Low risk, it's a few lines: standard gates plus acceptance criteria. For Medium, it's half a page: standard gates plus specific testing items (which endpoints need access control verification, which actions need audit log testing), a validation approach, and acceptance criteria. For High, it's a full page with security testing detail, independent review planning, and business workflow validation.
70
+
71
+ The critical compliance point: this document exists before implementation begins. It's evidence of planned testing, not retroactive documentation. An auditor can see that the testing approach was determined before the code was written.
72
+
73
+ ### Workflow 2: Implement and Test
74
+
75
+ The development loop: code, commit, run all gates, fix, repeat. Every commit on `develop` must leave all gates green.
76
+
77
+ The mandatory gates are TypeScript compilation, SAST scanning (Semgrep), dependency auditing (`npm audit`), and the full E2E test suite. These run locally on the developer's machine during implementation, providing comprehensive evidence.
78
+
79
+ For AI-assisted work, the developer logs significant prompts and outputs to the evidence directory (proportional to risk level), and flags any component regenerations. The regeneration protocol is one of the system's more nuanced rules: if you ask the AI to generate a component from scratch a second time (rather than incrementally editing the first output), that triggers a full retest of the component and its dependents. You can't assume two AI outputs for the same prompt are functionally equivalent.
80
+
81
+ ### Workflow 3: Compile Evidence
82
+
83
+ After implementation and testing, this workflow gathers all compliance artifacts: test results, SAST scan output, dependency audit results, security summary, AI use documentation, and the release ticket. It updates the RTM status and verifies that every item in the test scope (from workflow 1) has been addressed.
84
+
85
+ The release ticket template includes sections for AI involvement (which tool, which files, who reviewed the AI code, any regenerations), dependency changes (with vulnerability status), and a reviewer checklist that goes beyond code quality into security and compliance verification.
86
+
87
+ ### Workflow 4: Submit for Review
88
+
89
+ Creating the PR triggers two things simultaneously: GitHub Actions runs the independent verification pipeline (TypeScript, SAST, dependency audit, unauthenticated E2E tests), and the human reviewer is notified.
90
+
91
+ This is where the two-source evidence model comes together. The developer's local evidence (in the compliance directory) proves comprehensive testing was performed. The CI evidence (in GitHub Actions logs and artifacts) proves it independently — produced by GitHub's infrastructure, not by the developer. The human reviewer sees both before making their decision.
92
+
93
+ The PR checklist explicitly includes test scope verification: the reviewer confirms the test scope document exists, the risk classification is appropriate, the testing depth matches the risk level, and all items in the scope were addressed. This catches the case where the AI under-classifies risk or under-specifies the test scope.
94
+
95
+ Branch protection prevents merging until CI passes. This is enforced by GitHub, not by team discipline.
96
+
97
+ ### Workflow 5: Deploy to Main
98
+
99
+ After approval, the PR is merged (creating a merge commit that preserves the full audit trail), the hosting platform auto-deploys, and the developer performs post-deploy verification: health check, smoke test, and security verification (access control enforcement, security headers, no exposed stack traces).
100
+
101
+ The compliance artifacts are finalized: the release ticket moves from `pending-releases/` to `approved-releases/`, the RTM is updated to `APPROVED - DEPLOYED`, and the audit trail records every step from requirement to production.
102
+
103
+ ---
104
+
105
+ ## The AI Governance Model
106
+
107
+ The system's approach to AI governance is designed around a specific insight: AI-generated code doesn't need different testing — it needs more disciplined testing, applied consistently. The controls are:
108
+
109
+ **Accountability is non-transferable.** The Test Policy states it plainly: the human who commits the code is responsible for it. The reviewer who approves it shares that responsibility. This isn't a legal novelty — it's the same principle that applies to code from any source. But making it explicit in the policy prevents the "the AI wrote it" excuse from entering the culture.
110
+
111
+ **Human review is a formal compliance gate, not just good practice.** Every piece of AI-generated code must be reviewed by a qualified human before entering the test pipeline. "Qualified" means the reviewer understands the domain. The review is logged through the PR approval process — who reviewed it, when, and their decision.
112
+
113
+ **Risk classification accounts for AI involvement.** AI-generated code touching security-sensitive areas raises the risk level by one tier. This increases the review depth, testing requirements, and documentation burden proportionally.
114
+
115
+ **Regeneration triggers retest.** The non-determinism of AI output means you can't assume two generations of the same feature are equivalent. The system treats regeneration as a distinct event requiring full retesting, not an edit that follows standard gates.
116
+
117
+ **Documentation scales with risk.** Low-risk AI-assisted work needs only a `Co-Authored-By` commit tag. Medium risk adds a summary in the evidence directory. High risk requires detailed prompt and output retention. This avoids the trap of requiring full AI transcripts for every trivial change while ensuring the audit trail is robust where it matters.
118
+
119
+ **Permitted tools are explicit.** The Test Policy maintains a table of approved AI tools with their permitted uses and restrictions. Adding a new tool requires Engineering Leadership approval and policy updates. This prevents shadow AI usage from creating compliance blind spots.
120
+
121
+ ---
122
+
123
+ ## The Independent Verification Gate
124
+
125
+ One of the system's most important features emerged from a practical question: if a single developer is writing, testing, and committing code, where is the independent verification?
126
+
127
+ Running all tests locally means the developer produces all the evidence. An auditor has to trust that the tests ran against the actual code and the results weren't modified. That's a trust-based model.
128
+
129
+ Adding CI changes it to an evidence-based model. GitHub Actions runs TypeScript compilation, SAST scanning, dependency auditing, and the unauthenticated E2E test subset automatically on every PR. These results are produced by GitHub's infrastructure and recorded in GitHub's logs. The developer can't modify them.
130
+
131
+ The two evidence sources serve different purposes:
132
+
133
+ **Local testing** (developer machine) is the comprehensive gate. All tests, all security scans, the full suite. This is where defects are found and fixed.
134
+
135
+ **CI testing** (GitHub Actions) is the independent verification gate. A subset that provides tamper-resistant proof. This is what the auditor trusts.
136
+
137
+ Branch protection ties them together: the PR cannot merge unless CI passes. The human reviewer sees both sources — local evidence in the compliance directory and CI evidence in the PR checks — and approves based on the combination.
138
+
139
+ ---
140
+
141
+ ## What an Auditor Sees
142
+
143
+ When an auditor examines a change that went through this pipeline, they find a complete chain of evidence:
144
+
145
+ The **RTM** shows the requirement was defined with a risk classification before implementation began. The **test scope** document, timestamped before the first implementation commit, proves testing was planned, not retroactive. The **commit history** shows incremental development with requirement references and `Co-Authored-By` tags for AI involvement. The **compliance evidence directory** contains SAST scan results (JSON), dependency audit results (JSON), E2E test results, a security summary, and AI use documentation.
146
+
147
+ The **PR** shows the complete reviewer checklist, the CI check results (independent of the developer), the reviewer's approval with their identity and timestamp (recorded immutably by GitHub), and the compliance artifacts in the changed files. The **release ticket** in `approved-releases/` shows the full audit trail from requirement creation through deployment verification.
148
+
149
+ For AI-assisted changes, the auditor additionally finds: which AI tool was used, which files it generated, who reviewed the AI output, whether any components were regenerated (and whether retesting was performed), and — for Medium/High risk — the prompts and outputs that produced the code.
150
+
151
+ The chain is unbroken: requirement → planned test scope → implementation → comprehensive local testing → compiled evidence → independent CI verification → human review → deployment → post-deploy verification → finalization. Every link is documented and traceable.
152
+
153
+ ---
154
+
155
+ ## Compliance Framework Coverage
156
+
157
+ The system maps to specific compliance framework requirements:
158
+
159
+ **ISO/IEC/IEEE 29119-3** (Test Documentation): The tiered document structure directly implements the standard's artifact hierarchy. Test Policy, Test Strategy, Test Plan, Test Case Specifications (BDD feature files), Test Execution Logs (CI and local evidence), and Test Completion Reports (release tickets) are all present and correctly scoped.
160
+
161
+ **ISO 27001** (Information Security): Change control and traceability through the PR-based approval gate. Vulnerability management through per-commit SAST and dependency scanning plus periodic penetration testing. Access control testing as a per-release and quarterly activity. Audit log verification on the same schedule. Evidence retention (minimum 3 years) documented and implemented through Git history plus artifact storage.
162
+
163
+ **GDPR**: Test data privacy controls (no production PII in non-production environments, synthetic data generation). Data subject rights testing (access, rectification, erasure, portability, objection) included in test scope templates for relevant changes. Privacy by design testing built into the Medium/High risk test scope.
164
+
165
+ **EU AI Act preparedness**: AI tool documentation, human oversight integrated into the development process, prompt and output retention providing the basis for conformity assessment if required. The Test Policy explicitly addresses high-risk classification and its implications.
166
+
167
+ **SOC 2**: System monitoring (quarterly SAST + audit log review), logical access (quarterly access control review), change management (per-feature pipeline + quarterly reviews) map to CC7.1, CC6.1, and CC8.1 respectively.
168
+
169
+ ---
170
+
171
+ ## What Makes This Different
172
+
173
+ Most compliance frameworks for AI-assisted development fall into one of two traps: they're either so lightweight they don't satisfy auditors, or so heavy they make AI assistance counterproductive.
174
+
175
+ This system avoids both by making documentation proportional to risk. A Low-risk change (a UI tweak, a config update) goes through the same pipeline but produces minimal documentation — a commit tag, standard gate results, a brief PR. The overhead is negligible. A High-risk change (modifying authentication logic with AI assistance) produces a full test scope, detailed AI use documentation, independent security review, and comprehensive evidence. The overhead is significant but justified.
176
+
177
+ The workflow-driven approach means compliance isn't a separate activity bolted onto development — it's the development process itself. You don't "do compliance" after you finish coding. The compliance artifacts are natural outputs of the workflow steps you're already following.
178
+
179
+ The tiered document structure means you don't rewrite your compliance framework for every project. Tier 1 is written once and applies to everything. Tier 2 is customized per project from templates. Starting a new project means running the setup workflow, filling in placeholders, and executing the pipeline.
180
+
181
+ And the independent verification gate (CI) solves the single-developer trust problem without requiring a full QA team. GitHub Actions provides tamper-resistant evidence that the code passed all gates, regardless of who wrote it or who committed it.
182
+
183
+ ---
184
+
185
+ ## The Honest Limitations
186
+
187
+ This system doesn't solve everything.
188
+
189
+ It depends on the developer (or AI) actually following the workflows. If someone skips workflow 1 and starts coding without a test scope, the reviewer should catch it at workflow 4 — but the control is detective, not preventive.
190
+
191
+ The authenticated E2E tests (those requiring credentials) run only locally, not in CI. This means a subset of the test suite lacks independent verification. The mitigation is that the unauthenticated subset covers the majority of test cases, and the local evidence for the full suite is committed to the repository.
192
+
193
+ Periodic activities (penetration testing, disaster recovery testing) are defined in the schedule but require manual execution. They're not triggered by the pipeline. Someone has to remember to do them on schedule.
194
+
195
+ The AI regeneration protocol requires the developer to self-report when they regenerate a component. There's no automated detection of regeneration vs. incremental editing. The mitigation is the reviewer checklist, but this is a human control, not a technical one.
196
+
197
+ And the system is built for a single-developer or small-team context. Larger teams would need additional controls: more granular RBAC on who can approve which risk levels, automated test scope validation, and possibly automated evidence integrity checking.
198
+
199
+ ---
200
+
201
+ ## Getting Started
202
+
203
+ For teams wanting to implement this system:
204
+
205
+ 1. **Adopt Tier 1 as-is** — The Test Policy, Test Strategy, Test Architecture, and Periodic Security Review Schedule are designed to be used without modification. They're organizational standards, not project-specific.
206
+
207
+ 2. **Run the project setup workflow** — For each new project, execute `0-project-setup.md`. This creates the branch structure, CI pipeline, compliance directories, and project Test Plan from templates.
208
+
209
+ 3. **Follow the pipeline** — Every feature goes through workflows 1-5. The discipline comes from the workflow structure, not from individual memory.
210
+
211
+ 4. **Start with Low risk** — The first few requirements through the pipeline should be Low risk to validate that the process works end-to-end before the overhead of Medium/High risk documentation is introduced.
212
+
213
+ 5. **Let the reviewer calibrate** — The test scope verification in the PR checklist is where risk classification mistakes get caught. The first few reviews will calibrate what "appropriate testing depth" means for your specific project.
214
+
215
+ The complete template set — all 12 documents — is available and ready to be customized for any project using the placeholder markers in the Tier 2 templates.
216
+
217
+ ---
218
+
219
+ _This framework was developed iteratively through analysis of ISO 29119, ISO 27001, GDPR, SOC 2, and emerging AI governance guidance, then implemented as a practical workflow system designed for AI-assisted software development in a compliance context._
@@ -0,0 +1,410 @@
1
+ ---
2
+ description: One-time project setup — configure repository, CI pipeline, compliance structure, and tooling before workflows can run
3
+ ---
4
+
5
+ # Project Setup Guide
6
+
7
+ **Document Type:** Setup Guide | **Run Once:** At project start, before any workflow is executed
8
+
9
+ **Parent Documents:** Test Policy, Test Strategy, Test Architecture (all Tier 1, in devaudit/sdlc/files/)
10
+
11
+ ---
12
+
13
+ ## Purpose
14
+
15
+ This guide configures a new project so that the five pipeline workflows can run correctly. It sets up the repository structure, branch protection, CI pipeline (the independent verification gate), compliance directories, and local tooling.
16
+
17
+ **Run this guide once when starting a new project.** The pipeline workflows (1-5) assume this setup is complete.
18
+
19
+ ---
20
+
21
+ ## Fast path (recommended): `devaudit install`
22
+
23
+ The DevAudit CLI (`@metasession.co/devaudit-cli`) automates almost all of this guide. It writes `sdlc-config.json`, creates the DevAudit project, issues an API key, sets the GitHub secrets and variables, installs the hook framework, configures branch protection, and syncs the framework templates (CI workflows, hooks, scripts) — in roughly 30 seconds.
24
+
25
+ ```bash
26
+ npm install -g @metasession.co/devaudit-cli # requires Node >= 22
27
+ devaudit auth login # paste your PAT (issued at https://devaudit.metasession.co/settings/tokens)
28
+ devaudit install ../path/to/this-project
29
+ ```
30
+
31
+ The CLI ships the framework templates inside the package, so no DevAudit-Installer checkout is needed. If `sdlc-config.json` already exists in the target, `install` runs non-interactively from it and preserves your customisations (`app_env`, `build_env`, `e2e_*`, etc.). See [`docs/onboarding.md` in DevAudit-Installer](https://github.com/metasession-dev/DevAudit-Installer/blob/main/docs/onboarding.md) for the full walkthrough.
32
+
33
+ If you take the fast path, skip to Step 6 (Tier 1 docs reference) — Steps 1–5 are handled automatically. The remaining manual steps in this guide are still relevant: project-specific customisation of `INSTRUCTIONS.md` and `CLAUDE.md`, optional UAT-environment configuration, and verifying the first end-to-end release walks through Stages 1–5 cleanly.
34
+
35
+ The steps below are the **manual reference** — useful for understanding what the CLI does, or for environments where it can't run. Where a step produces a framework file (CI workflows, hooks, scripts), **copy the canonical template from the DevAudit-Installer repo (`sdlc/files/…`) and fill its placeholders** rather than authoring it by hand — a hand-written `ci.yml` routinely omits the DevAudit `register-release` / `upload-evidence` jobs and leaves releases unable to pass the compliance gates.
36
+
37
+ ---
38
+
39
+ ## Prerequisites
40
+
41
+ - GitHub repository created
42
+ - Hosting platform configured (Railway, Vercel, Fly, etc.) with auto-deploy from `main` (production)
43
+ - UAT environment configured (optional, opt-in by risk class — see Stage 3 Step 10)
44
+ - Local development environment working (app builds and runs)
45
+ - Stack toolchain: Node.js + npm (Node stack) OR Python 3.11+ + pip (Python stack)
46
+ - GitHub CLI (`gh`) installed and authenticated
47
+
48
+ ---
49
+
50
+ ## Step 1: Branch Structure
51
+
52
+ Create the permanent working branch:
53
+
54
+ ```bash
55
+ # Ensure main exists and is up to date
56
+ git checkout main
57
+ git pull origin main
58
+
59
+ # Create develop branch
60
+ git checkout -b develop
61
+ git push origin develop
62
+ ```
63
+
64
+ **Branch roles:**
65
+ - `main` — Production. Auto-deploys. Never commit directly.
66
+ - `develop` — All work happens here. Permanent, never deleted.
67
+
68
+ ---
69
+
70
+ ## Step 2: Branch Protection Rules
71
+
72
+ Configure in GitHub → Settings → Branches → Branch protection rules:
73
+
74
+ **`main` branch:**
75
+ - [x] Require a pull request before merging
76
+ - [x] Require approvals: 1 (adjust if team size requires more)
77
+ - [x] Require status checks to pass before merging
78
+ - Add these required checks after CI is configured (Step 4):
79
+ - `Quality Gates` (the consolidated CI job from `ci.yml` — TypeScript, SAST, dependency audit, E2E, build)
80
+ - `DevAudit Release Approval` (from `check-release-approval.yml` — blocks merge until the release is approved in DevAudit. Re-run the workflow after approving in DevAudit to turn the check green)
81
+ - [x] Require branches to be up to date before merging
82
+ - [x] Do not allow bypassing the above settings
83
+
84
+ **`develop` branch (optional but recommended):**
85
+ - [x] Require status checks to pass (push-to-develop checks)
86
+
87
+ ---
88
+
89
+ ## Step 3: Compliance Directory Structure
90
+
91
+ ```bash
92
+ # Create compliance directories
93
+ mkdir -p compliance/evidence/periodic/{sast-quarterly,dependency-audit,access-control,audit-log,pentest,dr-test,third-party}
94
+ mkdir -p compliance/pending-releases
95
+ mkdir -p compliance/approved-releases
96
+
97
+ # Create initial RTM
98
+ cat > compliance/RTM.md << 'EOF'
99
+ # Requirements Traceability Matrix
100
+
101
+ **Project:** [PROJECT NAME]
102
+ **Standard:** ISO/IEC/IEEE 29119-3:2021
103
+
104
+ ## Part A: Baseline Requirements
105
+
106
+ | ID | Requirement | Source | Status |
107
+ |---|---|---|---|
108
+
109
+ ## Part B: Change Request Traceability
110
+
111
+ | REQ-ID | Issue | Risk | Evidence | Status | Approver | Date |
112
+ |---|---|---|---|---|---|---|
113
+ EOF
114
+
115
+ # Commit
116
+ git add compliance/
117
+ git commit -m "compliance: initialize compliance directory structure"
118
+ git push origin develop
119
+ ```
120
+
121
+ ---
122
+
123
+ ## Step 4: CI Pipeline Configuration
124
+
125
+ This is the **independent verification gate**. Tests run locally during development (comprehensive, all tests). Tests run in CI during PR review (independent, tamper-resistant evidence produced by GitHub, not the developer).
126
+
127
+ ### What CI Must Run
128
+
129
+ | Pipeline | Trigger | Jobs | Purpose |
130
+ |---|---|---|---|
131
+ | CI | Push to `develop` | TypeScript + SAST + dependency audit + E2E + build | Quality gates + independent verification |
132
+ | Deploy | Merge to `main` | Auto-deploy to hosting platform | Production release |
133
+
134
+ PRs to `main` do not trigger a separate CI run. Branch protection required status checks ensure the commit already passed Quality Gates on the develop push. This avoids duplicate CI runs.
135
+
136
+ ### GitHub Actions Workflow File
137
+
138
+ > **Don't hand-author `ci.yml`.** `devaudit install` (and `devaudit update`)
139
+ > generate it from the canonical template; it is the load-bearing path for the
140
+ > compliance gates and drifts the moment it's retyped.
141
+
142
+ The generated `ci.yml` is a single `quality-gates` job (TypeScript + SAST +
143
+ dependency audit + E2E + build, all on every push to `develop`) **plus two
144
+ DevAudit jobs that are mandatory and easy to miss:**
145
+
146
+ - **`register-release`** — registers the release in DevAudit and syncs known
147
+ requirements from `RTM.md`, so the UAT gate can find the release.
148
+ - **`upload-evidence`** — uploads the gate results to DevAudit with the evidence
149
+ **categories the approval gate keys on**: `security_scan` (SAST + Dependency
150
+ Audit), `ci_pipeline` (E2E Tests), `test_report` (Playwright report / coverage).
151
+
152
+ ⚠️ **Without `upload-evidence`, releases can never pass the SAST Scan / Dependency
153
+ Audit / E2E Tests gates** — DevAudit derives gate pass/fail from uploaded evidence
154
+ categories, not from the GitHub check status. A `ci.yml` that only saves results
155
+ as `upload-artifact` artifacts produces a release that is permanently stuck in UAT
156
+ review with "Missing compliance gates". This is the single most common onboarding
157
+ failure; let the CLI generate the workflow.
158
+
159
+ **Manual install (fallback only):** copy the canonical template from the
160
+ DevAudit-Installer repo and fill its placeholders from `sdlc-config.json`:
161
+
162
+ ```bash
163
+ # `devaudit` here = a clone of the DevAudit-Installer repo, not the portal app.
164
+ cp path/to/DevAudit-Installer/sdlc/files/ci/ci.yml.template .github/workflows/ci.yml
165
+ # Then substitute {{PLACEHOLDERS}} ({{RUNNER}}, {{NODE_VERSION}}, {{SOURCE_DIRS}},
166
+ # {{DATABASE_*}}, {{APP_ENV}}, {{BUILD_ENV}}, {{E2E_*}}, {{PROJECT_SLUG}}, …) — the
167
+ # `devaudit update` command does this for you from sdlc-config.json.
168
+ ```
169
+
170
+ ### Customization Checklist
171
+
172
+ Customisation lives in **`sdlc-config.json`**, not in `ci.yml` (which is generated).
173
+ Confirm before the first install/sync:
174
+
175
+ - [ ] `node_version` matches your project
176
+ - [ ] `source_dirs` matches your source directory(ies) (e.g. `app/ lib/`)
177
+ - [ ] `database_service` / `database_image` / `database_env` match your DB (or empty for none)
178
+ - [ ] `app_env` + `build_env` carry every secret your app needs to start, as `${{ secrets.NAME }}`
179
+ - [ ] `e2e_start_command` + `e2e_project` are set
180
+ - [ ] `accepted_dep_risks` lists any documented, risk-registered audit acceptances
181
+
182
+ ### Verify CI Works
183
+
184
+ ```bash
185
+ # Commit the workflow file
186
+ git add .github/workflows/ci.yml
187
+ git commit -m "ci: configure CI pipeline with independent verification gates"
188
+ git push origin develop
189
+
190
+ # Create a test PR to verify all jobs pass
191
+ gh pr create --base main --head develop --title "ci: verify pipeline configuration" --body "Testing CI pipeline setup"
192
+
193
+ # Watch the checks run
194
+ gh pr checks
195
+ ```
196
+
197
+ Once all checks pass, go back to GitHub → Settings → Branches → `main` protection rules and add the required status check. The canonical workflow runs a single consolidated job, so add **`Quality Gates`** (and **`DevAudit Release Approval`** once that workflow is in place — see Additional Workflows below).
198
+
199
+ Now the PR **cannot be merged** unless the Quality Gates job passes. This is enforced by GitHub, not by the developer.
200
+
201
+ ### After CI Is Configured
202
+
203
+ The relationship between local testing and CI testing:
204
+
205
+ ```
206
+ Developer Machine (comprehensive) GitHub Actions (independent)
207
+ ───────────────────────────────── ──────────────────────────────
208
+ TypeScript check ──────────────────────→ TypeScript check
209
+ SAST scan (Semgrep) ───────────────────→ SAST scan (Semgrep)
210
+ Dependency audit ──────────────────────→ Dependency audit
211
+ E2E tests (ALL — e.g., 183) ──────────→ E2E tests (unauthenticated subset)
212
+ E2E tests (authenticated — local only) ✗ (credentials not in CI)
213
+ Post-deploy verification (local only) ✗ (runs after merge)
214
+
215
+ Evidence → compliance/evidence/REQ-XXX/ Evidence → uploaded to DevAudit
216
+ (comprehensive, developer-produced) (independent, tamper-resistant)
217
+ ```
218
+
219
+ Both are required. Local evidence proves comprehensive testing. CI evidence proves it independently. The `upload-evidence` job pushes the CI gate results **into DevAudit** (categorised `security_scan` / `ci_pipeline` / `test_report`) — GitHub artifacts alone are not evidence as far as the release gates are concerned.
220
+
221
+ ### Additional Workflows
222
+
223
+ These are generated for you by `devaudit install` / `devaudit update`. They come from the canonical templates in the DevAudit-Installer repo (`sdlc/files/ci/`):
224
+
225
+ **`check-release-approval.yml`** (workflow "Release Approval Gate") — release approval gate:
226
+ - Runs on PRs to `main` and `workflow_dispatch`
227
+ - Queries DevAudit for release approval status
228
+ - Blocks merge unless the release is approved
229
+ - Add `DevAudit Release Approval` (the job name) as a required status check on `main`
230
+
231
+ **`post-deploy-prod.yml`** — Production evidence capture:
232
+ - Runs on push to `main` (after merge)
233
+ - Waits for deployment, runs production smoke tests
234
+ - Uploads production evidence to DevAudit (`environment: production`)
235
+ - Marks the release as `released` in DevAudit
236
+
237
+ **Versioning convention:** Releases use date-based versions by default (`v2026.03.27`, or `v2026.03.27.2` for the second release on the same day). CI auto-creates releases in DevAudit when uploading evidence and auto-increments the sequence number if a release already exists for today. Update `PROJECT_SLUG` and `PRODUCTION_URL` in each template.
238
+
239
+ ---
240
+
241
+ ## Step 5: Local Tooling Setup (Required)
242
+
243
+ ### 5a. Security scanning tools
244
+
245
+ ```bash
246
+ # SAST
247
+ pip install semgrep
248
+ # or: npm install --save-dev semgrep
249
+
250
+ # Verify
251
+ semgrep --version
252
+ npm audit --help
253
+ ```
254
+
255
+ ### 5b. Playwright
256
+
257
+ ```bash
258
+ npx playwright install chromium
259
+ ```
260
+
261
+ ### 5c. Git hooks (required — enforces commit conventions and pre-push gates)
262
+
263
+ Install husky, commitlint, and lint-staged:
264
+
265
+ ```bash
266
+ npm install --save-dev husky @commitlint/cli @commitlint/config-conventional lint-staged
267
+ npx husky init
268
+ ```
269
+
270
+ Copy hook templates from DevAudit SDLC:
271
+
272
+ ```bash
273
+ # From your project root (adjust path to DevAudit)
274
+ # `devaudit install` bootstraps these for you. Manual fallback (path = a clone
275
+ # of the DevAudit-Installer repo). Hooks are stack-specific — swap `node` for `python`.
276
+ cp path/to/DevAudit-Installer/sdlc/files/stacks/node/hooks/commit-msg .husky/commit-msg
277
+ cp path/to/DevAudit-Installer/sdlc/files/stacks/node/hooks/pre-commit .husky/pre-commit
278
+ cp path/to/DevAudit-Installer/sdlc/files/stacks/node/hooks/pre-push .husky/pre-push
279
+ chmod +x .husky/commit-msg .husky/pre-commit .husky/pre-push
280
+
281
+ # Copy commitlint config
282
+ cp path/to/DevAudit-Installer/sdlc/files/stacks/node/hooks/commitlint.config.mjs commitlint.config.mjs
283
+ ```
284
+
285
+ Add lint-staged configuration to `package.json`:
286
+
287
+ ```json
288
+ {
289
+ "lint-staged": {
290
+ "*.{ts,tsx}": ["eslint --fix"],
291
+ "*.{ts,tsx,js,jsx,json,md}": ["prettier --write"]
292
+ }
293
+ }
294
+ ```
295
+
296
+ Add the prepare script so hooks install automatically for new clones:
297
+
298
+ ```bash
299
+ npm pkg set scripts.prepare="husky"
300
+ ```
301
+
302
+ ### 5d. JSDoc requirement check script (optional — for CI enforcement)
303
+
304
+ ```bash
305
+ cp path/to/DevAudit-Installer/sdlc/files/stacks/node/scripts/check-requirement-jsdoc.sh scripts/check-requirement-jsdoc.sh
306
+ chmod +x scripts/check-requirement-jsdoc.sh
307
+ ```
308
+
309
+ ### 5e. Verify hooks work
310
+
311
+ ```bash
312
+ # Test commitlint (should fail — missing type prefix)
313
+ echo "bad commit message" | npx commitlint
314
+ # Expected: error
315
+
316
+ # Test commitlint (should pass)
317
+ echo "feat: test commit message" | npx commitlint
318
+ # Expected: no errors (warnings about missing Ref/Co-Authored-By are OK)
319
+
320
+ # Test pre-push hook
321
+ npx tsc --noEmit
322
+ # Expected: passes (or shows real errors to fix)
323
+ ```
324
+
325
+ ---
326
+
327
+ ## Step 6: Create Project Test Plan
328
+
329
+ Copy `Test_Plan_TEMPLATE.md` from the DevAudit-Installer repo (`sdlc/files/_common/Test_Plan_TEMPLATE.md`) to `compliance/test-plan.md` and fill in:
330
+
331
+ - [ ] Project name and repository
332
+ - [ ] Stack and hosting details
333
+ - [ ] Production URL and health endpoint
334
+ - [ ] Database configuration
335
+ - [ ] Test suite counts and framework
336
+ - [ ] Entry/exit criteria with actual thresholds
337
+ - [ ] AI tool permissions for this project
338
+ - [ ] Disaster recovery targets (RTO/RPO)
339
+ - [ ] Penetration test scope
340
+
341
+ ```bash
342
+ git add compliance/test-plan.md
343
+ git commit -m "compliance: project test plan"
344
+ git push origin develop
345
+ ```
346
+
347
+ ---
348
+
349
+ ## Step 7: Verify Everything Works
350
+
351
+ Run through the complete pipeline once with a small test change:
352
+
353
+ ```bash
354
+ # 1. Plan (workflow 1)
355
+ # Create a test requirement in RTM
356
+
357
+ # 2. Implement (workflow 2)
358
+ # Make a small change, run all local gates
359
+
360
+ # 3. Compile evidence (workflow 3)
361
+ # Generate compliance artifacts
362
+
363
+ # 4. Submit for review (workflow 4)
364
+ # Create PR — verify all CI checks run and pass
365
+
366
+ # 5. Deploy (workflow 5)
367
+ # Merge, verify deployment, finalize compliance
368
+ ```
369
+
370
+ If any step fails, fix the configuration before starting real work.
371
+
372
+ ---
373
+
374
+ ## Setup Checklist
375
+
376
+ | Step | Status |
377
+ |---|---|
378
+ | Repository created | [ ] |
379
+ | `develop` branch created | [ ] |
380
+ | Production environment configured (auto-deploy from `main`) | [ ] |
381
+ | UAT environment configured (auto-deploy from `develop`) | [ ] |
382
+ | Branch protection configured | [ ] |
383
+ | Compliance directories created (including `periodic/` subdirs) | [ ] |
384
+ | RTM initialized | [ ] |
385
+ | CI workflow file created (`.github/workflows/ci.yml`) | [ ] |
386
+ | CI verified — all jobs pass on test PR | [ ] |
387
+ | Required status checks added to branch protection | [ ] |
388
+ | Local tooling installed (Semgrep, Playwright) | [ ] |
389
+ | Git hooks configured (Husky, Commitlint, lint-staged) | [ ] |
390
+ | Hook verification passed (commitlint, pre-push tsc) | [ ] |
391
+ | AI assistant SDLC rules configured (CLAUDE.md / .windsurfrules / .cursorrules) | [ ] |
392
+ | DevAudit evidence upload configured in CI | [ ] |
393
+ | Project Test Plan created | [ ] |
394
+ | End-to-end pipeline verified with test change | [ ] |
395
+
396
+ ---
397
+
398
+ ## What Happens Next
399
+
400
+ With setup complete, all development follows the five pipeline workflows:
401
+
402
+ ```
403
+ 1-plan-requirement.md → Define requirement, classify risk, generate test scope
404
+ 2-implement-and-test.md → Code, commit, run all local gates, auto-deploy to UAT
405
+ 3-compile-evidence.md → Gather evidence, verify on UAT, create release ticket
406
+ 4-submit-for-review.md → Create PR (CI runs independent verification)
407
+ 5-deploy-main.md → Merge, verify production deployment, finalize
408
+ ```
409
+
410
+ The CI pipeline runs automatically on every PR. The developer doesn't need to trigger it — GitHub handles it. The branch protection rules prevent merging if CI fails. The human reviewer sees both local evidence (in the compliance directory) and CI evidence (in the PR checks) before approving.