ronce-racine 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +90 -0
- package/LICENSE +21 -0
- package/README.md +281 -0
- package/agents/code-reviewer.md +33 -0
- package/agents/qa-tester.md +28 -0
- package/dist/hooks/README.md +182 -0
- package/dist/hooks/bash-npm-silent.mjs +81 -0
- package/dist/hooks/session-inject.mjs +53 -0
- package/dist/hooks/session-precompact.mjs +36 -0
- package/dist/hooks/session-writer.mjs +142 -0
- package/dist/hooks/skill-reminder.mjs +89 -0
- package/dist/hooks/truncate-bash-output.mjs +46 -0
- package/dist/hooks/truncate-output.mjs +88 -0
- package/dist/hooks/worktree-env-setup.mjs +123 -0
- package/dist/install.js +962 -0
- package/docs/adopting-a-repo.md +80 -0
- package/docs/architecture.md +76 -0
- package/docs/decisions.md +72 -0
- package/docs/developing.md +143 -0
- package/docs/evaluating-skills.md +41 -0
- package/docs/git-worktrees.md +62 -0
- package/docs/rules/clean-architecture-deps.md +66 -0
- package/docs/rules/commits.md +59 -0
- package/docs/rules/detection-gap-protocol.md +55 -0
- package/docs/rules/error-handling-discipline.md +62 -0
- package/docs/rules/minimal-code.md +58 -0
- package/docs/rules/no-raw-sql-interpolation.md +57 -0
- package/docs/rules/pre-commit-secret-detection.md +54 -0
- package/docs/rules/secure-logging.md +53 -0
- package/docs/rules/sql-migrations-discipline.md +58 -0
- package/docs/rules/subscription-cleanup.md +63 -0
- package/docs/rules/test-discipline.md +66 -0
- package/docs/rules/ui-states-complete.md +52 -0
- package/docs/templates/SKILL.template.md +66 -0
- package/docs/templates/rule.template.md +16 -0
- package/docs/writing-a-rule.md +43 -0
- package/docs/writing-a-skill.md +118 -0
- package/package.json +45 -0
- package/rules/clean-architecture-deps.md +20 -0
- package/rules/commits.md +16 -0
- package/rules/detection-gap-protocol.md +35 -0
- package/rules/error-handling-discipline.md +23 -0
- package/rules/minimal-code.md +32 -0
- package/rules/no-raw-sql-interpolation.md +20 -0
- package/rules/pre-commit-secret-detection.md +14 -0
- package/rules/secure-logging.md +16 -0
- package/rules/sql-migrations-discipline.md +18 -0
- package/rules/subscription-cleanup.md +22 -0
- package/rules/test-discipline.md +20 -0
- package/rules/ui-states-complete.md +23 -0
- package/scripts/subscription-leak-scan.ts +53 -0
- package/skills/adversarial-feature-challenge/README.md +65 -0
- package/skills/adversarial-feature-challenge/SKILL.md +96 -0
- package/skills/adversarial-feature-challenge/assets/fuzz-payloads.txt +60 -0
- package/skills/adversarial-feature-challenge/templates/challenge-report.md +42 -0
- package/skills/api-contract-versioning/README.md +62 -0
- package/skills/api-contract-versioning/SKILL.md +76 -0
- package/skills/api-contract-versioning/templates/consumer-inventory.md +29 -0
- package/skills/audit-architecture/README.md +56 -0
- package/skills/audit-architecture/SKILL.md +97 -0
- package/skills/audit-architecture/reference/ar-architecture.md +275 -0
- package/skills/audit-architecture/reference/di-disponibilite.md +252 -0
- package/skills/audit-architecture/reference/im-incidents.md +217 -0
- package/skills/audit-architecture/reference/pa-pannes.md +362 -0
- package/skills/audit-architecture/reference/re-recuperation.md +300 -0
- package/skills/audit-architecture/reference/sc-scalabilite.md +173 -0
- package/skills/audit-architecture/reference/su-support.md +180 -0
- package/skills/audit-ci-cd/README.md +54 -0
- package/skills/audit-ci-cd/SKILL.md +124 -0
- package/skills/audit-ci-cd/reference/artefacts.md +39 -0
- package/skills/audit-ci-cd/reference/deploiement.md +100 -0
- package/skills/audit-ci-cd/reference/dora.md +94 -0
- package/skills/audit-ci-cd/reference/iac.md +40 -0
- package/skills/audit-ci-cd/reference/pipeline-ci.md +36 -0
- package/skills/audit-ci-cd/reference/supply-chain.md +44 -0
- package/skills/audit-compliance/README.md +69 -0
- package/skills/audit-compliance/SKILL.md +176 -0
- package/skills/audit-compliance/reference/conformite-rgpd.md +295 -0
- package/skills/audit-compliance/reference/finops.md +318 -0
- package/skills/audit-compliance/reference/gouvernance-donnees.md +238 -0
- package/skills/audit-industrialisation/README.md +53 -0
- package/skills/audit-industrialisation/SKILL.md +251 -0
- package/skills/audit-observability/README.md +64 -0
- package/skills/audit-observability/SKILL.md +134 -0
- package/skills/audit-observability/reference/alerting.md +214 -0
- package/skills/audit-observability/reference/logging.md +114 -0
- package/skills/audit-observability/reference/metriques.md +166 -0
- package/skills/audit-observability/reference/traces-dashboards.md +90 -0
- package/skills/audit-performance-frontend/README.md +91 -0
- package/skills/audit-performance-frontend/SKILL.md +194 -0
- package/skills/audit-performance-frontend/reference/assets-cache-network.md +129 -0
- package/skills/audit-performance-frontend/reference/css-rendering-pipeline.md +154 -0
- package/skills/audit-performance-frontend/reference/html-critical-path.md +128 -0
- package/skills/audit-performance-frontend/reference/js-engine-runtime.md +280 -0
- package/skills/audit-performance-frontend/reference/memory-gc.md +126 -0
- package/skills/audit-performance-frontend/reference/profiling-tooling.md +98 -0
- package/skills/audit-quality/README.md +85 -0
- package/skills/audit-quality/SKILL.md +126 -0
- package/skills/audit-quality/reference/qa-devops.md +539 -0
- package/skills/audit-quality/reference/qu-qualite-donnees.md +402 -0
- package/skills/audit-report/README.md +72 -0
- package/skills/audit-report/SKILL.md +241 -0
- package/skills/audit-report/reference/scoring-model.md +45 -0
- package/skills/audit-security/README.md +72 -0
- package/skills/audit-security/SKILL.md +140 -0
- package/skills/audit-security/reference/auth-identite.md +98 -0
- package/skills/audit-security/reference/autorisation.md +66 -0
- package/skills/audit-security/reference/chiffrement-tls.md +97 -0
- package/skills/audit-security/reference/detection-patching.md +67 -0
- package/skills/audit-security/reference/incident-response.md +98 -0
- package/skills/audit-security/reference/reseau-conteneurs.md +72 -0
- package/skills/audit-security/reference/secrets.md +36 -0
- package/skills/audit-testing/README.md +82 -0
- package/skills/audit-testing/SKILL.md +131 -0
- package/skills/audit-testing/reference/golden-dataset.md +65 -0
- package/skills/audit-testing/reference/resilience-chaos.md +93 -0
- package/skills/audit-testing/reference/tests-contractuels.md +65 -0
- package/skills/audit-testing/reference/tests-e2e-flaky.md +67 -0
- package/skills/audit-testing/reference/tests-unit-integ-ci.md +125 -0
- package/skills/bug-ticket-root-cause/README.md +67 -0
- package/skills/bug-ticket-root-cause/SKILL.md +86 -0
- package/skills/bug-ticket-root-cause/templates/bug-ticket.md +43 -0
- package/skills/bug-triage-structured/README.md +63 -0
- package/skills/bug-triage-structured/SKILL.md +104 -0
- package/skills/bug-triage-structured/templates/bug-triage.md +38 -0
- package/skills/ci-pipeline-orchestration/README.md +55 -0
- package/skills/ci-pipeline-orchestration/SKILL.md +110 -0
- package/skills/commit-readiness-review/README.md +66 -0
- package/skills/commit-readiness-review/SKILL.md +82 -0
- package/skills/commit-readiness-review/scripts/precommit-scan.ts +81 -0
- package/skills/comprehensive-test-strategy/README.md +64 -0
- package/skills/comprehensive-test-strategy/SKILL.md +108 -0
- package/skills/comprehensive-test-strategy/reference/testing-advanced.md +85 -0
- package/skills/daily-workflow-optimization/README.md +56 -0
- package/skills/daily-workflow-optimization/SKILL.md +80 -0
- package/skills/database-schema-evolution/README.md +54 -0
- package/skills/database-schema-evolution/SKILL.md +107 -0
- package/skills/ddd-backend-implementation/README.md +63 -0
- package/skills/ddd-backend-implementation/SKILL.md +95 -0
- package/skills/design-system-component-lifecycle/README.md +56 -0
- package/skills/design-system-component-lifecycle/SKILL.md +97 -0
- package/skills/design-system-component-lifecycle/templates/component-contract.md +43 -0
- package/skills/detection-sweep/README.md +58 -0
- package/skills/detection-sweep/SKILL.md +93 -0
- package/skills/detection-sweep/scripts/sweep.ts +80 -0
- package/skills/domain-modeling-design/README.md +57 -0
- package/skills/domain-modeling-design/SKILL.md +83 -0
- package/skills/frontend-fullstack-implementation/README.md +60 -0
- package/skills/frontend-fullstack-implementation/SKILL.md +133 -0
- package/skills/frontend-spec-call-site-audit/README.md +66 -0
- package/skills/frontend-spec-call-site-audit/SKILL.md +82 -0
- package/skills/frontend-spec-call-site-audit/scripts/audit-entry-points.ts +79 -0
- package/skills/frontend-spec-call-site-audit/templates/ticket-frontend.md +32 -0
- package/skills/merge-request-review/README.md +60 -0
- package/skills/merge-request-review/SKILL.md +77 -0
- package/skills/merge-request-review/templates/mr-review-report.md +29 -0
- package/skills/production-incident-diagnostic/README.md +63 -0
- package/skills/production-incident-diagnostic/SKILL.md +103 -0
- package/skills/production-incident-diagnostic/templates/incident-report.md +33 -0
- package/skills/qa-session-intake/README.md +56 -0
- package/skills/qa-session-intake/SKILL.md +78 -0
- package/skills/recording-decisions/README.md +91 -0
- package/skills/recording-decisions/SKILL.md +84 -0
- package/skills/recurring-bug-root-cause/README.md +64 -0
- package/skills/recurring-bug-root-cause/SKILL.md +93 -0
- package/skills/recurring-bug-root-cause/scripts/detect-recurring-fixes.ts +46 -0
- package/skills/recurring-bug-root-cause/templates/postmortem-recidive.md +37 -0
- package/skills/refactoring-shared-component-api/README.md +50 -0
- package/skills/refactoring-shared-component-api/SKILL.md +84 -0
- package/skills/refactoring-shared-component-api/scripts/sweep-call-sites.ts +85 -0
- package/skills/refactoring-shared-component-api/templates/migration-checklist.md +19 -0
- package/skills/validating-features-end-to-end/README.md +50 -0
- package/skills/validating-features-end-to-end/SKILL.md +90 -0
- package/skills/visual-regression-check/README.md +51 -0
- package/skills/visual-regression-check/SKILL.md +89 -0
- package/skills/writing-robust-tests/README.md +53 -0
- package/skills/writing-robust-tests/SKILL.md +88 -0
- package/templates/.adopted.example +8 -0
- package/templates/anti-drift.gitlab-ci.yml +23 -0
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# `audit-ci-cd`
|
|
2
|
+
|
|
3
|
+
> Score a project's CI/CD and release-management maturity - pipelines, deployment, DORA metrics, IaC, artifacts, supply chain - on a 0-4 scale with prioritized, evidence-backed recommendations.
|
|
4
|
+
|
|
5
|
+
| | |
|
|
6
|
+
|---|---|
|
|
7
|
+
| **Type** | Skill (on-demand audit) |
|
|
8
|
+
| **Category** | `audit` |
|
|
9
|
+
| **Artifact** | [`SKILL.md`](SKILL.md) |
|
|
10
|
+
| **Tooling** | Bash (git, grep, ls) - read-only inspection; progressive-disclosure grids in [`reference/`](reference/) |
|
|
11
|
+
|
|
12
|
+
## What it is
|
|
13
|
+
|
|
14
|
+
`audit-ci-cd` is one domain audit in a family of maturity audits. It takes a repository and produces a structured assessment of how the team ships software: how the CI pipeline is built, how code reaches production, how fast and how safely that happens, and how the delivery chain is secured.
|
|
15
|
+
|
|
16
|
+
The domain is broken into **10 questions** (`ci-01` … `ci-10`) across 6 sections. Each question carries a criticality (`must` / `should` / `could`), a set of bash commands that gather objective evidence, an explicit checklist, and a 0-4 level rubric. The skill runs the commands, reads the pipeline files, scores each question with a confidence level, and emits a report with an overall (optionally weighted) score.
|
|
17
|
+
|
|
18
|
+
## Why it exists
|
|
19
|
+
|
|
20
|
+
CI/CD maturity is easy to assert and hard to prove. "We have a pipeline" says nothing about whether it caches dependencies, runs SAST, promotes a single immutable artifact between environments, or can roll back in under five minutes. This skill replaces impressions with evidence:
|
|
21
|
+
|
|
22
|
+
- **Objective first** - every question starts from commands (`git log`, `grep` over workflow files, `ls` of IaC directories), so a score is anchored to what is actually in the repo.
|
|
23
|
+
- **Honest about confidence** - DORA metrics in particular (`ci-03`/`ci-04`/`ci-05`) often can't be settled from the repo alone. The skill marks those `low` confidence and lists them as non-auditable rather than inventing a number.
|
|
24
|
+
- **Prioritized** - criticality weighting means a missing rollback procedure (`must`) outranks a missing SBOM (`could`) in the recommendations.
|
|
25
|
+
|
|
26
|
+
## When it triggers
|
|
27
|
+
|
|
28
|
+
Invoke it for a **domain-scoped** CI/CD audit. Trigger phrases: "audit ci", "audit pipeline", "audit deployment", "audit release".
|
|
29
|
+
|
|
30
|
+
For a **full multi-domain** project audit, use [`audit-industrialisation`](../audit-industrialisation/) instead - it orchestrates every domain audit (this one included) and consolidates the results. Do not invoke each domain skill by hand for a global audit.
|
|
31
|
+
|
|
32
|
+
## How it works
|
|
33
|
+
|
|
34
|
+
The audit starts objective: it detects and reads the CI pipeline files, then for each of the 10 questions (`ci-01` … `ci-10`) runs the verification commands from the matching `reference/` grid and inspects deployment configs and `package.json` scripts. Each question gets a level 0-4 with a confidence tag (high / medium / low) or N/A when the profile makes it irrelevant. DORA metrics (`ci-03`/`ci-04`/`ci-05`) are typically low-confidence and flagged non-auditable rather than guessed. Scores are averaged (and optionally criticality-weighted, `must`=3 / `should`=2 / `could`=1) and emitted through the standard report format with prioritized recommendations.
|
|
35
|
+
|
|
36
|
+
Full step-by-step protocol, section grids, and output format → [`SKILL.md`](SKILL.md).
|
|
37
|
+
|
|
38
|
+
## Worked example
|
|
39
|
+
|
|
40
|
+
> Auditing `acme-app`, a Node service with a GitHub Actions pipeline and a Helm chart, but no rollback runbook.
|
|
41
|
+
|
|
42
|
+
1. **Detect** - `ls` finds `.github/workflows/ci.yml`; it is read in full.
|
|
43
|
+
2. **ci-01** - the workflow has lint → build → test → publish, with `actions/cache` and a build matrix. Feedback ~8 min from the logs. Level **3**, confidence **high**.
|
|
44
|
+
3. **ci-02** - a `deploy` job runs `helm upgrade` with a rolling update, gated on a manual approval. No canary. Level **2**, confidence **high**.
|
|
45
|
+
4. **ci-09** - `grep` over `docs/`, `scripts/`, and the workflow finds no rollback procedure and no health-check-driven auto-revert. Level **1**, confidence **medium**. Because `ci-09` is a `must`, this drives the top recommendation.
|
|
46
|
+
5. **ci-03/04/05** - `git tag`/`git log` show weekly tags, but change failure rate can't be derived from the repo. Levels tentative, confidence **low**, listed under non-auditable items ("requires CI dashboard access").
|
|
47
|
+
6. **Report** - weighted overall score, with `[MUST] Document and automate a one-command rollback for acme-app` first in the recommendations.
|
|
48
|
+
|
|
49
|
+
## Related artifacts
|
|
50
|
+
|
|
51
|
+
- [`audit-industrialisation`](../audit-industrialisation/) - orchestrator that runs all domain audits and consolidates them; use it for a full audit.
|
|
52
|
+
- [`audit-report`](../audit-report/) - template and scoring rules for the consolidated report.
|
|
53
|
+
- [`audit-security`](../audit-security/) - application security audit; overlaps with `ci-10` on secrets and supply chain.
|
|
54
|
+
- [`audit-observability`](../audit-observability/) - logging, metrics, tracing, and alerting; complements deployment health and DORA signals.
|
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: audit-ci-cd
|
|
3
|
+
description: Audit CI/CD & Release Management. Use when auditing pipelines, deployment strategies, DORA metrics, IaC, artifact management, or supply chain security. Triggers on "audit ci", "audit pipeline", "audit déploiement", "audit deployment", "audit release".
|
|
4
|
+
version: 1.0.1
|
|
5
|
+
metadata:
|
|
6
|
+
last-reviewed: 2026-07-20
|
|
7
|
+
category: audit
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Audit CI/CD & Release Management
|
|
11
|
+
|
|
12
|
+
## When ME and not audit-industrialisation
|
|
13
|
+
|
|
14
|
+
- **ME** when: an audit scoped to this domain only
|
|
15
|
+
- **audit-industrialisation** instead if: a global multi-domain audit - it orchestrates every audit (including ME) and produces a consolidated report
|
|
16
|
+
|
|
17
|
+
> For a full project audit, use `audit-industrialisation` rather than invoking each skill separately.
|
|
18
|
+
|
|
19
|
+
## Overview
|
|
20
|
+
|
|
21
|
+
Audit a project's CI/CD maturity by analyzing pipelines, deployment strategies, DORA metrics, environment management, Infrastructure as Code, artifact management, rollback procedures, and software supply chain security. Produces a maturity score (0-4) per question and prioritized recommendations.
|
|
22
|
+
|
|
23
|
+
**10 questions** spread across 6 sections. The detailed grids (statement, analysis, checks, commands, levels 0-4) live in `reference/` - see [Grids by section](#grids-by-section).
|
|
24
|
+
|
|
25
|
+
Criticality levels:
|
|
26
|
+
- **must**: fundamental practice, absence = critical risk
|
|
27
|
+
- **should**: good practice, absence = moderate risk
|
|
28
|
+
- **could**: excellence, absence = room for improvement
|
|
29
|
+
|
|
30
|
+
## Score calculation
|
|
31
|
+
|
|
32
|
+
- Domain score = average of scored questions (exclude N/A)
|
|
33
|
+
- State "X questions scored out of Y total"
|
|
34
|
+
- A question is N/A if the project profile makes it non-applicable (e.g. a library with no deployment → ci-02/ci-06/ci-09 N/A)
|
|
35
|
+
- Criticality (used to prioritize remediation, NOT in the score - see `audit-report/reference/scoring-model.md`):
|
|
36
|
+
- **must**: ci-01, ci-02, ci-06, ci-09
|
|
37
|
+
- **should**: ci-07, ci-08, ci-10
|
|
38
|
+
- **could**: ci-03, ci-04, ci-05
|
|
39
|
+
|
|
40
|
+
## Audit protocol
|
|
41
|
+
|
|
42
|
+
1. **Detect the CI system**: run the ci-01 commands to find the pipeline files
|
|
43
|
+
2. **Read every pipeline file** found
|
|
44
|
+
3. **Run the verification commands** for each question ci-01 to ci-10
|
|
45
|
+
4. **Analyze each question** based on the command output and the file reading
|
|
46
|
+
5. **Look for deployment configs**: Dockerfile, docker-compose, Helm, Terraform, etc.
|
|
47
|
+
6. **Check the npm scripts in `package.json`** for deploy, release, etc. commands
|
|
48
|
+
7. **Evaluate the DORA metrics** (ci-03, ci-04, ci-05): these questions often require interviews or dashboard access for a high-confidence assessment
|
|
49
|
+
8. **Assign a level** (0-4) per question with justification and confidence level
|
|
50
|
+
9. **Mark N/A** the questions non-applicable to the project profile (e.g. no deployment → ci-02 N/A)
|
|
51
|
+
10. **List the non-auditable items** requiring an interview or external access
|
|
52
|
+
11. **Produce the report** in the standard format (see below)
|
|
53
|
+
|
|
54
|
+
Confidence levels:
|
|
55
|
+
- **high**: verified by an executed command or direct file reading
|
|
56
|
+
- **medium**: inferred from the structure/config but not verified in depth
|
|
57
|
+
- **low**: requires an interview or external access (CI logs, dashboard, team interview)
|
|
58
|
+
|
|
59
|
+
## Grids by section
|
|
60
|
+
|
|
61
|
+
Each file contains the statement, the items to analyze/check, the bash commands, and the level 0-4 tables.
|
|
62
|
+
|
|
63
|
+
| Section | Reference | Questions | Count |
|
|
64
|
+
|---------|-----------|-----------|-------|
|
|
65
|
+
| CI Pipeline | [reference/pipeline-ci.md](reference/pipeline-ci.md) | ci-01 | 1 |
|
|
66
|
+
| Deployment & Environments | [reference/deploiement.md](reference/deploiement.md) | ci-02, ci-06, ci-09 | 3 |
|
|
67
|
+
| DORA Metrics | [reference/dora.md](reference/dora.md) | ci-03, ci-04, ci-05 | 3 |
|
|
68
|
+
| Infrastructure as Code | [reference/iac.md](reference/iac.md) | ci-07 | 1 |
|
|
69
|
+
| Artifact Management | [reference/artefacts.md](reference/artefacts.md) | ci-08 | 1 |
|
|
70
|
+
| Supply Chain Security | [reference/supply-chain.md](reference/supply-chain.md) | ci-10 | 1 |
|
|
71
|
+
|
|
72
|
+
## Output format
|
|
73
|
+
|
|
74
|
+
```markdown
|
|
75
|
+
## CI/CD & Release Management - Overall score: X.X/4 (Y questions scored out of Z)
|
|
76
|
+
|
|
77
|
+
### Summary
|
|
78
|
+
[2-3 sentences summarizing CI/CD maturity]
|
|
79
|
+
|
|
80
|
+
### Detail per question
|
|
81
|
+
|
|
82
|
+
| Code | Question | Criticality | Level | Confidence | Justification |
|
|
83
|
+
|------|----------|-------------|-------|------------|---------------|
|
|
84
|
+
| ci-01 | CI pipeline architecture (build, lint, test, scan, artifacts) | must | X | high | ... |
|
|
85
|
+
| ci-02 | CD pipeline and deployment strategies | must | X | high | ... |
|
|
86
|
+
| ci-03 | Deployment frequency (DORA metric) | could | X | low | ... |
|
|
87
|
+
| ci-04 | Lead time for changes (DORA metric) | could | X | low | ... |
|
|
88
|
+
| ci-05 | Change failure rate (DORA metric) | could | X | low | ... |
|
|
89
|
+
| ci-06 | Environment management (dev, staging, preprod, prod) | must | X | high | ... |
|
|
90
|
+
| ci-07 | Infrastructure as Code (IaC) | should | X | medium | ... |
|
|
91
|
+
| ci-08 | Artifact management and registries | should | X | high | ... |
|
|
92
|
+
| ci-09 | Deployment rollback and recovery | must | X | low | ... |
|
|
93
|
+
| ci-10 | Pipeline security (Supply Chain Security) | should | X | high | ... |
|
|
94
|
+
|
|
95
|
+
### Strengths
|
|
96
|
+
- ...
|
|
97
|
+
|
|
98
|
+
### Weaknesses
|
|
99
|
+
- ...
|
|
100
|
+
|
|
101
|
+
### Recommendations
|
|
102
|
+
1. [MUST] ...
|
|
103
|
+
2. [SHOULD] ...
|
|
104
|
+
3. [COULD] ...
|
|
105
|
+
|
|
106
|
+
### Non-auditable items
|
|
107
|
+
- [ci-XX] Description - Reason (requires interview / CI dashboard access / production log access / ...)
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
## Exit condition
|
|
111
|
+
|
|
112
|
+
- [ ] CI system detected, pipeline files read
|
|
113
|
+
- [ ] All 10 questions evaluated (or justified N/A)
|
|
114
|
+
- [ ] Verification commands run for each scored question
|
|
115
|
+
- [ ] Level (0-4) + confidence + justification per question
|
|
116
|
+
- [ ] Domain score computed (average excluding N/A + weighted score, "X scored out of Y")
|
|
117
|
+
- [ ] Report produced in the output format
|
|
118
|
+
- [ ] Non-auditable items listed
|
|
119
|
+
|
|
120
|
+
## Changelog
|
|
121
|
+
|
|
122
|
+
- 1.0.1 (2026-07-20) - scoring aligned with scoring-model.md: flat mean only; criticality kept for remediation priority, weights removed from the score
|
|
123
|
+
|
|
124
|
+
- 1.0.0 (2026-06-19) - initial versioned release + state-of-the-art enrichment (routing, context, protocol, traps, exit condition); audit grids in reference/ (progressive disclosure)
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# Artifact Management section
|
|
2
|
+
|
|
3
|
+
Questions: ci-08 (Artifact management and registries).
|
|
4
|
+
|
|
5
|
+
## Table of contents
|
|
6
|
+
|
|
7
|
+
- [ci-08 - Artifact management and registries](#ci-08--artifact-management-and-registries--should)
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
### ci-08 - Artifact management and registries - `should`
|
|
12
|
+
|
|
13
|
+
**Analyze:** Docker registry, private npm registry, publish configs, signatures, SBOM
|
|
14
|
+
|
|
15
|
+
**Commands:**
|
|
16
|
+
- `grep -E "registry|artifact|publish|push|docker.*push" .gitlab-ci.yml .github/workflows/*.yml 2>/dev/null` → artifact publishing
|
|
17
|
+
- `git tag | head -5` → check tag-based versioning
|
|
18
|
+
- `grep -E "version|semver" package.json 2>/dev/null` → semantic versioning
|
|
19
|
+
- `grep -E "cosign\|sigstore\|sign\|sbom\|syft\|trivy" .gitlab-ci.yml .github/workflows/*.yml Dockerfile 2>/dev/null` → signing and scanning
|
|
20
|
+
- `grep -E "retention\|cleanup\|expire\|prune" .gitlab-ci.yml .github/workflows/*.yml 2>/dev/null` → retention policy
|
|
21
|
+
- `grep -E "slsa\|provenance\|attestation" .gitlab-ci.yml .github/workflows/*.yml 2>/dev/null` → SLSA provenance
|
|
22
|
+
|
|
23
|
+
**Check:**
|
|
24
|
+
- Centralized registry for artifacts (private preferably)
|
|
25
|
+
- Semantic versioning
|
|
26
|
+
- Immutable and signed artifacts (Cosign/Sigstore)
|
|
27
|
+
- SBOM generated automatically (Syft, Trivy)
|
|
28
|
+
- Retention policy (cleanup of old versions)
|
|
29
|
+
- Vulnerability scanning on images
|
|
30
|
+
- Provenance attestation (SLSA)
|
|
31
|
+
|
|
32
|
+
**Levels:**
|
|
33
|
+
| Level | Criteria |
|
|
34
|
+
|-------|----------|
|
|
35
|
+
| 0 | No artifact registry. Images built locally. Artifacts stored ad-hoc. |
|
|
36
|
+
| 1 | Basic artifact registry (Docker Hub, npm registry). Tagging by latest or build number. No retention policy. |
|
|
37
|
+
| 2 | Private registry with semantic versioning. Retention policy. Image vulnerability scanning. Single artifact promoted between environments. |
|
|
38
|
+
| 3 | Signed artifacts (Cosign/Sigstore). SBOM generated automatically for each build. Continuous registry scanning. Admission policy (only signed images are deployed). |
|
|
39
|
+
| 4 | Provenance attestations (SLSA Level 3+). SBOM integrated into the CVE response process (< 4h reaction). VEX (Vulnerability Exploitability eXchange). Reproducible hermetic builds. |
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
# Deployment & Environments section
|
|
2
|
+
|
|
3
|
+
Questions: ci-02 (CD pipeline), ci-06 (Environment management), ci-09 (Rollback & recovery).
|
|
4
|
+
|
|
5
|
+
## Table of contents
|
|
6
|
+
|
|
7
|
+
- [ci-02 - CD pipeline and deployment strategies](#ci-02--cd-pipeline-and-deployment-strategies--must)
|
|
8
|
+
- [ci-06 - Environment management (dev, staging, preprod, prod)](#ci-06--environment-management-dev-staging-preprod-prod--must)
|
|
9
|
+
- [ci-09 - Deployment rollback and recovery](#ci-09--deployment-rollback-and-recovery--must)
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
### ci-02 - CD pipeline and deployment strategies - `must`
|
|
14
|
+
|
|
15
|
+
**Analyze:** Deployment scripts, Kubernetes configs (Helm charts, kustomize), Terraform/Pulumi, docker-compose, ArgoCD/Flux configs
|
|
16
|
+
|
|
17
|
+
**Commands:**
|
|
18
|
+
- `grep -E "deploy|release|promote|deliver" .gitlab-ci.yml .github/workflows/*.yml 2>/dev/null` → detect deployment steps
|
|
19
|
+
- `ls Dockerfile docker-compose*.yml 2>/dev/null` → detect containerization
|
|
20
|
+
- `ls -d helm/ charts/ kustomize/ argocd/ 2>/dev/null` → detect K8s orchestration
|
|
21
|
+
- `grep -E "blue.green|canary|rolling|progressive" .gitlab-ci.yml .github/workflows/*.yml helm/*/values*.yml 2>/dev/null` → identify the deployment strategy
|
|
22
|
+
- `grep -E "approval|manual|when:.*manual" .gitlab-ci.yml .github/workflows/*.yml 2>/dev/null` → check the approval gates
|
|
23
|
+
- `grep -E "feature.flag|unleash|launchdarkly|flipper" package.json src/ 2>/dev/null` → detect feature flags
|
|
24
|
+
|
|
25
|
+
**Check:**
|
|
26
|
+
- Defined deployment strategy (rolling, blue-green, canary)
|
|
27
|
+
- Artifact promotion between environments (no rebuild)
|
|
28
|
+
- Environment separation (dev, staging, preprod, prod)
|
|
29
|
+
- Approval gates before production
|
|
30
|
+
- Feature flags for deploy/release decoupling
|
|
31
|
+
- GitOps with automatic reconciliation (ArgoCD, Flux)
|
|
32
|
+
|
|
33
|
+
**Levels:**
|
|
34
|
+
| Level | Criteria |
|
|
35
|
+
|-------|----------|
|
|
36
|
+
| 0 | Manual deployments (SSH, FTP, file copy). No automation. |
|
|
37
|
+
| 1 | Semi-automated deployment scripts. All-or-nothing deployment. Planned downtime during production releases. |
|
|
38
|
+
| 2 | Automated CD pipeline with rolling updates. Zero-downtime deployment. Deployment triggered manually after validation. |
|
|
39
|
+
| 3 | Automated canary or blue-green deployment. Feature flags to decouple deployment and activation. Automatic promotion between environments. |
|
|
40
|
+
| 4 | GitOps with automatic reconciliation. Automated canary analysis (metrics + auto rollback). Feature flags with advanced targeting (% users, segments). Continuous deployment to production several times a day. |
|
|
41
|
+
|
|
42
|
+
---
|
|
43
|
+
|
|
44
|
+
### ci-06 - Environment management (dev, staging, preprod, prod) - `must`
|
|
45
|
+
|
|
46
|
+
**Analyze:** docker-compose, environment configs, env variables, provisioning scripts
|
|
47
|
+
|
|
48
|
+
**Commands:**
|
|
49
|
+
- `grep -ri "staging\|preprod\|dev\|prod\|review" .gitlab-ci.yml docker-compose*.yml .github/workflows/*.yml 2>/dev/null` → identify the environments
|
|
50
|
+
- `ls .env .env.* env/ environments/ 2>/dev/null` → environment files
|
|
51
|
+
- `grep -ri "vault\|secret.manager\|sealed.secret\|sops\|age" .gitlab-ci.yml .github/workflows/*.yml 2>/dev/null` → secret management
|
|
52
|
+
- `grep -ri "environment:" .gitlab-ci.yml .github/workflows/*.yml 2>/dev/null` → environments declared in CI
|
|
53
|
+
- `grep -ri "preview\|ephemeral\|dynamic" .gitlab-ci.yml .github/workflows/*.yml 2>/dev/null` → ephemeral environments
|
|
54
|
+
|
|
55
|
+
**Check:**
|
|
56
|
+
- Separate environments (dev, staging, preprod, prod)
|
|
57
|
+
- Parity between environments (same stack, same config except secrets)
|
|
58
|
+
- Environment variable management (vault, secrets manager vs. hardcoded .env)
|
|
59
|
+
- Ephemeral environments for PRs (preview deployments)
|
|
60
|
+
- Self-service for developers (environment creation without ops)
|
|
61
|
+
|
|
62
|
+
**Levels:**
|
|
63
|
+
| Level | Criteria |
|
|
64
|
+
|-------|----------|
|
|
65
|
+
| 0 | A single environment (prod) or very different environments. Manual configuration. No staging. |
|
|
66
|
+
| 1 | Distinct dev and prod environments but with significant differences. Manual configuration by documentation. |
|
|
67
|
+
| 2 | dev, staging, prod environments with good parity. Configuration managed through environment variables. Semi-automated provisioning. |
|
|
68
|
+
| 3 | Full dev/staging/prod parity. Environments defined as code. Drift detection. Preview environments per PR. |
|
|
69
|
+
| 4 | Ephemeral environments on demand. Full self-service. Parity verified automatically. Environments created and destroyed in < 10 min. |
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
### ci-09 - Deployment rollback and recovery - `must`
|
|
74
|
+
|
|
75
|
+
**Analyze:** Documented rollback procedures, rollback scripts, health check configs, database migrations
|
|
76
|
+
|
|
77
|
+
**Commands:**
|
|
78
|
+
- `grep -ri "rollback\|revert\|previous\|recovery" docs/ scripts/ 2>/dev/null` → documented rollback procedures
|
|
79
|
+
- `grep -ri "health.check\|readiness\|liveness\|startup.probe" helm/ k8s/ docker-compose*.yml Dockerfile 2>/dev/null` → health checks
|
|
80
|
+
- `grep -ri "rollback\|undo\|restore" .gitlab-ci.yml .github/workflows/*.yml Makefile 2>/dev/null` → automated rollback
|
|
81
|
+
- `grep -ri "migration.*down\|reversible\|revert" db/ migrations/ database/ 2>/dev/null | head -10` → reversible migrations
|
|
82
|
+
- `grep -ri "feature.flag\|kill.switch\|toggle" src/ 2>/dev/null | head -5` → feature flags kill switch
|
|
83
|
+
- `grep -ri "expand.*contract\|backward.*compat" docs/ migrations/ 2>/dev/null` → expand/contract pattern
|
|
84
|
+
|
|
85
|
+
**Check:**
|
|
86
|
+
- Documented and tested rollback procedure
|
|
87
|
+
- Automatic rollback on health check failure or metric degradation
|
|
88
|
+
- Rollback time < 5 min
|
|
89
|
+
- Database: reversible migrations or expand/contract pattern
|
|
90
|
+
- Feature flags kill switch for instant deactivation
|
|
91
|
+
- Immutable artifacts (redeploy version N-1 with no rebuild)
|
|
92
|
+
|
|
93
|
+
**Levels:**
|
|
94
|
+
| Level | Criteria |
|
|
95
|
+
|-------|----------|
|
|
96
|
+
| 0 | No rollback procedure. Forward-fix only (hotfix). Recovery time > 4h. |
|
|
97
|
+
| 1 | Documented manual rollback. Procedure known but rarely tested. Non-reversible database migrations. |
|
|
98
|
+
| 2 | One-command automated rollback (redeploy version N-1). Reversible migrations. Recovery time < 30 min. |
|
|
99
|
+
| 3 | Automatic rollback triggered by smoke tests or SLO degradation. Feature flags kill switch. Expand/contract pattern for migrations. Recovery < 10 min. |
|
|
100
|
+
| 4 | Automatic rollback with canary analysis. Zero-downtime rollback. All migrations backward-compatible by design. Recovery < 2 min. Chaos engineering on rollback procedures. |
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# DORA Metrics section
|
|
2
|
+
|
|
3
|
+
Questions: ci-03 (Deployment frequency), ci-04 (Lead time for changes), ci-05 (Change failure rate).
|
|
4
|
+
|
|
5
|
+
## Table of contents
|
|
6
|
+
|
|
7
|
+
- [ci-03 - Deployment frequency (DORA metric)](#ci-03--deployment-frequency-dora-metric--could)
|
|
8
|
+
- [ci-04 - Lead time for changes (DORA metric)](#ci-04--lead-time-for-changes-dora-metric--could)
|
|
9
|
+
- [ci-05 - Change failure rate (DORA metric)](#ci-05--change-failure-rate-dora-metric--could)
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
### ci-03 - Deployment frequency (DORA metric) - `could`
|
|
14
|
+
|
|
15
|
+
**Analyze:** Deployment frequency via git log/tags, release history, CI/CD dashboards
|
|
16
|
+
|
|
17
|
+
**Commands:**
|
|
18
|
+
- `git tag --sort=-creatordate | head -20` → release frequency (dates of the latest tags)
|
|
19
|
+
- `git log --oneline --since="30 days" | wc -l` → recent activity
|
|
20
|
+
- `git log --oneline --since="30 days" --grep="deploy\|release" | wc -l` → deployment frequency
|
|
21
|
+
- `git log --merges --oneline --since="90 days" | head -20` → merge cadence
|
|
22
|
+
- `git log --format="%ad" --date=short --tags --since="180 days" | sort -u | head -30` → release dates over 6 months
|
|
23
|
+
- `grep -ri "dora\|deployment.frequency" docs/ 2>/dev/null` → documented metrics
|
|
24
|
+
|
|
25
|
+
**Check:**
|
|
26
|
+
- Deployment frequency measured and known
|
|
27
|
+
- Trend (improvement or degradation over recent months)
|
|
28
|
+
- Identified obstacles to a higher frequency
|
|
29
|
+
- Trunk-based development vs. long-lived branches (impacts frequency)
|
|
30
|
+
|
|
31
|
+
**Levels:**
|
|
32
|
+
| Level | Criteria |
|
|
33
|
+
|-------|----------|
|
|
34
|
+
| 0 | Rare deployments (< 1 per month). Releases planned far in advance. |
|
|
35
|
+
| 1 | Monthly or bi-monthly deployment. Heavy release process with manual coordination. |
|
|
36
|
+
| 2 | Weekly deployment. Documented and repeatable process. Defined deployment windows. |
|
|
37
|
+
| 3 | Daily or several times a week. On demand. Trunk-based development. Small batches. |
|
|
38
|
+
| 4 | Continuous deployment multiple times a day. Every validated commit can reach production automatically. Frequency measured and continuously optimized. |
|
|
39
|
+
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
### ci-04 - Lead time for changes (DORA metric) - `could`
|
|
43
|
+
|
|
44
|
+
**Analyze:** Time between the first commit and production deployment, review duration, CI pipeline duration
|
|
45
|
+
|
|
46
|
+
**Commands:**
|
|
47
|
+
- `git log --merges --format="%H %ad" --date=iso --since="90 days" | head -20` → recent merge dates
|
|
48
|
+
- `grep -ri "lead.time\|cycle.time\|value.stream" docs/ 2>/dev/null` → documented metrics
|
|
49
|
+
- `grep -E "timeout|max.*time|deadline" .github/workflows/*.yml .gitlab-ci.yml 2>/dev/null` → pipeline timeouts
|
|
50
|
+
- `git log --format="%ae %ad" --date=iso --since="30 days" | head -20` → contributor activity
|
|
51
|
+
|
|
52
|
+
**Check:**
|
|
53
|
+
- Lead time measured (from the first commit to production deployment)
|
|
54
|
+
- Lead time breakdown (dev, review, CI, deployment wait, deployment)
|
|
55
|
+
- Bottlenecks identified (slow review, long pipeline, release window)
|
|
56
|
+
- Value stream mapping performed
|
|
57
|
+
- Reasonably sized PRs (< 400 lines)
|
|
58
|
+
|
|
59
|
+
**Levels:**
|
|
60
|
+
| Level | Criteria |
|
|
61
|
+
|-------|----------|
|
|
62
|
+
| 0 | Lead time not measured. Estimated at several weeks or months. |
|
|
63
|
+
| 1 | Lead time measured informally. Between 1 week and 1 month. Bottlenecks identified but not addressed. |
|
|
64
|
+
| 2 | Lead time between 1 day and 1 week. Value stream mapping performed. Main waits reduced. |
|
|
65
|
+
| 3 | Lead time < 1 day. Automated quality gates. Small PRs reviewed quickly (< 4h). Deployment on the same day as the merge. |
|
|
66
|
+
| 4 | Lead time < 1 hour. Continuous deployment. Trunk-based development. Zero manual step between commit and production. Metric tracked in real time. |
|
|
67
|
+
|
|
68
|
+
---
|
|
69
|
+
|
|
70
|
+
### ci-05 - Change failure rate (DORA metric) - `could`
|
|
71
|
+
|
|
72
|
+
**Analyze:** History of post-deployment incidents, rollbacks, hotfixes, post-mortems
|
|
73
|
+
|
|
74
|
+
**Commands:**
|
|
75
|
+
- `git log --oneline --since="90 days" --grep="hotfix\|revert\|rollback\|fix.*deploy\|fix.*prod" | head -20` → recent hotfixes and reverts
|
|
76
|
+
- `git log --oneline --since="90 days" --grep="revert" | wc -l` → number of reverts
|
|
77
|
+
- `grep -ri "change.failure\|failure.rate\|post.mortem\|postmortem\|incident" docs/ 2>/dev/null` → documented metrics and process
|
|
78
|
+
- `grep -ri "slo\|sli\|error.budget" docs/ src/ 2>/dev/null` → defined SLOs
|
|
79
|
+
|
|
80
|
+
**Check:**
|
|
81
|
+
- Change failure rate measured (% of deployments causing an incident)
|
|
82
|
+
- Clear definition of what constitutes a "failure" (P1/P2 incident, rollback, hotfix)
|
|
83
|
+
- Blameless post-mortems performed after incidents
|
|
84
|
+
- Post-deployment smoke tests
|
|
85
|
+
- Correlation between quality metrics and failure rate
|
|
86
|
+
|
|
87
|
+
**Levels:**
|
|
88
|
+
| Level | Criteria |
|
|
89
|
+
|-------|----------|
|
|
90
|
+
| 0 | Change failure rate not measured. Frequent post-deployment incidents with no tracking. |
|
|
91
|
+
| 1 | Rate measured manually. Above 30%. No systematic root-cause analysis. |
|
|
92
|
+
| 2 | Rate measured automatically (15-30%). Post-mortems performed for major incidents. Corrective actions identified. |
|
|
93
|
+
| 3 | Rate between 5% and 15%. Systematic blameless post-mortems. Post-deployment smoke tests. Canary releases to reduce the blast radius. |
|
|
94
|
+
| 4 | Rate < 5%. Predictive analysis of deployment risks. Automated canary analysis. Documented continuous improvement. Correlation between quality metrics and failure rate. |
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
# Infrastructure as Code section
|
|
2
|
+
|
|
3
|
+
Questions: ci-07 (Infrastructure as Code).
|
|
4
|
+
|
|
5
|
+
## Table of contents
|
|
6
|
+
|
|
7
|
+
- [ci-07 - Infrastructure as Code (IaC)](#ci-07--infrastructure-as-code-iac--should)
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
### ci-07 - Infrastructure as Code (IaC) - `should`
|
|
12
|
+
|
|
13
|
+
**Analyze:** Terraform, Pulumi, CloudFormation, Ansible, CDK, Dockerfile, docker-compose files
|
|
14
|
+
|
|
15
|
+
**Commands:**
|
|
16
|
+
- `ls terraform/ pulumi/ cdk/ ansible/ cloudformation/ 2>/dev/null` → detect the IaC tools
|
|
17
|
+
- `ls *.tf *.tfvars 2>/dev/null` → Terraform files at the root
|
|
18
|
+
- `find . -maxdepth 3 -name "*.tf" -o -name "main.tf" 2>/dev/null | head -10` → Terraform in the project
|
|
19
|
+
- `ls Dockerfile docker-compose*.yml 2>/dev/null` → containerization
|
|
20
|
+
- `grep -ri "remote.*backend\|terraform.*backend\|state" *.tf terraform/*.tf 2>/dev/null` → remote state
|
|
21
|
+
- `grep -ri "module\|source.*=" *.tf terraform/*.tf 2>/dev/null | head -5` → reusable modules
|
|
22
|
+
- `grep -ri "checkov\|tfsec\|sentinel\|opa\|policy" .gitlab-ci.yml .github/workflows/*.yml 2>/dev/null` → Policy as Code
|
|
23
|
+
|
|
24
|
+
**Check:**
|
|
25
|
+
- Infrastructure defined as code (no manual cloud console config)
|
|
26
|
+
- IaC versioning (same repo or dedicated repo)
|
|
27
|
+
- State management (remote state, locking)
|
|
28
|
+
- Separate plan/apply with review
|
|
29
|
+
- Reusable modules
|
|
30
|
+
- Drift detection and alerting
|
|
31
|
+
- Policy as Code (OPA, Sentinel, Checkov)
|
|
32
|
+
|
|
33
|
+
**Levels:**
|
|
34
|
+
| Level | Criteria |
|
|
35
|
+
|-------|----------|
|
|
36
|
+
| 0 | Infrastructure managed manually via cloud consoles or SSH. No traceability of changes. |
|
|
37
|
+
| 1 | Basic provisioning scripts (shell, CLI). Partially documented. No systematic versioning. |
|
|
38
|
+
| 2 | IaC with Terraform/Pulumi/CDK for the core infrastructure. Code versioned in Git. Plan/Apply with review. Shared state (remote state). |
|
|
39
|
+
| 3 | IaC for all the infrastructure. GitOps with automatic reconciliation. Drift detection with alerting. Reusable modules. Policy as Code for validation. |
|
|
40
|
+
| 4 | Full IaC with automated tests (Terratest). Auto-remediated drift. Self-service infrastructure via a catalog. Compliance as Code. Rebuild time < 1h. |
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# CI Pipeline section
|
|
2
|
+
|
|
3
|
+
Questions: ci-01 (CI pipeline architecture).
|
|
4
|
+
|
|
5
|
+
## Table of contents
|
|
6
|
+
|
|
7
|
+
- [ci-01 - CI pipeline architecture (build, lint, test, scan, artifacts)](#ci-01--ci-pipeline-architecture-build-lint-test-scan-artifacts--must)
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
### ci-01 - CI pipeline architecture (build, lint, test, scan, artifacts) - `must`
|
|
12
|
+
|
|
13
|
+
**Analyze:** `.github/workflows/*.yml`, `.gitlab-ci.yml`, `Jenkinsfile`, `bitbucket-pipelines.yml`, `.circleci/config.yml` files
|
|
14
|
+
|
|
15
|
+
**Commands:**
|
|
16
|
+
- `ls .github/workflows/*.yml .gitlab-ci.yml Jenkinsfile bitbucket-pipelines.yml .circleci/config.yml 2>/dev/null` → detect the CI system
|
|
17
|
+
- `grep -E "lint|test|sast|scan|build" .gitlab-ci.yml .github/workflows/*.yml 2>/dev/null` → identify the stages
|
|
18
|
+
- `grep -E "cache|artifacts|save_cache|restore_cache" .gitlab-ci.yml .github/workflows/*.yml 2>/dev/null` → check the caching
|
|
19
|
+
- `grep -E "matrix|strategy" .github/workflows/*.yml 2>/dev/null` → detect matrix builds
|
|
20
|
+
- `grep -E "parallel|needs:|dependencies:" .gitlab-ci.yml 2>/dev/null` → check parallelization
|
|
21
|
+
|
|
22
|
+
**Check:**
|
|
23
|
+
- Ordered stages (fail-fast): lint → build → unit tests → integration tests → SAST → artifact publish
|
|
24
|
+
- Parallelization of independent steps
|
|
25
|
+
- Dependency caching (node_modules, .m2, pip cache, Docker layers)
|
|
26
|
+
- Matrix builds (multi-OS, multi-version)
|
|
27
|
+
- Feedback time < 10 min (check durations if the logs are accessible)
|
|
28
|
+
|
|
29
|
+
**Levels:**
|
|
30
|
+
| Level | Criteria |
|
|
31
|
+
|-------|----------|
|
|
32
|
+
| 0 | No CI pipeline. Manual builds on the developer machine. No standardization. |
|
|
33
|
+
| 1 | Basic CI pipeline: build + a few unit tests. Triggered manually or on the main branch only. |
|
|
34
|
+
| 2 | Structured CI pipeline on every PR: lint, build, unit tests, artifact publish. Dependency caching. Feedback in < 15 min. |
|
|
35
|
+
| 3 | Full multi-stage pipeline: lint, build, unit + integration tests, SAST, signed artifact generation. Parallelization. Matrix builds. Feedback in < 10 min. |
|
|
36
|
+
| 4 | Optimized pipeline with predictive test selection, distributed builds, feedback in < 5 min. Reusable Pipeline as Code (shared templates). Pipeline reliability metrics (flaky rate < 1%). |
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
# Supply Chain Security section
|
|
2
|
+
|
|
3
|
+
Questions: ci-10 (Pipeline security / Supply Chain Security).
|
|
4
|
+
|
|
5
|
+
## Table of contents
|
|
6
|
+
|
|
7
|
+
- [ci-10 - Pipeline security (Supply Chain Security)](#ci-10--pipeline-security-supply-chain-security--should)
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
### ci-10 - Pipeline security (Supply Chain Security) - `should`
|
|
12
|
+
|
|
13
|
+
**Analyze:** Actions/plugins used in the pipeline, permissions, secrets management in CI, SLSA compliance
|
|
14
|
+
|
|
15
|
+
**Commands:**
|
|
16
|
+
- `grep -E "@v[0-9]|@latest|@main|@master" .github/workflows/*.yml 2>/dev/null` → actions pinned or not
|
|
17
|
+
- `grep -E "@[a-f0-9]{40}" .github/workflows/*.yml 2>/dev/null` → actions pinned by hash (good practice)
|
|
18
|
+
- `ls .github/dependabot.yml renovate.json .renovaterc 2>/dev/null` → dependency management
|
|
19
|
+
- `grep -E "permissions:|GITHUB_TOKEN" .github/workflows/*.yml 2>/dev/null` → minimal permissions
|
|
20
|
+
- `grep -ri "OIDC\|workload.identity\|assume.role" .github/workflows/*.yml .gitlab-ci.yml 2>/dev/null` → OIDC for cloud access
|
|
21
|
+
- `grep -rn "password\|token\|secret\|api.key" .gitlab-ci.yml .github/workflows/*.yml 2>/dev/null | grep -v '\${{' | grep -v '\$CI_' | head -5` → hardcoded secrets (anti-pattern)
|
|
22
|
+
- `grep -ri "slsa\|provenance\|hermetic\|reproducible" .gitlab-ci.yml .github/workflows/*.yml 2>/dev/null` → SLSA compliance
|
|
23
|
+
- `grep -ri "gitsign\|signed.commit\|gpg\|ssh.*sign" .github/workflows/*.yml .gitlab-ci.yml 2>/dev/null` → signed commits
|
|
24
|
+
- `grep -ri "scorecard\|openssf" .github/workflows/*.yml 2>/dev/null` → OpenSSF Scorecard
|
|
25
|
+
|
|
26
|
+
**Check:**
|
|
27
|
+
- Actions/plugins pinned by hash (not `@latest`)
|
|
28
|
+
- Secrets injected via vault/CI secrets (not hardcoded)
|
|
29
|
+
- Minimal permissions (GITHUB_TOKEN scoped)
|
|
30
|
+
- Deployment audit trail
|
|
31
|
+
- Dependabot/Renovate configured
|
|
32
|
+
- Signed commits required (GPG, SSH signing, Gitsign)
|
|
33
|
+
- SLSA provenance of artifacts
|
|
34
|
+
- Ephemeral runners (no persistence between builds)
|
|
35
|
+
- Strict branch protection
|
|
36
|
+
|
|
37
|
+
**Levels:**
|
|
38
|
+
| Level | Criteria |
|
|
39
|
+
|-------|----------|
|
|
40
|
+
| 0 | No security measure on the pipeline. Shared runners without isolation. Secrets in cleartext in the CI variables. |
|
|
41
|
+
| 1 | Secrets in a dedicated manager (Vault, CI secrets). Dedicated runners. Dependency lockfiles present but not verified. |
|
|
42
|
+
| 2 | Secret scanning in CI (GitLeaks, TruffleHog). SAST integrated. Signed commits required. Dependency pinning with hash verification. Least privilege on the runners. |
|
|
43
|
+
| 3 | SLSA Level 2: artifact provenance generated. Ephemeral runners. Strict branch protection. DAST in staging. Full pipeline audit trail. OpenSSF Scorecard > 7. |
|
|
44
|
+
| 4 | SLSA Level 3+: hermetic and reproducible builds. Attestations verified at admission (Kyverno, OPA Gatekeeper). Supply chain policy enforcement. Signed commits + signed artifacts + signed SBOM. Zero detectable hardcoded secret. |
|
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
# `audit-compliance`
|
|
2
|
+
|
|
3
|
+
> Score a project's compliance, data-governance and FinOps maturity - GDPR, retention, right to erasure, data ownership, cloud cost control - on a 0-4 scale across 29 questions, with evidence gathered from the code itself.
|
|
4
|
+
|
|
5
|
+
| | |
|
|
6
|
+
|---|---|
|
|
7
|
+
| **Type** | Skill (on-demand audit) |
|
|
8
|
+
| **Category** | `audit` |
|
|
9
|
+
| **Artifact** | [`SKILL.md`](SKILL.md) |
|
|
10
|
+
| **Tooling** | Grep/find verification commands; progressive-disclosure grids in [`reference/`](reference/) |
|
|
11
|
+
|
|
12
|
+
## What it is
|
|
13
|
+
|
|
14
|
+
`audit-compliance` is a structured maturity assessment spanning three related domains that all revolve around **data you are legally and financially accountable for**:
|
|
15
|
+
|
|
16
|
+
- **CO - Compliance & Regulatory** (10 questions): GDPR minimization, anonymization, retention, DPIA, right to erasure, deletion logging.
|
|
17
|
+
- **DG - Data Governance** (8 questions): ownership, catalog, migrations, data contracts, quality, sensitive-data classification, lifecycle, Data Mesh.
|
|
18
|
+
- **FI - FinOps** (11 questions): cost monitoring, caching, batching, per-customer attribution, abuse detection, cloud-waste elimination, reservations.
|
|
19
|
+
|
|
20
|
+
Each question gets a level from 0 (absent) to 4 (state of the art), justified by concrete evidence from the repository. The detail for each section lives in a reference grid loaded on demand:
|
|
21
|
+
|
|
22
|
+
- [`reference/conformite-rgpd.md`](reference/conformite-rgpd.md) - CO-01 to CO-07a
|
|
23
|
+
- [`reference/gouvernance-donnees.md`](reference/gouvernance-donnees.md) - DG-01 to DG-08
|
|
24
|
+
- [`reference/finops.md`](reference/finops.md) - FI-01 to FI-08
|
|
25
|
+
|
|
26
|
+
## Why it exists
|
|
27
|
+
|
|
28
|
+
Compliance and cost debt are silent: nothing fails a test suite because there is no purge API or no cost tagging, yet both are real liabilities - a regulator's request you cannot fulfil, a cloud bill nobody can explain. A maturity audit turns that latent risk into a scored, evidence-backed grid, so the MUST-level gaps (a missing DPIA, no deletion propagation) rise to the top of the backlog ahead of the nice-to-haves.
|
|
29
|
+
|
|
30
|
+
It is deliberately **single-domain**. For a whole-project maturity review across all engineering domains, `audit-industrialisation` orchestrates this skill together with its siblings and consolidates the scores - invoke that instead of running each audit by hand.
|
|
31
|
+
|
|
32
|
+
## When it triggers
|
|
33
|
+
|
|
34
|
+
Invoke it when the user asks for a targeted compliance, data-governance or cost review:
|
|
35
|
+
|
|
36
|
+
- "audit compliance" / "compliance audit"
|
|
37
|
+
- "audit GDPR" / "audit data governance"
|
|
38
|
+
- "audit finops"
|
|
39
|
+
|
|
40
|
+
Do **not** invoke it for a full multi-domain audit - route to `audit-industrialisation`, which calls this skill as one of its components.
|
|
41
|
+
|
|
42
|
+
## How it works
|
|
43
|
+
|
|
44
|
+
The audit identifies the project type first (which drives the N/A sets - a local-first SPA marks the server-side CO and all FI questions N/A, a library marks entire CO and FI sections N/A), then works the CO, DG and FI sections in turn: running each grid's verification commands, reading the evidence, and assigning a level 0-4 with a confidence and justification. Conditional sub-questions are scored only when their parent reaches level ≥ 2. Per-section and overall scores are `Σ(levels) / count(scored questions)` excluding N/A, and the results feed the standard report with prioritized MUST/SHOULD/COULD recommendations.
|
|
45
|
+
|
|
46
|
+
Full step-by-step protocol, per-type variants, and output format → [`SKILL.md`](SKILL.md).
|
|
47
|
+
|
|
48
|
+
### Scoring at a glance
|
|
49
|
+
|
|
50
|
+
Criticality (MUST / SHOULD / COULD) drives recommendation priority; the level tables run roughly 0 = absent, 1 = ad-hoc, 2 = documented/basic, 3 = systematic with tooling, 4 = state of the art. N/A questions never drag the average down.
|
|
51
|
+
|
|
52
|
+
## Worked example
|
|
53
|
+
|
|
54
|
+
> A multi-tenant backend for **acme-app** is audited. There is a documented retention policy but no automated purge; a `DELETE /users/:id` endpoint exists but does not propagate to the third-party CRM; resources are tagged per tenant but there is no cost dashboard.
|
|
55
|
+
|
|
56
|
+
- **CO-03 (retention policy, SHOULD)** → level 1. Policy documented but not implemented (no TTL, no purge script found). Confidence: high.
|
|
57
|
+
- **CO-05 (purge API, MUST)** → level 1. Basic deletion endpoint, no propagation. Confidence: high.
|
|
58
|
+
- **CO-06 (third-party propagation, MUST)** → level 0. `grep` found the CRM integration but no deletion propagation. Confidence: high.
|
|
59
|
+
- **FI-04 (per-customer attribution, SHOULD)** → level 2. Per-tenant tagging present, reports manual. Confidence: medium.
|
|
60
|
+
- **FI-01a (granularity, SHOULD)** → scored, since FI-01 reached ≥ 2.
|
|
61
|
+
|
|
62
|
+
The report then recommends, in priority order: `[MUST]` implement deletion propagation to the CRM (CO-06), `[MUST]` complete the purge API and enforce the retention policy, `[SHOULD]` build a FinOps dashboard on top of the existing tags.
|
|
63
|
+
|
|
64
|
+
## Related artifacts
|
|
65
|
+
|
|
66
|
+
- [`audit-industrialisation`](../audit-industrialisation/) - orchestrates all domain audits (including this one) into a consolidated report; use it for a full project audit.
|
|
67
|
+
- [`audit-report`](../audit-report/) - template and scoring rules for the consolidated report.
|
|
68
|
+
- [`audit-security`](../audit-security/) - encryption, secrets and access control, which overlap with sensitive-data protection (DG-06) and GDPR.
|
|
69
|
+
- [`audit-quality`](../audit-quality/) - covers data validation and data lifecycle from the QA angle, complementing DG-02 and DG-07.
|