ronce-racine 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +90 -0
- package/LICENSE +21 -0
- package/README.md +281 -0
- package/agents/code-reviewer.md +33 -0
- package/agents/qa-tester.md +28 -0
- package/dist/hooks/README.md +182 -0
- package/dist/hooks/bash-npm-silent.mjs +81 -0
- package/dist/hooks/session-inject.mjs +53 -0
- package/dist/hooks/session-precompact.mjs +36 -0
- package/dist/hooks/session-writer.mjs +142 -0
- package/dist/hooks/skill-reminder.mjs +89 -0
- package/dist/hooks/truncate-bash-output.mjs +46 -0
- package/dist/hooks/truncate-output.mjs +88 -0
- package/dist/hooks/worktree-env-setup.mjs +123 -0
- package/dist/install.js +962 -0
- package/docs/adopting-a-repo.md +80 -0
- package/docs/architecture.md +76 -0
- package/docs/decisions.md +72 -0
- package/docs/developing.md +143 -0
- package/docs/evaluating-skills.md +41 -0
- package/docs/git-worktrees.md +62 -0
- package/docs/rules/clean-architecture-deps.md +66 -0
- package/docs/rules/commits.md +59 -0
- package/docs/rules/detection-gap-protocol.md +55 -0
- package/docs/rules/error-handling-discipline.md +62 -0
- package/docs/rules/minimal-code.md +58 -0
- package/docs/rules/no-raw-sql-interpolation.md +57 -0
- package/docs/rules/pre-commit-secret-detection.md +54 -0
- package/docs/rules/secure-logging.md +53 -0
- package/docs/rules/sql-migrations-discipline.md +58 -0
- package/docs/rules/subscription-cleanup.md +63 -0
- package/docs/rules/test-discipline.md +66 -0
- package/docs/rules/ui-states-complete.md +52 -0
- package/docs/templates/SKILL.template.md +66 -0
- package/docs/templates/rule.template.md +16 -0
- package/docs/writing-a-rule.md +43 -0
- package/docs/writing-a-skill.md +118 -0
- package/package.json +45 -0
- package/rules/clean-architecture-deps.md +20 -0
- package/rules/commits.md +16 -0
- package/rules/detection-gap-protocol.md +35 -0
- package/rules/error-handling-discipline.md +23 -0
- package/rules/minimal-code.md +32 -0
- package/rules/no-raw-sql-interpolation.md +20 -0
- package/rules/pre-commit-secret-detection.md +14 -0
- package/rules/secure-logging.md +16 -0
- package/rules/sql-migrations-discipline.md +18 -0
- package/rules/subscription-cleanup.md +22 -0
- package/rules/test-discipline.md +20 -0
- package/rules/ui-states-complete.md +23 -0
- package/scripts/subscription-leak-scan.ts +53 -0
- package/skills/adversarial-feature-challenge/README.md +65 -0
- package/skills/adversarial-feature-challenge/SKILL.md +96 -0
- package/skills/adversarial-feature-challenge/assets/fuzz-payloads.txt +60 -0
- package/skills/adversarial-feature-challenge/templates/challenge-report.md +42 -0
- package/skills/api-contract-versioning/README.md +62 -0
- package/skills/api-contract-versioning/SKILL.md +76 -0
- package/skills/api-contract-versioning/templates/consumer-inventory.md +29 -0
- package/skills/audit-architecture/README.md +56 -0
- package/skills/audit-architecture/SKILL.md +97 -0
- package/skills/audit-architecture/reference/ar-architecture.md +275 -0
- package/skills/audit-architecture/reference/di-disponibilite.md +252 -0
- package/skills/audit-architecture/reference/im-incidents.md +217 -0
- package/skills/audit-architecture/reference/pa-pannes.md +362 -0
- package/skills/audit-architecture/reference/re-recuperation.md +300 -0
- package/skills/audit-architecture/reference/sc-scalabilite.md +173 -0
- package/skills/audit-architecture/reference/su-support.md +180 -0
- package/skills/audit-ci-cd/README.md +54 -0
- package/skills/audit-ci-cd/SKILL.md +124 -0
- package/skills/audit-ci-cd/reference/artefacts.md +39 -0
- package/skills/audit-ci-cd/reference/deploiement.md +100 -0
- package/skills/audit-ci-cd/reference/dora.md +94 -0
- package/skills/audit-ci-cd/reference/iac.md +40 -0
- package/skills/audit-ci-cd/reference/pipeline-ci.md +36 -0
- package/skills/audit-ci-cd/reference/supply-chain.md +44 -0
- package/skills/audit-compliance/README.md +69 -0
- package/skills/audit-compliance/SKILL.md +176 -0
- package/skills/audit-compliance/reference/conformite-rgpd.md +295 -0
- package/skills/audit-compliance/reference/finops.md +318 -0
- package/skills/audit-compliance/reference/gouvernance-donnees.md +238 -0
- package/skills/audit-industrialisation/README.md +53 -0
- package/skills/audit-industrialisation/SKILL.md +251 -0
- package/skills/audit-observability/README.md +64 -0
- package/skills/audit-observability/SKILL.md +134 -0
- package/skills/audit-observability/reference/alerting.md +214 -0
- package/skills/audit-observability/reference/logging.md +114 -0
- package/skills/audit-observability/reference/metriques.md +166 -0
- package/skills/audit-observability/reference/traces-dashboards.md +90 -0
- package/skills/audit-performance-frontend/README.md +91 -0
- package/skills/audit-performance-frontend/SKILL.md +194 -0
- package/skills/audit-performance-frontend/reference/assets-cache-network.md +129 -0
- package/skills/audit-performance-frontend/reference/css-rendering-pipeline.md +154 -0
- package/skills/audit-performance-frontend/reference/html-critical-path.md +128 -0
- package/skills/audit-performance-frontend/reference/js-engine-runtime.md +280 -0
- package/skills/audit-performance-frontend/reference/memory-gc.md +126 -0
- package/skills/audit-performance-frontend/reference/profiling-tooling.md +98 -0
- package/skills/audit-quality/README.md +85 -0
- package/skills/audit-quality/SKILL.md +126 -0
- package/skills/audit-quality/reference/qa-devops.md +539 -0
- package/skills/audit-quality/reference/qu-qualite-donnees.md +402 -0
- package/skills/audit-report/README.md +72 -0
- package/skills/audit-report/SKILL.md +241 -0
- package/skills/audit-report/reference/scoring-model.md +45 -0
- package/skills/audit-security/README.md +72 -0
- package/skills/audit-security/SKILL.md +140 -0
- package/skills/audit-security/reference/auth-identite.md +98 -0
- package/skills/audit-security/reference/autorisation.md +66 -0
- package/skills/audit-security/reference/chiffrement-tls.md +97 -0
- package/skills/audit-security/reference/detection-patching.md +67 -0
- package/skills/audit-security/reference/incident-response.md +98 -0
- package/skills/audit-security/reference/reseau-conteneurs.md +72 -0
- package/skills/audit-security/reference/secrets.md +36 -0
- package/skills/audit-testing/README.md +82 -0
- package/skills/audit-testing/SKILL.md +131 -0
- package/skills/audit-testing/reference/golden-dataset.md +65 -0
- package/skills/audit-testing/reference/resilience-chaos.md +93 -0
- package/skills/audit-testing/reference/tests-contractuels.md +65 -0
- package/skills/audit-testing/reference/tests-e2e-flaky.md +67 -0
- package/skills/audit-testing/reference/tests-unit-integ-ci.md +125 -0
- package/skills/bug-ticket-root-cause/README.md +67 -0
- package/skills/bug-ticket-root-cause/SKILL.md +86 -0
- package/skills/bug-ticket-root-cause/templates/bug-ticket.md +43 -0
- package/skills/bug-triage-structured/README.md +63 -0
- package/skills/bug-triage-structured/SKILL.md +104 -0
- package/skills/bug-triage-structured/templates/bug-triage.md +38 -0
- package/skills/ci-pipeline-orchestration/README.md +55 -0
- package/skills/ci-pipeline-orchestration/SKILL.md +110 -0
- package/skills/commit-readiness-review/README.md +66 -0
- package/skills/commit-readiness-review/SKILL.md +82 -0
- package/skills/commit-readiness-review/scripts/precommit-scan.ts +81 -0
- package/skills/comprehensive-test-strategy/README.md +64 -0
- package/skills/comprehensive-test-strategy/SKILL.md +108 -0
- package/skills/comprehensive-test-strategy/reference/testing-advanced.md +85 -0
- package/skills/daily-workflow-optimization/README.md +56 -0
- package/skills/daily-workflow-optimization/SKILL.md +80 -0
- package/skills/database-schema-evolution/README.md +54 -0
- package/skills/database-schema-evolution/SKILL.md +107 -0
- package/skills/ddd-backend-implementation/README.md +63 -0
- package/skills/ddd-backend-implementation/SKILL.md +95 -0
- package/skills/design-system-component-lifecycle/README.md +56 -0
- package/skills/design-system-component-lifecycle/SKILL.md +97 -0
- package/skills/design-system-component-lifecycle/templates/component-contract.md +43 -0
- package/skills/detection-sweep/README.md +58 -0
- package/skills/detection-sweep/SKILL.md +93 -0
- package/skills/detection-sweep/scripts/sweep.ts +80 -0
- package/skills/domain-modeling-design/README.md +57 -0
- package/skills/domain-modeling-design/SKILL.md +83 -0
- package/skills/frontend-fullstack-implementation/README.md +60 -0
- package/skills/frontend-fullstack-implementation/SKILL.md +133 -0
- package/skills/frontend-spec-call-site-audit/README.md +66 -0
- package/skills/frontend-spec-call-site-audit/SKILL.md +82 -0
- package/skills/frontend-spec-call-site-audit/scripts/audit-entry-points.ts +79 -0
- package/skills/frontend-spec-call-site-audit/templates/ticket-frontend.md +32 -0
- package/skills/merge-request-review/README.md +60 -0
- package/skills/merge-request-review/SKILL.md +77 -0
- package/skills/merge-request-review/templates/mr-review-report.md +29 -0
- package/skills/production-incident-diagnostic/README.md +63 -0
- package/skills/production-incident-diagnostic/SKILL.md +103 -0
- package/skills/production-incident-diagnostic/templates/incident-report.md +33 -0
- package/skills/qa-session-intake/README.md +56 -0
- package/skills/qa-session-intake/SKILL.md +78 -0
- package/skills/recording-decisions/README.md +91 -0
- package/skills/recording-decisions/SKILL.md +84 -0
- package/skills/recurring-bug-root-cause/README.md +64 -0
- package/skills/recurring-bug-root-cause/SKILL.md +93 -0
- package/skills/recurring-bug-root-cause/scripts/detect-recurring-fixes.ts +46 -0
- package/skills/recurring-bug-root-cause/templates/postmortem-recidive.md +37 -0
- package/skills/refactoring-shared-component-api/README.md +50 -0
- package/skills/refactoring-shared-component-api/SKILL.md +84 -0
- package/skills/refactoring-shared-component-api/scripts/sweep-call-sites.ts +85 -0
- package/skills/refactoring-shared-component-api/templates/migration-checklist.md +19 -0
- package/skills/validating-features-end-to-end/README.md +50 -0
- package/skills/validating-features-end-to-end/SKILL.md +90 -0
- package/skills/visual-regression-check/README.md +51 -0
- package/skills/visual-regression-check/SKILL.md +89 -0
- package/skills/writing-robust-tests/README.md +53 -0
- package/skills/writing-robust-tests/SKILL.md +88 -0
- package/templates/.adopted.example +8 -0
- package/templates/anti-drift.gitlab-ci.yml +23 -0
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
# Encryption & TLS section
|
|
2
|
+
|
|
3
|
+
Questions: SE-01 (TLS), SE-02 (DB encryption), SE-02a (Key management).
|
|
4
|
+
|
|
5
|
+
## Table of contents
|
|
6
|
+
|
|
7
|
+
- [SE-01 - TLS 1.2+ / TLS 1.3](#se-01--tls-12--tls-13--criticality-must)
|
|
8
|
+
- [SE-02 - DB encryption (AES-256) of sensitive data](#se-02--db-encryption-aes-256-of-sensitive-data--criticality-must)
|
|
9
|
+
- [SE-02a - Encryption key management](#se-02a--encryption-key-management--criticality-must)
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
### SE-01 - TLS 1.2+ / TLS 1.3 - Criticality: **must**
|
|
14
|
+
|
|
15
|
+
**Analyze:** Nginx/Caddy/Traefik configs, certificates, HTTPS configs in the app
|
|
16
|
+
|
|
17
|
+
**Check:**
|
|
18
|
+
- TLS 1.2+ (ideally 1.3) on all endpoints
|
|
19
|
+
- HSTS enabled
|
|
20
|
+
- No mixed content (HTTP/HTTPS)
|
|
21
|
+
- Automated certificate management (Let's Encrypt/ACME)
|
|
22
|
+
- mTLS for inter-service communication (if microservices)
|
|
23
|
+
- Certificate Transparency monitoring
|
|
24
|
+
|
|
25
|
+
**Commands:**
|
|
26
|
+
```bash
|
|
27
|
+
grep -riE "ssl|tls|https|hsts|strict.transport" nginx.conf Caddyfile docker-compose*.yml 2>/dev/null
|
|
28
|
+
grep -riE "ssl_protocols|ssl_ciphers|tls_version" . --include="*.conf" --include="*.yml" 2>/dev/null | head -10
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
**Levels:**
|
|
32
|
+
| Level | Criticality | Criteria |
|
|
33
|
+
|--------|-----------|----------|
|
|
34
|
+
| 0 | must | No TLS or TLS 1.0/1.1. Data may travel in the clear. |
|
|
35
|
+
| 1 | must | TLS 1.2 on public endpoints. Default configuration. No audit. |
|
|
36
|
+
| 2 | must | TLS 1.2+ everywhere. Cipher suites reviewed. Basic certificate management. |
|
|
37
|
+
| 3 | must | TLS 1.3 preferred. HSTS enabled. Certificate automation (Let's Encrypt/ACME). Annual audit. |
|
|
38
|
+
| 4 | must | mTLS for internal communication. Certificate pinning. HSM for critical keys. PCI-DSS/SOC2 compliance. |
|
|
39
|
+
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
### SE-02 - DB encryption (AES-256) of sensitive data - Criticality: **must**
|
|
43
|
+
|
|
44
|
+
**Analyze:** DB config (encryption at rest), application-level encryption code, sensitive columns
|
|
45
|
+
|
|
46
|
+
**Check:**
|
|
47
|
+
- Encryption at rest enabled on the DB (TDE)
|
|
48
|
+
- Sensitive data encrypted at the application level (PII, secrets, tokens)
|
|
49
|
+
- Robust algorithm (AES-256, no MD5/SHA1 for crypto)
|
|
50
|
+
- Column-level encryption for regulated data
|
|
51
|
+
- Encrypted backups
|
|
52
|
+
|
|
53
|
+
**Commands:**
|
|
54
|
+
```bash
|
|
55
|
+
grep -ri "encrypt\|aes\|kms\|pgcrypto\|cipher" . --include="*.ts" --include="*.py" --include="*.php" 2>/dev/null | head -10
|
|
56
|
+
grep -ri "cast.*encrypted\|EncryptedString\|encryptable" . --include="*.php" 2>/dev/null | head -5
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
**Levels:**
|
|
60
|
+
| Level | Criticality | Criteria |
|
|
61
|
+
|--------|-----------|----------|
|
|
62
|
+
| 0 | must | No encryption. Sensitive data in the clear in the DB. |
|
|
63
|
+
| 1 | must | Ad-hoc encryption of a few fields. Keys in the code or config. |
|
|
64
|
+
| 2 | must | TDE (Transparent Data Encryption) enabled. Sensitive fields encrypted at the application level. |
|
|
65
|
+
| 3 | must | AES-256 with KMS. Key separation per environment. Encrypted backups. |
|
|
66
|
+
| 4 | must | Envelope encryption. Automatic key rotation. Audit trail of key access. HSM. |
|
|
67
|
+
|
|
68
|
+
---
|
|
69
|
+
|
|
70
|
+
### SE-02a - Encryption key management - Criticality: **must**
|
|
71
|
+
|
|
72
|
+
**Dependency:** Evaluate only if SE-02 >= level 2.
|
|
73
|
+
|
|
74
|
+
**Analyze:** Key storage, key lifecycle, DEK/KEK separation, use of KMS/Vault/HSM
|
|
75
|
+
|
|
76
|
+
**Check:**
|
|
77
|
+
- Encryption keys not hardcoded or in config files
|
|
78
|
+
- Use of a secret manager (Vault, AWS Secrets Manager, Azure Key Vault)
|
|
79
|
+
- DEK (Data Encryption Key) / KEK (Key Encryption Key) separation
|
|
80
|
+
- Documented and scheduled key rotation
|
|
81
|
+
- HSM for root keys in critical environments
|
|
82
|
+
|
|
83
|
+
**Commands:**
|
|
84
|
+
```bash
|
|
85
|
+
grep -ri "APP_KEY\|ENCRYPTION_KEY\|MASTER_KEY\|KMS\|vault" .env.example 2>/dev/null
|
|
86
|
+
grep -ri "kms\|vault\|hsm\|key.rotation\|key.management" . --include="*.ts" --include="*.php" --include="*.yml" 2>/dev/null | head -10
|
|
87
|
+
git log --all --diff-filter=A -- "*.key" "*.pem" 2>/dev/null | head -5
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
**Levels:**
|
|
91
|
+
| Level | Criticality | Criteria |
|
|
92
|
+
|--------|-----------|----------|
|
|
93
|
+
| 0 | must | Keys hardcoded or in config files. |
|
|
94
|
+
| 1 | must | Environment variables. No rotation. |
|
|
95
|
+
| 2 | must | Secret manager (Vault, AWS Secrets Manager). Manual rotation. |
|
|
96
|
+
| 3 | must | KMS with automatic rotation. DEK/KEK separation. |
|
|
97
|
+
| 4 | must | HSM for root keys. BYOK supported. Full audit. |
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
# Detection & Patching section
|
|
2
|
+
|
|
3
|
+
Questions: SE-08 (SIEM detection), SE-09 (Patching & communication).
|
|
4
|
+
|
|
5
|
+
## Table of contents
|
|
6
|
+
|
|
7
|
+
- [SE-08 - SIEM detection](#se-08--siem-detection--criticality-should)
|
|
8
|
+
- [SE-09 - Patching & communication](#se-09--patching--communication--criticality-must)
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
### SE-08 - SIEM detection - Criticality: **should**
|
|
13
|
+
|
|
14
|
+
**Analyze:** Deployed SIEM, correlation rules, log source integration, SOAR, UEBA
|
|
15
|
+
|
|
16
|
+
**Check:**
|
|
17
|
+
- SIEM deployed (Splunk, Elastic SIEM, Microsoft Sentinel, Google Chronicle)
|
|
18
|
+
- Log sources integrated (firewall, servers, apps, IdP, cloud)
|
|
19
|
+
- Custom correlation rules (not just the default rules)
|
|
20
|
+
- SOC or dedicated monitoring team
|
|
21
|
+
- SOAR to automate the response to common alerts
|
|
22
|
+
- Integrated threat intelligence feeds (MITRE ATT&CK, IoC feeds)
|
|
23
|
+
|
|
24
|
+
**Commands:**
|
|
25
|
+
```bash
|
|
26
|
+
grep -riE "siem\|splunk\|elastic\|sentinel\|soar\|threat.intel" . --include="*.yml" --include="*.tf" --include="*.md" 2>/dev/null | head -10
|
|
27
|
+
grep -riE "alert\|rule\|correlation\|anomaly\|detection" . --include="*.yml" --include="*.json" 2>/dev/null | grep -iv "node_modules\|test" | head -10
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
**Levels:**
|
|
31
|
+
| Level | Criticality | Criteria |
|
|
32
|
+
|--------|-----------|----------|
|
|
33
|
+
| 0 | should | No SIEM. Manual or nonexistent detection. |
|
|
34
|
+
| 1 | should | Centralized logs (ELK). Manual search. |
|
|
35
|
+
| 2 | should | Basic SIEM with predefined alerts (brute force, etc.). |
|
|
36
|
+
| 3 | should | SIEM with custom rules. Event correlation. SOC or dedicated team. |
|
|
37
|
+
| 4 | should | SOAR (Security Orchestration). ML for anomaly detection. Threat intelligence feeds. |
|
|
38
|
+
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
### SE-09 - Patching & communication - Criticality: **must**
|
|
42
|
+
|
|
43
|
+
**Analyze:** Patching process, SLA by CVSS severity, automation, vulnerability scanning, communication
|
|
44
|
+
|
|
45
|
+
**Check:**
|
|
46
|
+
- CVE and vulnerability tracking (Dependabot, Renovate, Snyk, Trivy)
|
|
47
|
+
- Patching SLA by severity (Critical <48h, High <7d, Medium <30d, Low <90d)
|
|
48
|
+
- Patching automation (Dependabot auto-merge, Renovate, AWS Systems Manager)
|
|
49
|
+
- Virtual patching via WAF for zero-day vulnerabilities
|
|
50
|
+
- Proactive customer communication on impactful patches
|
|
51
|
+
|
|
52
|
+
**Commands:**
|
|
53
|
+
```bash
|
|
54
|
+
npm audit --json 2>/dev/null | head -30
|
|
55
|
+
ls .github/dependabot.yml renovate.json 2>/dev/null
|
|
56
|
+
grep -riE "snyk\|trivy\|grype\|qualys\|nessus" . --include="*.yml" --include="*.json" 2>/dev/null | head -5
|
|
57
|
+
grep -riE "vulnerability\|cve\|patch\|security.update" . --include="*.md" --include="*.yml" 2>/dev/null | head -10
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
**Levels:**
|
|
61
|
+
| Level | Criticality | Criteria |
|
|
62
|
+
|--------|-----------|----------|
|
|
63
|
+
| 0 | must | No patching process. Vulnerabilities untracked. |
|
|
64
|
+
| 1 | must | Occasional manual patching. No CVE tracking. |
|
|
65
|
+
| 2 | must | Critical CVE tracking. Monthly patching. Basic communication. |
|
|
66
|
+
| 3 | must | Patching SLA by severity. Partial automation. Proactive customer communication. |
|
|
67
|
+
| 4 | must | Zero-day response process. Virtual patching. Automated vulnerability scanning. Full disclosure policy. |
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# Incident response section
|
|
2
|
+
|
|
3
|
+
Questions: SE-07 (Incident response plan), SE-07a (Crisis communication), SE-10 (Security post-mortem).
|
|
4
|
+
|
|
5
|
+
## Table of contents
|
|
6
|
+
|
|
7
|
+
- [SE-07 - Security incident response plan](#se-07--security-incident-response-plan--criticality-must)
|
|
8
|
+
- [SE-07a - Crisis communication](#se-07a--crisis-communication--criticality-should)
|
|
9
|
+
- [SE-10 - Security post-mortem](#se-10--security-post-mortem--criticality-should)
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
### SE-07 - Security incident response plan - Criticality: **must**
|
|
14
|
+
|
|
15
|
+
**Analyze:** IRP documentation, playbooks, roles (incident commander, technical lead, communications lead), simulation exercises
|
|
16
|
+
|
|
17
|
+
**Check:**
|
|
18
|
+
- Documented incident response plan (IRP)
|
|
19
|
+
- Defined roles and responsibilities (incident commander, technical lead, communications lead)
|
|
20
|
+
- Incident classification by severity (P1-P4) with response SLAs
|
|
21
|
+
- Playbooks per incident type (ransomware, data breach, account compromise, DDoS)
|
|
22
|
+
- Regular simulation exercises (tabletop exercises)
|
|
23
|
+
- GDPR articles 33-34 compliance (72h notification)
|
|
24
|
+
|
|
25
|
+
**Commands:**
|
|
26
|
+
```bash
|
|
27
|
+
find . -iname "*incident*" -o -iname "*playbook*" -o -iname "*irp*" -o -iname "*security*plan*" 2>/dev/null | head -10
|
|
28
|
+
grep -riE "incident.response\|playbook\|tabletop\|red.team\|bug.bounty" . --include="*.md" --include="*.yml" --include="*.adoc" 2>/dev/null | head -10
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
**Levels:**
|
|
32
|
+
| Level | Criticality | Criteria |
|
|
33
|
+
|--------|-----------|----------|
|
|
34
|
+
| 0 | must | No plan. Ad-hoc reaction on incident. |
|
|
35
|
+
| 1 | must | Informal plan. Emergency contacts known to a few people. |
|
|
36
|
+
| 2 | must | Documented plan with roles. Not regularly tested. |
|
|
37
|
+
| 3 | must | Plan tested annually (tabletop exercises). Systematic post-mortems. |
|
|
38
|
+
| 4 | must | Red team exercises. Bug bounty. Incident response automation. Internal CERT or partnership. |
|
|
39
|
+
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
### SE-07a - Crisis communication - Criticality: **should**
|
|
43
|
+
|
|
44
|
+
**Dependency:** Evaluate only if SE-07 >= level 2.
|
|
45
|
+
|
|
46
|
+
**Analyze:** Communication templates, war room process, status page, audience-differentiated communication
|
|
47
|
+
|
|
48
|
+
**Check:**
|
|
49
|
+
- Pre-written communication templates per severity level (P1-P4)
|
|
50
|
+
- Pre-configured distribution lists (internal, customers, authorities)
|
|
51
|
+
- War room process (physical or virtual)
|
|
52
|
+
- Public status page (Statuspage.io, Instatus, etc.)
|
|
53
|
+
- Audience-differentiated communication (technical, management, customers, authorities)
|
|
54
|
+
- GDPR articles 33-34 compliance (CNIL notification within 72h)
|
|
55
|
+
|
|
56
|
+
**Commands:**
|
|
57
|
+
```bash
|
|
58
|
+
grep -riE "status.page\|statuspage\|communication.*crise\|war.room\|template.*incident" . --include="*.md" --include="*.yml" --include="*.adoc" 2>/dev/null | head -10
|
|
59
|
+
find . -iname "*crisis*" -o -iname "*communication*plan*" -o -iname "*status*page*" 2>/dev/null | head -5
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
**Levels:**
|
|
63
|
+
| Level | Criticality | Criteria |
|
|
64
|
+
|--------|-----------|----------|
|
|
65
|
+
| 0 | should | No process. Improvised communication. |
|
|
66
|
+
| 1 | should | Emergency contacts. Communication by email/Slack. |
|
|
67
|
+
| 2 | should | Prepared communication templates. Defined distribution list. |
|
|
68
|
+
| 3 | should | War room process. Differentiated internal/external communication. CNIL notification if required. |
|
|
69
|
+
| 4 | should | Automated communication. Public status page. Notification SLA. Coordination with authorities. |
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
### SE-10 - Security post-mortem - Criticality: **should**
|
|
74
|
+
|
|
75
|
+
**Analyze:** Post-mortem process, templates, tracking of corrective actions, recurrence metrics
|
|
76
|
+
|
|
77
|
+
**Check:**
|
|
78
|
+
- Systematic post-mortem after each security incident
|
|
79
|
+
- Blameless approach (focus on systems, not individuals)
|
|
80
|
+
- Root cause analysis (5 Whys, timeline reconstruction)
|
|
81
|
+
- Corrective actions tracked in a tool (Jira, Linear) with deadlines
|
|
82
|
+
- Sharing of lessons learned across the organization
|
|
83
|
+
- Recurrence metrics and continuous improvement
|
|
84
|
+
|
|
85
|
+
**Commands:**
|
|
86
|
+
```bash
|
|
87
|
+
find . -iname "*post-mortem*" -o -iname "*postmortem*" -o -iname "*incident-report*" 2>/dev/null | head -10
|
|
88
|
+
grep -riE "post.mortem\|blameless\|root.cause\|lessons.learned\|5.whys" . --include="*.md" --include="*.adoc" 2>/dev/null | head -10
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
**Levels:**
|
|
92
|
+
| Level | Criticality | Criteria |
|
|
93
|
+
|--------|-----------|----------|
|
|
94
|
+
| 0 | should | No post-mortem. Incidents forgotten. |
|
|
95
|
+
| 1 | should | Informal discussion after a major incident. |
|
|
96
|
+
| 2 | should | Documented post-mortem for critical incidents. Actions identified. |
|
|
97
|
+
| 3 | should | Systematic post-mortem. Action tracking. Sharing of learnings. |
|
|
98
|
+
| 4 | should | Blameless post-mortems. Recurrence metrics. Measured continuous improvement. |
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
# Network & Containers section
|
|
2
|
+
|
|
3
|
+
Questions: SE-12 (Network security), SE-13 (Container security).
|
|
4
|
+
|
|
5
|
+
## Table of contents
|
|
6
|
+
|
|
7
|
+
- [SE-12 - Network security and perimeter protection](#se-12--network-security-and-perimeter-protection--criticality-must)
|
|
8
|
+
- [SE-13 - Security of containers and deployed images](#se-13--security-of-containers-and-deployed-images--criticality-must)
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
### SE-12 - Network security and perimeter protection - Criticality: **must**
|
|
13
|
+
|
|
14
|
+
**Analyze:** WAF, DDoS protection, network segmentation, microsegmentation, IDS/IPS, Zero Trust
|
|
15
|
+
|
|
16
|
+
**Check:**
|
|
17
|
+
- WAF configured (AWS WAF, Cloudflare WAF, ModSecurity) with OWASP rules
|
|
18
|
+
- DDoS protection (Cloudflare, AWS Shield)
|
|
19
|
+
- Network segmentation per environment (prod, staging, dev)
|
|
20
|
+
- Workload microsegmentation (Kubernetes NetworkPolicies, Security Groups)
|
|
21
|
+
- IDS/IPS deployed (Suricata, Palo Alto, AWS Network Firewall)
|
|
22
|
+
- Zero Trust approach (no implicit trust on the internal network)
|
|
23
|
+
|
|
24
|
+
**Commands:**
|
|
25
|
+
```bash
|
|
26
|
+
grep -riE "waf\|firewall\|ddos\|shield\|cloudflare" . --include="*.tf" --include="*.yml" --include="*.json" 2>/dev/null | head -10
|
|
27
|
+
grep -riE "networkpolicy\|securitygroup\|security.group\|ingress\|egress" . --include="*.yml" --include="*.yaml" --include="*.tf" 2>/dev/null | head -10
|
|
28
|
+
grep -riE "ids\|ips\|suricata\|zerotrust\|zero.trust\|beyondcorp" . --include="*.yml" --include="*.tf" --include="*.md" 2>/dev/null | head -5
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
**Levels:**
|
|
32
|
+
| Level | Criticality | Criteria |
|
|
33
|
+
|--------|-----------|----------|
|
|
34
|
+
| 0 | must | No network protection. No WAF, no segmentation. |
|
|
35
|
+
| 1 | must | Basic firewall. No WAF or DDoS protection. |
|
|
36
|
+
| 2 | must | WAF configured with basic rules. Network segmentation per environment. |
|
|
37
|
+
| 3 | must | Workload microsegmentation. Active DDoS protection. IDS/IPS deployed. |
|
|
38
|
+
| 4 | must | Zero Trust networking. Automated threat response. ML-based anomaly detection. Secure service mesh. |
|
|
39
|
+
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
### SE-13 - Security of containers and deployed images - Criticality: **must**
|
|
43
|
+
|
|
44
|
+
**Analyze:** Base images, vulnerability scanning, image signing, execution policies, runtime protection
|
|
45
|
+
|
|
46
|
+
**Check:**
|
|
47
|
+
- Private registry (no unverified public images)
|
|
48
|
+
- Automated image scanning (Trivy, Snyk, Grype) in CI/CD
|
|
49
|
+
- Minimal base images (distroless, Alpine, scratch)
|
|
50
|
+
- Image signing (Cosign/Sigstore, Notary)
|
|
51
|
+
- Pod Security Standards / Pod Security Policies enforced
|
|
52
|
+
- No containers running in privileged mode
|
|
53
|
+
- Runtime protection (Falco, Sysdig)
|
|
54
|
+
- SBOM (Software Bill of Materials) for all images
|
|
55
|
+
- Admission controllers (OPA/Gatekeeper, Kyverno)
|
|
56
|
+
|
|
57
|
+
**Commands:**
|
|
58
|
+
```bash
|
|
59
|
+
grep -riE "FROM\s+" . --include="Dockerfile*" 2>/dev/null | head -10
|
|
60
|
+
grep -riE "trivy\|snyk\|grype\|cosign\|notary\|sbom" . --include="*.yml" --include="*.yaml" 2>/dev/null | head -10
|
|
61
|
+
grep -riE "privileged\|hostNetwork\|hostPID\|securityContext\|runAsNonRoot" . --include="*.yml" --include="*.yaml" 2>/dev/null | head -10
|
|
62
|
+
grep -riE "falco\|gatekeeper\|kyverno\|admission.controller\|pod.security" . --include="*.yml" --include="*.yaml" --include="*.tf" 2>/dev/null | head -5
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
**Levels:**
|
|
66
|
+
| Level | Criticality | Criteria |
|
|
67
|
+
|--------|-----------|----------|
|
|
68
|
+
| 0 | must | No container security. Public images used without verification. |
|
|
69
|
+
| 1 | must | Public images without scanning. No pod security policy. |
|
|
70
|
+
| 2 | must | Private registry. Basic vulnerability scanning on images. |
|
|
71
|
+
| 3 | must | Automated scanning in CI/CD. Signed images. Minimal base images. Pod security policies enforced. |
|
|
72
|
+
| 4 | must | Runtime protection (Falco). Admission controllers (OPA/Kyverno). SBOM for all images. Continuous compliance. |
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Secrets section
|
|
2
|
+
|
|
3
|
+
Questions: SE-03 (Key/secret rotation).
|
|
4
|
+
|
|
5
|
+
## Table of contents
|
|
6
|
+
|
|
7
|
+
- [SE-03 - Key/secret rotation](#se-03--keysecret-rotation--criticality-must)
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
### SE-03 - Key/secret rotation - Criticality: **must**
|
|
12
|
+
|
|
13
|
+
**Analyze:** Secret rotation policy, automation, ephemeral secrets, TTL
|
|
14
|
+
|
|
15
|
+
**Check:**
|
|
16
|
+
- Documented rotation policy for all types of secrets (DB passwords, API keys, tokens, certificates)
|
|
17
|
+
- Automated rotation (not manual only)
|
|
18
|
+
- Rotation frequency matched to sensitivity (monthly to quarterly)
|
|
19
|
+
- Zero-downtime rotation (dual-write, then switch)
|
|
20
|
+
- Ephemeral / dynamic secrets for advanced environments
|
|
21
|
+
|
|
22
|
+
**Commands:**
|
|
23
|
+
```bash
|
|
24
|
+
grep -ri "rotation\|ttl\|expir\|renew\|dynamic.secret" . --include="*.ts" --include="*.py" --include="*.yml" --include="*.php" 2>/dev/null | head -10
|
|
25
|
+
grep -ri "password\|secret\|api_key\|token\|private_key" . --include="*.env" --include="*.env.example" 2>/dev/null | head -10
|
|
26
|
+
grep ".env" .gitignore 2>/dev/null
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
**Levels:**
|
|
30
|
+
| Level | Criticality | Criteria |
|
|
31
|
+
|--------|-----------|----------|
|
|
32
|
+
| 0 | must | No rotation. Secrets never changed since creation. |
|
|
33
|
+
| 1 | must | Manual rotation on incident or employee departure. |
|
|
34
|
+
| 2 | must | Scheduled annual rotation. Documented but manual process. |
|
|
35
|
+
| 3 | must | Automated quarterly rotation. Alerts on expired secrets. |
|
|
36
|
+
| 4 | must | Continuous automatic rotation. Zero-downtime rotation. Ephemeral secrets. |
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
# `audit-testing`
|
|
2
|
+
|
|
3
|
+
> Score a project's testing maturity across 13 questions and five sections, from unit-test structure to chaos engineering, on a 0-4 scale.
|
|
4
|
+
|
|
5
|
+
| | |
|
|
6
|
+
|---|---|
|
|
7
|
+
| **Type** | Skill (on-demand audit) |
|
|
8
|
+
| **Category** | `audit` |
|
|
9
|
+
| **Artifact** | [`SKILL.md`](SKILL.md) |
|
|
10
|
+
| **Tooling** | Bash verification commands + progressive-disclosure grids in [`reference/`](reference/) |
|
|
11
|
+
|
|
12
|
+
## What it is
|
|
13
|
+
|
|
14
|
+
`audit-testing` is a structured maturity assessment for a project's **test strategy and validation practices**. It walks 13 questions grouped into five sections - contract testing, golden dataset, E2E & flaky tests, resilience & chaos, and unit/integration/CI tests - and scores each one from 0 (absent) to 4 (state of the art). The output is a scored report with strengths, weaknesses, prioritized recommendations, and an explicit list of what could not be verified statically.
|
|
15
|
+
|
|
16
|
+
It is one of eight domain audits orchestrated by [`audit-industrialisation`](../audit-industrialisation/). Invoked on its own, it covers testing only.
|
|
17
|
+
|
|
18
|
+
## Why it exists
|
|
19
|
+
|
|
20
|
+
Test suites rot silently. A project can have thousands of green tests and still be fragile: no integration coverage against a real database, flaky E2E tests that everyone retries until they pass, no simulation of a third-party API going down, no way to replay a historical batch. A single "coverage %" number hides all of this.
|
|
21
|
+
|
|
22
|
+
This skill replaces gut feeling with a **repeatable rubric**. Every question has explicit bash checks and a five-level scale, so two auditors reach comparable scores, and the same project can be re-audited over time to see whether maturity is actually improving.
|
|
23
|
+
|
|
24
|
+
## When it triggers
|
|
25
|
+
|
|
26
|
+
Invoke it when someone asks for a testing-focused audit:
|
|
27
|
+
|
|
28
|
+
- "audit tests", "audit testing"
|
|
29
|
+
- "test coverage audit", "test quality audit"
|
|
30
|
+
- a request to assess test strategy, the test pyramid, flaky-test handling, chaos testing, or a test plan before a release
|
|
31
|
+
|
|
32
|
+
For a full multi-domain project audit, use [`audit-industrialisation`](../audit-industrialisation/) instead - it runs this skill along with the seven others and consolidates the results.
|
|
33
|
+
|
|
34
|
+
## How it works
|
|
35
|
+
|
|
36
|
+
### Scoring model
|
|
37
|
+
|
|
38
|
+
- **Domain score** = average of scored questions, excluding N/A.
|
|
39
|
+
- A question is **N/A** when the project profile makes it irrelevant (no third-party API → te-01/te-01a/te-05 N/A; no batch processing → te-06 N/A).
|
|
40
|
+
- **Conditional sub-questions** (te-01a, te-02a, te-03a, te-09a) are only scored if their parent reaches level ≥ 2.
|
|
41
|
+
- Weighting by criticality - **must** (weight 3): te-07, te-08, te-09; **should** (weight 2): te-02, te-02a, te-03, te-03a, te-05, te-09a; **could** (weight 1): te-01, te-01a, te-04, te-06.
|
|
42
|
+
- Weighted score = `sum(level × weight) / sum(weight)` over the scored questions.
|
|
43
|
+
|
|
44
|
+
### Audit protocol
|
|
45
|
+
|
|
46
|
+
The skill detects the test frameworks and counts test files by type, reads the coverage configs and CI stages, then runs each question's verification commands - scoring the conditional sub-questions only when their parent reaches level ≥ 2 - and inspects the test pyramid and resilience patterns before assigning levels, marking N/A, listing non-auditable items and producing the report.
|
|
47
|
+
|
|
48
|
+
Full step-by-step protocol → [`SKILL.md`](SKILL.md).
|
|
49
|
+
|
|
50
|
+
### The five reference grids
|
|
51
|
+
|
|
52
|
+
Each grid holds, per question: the statement, what to analyze, what to check, the bash commands, and the 0-4 level table.
|
|
53
|
+
|
|
54
|
+
| Section | Grid | Questions |
|
|
55
|
+
|---------|------|-----------|
|
|
56
|
+
| Contract testing | [`reference/tests-contractuels.md`](reference/tests-contractuels.md) | te-01, te-01a |
|
|
57
|
+
| Golden dataset | [`reference/golden-dataset.md`](reference/golden-dataset.md) | te-02, te-02a |
|
|
58
|
+
| E2E & flaky tests | [`reference/tests-e2e-flaky.md`](reference/tests-e2e-flaky.md) | te-03, te-03a |
|
|
59
|
+
| Resilience & chaos | [`reference/resilience-chaos.md`](reference/resilience-chaos.md) | te-04, te-05, te-06 |
|
|
60
|
+
| Unit, integration & CI | [`reference/tests-unit-integ-ci.md`](reference/tests-unit-integ-ci.md) | te-07, te-08, te-09, te-09a |
|
|
61
|
+
|
|
62
|
+
### Confidence levels
|
|
63
|
+
|
|
64
|
+
Each score carries a confidence: **high** (verified by an executed command or a direct file read), **medium** (inferred from structure but not deeply verified), or **low** (needs an interview or external access such as a coverage dashboard or historical reports).
|
|
65
|
+
|
|
66
|
+
## Worked example
|
|
67
|
+
|
|
68
|
+
> You audit `acme-app`, a backend service that talks to a third-party payment provider.
|
|
69
|
+
|
|
70
|
+
1. te-07: `find` counts 420 `*.spec.ts` files, `vitest.config.ts` sets a 80% branch threshold → **level 3, high confidence**.
|
|
71
|
+
2. te-08: Testcontainers spins up a real Postgres in the integration suite → **level 3, high confidence**.
|
|
72
|
+
3. te-03: a Playwright config exists but CI marks the E2E job `allow_failure: true` and three specs are `test.skip` → **level 2, high confidence**; te-03a triggers (parent ≥ 2) and, with no quarantine process, scores **level 1**.
|
|
73
|
+
4. te-05: the payment provider is never simulated - no Toxiproxy, no WireMock, no circuit breaker in the source → **level 0, medium confidence**, flagged as a top recommendation.
|
|
74
|
+
5. te-06 is marked **N/A** (no batch processing in `acme-app`).
|
|
75
|
+
|
|
76
|
+
The domain score is the weighted average of the scored questions, and te-05 lands in the `[MUST]`/`[SHOULD]` recommendations because an unsimulated payment dependency is a real resilience gap.
|
|
77
|
+
|
|
78
|
+
## Related artifacts
|
|
79
|
+
|
|
80
|
+
- [`audit-industrialisation`](../audit-industrialisation/) - orchestrates this audit and seven others into one consolidated report.
|
|
81
|
+
- [`audit-quality`](../audit-quality/) - QA & DevOps plus data quality; overlaps on coverage and CI but from a broader delivery angle.
|
|
82
|
+
- [`audit-report`](../audit-report/) - the consolidated report template and scoring rules.
|
|
@@ -0,0 +1,131 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: audit-testing
|
|
3
|
+
description: Testing & Validation audit. Use when auditing test strategy, coverage, test types, flaky tests, chaos testing, or test plans. Triggers on "audit tests", "audit testing", "audit couverture", "test coverage audit", "audit qualité tests", "test quality audit".
|
|
4
|
+
version: 1.0.1
|
|
5
|
+
metadata:
|
|
6
|
+
last-reviewed: 2026-07-20
|
|
7
|
+
category: audit
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Testing & Validation Audit
|
|
11
|
+
|
|
12
|
+
## When ME and not audit-industrialisation
|
|
13
|
+
|
|
14
|
+
- **ME** when: an audit targeting this domain only
|
|
15
|
+
- **audit-industrialisation** instead if: a global multi-domain audit - it orchestrates every audit (including ME) and produces a consolidated report
|
|
16
|
+
|
|
17
|
+
> For a full project audit, use `audit-industrialisation` rather than invoking each skill separately.
|
|
18
|
+
|
|
19
|
+
## Overview
|
|
20
|
+
|
|
21
|
+
Audit the maturity of a project's testing practices: coverage, test pyramid, stability, contract testing, chaos engineering, golden dataset, batch replay and CI feedback loop. Produces a maturity score (0-4) per question.
|
|
22
|
+
|
|
23
|
+
**13 questions** split into 5 sections. The detailed grids (statement, checks, commands, levels 0-4) live in `reference/` - see [Grids by section](#grids-by-section).
|
|
24
|
+
|
|
25
|
+
## Criticality levels
|
|
26
|
+
|
|
27
|
+
- **must**: fundamental practice, absence = critical risk
|
|
28
|
+
- **should**: good practice, absence = moderate risk
|
|
29
|
+
- **could**: excellence, absence = room for improvement
|
|
30
|
+
|
|
31
|
+
## Score calculation
|
|
32
|
+
|
|
33
|
+
- Domain score = average of scored questions (exclude N/A)
|
|
34
|
+
- State "X questions scored out of Y total"
|
|
35
|
+
- A question is N/A if the project profile makes it non-applicable (e.g. no third-party API → te-01/te-01a/te-05 N/A, no batch → te-06 N/A)
|
|
36
|
+
- Conditional sub-questions (te-01a, te-02a, te-03a, te-09a) are N/A if their parent question does not reach the required level
|
|
37
|
+
- Criticality (used to prioritize remediation, NOT in the score - see `audit-report/reference/scoring-model.md`):
|
|
38
|
+
- **must**: te-07, te-08, te-09
|
|
39
|
+
- **should**: te-02, te-02a, te-03, te-03a, te-05, te-09a
|
|
40
|
+
- **could**: te-01, te-01a, te-04, te-06
|
|
41
|
+
|
|
42
|
+
## Audit protocol
|
|
43
|
+
|
|
44
|
+
1. **Detect the test frameworks**: run the te-07 and te-03 commands to find the configs
|
|
45
|
+
2. **Count test files** by type (unit, integration, e2e) via the dedicated commands
|
|
46
|
+
3. **Read the coverage configs** and thresholds
|
|
47
|
+
4. **Check the CI pipeline** for test stages (te-09 commands)
|
|
48
|
+
5. **Run the verification commands** for every question te-01 to te-09a
|
|
49
|
+
6. **Evaluate the conditional sub-questions**: te-01a (if te-01 ≥ 2), te-02a (if te-02 ≥ 2), te-03a (if te-03 ≥ 2), te-09a (if te-09 ≥ 2)
|
|
50
|
+
7. **Look for patterns**: retries, skip, quarantine, fixtures, factories, seeds, circuit breakers, fallbacks
|
|
51
|
+
8. **Examine the ratio** unit/integ/e2e (test pyramid)
|
|
52
|
+
9. **Assign a level** per question with a justification and a confidence level
|
|
53
|
+
10. **Mark N/A** the questions non-applicable to the project profile
|
|
54
|
+
11. **List the non-auditable items** requiring an interview or external access
|
|
55
|
+
12. **Produce the report** in the standard format
|
|
56
|
+
|
|
57
|
+
## Grids by section
|
|
58
|
+
|
|
59
|
+
Each file contains the statement, the items to analyze/check, the bash commands and the tables of levels 0-4.
|
|
60
|
+
|
|
61
|
+
| Section | Reference | Questions | Count |
|
|
62
|
+
|---------|-----------|-----------|-------|
|
|
63
|
+
| Contract testing | [reference/tests-contractuels.md](reference/tests-contractuels.md) | te-01, te-01a | 2 |
|
|
64
|
+
| Golden dataset | [reference/golden-dataset.md](reference/golden-dataset.md) | te-02, te-02a | 2 |
|
|
65
|
+
| E2E & flaky tests | [reference/tests-e2e-flaky.md](reference/tests-e2e-flaky.md) | te-03, te-03a | 2 |
|
|
66
|
+
| Resilience & chaos | [reference/resilience-chaos.md](reference/resilience-chaos.md) | te-04, te-05, te-06 | 3 |
|
|
67
|
+
| Unit, integration & CI tests | [reference/tests-unit-integ-ci.md](reference/tests-unit-integ-ci.md) | te-07, te-08, te-09, te-09a | 4 |
|
|
68
|
+
|
|
69
|
+
## Confidence levels
|
|
70
|
+
|
|
71
|
+
- **high**: verified by an executed command or direct file read
|
|
72
|
+
- **medium**: inferred from structure/config but not deeply verified
|
|
73
|
+
- **low**: requires an interview or external access (coverage dashboard, historical reports, team interview)
|
|
74
|
+
|
|
75
|
+
## Output format
|
|
76
|
+
|
|
77
|
+
```markdown
|
|
78
|
+
## Testing & Validation - Global score: X.X/4 (Y questions scored out of Z)
|
|
79
|
+
|
|
80
|
+
### Summary
|
|
81
|
+
[2-3 sentences summarizing testing maturity]
|
|
82
|
+
|
|
83
|
+
### Detail per question
|
|
84
|
+
|
|
85
|
+
| Code | Question | Criticality | Level | Confidence | Justification |
|
|
86
|
+
|------|----------|-------------|-------|------------|---------------|
|
|
87
|
+
| te-01 | Contract testing with third-party APIs | could | X | medium | ... |
|
|
88
|
+
| te-01a | Coverage of tested contracts | could | X | low | ... |
|
|
89
|
+
| te-02 | Golden dataset | should | X | high | ... |
|
|
90
|
+
| te-02a | Golden dataset maintenance | should | X | medium | ... |
|
|
91
|
+
| te-03 | Full-injection E2E tests | should | X | high | ... |
|
|
92
|
+
| te-03a | Flaky test management | should | X | high | ... |
|
|
93
|
+
| te-04 | Chaos testing | could | X | low | ... |
|
|
94
|
+
| te-05 | Third-party API outage simulation | should | X | medium | ... |
|
|
95
|
+
| te-06 | Historical batch replay | could | X | low | ... |
|
|
96
|
+
| te-07 | Unit test structure | must | X | high | ... |
|
|
97
|
+
| te-08 | Integration test structure | must | X | high | ... |
|
|
98
|
+
| te-09 | Test execution in CI | must | X | high | ... |
|
|
99
|
+
| te-09a | Test feedback loop | should | X | medium | ... |
|
|
100
|
+
|
|
101
|
+
### Strengths
|
|
102
|
+
- ...
|
|
103
|
+
|
|
104
|
+
### Weaknesses
|
|
105
|
+
- ...
|
|
106
|
+
|
|
107
|
+
### Recommendations
|
|
108
|
+
1. [MUST] ...
|
|
109
|
+
2. [SHOULD] ...
|
|
110
|
+
3. [COULD] ...
|
|
111
|
+
|
|
112
|
+
### Non-auditable items
|
|
113
|
+
- [te-XX] Description - Reason (requires interview / dashboard access / log access / field observation / ...)
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
## Exit condition
|
|
117
|
+
|
|
118
|
+
- [ ] Test frameworks detected, files counted by type
|
|
119
|
+
- [ ] All 13 questions evaluated (or justified N/A)
|
|
120
|
+
- [ ] Conditional sub-questions evaluated if parent ≥ 2 (te-01a, te-02a, te-03a, te-09a)
|
|
121
|
+
- [ ] Verification commands executed for every scored question
|
|
122
|
+
- [ ] Level (0-4) + confidence + justification per question
|
|
123
|
+
- [ ] Domain score calculated (average excluding N/A, "X scored out of Y")
|
|
124
|
+
- [ ] Report produced in the output format
|
|
125
|
+
- [ ] Non-auditable items listed
|
|
126
|
+
|
|
127
|
+
## Changelog
|
|
128
|
+
|
|
129
|
+
- 1.0.1 (2026-07-20) - scoring aligned with scoring-model.md: flat mean only; criticality kept for remediation priority, weights removed from the score
|
|
130
|
+
|
|
131
|
+
- 1.0.0 (2026-06-19) - initial versioned release + state-of-the-art enrichment (routing, context, protocol, traps, exit condition); audit grids in reference/ (progressive disclosure)
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
# Golden dataset section
|
|
2
|
+
|
|
3
|
+
Questions: te-02 (Golden dataset), te-02a (Golden dataset maintenance).
|
|
4
|
+
|
|
5
|
+
## Table of contents
|
|
6
|
+
|
|
7
|
+
- [te-02 - Golden dataset (reference dataset)](#te-02--golden-dataset-reference-dataset--should)
|
|
8
|
+
- [te-02a - Golden dataset maintenance](#te-02a--golden-dataset-maintenance--should)
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
### te-02 - Golden dataset (reference dataset) - `should`
|
|
13
|
+
|
|
14
|
+
**Analyze:** Fixtures, seeds, factories, test-data files, seeding scripts
|
|
15
|
+
|
|
16
|
+
**Commands:**
|
|
17
|
+
- `find . -path "*/fixtures/*" -o -path "*/seeds/*" -o -path "*/factories/*" -o -path "*/__fixtures__/*" 2>/dev/null | grep -v node_modules | head -10` → test-data files
|
|
18
|
+
- `grep -ri "factory\|faker\|seed\|fixture" package.json 2>/dev/null` → data-generation libraries
|
|
19
|
+
- `find . -name "*.fixture.*" -o -name "*.seed.*" -o -name "*.factory.*" 2>/dev/null | grep -v node_modules | head -10` → fixture/seed/factory files
|
|
20
|
+
- `grep -ri "anonymi\|mask\|obfuscat" scripts/ docs/ 2>/dev/null | head -5` → production data anonymization
|
|
21
|
+
- `grep -ri "golden\|reference.*data\|dataset" docs/ README.md 2>/dev/null | head -5` → golden dataset documentation
|
|
22
|
+
|
|
23
|
+
**Check:**
|
|
24
|
+
- Reference dataset for the tests
|
|
25
|
+
- Data representative of production (anonymized)
|
|
26
|
+
- Versioning of the golden dataset alongside the code
|
|
27
|
+
- Edge-case coverage in the data
|
|
28
|
+
|
|
29
|
+
**Levels:**
|
|
30
|
+
| Level | Criteria |
|
|
31
|
+
|-------|----------|
|
|
32
|
+
| 0 | No golden dataset. Tests with ad-hoc data. |
|
|
33
|
+
| 1 | A few manually maintained test files. Not versioned. |
|
|
34
|
+
| 2 | Versioned golden dataset for the main cases. Partial coverage. |
|
|
35
|
+
| 3 | Complete dataset covering edge cases. Versioned with the code. Regular refresh. |
|
|
36
|
+
| 4 | Dataset generated automatically from production (anonymized). Coverage metrics. Mutation testing. |
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
### te-02a - Golden dataset maintenance - `should`
|
|
41
|
+
|
|
42
|
+
**Condition:** Applies if te-02 ≥ 2 (golden dataset in place).
|
|
43
|
+
|
|
44
|
+
**Analyze:** Test-data update process, refresh pipeline, bug-to-test workflow
|
|
45
|
+
|
|
46
|
+
**Commands:**
|
|
47
|
+
- `git log --oneline --all -- "*fixtures*" "*seeds*" "*factories*" "*__fixtures__*" 2>/dev/null | head -10` → fixture modification history
|
|
48
|
+
- `grep -ri "refresh\|update.*fixture\|update.*seed\|regenerat" scripts/ Makefile 2>/dev/null | head -5` → data-update scripts
|
|
49
|
+
- `find . -path "*/fixtures/*" -o -path "*/seeds/*" 2>/dev/null | grep -v node_modules | xargs ls -lt 2>/dev/null | head -10` → fixture modification dates
|
|
50
|
+
- `grep -ri "anonymi\|tonic\|delphix\|k.anonym" . --include="*.sh" --include="*.py" --include="*.js" 2>/dev/null | head -5` → anonymization pipeline
|
|
51
|
+
|
|
52
|
+
**Check:**
|
|
53
|
+
- Documented dataset-update process
|
|
54
|
+
- Update frequency (quarterly at minimum)
|
|
55
|
+
- Automated refresh pipeline with anonymization
|
|
56
|
+
- Bug-to-test workflow (prod bugs → new test cases)
|
|
57
|
+
|
|
58
|
+
**Levels:**
|
|
59
|
+
| Level | Criteria |
|
|
60
|
+
|-------|----------|
|
|
61
|
+
| 0 | No maintenance. Stale dataset. |
|
|
62
|
+
| 1 | Manual updates whenever someone thinks of it. |
|
|
63
|
+
| 2 | Documented update process. Quarterly. |
|
|
64
|
+
| 3 | Automated refresh pipeline. Anonymization. Representativeness validation. |
|
|
65
|
+
| 4 | Continuous data generation. Feedback loop with prod bugs. Coverage analysis. |
|