@ghostbyte/agent-docs-kit 4.0.2 → 5.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +32 -0
- package/README.md +43 -37
- package/SECURITY.md +11 -0
- package/content/catalog.json +191 -22
- package/content/documentation/01-system-documentation.md +13 -0
- package/content/documentation/02-project-governance.md +13 -0
- package/content/documentation/03-development-records.md +13 -0
- package/content/documentation/04-testing-evidence.md +13 -0
- package/content/documentation/05-operations-runbooks.md +13 -0
- package/content/documentation/06-future-proposals.md +13 -0
- package/content/documentation/07-archive-governance.md +13 -0
- package/content/documentation/full-reference.md +24 -0
- package/content/engineering/01-query-performance.md +17 -27
- package/content/engineering/02-networking-fundamentals.md +17 -27
- package/content/engineering/03-rate-limiting-algorithms.md +18 -28
- package/content/engineering/04-production-debugging.md +17 -27
- package/content/engineering/05-code-review-practices.md +17 -27
- package/content/engineering/06-data-modeling.md +17 -27
- package/content/engineering/07-cloud-cost-saving.md +17 -27
- package/content/engineering/08-testing-types.md +17 -27
- package/content/engineering/09-message-queues.md +17 -27
- package/content/engineering/10-api-design-compatibility.md +14 -0
- package/content/engineering/11-reliability-slos.md +14 -0
- package/content/engineering/12-observability-telemetry.md +14 -0
- package/content/engineering/13-deployment-change-safety.md +14 -0
- package/content/engineering/14-capacity-concurrency.md +14 -0
- package/content/engineering/full-reference.md +24 -144
- package/content/security/01-design-architecture.md +11 -19
- package/content/security/02-local-development.md +13 -21
- package/content/security/03-database-backend.md +18 -26
- package/content/security/04-api-layer.md +23 -31
- package/content/security/05-frontend-ui.md +16 -24
- package/content/security/06-application-security-testing.md +22 -28
- package/content/security/07-cicd-release-engineering.md +13 -21
- package/content/security/08-cloud-infrastructure.md +17 -25
- package/content/security/09-edge-cdn-dns.md +16 -24
- package/content/security/10-release-rollout.md +13 -21
- package/content/security/11-post-release-monitoring-ir.md +15 -23
- package/content/security/12-ongoing-operations.md +24 -32
- package/content/security/full-reference.md +30 -3163
- package/docs/ARCHITECTURE.md +31 -92
- package/docs/CONTENT_MAINTENANCE.md +5 -1
- package/docs/GITHUB_AND_NPM_SETUP.md +1 -1
- package/docs/RELEASE_PROCESS.md +5 -5
- package/package.json +12 -5
- package/scripts/build-site.mjs +13 -0
- package/scripts/build-skill.mjs +24 -0
- package/scripts/check-links.mjs +56 -0
- package/scripts/check-review-dates.mjs +21 -0
- package/scripts/preview-site.mjs +43 -0
- package/scripts/smoke-package.mjs +58 -0
- package/scripts/validate-content.mjs +85 -0
- package/site/app.js +121 -0
- package/site/index.html +346 -0
- package/site/styles.css +159 -0
- package/skills/manage-project-quality/SKILL.md +18 -2
- package/skills/manage-project-quality/assets/documentation-report.md +3 -3
- package/skills/manage-project-quality/assets/engineering-report.md +3 -3
- package/skills/manage-project-quality/assets/security-report.md +3 -3
- package/skills/manage-project-quality/references/documentation-01-system-documentation.md +13 -0
- package/skills/manage-project-quality/references/documentation-02-project-governance.md +13 -0
- package/skills/manage-project-quality/references/documentation-03-development-records.md +13 -0
- package/skills/manage-project-quality/references/documentation-04-testing-evidence.md +13 -0
- package/skills/manage-project-quality/references/documentation-05-operations-runbooks.md +13 -0
- package/skills/manage-project-quality/references/documentation-06-future-proposals.md +13 -0
- package/skills/manage-project-quality/references/documentation-07-archive-governance.md +13 -0
- package/skills/manage-project-quality/references/documentation-full-reference.md +24 -0
- package/skills/manage-project-quality/references/documentation-governance.md +2 -2
- package/skills/manage-project-quality/references/engineering-01-query-performance.md +17 -27
- package/skills/manage-project-quality/references/engineering-02-networking-fundamentals.md +17 -27
- package/skills/manage-project-quality/references/engineering-03-rate-limiting-algorithms.md +18 -28
- package/skills/manage-project-quality/references/engineering-04-production-debugging.md +17 -27
- package/skills/manage-project-quality/references/engineering-05-code-review-practices.md +17 -27
- package/skills/manage-project-quality/references/engineering-06-data-modeling.md +17 -27
- package/skills/manage-project-quality/references/engineering-07-cloud-cost-saving.md +17 -27
- package/skills/manage-project-quality/references/engineering-08-testing-types.md +17 -27
- package/skills/manage-project-quality/references/engineering-09-message-queues.md +17 -27
- package/skills/manage-project-quality/references/engineering-10-api-design-compatibility.md +14 -0
- package/skills/manage-project-quality/references/engineering-11-reliability-slos.md +14 -0
- package/skills/manage-project-quality/references/engineering-12-observability-telemetry.md +14 -0
- package/skills/manage-project-quality/references/engineering-13-deployment-change-safety.md +14 -0
- package/skills/manage-project-quality/references/engineering-14-capacity-concurrency.md +14 -0
- package/skills/manage-project-quality/references/engineering-full-reference.md +24 -144
- package/skills/manage-project-quality/references/engineering-protocol.md +1 -1
- package/skills/manage-project-quality/references/security-01-design-architecture.md +11 -19
- package/skills/manage-project-quality/references/security-02-local-development.md +13 -21
- package/skills/manage-project-quality/references/security-03-database-backend.md +18 -26
- package/skills/manage-project-quality/references/security-04-api-layer.md +23 -31
- package/skills/manage-project-quality/references/security-05-frontend-ui.md +16 -24
- package/skills/manage-project-quality/references/security-06-application-security-testing.md +22 -28
- package/skills/manage-project-quality/references/security-07-cicd-release-engineering.md +13 -21
- package/skills/manage-project-quality/references/security-08-cloud-infrastructure.md +17 -25
- package/skills/manage-project-quality/references/security-09-edge-cdn-dns.md +16 -24
- package/skills/manage-project-quality/references/security-10-release-rollout.md +13 -21
- package/skills/manage-project-quality/references/security-11-post-release-monitoring-ir.md +15 -23
- package/skills/manage-project-quality/references/security-12-ongoing-operations.md +24 -32
- package/skills/manage-project-quality/references/security-full-reference.md +30 -3163
- package/skills/manage-project-quality/scripts/check-report.mjs +67 -16
- package/src/api/index.mjs +4 -0
- package/src/cli/arguments.mjs +27 -3
- package/src/cli/help.mjs +53 -18
- package/src/cli/main.mjs +66 -9
- package/src/commands/audit.mjs +146 -31
- package/src/commands/doctor.mjs +1 -1
- package/src/commands/guide.mjs +12 -0
- package/src/commands/list.mjs +1 -1
- package/src/commands/setup.mjs +9 -4
- package/src/commands/skill.mjs +1 -0
- package/src/commands/template.mjs +32 -0
- package/src/commands/validate.mjs +36 -4
- package/src/commands/website.mjs +26 -0
- package/src/core/config.mjs +61 -0
- package/src/core/files.mjs +47 -6
- package/templates/adr.md +20 -0
- package/templates/change-record.md +18 -0
- package/templates/incident-review.md +20 -0
- package/templates/runbook.md +18 -0
- package/templates/test-evidence.md +18 -0
- package/tests/cli.test.mjs +226 -0
- package/tests/content.test.mjs +18 -0
|
@@ -1,48 +1,38 @@
|
|
|
1
1
|
# Query Performance & Slow Query Diagnosis
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
## Engineering Topic 1: Query Performance & Slow Query Diagnosis
|
|
4
4
|
|
|
5
5
|
**When to run this:** When a query, endpoint, or report is running slower than expected, or before shipping a new query against a table that will grow.
|
|
6
6
|
|
|
7
7
|
**Goal of this pass:** Work through the standard diagnostic order for a slow query instead of guessing at fixes — read the plan first, fix the cheapest thing, and don't reach for denormalization until indexing is exhausted.
|
|
8
8
|
|
|
9
|
-
##
|
|
9
|
+
## How to use this reference
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|---|---|
|
|
13
|
-
| `CHECKLIST.md` | The reference list of concepts for this topic — what each one is, why it matters, and the self-check question to ask. Doesn't get filled in; copy `RESULTS_TEMPLATE.md` to record a real pass. |
|
|
14
|
-
| `RESULTS_TEMPLATE.md` | Fillable version of the same list, with a Status column and notes. Copy this file per pass (e.g. `RESULTS_2026-08-29.md`) so history isn't overwritten. |
|
|
15
|
-
|
|
16
|
-
Full source list lives in `../ENGINEERING_REFERENCE.md`. Unlike
|
|
17
|
-
`../../security-gate/phases/`, this is knowledge/practice review, not pass/fail security
|
|
18
|
-
testing — there's no severity rating, and "Fail" isn't the right framing.
|
|
19
|
-
Use the Status options in `RESULTS_TEMPLATE.md` instead: Applied, Partial,
|
|
20
|
-
Should adopt, or Not applicable.
|
|
11
|
+
Use each stable practice ID in a generated engineering worksheet. Record concrete evidence and choose Applied, Partial, Recommended, Unverified, or N/A. Preserve each dated worksheet as review history.
|
|
21
12
|
|
|
22
13
|
## Wrap-up
|
|
23
14
|
|
|
24
15
|
- This isn't a gate — nothing here blocks a release the way
|
|
25
|
-
Critical/High security items in
|
|
16
|
+
Critical/High security items in security phase reviews do.
|
|
26
17
|
- Total concepts in this topic: 10
|
|
27
18
|
- Useful as an onboarding read-through, a periodic self-audit, or context
|
|
28
19
|
to hand an agent before it reviews related code.
|
|
29
20
|
|
|
30
21
|
---
|
|
31
22
|
|
|
32
|
-
|
|
23
|
+
## Concepts — Topic 1: Query Performance & Slow Query Diagnosis
|
|
33
24
|
|
|
34
|
-
|
|
35
|
-
this same folder to actually record a pass.
|
|
25
|
+
Use the applicable practices below to populate the worksheet findings table.
|
|
36
26
|
|
|
37
|
-
|
|
|
27
|
+
| Practice ID | Practice | Why it matters | Evidence question |
|
|
38
28
|
|---|---|---|---|
|
|
39
|
-
|
|
|
40
|
-
|
|
|
41
|
-
|
|
|
42
|
-
|
|
|
43
|
-
|
|
|
44
|
-
|
|
|
45
|
-
|
|
|
46
|
-
|
|
|
47
|
-
|
|
|
48
|
-
| 10 | Update table statistics | A stale planner (out-of-date row-count/distribution stats) can pick a bad plan even with the right indexes present. | Are table statistics refreshed on a schedule (ANALYZE / equivalent), especially after large data changes? |
|
|
29
|
+
| ENG-01-01 | Start with the explain plan | Reveals whether the DB is scanning, sorting, or joining in a way that doesn't match intent — guessing at fixes without this wastes time. | Have you run EXPLAIN (or EXPLAIN ANALYZE) on the slow query and read where the time actually goes? |
|
|
30
|
+
| ENG-01-02 | Add the missing index | Full table scans get linearly worse as data grows; the right index turns O(n) into O(log n). | Does the WHERE/JOIN/ORDER BY column set have a supporting index? |
|
|
31
|
+
| ENG-01-03 | Stop selecting * | Pulling unused columns wastes I/O and network, and can defeat a covering index. | Does the query request only the columns actually used downstream? |
|
|
32
|
+
| ENG-01-04 | Fix N+1 queries | One query per row in a loop turns a single request into hundreds; batching/joining collapses it back to one or few. | Does any code path issue a query inside a loop over another query's results? |
|
|
33
|
+
| ENG-01-05 | Use a covering index | An index that includes all selected columns lets the DB answer from the index alone, skipping the table lookup. | For hot read queries, does the index include every column the query selects? |
|
|
34
|
+
| ENG-01-06 | Cache repeated queries | Identical, expensive, low-volatility queries don't need to hit the DB every time. | Are the same expensive read queries re-run frequently with unchanged results, and if so, are they cached? |
|
|
35
|
+
| ENG-01-07 | Prune partitions | Partition pruning lets the DB skip irrelevant partitions entirely instead of scanning the whole table. | For large time-series/partitioned tables, does the query filter on the partition key so pruning can kick in? |
|
|
36
|
+
| ENG-01-08 | Denormalize for reads | Trading write complexity/redundancy for fewer joins can be the right call once indexing alone isn't enough. | Has indexing been exhausted before reaching for denormalization, and is the redundancy kept in sync deliberately? |
|
|
37
|
+
| ENG-01-09 | Batch your writes | Many small write round-trips cost more in overhead than one larger batched write. | Are writes that happen together (e.g. in a loop or transaction) batched instead of issued one at a time? |
|
|
38
|
+
| ENG-01-10 | Update table statistics | A stale planner (out-of-date row-count/distribution stats) can pick a bad plan even with the right indexes present. | Are table statistics refreshed on a schedule (ANALYZE / equivalent), especially after large data changes? |
|
|
@@ -1,48 +1,38 @@
|
|
|
1
1
|
# Networking Fundamentals
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
## Engineering Topic 2: Networking Fundamentals
|
|
4
4
|
|
|
5
5
|
**When to run this:** When debugging connectivity, latency, or TLS issues, or before making infra decisions (load balancer, CDN, multi-region) that depend on understanding the network path.
|
|
6
6
|
|
|
7
7
|
**Goal of this pass:** Confirm the team can reason about what happens between a client request and a server response — not memorize trivia, but recognize which layer a given production issue actually lives in.
|
|
8
8
|
|
|
9
|
-
##
|
|
9
|
+
## How to use this reference
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|---|---|
|
|
13
|
-
| `CHECKLIST.md` | The reference list of concepts for this topic — what each one is, why it matters, and the self-check question to ask. Doesn't get filled in; copy `RESULTS_TEMPLATE.md` to record a real pass. |
|
|
14
|
-
| `RESULTS_TEMPLATE.md` | Fillable version of the same list, with a Status column and notes. Copy this file per pass (e.g. `RESULTS_2026-08-29.md`) so history isn't overwritten. |
|
|
15
|
-
|
|
16
|
-
Full source list lives in `../ENGINEERING_REFERENCE.md`. Unlike
|
|
17
|
-
`../../security-gate/phases/`, this is knowledge/practice review, not pass/fail security
|
|
18
|
-
testing — there's no severity rating, and "Fail" isn't the right framing.
|
|
19
|
-
Use the Status options in `RESULTS_TEMPLATE.md` instead: Applied, Partial,
|
|
20
|
-
Should adopt, or Not applicable.
|
|
11
|
+
Use each stable practice ID in a generated engineering worksheet. Record concrete evidence and choose Applied, Partial, Recommended, Unverified, or N/A. Preserve each dated worksheet as review history.
|
|
21
12
|
|
|
22
13
|
## Wrap-up
|
|
23
14
|
|
|
24
15
|
- This isn't a gate — nothing here blocks a release the way
|
|
25
|
-
Critical/High security items in
|
|
16
|
+
Critical/High security items in security phase reviews do.
|
|
26
17
|
- Total concepts in this topic: 10
|
|
27
18
|
- Useful as an onboarding read-through, a periodic self-audit, or context
|
|
28
19
|
to hand an agent before it reviews related code.
|
|
29
20
|
|
|
30
21
|
---
|
|
31
22
|
|
|
32
|
-
|
|
23
|
+
## Concepts — Topic 2: Networking Fundamentals
|
|
33
24
|
|
|
34
|
-
|
|
35
|
-
this same folder to actually record a pass.
|
|
25
|
+
Use the applicable practices below to populate the worksheet findings table.
|
|
36
26
|
|
|
37
|
-
|
|
|
27
|
+
| Practice ID | Practice | Why it matters | Evidence question |
|
|
38
28
|
|---|---|---|---|
|
|
39
|
-
|
|
|
40
|
-
|
|
|
41
|
-
|
|
|
42
|
-
|
|
|
43
|
-
|
|
|
44
|
-
|
|
|
45
|
-
|
|
|
46
|
-
|
|
|
47
|
-
|
|
|
48
|
-
| 10 | BGP routing | Explains internet-scale outages and route leaks — usually outside your control, but recognizable when it's the actual cause. | When a broad, upstream-looking outage occurs, is BGP/routing considered as a possible cause rather than assumed to be your infra? |
|
|
29
|
+
| ENG-02-01 | DNS resolution | Wrong TTLs or stale records cause outages/slow failover that look like application bugs. | Do you know your DNS TTLs and what happens on a DNS provider outage or a failed cutover? |
|
|
30
|
+
| ENG-02-02 | TCP handshake | SYN/SYN-ACK/ACK latency adds to every new connection; connection reuse avoids paying it repeatedly. | Are connections being reused (keep-alive/pooling) where the handshake cost matters? |
|
|
31
|
+
| ENG-02-03 | TLS handshake | Adds round trips and CPU cost on top of TCP; session resumption and TLS version choice affect both. | Is TLS session resumption enabled, and is the minimum TLS version intentionally set? |
|
|
32
|
+
| ENG-02-04 | NAT | Explains why a server sees one client IP for many real users, and why some P2P/inbound-connection patterns don't work as expected. | Does any rate-limiting or per-client logic depend on client IP, and does it account for NAT'd clients sharing an IP? |
|
|
33
|
+
| ENG-02-05 | Load balancer types | L4 vs L7 balancers differ in what they can inspect and route on (raw TCP vs HTTP headers/paths). | Does the current load balancer type actually support the routing/health-check behavior the app needs? |
|
|
34
|
+
| ENG-02-06 | CDN edge caching | Wrong cache-control headers cause either stale content served for too long or origin overload from no caching at all. | Are cache-control/CDN cache-key rules deliberately set per route, not left at CDN defaults? |
|
|
35
|
+
| ENG-02-07 | Packet loss & retransmit | Retransmits show up as latency and timeouts that look application-side but are actually network-layer. | When debugging intermittent slowness, has packet loss been ruled out before blaming application code? |
|
|
36
|
+
| ENG-02-08 | MTU / fragmentation | Oversized packets get fragmented or dropped, which shows up as weird partial-failure behavior, especially over VPNs/tunnels. | Has MTU mismatch been considered for any VPN, tunnel, or container network path in play? |
|
|
37
|
+
| ENG-02-09 | Keep-alive | Persistent connections avoid the handshake cost per request but tie up server resources if left open too long. | Are keep-alive timeouts tuned deliberately on both client and server, not left at framework defaults? |
|
|
38
|
+
| ENG-02-10 | BGP routing | Explains internet-scale outages and route leaks — usually outside your control, but recognizable when it's the actual cause. | When a broad, upstream-looking outage occurs, is BGP/routing considered as a possible cause rather than assumed to be your infra? |
|
|
@@ -1,48 +1,38 @@
|
|
|
1
1
|
# Rate Limiting Algorithm Choices
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
## Engineering Topic 3: Rate Limiting Algorithm Choices
|
|
4
4
|
|
|
5
|
-
**When to run this:** When designing or reviewing a rate limiter
|
|
5
|
+
**When to run this:** When designing or reviewing a rate limiter. This topic focuses on selecting the appropriate algorithm; the API security phase separately verifies that protective limits exist and are effective.
|
|
6
6
|
|
|
7
7
|
**Goal of this pass:** Make sure the chosen algorithm actually matches the traffic pattern being protected against, instead of defaulting to whatever's easiest to implement.
|
|
8
8
|
|
|
9
|
-
##
|
|
9
|
+
## How to use this reference
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|---|---|
|
|
13
|
-
| `CHECKLIST.md` | The reference list of concepts for this topic — what each one is, why it matters, and the self-check question to ask. Doesn't get filled in; copy `RESULTS_TEMPLATE.md` to record a real pass. |
|
|
14
|
-
| `RESULTS_TEMPLATE.md` | Fillable version of the same list, with a Status column and notes. Copy this file per pass (e.g. `RESULTS_2026-08-29.md`) so history isn't overwritten. |
|
|
15
|
-
|
|
16
|
-
Full source list lives in `../ENGINEERING_REFERENCE.md`. Unlike
|
|
17
|
-
`../../security-gate/phases/`, this is knowledge/practice review, not pass/fail security
|
|
18
|
-
testing — there's no severity rating, and "Fail" isn't the right framing.
|
|
19
|
-
Use the Status options in `RESULTS_TEMPLATE.md` instead: Applied, Partial,
|
|
20
|
-
Should adopt, or Not applicable.
|
|
11
|
+
Use each stable practice ID in a generated engineering worksheet. Record concrete evidence and choose Applied, Partial, Recommended, Unverified, or N/A. Preserve each dated worksheet as review history.
|
|
21
12
|
|
|
22
13
|
## Wrap-up
|
|
23
14
|
|
|
24
15
|
- This isn't a gate — nothing here blocks a release the way
|
|
25
|
-
Critical/High security items in
|
|
16
|
+
Critical/High security items in security phase reviews do.
|
|
26
17
|
- Total concepts in this topic: 10
|
|
27
18
|
- Useful as an onboarding read-through, a periodic self-audit, or context
|
|
28
19
|
to hand an agent before it reviews related code.
|
|
29
20
|
|
|
30
21
|
---
|
|
31
22
|
|
|
32
|
-
|
|
23
|
+
## Concepts — Topic 3: Rate Limiting Algorithm Choices
|
|
33
24
|
|
|
34
|
-
|
|
35
|
-
this same folder to actually record a pass.
|
|
25
|
+
Use the applicable practices below to populate the worksheet findings table.
|
|
36
26
|
|
|
37
|
-
|
|
|
27
|
+
| Practice ID | Practice | Why it matters | Evidence question |
|
|
38
28
|
|---|---|---|---|
|
|
39
|
-
|
|
|
40
|
-
|
|
|
41
|
-
|
|
|
42
|
-
|
|
|
43
|
-
|
|
|
44
|
-
|
|
|
45
|
-
|
|
|
46
|
-
|
|
|
47
|
-
|
|
|
48
|
-
| 10 | Back pressure | Signals upstream to slow down instead of silently dropping or queuing indefinitely when a system is overloaded. | When downstream capacity is exceeded, does the system push back (429/503, queue limits) rather than accepting unbounded work? |
|
|
29
|
+
| ENG-03-01 | Fixed window | Simple to implement but allows a burst of 2x the limit right at the window boundary. | If fixed window is used, is the boundary-burst behavior acceptable for this endpoint? |
|
|
30
|
+
| ENG-03-02 | Sliding window log | Precise (tracks every request timestamp) but memory-heavy at scale. | If precision matters and volume is low/medium, is sliding window log a better fit than fixed window? |
|
|
31
|
+
| ENG-03-03 | Sliding window counter | Approximates the sliding log with much less memory by weighting two adjacent fixed windows. | For high-volume endpoints needing better accuracy than fixed window, is sliding window counter being used instead of the log variant? |
|
|
32
|
+
| ENG-03-04 | Token bucket | Allows controlled bursts up to the bucket size while enforcing a steady refill rate — good for bursty-but-bounded traffic. | Does the traffic pattern need to allow short bursts, and if so, is token bucket the chosen algorithm? |
|
|
33
|
+
| ENG-03-05 | Leaky bucket | Smooths bursty input into a constant output rate — good for protecting a downstream system that can't handle spikes at all. | Is there a downstream system that needs a strictly constant request rate, and if so, is leaky bucket used to enforce it? |
|
|
34
|
+
| ENG-03-06 | Distributed rate limiting | A per-instance limiter is meaningless behind a load balancer with multiple app servers — limits get multiplied by instance count. | If the app runs on more than one instance, is the rate limit state shared (e.g. via Redis) rather than per-instance? |
|
|
35
|
+
| ENG-03-07 | Client-side throttling | Reduces wasted requests and improves UX, but is never sufficient alone since a client can be bypassed or malicious. | Does the client throttle its own request rate, and is server-side enforcement still present as the actual control? |
|
|
36
|
+
| ENG-03-08 | Adaptive rate limiting | Static limits can be too strict in quiet periods or too loose under load; adaptive limits adjust to current system health. | Does the limiter adjust based on current load/error rate, or is it a single static number regardless of system state? |
|
|
37
|
+
| ENG-03-09 | Concurrency limiting | Limits in-flight requests, not just requests-per-second — protects against slow requests piling up and exhausting resources. | Is there a cap on concurrent in-flight requests per client, separate from the requests-per-second limit? |
|
|
38
|
+
| ENG-03-10 | Back pressure | Signals upstream to slow down instead of silently dropping or queuing indefinitely when a system is overloaded. | When downstream capacity is exceeded, does the system push back (429/503, queue limits) rather than accepting unbounded work? |
|
|
@@ -1,48 +1,38 @@
|
|
|
1
1
|
# Production Debugging
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
## Engineering Topic 4: Debugging Production Issues
|
|
4
4
|
|
|
5
5
|
**When to run this:** Before an incident happens — this is about having the right tooling and habits in place, not what to do during a live fire.
|
|
6
6
|
|
|
7
7
|
**Goal of this pass:** Confirm the team has the observability primitives needed to actually diagnose a production issue quickly, not just react to alerts.
|
|
8
8
|
|
|
9
|
-
##
|
|
9
|
+
## How to use this reference
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|---|---|
|
|
13
|
-
| `CHECKLIST.md` | The reference list of concepts for this topic — what each one is, why it matters, and the self-check question to ask. Doesn't get filled in; copy `RESULTS_TEMPLATE.md` to record a real pass. |
|
|
14
|
-
| `RESULTS_TEMPLATE.md` | Fillable version of the same list, with a Status column and notes. Copy this file per pass (e.g. `RESULTS_2026-08-29.md`) so history isn't overwritten. |
|
|
15
|
-
|
|
16
|
-
Full source list lives in `../ENGINEERING_REFERENCE.md`. Unlike
|
|
17
|
-
`../../security-gate/phases/`, this is knowledge/practice review, not pass/fail security
|
|
18
|
-
testing — there's no severity rating, and "Fail" isn't the right framing.
|
|
19
|
-
Use the Status options in `RESULTS_TEMPLATE.md` instead: Applied, Partial,
|
|
20
|
-
Should adopt, or Not applicable.
|
|
11
|
+
Use each stable practice ID in a generated engineering worksheet. Record concrete evidence and choose Applied, Partial, Recommended, Unverified, or N/A. Preserve each dated worksheet as review history.
|
|
21
12
|
|
|
22
13
|
## Wrap-up
|
|
23
14
|
|
|
24
15
|
- This isn't a gate — nothing here blocks a release the way
|
|
25
|
-
Critical/High security items in
|
|
16
|
+
Critical/High security items in security phase reviews do.
|
|
26
17
|
- Total concepts in this topic: 10
|
|
27
18
|
- Useful as an onboarding read-through, a periodic self-audit, or context
|
|
28
19
|
to hand an agent before it reviews related code.
|
|
29
20
|
|
|
30
21
|
---
|
|
31
22
|
|
|
32
|
-
|
|
23
|
+
## Concepts — Topic 4: Debugging Production Issues
|
|
33
24
|
|
|
34
|
-
|
|
35
|
-
this same folder to actually record a pass.
|
|
25
|
+
Use the applicable practices below to populate the worksheet findings table.
|
|
36
26
|
|
|
37
|
-
|
|
|
27
|
+
| Practice ID | Practice | Why it matters | Evidence question |
|
|
38
28
|
|---|---|---|---|
|
|
39
|
-
|
|
|
40
|
-
|
|
|
41
|
-
|
|
|
42
|
-
|
|
|
43
|
-
|
|
|
44
|
-
|
|
|
45
|
-
|
|
|
46
|
-
|
|
|
47
|
-
|
|
|
48
|
-
| 10 | Postmortems | Without a blameless, written record, the same class of incident tends to recur because the lesson isn't institutionalized. | Does every significant incident get a blameless postmortem with concrete follow-up actions that are actually tracked to completion? |
|
|
29
|
+
| ENG-04-01 | Correlation IDs | Without a shared ID, tracing one user's request across multiple services means grepping logs by timestamp and guessing. | Does every request get a correlation/request ID that's propagated across all services it touches and included in every log line? |
|
|
30
|
+
| ENG-04-02 | Distributed tracing | Shows exactly where time is spent across a multi-service call chain — impossible to reconstruct from logs alone at scale. | Is distributed tracing (e.g. OpenTelemetry) instrumented across service boundaries? |
|
|
31
|
+
| ENG-04-03 | Feature flag rollback | Turning off a bad feature via flag is faster and safer than a code rollback/redeploy under incident pressure. | Can the riskiest recent features be disabled via flag without a deploy? |
|
|
32
|
+
| ENG-04-04 | Canary analysis | Rolling out to a small subset first catches regressions before they hit 100% of traffic. | Are risky changes rolled out to a canary/subset with automated comparison against the baseline before full rollout? |
|
|
33
|
+
| ENG-04-05 | Replay traffic | Re-running real production requests against a fix confirms it actually resolves the issue, not just a synthetic test case. | Is there a way to replay real production traffic (or a sample) against a candidate fix before it ships? |
|
|
34
|
+
| ENG-04-06 | Heap dumps | A heap dump is often the only way to root-cause a memory leak instead of guessing from symptoms. | Is heap-dump capture available (manually or via crash trigger) for services with memory-related symptoms? |
|
|
35
|
+
| ENG-04-07 | Chaos testing | Deliberately injecting failure in a controlled way surfaces weaknesses before real, uncontrolled failure does. | Has chaos testing (killing instances, injecting latency/errors) been run against critical paths? |
|
|
36
|
+
| ENG-04-08 | Synthetic monitoring | Catches outages and regressions from the outside, the way a real user would experience them, independent of internal metrics. | Are synthetic checks running against critical user journeys from outside the infrastructure? |
|
|
37
|
+
| ENG-04-09 | Error budgets | Gives a quantified, agreed-upon threshold for how much unreliability is acceptable, so incident response isn't a judgment call every time. | Is there an agreed error budget per service, and does exceeding it trigger a defined response (e.g. freeze on new features)? |
|
|
38
|
+
| ENG-04-10 | Postmortems | Without a blameless, written record, the same class of incident tends to recur because the lesson isn't institutionalized. | Does every significant incident get a blameless postmortem with concrete follow-up actions that are actually tracked to completion? |
|
|
@@ -1,48 +1,38 @@
|
|
|
1
1
|
# Code Review Practices
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
## Engineering Topic 5: Code Review Practices
|
|
4
4
|
|
|
5
5
|
**When to run this:** Ongoing — use this to periodically check whether the team's actual review habits match its stated process, not just once.
|
|
6
6
|
|
|
7
7
|
**Goal of this pass:** Keep review effective and fast rather than either a rubber stamp or a bottleneck.
|
|
8
8
|
|
|
9
|
-
##
|
|
9
|
+
## How to use this reference
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|---|---|
|
|
13
|
-
| `CHECKLIST.md` | The reference list of concepts for this topic — what each one is, why it matters, and the self-check question to ask. Doesn't get filled in; copy `RESULTS_TEMPLATE.md` to record a real pass. |
|
|
14
|
-
| `RESULTS_TEMPLATE.md` | Fillable version of the same list, with a Status column and notes. Copy this file per pass (e.g. `RESULTS_2026-08-29.md`) so history isn't overwritten. |
|
|
15
|
-
|
|
16
|
-
Full source list lives in `../ENGINEERING_REFERENCE.md`. Unlike
|
|
17
|
-
`../../security-gate/phases/`, this is knowledge/practice review, not pass/fail security
|
|
18
|
-
testing — there's no severity rating, and "Fail" isn't the right framing.
|
|
19
|
-
Use the Status options in `RESULTS_TEMPLATE.md` instead: Applied, Partial,
|
|
20
|
-
Should adopt, or Not applicable.
|
|
11
|
+
Use each stable practice ID in a generated engineering worksheet. Record concrete evidence and choose Applied, Partial, Recommended, Unverified, or N/A. Preserve each dated worksheet as review history.
|
|
21
12
|
|
|
22
13
|
## Wrap-up
|
|
23
14
|
|
|
24
15
|
- This isn't a gate — nothing here blocks a release the way
|
|
25
|
-
Critical/High security items in
|
|
16
|
+
Critical/High security items in security phase reviews do.
|
|
26
17
|
- Total concepts in this topic: 10
|
|
27
18
|
- Useful as an onboarding read-through, a periodic self-audit, or context
|
|
28
19
|
to hand an agent before it reviews related code.
|
|
29
20
|
|
|
30
21
|
---
|
|
31
22
|
|
|
32
|
-
|
|
23
|
+
## Concepts — Topic 5: Code Review Practices
|
|
33
24
|
|
|
34
|
-
|
|
35
|
-
this same folder to actually record a pass.
|
|
25
|
+
Use the applicable practices below to populate the worksheet findings table.
|
|
36
26
|
|
|
37
|
-
|
|
|
27
|
+
| Practice ID | Practice | Why it matters | Evidence question |
|
|
38
28
|
|---|---|---|---|
|
|
39
|
-
|
|
|
40
|
-
|
|
|
41
|
-
|
|
|
42
|
-
|
|
|
43
|
-
|
|
|
44
|
-
|
|
|
45
|
-
|
|
|
46
|
-
|
|
|
47
|
-
|
|
|
48
|
-
| 10 | Learning from reviews | Review comments that aren't retained just repeat themselves across future PRs instead of becoming shared knowledge. | Do recurring review comments get turned into a lint rule, style-guide entry, or documented convention? |
|
|
29
|
+
| ENG-05-01 | Small PRs | Large PRs get worse review (reviewers skim or rubber-stamp) and take longer to land. | Is there a rough size norm for PRs, and are oversized PRs flagged for splitting? |
|
|
30
|
+
| ENG-05-02 | Clear description | A reviewer without context re-derives the 'why' from the diff alone, which is slower and more error-prone than being told. | Does every PR description explain what changed and why, not just restate the diff? |
|
|
31
|
+
| ENG-05-03 | Constructive feedback | Feedback that reads as personal or vague either gets ignored or damages the reviewer/author relationship over time. | Is feedback phrased around the code/tradeoff rather than the author, with a suggested direction, not just a criticism? |
|
|
32
|
+
| ENG-05-04 | Nitpick vs blocker | Without this distinction, minor style opinions can block a merge as if they were correctness issues. | Are non-blocking comments explicitly labeled (e.g. 'nit:') so the author knows what's optional vs required? |
|
|
33
|
+
| ENG-05-05 | Fast turnaround | A PR that sits for days loses context for both author and reviewer and blocks downstream work. | Is there a target turnaround time for first review, and is it generally being met? |
|
|
34
|
+
| ENG-05-06 | Automated checks first | Humans shouldn't spend review time on things a linter/CI can catch automatically. | Do lint, format, and test checks run and pass before a human reviews the PR? |
|
|
35
|
+
| ENG-05-07 | Context before code | Reviewing code without understanding the surrounding system leads to reviews that miss integration-level problems. | Does the reviewer understand the relevant part of the system before diving into line-by-line comments? |
|
|
36
|
+
| ENG-05-08 | Approve vs request changes | Conflating 'I have a question' with 'this is blocked' slows things down or sends mixed signals. | Is the approve/comment/request-changes distinction used deliberately, not defaulted to whichever is habit? |
|
|
37
|
+
| ENG-05-09 | Self-review first | Authors catch a meaningful fraction of their own issues by re-reading the diff before asking someone else to. | Does the author review their own diff (and leave self-notes where useful) before requesting review? |
|
|
38
|
+
| ENG-05-10 | Learning from reviews | Review comments that aren't retained just repeat themselves across future PRs instead of becoming shared knowledge. | Do recurring review comments get turned into a lint rule, style-guide entry, or documented convention? |
|
|
@@ -1,48 +1,38 @@
|
|
|
1
1
|
# Data Modeling
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
## Engineering Topic 6: Data Modeling Concepts
|
|
4
4
|
|
|
5
5
|
**When to run this:** When designing a new schema, adding a table/relationship to an existing one, or reviewing a migration before it ships.
|
|
6
6
|
|
|
7
7
|
**Goal of this pass:** Confirm the schema decisions are deliberate — the right relationship type, the right key strategy, the right precision — rather than whatever the ORM defaulted to.
|
|
8
8
|
|
|
9
|
-
##
|
|
9
|
+
## How to use this reference
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|---|---|
|
|
13
|
-
| `CHECKLIST.md` | The reference list of concepts for this topic — what each one is, why it matters, and the self-check question to ask. Doesn't get filled in; copy `RESULTS_TEMPLATE.md` to record a real pass. |
|
|
14
|
-
| `RESULTS_TEMPLATE.md` | Fillable version of the same list, with a Status column and notes. Copy this file per pass (e.g. `RESULTS_2026-08-29.md`) so history isn't overwritten. |
|
|
15
|
-
|
|
16
|
-
Full source list lives in `../ENGINEERING_REFERENCE.md`. Unlike
|
|
17
|
-
`../../security-gate/phases/`, this is knowledge/practice review, not pass/fail security
|
|
18
|
-
testing — there's no severity rating, and "Fail" isn't the right framing.
|
|
19
|
-
Use the Status options in `RESULTS_TEMPLATE.md` instead: Applied, Partial,
|
|
20
|
-
Should adopt, or Not applicable.
|
|
11
|
+
Use each stable practice ID in a generated engineering worksheet. Record concrete evidence and choose Applied, Partial, Recommended, Unverified, or N/A. Preserve each dated worksheet as review history.
|
|
21
12
|
|
|
22
13
|
## Wrap-up
|
|
23
14
|
|
|
24
15
|
- This isn't a gate — nothing here blocks a release the way
|
|
25
|
-
Critical/High security items in
|
|
16
|
+
Critical/High security items in security phase reviews do.
|
|
26
17
|
- Total concepts in this topic: 10
|
|
27
18
|
- Useful as an onboarding read-through, a periodic self-audit, or context
|
|
28
19
|
to hand an agent before it reviews related code.
|
|
29
20
|
|
|
30
21
|
---
|
|
31
22
|
|
|
32
|
-
|
|
23
|
+
## Concepts — Topic 6: Data Modeling Concepts
|
|
33
24
|
|
|
34
|
-
|
|
35
|
-
this same folder to actually record a pass.
|
|
25
|
+
Use the applicable practices below to populate the worksheet findings table.
|
|
36
26
|
|
|
37
|
-
|
|
|
27
|
+
| Practice ID | Practice | Why it matters | Evidence question |
|
|
38
28
|
|---|---|---|---|
|
|
39
|
-
|
|
|
40
|
-
|
|
|
41
|
-
|
|
|
42
|
-
|
|
|
43
|
-
|
|
|
44
|
-
|
|
|
45
|
-
|
|
|
46
|
-
|
|
|
47
|
-
|
|
|
48
|
-
| 10 | Precision & data types | Using float for money, or too-small an int type for a growing counter, causes rounding errors or overflow that surface much later. | Are numeric types chosen deliberately (e.g. decimal for currency, right-sized int for counters expected to grow)? |
|
|
29
|
+
| ENG-06-01 | Primary vs foreign keys | Gets the basic referential integrity of the schema right; missing FK constraints let orphaned/inconsistent data accumulate silently. | Does every relationship that should be enforced have an actual FK constraint, not just an implied convention? |
|
|
30
|
+
| ENG-06-02 | One-to-many | The most common relationship shape; modeling it as many-to-many or vice versa causes either data duplication or missing constraints. | For each relationship, has one-to-many vs many-to-many actually been verified against real-world cardinality, not assumed? |
|
|
31
|
+
| ENG-06-03 | Many-to-many | Requires a join table; skipping it (e.g. storing a comma-separated list) breaks referential integrity and queryability. | Are many-to-many relationships modeled with a proper join table rather than a denormalized list column? |
|
|
32
|
+
| ENG-06-04 | Composite keys | Sometimes the natural key is a combination of columns, not a synthetic ID — forcing a synthetic key can hide a real uniqueness constraint. | Where a natural composite key exists (e.g. user_id + role_id), is it enforced as a unique constraint even if a synthetic PK is also used? |
|
|
33
|
+
| ENG-06-05 | Soft deletes | A `deleted_at` column preserves history/auditability but means every query must remember to filter it, or leaked 'deleted' data appears. | If soft deletes are used, do all reads consistently filter them out (e.g. via a default scope), and is that not forgotten in raw queries? |
|
|
34
|
+
| ENG-06-06 | Polymorphic associations | Lets one table reference multiple other table types, but sacrifices FK constraint enforcement at the DB level — a known tradeoff, not a free pattern. | Where a polymorphic association is used, was that tradeoff (no DB-level FK) made deliberately, with integrity checks handled elsewhere? |
|
|
35
|
+
| ENG-06-07 | Enum vs lookup table | A hardcoded enum is faster to query but requires a migration/deploy to add a value; a lookup table is more flexible but adds a join. | Was the enum-vs-lookup-table choice made based on how often the value set changes, not just habit? |
|
|
36
|
+
| ENG-06-08 | Audit columns | Without created_at/updated_at/created_by-style columns, debugging 'when did this change and who did it' becomes guesswork. | Do tables that matter for debugging/compliance have consistent audit columns? |
|
|
37
|
+
| ENG-06-09 | Schema migrations | Migrations that aren't reversible or aren't tested against production-scale data can cause downtime or data loss when they run for real. | Are migrations reviewed for reversibility and tested against realistic data volume before running on production? |
|
|
38
|
+
| ENG-06-10 | Precision & data types | Using float for money, or too-small an int type for a growing counter, causes rounding errors or overflow that surface much later. | Are numeric types chosen deliberately (e.g. decimal for currency, right-sized int for counters expected to grow)? |
|
|
@@ -1,48 +1,38 @@
|
|
|
1
1
|
# Cloud Cost Saving
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
## Engineering Topic 7: Cloud Cost-Saving Techniques
|
|
4
4
|
|
|
5
5
|
**When to run this:** During a cost review, before scaling up infrastructure, or when a cloud bill increase needs investigating.
|
|
6
6
|
|
|
7
7
|
**Goal of this pass:** Work through the standard cost levers systematically instead of just cutting the first expensive-looking line item.
|
|
8
8
|
|
|
9
|
-
##
|
|
9
|
+
## How to use this reference
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|---|---|
|
|
13
|
-
| `CHECKLIST.md` | The reference list of concepts for this topic — what each one is, why it matters, and the self-check question to ask. Doesn't get filled in; copy `RESULTS_TEMPLATE.md` to record a real pass. |
|
|
14
|
-
| `RESULTS_TEMPLATE.md` | Fillable version of the same list, with a Status column and notes. Copy this file per pass (e.g. `RESULTS_2026-08-29.md`) so history isn't overwritten. |
|
|
15
|
-
|
|
16
|
-
Full source list lives in `../ENGINEERING_REFERENCE.md`. Unlike
|
|
17
|
-
`../../security-gate/phases/`, this is knowledge/practice review, not pass/fail security
|
|
18
|
-
testing — there's no severity rating, and "Fail" isn't the right framing.
|
|
19
|
-
Use the Status options in `RESULTS_TEMPLATE.md` instead: Applied, Partial,
|
|
20
|
-
Should adopt, or Not applicable.
|
|
11
|
+
Use each stable practice ID in a generated engineering worksheet. Record concrete evidence and choose Applied, Partial, Recommended, Unverified, or N/A. Preserve each dated worksheet as review history.
|
|
21
12
|
|
|
22
13
|
## Wrap-up
|
|
23
14
|
|
|
24
15
|
- This isn't a gate — nothing here blocks a release the way
|
|
25
|
-
Critical/High security items in
|
|
16
|
+
Critical/High security items in security phase reviews do.
|
|
26
17
|
- Total concepts in this topic: 10
|
|
27
18
|
- Useful as an onboarding read-through, a periodic self-audit, or context
|
|
28
19
|
to hand an agent before it reviews related code.
|
|
29
20
|
|
|
30
21
|
---
|
|
31
22
|
|
|
32
|
-
|
|
23
|
+
## Concepts — Topic 7: Cloud Cost-Saving Techniques
|
|
33
24
|
|
|
34
|
-
|
|
35
|
-
this same folder to actually record a pass.
|
|
25
|
+
Use the applicable practices below to populate the worksheet findings table.
|
|
36
26
|
|
|
37
|
-
|
|
|
27
|
+
| Practice ID | Practice | Why it matters | Evidence question |
|
|
38
28
|
|---|---|---|---|
|
|
39
|
-
|
|
|
40
|
-
|
|
|
41
|
-
|
|
|
42
|
-
|
|
|
43
|
-
|
|
|
44
|
-
|
|
|
45
|
-
|
|
|
46
|
-
|
|
|
47
|
-
|
|
|
48
|
-
| 10 | Multi-region arbitrage | Compute/storage/egress pricing varies meaningfully by region — running in the highest-cost region without a reason is avoidable spend. | Has region choice been checked against pricing, not just picked for proximity or default? |
|
|
29
|
+
| ENG-07-01 | Spot instances | Substantially cheaper than on-demand, but can be reclaimed with short notice — only right for interruption-tolerant workloads. | Are interruption-tolerant workloads (batch jobs, stateless workers) running on spot instead of on-demand? |
|
|
30
|
+
| ENG-07-02 | Autoscaling | Paying for peak capacity 24/7 wastes money during predictable low-traffic periods. | Does capacity actually scale down during known low-traffic periods, not just scale up under load? |
|
|
31
|
+
| ENG-07-03 | Reserved capacity | Committing to steady-state baseline load ahead of time is cheaper than paying on-demand rates for it continuously. | Has the stable, predictable-load portion of infrastructure been covered by reserved/committed-use pricing? |
|
|
32
|
+
| ENG-07-04 | Right-sizing | Over-provisioned instances (low CPU/memory utilization) are a direct, ongoing waste that compounds over time. | Have instance sizes been checked against actual utilization recently, not just set once at launch and left alone? |
|
|
33
|
+
| ENG-07-05 | Storage tiering | Infrequently accessed data doesn't need to sit on the most expensive, fastest storage tier. | Is cold/infrequently-accessed data moved to a cheaper storage tier rather than left on the default hot tier? |
|
|
34
|
+
| ENG-07-06 | CDN offloading | Serving static/cacheable content from origin instead of a CDN costs more in bandwidth and origin load. | Is static and cacheable content served from a CDN rather than hitting origin servers for every request? |
|
|
35
|
+
| ENG-07-07 | Serverless for spiky loads | Paying for always-on servers to handle infrequent, spiky workloads is wasteful compared to pay-per-invocation. | Are low-frequency, spiky workloads running serverless rather than on dedicated always-on infrastructure? |
|
|
36
|
+
| ENG-07-08 | Idle resource cleanup | Unattached volumes, unused load balancers, and orphaned snapshots accumulate cost silently over time. | Is there a recurring process to find and remove idle/orphaned cloud resources? |
|
|
37
|
+
| ENG-07-09 | Compression | Uncompressed data transfer and storage costs more in both bandwidth and storage fees than it needs to. | Is compression enabled where it's cheap to apply (asset delivery, log storage, backups)? |
|
|
38
|
+
| ENG-07-10 | Multi-region arbitrage | Compute/storage/egress pricing varies meaningfully by region — running in the highest-cost region without a reason is avoidable spend. | Has region choice been checked against pricing, not just picked for proximity or default? |
|
|
@@ -1,48 +1,38 @@
|
|
|
1
1
|
# Testing Types
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
## Engineering Topic 8: Types of Testing
|
|
4
4
|
|
|
5
5
|
**When to run this:** When designing a test strategy for a new project/feature, or auditing whether the current test suite has coverage gaps.
|
|
6
6
|
|
|
7
7
|
**Goal of this pass:** Confirm the right mix of test types is in place for what's actually being built — not every project needs every type, but the choice should be deliberate.
|
|
8
8
|
|
|
9
|
-
##
|
|
9
|
+
## How to use this reference
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|---|---|
|
|
13
|
-
| `CHECKLIST.md` | The reference list of concepts for this topic — what each one is, why it matters, and the self-check question to ask. Doesn't get filled in; copy `RESULTS_TEMPLATE.md` to record a real pass. |
|
|
14
|
-
| `RESULTS_TEMPLATE.md` | Fillable version of the same list, with a Status column and notes. Copy this file per pass (e.g. `RESULTS_2026-08-29.md`) so history isn't overwritten. |
|
|
15
|
-
|
|
16
|
-
Full source list lives in `../ENGINEERING_REFERENCE.md`. Unlike
|
|
17
|
-
`../../security-gate/phases/`, this is knowledge/practice review, not pass/fail security
|
|
18
|
-
testing — there's no severity rating, and "Fail" isn't the right framing.
|
|
19
|
-
Use the Status options in `RESULTS_TEMPLATE.md` instead: Applied, Partial,
|
|
20
|
-
Should adopt, or Not applicable.
|
|
11
|
+
Use each stable practice ID in a generated engineering worksheet. Record concrete evidence and choose Applied, Partial, Recommended, Unverified, or N/A. Preserve each dated worksheet as review history.
|
|
21
12
|
|
|
22
13
|
## Wrap-up
|
|
23
14
|
|
|
24
15
|
- This isn't a gate — nothing here blocks a release the way
|
|
25
|
-
Critical/High security items in
|
|
16
|
+
Critical/High security items in security phase reviews do.
|
|
26
17
|
- Total concepts in this topic: 10
|
|
27
18
|
- Useful as an onboarding read-through, a periodic self-audit, or context
|
|
28
19
|
to hand an agent before it reviews related code.
|
|
29
20
|
|
|
30
21
|
---
|
|
31
22
|
|
|
32
|
-
|
|
23
|
+
## Concepts — Topic 8: Types of Testing
|
|
33
24
|
|
|
34
|
-
|
|
35
|
-
this same folder to actually record a pass.
|
|
25
|
+
Use the applicable practices below to populate the worksheet findings table.
|
|
36
26
|
|
|
37
|
-
|
|
|
27
|
+
| Practice ID | Practice | Why it matters | Evidence question |
|
|
38
28
|
|---|---|---|---|
|
|
39
|
-
|
|
|
40
|
-
|
|
|
41
|
-
|
|
|
42
|
-
|
|
|
43
|
-
|
|
|
44
|
-
|
|
|
45
|
-
|
|
|
46
|
-
|
|
|
47
|
-
|
|
|
48
|
-
| 10 | Regression test | Confirms a previously fixed bug doesn't come back — without it, the same bug can resurface silently. | Does every fixed bug get a corresponding regression test added to the suite? |
|
|
29
|
+
| ENG-08-01 | Unit test | Tests one function/module in isolation — fast feedback, but doesn't catch integration problems. | Do core business-logic units have direct unit test coverage, isolated from external dependencies? |
|
|
30
|
+
| ENG-08-02 | Integration test | Tests that multiple components work together correctly (e.g. app + real DB) — catches what unit tests can't. | Are integration tests run against real (or realistic) dependencies, not just mocks, for critical paths? |
|
|
31
|
+
| ENG-08-03 | End-to-end test | Tests a full user flow through the real system — the closest simulation of actual usage, but slow and more brittle. | Do the most critical user journeys have E2E coverage, kept to a small, high-value set rather than everything? |
|
|
32
|
+
| ENG-08-04 | Snapshot test | Catches unintended changes to output (UI render, API response shape) by diffing against a saved reference. | Where snapshot tests are used, are snapshots actually reviewed on change, not blindly re-approved? |
|
|
33
|
+
| ENG-08-05 | Contract test | Verifies that a service's API still matches what its consumers expect, without needing a full integration environment. | For services with external consumers, are contract tests in place to catch breaking API changes before deploy? |
|
|
34
|
+
| ENG-08-06 | Load test | Confirms the system holds up under expected (and above-expected) traffic before it meets that traffic in production. | Has the system been load-tested at a realistic multiple of expected peak traffic? |
|
|
35
|
+
| ENG-08-07 | Chaos test | Confirms the system degrades gracefully under real failure conditions, not just happy-path load. | Has chaos testing been run against the critical paths (see also Debugging Production Issues)? |
|
|
36
|
+
| ENG-08-08 | Mutation test | Checks whether the test suite would actually catch a real bug by deliberately introducing small code mutations and seeing if tests fail. | Has mutation testing been run at least once to sanity-check that the test suite isn't just achieving coverage without catching bugs? |
|
|
37
|
+
| ENG-08-09 | Smoke test | A fast, shallow check that the basic system is up and functional — meant to catch catastrophic failures immediately after deploy. | Does every deploy run a fast smoke test before being considered successful? |
|
|
38
|
+
| ENG-08-10 | Regression test | Confirms a previously fixed bug doesn't come back — without it, the same bug can resurface silently. | Does every fixed bug get a corresponding regression test added to the suite? |
|