azcodr 1.3.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/hooks.json.example +42 -0
- package/.agents/mcp_config.json.example +24 -0
- package/.agents/skills/agentic-architect/SKILL.md +14 -7
- package/.agents/skills/agentic-architect/references/agents_md_template.md +6 -3
- package/.agents/skills/agentic-architect/references/skill_architecture_inquiry.md +1 -1
- package/.agents/skills/agentic-architect/references/skill_template.md +2 -1
- package/.agents/skills/agentic-architect/scripts/validate_agentic_configs.sh +156 -4
- package/.agents/skills/clean-code-refactor/SKILL.md +6 -6
- package/.agents/skills/compliance-audit/SKILL.md +1 -1
- package/.agents/skills/lets-build/SKILL.md +12 -4
- package/.agents/skills/lets-build/references/hexagonal_bootstrap_scaffolds.md +1 -1
- package/.agents/skills/lets-build/references/project_readme_template.md +4 -4
- package/.agents/skills/lets-build/scripts/bootstrap_workspace.sh +63 -35
- package/.agents/skills/relentless-questioner/SKILL.md +10 -5
- package/AGENTS.md +25 -40
- package/README.md +27 -40
- package/bin/azcodr.js +1 -1
- package/docs/rules/agentic_configuration.md +120 -32
- package/docs/rules/api_architecture.md +179 -0
- package/docs/rules/caching.md +30 -13
- package/docs/rules/cloud_native.md +10 -12
- package/docs/rules/cqrs.md +203 -0
- package/docs/rules/database_design.md +125 -0
- package/docs/rules/database_operations.md +56 -14
- package/docs/rules/design_patterns.md +18 -11
- package/docs/rules/devops_ci_cd.md +76 -0
- package/docs/rules/domain_driven_design.md +17 -13
- package/docs/rules/feature_flags.md +21 -4
- package/docs/rules/frontend_architecture.md +157 -0
- package/docs/rules/multitenancy_architecture.md +98 -0
- package/docs/rules/product_ownership.md +22 -27
- package/docs/rules/requirements_engineering.md +16 -14
- package/docs/rules/security_compliance.md +53 -0
- package/docs/rules/server_driven_ui.md +20 -3
- package/docs/rules/test_driven_development.md +118 -62
- package/docs/rules/type_safety.md +65 -0
- package/docs/rules/ui_ux_architecture.md +33 -30
- package/docs/rules/workflow_state_machines.md +20 -3
- package/lib/scaffold.js +9 -0
- package/memory.md +149 -15
- package/package.json +2 -2
- package/docs/rules/accessibility.md +0 -31
- package/docs/rules/advanced_api_patterns.md +0 -104
- package/docs/rules/api_versioning.md +0 -113
- package/docs/rules/application_security.md +0 -23
- package/docs/rules/architecture_decision_records.md +0 -42
- package/docs/rules/compliance.md +0 -25
- package/docs/rules/container_infrastructure.md +0 -32
- package/docs/rules/continuous_deployment.md +0 -24
- package/docs/rules/continuous_integration.md +0 -20
- package/docs/rules/continuous_learning.md +0 -29
- package/docs/rules/database_integrity.md +0 -80
- package/docs/rules/database_migrations.md +0 -41
- package/docs/rules/database_performance.md +0 -44
- package/docs/rules/database_transactions.md +0 -81
- package/docs/rules/devsecops.md +0 -33
- package/docs/rules/multitenancy_isolation.md +0 -88
- package/docs/rules/react.md +0 -78
- package/docs/rules/rest_api_conventions.md +0 -46
- package/docs/rules/tenant_dynamic_schemas.md +0 -88
- package/docs/rules/tenant_pluggable_logic.md +0 -59
- package/docs/rules/test_isolation.md +0 -26
- package/docs/rules/typescript.md +0 -55
- package/docs/rules/ui_navigation.md +0 -20
- package/docs/rules/workspace_isolation.md +0 -25
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
# API Architecture, Protocols & Communication Standards
|
|
2
|
+
|
|
3
|
+
> **Core Mandate:** Enforce standard HTTP semantics, synchronous vs. asynchronous processing (`202 Accepted`), capability metadata (`_actions`), safe mutations via idempotency keys, keyset cursor pagination, optimistic concurrency control (OCC), and RFC 8594 lifecycle versioning.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 1. Standard HTTP Semantics & Status Codes
|
|
8
|
+
|
|
9
|
+
APIs must adhere strictly to standard HTTP semantics. Never return `200 OK` for error envelopes:
|
|
10
|
+
|
|
11
|
+
| Status Code | Semantic Purpose | When to Return |
|
|
12
|
+
|---|---|---|
|
|
13
|
+
| **`200 OK`** | Successful read or synchronous update | Standard `GET`, `PATCH`, `PUT` queries that return payloads. |
|
|
14
|
+
| **`201 Created`** | Successful resource creation | Synchronous `POST` with `Location: /api/v1/resources/:id` header. |
|
|
15
|
+
| **`202 Accepted`** | Asynchronous task accepted | Tasks exceeding latency budgets (> 1.5s) delegated to background queues. |
|
|
16
|
+
| **`204 No Content`** | Successful action with zero payload | Standard `DELETE` or empty mutation responses. |
|
|
17
|
+
| **`400 Bad Request`** | Malformed syntax or protocol violation | Unparseable JSON, invalid query parameters. |
|
|
18
|
+
| **`401 Unauthorized`** | Missing or invalid authentication | Missing, expired, or tampered JWT / API token. |
|
|
19
|
+
| **`403 Forbidden`** | Authenticated but insufficient permission | Role, tenant boundary, or policy guard denial. |
|
|
20
|
+
| **`404 Not Found`** | Resource does not exist | Unknown entity identifier (or masked tenant resource). |
|
|
21
|
+
| **`409 Conflict`** | State conflict or race condition | Concurrency mismatch (`If-Match`), unique constraint, or in-flight idempotency. |
|
|
22
|
+
| **`422 Unprocessable`** | Semantic validation failure | Schema constraint violation (RFC 7807 problem details). |
|
|
23
|
+
| **`500 Internal Error`** | Unhandled server exception | Unexpected server failure; never leak internal stack traces. |
|
|
24
|
+
|
|
25
|
+
### Subresource URL Conventions
|
|
26
|
+
- Express relational hierarchy cleanly: `/api/v1/organizations/:orgId/projects/:projectId/members`.
|
|
27
|
+
- Limit URL nesting to a maximum of 2 subresource levels; for deeper resources, access directly via canonical ID (`/api/v1/tasks/:taskId`).
|
|
28
|
+
|
|
29
|
+
### Enumeration Masking
|
|
30
|
+
- Authentication and recovery endpoints must never reveal user existence (e.g. return `"If an account exists, a recovery link has been sent"` with identical timing).
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## 2. Synchronous vs. Asynchronous Processing (`202 Accepted`)
|
|
35
|
+
|
|
36
|
+
Operations with unpredictable or long execution durations (> 1.5 seconds, such as video rendering, large PDF exports, batch imports, or complex report generation) must never block synchronous HTTP request threads:
|
|
37
|
+
|
|
38
|
+
```mermaid
|
|
39
|
+
sequenceDiagram
|
|
40
|
+
autonumber
|
|
41
|
+
actor Client
|
|
42
|
+
participant Gateway as API Gateway
|
|
43
|
+
participant Queue as Task Queue / Worker
|
|
44
|
+
|
|
45
|
+
Client->>Gateway: POST /reports/export (Long-running > 1.5s)
|
|
46
|
+
Gateway->>Queue: Dispatch background job
|
|
47
|
+
Gateway-->>Client: 202 Accepted (Location: /api/v1/tasks/tsk_123)
|
|
48
|
+
Note over Client,Gateway: Client polls GET /api/v1/tasks/tsk_123 until complete
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
### Protocol Standards:
|
|
52
|
+
1. Dispatch the payload to a persistent worker queue (e.g., BullMQ, Temporal, Celery).
|
|
53
|
+
2. Respond immediately with **`202 Accepted`** containing:
|
|
54
|
+
- Header: `Location: /api/v1/tasks/:taskId`
|
|
55
|
+
- Envelope:
|
|
56
|
+
```json
|
|
57
|
+
{
|
|
58
|
+
"taskId": "tsk_123",
|
|
59
|
+
"status": "QUEUED",
|
|
60
|
+
"pollIntervalMs": 2000,
|
|
61
|
+
"_links": {
|
|
62
|
+
"status": { "href": "/api/v1/tasks/tsk_123", "method": "GET" },
|
|
63
|
+
"cancel": { "href": "/api/v1/tasks/tsk_123", "method": "DELETE" }
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
```
|
|
67
|
+
3. Polling endpoint (`GET /api/v1/tasks/:taskId`) returns:
|
|
68
|
+
- In-progress: `200 OK` with status `PROCESSING` and progress percentage.
|
|
69
|
+
- Finished: `303 See Other` with `Location: /api/v1/reports/rep_789` or `200 OK` with `status: "COMPLETED"` and the final artifact URI.
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
## 3. Allowed Actions & Capability Metadata (`_actions` Envelope)
|
|
74
|
+
|
|
75
|
+
Clients must not duplicate complex server-side business and authorization rules to decide whether UI actions (edit, delete, approve, cancel, refund) are permitted. **The server is the authoritative source of truth.**
|
|
76
|
+
|
|
77
|
+
### Pattern: `_actions` and `_links` Envelope
|
|
78
|
+
Every resource response must embed an `_actions` boolean map and optional `_links` hypermedia block indicating what the requesting caller is permitted to do based on their role, tenant boundaries, and the entity's current lifecycle state:
|
|
79
|
+
|
|
80
|
+
```json
|
|
81
|
+
{
|
|
82
|
+
"id": "ord_9876",
|
|
83
|
+
"status": "SHIPPED",
|
|
84
|
+
"totalAmount": 149.99,
|
|
85
|
+
"currency": "USD",
|
|
86
|
+
"_actions": {
|
|
87
|
+
"canEdit": false,
|
|
88
|
+
"canCancel": false,
|
|
89
|
+
"canTrack": true,
|
|
90
|
+
"canRequestRefund": true
|
|
91
|
+
},
|
|
92
|
+
"_links": {
|
|
93
|
+
"self": { "href": "/api/v1/orders/ord_9876", "method": "GET" },
|
|
94
|
+
"track": { "href": "/api/v1/orders/ord_9876/tracking", "method": "GET" },
|
|
95
|
+
"refund": { "href": "/api/v1/orders/ord_9876/refunds", "method": "POST" }
|
|
96
|
+
}
|
|
97
|
+
}
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
### Frontend Binding:
|
|
101
|
+
- UI action buttons directly bind visibility or disabled state to `resource._actions.canCancel`.
|
|
102
|
+
- When business logic evolves (e.g. orders over $1,000 require manager approval), only backend policy changes—zero frontend redeployment required.
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## 4. Safe Mutations via Idempotency Keys (IETF Draft)
|
|
107
|
+
|
|
108
|
+
To prevent duplicate execution (double charging, duplicate orders) caused by network retries or transient connection drops:
|
|
109
|
+
|
|
110
|
+
### Protocol Standards:
|
|
111
|
+
- Clients generating mutating requests (`POST`, `PATCH`) must supply a unique `Idempotency-Key: <uuid-v4>` header.
|
|
112
|
+
- **Server Execution Lifecycle**:
|
|
113
|
+
1. Check distributed idempotency cache for key `idemp:<tenantId>:<idempotencyKey>`.
|
|
114
|
+
2. If found with status `IN_FLIGHT`: return **`409 Conflict`** (`IDEMPOTENT_OPERATION_IN_PROGRESS`).
|
|
115
|
+
3. If found with status `COMPLETED`: return the cached HTTP status code, headers, and response payload without re-executing.
|
|
116
|
+
4. If not found: Acquire distributed lock, execute mutation within an atomic database transaction, cache the response envelope with a 24-hour TTL, and release the lock.
|
|
117
|
+
|
|
118
|
+
---
|
|
119
|
+
|
|
120
|
+
## 5. High-Scale Keyset / Cursor-Based Pagination
|
|
121
|
+
|
|
122
|
+
Never use offset pagination (`OFFSET 10000 LIMIT 20`) on large tables. Offsets degrade linearly ($O(N)$) and suffer from page-drift anomalies as rows are inserted or deleted.
|
|
123
|
+
|
|
124
|
+
### Specification & Envelope:
|
|
125
|
+
- Query Parameters: `?cursor=<opaque_base64>&limit=20` (default limit 20, max 100).
|
|
126
|
+
- Response Envelope:
|
|
127
|
+
```json
|
|
128
|
+
{
|
|
129
|
+
"data": [...],
|
|
130
|
+
"pagination": {
|
|
131
|
+
"nextCursor": "ZXlKaWRI...==",
|
|
132
|
+
"hasMore": true,
|
|
133
|
+
"limit": 20
|
|
134
|
+
}
|
|
135
|
+
}
|
|
136
|
+
```
|
|
137
|
+
- **Agnostic Keyset Query Pattern**:
|
|
138
|
+
```sql
|
|
139
|
+
SELECT * FROM orders
|
|
140
|
+
WHERE tenant_id = :tenantId
|
|
141
|
+
AND (created_at, id) < (:cursorCreatedAt, :cursorId)
|
|
142
|
+
ORDER BY created_at DESC, id DESC
|
|
143
|
+
LIMIT :limit + 1;
|
|
144
|
+
```
|
|
145
|
+
If `results.length > limit`, slice the extra item and encode its composite values (`created_at`, `id`) into the base64 `nextCursor`.
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
## 6. Optimistic Concurrency Control (OCC)
|
|
150
|
+
|
|
151
|
+
Prevent lost-update anomalies during concurrent edits without pessimistic database row locking:
|
|
152
|
+
|
|
153
|
+
### Protocol Standards:
|
|
154
|
+
- Every mutable entity contains an incrementing integer `version` column.
|
|
155
|
+
- The server returns the current entity version in the `ETag` response header: `ETag: W/"v4"`.
|
|
156
|
+
- Clients submitting updates (`PUT`, `PATCH`) must include `If-Match: W/"v4"`.
|
|
157
|
+
- **Atomic Concurrency Handling**:
|
|
158
|
+
```sql
|
|
159
|
+
UPDATE orders
|
|
160
|
+
SET status = :status, version = version + 1
|
|
161
|
+
WHERE id = :id AND version = :expectedVersion;
|
|
162
|
+
```
|
|
163
|
+
- If `rows_affected == 0`: Return **`409 Conflict`** with error code `CONCURRENCY_CONFLICT` and the latest entity representation.
|
|
164
|
+
|
|
165
|
+
---
|
|
166
|
+
|
|
167
|
+
## 7. API Versioning & RFC 8594 Lifecycle Deprecation
|
|
168
|
+
|
|
169
|
+
### URI Versioning Standard
|
|
170
|
+
- Standardize on explicit path versioning: `/v1/`, `/v2/`.
|
|
171
|
+
- Never introduce breaking changes within an active major version:
|
|
172
|
+
- *Non-Breaking (Permitted in `/v1/`):* Adding optional fields, adding new endpoints, adding new enum variants.
|
|
173
|
+
- *Breaking (Demands `/v2/`):* Renaming/removing fields, changing validation constraints, altering status codes.
|
|
174
|
+
|
|
175
|
+
### RFC 8594 Sunset & Deprecation Headers
|
|
176
|
+
When deprecating an endpoint, provide clients with a minimum 90-day grace period:
|
|
177
|
+
- `Deprecation: @<unix-timestamp>`: Date when the endpoint was deprecated.
|
|
178
|
+
- `Sunset: <HTTP-date>`: Absolute date when the endpoint will return `410 Gone`.
|
|
179
|
+
- `Link: </api/v2/docs>; rel="sunset"`: Link to migration documentation.
|
package/docs/rules/caching.md
CHANGED
|
@@ -4,20 +4,37 @@
|
|
|
4
4
|
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
-
## 1.
|
|
7
|
+
## 1. The YAGNI Gate: Database First, Caching Second
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Caching introduces state duplication, cache invalidation race conditions, and memory overhead. **Caching is never a substitute for missing database indexes or poorly structured SQL queries.**
|
|
10
10
|
|
|
11
|
+
```mermaid
|
|
12
|
+
flowchart TD
|
|
13
|
+
subgraph CachingGate["Caching YAGNI Gate"]
|
|
14
|
+
B1["1. Simple Baseline (Day 1)<br/>• Relational queries with composite indexes<br/>• Request-scoped in-memory DataLoader batching to eliminate N+1<br/>• Zero distributed cache infrastructure (no Redis / Memcached)"]
|
|
15
|
+
B2["2. Anti-Triggers (Forbidden)<br/>• Queries that are slow due to missing indexes or sequential scans<br/>• High-write / high-churn entities (write-heavy mutation streams)<br/>• Low-traffic administrative or internal operational queries"]
|
|
16
|
+
B3["3. The Tipping Point (Graduation)<br/>• Query has been optimized with EXPLAIN ANALYZE, but p99 latency still exceeds SLA (> 100ms)<br/>• Read-to-write asymmetry on the entity exceeds 20:1<br/>• Downstream external API rate limits or third-party egress costs demand response caching"]
|
|
17
|
+
B1 -->|Forbidden if missing indexes| B2
|
|
18
|
+
B1 -->|Triggered by high read/write ratio| B3
|
|
19
|
+
end
|
|
11
20
|
```
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 2. Abstract Cache Port & Cache-Aside Pattern
|
|
25
|
+
|
|
26
|
+
Application services interact with caching infrastructure through a swappable **Cache Port**, supporting any backend (Redis, Valkey, Dragonfly, KeyDB, Memcached, or in-memory LRU):
|
|
27
|
+
|
|
28
|
+
```mermaid
|
|
29
|
+
classDiagram
|
|
30
|
+
class CachePort {
|
|
31
|
+
<<interface>>
|
|
32
|
+
+get(key: string) Optional~string~
|
|
33
|
+
+set(key: string, value: string, ttlSeconds: number) void
|
|
34
|
+
+delete(key: string) void
|
|
35
|
+
+deletePattern(pattern: string) void
|
|
36
|
+
+acquireLock(lockKey: string, ttlMs: number) boolean
|
|
37
|
+
}
|
|
21
38
|
```
|
|
22
39
|
|
|
23
40
|
### Cache-Aside Implementation & Stampede Defense
|
|
@@ -38,7 +55,7 @@ For high-throughput cache regeneration, employ the **XFetch algorithm** (probabi
|
|
|
38
55
|
|
|
39
56
|
---
|
|
40
57
|
|
|
41
|
-
##
|
|
58
|
+
## 3. Key Namespacing & Event-Driven Invalidation
|
|
42
59
|
|
|
43
60
|
- **Universal Key Hierarchy**: Structure all keys hierarchically:
|
|
44
61
|
`tenant:{tenantId}:{entity}:{entityId}` (e.g. `tenant:123:order:987`)
|
|
@@ -46,7 +63,7 @@ For high-throughput cache regeneration, employ the **XFetch algorithm** (probabi
|
|
|
46
63
|
|
|
47
64
|
---
|
|
48
65
|
|
|
49
|
-
##
|
|
66
|
+
## 4. HTTP Conditional Caching (ETags)
|
|
50
67
|
|
|
51
68
|
- Generate strong cryptographic `ETag` hashes (e.g. SHA-256 of representation or resource version) for cacheable `GET` endpoints.
|
|
52
69
|
- Return **`304 Not Modified`** with zero payload body when inbound requests present matching `If-None-Match` headers, preserving bandwidth and client CPU.
|
|
@@ -28,16 +28,14 @@ All application processes and containers must handle graceful termination:
|
|
|
28
28
|
- Drain active, in-flight connections within a bounded timeout window (e.g. 10 seconds).
|
|
29
29
|
- Gracefully flush telemetry buffers, terminate background workers, and close database/cache connection pools cleanly before exiting with code 0:
|
|
30
30
|
|
|
31
|
-
```
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
│ 6. Terminate process with exit code 0 │
|
|
42
|
-
└────────────────────────────────────────────────────────┘
|
|
31
|
+
```mermaid
|
|
32
|
+
flowchart TD
|
|
33
|
+
subgraph GracefulShutdown["Graceful Shutdown Flow (Universal / Agnostic)"]
|
|
34
|
+
Sig["onSignal (SIGTERM | SIGINT)"] --> S1["1. Set health check probe to UNHEALTHY (drain LB)"]
|
|
35
|
+
S1 --> S2["2. Stop server listening for new connections"]
|
|
36
|
+
S2 --> S3["3. Wait for in-flight requests (timeout: 10s)"]
|
|
37
|
+
S3 --> S4["4. Close database and cache connection pools"]
|
|
38
|
+
S4 --> S5["5. Flush OpenTelemetry trace & log buffers"]
|
|
39
|
+
S5 --> S6["6. Terminate process with exit code 0"]
|
|
40
|
+
end
|
|
43
41
|
```
|
|
@@ -0,0 +1,203 @@
|
|
|
1
|
+
# CQRS (Command Query Responsibility Segregation) & YAGNI Defense
|
|
2
|
+
|
|
3
|
+
> **Core Mandate:** Enforce evolutionary graduation across the CQRS spectrum, protect against premature abstraction (YAGNI), bound CQRS strictly to high-contention subdomains, decouple write aggregates from read projections, and prevent eventual consistency hazards.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 1. The YAGNI Defense: Default Architecture vs. CQRS
|
|
8
|
+
|
|
9
|
+
CQRS separates the data model used for state mutations (**Commands**) from the data model used for read operations (**Queries**). While powerful, **CQRS is one of the most frequently over-engineered patterns in modern software**.
|
|
10
|
+
|
|
11
|
+
```mermaid
|
|
12
|
+
flowchart LR
|
|
13
|
+
subgraph DefaultModel["Default Single-Model (CRUD) - Start Here"]
|
|
14
|
+
D1["• 1 Unified Data Schema (ACID)<br/>• Immediate Strong Consistency<br/>• Zero Projection Infrastructure<br/>• Minimal Mental Overhead"]
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
subgraph DistributedCQRS["Distributed CQRS (Level 3) - Graduate on Proof"]
|
|
18
|
+
C1["• Dual Schemas & Dual Databases<br/>• Eventual Consistency & Lag<br/>• Outbox, Message Bus, CDC Workers<br/>• Projection Versioning & Replays"]
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
DefaultModel -.->|Graduate ONLY on high read-write asymmetry or SLA breaches| DistributedCQRS
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
### The YAGNI Rule for CQRS
|
|
25
|
+
1. **Default to Single Model / Single Database**: Start every application, microservice, or bounded context with a standard single data model (Hexagonal/DDD) where repositories handle both reads and writes.
|
|
26
|
+
2. **Never Use CQRS as Top-Level System Architecture**: CQRS must never be applied globally across an entire system. It is strictly a bounded-context tactical pattern. Generic subdomains (Auth, Settings, Billing, Organizations) must remain simple transactional CRUD.
|
|
27
|
+
3. **Explicit Graduation Tipping Points**: Do not introduce segregated read databases or asynchronous projections until empirical metrics prove that a unified relational model cannot meet operational constraints:
|
|
28
|
+
- **Read/Write Asymmetry**: Read volume exceeds writes by > 50:1, and heavy read queries cause lock contention that starves transactional write operations.
|
|
29
|
+
- **Impedance Mismatch**: Constructing UI views requires joining 10+ relational tables across multiple consistency boundaries, causing query latency to violate SLAs (> 500ms).
|
|
30
|
+
- **Polyglot Query Topology**: Complex full-text search, spatial filtering, or columnar aggregation fundamentally requires specialized search engines (Elasticsearch, Meilisearch, ClickHouse) that cannot act as the primary write store.
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## 2. The 4-Tier Evolutionary CQRS Spectrum
|
|
35
|
+
|
|
36
|
+
Rather than treating CQRS as a binary switch, systems must graduate incrementally across four explicit architectural tiers:
|
|
37
|
+
|
|
38
|
+
```mermaid
|
|
39
|
+
flowchart TD
|
|
40
|
+
L0["Level 0: Method CQS (Bertrand Meyer)<br/>Functions either mutate state or return data; zero architectural overhead. Always mandatory."]
|
|
41
|
+
L1["Level 1: Segregated Handlers in Code (Single DB, Single Schema)<br/>Command Handlers load Aggregates; Query Handlers bypass domain model for flat DTOs."]
|
|
42
|
+
L2["Level 2: Segregated Read Models / Materialized Views (Single DB)<br/>Read queries query indexed SQL views or JSON cache tables populated synchronously via ACID transactions."]
|
|
43
|
+
L3["Level 3: Polyglot Persistence & Asynchronous Projections (Multi-Store)<br/>Write DB (Postgres) + Read DB (Elastic/Redis). Synchronized asynchronously via Transactional Outbox + CDC."]
|
|
44
|
+
|
|
45
|
+
L0 --> L1 --> L2 --> L3
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
### Level 0: Method-Level CQS (Command-Query Separation)
|
|
49
|
+
- Mandatory across all codebases (see [`docs/rules/clean_code.md`](./clean_code.md)).
|
|
50
|
+
- Any function that modifies state must return `void` or a `Result<void, DomainError>`. It must never return the mutated entity.
|
|
51
|
+
- Any function that queries state must be pure and cause zero observable side-effects.
|
|
52
|
+
|
|
53
|
+
### Level 1: Segregated Handlers in Code (Single DB, Single Schema)
|
|
54
|
+
- When domain aggregates become rich with business logic, hydrating deep aggregate graphs (with children, value objects, and invariant checks) simply to render a flat summary list is wasteful.
|
|
55
|
+
- **Commands**: Flow through `CommandHandler` ➔ loads `AggregateRoot` from `Repository` ➔ executes domain methods ➔ commits transaction.
|
|
56
|
+
- **Queries**: Flow through `QueryHandler` ➔ bypasses domain repositories ➔ issues direct projection queries (SQL `SELECT` or ORM raw select) ➔ returns read-only `DTO`.
|
|
57
|
+
- *Zero distributed complexity; 100% ACID consistency.*
|
|
58
|
+
|
|
59
|
+
### Level 2: Segregated Read Models (Single DB, Synchronous)
|
|
60
|
+
- When read queries require expensive aggregations or multi-table joins, create dedicated read tables, database views, or PostgreSQL JSONB projection columns in the same database.
|
|
61
|
+
- Projections are updated **synchronously within the same database transaction** as the command mutation (or via database triggers).
|
|
62
|
+
- *Guarantees immediate read-your-own-writes consistency without distributed outbox pipelines.*
|
|
63
|
+
|
|
64
|
+
### Level 3: Polyglot Persistence & Asynchronous Projections
|
|
65
|
+
- Writes commit to the transactional primary database (e.g. PostgreSQL).
|
|
66
|
+
- Events are emitted via the Transactional Outbox pattern and streamed to secondary read stores (e.g. Elasticsearch for search, Redis for caches, ClickHouse for telemetry).
|
|
67
|
+
- *Requires strict handling of eventual consistency, projection replay, and outbox delivery.*
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## 3. Tactical Implementation Patterns (TypeScript)
|
|
72
|
+
|
|
73
|
+
### 1. Command Side: Enforcing Invariants via Aggregates
|
|
74
|
+
Commands express user intent, encapsulate validation, and mutate state through the Aggregate Root:
|
|
75
|
+
|
|
76
|
+
```typescript
|
|
77
|
+
// commands/create_order.command.ts
|
|
78
|
+
export interface CreateOrderCommand {
|
|
79
|
+
readonly orderId: string;
|
|
80
|
+
readonly customerId: string;
|
|
81
|
+
readonly items: ReadonlyArray<{ productId: string; quantity: number; unitPriceCents: number }>;
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
// handlers/create_order.handler.ts
|
|
85
|
+
export class CreateOrderCommandHandler {
|
|
86
|
+
constructor(
|
|
87
|
+
private readonly orderRepo: OrderRepositoryPort,
|
|
88
|
+
private readonly outbox: TransactionalOutboxPort
|
|
89
|
+
) {}
|
|
90
|
+
|
|
91
|
+
async execute(command: CreateOrderCommand): Promise<Result<void, OrderDomainError>> {
|
|
92
|
+
// 1. Load or instantiate Aggregate Root
|
|
93
|
+
const orderResult = OrderAggregate.create({
|
|
94
|
+
id: command.orderId,
|
|
95
|
+
customerId: command.customerId,
|
|
96
|
+
items: command.items
|
|
97
|
+
});
|
|
98
|
+
|
|
99
|
+
if (orderResult.isFailure) {
|
|
100
|
+
return Result.fail(orderResult.error);
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
const order = orderResult.value;
|
|
104
|
+
|
|
105
|
+
// 2. Commit aggregate mutation and outbox event atomically in 1 transaction
|
|
106
|
+
await this.orderRepo.transaction(async (tx) => {
|
|
107
|
+
await this.orderRepo.save(order, tx);
|
|
108
|
+
for (const event of order.pullDomainEvents()) {
|
|
109
|
+
await this.outbox.stageEvent(event, tx);
|
|
110
|
+
}
|
|
111
|
+
});
|
|
112
|
+
|
|
113
|
+
return Result.ok();
|
|
114
|
+
}
|
|
115
|
+
}
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
### 2. Query Side: Fast Direct Projection DTOs
|
|
119
|
+
Queries bypass domain entities and ORM aggregate hydration entirely, executing direct, index-optimized reads:
|
|
120
|
+
|
|
121
|
+
```typescript
|
|
122
|
+
// queries/get_customer_orders.query.ts
|
|
123
|
+
export interface GetCustomerOrdersQuery {
|
|
124
|
+
readonly customerId: string;
|
|
125
|
+
readonly limit: number;
|
|
126
|
+
readonly cursor?: string;
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
export interface CustomerOrderSummaryDTO {
|
|
130
|
+
readonly orderId: string;
|
|
131
|
+
readonly totalCents: number;
|
|
132
|
+
readonly status: string;
|
|
133
|
+
readonly itemCount: number;
|
|
134
|
+
readonly createdAt: string;
|
|
135
|
+
}
|
|
136
|
+
|
|
137
|
+
// handlers/get_customer_orders.handler.ts
|
|
138
|
+
export class GetCustomerOrdersQueryHandler {
|
|
139
|
+
constructor(private readonly db: ReadDatabaseClient) {}
|
|
140
|
+
|
|
141
|
+
async execute(query: GetCustomerOrdersQuery): Promise<CustomerOrderSummaryDTO[]> {
|
|
142
|
+
// Direct SQL projection - zero domain aggregate hydration overhead
|
|
143
|
+
return await this.db.query<CustomerOrderSummaryDTO>(
|
|
144
|
+
`SELECT order_id as "orderId", total_cents as "totalCents",
|
|
145
|
+
status, item_count as "itemCount", created_at as "createdAt"
|
|
146
|
+
FROM order_summaries_view
|
|
147
|
+
WHERE customer_id = $1
|
|
148
|
+
ORDER BY created_at DESC
|
|
149
|
+
LIMIT $2`,
|
|
150
|
+
[query.customerId, query.limit]
|
|
151
|
+
);
|
|
152
|
+
}
|
|
153
|
+
}
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
---
|
|
157
|
+
|
|
158
|
+
## 4. Asynchronous Projection Invariants (Level 3 CQRS)
|
|
159
|
+
|
|
160
|
+
When graduating to Level 3 (asynchronous read stores), you must prevent data corruption, drift, and race conditions:
|
|
161
|
+
|
|
162
|
+
### 1. Zero Dual-Writes (Strict Outbox Mandate)
|
|
163
|
+
Never write to the command database and then publish to a message broker (RabbitMQ/Kafka) in two independent operations. Network failure between the two operations corrupts read models. Always use the **Transactional Outbox Pattern** (see [`docs/rules/database_design.md`](./database_design.md)).
|
|
164
|
+
|
|
165
|
+
### 2. Monotonic Sequence & Idempotent Projectors
|
|
166
|
+
Every domain event must carry an aggregate version number. Read projectors must discard duplicate or out-of-order events:
|
|
167
|
+
```typescript
|
|
168
|
+
export async function projectOrderUpdated(event: OrderUpdatedEvent, db: ReadDb): Promise<void> {
|
|
169
|
+
// Idempotent upsert guarded by monotonic version sequence
|
|
170
|
+
await db.query(
|
|
171
|
+
`UPDATE order_read_models
|
|
172
|
+
SET status = $1, version = $2, updated_at = $3
|
|
173
|
+
WHERE order_id = $4 AND version < $2`,
|
|
174
|
+
[event.newStatus, event.version, event.occurredAt, event.orderId]
|
|
175
|
+
);
|
|
176
|
+
}
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
---
|
|
180
|
+
|
|
181
|
+
## 5. Defending Read-Your-Own-Writes Consistency
|
|
182
|
+
|
|
183
|
+
The biggest hazard of asynchronous CQRS is **eventual consistency lag**: a user submits a mutation (e.g. updating their display name), receives an HTTP `200 OK`, gets redirected to their profile, but still sees their old name because the background projection worker is lagging.
|
|
184
|
+
|
|
185
|
+
### Strategies to Eliminate User-Perceived Lag
|
|
186
|
+
1. **Optimistic UI Updates**: The frontend immediately updates the client cache with the submitted values upon HTTP 200 response, without refetching from the query API.
|
|
187
|
+
2. **Monotonic Version Headers**:
|
|
188
|
+
- The command response returns the new entity version: `ETag: "v14"`.
|
|
189
|
+
- Subsequent query requests include `If-None-Match: "v14"` or `X-Min-Version: 14`.
|
|
190
|
+
- If the read projection has not caught up to `v14`, the query service waits (up to 500ms) or falls back to querying the primary write database.
|
|
191
|
+
3. **Session Read-Model Pinning**: Direct read requests from the mutating user's session to the primary database for 5 seconds after a command execution, while general traffic continues reading from the read replica.
|
|
192
|
+
|
|
193
|
+
---
|
|
194
|
+
|
|
195
|
+
## 6. Gotchas & Anti-Patterns
|
|
196
|
+
|
|
197
|
+
| Anti-Pattern | Why It Fails | Mandated Correction |
|
|
198
|
+
|---|---|---|
|
|
199
|
+
| **CQRS as Default Architecture** | Multiplies boilerplate, schemas, and cognitive load 3x on simple applications (YAGNI violation). | Default to Single Model / Single Database. Graduate only on measured bottlenecks. |
|
|
200
|
+
| **Conflating CQRS with Event Sourcing** | Event Sourcing stores all state as event streams; CQRS only separates read and write pathways. ES adds severe schema migration complexity. | Implement CQRS with standard relational state. Only add Event Sourcing if the business requires legal/temporal audit ledgers. |
|
|
201
|
+
| **Leaking Entities into Queries** | Hydrating full Domain Aggregates inside Query Handlers causes severe N+1 query cascades and memory bloat. | Query Handlers must bypass entities and return flat, serializable DTOs directly from the database. |
|
|
202
|
+
| **Dual-Write Projections** | Emitting HTTP calls or message queue pushes directly inside mutation services causes silent sync failures. | Write events to a local `outbox` table in the same DB transaction as the aggregate state change. |
|
|
203
|
+
| **CRUD Queries via Aggregate Roots** | Loading an entire Aggregate Root just to read 2 fields creates unnecessary lock contention and memory footprint. | Use direct projection queries (Level 1 CQRS) for read-only flows. |
|
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
# Database Design, Relational Integrity & Transactions
|
|
2
|
+
|
|
3
|
+
> **Core Mandate:** Enforce relational constraints at the database engine level (FKs, CHECKs, EXCLUDEs), the Canonical 6 Total Audit Fields, partial soft-delete indexes, ACID transactional boundaries, and the Transactional Outbox pattern.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 1. Relational Integrity & Engine-Level Constraints
|
|
8
|
+
|
|
9
|
+
Data integrity must be enforced by the relational database engine, not delegated exclusively to application code:
|
|
10
|
+
|
|
11
|
+
```mermaid
|
|
12
|
+
flowchart TD
|
|
13
|
+
App["Application Layer<br/>Validates DTOs (Zod / Pydantic / Bean Validation)"] --> DB["Database Engine (Final Arbiter of Truth)"]
|
|
14
|
+
subgraph EngineRules["Engine Constraints"]
|
|
15
|
+
DB --> FK["FOREIGN KEY<br/>Referential Integrity"]
|
|
16
|
+
DB --> CHECK["CHECK<br/>Domain Invariants & Ranges"]
|
|
17
|
+
DB --> EXCLUDE["EXCLUDE USING gist<br/>Temporal Non-Overlaps"]
|
|
18
|
+
DB --> PARTIAL["Partial Unique Indexes<br/>UNIQUE ... WHERE deleted_at IS NULL"]
|
|
19
|
+
end
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
### Invariants:
|
|
23
|
+
1. **Mandatory Foreign Keys**: Every relational association must define an explicit foreign key constraint with explicit `ON DELETE` semantics (`ON DELETE RESTRICT` or `ON DELETE CASCADE`).
|
|
24
|
+
2. **Domain `CHECK` Constraints**: Enforce domain value ranges at the schema level:
|
|
25
|
+
```sql
|
|
26
|
+
ALTER TABLE orders ADD CONSTRAINT check_positive_amount CHECK (total_amount >= 0);
|
|
27
|
+
ALTER TABLE users ADD CONSTRAINT check_valid_role CHECK (role IN ('ADMIN', 'OPERATOR', 'MEMBER'));
|
|
28
|
+
```
|
|
29
|
+
3. **Temporal Non-Overlap (`EXCLUDE`)**: Scheduling, booking, or reservation systems must prevent double-booking using PostgreSQL range exclusion constraints:
|
|
30
|
+
```sql
|
|
31
|
+
ALTER TABLE bookings ADD CONSTRAINT no_overlapping_reservations
|
|
32
|
+
EXCLUDE USING gist (room_id WITH =, reservation_period WITH &&);
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
---
|
|
36
|
+
|
|
37
|
+
## 2. The Canonical 6 Total Audit Fields
|
|
38
|
+
|
|
39
|
+
Every mutable stateful entity in the database must implement the **Canonical 6 Total Audit Fields** to guarantee complete traceability for compliance and security audits:
|
|
40
|
+
|
|
41
|
+
```sql
|
|
42
|
+
CREATE TABLE resources (
|
|
43
|
+
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
|
44
|
+
tenant_id UUID NOT NULL,
|
|
45
|
+
name VARCHAR(255) NOT NULL,
|
|
46
|
+
|
|
47
|
+
-- The Canonical 6 Total Audit Fields
|
|
48
|
+
created_at TIMESTAMPTZ NOT NULL DEFAULT clock_timestamp(),
|
|
49
|
+
created_by UUID NOT NULL,
|
|
50
|
+
updated_at TIMESTAMPTZ NOT NULL DEFAULT clock_timestamp(),
|
|
51
|
+
updated_by UUID NOT NULL,
|
|
52
|
+
deleted_at TIMESTAMPTZ,
|
|
53
|
+
deleted_by UUID
|
|
54
|
+
);
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
### Rules & Append-Only Exception:
|
|
58
|
+
- **Mutable Tables:** All 6 fields are mandatory. Application repositories must automatically populate `createdBy`/`updatedBy` from the ambient authenticated user context.
|
|
59
|
+
- **Append-Only Event / Ledger Tables:** Immutable financial ledgers, audit logs, and domain events omit `updated_at`, `updated_by`, `deleted_at`, and `deleted_by` (immutable rows are never updated or deleted).
|
|
60
|
+
|
|
61
|
+
---
|
|
62
|
+
|
|
63
|
+
## 3. Soft-Delete Invariants & Partial Unique Indexes
|
|
64
|
+
|
|
65
|
+
When implementing soft-delete (`deleted_at IS NOT NULL`):
|
|
66
|
+
1. **Never Use Unfiltered Unique Constraints**: A standard `UNIQUE(slug)` constraint prevents creating a new record with the same slug after deleting an old record.
|
|
67
|
+
2. **Mandatory Partial Unique Index**:
|
|
68
|
+
```sql
|
|
69
|
+
-- Correct: Allows reusing slug after soft-deletion
|
|
70
|
+
CREATE UNIQUE INDEX idx_resources_tenant_slug_active
|
|
71
|
+
ON resources (tenant_id, slug)
|
|
72
|
+
WHERE deleted_at IS NULL;
|
|
73
|
+
```
|
|
74
|
+
3. **Soft-Delete Query Filtering**: All read queries must filter `WHERE deleted_at IS NULL` unless explicitly requesting archived records.
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## 4. ACID Transactions & Isolation Levels
|
|
79
|
+
|
|
80
|
+
Every operation mutating multiple database rows or coordinating interrelated aggregates must execute within an explicit **ACID Transaction**:
|
|
81
|
+
|
|
82
|
+
### Transaction Guidelines:
|
|
83
|
+
- **Single Aggregate Transactions**: Strive to keep transactions bounded to a single aggregate root.
|
|
84
|
+
- **Isolation Level Selection**:
|
|
85
|
+
- `READ COMMITTED` (Default): Acceptable for standard CRUD operations.
|
|
86
|
+
- `REPEATABLE READ` / `SERIALIZABLE`: Mandatory for financial ledger balances, inventory decrements, and ticket allocations where phantom reads cause double-spending.
|
|
87
|
+
- **Defensive Timeouts**: Transactions must never run indefinitely. Always set a defensive statement and transaction timeout:
|
|
88
|
+
```sql
|
|
89
|
+
SET LOCAL statement_timeout = '5s';
|
|
90
|
+
SET LOCAL idle_in_transaction_session_timeout = '10s';
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
---
|
|
94
|
+
|
|
95
|
+
## 5. The Transactional Outbox Pattern (Dual-Write Defense)
|
|
96
|
+
|
|
97
|
+
Never execute a database write and a message broker publish sequentially in application code. If the network fails between the two operations, the system enters an inconsistent, corrupted state (**The Dual-Write Anti-Pattern**).
|
|
98
|
+
|
|
99
|
+
```mermaid
|
|
100
|
+
sequenceDiagram
|
|
101
|
+
autonumber
|
|
102
|
+
actor Caller
|
|
103
|
+
participant App as Application Service
|
|
104
|
+
participant DB as PostgreSQL Database
|
|
105
|
+
participant Relay as CDC Relay / Worker
|
|
106
|
+
participant Broker as Message Broker (Kafka/SQS)
|
|
107
|
+
|
|
108
|
+
Caller->>App: Execute Command
|
|
109
|
+
activate App
|
|
110
|
+
Note over App,DB: Atomic ACID Transaction
|
|
111
|
+
App->>DB: 1. Mutate Domain State (UPDATE accounts...)
|
|
112
|
+
App->>DB: 2. Insert Outbox Event (INSERT INTO outbox_events...)
|
|
113
|
+
App->>DB: COMMIT
|
|
114
|
+
App-->>Caller: Success Response
|
|
115
|
+
deactivate App
|
|
116
|
+
|
|
117
|
+
Note over Relay,Broker: Asynchronous Processing
|
|
118
|
+
Relay->>DB: 3. Read unpublished outbox_events (CDC / Polling)
|
|
119
|
+
Relay->>Broker: 4. Publish Event to Broker
|
|
120
|
+
Relay->>DB: 5. Mark Event Published / Purge
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
### Outbox Invariants:
|
|
124
|
+
1. **Atomic Ingestion**: Domain entity mutation and event insertion must share the identical database transaction.
|
|
125
|
+
2. **Guaranteed At-Least-Once Delivery**: The outbox relay guarantees delivery to consumers; consumers must implement idempotency.
|