skillstore-cli 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +95 -0
- package/data/bundles/devflow-complete.json +19 -0
- package/data/free-skills/devflow-agile/manifest.json +19 -0
- package/data/free-skills/devflow-agile/plugin/commands/agile/retro.md +23 -0
- package/data/free-skills/devflow-agile/plugin/commands/agile/review.md +21 -0
- package/data/free-skills/devflow-agile/plugin/commands/agile/sprint.md +30 -0
- package/data/free-skills/devflow-agile/plugin/commands/agile/standup.md +20 -0
- package/data/free-skills/devflow-agile/plugin/commands/agile.md +35 -0
- package/data/free-skills/devflow-agile/plugin/commands/devflow.md +42 -0
- package/data/free-skills/devflow-agile/plugin/skills/developer/SKILL.md +93 -0
- package/data/free-skills/devflow-agile/plugin/skills/developer/assets/sample-output.md +182 -0
- package/data/free-skills/devflow-agile/plugin/skills/developer/references/clean-architecture.md +361 -0
- package/data/free-skills/devflow-agile/plugin/skills/developer/references/clean-code-guide.md +207 -0
- package/data/free-skills/devflow-agile/plugin/skills/developer/references/debugging-methodology.md +191 -0
- package/data/free-skills/devflow-agile/template/agents/agile-coach.md +76 -0
- package/data/free-skills/devflow-agile/template/workflows/agile-sprint-workflow.md +81 -0
- package/data/free-skills/devflow-bootstrap/manifest.json +8 -0
- package/data/free-skills/devflow-bootstrap/plugin/commands/bootstrap/auto.md +31 -0
- package/data/free-skills/devflow-bootstrap/plugin/commands/bootstrap.md +38 -0
- package/data/free-skills/devflow-bootstrap/plugin/commands/devflow.md +20 -0
- package/data/free-skills/devflow-bootstrap/plugin/skills/project-scaffold/SKILL.md +56 -0
- package/data/free-skills/devflow-bootstrap/plugin/skills/project-scaffold/assets/sample-output.md +216 -0
- package/data/free-skills/devflow-bootstrap/plugin/skills/project-scaffold/references/architecture-decisions.md +254 -0
- package/data/free-skills/devflow-bootstrap/plugin/skills/project-scaffold/references/stack-templates.md +400 -0
- package/data/free-skills/devflow-bootstrap/template/agents/bootstrap-specialist.md +56 -0
- package/data/free-skills/devflow-bootstrap/template/workflows/bootstrap-workflow.md +70 -0
- package/data/free-skills/devflow-docs/manifest.json +8 -0
- package/data/free-skills/devflow-docs/plugin/commands/devflow.md +20 -0
- package/data/free-skills/devflow-docs/plugin/commands/docs/generate.md +17 -0
- package/data/free-skills/devflow-docs/plugin/commands/docs/parse.md +19 -0
- package/data/free-skills/devflow-docs/plugin/commands/docs.md +26 -0
- package/data/free-skills/devflow-docs/plugin/skills/pdf-processor/SKILL.md +59 -0
- package/data/free-skills/devflow-docs/plugin/skills/pdf-processor/assets/sample-output.md +114 -0
- package/data/free-skills/devflow-docs/plugin/skills/pdf-processor/references/extraction-techniques.md +115 -0
- package/data/free-skills/devflow-docs/plugin/skills/pdf-processor/references/ocr-strategies.md +167 -0
- package/data/free-skills/devflow-docs/template/agents/docs-specialist.md +35 -0
- package/data/free-skills/devflow-docs/template/workflows/docs-workflow.md +70 -0
- package/data/free-skills/devflow-postproject/manifest.json +13 -0
- package/data/free-skills/devflow-postproject/plugin/commands/devflow.md +34 -0
- package/data/free-skills/devflow-postproject/plugin/commands/postproject/handover.md +21 -0
- package/data/free-skills/devflow-postproject/plugin/commands/postproject/retro.md +21 -0
- package/data/free-skills/devflow-postproject/plugin/commands/postproject/support.md +21 -0
- package/data/free-skills/devflow-postproject/plugin/commands/postproject.md +32 -0
- package/data/free-skills/devflow-postproject/plugin/skills/retrospective/SKILL.md +70 -0
- package/data/free-skills/devflow-postproject/plugin/skills/retrospective/assets/sample-output.md +79 -0
- package/data/free-skills/devflow-postproject/plugin/skills/retrospective/references/facilitation-techniques.md +178 -0
- package/data/free-skills/devflow-postproject/plugin/skills/retrospective/references/lessons-learned-template.md +118 -0
- package/data/free-skills/devflow-postproject/plugin/skills/retrospective/references/retro-techniques.md +100 -0
- package/data/free-skills/devflow-postproject/template/agents/transition-manager.md +71 -0
- package/data/free-skills/devflow-postproject/template/workflows/transition-workflow.md +72 -0
- package/data/free-skills/devflow-presale/manifest.json +15 -0
- package/data/free-skills/devflow-presale/plugin/commands/devflow.md +47 -0
- package/data/free-skills/devflow-presale/plugin/commands/presale/analyze.md +30 -0
- package/data/free-skills/devflow-presale/plugin/commands/presale/estimate.md +30 -0
- package/data/free-skills/devflow-presale/plugin/commands/presale/price.md +30 -0
- package/data/free-skills/devflow-presale/plugin/commands/presale/propose.md +30 -0
- package/data/free-skills/devflow-presale/plugin/commands/presale.md +42 -0
- package/data/free-skills/devflow-presale/plugin/skills/requirement-analysis/SKILL.md +63 -0
- package/data/free-skills/devflow-presale/plugin/skills/requirement-analysis/assets/sample-output.md +129 -0
- package/data/free-skills/devflow-presale/plugin/skills/requirement-analysis/references/extraction-framework.md +140 -0
- package/data/free-skills/devflow-presale/plugin/skills/requirement-analysis/references/output-template.md +132 -0
- package/data/free-skills/devflow-presale/template/agents/presale-lead.md +83 -0
- package/data/free-skills/devflow-presale/template/agents/proposal-reviewer.md +63 -0
- package/data/free-skills/devflow-presale/template/workflows/presale-workflow.md +70 -0
- package/data/registry/categories.json +7 -0
- package/data/registry/packages.json +184 -0
- package/data/shared/framework/agents/brainstormer.md +74 -0
- package/data/shared/framework/agents/code-reviewer.md +87 -0
- package/data/shared/framework/agents/debugger.md +84 -0
- package/data/shared/framework/agents/docs-manager.md +55 -0
- package/data/shared/framework/agents/git-manager.md +59 -0
- package/data/shared/framework/agents/planner.md +68 -0
- package/data/shared/framework/agents/researcher.md +66 -0
- package/data/shared/framework/agents/tester.md +65 -0
- package/data/shared/framework/commands/cook/auto.md +27 -0
- package/data/shared/framework/commands/cook.md +45 -0
- package/data/shared/framework/commands/fix/ci.md +21 -0
- package/data/shared/framework/commands/fix/test.md +26 -0
- package/data/shared/framework/commands/fix/types.md +29 -0
- package/data/shared/framework/commands/fix.md +26 -0
- package/data/shared/framework/commands/git/cm.md +37 -0
- package/data/shared/framework/commands/git/pr.md +40 -0
- package/data/shared/framework/config/CLAUDE.md.template +26 -0
- package/data/shared/framework/config/settings.json +41 -0
- package/data/shared/framework/config/skillstore.config.json +29 -0
- package/data/shared/framework/hooks/discord-notify.sh +85 -0
- package/data/shared/framework/hooks/docs-sync.sh +53 -0
- package/data/shared/framework/hooks/modularization-hook.js +103 -0
- package/data/shared/framework/hooks/notification.js +94 -0
- package/data/shared/framework/hooks/quality-gate.js +109 -0
- package/data/shared/framework/hooks/scout-block.js +77 -0
- package/data/shared/framework/hooks/telegram-notify.sh +77 -0
- package/data/shared/framework/protocols/error-recovery.md +80 -0
- package/data/shared/framework/protocols/orchestration-protocol.md +112 -0
- package/data/shared/framework/quality/review-protocol.md +76 -0
- package/data/shared/framework/quality/verification-protocol.md +66 -0
- package/data/shared/framework/rules/development-rules.md +75 -0
- package/data/shared/framework/skills/backend-development/SKILL.md +77 -0
- package/data/shared/framework/skills/backend-development/assets/sample-output.md +175 -0
- package/data/shared/framework/skills/backend-development/references/advanced-patterns.md +180 -0
- package/data/shared/framework/skills/backend-development/references/api-design-guide.md +160 -0
- package/data/shared/framework/skills/backend-development/references/architecture-patterns.md +183 -0
- package/data/shared/framework/skills/backend-development/references/observability-resilience.md +155 -0
- package/data/shared/framework/skills/backend-development/references/troubleshooting.md +199 -0
- package/data/shared/framework/skills/codebase-analysis/SKILL.md +72 -0
- package/data/shared/framework/skills/codebase-analysis/assets/sample-output.md +263 -0
- package/data/shared/framework/skills/codebase-analysis/references/analysis-techniques.md +241 -0
- package/data/shared/framework/skills/codebase-analysis/references/dependency-mapping.md +280 -0
- package/data/shared/framework/skills/codebase-analysis/references/tech-debt-assessment.md +208 -0
- package/data/shared/framework/skills/databases/SKILL.md +72 -0
- package/data/shared/framework/skills/databases/assets/sample-output.md +212 -0
- package/data/shared/framework/skills/databases/references/advanced-data-patterns.md +259 -0
- package/data/shared/framework/skills/databases/references/query-optimization.md +214 -0
- package/data/shared/framework/skills/databases/references/schema-design.md +159 -0
- package/data/shared/framework/skills/databases/references/troubleshooting.md +214 -0
- package/data/shared/framework/skills/debugging-investigation/SKILL.md +84 -0
- package/data/shared/framework/skills/debugging-investigation/assets/sample-output.md +314 -0
- package/data/shared/framework/skills/debugging-investigation/references/systematic-debugging.md +197 -0
- package/data/shared/framework/skills/debugging-investigation/references/tool-specific-guides.md +202 -0
- package/data/shared/framework/skills/debugging-investigation/references/troubleshooting-patterns.md +196 -0
- package/data/shared/framework/skills/frontend-development/SKILL.md +67 -0
- package/data/shared/framework/skills/frontend-development/assets/sample-output.md +110 -0
- package/data/shared/framework/skills/frontend-development/references/component-patterns.md +112 -0
- package/data/shared/framework/skills/frontend-development/references/performance-guide.md +169 -0
- package/data/shared/framework/skills/frontend-development/references/routing-forms-realtime.md +374 -0
- package/data/shared/framework/skills/frontend-development/references/ssr-rsc-patterns.md +284 -0
- package/data/shared/framework/skills/frontend-development/references/troubleshooting.md +154 -0
- package/data/shared/framework/skills/mobile-development/SKILL.md +67 -0
- package/data/shared/framework/skills/mobile-development/assets/sample-output.md +382 -0
- package/data/shared/framework/skills/mobile-development/references/mobile-patterns.md +681 -0
- package/data/shared/framework/skills/mobile-development/references/mobile-performance.md +524 -0
- package/data/shared/framework/skills/mobile-development/references/troubleshooting.md +158 -0
- package/data/shared/framework/skills/security-audit/SKILL.md +83 -0
- package/data/shared/framework/skills/security-audit/assets/sample-output.md +451 -0
- package/data/shared/framework/skills/security-audit/references/owasp-checklist.md +580 -0
- package/data/shared/framework/skills/security-audit/references/secure-coding-patterns.md +433 -0
- package/data/shared/framework/skills/security-audit/references/vulnerability-remediation.md +331 -0
- package/data/shared/framework/skills/ui-generation/SKILL.md +70 -0
- package/data/shared/framework/skills/ui-generation/assets/sample-output.md +139 -0
- package/data/shared/framework/skills/ui-generation/references/accessibility-responsive.md +127 -0
- package/data/shared/framework/skills/ui-generation/references/compound-components.md +252 -0
- package/data/shared/framework/skills/ui-generation/references/generation-patterns.md +110 -0
- package/data/shared/framework/skills/ui-generation/references/storybook-design-system.md +278 -0
- package/data/shared/framework/skills/ui-generation/references/troubleshooting.md +198 -0
- package/data/shared/framework/workflows/documentation-management.md +58 -0
- package/data/shared/framework/workflows/primary-workflow.md +88 -0
- package/dist/commands/activate.d.ts +3 -0
- package/dist/commands/activate.d.ts.map +1 -0
- package/dist/commands/activate.js +34 -0
- package/dist/commands/activate.js.map +1 -0
- package/dist/commands/bundle.d.ts +3 -0
- package/dist/commands/bundle.d.ts.map +1 -0
- package/dist/commands/bundle.js +64 -0
- package/dist/commands/bundle.js.map +1 -0
- package/dist/commands/install.d.ts +3 -0
- package/dist/commands/install.d.ts.map +1 -0
- package/dist/commands/install.js +99 -0
- package/dist/commands/install.js.map +1 -0
- package/dist/commands/list.d.ts +3 -0
- package/dist/commands/list.d.ts.map +1 -0
- package/dist/commands/list.js +37 -0
- package/dist/commands/list.js.map +1 -0
- package/dist/commands/search.d.ts +3 -0
- package/dist/commands/search.d.ts.map +1 -0
- package/dist/commands/search.js +30 -0
- package/dist/commands/search.js.map +1 -0
- package/dist/commands/status.d.ts +3 -0
- package/dist/commands/status.d.ts.map +1 -0
- package/dist/commands/status.js +35 -0
- package/dist/commands/status.js.map +1 -0
- package/dist/commands/update.d.ts +3 -0
- package/dist/commands/update.d.ts.map +1 -0
- package/dist/commands/update.js +68 -0
- package/dist/commands/update.js.map +1 -0
- package/dist/download/cache.d.ts +3 -0
- package/dist/download/cache.d.ts.map +1 -0
- package/dist/download/cache.js +18 -0
- package/dist/download/cache.js.map +1 -0
- package/dist/download/client.d.ts +2 -0
- package/dist/download/client.d.ts.map +1 -0
- package/dist/download/client.js +58 -0
- package/dist/download/client.js.map +1 -0
- package/dist/index.d.ts +3 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +23 -0
- package/dist/index.js.map +1 -0
- package/dist/installer/file-copier.d.ts +6 -0
- package/dist/installer/file-copier.d.ts.map +1 -0
- package/dist/installer/file-copier.js +32 -0
- package/dist/installer/file-copier.js.map +1 -0
- package/dist/installer/plugin-installer.d.ts +12 -0
- package/dist/installer/plugin-installer.d.ts.map +1 -0
- package/dist/installer/plugin-installer.js +33 -0
- package/dist/installer/plugin-installer.js.map +1 -0
- package/dist/installer/template-installer.d.ts +12 -0
- package/dist/installer/template-installer.d.ts.map +1 -0
- package/dist/installer/template-installer.js +45 -0
- package/dist/installer/template-installer.js.map +1 -0
- package/dist/license/crypto.d.ts +16 -0
- package/dist/license/crypto.d.ts.map +1 -0
- package/dist/license/crypto.js +50 -0
- package/dist/license/crypto.js.map +1 -0
- package/dist/license/license-store.d.ts +19 -0
- package/dist/license/license-store.d.ts.map +1 -0
- package/dist/license/license-store.js +99 -0
- package/dist/license/license-store.js.map +1 -0
- package/dist/license/validator.d.ts +32 -0
- package/dist/license/validator.d.ts.map +1 -0
- package/dist/license/validator.js +81 -0
- package/dist/license/validator.js.map +1 -0
- package/dist/registry/loader.d.ts +30 -0
- package/dist/registry/loader.d.ts.map +1 -0
- package/dist/registry/loader.js +22 -0
- package/dist/registry/loader.js.map +1 -0
- package/dist/registry/search-engine.d.ts +9 -0
- package/dist/registry/search-engine.d.ts.map +1 -0
- package/dist/registry/search-engine.js +30 -0
- package/dist/registry/search-engine.js.map +1 -0
- package/dist/utils/config.d.ts +14 -0
- package/dist/utils/config.d.ts.map +1 -0
- package/dist/utils/config.js +28 -0
- package/dist/utils/config.js.map +1 -0
- package/dist/utils/logger.d.ts +9 -0
- package/dist/utils/logger.d.ts.map +1 -0
- package/dist/utils/logger.js +22 -0
- package/dist/utils/logger.js.map +1 -0
- package/dist/utils/paths.d.ts +20 -0
- package/dist/utils/paths.d.ts.map +1 -0
- package/dist/utils/paths.js +79 -0
- package/dist/utils/paths.js.map +1 -0
- package/package.json +54 -0
|
@@ -0,0 +1,259 @@
|
|
|
1
|
+
# Advanced Data Patterns
|
|
2
|
+
|
|
3
|
+
## Sharding Strategies
|
|
4
|
+
|
|
5
|
+
### Decision Matrix
|
|
6
|
+
|
|
7
|
+
| Strategy | Best For | Shard Key Example | Trade-off |
|
|
8
|
+
|---|---|---|---|
|
|
9
|
+
| Hash-based | Even distribution, no range queries needed | `hash(user_id) % N` | Cannot do efficient range scans |
|
|
10
|
+
| Range-based | Time-series, sequential access patterns | `created_date`, `region_id` | Risk of hot spots on recent ranges |
|
|
11
|
+
| Geographic | Data locality, compliance (GDPR, residency) | `country_code`, `region` | Cross-region queries are expensive |
|
|
12
|
+
|
|
13
|
+
### Shard Key Selection Criteria
|
|
14
|
+
|
|
15
|
+
Choose a shard key that satisfies:
|
|
16
|
+
1. **High cardinality** — enough distinct values to distribute evenly across shards
|
|
17
|
+
2. **Even distribution** — no single value dominates (avoid skewed keys like `status`)
|
|
18
|
+
3. **Query isolation** — most queries include the shard key, avoiding scatter-gather
|
|
19
|
+
4. **Immutability** — the key should not change after insertion (resharding is expensive)
|
|
20
|
+
|
|
21
|
+
```sql
|
|
22
|
+
-- Example: hash-based sharding in application layer (PostgreSQL + Citus)
|
|
23
|
+
SELECT create_distributed_table('orders', 'customer_id');
|
|
24
|
+
|
|
25
|
+
-- Verify shard distribution
|
|
26
|
+
SELECT nodename, count(*)
|
|
27
|
+
FROM citus_shards
|
|
28
|
+
GROUP BY nodename;
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
### Cross-Shard Query Handling
|
|
32
|
+
|
|
33
|
+
- **Scatter-gather**: query all shards, merge results — acceptable for analytics, not for OLTP hot paths
|
|
34
|
+
- **Global tables**: replicate small reference tables (countries, currencies) to every shard
|
|
35
|
+
- **Entity groups**: co-locate related tables on the same shard key (orders + order_items both sharded by customer_id)
|
|
36
|
+
|
|
37
|
+
### Rebalancing
|
|
38
|
+
|
|
39
|
+
- **Citus**: `SELECT rebalance_table_shards('orders');` — moves shards to equalize storage
|
|
40
|
+
- **Vitess**: `Reshard` workflow splits or merges shards with zero downtime
|
|
41
|
+
- **Rule of thumb**: plan rebalancing when any shard exceeds 1.5× average size or 80% disk capacity
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Change Data Capture (CDC)
|
|
46
|
+
|
|
47
|
+
### Debezium + PostgreSQL Setup
|
|
48
|
+
|
|
49
|
+
1. **Enable logical replication** in `postgresql.conf`:
|
|
50
|
+
```
|
|
51
|
+
wal_level = logical
|
|
52
|
+
max_replication_slots = 4
|
|
53
|
+
max_wal_senders = 4
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
2. **Create a publication**:
|
|
57
|
+
```sql
|
|
58
|
+
CREATE PUBLICATION cdc_publication FOR TABLE orders, customers;
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
3. **Debezium connector configuration** (Kafka Connect JSON):
|
|
62
|
+
```json
|
|
63
|
+
{
|
|
64
|
+
"name": "pg-cdc-connector",
|
|
65
|
+
"config": {
|
|
66
|
+
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
|
|
67
|
+
"database.hostname": "db-primary",
|
|
68
|
+
"database.port": "5432",
|
|
69
|
+
"database.dbname": "app_db",
|
|
70
|
+
"database.user": "cdc_user",
|
|
71
|
+
"plugin.name": "pgoutput",
|
|
72
|
+
"publication.name": "cdc_publication",
|
|
73
|
+
"topic.prefix": "app",
|
|
74
|
+
"table.include.list": "public.orders,public.customers",
|
|
75
|
+
"snapshot.mode": "initial",
|
|
76
|
+
"tombstones.on.delete": true,
|
|
77
|
+
"transforms": "unwrap",
|
|
78
|
+
"transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState"
|
|
79
|
+
}
|
|
80
|
+
}
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
4. **Event structure** — each change event contains:
|
|
84
|
+
- `before`: row state before the change (null for inserts)
|
|
85
|
+
- `after`: row state after the change (null for deletes)
|
|
86
|
+
- `source`: metadata (lsn, txId, timestamp, table, schema)
|
|
87
|
+
- `op`: operation type — `c` (create), `u` (update), `d` (delete), `r` (snapshot read)
|
|
88
|
+
|
|
89
|
+
### CDC Use Cases
|
|
90
|
+
- Sync data to Elasticsearch or analytics warehouse
|
|
91
|
+
- Trigger downstream microservice workflows on data changes
|
|
92
|
+
- Build audit logs without application-level instrumentation
|
|
93
|
+
- Cache invalidation in Redis based on database writes
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## Temporal Tables
|
|
98
|
+
|
|
99
|
+
### PostgreSQL (temporal_tables Extension)
|
|
100
|
+
|
|
101
|
+
```sql
|
|
102
|
+
-- Install the extension
|
|
103
|
+
CREATE EXTENSION IF NOT EXISTS temporal_tables;
|
|
104
|
+
|
|
105
|
+
-- Create the main table
|
|
106
|
+
CREATE TABLE products (
|
|
107
|
+
id SERIAL PRIMARY KEY,
|
|
108
|
+
name TEXT NOT NULL,
|
|
109
|
+
price NUMERIC(10,2) NOT NULL,
|
|
110
|
+
sys_period tstzrange NOT NULL DEFAULT tstzrange(current_timestamp, null)
|
|
111
|
+
);
|
|
112
|
+
|
|
113
|
+
-- Create the history table (same structure)
|
|
114
|
+
CREATE TABLE products_history (LIKE products);
|
|
115
|
+
|
|
116
|
+
-- Attach the trigger
|
|
117
|
+
CREATE TRIGGER products_versioning
|
|
118
|
+
BEFORE INSERT OR UPDATE OR DELETE ON products
|
|
119
|
+
FOR EACH ROW EXECUTE FUNCTION versioning(
|
|
120
|
+
'sys_period', 'products_history', true
|
|
121
|
+
);
|
|
122
|
+
|
|
123
|
+
-- Query: what was the price on a specific date?
|
|
124
|
+
SELECT name, price
|
|
125
|
+
FROM products_history
|
|
126
|
+
WHERE id = 42
|
|
127
|
+
AND sys_period @> '2025-06-15T00:00:00Z'::timestamptz;
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
### SQL Server (System-Versioned)
|
|
131
|
+
|
|
132
|
+
```sql
|
|
133
|
+
CREATE TABLE products (
|
|
134
|
+
id INT PRIMARY KEY,
|
|
135
|
+
name NVARCHAR(200) NOT NULL,
|
|
136
|
+
price DECIMAL(10,2) NOT NULL,
|
|
137
|
+
valid_from DATETIME2 GENERATED ALWAYS AS ROW START,
|
|
138
|
+
valid_to DATETIME2 GENERATED ALWAYS AS ROW END,
|
|
139
|
+
PERIOD FOR SYSTEM_TIME (valid_from, valid_to)
|
|
140
|
+
) WITH (SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.products_history));
|
|
141
|
+
|
|
142
|
+
-- Point-in-time query
|
|
143
|
+
SELECT * FROM products
|
|
144
|
+
FOR SYSTEM_TIME AS OF '2025-06-15T00:00:00';
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
## Full-Text Search: PostgreSQL vs Elasticsearch
|
|
150
|
+
|
|
151
|
+
### Decision Guide
|
|
152
|
+
|
|
153
|
+
| Factor | PostgreSQL tsvector/tsquery | Elasticsearch |
|
|
154
|
+
|---|---|---|
|
|
155
|
+
| Data under 10M docs, simple search | Preferred — no extra infra | Overkill |
|
|
156
|
+
| Fuzzy matching, typo tolerance | Limited (trigram + similarity) | Excellent (built-in fuzziness) |
|
|
157
|
+
| Faceted search, aggregations | Weak | Excellent |
|
|
158
|
+
| Relevance tuning | Basic (weights A-D) | Fine-grained (BM25, boosting, function_score) |
|
|
159
|
+
| Operational cost | Zero — runs inside your DB | Separate cluster to manage |
|
|
160
|
+
| Real-time indexing | Immediate (same transaction) | Near real-time (~1s refresh) |
|
|
161
|
+
|
|
162
|
+
### PostgreSQL Full-Text Setup
|
|
163
|
+
|
|
164
|
+
```sql
|
|
165
|
+
-- Add a tsvector column with GIN index
|
|
166
|
+
ALTER TABLE articles ADD COLUMN search_vector tsvector
|
|
167
|
+
GENERATED ALWAYS AS (
|
|
168
|
+
setweight(to_tsvector('english', coalesce(title, '')), 'A') ||
|
|
169
|
+
setweight(to_tsvector('english', coalesce(body, '')), 'B')
|
|
170
|
+
) STORED;
|
|
171
|
+
|
|
172
|
+
CREATE INDEX idx_articles_search ON articles USING GIN (search_vector);
|
|
173
|
+
|
|
174
|
+
-- Search with ranking
|
|
175
|
+
SELECT title, ts_rank(search_vector, query) AS rank
|
|
176
|
+
FROM articles, plainto_tsquery('english', 'database sharding') AS query
|
|
177
|
+
WHERE search_vector @@ query
|
|
178
|
+
ORDER BY rank DESC
|
|
179
|
+
LIMIT 20;
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
---
|
|
183
|
+
|
|
184
|
+
## Database-per-Service (Microservices)
|
|
185
|
+
|
|
186
|
+
### Pros
|
|
187
|
+
- Independent schema evolution — no cross-team migration coordination
|
|
188
|
+
- Technology freedom — use PostgreSQL for one service, MongoDB for another
|
|
189
|
+
- Fault isolation — one database failure does not cascade
|
|
190
|
+
- Independent scaling of storage per service
|
|
191
|
+
|
|
192
|
+
### Cons
|
|
193
|
+
- **Cross-service queries** require API composition or CQRS read models
|
|
194
|
+
- **Distributed transactions** are hard — use Saga pattern instead of 2PC
|
|
195
|
+
- **Data duplication** is inevitable; accept eventual consistency
|
|
196
|
+
- **Operational overhead** — more databases to monitor, back up, patch
|
|
197
|
+
|
|
198
|
+
### Data Consistency Patterns
|
|
199
|
+
- **Saga (choreography)**: services emit events; each service reacts and compensates on failure
|
|
200
|
+
- **Saga (orchestration)**: a central coordinator directs the transaction steps
|
|
201
|
+
- **Outbox pattern**: write events to an outbox table in the same transaction, then publish asynchronously
|
|
202
|
+
- **CQRS**: separate command and query models; sync via events for cross-service reads
|
|
203
|
+
|
|
204
|
+
---
|
|
205
|
+
|
|
206
|
+
## Multi-Tenant Data Isolation
|
|
207
|
+
|
|
208
|
+
### Strategy Comparison
|
|
209
|
+
|
|
210
|
+
| Strategy | Isolation | Cost | Complexity | Best For |
|
|
211
|
+
|---|---|---|---|---|
|
|
212
|
+
| Separate databases | Strongest | Highest | Low code, high ops | Enterprise / regulated |
|
|
213
|
+
| Schema-per-tenant | Strong | Medium | Medium | Mid-market SaaS |
|
|
214
|
+
| Row-level security | Weakest | Lowest | Medium code, low ops | High-volume SMB SaaS |
|
|
215
|
+
|
|
216
|
+
### Schema-per-Tenant (PostgreSQL)
|
|
217
|
+
|
|
218
|
+
```sql
|
|
219
|
+
-- Create a schema for each tenant on signup
|
|
220
|
+
CREATE SCHEMA tenant_acme;
|
|
221
|
+
CREATE TABLE tenant_acme.orders (LIKE public.orders_template INCLUDING ALL);
|
|
222
|
+
|
|
223
|
+
-- Set search_path per connection
|
|
224
|
+
SET search_path TO tenant_acme, public;
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
### Row-Level Security (RLS)
|
|
228
|
+
|
|
229
|
+
```sql
|
|
230
|
+
-- Enable RLS on the table
|
|
231
|
+
ALTER TABLE orders ENABLE ROW LEVEL SECURITY;
|
|
232
|
+
|
|
233
|
+
-- Policy: users see only their tenant's rows
|
|
234
|
+
CREATE POLICY tenant_isolation ON orders
|
|
235
|
+
USING (tenant_id = current_setting('app.current_tenant')::uuid);
|
|
236
|
+
|
|
237
|
+
-- Force RLS even for table owners
|
|
238
|
+
ALTER TABLE orders FORCE ROW LEVEL SECURITY;
|
|
239
|
+
|
|
240
|
+
-- Application sets tenant context per request
|
|
241
|
+
SET LOCAL app.current_tenant = 'a1b2c3d4-...';
|
|
242
|
+
|
|
243
|
+
-- Now all queries are automatically filtered
|
|
244
|
+
SELECT * FROM orders; -- returns only current tenant's orders
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
### Application-Layer Setup (Node.js Example)
|
|
248
|
+
|
|
249
|
+
```typescript
|
|
250
|
+
// Middleware: set tenant context on every request
|
|
251
|
+
app.use(async (req, res, next) => {
|
|
252
|
+
const tenantId = req.headers['x-tenant-id'];
|
|
253
|
+
if (!tenantId) return res.status(400).json({ error: 'Tenant ID required' });
|
|
254
|
+
|
|
255
|
+
// Set RLS context for this database session
|
|
256
|
+
await db.query(`SET LOCAL app.current_tenant = $1`, [tenantId]);
|
|
257
|
+
next();
|
|
258
|
+
});
|
|
259
|
+
```
|
|
@@ -0,0 +1,214 @@
|
|
|
1
|
+
# Query Optimization
|
|
2
|
+
|
|
3
|
+
## EXPLAIN / ANALYZE Interpretation
|
|
4
|
+
|
|
5
|
+
### PostgreSQL
|
|
6
|
+
```sql
|
|
7
|
+
EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT) SELECT * FROM orders WHERE user_id = 'abc' AND status = 'open';
|
|
8
|
+
```
|
|
9
|
+
|
|
10
|
+
### Key Metrics to Check
|
|
11
|
+
|
|
12
|
+
| Metric | Good | Investigate |
|
|
13
|
+
|--------|------|-------------|
|
|
14
|
+
| Seq Scan on large table | < 1,000 rows | > 10,000 rows |
|
|
15
|
+
| Nested Loop | Inner table < 100 rows | Inner table > 1,000 rows |
|
|
16
|
+
| Sort | In-memory | On disk (work_mem too low) |
|
|
17
|
+
| Actual rows vs. estimated | Within 10x | Off by > 100x (stale stats) |
|
|
18
|
+
| Buffers shared hit ratio | > 95% | < 80% (increase shared_buffers) |
|
|
19
|
+
|
|
20
|
+
### Common EXPLAIN Nodes
|
|
21
|
+
- **Seq Scan**: Full table scan. OK for small tables. Add index for large tables.
|
|
22
|
+
- **Index Scan**: Uses index to find rows, then fetches from table. Good.
|
|
23
|
+
- **Index Only Scan**: All data from index, no table fetch. Best case.
|
|
24
|
+
- **Bitmap Index Scan**: Combines multiple indexes or handles many matches. OK.
|
|
25
|
+
- **Hash Join / Merge Join**: Joining two sets. Hash is better for unequal sizes, Merge for pre-sorted data.
|
|
26
|
+
|
|
27
|
+
### When Stats Are Stale
|
|
28
|
+
```sql
|
|
29
|
+
-- Force PostgreSQL to update statistics
|
|
30
|
+
ANALYZE orders;
|
|
31
|
+
|
|
32
|
+
-- Check when stats were last updated
|
|
33
|
+
SELECT relname, last_analyze, last_autoanalyze FROM pg_stat_user_tables WHERE relname = 'orders';
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
## Common Query Anti-Patterns
|
|
37
|
+
|
|
38
|
+
### 1. SELECT * (Fetching All Columns)
|
|
39
|
+
```sql
|
|
40
|
+
-- Bad: fetches all columns including large TEXT/BLOB fields
|
|
41
|
+
SELECT * FROM users WHERE status = 'active';
|
|
42
|
+
|
|
43
|
+
-- Good: fetch only what you need
|
|
44
|
+
SELECT id, name, email FROM users WHERE status = 'active';
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
### 2. N+1 Queries
|
|
48
|
+
```python
|
|
49
|
+
# Bad: 1 query for orders + N queries for users (one per order)
|
|
50
|
+
orders = db.query("SELECT * FROM orders LIMIT 100")
|
|
51
|
+
for order in orders:
|
|
52
|
+
user = db.query("SELECT * FROM users WHERE id = %s", order.user_id)
|
|
53
|
+
|
|
54
|
+
# Good: 1 query with JOIN or 2 queries with IN
|
|
55
|
+
orders = db.query("""
|
|
56
|
+
SELECT o.*, u.name as user_name
|
|
57
|
+
FROM orders o JOIN users u ON o.user_id = u.id
|
|
58
|
+
LIMIT 100
|
|
59
|
+
""")
|
|
60
|
+
|
|
61
|
+
# Also good: batch fetch
|
|
62
|
+
user_ids = [o.user_id for o in orders]
|
|
63
|
+
users = db.query("SELECT * FROM users WHERE id = ANY(%s)", user_ids)
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
### 3. Missing Indexes on Foreign Keys
|
|
67
|
+
Every foreign key column should have an index. Without it, JOINs and cascading deletes trigger full table scans.
|
|
68
|
+
|
|
69
|
+
### 4. OR vs. UNION
|
|
70
|
+
```sql
|
|
71
|
+
-- Bad: OR often prevents index usage
|
|
72
|
+
SELECT * FROM tasks WHERE assignee_id = 'a' OR creator_id = 'a';
|
|
73
|
+
|
|
74
|
+
-- Better: UNION uses both indexes
|
|
75
|
+
SELECT * FROM tasks WHERE assignee_id = 'a'
|
|
76
|
+
UNION
|
|
77
|
+
SELECT * FROM tasks WHERE creator_id = 'a';
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
### 5. Functions on Indexed Columns
|
|
81
|
+
```sql
|
|
82
|
+
-- Bad: index on created_at is useless because of DATE()
|
|
83
|
+
SELECT * FROM orders WHERE DATE(created_at) = '2026-03-20';
|
|
84
|
+
|
|
85
|
+
-- Good: use range query
|
|
86
|
+
SELECT * FROM orders WHERE created_at >= '2026-03-20' AND created_at < '2026-03-21';
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
### 6. OFFSET for Deep Pagination
|
|
90
|
+
```sql
|
|
91
|
+
-- Bad: database reads and discards 10,000 rows
|
|
92
|
+
SELECT * FROM products ORDER BY id LIMIT 25 OFFSET 10000;
|
|
93
|
+
|
|
94
|
+
-- Good: cursor-based pagination (keyset)
|
|
95
|
+
SELECT * FROM products WHERE id > 'last-seen-id' ORDER BY id LIMIT 25;
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
## Index Optimization Strategies
|
|
99
|
+
|
|
100
|
+
### Composite Index Column Order
|
|
101
|
+
Put the most selective column first (highest cardinality):
|
|
102
|
+
```sql
|
|
103
|
+
-- If user_id has 100,000 unique values and status has 5:
|
|
104
|
+
CREATE INDEX idx_tasks_user_status ON tasks (user_id, status); -- Good
|
|
105
|
+
CREATE INDEX idx_tasks_status_user ON tasks (status, user_id); -- Less effective
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
### Index for Sorting
|
|
109
|
+
```sql
|
|
110
|
+
-- Query: WHERE status = 'open' ORDER BY created_at DESC
|
|
111
|
+
-- Index must match both filter and sort:
|
|
112
|
+
CREATE INDEX idx_tasks_status_created ON tasks (status, created_at DESC);
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
### Partial Indexes for Hot Data
|
|
116
|
+
```sql
|
|
117
|
+
-- 90% of queries are for active records
|
|
118
|
+
CREATE INDEX idx_active_users ON users (email) WHERE is_active = true;
|
|
119
|
+
-- Much smaller index, faster lookups
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
### Monitor Unused Indexes
|
|
123
|
+
```sql
|
|
124
|
+
-- PostgreSQL: find indexes with zero scans
|
|
125
|
+
SELECT indexrelname, idx_scan, pg_size_pretty(pg_relation_size(indexrelid))
|
|
126
|
+
FROM pg_stat_user_indexes
|
|
127
|
+
WHERE idx_scan = 0 AND indexrelname NOT LIKE '%pkey'
|
|
128
|
+
ORDER BY pg_relation_size(indexrelid) DESC;
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
## Connection Pooling
|
|
132
|
+
|
|
133
|
+
### Why
|
|
134
|
+
Database connections are expensive (TCP handshake, auth, memory allocation). A pool reuses connections.
|
|
135
|
+
|
|
136
|
+
### Configuration Guidelines
|
|
137
|
+
|
|
138
|
+
| App Type | Pool Size | Formula |
|
|
139
|
+
|----------|-----------|---------|
|
|
140
|
+
| Web server | 10-20 per instance | `connections = (cores * 2) + disk_spindles` |
|
|
141
|
+
| Background workers | 3-5 per worker | Fewer concurrent queries needed |
|
|
142
|
+
| Total max | Check DB limit | PostgreSQL default: 100 connections |
|
|
143
|
+
|
|
144
|
+
### Tools
|
|
145
|
+
- **Application-level**: HikariCP (Java), pgBouncer (PostgreSQL), Prisma connection pool
|
|
146
|
+
- **Standalone proxy**: PgBouncer, ProxySQL (MySQL)
|
|
147
|
+
|
|
148
|
+
### PgBouncer Modes
|
|
149
|
+
| Mode | When |
|
|
150
|
+
|------|------|
|
|
151
|
+
| Transaction | Default choice. Releases connection after each transaction. |
|
|
152
|
+
| Session | App uses prepared statements or temp tables. |
|
|
153
|
+
| Statement | High-throughput, no transactions needed. |
|
|
154
|
+
|
|
155
|
+
## Read Replicas and Partitioning
|
|
156
|
+
|
|
157
|
+
### Read Replicas
|
|
158
|
+
- Route read queries (reports, search, dashboards) to replicas
|
|
159
|
+
- Keep writes on primary
|
|
160
|
+
- Handle replication lag (data may be 0-5 seconds behind)
|
|
161
|
+
- Use for: analytics queries, search indexes, backup
|
|
162
|
+
|
|
163
|
+
### Table Partitioning
|
|
164
|
+
```sql
|
|
165
|
+
-- Partition by range (time-series data)
|
|
166
|
+
CREATE TABLE events (id UUID, created_at TIMESTAMPTZ, payload JSONB)
|
|
167
|
+
PARTITION BY RANGE (created_at);
|
|
168
|
+
|
|
169
|
+
CREATE TABLE events_2026_q1 PARTITION OF events
|
|
170
|
+
FOR VALUES FROM ('2026-01-01') TO ('2026-04-01');
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
**When to partition**: table exceeds 50-100 million rows, or queries always filter by partition key.
|
|
174
|
+
|
|
175
|
+
## Migration Strategies
|
|
176
|
+
|
|
177
|
+
### Zero-Downtime Migrations (Expand-Contract)
|
|
178
|
+
|
|
179
|
+
**Adding a column:**
|
|
180
|
+
1. Add column as nullable (no lock)
|
|
181
|
+
2. Deploy code that writes to both old and new columns
|
|
182
|
+
3. Backfill existing rows in batches
|
|
183
|
+
4. Deploy code that reads from new column
|
|
184
|
+
5. Add NOT NULL constraint (if needed)
|
|
185
|
+
6. Remove old column usage from code
|
|
186
|
+
|
|
187
|
+
**Renaming a column:**
|
|
188
|
+
1. Add new column
|
|
189
|
+
2. Write to both columns
|
|
190
|
+
3. Backfill
|
|
191
|
+
4. Read from new column
|
|
192
|
+
5. Drop old column
|
|
193
|
+
|
|
194
|
+
### Migration Best Practices
|
|
195
|
+
- Every migration must have a rollback (down migration)
|
|
196
|
+
- Test migrations on production-size data locally
|
|
197
|
+
- Run migrations in a transaction when possible
|
|
198
|
+
- Set a statement timeout to prevent long locks: `SET lock_timeout = '5s'`
|
|
199
|
+
- Never run `ALTER TABLE ... ADD COLUMN ... DEFAULT` on large tables in older PostgreSQL (< 11) — it rewrites the entire table
|
|
200
|
+
|
|
201
|
+
## Backup and Recovery
|
|
202
|
+
|
|
203
|
+
### Backup Types
|
|
204
|
+
| Type | Frequency | Use Case |
|
|
205
|
+
|------|-----------|----------|
|
|
206
|
+
| Full backup (pg_dump) | Daily | Point-in-time recovery base |
|
|
207
|
+
| WAL archiving | Continuous | Point-in-time recovery to any second |
|
|
208
|
+
| Logical replication | Continuous | Cross-version or selective backup |
|
|
209
|
+
|
|
210
|
+
### Testing Recovery
|
|
211
|
+
- Schedule monthly recovery drills
|
|
212
|
+
- Verify backup integrity: restore to a test instance and run validation queries
|
|
213
|
+
- Measure Recovery Time Objective (RTO) — how long to restore
|
|
214
|
+
- Measure Recovery Point Objective (RPO) — maximum data loss acceptable
|
|
@@ -0,0 +1,159 @@
|
|
|
1
|
+
# Schema Design
|
|
2
|
+
|
|
3
|
+
## Normalization
|
|
4
|
+
|
|
5
|
+
### 1NF (First Normal Form)
|
|
6
|
+
- Every column contains atomic (indivisible) values
|
|
7
|
+
- No repeating groups or arrays in a single column
|
|
8
|
+
|
|
9
|
+
```sql
|
|
10
|
+
-- Bad: tags stored as comma-separated string
|
|
11
|
+
CREATE TABLE products (id INT, name TEXT, tags TEXT); -- tags = "electronics,sale,new"
|
|
12
|
+
|
|
13
|
+
-- Good: separate table for tags
|
|
14
|
+
CREATE TABLE products (id INT PRIMARY KEY, name TEXT);
|
|
15
|
+
CREATE TABLE product_tags (product_id INT REFERENCES products(id), tag TEXT);
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
### 2NF (Second Normal Form)
|
|
19
|
+
- Must be in 1NF
|
|
20
|
+
- Every non-key column depends on the ENTIRE primary key (no partial dependencies)
|
|
21
|
+
|
|
22
|
+
### 3NF (Third Normal Form)
|
|
23
|
+
- Must be in 2NF
|
|
24
|
+
- No transitive dependencies (non-key column depending on another non-key column)
|
|
25
|
+
|
|
26
|
+
```sql
|
|
27
|
+
-- Bad: city_name depends on zip_code, not on order_id
|
|
28
|
+
CREATE TABLE orders (id INT, zip_code TEXT, city_name TEXT);
|
|
29
|
+
|
|
30
|
+
-- Good: separate addresses table
|
|
31
|
+
CREATE TABLE orders (id INT, address_id INT REFERENCES addresses(id));
|
|
32
|
+
CREATE TABLE addresses (id INT PRIMARY KEY, zip_code TEXT, city_name TEXT);
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
### BCNF (Boyce-Codd Normal Form)
|
|
36
|
+
- Every determinant is a candidate key
|
|
37
|
+
- Rarely needed in practice — 3NF is sufficient for most applications
|
|
38
|
+
|
|
39
|
+
**Practical rule**: Normalize to 3NF, then denormalize specific tables based on measured query performance needs.
|
|
40
|
+
|
|
41
|
+
## Denormalization Strategies
|
|
42
|
+
|
|
43
|
+
| Strategy | When | Trade-off |
|
|
44
|
+
|----------|------|-----------|
|
|
45
|
+
| Duplicate columns | Frequently joined fields (e.g., user_name in orders) | Stale data risk, update anomalies |
|
|
46
|
+
| Pre-computed aggregates | Dashboard counters, report summaries | Must update on every write |
|
|
47
|
+
| Materialized views | Complex reports queried often | Refresh latency |
|
|
48
|
+
| JSON columns | Flexible metadata, rare querying needs | Harder to index and validate |
|
|
49
|
+
|
|
50
|
+
**Rule**: Only denormalize when you have evidence of a performance problem. Premature denormalization causes data inconsistency bugs.
|
|
51
|
+
|
|
52
|
+
## Index Design
|
|
53
|
+
|
|
54
|
+
### B-tree Index (Default)
|
|
55
|
+
- Good for: equality, range queries, sorting, LIKE 'prefix%'
|
|
56
|
+
- Bad for: LIKE '%suffix', full-text search, high-cardinality writes
|
|
57
|
+
|
|
58
|
+
### Composite Index
|
|
59
|
+
```sql
|
|
60
|
+
-- Follows leftmost prefix rule
|
|
61
|
+
CREATE INDEX idx_orders_user_status ON orders (user_id, status);
|
|
62
|
+
|
|
63
|
+
-- This index supports:
|
|
64
|
+
-- WHERE user_id = ? ✓
|
|
65
|
+
-- WHERE user_id = ? AND status = ? ✓
|
|
66
|
+
-- WHERE status = ? ✗ (needs separate index)
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
### Covering Index
|
|
70
|
+
Include all columns needed by the query to avoid table lookups:
|
|
71
|
+
```sql
|
|
72
|
+
CREATE INDEX idx_orders_covering ON orders (user_id, status) INCLUDE (total, created_at);
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
### Partial Index (PostgreSQL)
|
|
76
|
+
Index only rows matching a condition — smaller index, faster queries:
|
|
77
|
+
```sql
|
|
78
|
+
CREATE INDEX idx_active_tasks ON tasks (assignee_id, due_date) WHERE status != 'done';
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
### Index Guidelines
|
|
82
|
+
- Always index foreign keys
|
|
83
|
+
- Index columns used in WHERE, JOIN, ORDER BY
|
|
84
|
+
- Don't index low-cardinality columns alone (boolean, enum with 3 values)
|
|
85
|
+
- Monitor index usage: `pg_stat_user_indexes` (PostgreSQL)
|
|
86
|
+
- Remove unused indexes — each index slows down INSERT/UPDATE/DELETE
|
|
87
|
+
|
|
88
|
+
## SQL vs NoSQL Decision Matrix
|
|
89
|
+
|
|
90
|
+
| Factor | Choose SQL | Choose NoSQL |
|
|
91
|
+
|--------|-----------|--------------|
|
|
92
|
+
| Data structure | Well-defined, relational | Flexible, evolving schema |
|
|
93
|
+
| Consistency | Strong ACID required | Eventual consistency OK |
|
|
94
|
+
| Query patterns | Complex joins, aggregations | Simple key-value or document lookups |
|
|
95
|
+
| Scale | Vertical (single node OK) | Horizontal (distributed, high volume) |
|
|
96
|
+
| Transactions | Multi-table transactions | Single-document operations |
|
|
97
|
+
| Examples | PostgreSQL, MySQL | MongoDB, DynamoDB, Redis |
|
|
98
|
+
|
|
99
|
+
**Default choice**: PostgreSQL. It handles 95% of application needs. Only reach for NoSQL when you have a specific reason.
|
|
100
|
+
|
|
101
|
+
## Relational Patterns
|
|
102
|
+
|
|
103
|
+
### One-to-Many
|
|
104
|
+
```sql
|
|
105
|
+
CREATE TABLE projects (id UUID PRIMARY KEY, name TEXT NOT NULL);
|
|
106
|
+
CREATE TABLE tasks (id UUID PRIMARY KEY, project_id UUID REFERENCES projects(id), title TEXT NOT NULL);
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
### Many-to-Many
|
|
110
|
+
```sql
|
|
111
|
+
CREATE TABLE users (id UUID PRIMARY KEY, name TEXT NOT NULL);
|
|
112
|
+
CREATE TABLE roles (id UUID PRIMARY KEY, name TEXT NOT NULL);
|
|
113
|
+
CREATE TABLE user_roles (
|
|
114
|
+
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
|
|
115
|
+
role_id UUID REFERENCES roles(id) ON DELETE CASCADE,
|
|
116
|
+
granted_at TIMESTAMPTZ DEFAULT now(),
|
|
117
|
+
PRIMARY KEY (user_id, role_id)
|
|
118
|
+
);
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
### Polymorphic Associations
|
|
122
|
+
Avoid single foreign key pointing to multiple tables. Use one of:
|
|
123
|
+
- **Separate tables**: `task_comments`, `project_comments` (simplest, best for SQL)
|
|
124
|
+
- **Shared table with type column**: `comments` with `commentable_type` + `commentable_id` (no FK enforcement)
|
|
125
|
+
- **Join table per type**: `task_comments(task_id, comment_id)`, `project_comments(project_id, comment_id)`
|
|
126
|
+
|
|
127
|
+
## NoSQL Patterns
|
|
128
|
+
|
|
129
|
+
### Document Store (MongoDB, DynamoDB)
|
|
130
|
+
- Embed related data that is always accessed together
|
|
131
|
+
- Reference data that is shared or accessed independently
|
|
132
|
+
- Design for access patterns, not for normalization
|
|
133
|
+
|
|
134
|
+
### Key-Value (Redis)
|
|
135
|
+
- Session storage, caching, rate limiting, leaderboards
|
|
136
|
+
- Use key naming conventions: `app:entity:id:field` (e.g., `myapp:user:123:session`)
|
|
137
|
+
- Set TTL on every key to prevent memory leaks
|
|
138
|
+
|
|
139
|
+
## Data Types Best Practices
|
|
140
|
+
|
|
141
|
+
| Data | Type | NOT This |
|
|
142
|
+
|------|------|----------|
|
|
143
|
+
| Primary key | UUID / ULID | AUTO_INCREMENT (unless single-node) |
|
|
144
|
+
| Money | DECIMAL(19,4) or integer cents | FLOAT / DOUBLE |
|
|
145
|
+
| Timestamps | TIMESTAMPTZ (with timezone) | TIMESTAMP (without timezone) |
|
|
146
|
+
| Email/URL | TEXT with CHECK constraint | VARCHAR(255) |
|
|
147
|
+
| Boolean | BOOLEAN | TINYINT / CHAR(1) |
|
|
148
|
+
| Enum | TEXT with CHECK or enum type | Magic numbers |
|
|
149
|
+
| JSON metadata | JSONB (PostgreSQL) | TEXT (can't query or index) |
|
|
150
|
+
|
|
151
|
+
## Naming Conventions
|
|
152
|
+
|
|
153
|
+
- Tables: plural, snake_case (`order_items`, `user_roles`)
|
|
154
|
+
- Columns: singular, snake_case (`created_at`, `user_id`)
|
|
155
|
+
- Primary keys: `id`
|
|
156
|
+
- Foreign keys: `{referenced_table_singular}_id` (`user_id`, `project_id`)
|
|
157
|
+
- Indexes: `idx_{table}_{columns}` (`idx_orders_user_id_status`)
|
|
158
|
+
- Constraints: `chk_{table}_{column}` (`chk_orders_status`)
|
|
159
|
+
- Booleans: `is_` or `has_` prefix (`is_active`, `has_verified_email`)
|