skillstore-cli 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (231) hide show
  1. package/README.md +95 -0
  2. package/data/bundles/devflow-complete.json +19 -0
  3. package/data/free-skills/devflow-agile/manifest.json +19 -0
  4. package/data/free-skills/devflow-agile/plugin/commands/agile/retro.md +23 -0
  5. package/data/free-skills/devflow-agile/plugin/commands/agile/review.md +21 -0
  6. package/data/free-skills/devflow-agile/plugin/commands/agile/sprint.md +30 -0
  7. package/data/free-skills/devflow-agile/plugin/commands/agile/standup.md +20 -0
  8. package/data/free-skills/devflow-agile/plugin/commands/agile.md +35 -0
  9. package/data/free-skills/devflow-agile/plugin/commands/devflow.md +42 -0
  10. package/data/free-skills/devflow-agile/plugin/skills/developer/SKILL.md +93 -0
  11. package/data/free-skills/devflow-agile/plugin/skills/developer/assets/sample-output.md +182 -0
  12. package/data/free-skills/devflow-agile/plugin/skills/developer/references/clean-architecture.md +361 -0
  13. package/data/free-skills/devflow-agile/plugin/skills/developer/references/clean-code-guide.md +207 -0
  14. package/data/free-skills/devflow-agile/plugin/skills/developer/references/debugging-methodology.md +191 -0
  15. package/data/free-skills/devflow-agile/template/agents/agile-coach.md +76 -0
  16. package/data/free-skills/devflow-agile/template/workflows/agile-sprint-workflow.md +81 -0
  17. package/data/free-skills/devflow-bootstrap/manifest.json +8 -0
  18. package/data/free-skills/devflow-bootstrap/plugin/commands/bootstrap/auto.md +31 -0
  19. package/data/free-skills/devflow-bootstrap/plugin/commands/bootstrap.md +38 -0
  20. package/data/free-skills/devflow-bootstrap/plugin/commands/devflow.md +20 -0
  21. package/data/free-skills/devflow-bootstrap/plugin/skills/project-scaffold/SKILL.md +56 -0
  22. package/data/free-skills/devflow-bootstrap/plugin/skills/project-scaffold/assets/sample-output.md +216 -0
  23. package/data/free-skills/devflow-bootstrap/plugin/skills/project-scaffold/references/architecture-decisions.md +254 -0
  24. package/data/free-skills/devflow-bootstrap/plugin/skills/project-scaffold/references/stack-templates.md +400 -0
  25. package/data/free-skills/devflow-bootstrap/template/agents/bootstrap-specialist.md +56 -0
  26. package/data/free-skills/devflow-bootstrap/template/workflows/bootstrap-workflow.md +70 -0
  27. package/data/free-skills/devflow-docs/manifest.json +8 -0
  28. package/data/free-skills/devflow-docs/plugin/commands/devflow.md +20 -0
  29. package/data/free-skills/devflow-docs/plugin/commands/docs/generate.md +17 -0
  30. package/data/free-skills/devflow-docs/plugin/commands/docs/parse.md +19 -0
  31. package/data/free-skills/devflow-docs/plugin/commands/docs.md +26 -0
  32. package/data/free-skills/devflow-docs/plugin/skills/pdf-processor/SKILL.md +59 -0
  33. package/data/free-skills/devflow-docs/plugin/skills/pdf-processor/assets/sample-output.md +114 -0
  34. package/data/free-skills/devflow-docs/plugin/skills/pdf-processor/references/extraction-techniques.md +115 -0
  35. package/data/free-skills/devflow-docs/plugin/skills/pdf-processor/references/ocr-strategies.md +167 -0
  36. package/data/free-skills/devflow-docs/template/agents/docs-specialist.md +35 -0
  37. package/data/free-skills/devflow-docs/template/workflows/docs-workflow.md +70 -0
  38. package/data/free-skills/devflow-postproject/manifest.json +13 -0
  39. package/data/free-skills/devflow-postproject/plugin/commands/devflow.md +34 -0
  40. package/data/free-skills/devflow-postproject/plugin/commands/postproject/handover.md +21 -0
  41. package/data/free-skills/devflow-postproject/plugin/commands/postproject/retro.md +21 -0
  42. package/data/free-skills/devflow-postproject/plugin/commands/postproject/support.md +21 -0
  43. package/data/free-skills/devflow-postproject/plugin/commands/postproject.md +32 -0
  44. package/data/free-skills/devflow-postproject/plugin/skills/retrospective/SKILL.md +70 -0
  45. package/data/free-skills/devflow-postproject/plugin/skills/retrospective/assets/sample-output.md +79 -0
  46. package/data/free-skills/devflow-postproject/plugin/skills/retrospective/references/facilitation-techniques.md +178 -0
  47. package/data/free-skills/devflow-postproject/plugin/skills/retrospective/references/lessons-learned-template.md +118 -0
  48. package/data/free-skills/devflow-postproject/plugin/skills/retrospective/references/retro-techniques.md +100 -0
  49. package/data/free-skills/devflow-postproject/template/agents/transition-manager.md +71 -0
  50. package/data/free-skills/devflow-postproject/template/workflows/transition-workflow.md +72 -0
  51. package/data/free-skills/devflow-presale/manifest.json +15 -0
  52. package/data/free-skills/devflow-presale/plugin/commands/devflow.md +47 -0
  53. package/data/free-skills/devflow-presale/plugin/commands/presale/analyze.md +30 -0
  54. package/data/free-skills/devflow-presale/plugin/commands/presale/estimate.md +30 -0
  55. package/data/free-skills/devflow-presale/plugin/commands/presale/price.md +30 -0
  56. package/data/free-skills/devflow-presale/plugin/commands/presale/propose.md +30 -0
  57. package/data/free-skills/devflow-presale/plugin/commands/presale.md +42 -0
  58. package/data/free-skills/devflow-presale/plugin/skills/requirement-analysis/SKILL.md +63 -0
  59. package/data/free-skills/devflow-presale/plugin/skills/requirement-analysis/assets/sample-output.md +129 -0
  60. package/data/free-skills/devflow-presale/plugin/skills/requirement-analysis/references/extraction-framework.md +140 -0
  61. package/data/free-skills/devflow-presale/plugin/skills/requirement-analysis/references/output-template.md +132 -0
  62. package/data/free-skills/devflow-presale/template/agents/presale-lead.md +83 -0
  63. package/data/free-skills/devflow-presale/template/agents/proposal-reviewer.md +63 -0
  64. package/data/free-skills/devflow-presale/template/workflows/presale-workflow.md +70 -0
  65. package/data/registry/categories.json +7 -0
  66. package/data/registry/packages.json +184 -0
  67. package/data/shared/framework/agents/brainstormer.md +74 -0
  68. package/data/shared/framework/agents/code-reviewer.md +87 -0
  69. package/data/shared/framework/agents/debugger.md +84 -0
  70. package/data/shared/framework/agents/docs-manager.md +55 -0
  71. package/data/shared/framework/agents/git-manager.md +59 -0
  72. package/data/shared/framework/agents/planner.md +68 -0
  73. package/data/shared/framework/agents/researcher.md +66 -0
  74. package/data/shared/framework/agents/tester.md +65 -0
  75. package/data/shared/framework/commands/cook/auto.md +27 -0
  76. package/data/shared/framework/commands/cook.md +45 -0
  77. package/data/shared/framework/commands/fix/ci.md +21 -0
  78. package/data/shared/framework/commands/fix/test.md +26 -0
  79. package/data/shared/framework/commands/fix/types.md +29 -0
  80. package/data/shared/framework/commands/fix.md +26 -0
  81. package/data/shared/framework/commands/git/cm.md +37 -0
  82. package/data/shared/framework/commands/git/pr.md +40 -0
  83. package/data/shared/framework/config/CLAUDE.md.template +26 -0
  84. package/data/shared/framework/config/settings.json +41 -0
  85. package/data/shared/framework/config/skillstore.config.json +29 -0
  86. package/data/shared/framework/hooks/discord-notify.sh +85 -0
  87. package/data/shared/framework/hooks/docs-sync.sh +53 -0
  88. package/data/shared/framework/hooks/modularization-hook.js +103 -0
  89. package/data/shared/framework/hooks/notification.js +94 -0
  90. package/data/shared/framework/hooks/quality-gate.js +109 -0
  91. package/data/shared/framework/hooks/scout-block.js +77 -0
  92. package/data/shared/framework/hooks/telegram-notify.sh +77 -0
  93. package/data/shared/framework/protocols/error-recovery.md +80 -0
  94. package/data/shared/framework/protocols/orchestration-protocol.md +112 -0
  95. package/data/shared/framework/quality/review-protocol.md +76 -0
  96. package/data/shared/framework/quality/verification-protocol.md +66 -0
  97. package/data/shared/framework/rules/development-rules.md +75 -0
  98. package/data/shared/framework/skills/backend-development/SKILL.md +77 -0
  99. package/data/shared/framework/skills/backend-development/assets/sample-output.md +175 -0
  100. package/data/shared/framework/skills/backend-development/references/advanced-patterns.md +180 -0
  101. package/data/shared/framework/skills/backend-development/references/api-design-guide.md +160 -0
  102. package/data/shared/framework/skills/backend-development/references/architecture-patterns.md +183 -0
  103. package/data/shared/framework/skills/backend-development/references/observability-resilience.md +155 -0
  104. package/data/shared/framework/skills/backend-development/references/troubleshooting.md +199 -0
  105. package/data/shared/framework/skills/codebase-analysis/SKILL.md +72 -0
  106. package/data/shared/framework/skills/codebase-analysis/assets/sample-output.md +263 -0
  107. package/data/shared/framework/skills/codebase-analysis/references/analysis-techniques.md +241 -0
  108. package/data/shared/framework/skills/codebase-analysis/references/dependency-mapping.md +280 -0
  109. package/data/shared/framework/skills/codebase-analysis/references/tech-debt-assessment.md +208 -0
  110. package/data/shared/framework/skills/databases/SKILL.md +72 -0
  111. package/data/shared/framework/skills/databases/assets/sample-output.md +212 -0
  112. package/data/shared/framework/skills/databases/references/advanced-data-patterns.md +259 -0
  113. package/data/shared/framework/skills/databases/references/query-optimization.md +214 -0
  114. package/data/shared/framework/skills/databases/references/schema-design.md +159 -0
  115. package/data/shared/framework/skills/databases/references/troubleshooting.md +214 -0
  116. package/data/shared/framework/skills/debugging-investigation/SKILL.md +84 -0
  117. package/data/shared/framework/skills/debugging-investigation/assets/sample-output.md +314 -0
  118. package/data/shared/framework/skills/debugging-investigation/references/systematic-debugging.md +197 -0
  119. package/data/shared/framework/skills/debugging-investigation/references/tool-specific-guides.md +202 -0
  120. package/data/shared/framework/skills/debugging-investigation/references/troubleshooting-patterns.md +196 -0
  121. package/data/shared/framework/skills/frontend-development/SKILL.md +67 -0
  122. package/data/shared/framework/skills/frontend-development/assets/sample-output.md +110 -0
  123. package/data/shared/framework/skills/frontend-development/references/component-patterns.md +112 -0
  124. package/data/shared/framework/skills/frontend-development/references/performance-guide.md +169 -0
  125. package/data/shared/framework/skills/frontend-development/references/routing-forms-realtime.md +374 -0
  126. package/data/shared/framework/skills/frontend-development/references/ssr-rsc-patterns.md +284 -0
  127. package/data/shared/framework/skills/frontend-development/references/troubleshooting.md +154 -0
  128. package/data/shared/framework/skills/mobile-development/SKILL.md +67 -0
  129. package/data/shared/framework/skills/mobile-development/assets/sample-output.md +382 -0
  130. package/data/shared/framework/skills/mobile-development/references/mobile-patterns.md +681 -0
  131. package/data/shared/framework/skills/mobile-development/references/mobile-performance.md +524 -0
  132. package/data/shared/framework/skills/mobile-development/references/troubleshooting.md +158 -0
  133. package/data/shared/framework/skills/security-audit/SKILL.md +83 -0
  134. package/data/shared/framework/skills/security-audit/assets/sample-output.md +451 -0
  135. package/data/shared/framework/skills/security-audit/references/owasp-checklist.md +580 -0
  136. package/data/shared/framework/skills/security-audit/references/secure-coding-patterns.md +433 -0
  137. package/data/shared/framework/skills/security-audit/references/vulnerability-remediation.md +331 -0
  138. package/data/shared/framework/skills/ui-generation/SKILL.md +70 -0
  139. package/data/shared/framework/skills/ui-generation/assets/sample-output.md +139 -0
  140. package/data/shared/framework/skills/ui-generation/references/accessibility-responsive.md +127 -0
  141. package/data/shared/framework/skills/ui-generation/references/compound-components.md +252 -0
  142. package/data/shared/framework/skills/ui-generation/references/generation-patterns.md +110 -0
  143. package/data/shared/framework/skills/ui-generation/references/storybook-design-system.md +278 -0
  144. package/data/shared/framework/skills/ui-generation/references/troubleshooting.md +198 -0
  145. package/data/shared/framework/workflows/documentation-management.md +58 -0
  146. package/data/shared/framework/workflows/primary-workflow.md +88 -0
  147. package/dist/commands/activate.d.ts +3 -0
  148. package/dist/commands/activate.d.ts.map +1 -0
  149. package/dist/commands/activate.js +34 -0
  150. package/dist/commands/activate.js.map +1 -0
  151. package/dist/commands/bundle.d.ts +3 -0
  152. package/dist/commands/bundle.d.ts.map +1 -0
  153. package/dist/commands/bundle.js +64 -0
  154. package/dist/commands/bundle.js.map +1 -0
  155. package/dist/commands/install.d.ts +3 -0
  156. package/dist/commands/install.d.ts.map +1 -0
  157. package/dist/commands/install.js +99 -0
  158. package/dist/commands/install.js.map +1 -0
  159. package/dist/commands/list.d.ts +3 -0
  160. package/dist/commands/list.d.ts.map +1 -0
  161. package/dist/commands/list.js +37 -0
  162. package/dist/commands/list.js.map +1 -0
  163. package/dist/commands/search.d.ts +3 -0
  164. package/dist/commands/search.d.ts.map +1 -0
  165. package/dist/commands/search.js +30 -0
  166. package/dist/commands/search.js.map +1 -0
  167. package/dist/commands/status.d.ts +3 -0
  168. package/dist/commands/status.d.ts.map +1 -0
  169. package/dist/commands/status.js +35 -0
  170. package/dist/commands/status.js.map +1 -0
  171. package/dist/commands/update.d.ts +3 -0
  172. package/dist/commands/update.d.ts.map +1 -0
  173. package/dist/commands/update.js +68 -0
  174. package/dist/commands/update.js.map +1 -0
  175. package/dist/download/cache.d.ts +3 -0
  176. package/dist/download/cache.d.ts.map +1 -0
  177. package/dist/download/cache.js +18 -0
  178. package/dist/download/cache.js.map +1 -0
  179. package/dist/download/client.d.ts +2 -0
  180. package/dist/download/client.d.ts.map +1 -0
  181. package/dist/download/client.js +58 -0
  182. package/dist/download/client.js.map +1 -0
  183. package/dist/index.d.ts +3 -0
  184. package/dist/index.d.ts.map +1 -0
  185. package/dist/index.js +23 -0
  186. package/dist/index.js.map +1 -0
  187. package/dist/installer/file-copier.d.ts +6 -0
  188. package/dist/installer/file-copier.d.ts.map +1 -0
  189. package/dist/installer/file-copier.js +32 -0
  190. package/dist/installer/file-copier.js.map +1 -0
  191. package/dist/installer/plugin-installer.d.ts +12 -0
  192. package/dist/installer/plugin-installer.d.ts.map +1 -0
  193. package/dist/installer/plugin-installer.js +33 -0
  194. package/dist/installer/plugin-installer.js.map +1 -0
  195. package/dist/installer/template-installer.d.ts +12 -0
  196. package/dist/installer/template-installer.d.ts.map +1 -0
  197. package/dist/installer/template-installer.js +45 -0
  198. package/dist/installer/template-installer.js.map +1 -0
  199. package/dist/license/crypto.d.ts +16 -0
  200. package/dist/license/crypto.d.ts.map +1 -0
  201. package/dist/license/crypto.js +50 -0
  202. package/dist/license/crypto.js.map +1 -0
  203. package/dist/license/license-store.d.ts +19 -0
  204. package/dist/license/license-store.d.ts.map +1 -0
  205. package/dist/license/license-store.js +99 -0
  206. package/dist/license/license-store.js.map +1 -0
  207. package/dist/license/validator.d.ts +32 -0
  208. package/dist/license/validator.d.ts.map +1 -0
  209. package/dist/license/validator.js +81 -0
  210. package/dist/license/validator.js.map +1 -0
  211. package/dist/registry/loader.d.ts +30 -0
  212. package/dist/registry/loader.d.ts.map +1 -0
  213. package/dist/registry/loader.js +22 -0
  214. package/dist/registry/loader.js.map +1 -0
  215. package/dist/registry/search-engine.d.ts +9 -0
  216. package/dist/registry/search-engine.d.ts.map +1 -0
  217. package/dist/registry/search-engine.js +30 -0
  218. package/dist/registry/search-engine.js.map +1 -0
  219. package/dist/utils/config.d.ts +14 -0
  220. package/dist/utils/config.d.ts.map +1 -0
  221. package/dist/utils/config.js +28 -0
  222. package/dist/utils/config.js.map +1 -0
  223. package/dist/utils/logger.d.ts +9 -0
  224. package/dist/utils/logger.d.ts.map +1 -0
  225. package/dist/utils/logger.js +22 -0
  226. package/dist/utils/logger.js.map +1 -0
  227. package/dist/utils/paths.d.ts +20 -0
  228. package/dist/utils/paths.d.ts.map +1 -0
  229. package/dist/utils/paths.js +79 -0
  230. package/dist/utils/paths.js.map +1 -0
  231. package/package.json +54 -0
@@ -0,0 +1,259 @@
1
+ # Advanced Data Patterns
2
+
3
+ ## Sharding Strategies
4
+
5
+ ### Decision Matrix
6
+
7
+ | Strategy | Best For | Shard Key Example | Trade-off |
8
+ |---|---|---|---|
9
+ | Hash-based | Even distribution, no range queries needed | `hash(user_id) % N` | Cannot do efficient range scans |
10
+ | Range-based | Time-series, sequential access patterns | `created_date`, `region_id` | Risk of hot spots on recent ranges |
11
+ | Geographic | Data locality, compliance (GDPR, residency) | `country_code`, `region` | Cross-region queries are expensive |
12
+
13
+ ### Shard Key Selection Criteria
14
+
15
+ Choose a shard key that satisfies:
16
+ 1. **High cardinality** — enough distinct values to distribute evenly across shards
17
+ 2. **Even distribution** — no single value dominates (avoid skewed keys like `status`)
18
+ 3. **Query isolation** — most queries include the shard key, avoiding scatter-gather
19
+ 4. **Immutability** — the key should not change after insertion (resharding is expensive)
20
+
21
+ ```sql
22
+ -- Example: hash-based sharding in application layer (PostgreSQL + Citus)
23
+ SELECT create_distributed_table('orders', 'customer_id');
24
+
25
+ -- Verify shard distribution
26
+ SELECT nodename, count(*)
27
+ FROM citus_shards
28
+ GROUP BY nodename;
29
+ ```
30
+
31
+ ### Cross-Shard Query Handling
32
+
33
+ - **Scatter-gather**: query all shards, merge results — acceptable for analytics, not for OLTP hot paths
34
+ - **Global tables**: replicate small reference tables (countries, currencies) to every shard
35
+ - **Entity groups**: co-locate related tables on the same shard key (orders + order_items both sharded by customer_id)
36
+
37
+ ### Rebalancing
38
+
39
+ - **Citus**: `SELECT rebalance_table_shards('orders');` — moves shards to equalize storage
40
+ - **Vitess**: `Reshard` workflow splits or merges shards with zero downtime
41
+ - **Rule of thumb**: plan rebalancing when any shard exceeds 1.5× average size or 80% disk capacity
42
+
43
+ ---
44
+
45
+ ## Change Data Capture (CDC)
46
+
47
+ ### Debezium + PostgreSQL Setup
48
+
49
+ 1. **Enable logical replication** in `postgresql.conf`:
50
+ ```
51
+ wal_level = logical
52
+ max_replication_slots = 4
53
+ max_wal_senders = 4
54
+ ```
55
+
56
+ 2. **Create a publication**:
57
+ ```sql
58
+ CREATE PUBLICATION cdc_publication FOR TABLE orders, customers;
59
+ ```
60
+
61
+ 3. **Debezium connector configuration** (Kafka Connect JSON):
62
+ ```json
63
+ {
64
+ "name": "pg-cdc-connector",
65
+ "config": {
66
+ "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
67
+ "database.hostname": "db-primary",
68
+ "database.port": "5432",
69
+ "database.dbname": "app_db",
70
+ "database.user": "cdc_user",
71
+ "plugin.name": "pgoutput",
72
+ "publication.name": "cdc_publication",
73
+ "topic.prefix": "app",
74
+ "table.include.list": "public.orders,public.customers",
75
+ "snapshot.mode": "initial",
76
+ "tombstones.on.delete": true,
77
+ "transforms": "unwrap",
78
+ "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState"
79
+ }
80
+ }
81
+ ```
82
+
83
+ 4. **Event structure** — each change event contains:
84
+ - `before`: row state before the change (null for inserts)
85
+ - `after`: row state after the change (null for deletes)
86
+ - `source`: metadata (lsn, txId, timestamp, table, schema)
87
+ - `op`: operation type — `c` (create), `u` (update), `d` (delete), `r` (snapshot read)
88
+
89
+ ### CDC Use Cases
90
+ - Sync data to Elasticsearch or analytics warehouse
91
+ - Trigger downstream microservice workflows on data changes
92
+ - Build audit logs without application-level instrumentation
93
+ - Cache invalidation in Redis based on database writes
94
+
95
+ ---
96
+
97
+ ## Temporal Tables
98
+
99
+ ### PostgreSQL (temporal_tables Extension)
100
+
101
+ ```sql
102
+ -- Install the extension
103
+ CREATE EXTENSION IF NOT EXISTS temporal_tables;
104
+
105
+ -- Create the main table
106
+ CREATE TABLE products (
107
+ id SERIAL PRIMARY KEY,
108
+ name TEXT NOT NULL,
109
+ price NUMERIC(10,2) NOT NULL,
110
+ sys_period tstzrange NOT NULL DEFAULT tstzrange(current_timestamp, null)
111
+ );
112
+
113
+ -- Create the history table (same structure)
114
+ CREATE TABLE products_history (LIKE products);
115
+
116
+ -- Attach the trigger
117
+ CREATE TRIGGER products_versioning
118
+ BEFORE INSERT OR UPDATE OR DELETE ON products
119
+ FOR EACH ROW EXECUTE FUNCTION versioning(
120
+ 'sys_period', 'products_history', true
121
+ );
122
+
123
+ -- Query: what was the price on a specific date?
124
+ SELECT name, price
125
+ FROM products_history
126
+ WHERE id = 42
127
+ AND sys_period @> '2025-06-15T00:00:00Z'::timestamptz;
128
+ ```
129
+
130
+ ### SQL Server (System-Versioned)
131
+
132
+ ```sql
133
+ CREATE TABLE products (
134
+ id INT PRIMARY KEY,
135
+ name NVARCHAR(200) NOT NULL,
136
+ price DECIMAL(10,2) NOT NULL,
137
+ valid_from DATETIME2 GENERATED ALWAYS AS ROW START,
138
+ valid_to DATETIME2 GENERATED ALWAYS AS ROW END,
139
+ PERIOD FOR SYSTEM_TIME (valid_from, valid_to)
140
+ ) WITH (SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.products_history));
141
+
142
+ -- Point-in-time query
143
+ SELECT * FROM products
144
+ FOR SYSTEM_TIME AS OF '2025-06-15T00:00:00';
145
+ ```
146
+
147
+ ---
148
+
149
+ ## Full-Text Search: PostgreSQL vs Elasticsearch
150
+
151
+ ### Decision Guide
152
+
153
+ | Factor | PostgreSQL tsvector/tsquery | Elasticsearch |
154
+ |---|---|---|
155
+ | Data under 10M docs, simple search | Preferred — no extra infra | Overkill |
156
+ | Fuzzy matching, typo tolerance | Limited (trigram + similarity) | Excellent (built-in fuzziness) |
157
+ | Faceted search, aggregations | Weak | Excellent |
158
+ | Relevance tuning | Basic (weights A-D) | Fine-grained (BM25, boosting, function_score) |
159
+ | Operational cost | Zero — runs inside your DB | Separate cluster to manage |
160
+ | Real-time indexing | Immediate (same transaction) | Near real-time (~1s refresh) |
161
+
162
+ ### PostgreSQL Full-Text Setup
163
+
164
+ ```sql
165
+ -- Add a tsvector column with GIN index
166
+ ALTER TABLE articles ADD COLUMN search_vector tsvector
167
+ GENERATED ALWAYS AS (
168
+ setweight(to_tsvector('english', coalesce(title, '')), 'A') ||
169
+ setweight(to_tsvector('english', coalesce(body, '')), 'B')
170
+ ) STORED;
171
+
172
+ CREATE INDEX idx_articles_search ON articles USING GIN (search_vector);
173
+
174
+ -- Search with ranking
175
+ SELECT title, ts_rank(search_vector, query) AS rank
176
+ FROM articles, plainto_tsquery('english', 'database sharding') AS query
177
+ WHERE search_vector @@ query
178
+ ORDER BY rank DESC
179
+ LIMIT 20;
180
+ ```
181
+
182
+ ---
183
+
184
+ ## Database-per-Service (Microservices)
185
+
186
+ ### Pros
187
+ - Independent schema evolution — no cross-team migration coordination
188
+ - Technology freedom — use PostgreSQL for one service, MongoDB for another
189
+ - Fault isolation — one database failure does not cascade
190
+ - Independent scaling of storage per service
191
+
192
+ ### Cons
193
+ - **Cross-service queries** require API composition or CQRS read models
194
+ - **Distributed transactions** are hard — use Saga pattern instead of 2PC
195
+ - **Data duplication** is inevitable; accept eventual consistency
196
+ - **Operational overhead** — more databases to monitor, back up, patch
197
+
198
+ ### Data Consistency Patterns
199
+ - **Saga (choreography)**: services emit events; each service reacts and compensates on failure
200
+ - **Saga (orchestration)**: a central coordinator directs the transaction steps
201
+ - **Outbox pattern**: write events to an outbox table in the same transaction, then publish asynchronously
202
+ - **CQRS**: separate command and query models; sync via events for cross-service reads
203
+
204
+ ---
205
+
206
+ ## Multi-Tenant Data Isolation
207
+
208
+ ### Strategy Comparison
209
+
210
+ | Strategy | Isolation | Cost | Complexity | Best For |
211
+ |---|---|---|---|---|
212
+ | Separate databases | Strongest | Highest | Low code, high ops | Enterprise / regulated |
213
+ | Schema-per-tenant | Strong | Medium | Medium | Mid-market SaaS |
214
+ | Row-level security | Weakest | Lowest | Medium code, low ops | High-volume SMB SaaS |
215
+
216
+ ### Schema-per-Tenant (PostgreSQL)
217
+
218
+ ```sql
219
+ -- Create a schema for each tenant on signup
220
+ CREATE SCHEMA tenant_acme;
221
+ CREATE TABLE tenant_acme.orders (LIKE public.orders_template INCLUDING ALL);
222
+
223
+ -- Set search_path per connection
224
+ SET search_path TO tenant_acme, public;
225
+ ```
226
+
227
+ ### Row-Level Security (RLS)
228
+
229
+ ```sql
230
+ -- Enable RLS on the table
231
+ ALTER TABLE orders ENABLE ROW LEVEL SECURITY;
232
+
233
+ -- Policy: users see only their tenant's rows
234
+ CREATE POLICY tenant_isolation ON orders
235
+ USING (tenant_id = current_setting('app.current_tenant')::uuid);
236
+
237
+ -- Force RLS even for table owners
238
+ ALTER TABLE orders FORCE ROW LEVEL SECURITY;
239
+
240
+ -- Application sets tenant context per request
241
+ SET LOCAL app.current_tenant = 'a1b2c3d4-...';
242
+
243
+ -- Now all queries are automatically filtered
244
+ SELECT * FROM orders; -- returns only current tenant's orders
245
+ ```
246
+
247
+ ### Application-Layer Setup (Node.js Example)
248
+
249
+ ```typescript
250
+ // Middleware: set tenant context on every request
251
+ app.use(async (req, res, next) => {
252
+ const tenantId = req.headers['x-tenant-id'];
253
+ if (!tenantId) return res.status(400).json({ error: 'Tenant ID required' });
254
+
255
+ // Set RLS context for this database session
256
+ await db.query(`SET LOCAL app.current_tenant = $1`, [tenantId]);
257
+ next();
258
+ });
259
+ ```
@@ -0,0 +1,214 @@
1
+ # Query Optimization
2
+
3
+ ## EXPLAIN / ANALYZE Interpretation
4
+
5
+ ### PostgreSQL
6
+ ```sql
7
+ EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT) SELECT * FROM orders WHERE user_id = 'abc' AND status = 'open';
8
+ ```
9
+
10
+ ### Key Metrics to Check
11
+
12
+ | Metric | Good | Investigate |
13
+ |--------|------|-------------|
14
+ | Seq Scan on large table | < 1,000 rows | > 10,000 rows |
15
+ | Nested Loop | Inner table < 100 rows | Inner table > 1,000 rows |
16
+ | Sort | In-memory | On disk (work_mem too low) |
17
+ | Actual rows vs. estimated | Within 10x | Off by > 100x (stale stats) |
18
+ | Buffers shared hit ratio | > 95% | < 80% (increase shared_buffers) |
19
+
20
+ ### Common EXPLAIN Nodes
21
+ - **Seq Scan**: Full table scan. OK for small tables. Add index for large tables.
22
+ - **Index Scan**: Uses index to find rows, then fetches from table. Good.
23
+ - **Index Only Scan**: All data from index, no table fetch. Best case.
24
+ - **Bitmap Index Scan**: Combines multiple indexes or handles many matches. OK.
25
+ - **Hash Join / Merge Join**: Joining two sets. Hash is better for unequal sizes, Merge for pre-sorted data.
26
+
27
+ ### When Stats Are Stale
28
+ ```sql
29
+ -- Force PostgreSQL to update statistics
30
+ ANALYZE orders;
31
+
32
+ -- Check when stats were last updated
33
+ SELECT relname, last_analyze, last_autoanalyze FROM pg_stat_user_tables WHERE relname = 'orders';
34
+ ```
35
+
36
+ ## Common Query Anti-Patterns
37
+
38
+ ### 1. SELECT * (Fetching All Columns)
39
+ ```sql
40
+ -- Bad: fetches all columns including large TEXT/BLOB fields
41
+ SELECT * FROM users WHERE status = 'active';
42
+
43
+ -- Good: fetch only what you need
44
+ SELECT id, name, email FROM users WHERE status = 'active';
45
+ ```
46
+
47
+ ### 2. N+1 Queries
48
+ ```python
49
+ # Bad: 1 query for orders + N queries for users (one per order)
50
+ orders = db.query("SELECT * FROM orders LIMIT 100")
51
+ for order in orders:
52
+ user = db.query("SELECT * FROM users WHERE id = %s", order.user_id)
53
+
54
+ # Good: 1 query with JOIN or 2 queries with IN
55
+ orders = db.query("""
56
+ SELECT o.*, u.name as user_name
57
+ FROM orders o JOIN users u ON o.user_id = u.id
58
+ LIMIT 100
59
+ """)
60
+
61
+ # Also good: batch fetch
62
+ user_ids = [o.user_id for o in orders]
63
+ users = db.query("SELECT * FROM users WHERE id = ANY(%s)", user_ids)
64
+ ```
65
+
66
+ ### 3. Missing Indexes on Foreign Keys
67
+ Every foreign key column should have an index. Without it, JOINs and cascading deletes trigger full table scans.
68
+
69
+ ### 4. OR vs. UNION
70
+ ```sql
71
+ -- Bad: OR often prevents index usage
72
+ SELECT * FROM tasks WHERE assignee_id = 'a' OR creator_id = 'a';
73
+
74
+ -- Better: UNION uses both indexes
75
+ SELECT * FROM tasks WHERE assignee_id = 'a'
76
+ UNION
77
+ SELECT * FROM tasks WHERE creator_id = 'a';
78
+ ```
79
+
80
+ ### 5. Functions on Indexed Columns
81
+ ```sql
82
+ -- Bad: index on created_at is useless because of DATE()
83
+ SELECT * FROM orders WHERE DATE(created_at) = '2026-03-20';
84
+
85
+ -- Good: use range query
86
+ SELECT * FROM orders WHERE created_at >= '2026-03-20' AND created_at < '2026-03-21';
87
+ ```
88
+
89
+ ### 6. OFFSET for Deep Pagination
90
+ ```sql
91
+ -- Bad: database reads and discards 10,000 rows
92
+ SELECT * FROM products ORDER BY id LIMIT 25 OFFSET 10000;
93
+
94
+ -- Good: cursor-based pagination (keyset)
95
+ SELECT * FROM products WHERE id > 'last-seen-id' ORDER BY id LIMIT 25;
96
+ ```
97
+
98
+ ## Index Optimization Strategies
99
+
100
+ ### Composite Index Column Order
101
+ Put the most selective column first (highest cardinality):
102
+ ```sql
103
+ -- If user_id has 100,000 unique values and status has 5:
104
+ CREATE INDEX idx_tasks_user_status ON tasks (user_id, status); -- Good
105
+ CREATE INDEX idx_tasks_status_user ON tasks (status, user_id); -- Less effective
106
+ ```
107
+
108
+ ### Index for Sorting
109
+ ```sql
110
+ -- Query: WHERE status = 'open' ORDER BY created_at DESC
111
+ -- Index must match both filter and sort:
112
+ CREATE INDEX idx_tasks_status_created ON tasks (status, created_at DESC);
113
+ ```
114
+
115
+ ### Partial Indexes for Hot Data
116
+ ```sql
117
+ -- 90% of queries are for active records
118
+ CREATE INDEX idx_active_users ON users (email) WHERE is_active = true;
119
+ -- Much smaller index, faster lookups
120
+ ```
121
+
122
+ ### Monitor Unused Indexes
123
+ ```sql
124
+ -- PostgreSQL: find indexes with zero scans
125
+ SELECT indexrelname, idx_scan, pg_size_pretty(pg_relation_size(indexrelid))
126
+ FROM pg_stat_user_indexes
127
+ WHERE idx_scan = 0 AND indexrelname NOT LIKE '%pkey'
128
+ ORDER BY pg_relation_size(indexrelid) DESC;
129
+ ```
130
+
131
+ ## Connection Pooling
132
+
133
+ ### Why
134
+ Database connections are expensive (TCP handshake, auth, memory allocation). A pool reuses connections.
135
+
136
+ ### Configuration Guidelines
137
+
138
+ | App Type | Pool Size | Formula |
139
+ |----------|-----------|---------|
140
+ | Web server | 10-20 per instance | `connections = (cores * 2) + disk_spindles` |
141
+ | Background workers | 3-5 per worker | Fewer concurrent queries needed |
142
+ | Total max | Check DB limit | PostgreSQL default: 100 connections |
143
+
144
+ ### Tools
145
+ - **Application-level**: HikariCP (Java), pgBouncer (PostgreSQL), Prisma connection pool
146
+ - **Standalone proxy**: PgBouncer, ProxySQL (MySQL)
147
+
148
+ ### PgBouncer Modes
149
+ | Mode | When |
150
+ |------|------|
151
+ | Transaction | Default choice. Releases connection after each transaction. |
152
+ | Session | App uses prepared statements or temp tables. |
153
+ | Statement | High-throughput, no transactions needed. |
154
+
155
+ ## Read Replicas and Partitioning
156
+
157
+ ### Read Replicas
158
+ - Route read queries (reports, search, dashboards) to replicas
159
+ - Keep writes on primary
160
+ - Handle replication lag (data may be 0-5 seconds behind)
161
+ - Use for: analytics queries, search indexes, backup
162
+
163
+ ### Table Partitioning
164
+ ```sql
165
+ -- Partition by range (time-series data)
166
+ CREATE TABLE events (id UUID, created_at TIMESTAMPTZ, payload JSONB)
167
+ PARTITION BY RANGE (created_at);
168
+
169
+ CREATE TABLE events_2026_q1 PARTITION OF events
170
+ FOR VALUES FROM ('2026-01-01') TO ('2026-04-01');
171
+ ```
172
+
173
+ **When to partition**: table exceeds 50-100 million rows, or queries always filter by partition key.
174
+
175
+ ## Migration Strategies
176
+
177
+ ### Zero-Downtime Migrations (Expand-Contract)
178
+
179
+ **Adding a column:**
180
+ 1. Add column as nullable (no lock)
181
+ 2. Deploy code that writes to both old and new columns
182
+ 3. Backfill existing rows in batches
183
+ 4. Deploy code that reads from new column
184
+ 5. Add NOT NULL constraint (if needed)
185
+ 6. Remove old column usage from code
186
+
187
+ **Renaming a column:**
188
+ 1. Add new column
189
+ 2. Write to both columns
190
+ 3. Backfill
191
+ 4. Read from new column
192
+ 5. Drop old column
193
+
194
+ ### Migration Best Practices
195
+ - Every migration must have a rollback (down migration)
196
+ - Test migrations on production-size data locally
197
+ - Run migrations in a transaction when possible
198
+ - Set a statement timeout to prevent long locks: `SET lock_timeout = '5s'`
199
+ - Never run `ALTER TABLE ... ADD COLUMN ... DEFAULT` on large tables in older PostgreSQL (< 11) — it rewrites the entire table
200
+
201
+ ## Backup and Recovery
202
+
203
+ ### Backup Types
204
+ | Type | Frequency | Use Case |
205
+ |------|-----------|----------|
206
+ | Full backup (pg_dump) | Daily | Point-in-time recovery base |
207
+ | WAL archiving | Continuous | Point-in-time recovery to any second |
208
+ | Logical replication | Continuous | Cross-version or selective backup |
209
+
210
+ ### Testing Recovery
211
+ - Schedule monthly recovery drills
212
+ - Verify backup integrity: restore to a test instance and run validation queries
213
+ - Measure Recovery Time Objective (RTO) — how long to restore
214
+ - Measure Recovery Point Objective (RPO) — maximum data loss acceptable
@@ -0,0 +1,159 @@
1
+ # Schema Design
2
+
3
+ ## Normalization
4
+
5
+ ### 1NF (First Normal Form)
6
+ - Every column contains atomic (indivisible) values
7
+ - No repeating groups or arrays in a single column
8
+
9
+ ```sql
10
+ -- Bad: tags stored as comma-separated string
11
+ CREATE TABLE products (id INT, name TEXT, tags TEXT); -- tags = "electronics,sale,new"
12
+
13
+ -- Good: separate table for tags
14
+ CREATE TABLE products (id INT PRIMARY KEY, name TEXT);
15
+ CREATE TABLE product_tags (product_id INT REFERENCES products(id), tag TEXT);
16
+ ```
17
+
18
+ ### 2NF (Second Normal Form)
19
+ - Must be in 1NF
20
+ - Every non-key column depends on the ENTIRE primary key (no partial dependencies)
21
+
22
+ ### 3NF (Third Normal Form)
23
+ - Must be in 2NF
24
+ - No transitive dependencies (non-key column depending on another non-key column)
25
+
26
+ ```sql
27
+ -- Bad: city_name depends on zip_code, not on order_id
28
+ CREATE TABLE orders (id INT, zip_code TEXT, city_name TEXT);
29
+
30
+ -- Good: separate addresses table
31
+ CREATE TABLE orders (id INT, address_id INT REFERENCES addresses(id));
32
+ CREATE TABLE addresses (id INT PRIMARY KEY, zip_code TEXT, city_name TEXT);
33
+ ```
34
+
35
+ ### BCNF (Boyce-Codd Normal Form)
36
+ - Every determinant is a candidate key
37
+ - Rarely needed in practice — 3NF is sufficient for most applications
38
+
39
+ **Practical rule**: Normalize to 3NF, then denormalize specific tables based on measured query performance needs.
40
+
41
+ ## Denormalization Strategies
42
+
43
+ | Strategy | When | Trade-off |
44
+ |----------|------|-----------|
45
+ | Duplicate columns | Frequently joined fields (e.g., user_name in orders) | Stale data risk, update anomalies |
46
+ | Pre-computed aggregates | Dashboard counters, report summaries | Must update on every write |
47
+ | Materialized views | Complex reports queried often | Refresh latency |
48
+ | JSON columns | Flexible metadata, rare querying needs | Harder to index and validate |
49
+
50
+ **Rule**: Only denormalize when you have evidence of a performance problem. Premature denormalization causes data inconsistency bugs.
51
+
52
+ ## Index Design
53
+
54
+ ### B-tree Index (Default)
55
+ - Good for: equality, range queries, sorting, LIKE 'prefix%'
56
+ - Bad for: LIKE '%suffix', full-text search, high-cardinality writes
57
+
58
+ ### Composite Index
59
+ ```sql
60
+ -- Follows leftmost prefix rule
61
+ CREATE INDEX idx_orders_user_status ON orders (user_id, status);
62
+
63
+ -- This index supports:
64
+ -- WHERE user_id = ? ✓
65
+ -- WHERE user_id = ? AND status = ? ✓
66
+ -- WHERE status = ? ✗ (needs separate index)
67
+ ```
68
+
69
+ ### Covering Index
70
+ Include all columns needed by the query to avoid table lookups:
71
+ ```sql
72
+ CREATE INDEX idx_orders_covering ON orders (user_id, status) INCLUDE (total, created_at);
73
+ ```
74
+
75
+ ### Partial Index (PostgreSQL)
76
+ Index only rows matching a condition — smaller index, faster queries:
77
+ ```sql
78
+ CREATE INDEX idx_active_tasks ON tasks (assignee_id, due_date) WHERE status != 'done';
79
+ ```
80
+
81
+ ### Index Guidelines
82
+ - Always index foreign keys
83
+ - Index columns used in WHERE, JOIN, ORDER BY
84
+ - Don't index low-cardinality columns alone (boolean, enum with 3 values)
85
+ - Monitor index usage: `pg_stat_user_indexes` (PostgreSQL)
86
+ - Remove unused indexes — each index slows down INSERT/UPDATE/DELETE
87
+
88
+ ## SQL vs NoSQL Decision Matrix
89
+
90
+ | Factor | Choose SQL | Choose NoSQL |
91
+ |--------|-----------|--------------|
92
+ | Data structure | Well-defined, relational | Flexible, evolving schema |
93
+ | Consistency | Strong ACID required | Eventual consistency OK |
94
+ | Query patterns | Complex joins, aggregations | Simple key-value or document lookups |
95
+ | Scale | Vertical (single node OK) | Horizontal (distributed, high volume) |
96
+ | Transactions | Multi-table transactions | Single-document operations |
97
+ | Examples | PostgreSQL, MySQL | MongoDB, DynamoDB, Redis |
98
+
99
+ **Default choice**: PostgreSQL. It handles 95% of application needs. Only reach for NoSQL when you have a specific reason.
100
+
101
+ ## Relational Patterns
102
+
103
+ ### One-to-Many
104
+ ```sql
105
+ CREATE TABLE projects (id UUID PRIMARY KEY, name TEXT NOT NULL);
106
+ CREATE TABLE tasks (id UUID PRIMARY KEY, project_id UUID REFERENCES projects(id), title TEXT NOT NULL);
107
+ ```
108
+
109
+ ### Many-to-Many
110
+ ```sql
111
+ CREATE TABLE users (id UUID PRIMARY KEY, name TEXT NOT NULL);
112
+ CREATE TABLE roles (id UUID PRIMARY KEY, name TEXT NOT NULL);
113
+ CREATE TABLE user_roles (
114
+ user_id UUID REFERENCES users(id) ON DELETE CASCADE,
115
+ role_id UUID REFERENCES roles(id) ON DELETE CASCADE,
116
+ granted_at TIMESTAMPTZ DEFAULT now(),
117
+ PRIMARY KEY (user_id, role_id)
118
+ );
119
+ ```
120
+
121
+ ### Polymorphic Associations
122
+ Avoid single foreign key pointing to multiple tables. Use one of:
123
+ - **Separate tables**: `task_comments`, `project_comments` (simplest, best for SQL)
124
+ - **Shared table with type column**: `comments` with `commentable_type` + `commentable_id` (no FK enforcement)
125
+ - **Join table per type**: `task_comments(task_id, comment_id)`, `project_comments(project_id, comment_id)`
126
+
127
+ ## NoSQL Patterns
128
+
129
+ ### Document Store (MongoDB, DynamoDB)
130
+ - Embed related data that is always accessed together
131
+ - Reference data that is shared or accessed independently
132
+ - Design for access patterns, not for normalization
133
+
134
+ ### Key-Value (Redis)
135
+ - Session storage, caching, rate limiting, leaderboards
136
+ - Use key naming conventions: `app:entity:id:field` (e.g., `myapp:user:123:session`)
137
+ - Set TTL on every key to prevent memory leaks
138
+
139
+ ## Data Types Best Practices
140
+
141
+ | Data | Type | NOT This |
142
+ |------|------|----------|
143
+ | Primary key | UUID / ULID | AUTO_INCREMENT (unless single-node) |
144
+ | Money | DECIMAL(19,4) or integer cents | FLOAT / DOUBLE |
145
+ | Timestamps | TIMESTAMPTZ (with timezone) | TIMESTAMP (without timezone) |
146
+ | Email/URL | TEXT with CHECK constraint | VARCHAR(255) |
147
+ | Boolean | BOOLEAN | TINYINT / CHAR(1) |
148
+ | Enum | TEXT with CHECK or enum type | Magic numbers |
149
+ | JSON metadata | JSONB (PostgreSQL) | TEXT (can't query or index) |
150
+
151
+ ## Naming Conventions
152
+
153
+ - Tables: plural, snake_case (`order_items`, `user_roles`)
154
+ - Columns: singular, snake_case (`created_at`, `user_id`)
155
+ - Primary keys: `id`
156
+ - Foreign keys: `{referenced_table_singular}_id` (`user_id`, `project_id`)
157
+ - Indexes: `idx_{table}_{columns}` (`idx_orders_user_id_status`)
158
+ - Constraints: `chk_{table}_{column}` (`chk_orders_status`)
159
+ - Booleans: `is_` or `has_` prefix (`is_active`, `has_verified_email`)