0xkobold 0.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (258) hide show
  1. package/.agents/skills/nextjs-best-practices/SKILL.md +208 -0
  2. package/.agents/skills/sql-optimization-patterns/SKILL.md +509 -0
  3. package/HEARTBEAT.md +45 -0
  4. package/README.md +197 -0
  5. package/USAGE.md +191 -0
  6. package/dist/package.json +77 -0
  7. package/dist/src/agent/pi-adapter.js +307 -0
  8. package/dist/src/agent/pi-adapter.js.map +1 -0
  9. package/dist/src/agent/tool-adapter.js +86 -0
  10. package/dist/src/agent/tool-adapter.js.map +1 -0
  11. package/dist/src/approval/queue.js +114 -0
  12. package/dist/src/approval/queue.js.map +1 -0
  13. package/dist/src/ascii-kobold.js +76 -0
  14. package/dist/src/ascii-kobold.js.map +1 -0
  15. package/dist/src/cli/client.js +217 -0
  16. package/dist/src/cli/client.js.map +1 -0
  17. package/dist/src/cli/commands/agent.js +272 -0
  18. package/dist/src/cli/commands/agent.js.map +1 -0
  19. package/dist/src/cli/commands/chat.js +234 -0
  20. package/dist/src/cli/commands/chat.js.map +1 -0
  21. package/dist/src/cli/commands/config.js +202 -0
  22. package/dist/src/cli/commands/config.js.map +1 -0
  23. package/dist/src/cli/commands/daemon.js +203 -0
  24. package/dist/src/cli/commands/daemon.js.map +1 -0
  25. package/dist/src/cli/commands/gateway.js +184 -0
  26. package/dist/src/cli/commands/gateway.js.map +1 -0
  27. package/dist/src/cli/commands/init.js +175 -0
  28. package/dist/src/cli/commands/init.js.map +1 -0
  29. package/dist/src/cli/commands/kobold.js +21 -0
  30. package/dist/src/cli/commands/kobold.js.map +1 -0
  31. package/dist/src/cli/commands/logs.js +27 -0
  32. package/dist/src/cli/commands/logs.js.map +1 -0
  33. package/dist/src/cli/commands/mode.js +121 -0
  34. package/dist/src/cli/commands/mode.js.map +1 -0
  35. package/dist/src/cli/commands/persona.js +261 -0
  36. package/dist/src/cli/commands/persona.js.map +1 -0
  37. package/dist/src/cli/commands/start.js +66 -0
  38. package/dist/src/cli/commands/start.js.map +1 -0
  39. package/dist/src/cli/commands/status.js +117 -0
  40. package/dist/src/cli/commands/status.js.map +1 -0
  41. package/dist/src/cli/commands/stop.js +27 -0
  42. package/dist/src/cli/commands/stop.js.map +1 -0
  43. package/dist/src/cli/commands/system.js +128 -0
  44. package/dist/src/cli/commands/system.js.map +1 -0
  45. package/dist/src/cli/commands/tui.js +103 -0
  46. package/dist/src/cli/commands/tui.js.map +1 -0
  47. package/dist/src/cli/commands/update.js +133 -0
  48. package/dist/src/cli/commands/update.js.map +1 -0
  49. package/dist/src/cli/extensions/discord.js +113 -0
  50. package/dist/src/cli/extensions/discord.js.map +1 -0
  51. package/dist/src/cli/extensions/env.js +91 -0
  52. package/dist/src/cli/extensions/env.js.map +1 -0
  53. package/dist/src/cli/extensions/heartbeat.js +78 -0
  54. package/dist/src/cli/extensions/heartbeat.js.map +1 -0
  55. package/dist/src/cli/index.js +24 -0
  56. package/dist/src/cli/index.js.map +1 -0
  57. package/dist/src/cli/program.js +70 -0
  58. package/dist/src/cli/program.js.map +1 -0
  59. package/dist/src/cli/repl.js +184 -0
  60. package/dist/src/cli/repl.js.map +1 -0
  61. package/dist/src/cli/shared/discord-service.js +102 -0
  62. package/dist/src/cli/shared/discord-service.js.map +1 -0
  63. package/dist/src/config/index.js +10 -0
  64. package/dist/src/config/index.js.map +1 -0
  65. package/dist/src/config/loader.js +401 -0
  66. package/dist/src/config/loader.js.map +1 -0
  67. package/dist/src/config/paths.js +84 -0
  68. package/dist/src/config/paths.js.map +1 -0
  69. package/dist/src/config/types.js +8 -0
  70. package/dist/src/config/types.js.map +1 -0
  71. package/dist/src/context-detector.js +60 -0
  72. package/dist/src/context-detector.js.map +1 -0
  73. package/dist/src/discord/index.js +376 -0
  74. package/dist/src/discord/index.js.map +1 -0
  75. package/dist/src/event-bus/index.js +97 -0
  76. package/dist/src/event-bus/index.js.map +1 -0
  77. package/dist/src/extensions/command-args.js +68 -0
  78. package/dist/src/extensions/command-args.js.map +1 -0
  79. package/dist/src/extensions/core/agent-registry-extension.js +541 -0
  80. package/dist/src/extensions/core/agent-registry-extension.js.map +1 -0
  81. package/dist/src/extensions/core/agent-worker.js +148 -0
  82. package/dist/src/extensions/core/agent-worker.js.map +1 -0
  83. package/dist/src/extensions/core/compaction-safeguard.js +154 -0
  84. package/dist/src/extensions/core/compaction-safeguard.js.map +1 -0
  85. package/dist/src/extensions/core/confirm-destructive.js +43 -0
  86. package/dist/src/extensions/core/confirm-destructive.js.map +1 -0
  87. package/dist/src/extensions/core/context-aware-extension.js +124 -0
  88. package/dist/src/extensions/core/context-aware-extension.js.map +1 -0
  89. package/dist/src/extensions/core/context-pruning/extension.js +124 -0
  90. package/dist/src/extensions/core/context-pruning/extension.js.map +1 -0
  91. package/dist/src/extensions/core/context-pruning/pruner.js +312 -0
  92. package/dist/src/extensions/core/context-pruning/pruner.js.map +1 -0
  93. package/dist/src/extensions/core/context-pruning/runtime.js +48 -0
  94. package/dist/src/extensions/core/context-pruning/runtime.js.map +1 -0
  95. package/dist/src/extensions/core/context-pruning/settings.js +105 -0
  96. package/dist/src/extensions/core/context-pruning/settings.js.map +1 -0
  97. package/dist/src/extensions/core/dirty-repo-guard.js +42 -0
  98. package/dist/src/extensions/core/dirty-repo-guard.js.map +1 -0
  99. package/dist/src/extensions/core/discord-channel-extension.js +205 -0
  100. package/dist/src/extensions/core/discord-channel-extension.js.map +1 -0
  101. package/dist/src/extensions/core/discord-extension.js +142 -0
  102. package/dist/src/extensions/core/discord-extension.js.map +1 -0
  103. package/dist/src/extensions/core/env-loader-extension.js +157 -0
  104. package/dist/src/extensions/core/env-loader-extension.js.map +1 -0
  105. package/dist/src/extensions/core/fileops-extension.js +699 -0
  106. package/dist/src/extensions/core/fileops-extension.js.map +1 -0
  107. package/dist/src/extensions/core/gateway-extension.js +730 -0
  108. package/dist/src/extensions/core/gateway-extension.js.map +1 -0
  109. package/dist/src/extensions/core/git-checkpoint.js +46 -0
  110. package/dist/src/extensions/core/git-checkpoint.js.map +1 -0
  111. package/dist/src/extensions/core/handoff-extension.js +206 -0
  112. package/dist/src/extensions/core/handoff-extension.js.map +1 -0
  113. package/dist/src/extensions/core/heartbeat-extension.js +373 -0
  114. package/dist/src/extensions/core/heartbeat-extension.js.map +1 -0
  115. package/dist/src/extensions/core/mcp-extension.js +413 -0
  116. package/dist/src/extensions/core/mcp-extension.js.map +1 -0
  117. package/dist/src/extensions/core/mode-manager-extension.js +562 -0
  118. package/dist/src/extensions/core/mode-manager-extension.js.map +1 -0
  119. package/dist/src/extensions/core/multi-channel-extension.js +435 -0
  120. package/dist/src/extensions/core/multi-channel-extension.js.map +1 -0
  121. package/dist/src/extensions/core/ollama-provider-extension.js +66 -0
  122. package/dist/src/extensions/core/ollama-provider-extension.js.map +1 -0
  123. package/dist/src/extensions/core/onboarding-extension.js +122 -0
  124. package/dist/src/extensions/core/onboarding-extension.js.map +1 -0
  125. package/dist/src/extensions/core/persona-loader-extension.js +139 -0
  126. package/dist/src/extensions/core/persona-loader-extension.js.map +1 -0
  127. package/dist/src/extensions/core/pi-notify-extension.js +70 -0
  128. package/dist/src/extensions/core/pi-notify-extension.js.map +1 -0
  129. package/dist/src/extensions/core/protected-paths.js +24 -0
  130. package/dist/src/extensions/core/protected-paths.js.map +1 -0
  131. package/dist/src/extensions/core/questionnaire-extension.js +242 -0
  132. package/dist/src/extensions/core/questionnaire-extension.js.map +1 -0
  133. package/dist/src/extensions/core/self-update-extension.js +181 -0
  134. package/dist/src/extensions/core/self-update-extension.js.map +1 -0
  135. package/dist/src/extensions/core/session-bridge-extension.js +78 -0
  136. package/dist/src/extensions/core/session-bridge-extension.js.map +1 -0
  137. package/dist/src/extensions/core/session-manager-extension.js +319 -0
  138. package/dist/src/extensions/core/session-manager-extension.js.map +1 -0
  139. package/dist/src/extensions/core/session-name-extension.js +88 -0
  140. package/dist/src/extensions/core/session-name-extension.js.map +1 -0
  141. package/dist/src/extensions/core/session-pruning-extension.js +480 -0
  142. package/dist/src/extensions/core/session-pruning-extension.js.map +1 -0
  143. package/dist/src/extensions/core/task-manager-extension.js +661 -0
  144. package/dist/src/extensions/core/task-manager-extension.js.map +1 -0
  145. package/dist/src/extensions/core/update-extension.js +438 -0
  146. package/dist/src/extensions/core/update-extension.js.map +1 -0
  147. package/dist/src/extensions/core/websearch-extension.js +463 -0
  148. package/dist/src/extensions/core/websearch-extension.js.map +1 -0
  149. package/dist/src/extensions/index.js +5 -0
  150. package/dist/src/extensions/index.js.map +1 -0
  151. package/dist/src/extensions/loader.js +80 -0
  152. package/dist/src/extensions/loader.js.map +1 -0
  153. package/dist/src/gateway/index.js +353 -0
  154. package/dist/src/gateway/index.js.map +1 -0
  155. package/dist/src/index.js +150 -0
  156. package/dist/src/index.js.map +1 -0
  157. package/dist/src/llm/anthropic.js +86 -0
  158. package/dist/src/llm/anthropic.js.map +1 -0
  159. package/dist/src/llm/index.js +9 -0
  160. package/dist/src/llm/index.js.map +1 -0
  161. package/dist/src/llm/ollama.js +113 -0
  162. package/dist/src/llm/ollama.js.map +1 -0
  163. package/dist/src/llm/router.js +145 -0
  164. package/dist/src/llm/router.js.map +1 -0
  165. package/dist/src/llm/types.js +7 -0
  166. package/dist/src/llm/types.js.map +1 -0
  167. package/dist/src/memory/index.js +5 -0
  168. package/dist/src/memory/index.js.map +1 -0
  169. package/dist/src/memory/store.js +91 -0
  170. package/dist/src/memory/store.js.map +1 -0
  171. package/dist/src/pi-config.js +80 -0
  172. package/dist/src/pi-config.js.map +1 -0
  173. package/dist/src/skills/builtin/file.js +184 -0
  174. package/dist/src/skills/builtin/file.js.map +1 -0
  175. package/dist/src/skills/builtin/shell.js +100 -0
  176. package/dist/src/skills/builtin/shell.js.map +1 -0
  177. package/dist/src/skills/builtin/subagent.js +62 -0
  178. package/dist/src/skills/builtin/subagent.js.map +1 -0
  179. package/dist/src/skills/hello.js +42 -0
  180. package/dist/src/skills/hello.js.map +1 -0
  181. package/dist/src/skills/index.js +11 -0
  182. package/dist/src/skills/index.js.map +1 -0
  183. package/dist/src/skills/loader.js +382 -0
  184. package/dist/src/skills/loader.js.map +1 -0
  185. package/dist/src/skills/types.js +8 -0
  186. package/dist/src/skills/types.js.map +1 -0
  187. package/dist/src/utils/working-dir.js +71 -0
  188. package/dist/src/utils/working-dir.js.map +1 -0
  189. package/package.json +77 -0
  190. package/skills/1password/SKILL.md +70 -0
  191. package/skills/1password/references/cli-examples.md +29 -0
  192. package/skills/1password/references/get-started.md +17 -0
  193. package/skills/apple-notes/SKILL.md +77 -0
  194. package/skills/apple-reminders/SKILL.md +118 -0
  195. package/skills/bear-notes/SKILL.md +107 -0
  196. package/skills/blogwatcher/SKILL.md +69 -0
  197. package/skills/blucli/SKILL.md +47 -0
  198. package/skills/bluebubbles/SKILL.md +131 -0
  199. package/skills/camsnap/SKILL.md +45 -0
  200. package/skills/canvas/SKILL.md +198 -0
  201. package/skills/clawhub/SKILL.md +77 -0
  202. package/skills/coding-agent/SKILL.md +284 -0
  203. package/skills/discord/SKILL.md +197 -0
  204. package/skills/eightctl/SKILL.md +50 -0
  205. package/skills/food-order/SKILL.md +48 -0
  206. package/skills/gemini/SKILL.md +43 -0
  207. package/skills/gh-issues/SKILL.md +865 -0
  208. package/skills/gifgrep/SKILL.md +79 -0
  209. package/skills/github/SKILL.md +163 -0
  210. package/skills/gog/SKILL.md +116 -0
  211. package/skills/goplaces/SKILL.md +52 -0
  212. package/skills/healthcheck/SKILL.md +245 -0
  213. package/skills/himalaya/SKILL.md +257 -0
  214. package/skills/himalaya/references/configuration.md +184 -0
  215. package/skills/himalaya/references/message-composition.md +199 -0
  216. package/skills/imsg/SKILL.md +122 -0
  217. package/skills/mcporter/SKILL.md +61 -0
  218. package/skills/model-usage/SKILL.md +69 -0
  219. package/skills/model-usage/references/codexbar-cli.md +33 -0
  220. package/skills/model-usage/scripts/model_usage.py +310 -0
  221. package/skills/nano-banana-pro/SKILL.md +58 -0
  222. package/skills/nano-banana-pro/scripts/generate_image.py +184 -0
  223. package/skills/nano-pdf/SKILL.md +38 -0
  224. package/skills/notion/SKILL.md +172 -0
  225. package/skills/obsidian/SKILL.md +81 -0
  226. package/skills/openai-image-gen/SKILL.md +89 -0
  227. package/skills/openai-image-gen/scripts/gen.py +240 -0
  228. package/skills/openai-whisper/SKILL.md +38 -0
  229. package/skills/openai-whisper-api/SKILL.md +52 -0
  230. package/skills/openai-whisper-api/scripts/transcribe.sh +85 -0
  231. package/skills/openhue/SKILL.md +112 -0
  232. package/skills/oracle/SKILL.md +125 -0
  233. package/skills/ordercli/SKILL.md +78 -0
  234. package/skills/peekaboo/SKILL.md +190 -0
  235. package/skills/sag/SKILL.md +87 -0
  236. package/skills/session-logs/SKILL.md +115 -0
  237. package/skills/sherpa-onnx-tts/SKILL.md +103 -0
  238. package/skills/sherpa-onnx-tts/bin/sherpa-onnx-tts +178 -0
  239. package/skills/skill-creator/SKILL.md +370 -0
  240. package/skills/skill-creator/license.txt +202 -0
  241. package/skills/skill-creator/scripts/init_skill.py +378 -0
  242. package/skills/skill-creator/scripts/package_skill.py +111 -0
  243. package/skills/skill-creator/scripts/quick_validate.py +101 -0
  244. package/skills/slack/SKILL.md +144 -0
  245. package/skills/songsee/SKILL.md +49 -0
  246. package/skills/sonoscli/SKILL.md +46 -0
  247. package/skills/spotify-player/SKILL.md +64 -0
  248. package/skills/summarize/SKILL.md +87 -0
  249. package/skills/things-mac/SKILL.md +86 -0
  250. package/skills/tmux/SKILL.md +153 -0
  251. package/skills/tmux/scripts/find-sessions.sh +112 -0
  252. package/skills/tmux/scripts/wait-for-text.sh +83 -0
  253. package/skills/trello/SKILL.md +95 -0
  254. package/skills/video-frames/SKILL.md +46 -0
  255. package/skills/video-frames/scripts/frame.sh +81 -0
  256. package/skills/voice-call/SKILL.md +45 -0
  257. package/skills/wacli/SKILL.md +72 -0
  258. package/skills/weather/SKILL.md +112 -0
@@ -0,0 +1,208 @@
1
+ ---
2
+ name: nextjs-best-practices
3
+ description: "Next.js App Router principles. Server Components, data fetching, routing patterns."
4
+ risk: unknown
5
+ source: community
6
+ date_added: "2026-02-27"
7
+ ---
8
+
9
+ # Next.js Best Practices
10
+
11
+ > Principles for Next.js App Router development.
12
+
13
+ ---
14
+
15
+ ## 1. Server vs Client Components
16
+
17
+ ### Decision Tree
18
+
19
+ ```
20
+ Does it need...?
21
+
22
+ ├── useState, useEffect, event handlers
23
+ │ └── Client Component ('use client')
24
+
25
+ ├── Direct data fetching, no interactivity
26
+ │ └── Server Component (default)
27
+
28
+ └── Both?
29
+ └── Split: Server parent + Client child
30
+ ```
31
+
32
+ ### By Default
33
+
34
+ | Type | Use |
35
+ |------|-----|
36
+ | **Server** | Data fetching, layout, static content |
37
+ | **Client** | Forms, buttons, interactive UI |
38
+
39
+ ---
40
+
41
+ ## 2. Data Fetching Patterns
42
+
43
+ ### Fetch Strategy
44
+
45
+ | Pattern | Use |
46
+ |---------|-----|
47
+ | **Default** | Static (cached at build) |
48
+ | **Revalidate** | ISR (time-based refresh) |
49
+ | **No-store** | Dynamic (every request) |
50
+
51
+ ### Data Flow
52
+
53
+ | Source | Pattern |
54
+ |--------|---------|
55
+ | Database | Server Component fetch |
56
+ | API | fetch with caching |
57
+ | User input | Client state + server action |
58
+
59
+ ---
60
+
61
+ ## 3. Routing Principles
62
+
63
+ ### File Conventions
64
+
65
+ | File | Purpose |
66
+ |------|---------|
67
+ | `page.tsx` | Route UI |
68
+ | `layout.tsx` | Shared layout |
69
+ | `loading.tsx` | Loading state |
70
+ | `error.tsx` | Error boundary |
71
+ | `not-found.tsx` | 404 page |
72
+
73
+ ### Route Organization
74
+
75
+ | Pattern | Use |
76
+ |---------|-----|
77
+ | Route groups `(name)` | Organize without URL |
78
+ | Parallel routes `@slot` | Multiple same-level pages |
79
+ | Intercepting `(.)` | Modal overlays |
80
+
81
+ ---
82
+
83
+ ## 4. API Routes
84
+
85
+ ### Route Handlers
86
+
87
+ | Method | Use |
88
+ |--------|-----|
89
+ | GET | Read data |
90
+ | POST | Create data |
91
+ | PUT/PATCH | Update data |
92
+ | DELETE | Remove data |
93
+
94
+ ### Best Practices
95
+
96
+ - Validate input with Zod
97
+ - Return proper status codes
98
+ - Handle errors gracefully
99
+ - Use Edge runtime when possible
100
+
101
+ ---
102
+
103
+ ## 5. Performance Principles
104
+
105
+ ### Image Optimization
106
+
107
+ - Use next/image component
108
+ - Set priority for above-fold
109
+ - Provide blur placeholder
110
+ - Use responsive sizes
111
+
112
+ ### Bundle Optimization
113
+
114
+ - Dynamic imports for heavy components
115
+ - Route-based code splitting (automatic)
116
+ - Analyze with bundle analyzer
117
+
118
+ ---
119
+
120
+ ## 6. Metadata
121
+
122
+ ### Static vs Dynamic
123
+
124
+ | Type | Use |
125
+ |------|-----|
126
+ | Static export | Fixed metadata |
127
+ | generateMetadata | Dynamic per-route |
128
+
129
+ ### Essential Tags
130
+
131
+ - title (50-60 chars)
132
+ - description (150-160 chars)
133
+ - Open Graph images
134
+ - Canonical URL
135
+
136
+ ---
137
+
138
+ ## 7. Caching Strategy
139
+
140
+ ### Cache Layers
141
+
142
+ | Layer | Control |
143
+ |-------|---------|
144
+ | Request | fetch options |
145
+ | Data | revalidate/tags |
146
+ | Full route | route config |
147
+
148
+ ### Revalidation
149
+
150
+ | Method | Use |
151
+ |--------|-----|
152
+ | Time-based | `revalidate: 60` |
153
+ | On-demand | `revalidatePath/Tag` |
154
+ | No cache | `no-store` |
155
+
156
+ ---
157
+
158
+ ## 8. Server Actions
159
+
160
+ ### Use Cases
161
+
162
+ - Form submissions
163
+ - Data mutations
164
+ - Revalidation triggers
165
+
166
+ ### Best Practices
167
+
168
+ - Mark with 'use server'
169
+ - Validate all inputs
170
+ - Return typed responses
171
+ - Handle errors
172
+
173
+ ---
174
+
175
+ ## 9. Anti-Patterns
176
+
177
+ | ❌ Don't | ✅ Do |
178
+ |----------|-------|
179
+ | 'use client' everywhere | Server by default |
180
+ | Fetch in client components | Fetch in server |
181
+ | Skip loading states | Use loading.tsx |
182
+ | Ignore error boundaries | Use error.tsx |
183
+ | Large client bundles | Dynamic imports |
184
+
185
+ ---
186
+
187
+ ## 10. Project Structure
188
+
189
+ ```
190
+ app/
191
+ ├── (marketing)/ # Route group
192
+ │ └── page.tsx
193
+ ├── (dashboard)/
194
+ │ ├── layout.tsx # Dashboard layout
195
+ │ └── page.tsx
196
+ ├── api/
197
+ │ └── [resource]/
198
+ │ └── route.ts
199
+ └── components/
200
+ └── ui/
201
+ ```
202
+
203
+ ---
204
+
205
+ > **Remember:** Server Components are the default for a reason. Start there, add client only when needed.
206
+
207
+ ## When to Use
208
+ This skill is applicable to execute the workflow or actions described in the overview.
@@ -0,0 +1,509 @@
1
+ ---
2
+ name: sql-optimization-patterns
3
+ description: Master SQL query optimization, indexing strategies, and EXPLAIN analysis to dramatically improve database performance and eliminate slow queries. Use when debugging slow queries, designing database schemas, or optimizing application performance.
4
+ ---
5
+
6
+ # SQL Optimization Patterns
7
+
8
+ Transform slow database queries into lightning-fast operations through systematic optimization, proper indexing, and query plan analysis.
9
+
10
+ ## When to Use This Skill
11
+
12
+ - Debugging slow-running queries
13
+ - Designing performant database schemas
14
+ - Optimizing application response times
15
+ - Reducing database load and costs
16
+ - Improving scalability for growing datasets
17
+ - Analyzing EXPLAIN query plans
18
+ - Implementing efficient indexes
19
+ - Resolving N+1 query problems
20
+
21
+ ## Core Concepts
22
+
23
+ ### 1. Query Execution Plans (EXPLAIN)
24
+
25
+ Understanding EXPLAIN output is fundamental to optimization.
26
+
27
+ **PostgreSQL EXPLAIN:**
28
+
29
+ ```sql
30
+ -- Basic explain
31
+ EXPLAIN SELECT * FROM users WHERE email = 'user@example.com';
32
+
33
+ -- With actual execution stats
34
+ EXPLAIN ANALYZE
35
+ SELECT * FROM users WHERE email = 'user@example.com';
36
+
37
+ -- Verbose output with more details
38
+ EXPLAIN (ANALYZE, BUFFERS, VERBOSE)
39
+ SELECT u.*, o.order_total
40
+ FROM users u
41
+ JOIN orders o ON u.id = o.user_id
42
+ WHERE u.created_at > NOW() - INTERVAL '30 days';
43
+ ```
44
+
45
+ **Key Metrics to Watch:**
46
+
47
+ - **Seq Scan**: Full table scan (usually slow for large tables)
48
+ - **Index Scan**: Using index (good)
49
+ - **Index Only Scan**: Using index without touching table (best)
50
+ - **Nested Loop**: Join method (okay for small datasets)
51
+ - **Hash Join**: Join method (good for larger datasets)
52
+ - **Merge Join**: Join method (good for sorted data)
53
+ - **Cost**: Estimated query cost (lower is better)
54
+ - **Rows**: Estimated rows returned
55
+ - **Actual Time**: Real execution time
56
+
57
+ ### 2. Index Strategies
58
+
59
+ Indexes are the most powerful optimization tool.
60
+
61
+ **Index Types:**
62
+
63
+ - **B-Tree**: Default, good for equality and range queries
64
+ - **Hash**: Only for equality (=) comparisons
65
+ - **GIN**: Full-text search, array queries, JSONB
66
+ - **GiST**: Geometric data, full-text search
67
+ - **BRIN**: Block Range INdex for very large tables with correlation
68
+
69
+ ```sql
70
+ -- Standard B-Tree index
71
+ CREATE INDEX idx_users_email ON users(email);
72
+
73
+ -- Composite index (order matters!)
74
+ CREATE INDEX idx_orders_user_status ON orders(user_id, status);
75
+
76
+ -- Partial index (index subset of rows)
77
+ CREATE INDEX idx_active_users ON users(email)
78
+ WHERE status = 'active';
79
+
80
+ -- Expression index
81
+ CREATE INDEX idx_users_lower_email ON users(LOWER(email));
82
+
83
+ -- Covering index (include additional columns)
84
+ CREATE INDEX idx_users_email_covering ON users(email)
85
+ INCLUDE (name, created_at);
86
+
87
+ -- Full-text search index
88
+ CREATE INDEX idx_posts_search ON posts
89
+ USING GIN(to_tsvector('english', title || ' ' || body));
90
+
91
+ -- JSONB index
92
+ CREATE INDEX idx_metadata ON events USING GIN(metadata);
93
+ ```
94
+
95
+ ### 3. Query Optimization Patterns
96
+
97
+ **Avoid SELECT \*:**
98
+
99
+ ```sql
100
+ -- Bad: Fetches unnecessary columns
101
+ SELECT * FROM users WHERE id = 123;
102
+
103
+ -- Good: Fetch only what you need
104
+ SELECT id, email, name FROM users WHERE id = 123;
105
+ ```
106
+
107
+ **Use WHERE Clause Efficiently:**
108
+
109
+ ```sql
110
+ -- Bad: Function prevents index usage
111
+ SELECT * FROM users WHERE LOWER(email) = 'user@example.com';
112
+
113
+ -- Good: Create functional index or use exact match
114
+ CREATE INDEX idx_users_email_lower ON users(LOWER(email));
115
+ -- Then:
116
+ SELECT * FROM users WHERE LOWER(email) = 'user@example.com';
117
+
118
+ -- Or store normalized data
119
+ SELECT * FROM users WHERE email = 'user@example.com';
120
+ ```
121
+
122
+ **Optimize JOINs:**
123
+
124
+ ```sql
125
+ -- Bad: Cartesian product then filter
126
+ SELECT u.name, o.total
127
+ FROM users u, orders o
128
+ WHERE u.id = o.user_id AND u.created_at > '2024-01-01';
129
+
130
+ -- Good: Filter before join
131
+ SELECT u.name, o.total
132
+ FROM users u
133
+ JOIN orders o ON u.id = o.user_id
134
+ WHERE u.created_at > '2024-01-01';
135
+
136
+ -- Better: Filter both tables
137
+ SELECT u.name, o.total
138
+ FROM (SELECT * FROM users WHERE created_at > '2024-01-01') u
139
+ JOIN orders o ON u.id = o.user_id;
140
+ ```
141
+
142
+ ## Optimization Patterns
143
+
144
+ ### Pattern 1: Eliminate N+1 Queries
145
+
146
+ **Problem: N+1 Query Anti-Pattern**
147
+
148
+ ```python
149
+ # Bad: Executes N+1 queries
150
+ users = db.query("SELECT * FROM users LIMIT 10")
151
+ for user in users:
152
+ orders = db.query("SELECT * FROM orders WHERE user_id = ?", user.id)
153
+ # Process orders
154
+ ```
155
+
156
+ **Solution: Use JOINs or Batch Loading**
157
+
158
+ ```sql
159
+ -- Solution 1: JOIN
160
+ SELECT
161
+ u.id, u.name,
162
+ o.id as order_id, o.total
163
+ FROM users u
164
+ LEFT JOIN orders o ON u.id = o.user_id
165
+ WHERE u.id IN (1, 2, 3, 4, 5);
166
+
167
+ -- Solution 2: Batch query
168
+ SELECT * FROM orders
169
+ WHERE user_id IN (1, 2, 3, 4, 5);
170
+ ```
171
+
172
+ ```python
173
+ # Good: Single query with JOIN or batch load
174
+ # Using JOIN
175
+ results = db.query("""
176
+ SELECT u.id, u.name, o.id as order_id, o.total
177
+ FROM users u
178
+ LEFT JOIN orders o ON u.id = o.user_id
179
+ WHERE u.id IN (1, 2, 3, 4, 5)
180
+ """)
181
+
182
+ # Or batch load
183
+ users = db.query("SELECT * FROM users LIMIT 10")
184
+ user_ids = [u.id for u in users]
185
+ orders = db.query(
186
+ "SELECT * FROM orders WHERE user_id IN (?)",
187
+ user_ids
188
+ )
189
+ # Group orders by user_id
190
+ orders_by_user = {}
191
+ for order in orders:
192
+ orders_by_user.setdefault(order.user_id, []).append(order)
193
+ ```
194
+
195
+ ### Pattern 2: Optimize Pagination
196
+
197
+ **Bad: OFFSET on Large Tables**
198
+
199
+ ```sql
200
+ -- Slow for large offsets
201
+ SELECT * FROM users
202
+ ORDER BY created_at DESC
203
+ LIMIT 20 OFFSET 100000; -- Very slow!
204
+ ```
205
+
206
+ **Good: Cursor-Based Pagination**
207
+
208
+ ```sql
209
+ -- Much faster: Use cursor (last seen ID)
210
+ SELECT * FROM users
211
+ WHERE created_at < '2024-01-15 10:30:00' -- Last cursor
212
+ ORDER BY created_at DESC
213
+ LIMIT 20;
214
+
215
+ -- With composite sorting
216
+ SELECT * FROM users
217
+ WHERE (created_at, id) < ('2024-01-15 10:30:00', 12345)
218
+ ORDER BY created_at DESC, id DESC
219
+ LIMIT 20;
220
+
221
+ -- Requires index
222
+ CREATE INDEX idx_users_cursor ON users(created_at DESC, id DESC);
223
+ ```
224
+
225
+ ### Pattern 3: Aggregate Efficiently
226
+
227
+ **Optimize COUNT Queries:**
228
+
229
+ ```sql
230
+ -- Bad: Counts all rows
231
+ SELECT COUNT(*) FROM orders; -- Slow on large tables
232
+
233
+ -- Good: Use estimates for approximate counts
234
+ SELECT reltuples::bigint AS estimate
235
+ FROM pg_class
236
+ WHERE relname = 'orders';
237
+
238
+ -- Good: Filter before counting
239
+ SELECT COUNT(*) FROM orders
240
+ WHERE created_at > NOW() - INTERVAL '7 days';
241
+
242
+ -- Better: Use index-only scan
243
+ CREATE INDEX idx_orders_created ON orders(created_at);
244
+ SELECT COUNT(*) FROM orders
245
+ WHERE created_at > NOW() - INTERVAL '7 days';
246
+ ```
247
+
248
+ **Optimize GROUP BY:**
249
+
250
+ ```sql
251
+ -- Bad: Group by then filter
252
+ SELECT user_id, COUNT(*) as order_count
253
+ FROM orders
254
+ GROUP BY user_id
255
+ HAVING COUNT(*) > 10;
256
+
257
+ -- Better: Filter first, then group (if possible)
258
+ SELECT user_id, COUNT(*) as order_count
259
+ FROM orders
260
+ WHERE status = 'completed'
261
+ GROUP BY user_id
262
+ HAVING COUNT(*) > 10;
263
+
264
+ -- Best: Use covering index
265
+ CREATE INDEX idx_orders_user_status ON orders(user_id, status);
266
+ ```
267
+
268
+ ### Pattern 4: Subquery Optimization
269
+
270
+ **Transform Correlated Subqueries:**
271
+
272
+ ```sql
273
+ -- Bad: Correlated subquery (runs for each row)
274
+ SELECT u.name, u.email,
275
+ (SELECT COUNT(*) FROM orders o WHERE o.user_id = u.id) as order_count
276
+ FROM users u;
277
+
278
+ -- Good: JOIN with aggregation
279
+ SELECT u.name, u.email, COUNT(o.id) as order_count
280
+ FROM users u
281
+ LEFT JOIN orders o ON o.user_id = u.id
282
+ GROUP BY u.id, u.name, u.email;
283
+
284
+ -- Better: Use window functions
285
+ SELECT DISTINCT ON (u.id)
286
+ u.name, u.email,
287
+ COUNT(o.id) OVER (PARTITION BY u.id) as order_count
288
+ FROM users u
289
+ LEFT JOIN orders o ON o.user_id = u.id;
290
+ ```
291
+
292
+ **Use CTEs for Clarity:**
293
+
294
+ ```sql
295
+ -- Using Common Table Expressions
296
+ WITH recent_users AS (
297
+ SELECT id, name, email
298
+ FROM users
299
+ WHERE created_at > NOW() - INTERVAL '30 days'
300
+ ),
301
+ user_order_counts AS (
302
+ SELECT user_id, COUNT(*) as order_count
303
+ FROM orders
304
+ WHERE created_at > NOW() - INTERVAL '30 days'
305
+ GROUP BY user_id
306
+ )
307
+ SELECT ru.name, ru.email, COALESCE(uoc.order_count, 0) as orders
308
+ FROM recent_users ru
309
+ LEFT JOIN user_order_counts uoc ON ru.id = uoc.user_id;
310
+ ```
311
+
312
+ ### Pattern 5: Batch Operations
313
+
314
+ **Batch INSERT:**
315
+
316
+ ```sql
317
+ -- Bad: Multiple individual inserts
318
+ INSERT INTO users (name, email) VALUES ('Alice', 'alice@example.com');
319
+ INSERT INTO users (name, email) VALUES ('Bob', 'bob@example.com');
320
+ INSERT INTO users (name, email) VALUES ('Carol', 'carol@example.com');
321
+
322
+ -- Good: Batch insert
323
+ INSERT INTO users (name, email) VALUES
324
+ ('Alice', 'alice@example.com'),
325
+ ('Bob', 'bob@example.com'),
326
+ ('Carol', 'carol@example.com');
327
+
328
+ -- Better: Use COPY for bulk inserts (PostgreSQL)
329
+ COPY users (name, email) FROM '/tmp/users.csv' CSV HEADER;
330
+ ```
331
+
332
+ **Batch UPDATE:**
333
+
334
+ ```sql
335
+ -- Bad: Update in loop
336
+ UPDATE users SET status = 'active' WHERE id = 1;
337
+ UPDATE users SET status = 'active' WHERE id = 2;
338
+ -- ... repeat for many IDs
339
+
340
+ -- Good: Single UPDATE with IN clause
341
+ UPDATE users
342
+ SET status = 'active'
343
+ WHERE id IN (1, 2, 3, 4, 5, ...);
344
+
345
+ -- Better: Use temporary table for large batches
346
+ CREATE TEMP TABLE temp_user_updates (id INT, new_status VARCHAR);
347
+ INSERT INTO temp_user_updates VALUES (1, 'active'), (2, 'active'), ...;
348
+
349
+ UPDATE users u
350
+ SET status = t.new_status
351
+ FROM temp_user_updates t
352
+ WHERE u.id = t.id;
353
+ ```
354
+
355
+ ## Advanced Techniques
356
+
357
+ ### Materialized Views
358
+
359
+ Pre-compute expensive queries.
360
+
361
+ ```sql
362
+ -- Create materialized view
363
+ CREATE MATERIALIZED VIEW user_order_summary AS
364
+ SELECT
365
+ u.id,
366
+ u.name,
367
+ COUNT(o.id) as total_orders,
368
+ SUM(o.total) as total_spent,
369
+ MAX(o.created_at) as last_order_date
370
+ FROM users u
371
+ LEFT JOIN orders o ON u.id = o.user_id
372
+ GROUP BY u.id, u.name;
373
+
374
+ -- Add index to materialized view
375
+ CREATE INDEX idx_user_summary_spent ON user_order_summary(total_spent DESC);
376
+
377
+ -- Refresh materialized view
378
+ REFRESH MATERIALIZED VIEW user_order_summary;
379
+
380
+ -- Concurrent refresh (PostgreSQL)
381
+ REFRESH MATERIALIZED VIEW CONCURRENTLY user_order_summary;
382
+
383
+ -- Query materialized view (very fast)
384
+ SELECT * FROM user_order_summary
385
+ WHERE total_spent > 1000
386
+ ORDER BY total_spent DESC;
387
+ ```
388
+
389
+ ### Partitioning
390
+
391
+ Split large tables for better performance.
392
+
393
+ ```sql
394
+ -- Range partitioning by date (PostgreSQL)
395
+ CREATE TABLE orders (
396
+ id SERIAL,
397
+ user_id INT,
398
+ total DECIMAL,
399
+ created_at TIMESTAMP
400
+ ) PARTITION BY RANGE (created_at);
401
+
402
+ -- Create partitions
403
+ CREATE TABLE orders_2024_q1 PARTITION OF orders
404
+ FOR VALUES FROM ('2024-01-01') TO ('2024-04-01');
405
+
406
+ CREATE TABLE orders_2024_q2 PARTITION OF orders
407
+ FOR VALUES FROM ('2024-04-01') TO ('2024-07-01');
408
+
409
+ -- Queries automatically use appropriate partition
410
+ SELECT * FROM orders
411
+ WHERE created_at BETWEEN '2024-02-01' AND '2024-02-28';
412
+ -- Only scans orders_2024_q1 partition
413
+ ```
414
+
415
+ ### Query Hints and Optimization
416
+
417
+ ```sql
418
+ -- Force index usage (MySQL)
419
+ SELECT * FROM users
420
+ USE INDEX (idx_users_email)
421
+ WHERE email = 'user@example.com';
422
+
423
+ -- Parallel query (PostgreSQL)
424
+ SET max_parallel_workers_per_gather = 4;
425
+ SELECT * FROM large_table WHERE condition;
426
+
427
+ -- Join hints (PostgreSQL)
428
+ SET enable_nestloop = OFF; -- Force hash or merge join
429
+ ```
430
+
431
+ ## Best Practices
432
+
433
+ 1. **Index Selectively**: Too many indexes slow down writes
434
+ 2. **Monitor Query Performance**: Use slow query logs
435
+ 3. **Keep Statistics Updated**: Run ANALYZE regularly
436
+ 4. **Use Appropriate Data Types**: Smaller types = better performance
437
+ 5. **Normalize Thoughtfully**: Balance normalization vs performance
438
+ 6. **Cache Frequently Accessed Data**: Use application-level caching
439
+ 7. **Connection Pooling**: Reuse database connections
440
+ 8. **Regular Maintenance**: VACUUM, ANALYZE, rebuild indexes
441
+
442
+ ```sql
443
+ -- Update statistics
444
+ ANALYZE users;
445
+ ANALYZE VERBOSE orders;
446
+
447
+ -- Vacuum (PostgreSQL)
448
+ VACUUM ANALYZE users;
449
+ VACUUM FULL users; -- Reclaim space (locks table)
450
+
451
+ -- Reindex
452
+ REINDEX INDEX idx_users_email;
453
+ REINDEX TABLE users;
454
+ ```
455
+
456
+ ## Common Pitfalls
457
+
458
+ - **Over-Indexing**: Each index slows down INSERT/UPDATE/DELETE
459
+ - **Unused Indexes**: Waste space and slow writes
460
+ - **Missing Indexes**: Slow queries, full table scans
461
+ - **Implicit Type Conversion**: Prevents index usage
462
+ - **OR Conditions**: Can't use indexes efficiently
463
+ - **LIKE with Leading Wildcard**: `LIKE '%abc'` can't use index
464
+ - **Function in WHERE**: Prevents index usage unless functional index exists
465
+
466
+ ## Monitoring Queries
467
+
468
+ ```sql
469
+ -- Find slow queries (PostgreSQL)
470
+ SELECT query, calls, total_time, mean_time
471
+ FROM pg_stat_statements
472
+ ORDER BY mean_time DESC
473
+ LIMIT 10;
474
+
475
+ -- Find missing indexes (PostgreSQL)
476
+ SELECT
477
+ schemaname,
478
+ tablename,
479
+ seq_scan,
480
+ seq_tup_read,
481
+ idx_scan,
482
+ seq_tup_read / seq_scan AS avg_seq_tup_read
483
+ FROM pg_stat_user_tables
484
+ WHERE seq_scan > 0
485
+ ORDER BY seq_tup_read DESC
486
+ LIMIT 10;
487
+
488
+ -- Find unused indexes (PostgreSQL)
489
+ SELECT
490
+ schemaname,
491
+ tablename,
492
+ indexname,
493
+ idx_scan,
494
+ idx_tup_read,
495
+ idx_tup_fetch
496
+ FROM pg_stat_user_indexes
497
+ WHERE idx_scan = 0
498
+ ORDER BY pg_relation_size(indexrelid) DESC;
499
+ ```
500
+
501
+ ## Resources
502
+
503
+ - **references/postgres-optimization-guide.md**: PostgreSQL-specific optimization
504
+ - **references/mysql-optimization-guide.md**: MySQL/MariaDB optimization
505
+ - **references/query-plan-analysis.md**: Deep dive into EXPLAIN plans
506
+ - **assets/index-strategy-checklist.md**: When and how to create indexes
507
+ - **assets/query-optimization-checklist.md**: Step-by-step optimization guide
508
+ - **scripts/analyze-slow-queries.sql**: Identify slow queries in your database
509
+ - **scripts/index-recommendations.sql**: Generate index recommendations