opencode-swarm-plugin 0.44.0 → 0.44.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (205) hide show
  1. package/bin/swarm.serve.test.ts +6 -4
  2. package/bin/swarm.ts +16 -10
  3. package/dist/compaction-prompt-scoring.js +139 -0
  4. package/dist/eval-capture.js +12811 -0
  5. package/dist/hive.d.ts.map +1 -1
  6. package/dist/index.js +7644 -62599
  7. package/dist/plugin.js +23766 -78721
  8. package/dist/swarm-orchestrate.d.ts.map +1 -1
  9. package/dist/swarm-prompts.d.ts.map +1 -1
  10. package/dist/swarm-review.d.ts.map +1 -1
  11. package/package.json +17 -5
  12. package/.changeset/swarm-insights-data-layer.md +0 -63
  13. package/.hive/analysis/eval-failure-analysis-2025-12-25.md +0 -331
  14. package/.hive/analysis/session-data-quality-audit.md +0 -320
  15. package/.hive/eval-results.json +0 -483
  16. package/.hive/issues.jsonl +0 -138
  17. package/.hive/memories.jsonl +0 -729
  18. package/.opencode/eval-history.jsonl +0 -327
  19. package/.turbo/turbo-build.log +0 -9
  20. package/CHANGELOG.md +0 -2286
  21. package/SCORER-ANALYSIS.md +0 -598
  22. package/docs/analysis/subagent-coordination-patterns.md +0 -902
  23. package/docs/analysis-socratic-planner-pattern.md +0 -504
  24. package/docs/planning/ADR-001-monorepo-structure.md +0 -171
  25. package/docs/planning/ADR-002-package-extraction.md +0 -393
  26. package/docs/planning/ADR-003-performance-improvements.md +0 -451
  27. package/docs/planning/ADR-004-message-queue-features.md +0 -187
  28. package/docs/planning/ADR-005-devtools-observability.md +0 -202
  29. package/docs/planning/ADR-007-swarm-enhancements-worktree-review.md +0 -168
  30. package/docs/planning/ADR-008-worker-handoff-protocol.md +0 -293
  31. package/docs/planning/ADR-009-oh-my-opencode-patterns.md +0 -353
  32. package/docs/planning/ADR-010-cass-inhousing.md +0 -1215
  33. package/docs/planning/ROADMAP.md +0 -368
  34. package/docs/semantic-memory-cli-syntax.md +0 -123
  35. package/docs/swarm-mail-architecture.md +0 -1147
  36. package/docs/testing/context-recovery-test.md +0 -470
  37. package/evals/ARCHITECTURE.md +0 -1189
  38. package/evals/README.md +0 -768
  39. package/evals/compaction-prompt.eval.ts +0 -149
  40. package/evals/compaction-resumption.eval.ts +0 -289
  41. package/evals/coordinator-behavior.eval.ts +0 -307
  42. package/evals/coordinator-session.eval.ts +0 -154
  43. package/evals/evalite.config.ts.bak +0 -15
  44. package/evals/example.eval.ts +0 -31
  45. package/evals/fixtures/cass-baseline.ts +0 -217
  46. package/evals/fixtures/compaction-cases.ts +0 -350
  47. package/evals/fixtures/compaction-prompt-cases.ts +0 -311
  48. package/evals/fixtures/coordinator-sessions.ts +0 -328
  49. package/evals/fixtures/decomposition-cases.ts +0 -105
  50. package/evals/lib/compaction-loader.test.ts +0 -248
  51. package/evals/lib/compaction-loader.ts +0 -320
  52. package/evals/lib/data-loader.evalite-test.ts +0 -289
  53. package/evals/lib/data-loader.test.ts +0 -345
  54. package/evals/lib/data-loader.ts +0 -281
  55. package/evals/lib/llm.ts +0 -115
  56. package/evals/scorers/compaction-prompt-scorers.ts +0 -145
  57. package/evals/scorers/compaction-scorers.ts +0 -305
  58. package/evals/scorers/coordinator-discipline.evalite-test.ts +0 -539
  59. package/evals/scorers/coordinator-discipline.ts +0 -325
  60. package/evals/scorers/index.test.ts +0 -146
  61. package/evals/scorers/index.ts +0 -328
  62. package/evals/scorers/outcome-scorers.evalite-test.ts +0 -27
  63. package/evals/scorers/outcome-scorers.ts +0 -349
  64. package/evals/swarm-decomposition.eval.ts +0 -121
  65. package/examples/commands/swarm.md +0 -745
  66. package/examples/plugin-wrapper-template.ts +0 -2515
  67. package/examples/skills/hive-workflow/SKILL.md +0 -212
  68. package/examples/skills/skill-creator/SKILL.md +0 -223
  69. package/examples/skills/swarm-coordination/SKILL.md +0 -292
  70. package/global-skills/cli-builder/SKILL.md +0 -344
  71. package/global-skills/cli-builder/references/advanced-patterns.md +0 -244
  72. package/global-skills/learning-systems/SKILL.md +0 -644
  73. package/global-skills/skill-creator/LICENSE.txt +0 -202
  74. package/global-skills/skill-creator/SKILL.md +0 -352
  75. package/global-skills/skill-creator/references/output-patterns.md +0 -82
  76. package/global-skills/skill-creator/references/workflows.md +0 -28
  77. package/global-skills/swarm-coordination/SKILL.md +0 -995
  78. package/global-skills/swarm-coordination/references/coordinator-patterns.md +0 -235
  79. package/global-skills/swarm-coordination/references/strategies.md +0 -138
  80. package/global-skills/system-design/SKILL.md +0 -213
  81. package/global-skills/testing-patterns/SKILL.md +0 -430
  82. package/global-skills/testing-patterns/references/dependency-breaking-catalog.md +0 -586
  83. package/opencode-swarm-plugin-0.30.7.tgz +0 -0
  84. package/opencode-swarm-plugin-0.31.0.tgz +0 -0
  85. package/scripts/cleanup-test-memories.ts +0 -346
  86. package/scripts/init-skill.ts +0 -222
  87. package/scripts/migrate-unknown-sessions.ts +0 -349
  88. package/scripts/validate-skill.ts +0 -204
  89. package/src/agent-mail.ts +0 -1724
  90. package/src/anti-patterns.test.ts +0 -1167
  91. package/src/anti-patterns.ts +0 -448
  92. package/src/compaction-capture.integration.test.ts +0 -257
  93. package/src/compaction-hook.test.ts +0 -838
  94. package/src/compaction-hook.ts +0 -1204
  95. package/src/compaction-observability.integration.test.ts +0 -139
  96. package/src/compaction-observability.test.ts +0 -187
  97. package/src/compaction-observability.ts +0 -324
  98. package/src/compaction-prompt-scorers.test.ts +0 -475
  99. package/src/compaction-prompt-scoring.ts +0 -300
  100. package/src/contributor-tools.test.ts +0 -133
  101. package/src/contributor-tools.ts +0 -201
  102. package/src/dashboard.test.ts +0 -611
  103. package/src/dashboard.ts +0 -462
  104. package/src/error-enrichment.test.ts +0 -403
  105. package/src/error-enrichment.ts +0 -219
  106. package/src/eval-capture.test.ts +0 -1015
  107. package/src/eval-capture.ts +0 -929
  108. package/src/eval-gates.test.ts +0 -306
  109. package/src/eval-gates.ts +0 -218
  110. package/src/eval-history.test.ts +0 -508
  111. package/src/eval-history.ts +0 -214
  112. package/src/eval-learning.test.ts +0 -378
  113. package/src/eval-learning.ts +0 -360
  114. package/src/eval-runner.test.ts +0 -223
  115. package/src/eval-runner.ts +0 -402
  116. package/src/export-tools.test.ts +0 -476
  117. package/src/export-tools.ts +0 -257
  118. package/src/hive.integration.test.ts +0 -2241
  119. package/src/hive.ts +0 -1628
  120. package/src/index.ts +0 -940
  121. package/src/learning.integration.test.ts +0 -1815
  122. package/src/learning.ts +0 -1079
  123. package/src/logger.test.ts +0 -189
  124. package/src/logger.ts +0 -135
  125. package/src/mandate-promotion.test.ts +0 -473
  126. package/src/mandate-promotion.ts +0 -239
  127. package/src/mandate-storage.integration.test.ts +0 -601
  128. package/src/mandate-storage.test.ts +0 -578
  129. package/src/mandate-storage.ts +0 -794
  130. package/src/mandates.ts +0 -540
  131. package/src/memory-tools.test.ts +0 -195
  132. package/src/memory-tools.ts +0 -344
  133. package/src/memory.integration.test.ts +0 -334
  134. package/src/memory.test.ts +0 -158
  135. package/src/memory.ts +0 -527
  136. package/src/model-selection.test.ts +0 -188
  137. package/src/model-selection.ts +0 -68
  138. package/src/observability-tools.test.ts +0 -359
  139. package/src/observability-tools.ts +0 -871
  140. package/src/output-guardrails.test.ts +0 -438
  141. package/src/output-guardrails.ts +0 -381
  142. package/src/pattern-maturity.test.ts +0 -1160
  143. package/src/pattern-maturity.ts +0 -525
  144. package/src/planning-guardrails.test.ts +0 -491
  145. package/src/planning-guardrails.ts +0 -438
  146. package/src/plugin.ts +0 -23
  147. package/src/post-compaction-tracker.test.ts +0 -251
  148. package/src/post-compaction-tracker.ts +0 -237
  149. package/src/query-tools.test.ts +0 -636
  150. package/src/query-tools.ts +0 -324
  151. package/src/rate-limiter.integration.test.ts +0 -466
  152. package/src/rate-limiter.ts +0 -774
  153. package/src/replay-tools.test.ts +0 -496
  154. package/src/replay-tools.ts +0 -240
  155. package/src/repo-crawl.integration.test.ts +0 -441
  156. package/src/repo-crawl.ts +0 -610
  157. package/src/schemas/cell-events.test.ts +0 -347
  158. package/src/schemas/cell-events.ts +0 -807
  159. package/src/schemas/cell.ts +0 -257
  160. package/src/schemas/evaluation.ts +0 -166
  161. package/src/schemas/index.test.ts +0 -199
  162. package/src/schemas/index.ts +0 -286
  163. package/src/schemas/mandate.ts +0 -232
  164. package/src/schemas/swarm-context.ts +0 -115
  165. package/src/schemas/task.ts +0 -161
  166. package/src/schemas/worker-handoff.test.ts +0 -302
  167. package/src/schemas/worker-handoff.ts +0 -131
  168. package/src/sessions/agent-discovery.test.ts +0 -137
  169. package/src/sessions/agent-discovery.ts +0 -112
  170. package/src/sessions/index.ts +0 -15
  171. package/src/skills.integration.test.ts +0 -1192
  172. package/src/skills.test.ts +0 -643
  173. package/src/skills.ts +0 -1549
  174. package/src/storage.integration.test.ts +0 -341
  175. package/src/storage.ts +0 -884
  176. package/src/structured.integration.test.ts +0 -817
  177. package/src/structured.test.ts +0 -1046
  178. package/src/structured.ts +0 -762
  179. package/src/swarm-decompose.test.ts +0 -188
  180. package/src/swarm-decompose.ts +0 -1302
  181. package/src/swarm-deferred.integration.test.ts +0 -157
  182. package/src/swarm-deferred.test.ts +0 -38
  183. package/src/swarm-insights.test.ts +0 -214
  184. package/src/swarm-insights.ts +0 -459
  185. package/src/swarm-mail.integration.test.ts +0 -970
  186. package/src/swarm-mail.ts +0 -739
  187. package/src/swarm-orchestrate.integration.test.ts +0 -282
  188. package/src/swarm-orchestrate.test.ts +0 -548
  189. package/src/swarm-orchestrate.ts +0 -3084
  190. package/src/swarm-prompts.test.ts +0 -1270
  191. package/src/swarm-prompts.ts +0 -2077
  192. package/src/swarm-research.integration.test.ts +0 -701
  193. package/src/swarm-research.test.ts +0 -698
  194. package/src/swarm-research.ts +0 -472
  195. package/src/swarm-review.integration.test.ts +0 -285
  196. package/src/swarm-review.test.ts +0 -879
  197. package/src/swarm-review.ts +0 -709
  198. package/src/swarm-strategies.ts +0 -407
  199. package/src/swarm-worktree.test.ts +0 -501
  200. package/src/swarm-worktree.ts +0 -575
  201. package/src/swarm.integration.test.ts +0 -2377
  202. package/src/swarm.ts +0 -38
  203. package/src/tool-adapter.integration.test.ts +0 -1221
  204. package/src/tool-availability.ts +0 -461
  205. package/tsconfig.json +0 -28
@@ -1,1147 +0,0 @@
1
- # Swarm Mail Architecture
2
-
3
- ```
4
- _.------._
5
- .' .--. '. 🐝
6
- / .' '. \ 🐝
7
- | / __ \ | 🐝
8
- 🐝 | | ( ) | | 🐝
9
- 🐝 _ _ | | |__| | |
10
- ( \/ ) \ '. .' / 🐝
11
- 🐝 ____/ \____ '. '----' .'
12
- / \ / \ '-._____.-' 🐝
13
- / () \ / () \
14
- | /\ || /\ | ███████╗██╗ ██╗ █████╗ ██████╗ ███╗ ███╗
15
- | /__\ || /__\ | ██╔════╝██║ ██║██╔══██╗██╔══██╗████╗ ████║
16
- \ / \ / ███████╗██║ █╗ ██║███████║██████╔╝██╔████╔██║
17
- 🐝 '----' '----' ╚════██║██║███╗██║██╔══██║██╔══██╗██║╚██╔╝██║
18
- ███████║╚███╔███╔╝██║ ██║██║ ██║██║ ╚═╝ ██║
19
- 🐝 ╚══════╝ ╚══╝╚══╝ ╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝
20
- 🐝
21
- 🐝 ███╗ ███╗ █████╗ ██╗██╗
22
- ████╗ ████║██╔══██╗██║██║ 🐝
23
- 🐝 ██╔████╔██║███████║██║██║
24
- ██║╚██╔╝██║██╔══██║██║██║ 🐝
25
- 🐝 🐝 ██║ ╚═╝ ██║██║ ██║██║███████╗
26
- ╚═╝ ╚═╝╚═╝ ╚═╝╚═╝╚══════╝ 🐝
27
- 🐝
28
- ⚡ Actor-Model Primitives for Agent Coordination ⚡
29
- ```
30
-
31
- ## Overview
32
-
33
- **Swarm Mail** is an embedded, event-sourced messaging system for multi-agent coordination. Built on **Durable Streams** primitives with Effect-TS, it provides actor-model communication without external server dependencies.
34
-
35
- ### What Problem Does It Solve?
36
-
37
- When multiple AI agents work on the same codebase in parallel, they need to:
38
-
39
- - **Coordinate file access** - prevent edit conflicts via reservations
40
- - **Exchange messages** - async communication for status, blockers, handoffs
41
- - **Request/response** - synchronous-style RPC for data queries
42
- - **Resume after crashes** - positioned consumption with checkpointing
43
- - **Audit all actions** - full event history for debugging and learning
44
-
45
- Traditional solutions require external servers (Redis, Kafka, NATS). Swarm Mail is **embedded** - just PGLite (embedded Postgres) + Effect-TS.
46
-
47
- ### Key Features
48
-
49
- - ✅ **Local-first** - No external servers, no network dependencies
50
- - ✅ **Event-sourced** - Full audit trail of all agent actions
51
- - ✅ **Resumable** - Checkpointed cursors for exactly-once processing
52
- - ✅ **Type-safe** - Effect-TS with full type inference
53
- - ✅ **Actor-model** - Mailboxes, envelopes, distributed promises
54
- - ✅ **File safety** - CAS-based locks for mutual exclusion
55
-
56
- ---
57
-
58
- ## Architecture Stack
59
-
60
- Swarm Mail is built in **3 tiers** - primitives, patterns, and coordination layers.
61
-
62
- ```
63
- ┌─────────────────────────────────────────────────────────────────────────────┐
64
- │ SWARM MAIL STACK │
65
- ├─────────────────────────────────────────────────────────────────────────────┤
66
- │ │
67
- │ TIER 3: COORDINATION │
68
- │ ┌─────────────────────────────────────────────────────────────────────┐ │
69
- │ │ ask<Req, Res>() - Request/Response over Streams (RPC-style) │ │
70
- │ └─────────────────────────────────────────────────────────────────────┘ │
71
- │ │ │
72
- │ TIER 2: PATTERNS ▼ │
73
- │ ┌───────────────────────┐ ┌───────────────────────┐ │
74
- │ │ DurableMailbox │ │ DurableLock │ │
75
- │ │ Actor Inbox + Reply │ │ CAS Mutual Exclusion │ │
76
- │ └───────────────────────┘ └───────────────────────┘ │
77
- │ │ │ │
78
- │ TIER 1: PRIMITIVES ▼ │
79
- │ ┌───────────────────────┐ ┌───────────────────────┐ │
80
- │ │ DurableCursor │ │ DurableDeferred │ │
81
- │ │ Checkpointed Reader │ │ Distributed Promise │ │
82
- │ └───────────────────────┘ └───────────────────────┘ │
83
- │ │ │
84
- │ STORAGE ▼ │
85
- │ ┌─────────────────────────────────────────────────────────────────────┐ │
86
- │ │ PGLite (Embedded Postgres) + Migrations │ │
87
- │ └─────────────────────────────────────────────────────────────────────┘ │
88
- │ │
89
- └─────────────────────────────────────────────────────────────────────────────┘
90
- ```
91
-
92
- ---
93
-
94
- ## Tier 1: Durable Streams Primitives
95
-
96
- The foundational building blocks. Inspired by Kyle Matthews' [Durable Streams protocol](https://x.com/kylemathews/status/1999896667030700098).
97
-
98
- ### DurableCursor - Positioned Event Stream Consumption
99
-
100
- **Purpose:** Read events from a stream with resumable position tracking.
101
-
102
- **Key Concept:** Event streams are append-only logs. Cursors track the "last read position" (sequence number) and checkpoint it to the database. If an agent crashes, it resumes from the last committed position.
103
-
104
- **API:**
105
-
106
- ```typescript
107
- const cursor =
108
- yield *
109
- cursorService.create({
110
- stream: "projects/foo/events", // Stream identifier
111
- checkpoint: "agents/bar/position", // Unique checkpoint name
112
- batchSize: 100, // Read 100 events at a time
113
- });
114
-
115
- // Consume events as async iterable
116
- for await (const msg of cursor.consume()) {
117
- yield * handleMessage(msg.value);
118
- yield * msg.commit(); // Checkpoint this position
119
- }
120
- ```
121
-
122
- **Implementation Details:**
123
-
124
- - **Schema:** `cursors` table with `(stream, checkpoint)` UNIQUE constraint
125
- - **Batching:** Reads events in batches (default 100) for efficiency
126
- - **Commit:** Updates `position` in database + in-memory `Ref`
127
- - **Resumability:** On restart, loads last committed position from DB
128
-
129
- **Use Cases:**
130
-
131
- - Inbox consumption - agent reads messages from stream
132
- - Event processing - workers consume tasks from event log
133
- - Exactly-once semantics - commit after processing, skip on replay
134
-
135
- ---
136
-
137
- ### DurableDeferred - Distributed Promises
138
-
139
- **Purpose:** Create a "distributed promise" that can be resolved from anywhere (think of it as a URL-addressable future value).
140
-
141
- **Key Concept:** You create a deferred with a unique URL, pass that URL to another agent, and block waiting for the response. The other agent resolves the deferred by URL, unblocking you.
142
-
143
- **API:**
144
-
145
- ```typescript
146
- // Agent A: Create deferred and send request
147
- const deferred =
148
- yield *
149
- deferredService.create<Response>({
150
- ttlSeconds: 60,
151
- });
152
-
153
- yield *
154
- mailbox.send("agent-b", {
155
- payload: { task: "getData" },
156
- replyTo: deferred.url, // URL like "deferred:abc123"
157
- });
158
-
159
- const response = yield * deferred.value; // Blocks until resolved or timeout
160
-
161
- // Agent B: Resolve deferred
162
- yield * deferredService.resolve(envelope.replyTo, { data: "result" });
163
- ```
164
-
165
- **Implementation Details:**
166
-
167
- - **Schema:** `deferred` table with `url UNIQUE`, `resolved BOOLEAN`, `value JSONB`
168
- - **In-memory registry:** `Map<url, Effect.Deferred>` for instant resolution (no polling)
169
- - **Fallback polling:** If in-memory deferred missing (agent restart), polls database every 100ms
170
- - **TTL expiry:** Auto-cleanup of expired deferreds
171
- - **Errors:** `TimeoutError` if TTL expires, `NotFoundError` if URL doesn't exist
172
-
173
- **Use Cases:**
174
-
175
- - Request/response - ask pattern (see Tier 3)
176
- - RPC over streams - synchronous-style calls between agents
177
- - Coordination - agent waits for signal from another agent
178
-
179
- ---
180
-
181
- ### DurableLock - Distributed Mutual Exclusion
182
-
183
- **Purpose:** Acquire exclusive locks on resources using Compare-And-Swap (CAS) pattern.
184
-
185
- **Key Concept:** CAS (seq=0) pattern - try to INSERT (no lock exists) or UPDATE (lock expired or we already hold it). Uses exponential backoff for retries on contention.
186
-
187
- **API:**
188
-
189
- ```typescript
190
- // Acquire lock with retry
191
- const lock =
192
- yield *
193
- lockService.acquire("my-resource", {
194
- ttlSeconds: 30,
195
- maxRetries: 10,
196
- });
197
-
198
- try {
199
- // Critical section - only one agent here at a time
200
- yield * doWork();
201
- } finally {
202
- yield * lock.release();
203
- }
204
-
205
- // Or use helper
206
- yield * lockService.withLock("my-resource", Effect.succeed(42));
207
- ```
208
-
209
- **Implementation Details:**
210
-
211
- - **Schema:** `locks` table with `resource UNIQUE`, `holder TEXT`, `seq INTEGER`
212
- - **CAS logic:**
213
- 1. Try `INSERT` (no lock exists) → success
214
- 2. If INSERT fails, try `UPDATE WHERE expires_at < now OR holder = me` → success if stale/reentrant
215
- 3. If UPDATE returns 0 rows → contention, retry with backoff
216
- - **Exponential backoff:** 50ms base delay, doubles each retry (50ms, 100ms, 200ms...)
217
- - **Auto-expiry:** TTL stored in `expires_at`, stale locks can be claimed
218
- - **Errors:** `LockTimeout` if max retries exceeded, `LockNotHeld` if release by wrong holder
219
-
220
- **Use Cases:**
221
-
222
- - File reservations - prevent edit conflicts (Swarm Mail uses this)
223
- - Critical sections - only one agent modifying shared state
224
- - Leader election - first to acquire lock becomes leader
225
-
226
- ---
227
-
228
- ### DurableMailbox - Actor-Style Messaging
229
-
230
- **Purpose:** Send/receive envelopes between agents using cursor-based positioned consumption.
231
-
232
- **Key Concept:** Combines DurableCursor (positioned reading) + Envelope pattern (payload + metadata). Each agent has a named mailbox, messages are filtered by recipient during consumption.
233
-
234
- **API:**
235
-
236
- ```typescript
237
- const mailbox =
238
- yield *
239
- mailboxService.create({
240
- agent: "worker-1",
241
- projectKey: "proj-123",
242
- });
243
-
244
- // Send message with optional reply channel
245
- yield *
246
- mailbox.send("worker-2", {
247
- payload: { task: "process-data" },
248
- replyTo: "deferred:xyz", // For request/response
249
- threadId: "bd-123", // Conversation tracking
250
- });
251
-
252
- // Receive messages (filters to only messages for this agent)
253
- for await (const envelope of mailbox.receive()) {
254
- console.log(envelope.payload);
255
- if (envelope.replyTo) {
256
- yield * DurableDeferred.resolve(envelope.replyTo, result);
257
- }
258
- yield * envelope.commit(); // Checkpoint position
259
- }
260
- ```
261
-
262
- **Implementation Details:**
263
-
264
- - **Cursor creation:** Creates `DurableCursor` with filter `types: ["message_sent"]`
265
- - **Filtering:** `eventToEnvelope()` skips messages not addressed to this agent
266
- - **Envelope structure:**
267
- ```typescript
268
- {
269
- payload: T, // Your message data
270
- replyTo?: string, // Deferred URL for response
271
- sender: string, // From agent
272
- messageId: number, // Message ID
273
- threadId?: string, // Conversation grouping
274
- commit: () => Effect // Checkpoint this message
275
- }
276
- ```
277
- - **Storage:** Messages stored as `message_sent` events in event stream
278
-
279
- **Use Cases:**
280
-
281
- - Agent inbox - receive tasks, status updates, blockers
282
- - Broadcast - send message to multiple agents
283
- - Request/response - combine with DurableDeferred (see ask pattern)
284
-
285
- ---
286
-
287
- ## Tier 2: Coordination Patterns
288
-
289
- Higher-level patterns built by composing primitives.
290
-
291
- ### The Ask Pattern - Request/Response over Streams
292
-
293
- **Purpose:** Synchronous-style RPC between agents using async streams.
294
-
295
- **How It Works:**
296
-
297
- ```
298
- ┌─────────┐ ┌─────────┐
299
- │ Agent A │ │ Agent B │
300
- └────┬────┘ └────┬────┘
301
- │ │
302
- │ 1. Create deferred │
303
- │ url = "deferred:abc123" │
304
- │ │
305
- │ 2. Send message with replyTo=url │
306
- ├──────────────────────────────────────────────────────────>│
307
- │ │
308
- │ 3. Block on deferred.value │ 4. Process request
309
- │ (waits...) │
310
- │ │
311
- │ 5. Resolve deferred(url) │
312
- │<───────────────────────────────────────────────────────────┤
313
- │ │
314
- │ 6. Unblocked, return response │
315
- │ │
316
- ```
317
-
318
- **Code Example:**
319
-
320
- ```typescript
321
- // Agent A (caller)
322
- const response =
323
- yield *
324
- ask<Request, Response>({
325
- mailbox: myMailbox,
326
- to: "worker-2",
327
- payload: { query: "getUserData", userId: 123 },
328
- ttlSeconds: 30,
329
- });
330
-
331
- // Agent B (responder)
332
- for await (const envelope of mailbox.receive()) {
333
- const result = processRequest(envelope.payload);
334
- if (envelope.replyTo) {
335
- yield * DurableDeferred.resolve(envelope.replyTo, result);
336
- }
337
- yield * envelope.commit();
338
- }
339
- ```
340
-
341
- **Why This Matters:**
342
-
343
- - **Synchronous feel, async reality** - code looks like RPC, but it's event-driven
344
- - **Resilient** - if responder crashes, caller gets timeout (not hung forever)
345
- - **Auditable** - all request/response pairs in event log
346
- - **Type-safe** - full TypeScript inference for request/response types
347
-
348
- ---
349
-
350
- ### File Reservation Protocol
351
-
352
- **Purpose:** Prevent edit conflicts when multiple agents modify files.
353
-
354
- **Flow:**
355
-
356
- ```
357
- ┌──────────┐ ┌──────────┐
358
- │ Agent A │ │ Agent B │
359
- └────┬─────┘ └────┬─────┘
360
- │ │
361
- │ 1. Reserve src/auth.ts (exclusive) │
362
- ├────────────────────────────────────────────┐ │
363
- │ │ │
364
- │ 2. DurableLock.acquire("src/auth.ts") │ │
365
- │ → Granted (no conflicts) │ │
366
- │ │ │
367
- │ 3. Edit src/auth.ts │ │
368
- │ │ │ 4. Reserve src/auth.ts
369
- │ │ ├──────────────────────┐
370
- │ │ │ │
371
- │ │ │ 5. Lock contention │
372
- │ │ │ → Warned (Agent A │
373
- │ │ │ holds lock) │
374
- │ │ │ │
375
- │ 6. Release src/auth.ts │ │ │
376
- ├────────────────────────────────────────────┤ │ │
377
- │ │ │ │
378
- │ 7. DurableLock.release() │ │ 8. Retry acquire │
379
- │ │ │ → Granted │
380
- │ │ │ │
381
- ```
382
-
383
- **Implementation:**
384
-
385
- - Uses `DurableLock` for mutual exclusion
386
- - Conflicts are **warnings**, not blockers (agents can proceed with awareness)
387
- - TTL expiry (default 1 hour) prevents deadlocks from crashed agents
388
- - Materialized view (`reservations` table) for fast conflict queries
389
-
390
- ---
391
-
392
- ## Tier 3: High-Level APIs
393
-
394
- The public API exposed to agents (wraps primitives + patterns).
395
-
396
- ### Swarm Mail API
397
-
398
- These are the functions called by agents (via plugin tools or direct import):
399
-
400
- ```typescript
401
- // Initialization
402
- const { agentName, projectKey } = await initSwarmAgent({
403
- projectPath: "/abs/path",
404
- agentName: "BlueLake", // Optional, auto-generated if omitted
405
- taskDescription: "Implementing auth",
406
- });
407
-
408
- // Send message
409
- await sendSwarmMessage({
410
- projectPath,
411
- fromAgent: "BlueLake",
412
- toAgents: ["coordinator"],
413
- subject: "Progress: bd-123.2",
414
- body: "Auth service complete",
415
- threadId: "bd-123",
416
- importance: "normal",
417
- });
418
-
419
- // Read inbox
420
- const { messages } = await getSwarmInbox({
421
- projectPath,
422
- agentName: "BlueLake",
423
- limit: 5, // Max 5 (hard cap for context)
424
- urgentOnly: false,
425
- includeBodies: false, // Headers only by default
426
- });
427
-
428
- // Read full message
429
- const message = await readSwarmMessage({
430
- projectPath,
431
- messageId: 123,
432
- markAsRead: true,
433
- });
434
-
435
- // Reserve files
436
- const { granted, conflicts } = await reserveSwarmFiles({
437
- projectPath,
438
- agentName: "BlueLake",
439
- paths: ["src/auth/**"],
440
- reason: "bd-123.2: Auth service",
441
- exclusive: true,
442
- ttlSeconds: 3600,
443
- });
444
-
445
- // Release files
446
- await releaseSwarmFiles({
447
- projectPath,
448
- agentName: "BlueLake",
449
- paths: ["src/auth/**"],
450
- });
451
- ```
452
-
453
- ---
454
-
455
- ## Event Sourcing Architecture
456
-
457
- Swarm Mail is fully event-sourced. **All state changes are events first, materialized views second.**
458
-
459
- ### Event Flow
460
-
461
- ```
462
- ┌─────────────┐
463
- │ Action │ Agent calls API (e.g., sendMessage)
464
- └──────┬──────┘
465
-
466
-
467
- ┌─────────────┐
468
- │ Create │ createEvent("message_sent", {...})
469
- │ Event │ Returns: { type, timestamp, ...payload }
470
- └──────┬──────┘
471
-
472
-
473
- ┌─────────────┐
474
- │ Append │ appendEvent() → INSERT INTO events
475
- │ to Log │ Gets auto-increment id + sequence
476
- └──────┬──────┘
477
-
478
-
479
- ┌─────────────┐
480
- │ Update │ updateProjections() triggers based on event type
481
- │ Views │ - message_sent → INSERT messages, UPDATE agent read status
482
- └──────┬──────┘ - file_reserved → INSERT reservations
483
- │ - message_read → UPDATE message read flags
484
-
485
- ┌─────────────┐
486
- │ Query │ getInbox(), getMessage(), getReservations()
487
- │ Views │ Fast queries on materialized tables
488
- └─────────────┘
489
- ```
490
-
491
- ### Event Types
492
-
493
- | Event Type | Trigger | Projections Updated |
494
- | ------------------ | ---------------------------- | -------------------------------- |
495
- | `agent_registered` | Agent init | `agents` table |
496
- | `message_sent` | sendMessage() | `messages`, `message_recipients` |
497
- | `message_read` | readMessage(markAsRead=true) | `messages.read_by` JSONB |
498
- | `message_acked` | acknowledgeMessage() | `messages.acked_by` JSONB |
499
- | `file_reserved` | reserveFiles() | `reservations` |
500
- | `file_released` | releaseFiles() | `reservations` (DELETE expired) |
501
-
502
- ### Why Event Sourcing?
503
-
504
- - ✅ **Full audit trail** - every agent action is logged forever
505
- - ✅ **Time travel** - replay events to reconstruct past state
506
- - ✅ **Debugging** - when agents conflict, trace exact sequence of events
507
- - ✅ **Learning** - analyze event patterns to improve swarm strategies
508
- - ✅ **Resumability** - cursors checkpoint position, replay from there on crash
509
-
510
- ---
511
-
512
- ## Comparison to Agent Mail
513
-
514
- Swarm Mail is **inspired by** [Agent Mail](https://github.com/sst/opencode) (SST's multi-agent coordination layer) but built from scratch with different trade-offs.
515
-
516
- | Aspect | Agent Mail (SST) | Swarm Mail (This Plugin) |
517
- | ------------------------ | ----------------------------- | -------------------------------------- |
518
- | **Architecture** | MCP server (external process) | Embedded (PGLite in-process) |
519
- | **Storage** | PGLite (embedded Postgres) | PGLite (WASM Postgres) |
520
- | **Dependencies** | Requires MCP server running | Zero external deps, just npm install |
521
- | **Effect-TS** | No | Yes (full Effect integration) |
522
- | **Event Sourcing** | No (CRUD operations) | Yes (append-only event log) |
523
- | **Cursors** | No (queries are one-shot) | Yes (resumable positioned consumption) |
524
- | **Distributed Promises** | No | Yes (DurableDeferred) |
525
- | **Type Safety** | MCP JSON-RPC (strings) | Full TypeScript + Zod validation |
526
- | **Local-First** | Requires server | True local-first (no network) |
527
- | **Learning System** | No | Yes (eval records, pattern maturity) |
528
-
529
- **When to Use Agent Mail:**
530
-
531
- - You're already using OpenCode with MCP infrastructure
532
- - You need cross-project coordination (multiple repos)
533
- - You want battle-tested SST ecosystem integration
534
-
535
- **When to Use Swarm Mail:**
536
-
537
- - You want embedded, zero-config coordination
538
- - You're building Effect-TS applications
539
- - You need event sourcing and audit trails
540
- - You want resumable cursors for exactly-once semantics
541
- - You're using this plugin's learning/swarm features
542
-
543
- ---
544
-
545
- ## Database Schema
546
-
547
- Swarm Mail uses **PGLite** (embedded Postgres compiled to WASM). Schema is managed via migrations (see `src/streams/migrations.ts`).
548
-
549
- ### Core Tables
550
-
551
- #### `events` - Append-Only Event Log
552
-
553
- ```sql
554
- CREATE TABLE events (
555
- id SERIAL PRIMARY KEY,
556
- sequence SERIAL, -- Auto-increment, never gaps
557
- type TEXT NOT NULL, -- Event type (message_sent, file_reserved, etc.)
558
- timestamp BIGINT NOT NULL, -- Unix timestamp in ms
559
- payload JSONB NOT NULL, -- Event-specific data
560
- project_key TEXT NOT NULL,
561
- agent_name TEXT
562
- );
563
- CREATE INDEX idx_events_sequence ON events(sequence);
564
- CREATE INDEX idx_events_type ON events(type);
565
- CREATE INDEX idx_events_project ON events(project_key);
566
- ```
567
-
568
- **All state changes flow through this table.** Other tables are materialized views.
569
-
570
- ---
571
-
572
- #### `cursors` - Checkpointed Read Positions
573
-
574
- ```sql
575
- CREATE TABLE cursors (
576
- id SERIAL PRIMARY KEY,
577
- stream TEXT NOT NULL, -- Stream name (e.g., "projects/foo/events")
578
- checkpoint TEXT NOT NULL, -- Checkpoint name (e.g., "agents/bar/position")
579
- position BIGINT NOT NULL, -- Last committed sequence number
580
- updated_at BIGINT NOT NULL,
581
- UNIQUE(stream, checkpoint)
582
- );
583
- CREATE INDEX idx_cursors_checkpoint ON cursors(checkpoint);
584
- ```
585
-
586
- **DurableCursor uses this to resume from last position after restart.**
587
-
588
- ---
589
-
590
- #### `deferred` - Distributed Promises
591
-
592
- ```sql
593
- CREATE TABLE deferred (
594
- id SERIAL PRIMARY KEY,
595
- url TEXT NOT NULL UNIQUE, -- Unique identifier (e.g., "deferred:abc123")
596
- resolved BOOLEAN NOT NULL DEFAULT FALSE,
597
- value JSONB, -- Resolution value
598
- error TEXT, -- Rejection error message
599
- expires_at BIGINT NOT NULL, -- TTL expiry timestamp
600
- created_at BIGINT NOT NULL
601
- );
602
- CREATE INDEX idx_deferred_url ON deferred(url);
603
- CREATE INDEX idx_deferred_expires ON deferred(expires_at);
604
- ```
605
-
606
- **DurableDeferred uses this for URL-addressable futures.**
607
-
608
- ---
609
-
610
- #### `locks` - CAS-Based Mutual Exclusion
611
-
612
- ```sql
613
- CREATE TABLE locks (
614
- id SERIAL PRIMARY KEY,
615
- resource TEXT NOT NULL UNIQUE, -- Resource being locked
616
- holder TEXT NOT NULL, -- Agent holding lock
617
- seq INTEGER NOT NULL, -- CAS sequence number
618
- acquired_at BIGINT NOT NULL,
619
- expires_at BIGINT NOT NULL,
620
- UNIQUE(resource)
621
- );
622
- ```
623
-
624
- **DurableLock uses this for file reservations and critical sections.**
625
-
626
- ---
627
-
628
- #### `messages` - Materialized Message View
629
-
630
- ```sql
631
- CREATE TABLE messages (
632
- id SERIAL PRIMARY KEY,
633
- project_key TEXT NOT NULL,
634
- from_agent TEXT NOT NULL,
635
- subject TEXT NOT NULL,
636
- body TEXT NOT NULL,
637
- thread_id TEXT,
638
- importance TEXT DEFAULT 'normal',
639
- created_at BIGINT NOT NULL,
640
- read_by JSONB DEFAULT '[]', -- Array of agent names who read it
641
- acked_by JSONB DEFAULT '[]' -- Array of agent names who acked it
642
- );
643
- ```
644
-
645
- **Built from `message_sent`, `message_read`, `message_acked` events.**
646
-
647
- ---
648
-
649
- #### `message_recipients` - Message Routing
650
-
651
- ```sql
652
- CREATE TABLE message_recipients (
653
- id SERIAL PRIMARY KEY,
654
- message_id INTEGER NOT NULL REFERENCES messages(id),
655
- agent_name TEXT NOT NULL,
656
- UNIQUE(message_id, agent_name)
657
- );
658
- CREATE INDEX idx_message_recipients_agent ON message_recipients(agent_name);
659
- ```
660
-
661
- **Tracks which agents should see which messages (for inbox filtering).**
662
-
663
- ---
664
-
665
- #### `reservations` - Active File Locks
666
-
667
- ```sql
668
- CREATE TABLE reservations (
669
- id SERIAL PRIMARY KEY,
670
- project_key TEXT NOT NULL,
671
- agent_name TEXT NOT NULL,
672
- path_pattern TEXT NOT NULL, -- File path or glob pattern
673
- exclusive BOOLEAN NOT NULL,
674
- reason TEXT,
675
- expires_at BIGINT NOT NULL
676
- );
677
- CREATE INDEX idx_reservations_agent ON reservations(agent_name);
678
- CREATE INDEX idx_reservations_path ON reservations(path_pattern);
679
- ```
680
-
681
- **Built from `file_reserved` events, expired entries removed by `file_released`.**
682
-
683
- ---
684
-
685
- ### Migrations
686
-
687
- Schema versioning via `migrations.ts`:
688
-
689
- ```typescript
690
- export const migrations: Migration[] = [
691
- {
692
- version: 1,
693
- description: "Add cursors table for DurableCursor",
694
- up: `CREATE TABLE cursors (...)`,
695
- down: `DROP TABLE cursors`,
696
- },
697
- {
698
- version: 2,
699
- description: "Add deferred table for DurableDeferred",
700
- up: `CREATE TABLE deferred (...)`,
701
- down: `DROP TABLE deferred`,
702
- },
703
- // ... more migrations
704
- ];
705
- ```
706
-
707
- **Migrations run automatically on first database access.** New migrations are applied in order, version tracked in `schema_version` table.
708
-
709
- ---
710
-
711
- ## Effect-TS Integration
712
-
713
- Swarm Mail is **built with Effect** - a functional effect system for TypeScript.
714
-
715
- ### Why Effect?
716
-
717
- - ✅ **Composability** - combine primitives into patterns without callback hell
718
- - ✅ **Type inference** - full TypeScript inference for errors and dependencies
719
- - ✅ **Retries** - built-in retry schedules with exponential backoff
720
- - ✅ **Resource safety** - `Effect.ensuring` guarantees cleanup (like try/finally)
721
- - ✅ **Dependency injection** - Layers for services, no globals
722
-
723
- ### Services and Layers
724
-
725
- Each primitive is an Effect **Service** (via `Context.Tag`):
726
-
727
- ```typescript
728
- // Define service interface
729
- export class DurableCursor extends Context.Tag("DurableCursor")<
730
- DurableCursor,
731
- DurableCursorService
732
- >() {}
733
-
734
- // Implement service
735
- export const DurableCursorLive = DurableCursor.of({
736
- create: createCursorImpl,
737
- });
738
-
739
- // Use service in Effect
740
- const program = Effect.gen(function* () {
741
- const cursor = yield* DurableCursor; // Service dependency
742
- const consumer = yield* cursor.create({ stream: "..." });
743
- // ...
744
- });
745
-
746
- // Provide service implementation
747
- Effect.runPromise(program.pipe(Effect.provide(DurableCursorLive)));
748
- ```
749
-
750
- **Layers compose** - higher-level services depend on lower-level ones:
751
-
752
- ```typescript
753
- // DurableMailbox depends on DurableCursor
754
- const MailboxLayer = Layer.mergeAll(CursorLayer, DurableMailboxLive);
755
-
756
- // Ask pattern depends on Mailbox + Deferred
757
- export const DurableAskLive = Layer.mergeAll(DurableDeferredLive, MailboxLayer);
758
-
759
- // Use in program
760
- const program = Effect.gen(function* () {
761
- const mailbox = yield* DurableMailbox;
762
- const response = yield* ask({ mailbox, to: "worker-2", payload: {...} });
763
- });
764
-
765
- Effect.runPromise(program.pipe(Effect.provide(DurableAskLive)));
766
- ```
767
-
768
- ### Error Handling
769
-
770
- Effect has **typed errors** - errors are part of the Effect signature:
771
-
772
- ```typescript
773
- // Effect<Success, Error, Requirements>
774
- type MyEffect = Effect.Effect<number, LockError | TimeoutError, DurableLock>;
775
-
776
- // Handle specific error types
777
- yield *
778
- lockService.acquire("resource").pipe(
779
- Effect.catchTag("LockTimeout", () => Effect.succeed(null)),
780
- Effect.catchTag("LockContention", () => Effect.fail(new MyError())),
781
- );
782
- ```
783
-
784
- ---
785
-
786
- ## Use Cases and Examples
787
-
788
- ### Example 1: Swarm Worker Agent
789
-
790
- **Scenario:** Worker agent receives subtask, reserves files, reports progress, completes.
791
-
792
- ```typescript
793
- import { Effect } from "effect";
794
- import { DurableAskLive } from "./streams/effect/layers";
795
- import {
796
- initSwarmAgent,
797
- sendSwarmMessage,
798
- reserveSwarmFiles,
799
- } from "./swarm-mail";
800
-
801
- const program = Effect.gen(function* () {
802
- // 1. Initialize agent
803
- const { agentName, projectKey } = yield* Effect.promise(() =>
804
- initSwarmAgent({
805
- projectPath: "/abs/path",
806
- taskDescription: "bd-123.2: Auth service",
807
- }),
808
- );
809
-
810
- console.log(`Agent: ${agentName}`); // e.g., "DarkStone"
811
-
812
- // 2. Reserve files
813
- const { granted, conflicts } = yield* Effect.promise(() =>
814
- reserveSwarmFiles({
815
- projectPath: projectKey,
816
- agentName,
817
- paths: ["src/auth/**"],
818
- reason: "bd-123.2: Auth service",
819
- exclusive: true,
820
- }),
821
- );
822
-
823
- if (conflicts.length > 0) {
824
- console.warn("File conflicts detected:", conflicts);
825
- }
826
-
827
- // 3. Report progress
828
- yield* Effect.promise(() =>
829
- sendSwarmMessage({
830
- projectPath: projectKey,
831
- fromAgent: agentName,
832
- toAgents: ["coordinator"],
833
- subject: "Progress: bd-123.2",
834
- body: "Reserved files, starting work",
835
- threadId: "bd-123",
836
- importance: "normal",
837
- }),
838
- );
839
-
840
- // 4. Do work...
841
- yield* Effect.sleep("5 seconds");
842
-
843
- // 5. Report completion
844
- yield* Effect.promise(() =>
845
- sendSwarmMessage({
846
- projectPath: projectKey,
847
- fromAgent: agentName,
848
- toAgents: ["coordinator"],
849
- subject: "Complete: bd-123.2",
850
- body: "Auth service implemented",
851
- threadId: "bd-123",
852
- importance: "high",
853
- }),
854
- );
855
- });
856
-
857
- Effect.runPromise(program.pipe(Effect.provide(DurableAskLive)));
858
- ```
859
-
860
- ---
861
-
862
- ### Example 2: Request/Response with Ask Pattern
863
-
864
- **Scenario:** Agent A asks Agent B for data, blocks until response.
865
-
866
- ```typescript
867
- import { ask, respond } from "./streams/effect/ask";
868
- import { DurableAskLive } from "./streams/effect/layers";
869
-
870
- // Agent A (requester)
871
- const agentA = Effect.gen(function* () {
872
- const mailboxService = yield* DurableMailbox;
873
- const mailbox = yield* mailboxService.create({
874
- agent: "agent-a",
875
- projectKey: "proj",
876
- });
877
-
878
- console.log("Requesting user data...");
879
-
880
- const response = yield* ask<Request, Response>({
881
- mailbox,
882
- to: "agent-b",
883
- payload: { userId: 123 },
884
- ttlSeconds: 30,
885
- });
886
-
887
- console.log("Got response:", response);
888
- });
889
-
890
- // Agent B (responder)
891
- const agentB = Effect.gen(function* () {
892
- const mailboxService = yield* DurableMailbox;
893
- const mailbox = yield* mailboxService.create({
894
- agent: "agent-b",
895
- projectKey: "proj",
896
- });
897
-
898
- console.log("Listening for requests...");
899
-
900
- for await (const envelope of mailbox.receive<Request>()) {
901
- console.log("Processing request:", envelope.payload);
902
-
903
- const result = { name: "John", id: envelope.payload.userId };
904
-
905
- yield* respond(envelope, result);
906
- yield* envelope.commit();
907
- }
908
- });
909
-
910
- // Run both agents in parallel
911
- const program = Effect.all([agentA, agentB], { concurrency: "unbounded" });
912
- Effect.runPromise(program.pipe(Effect.provide(DurableAskLive)));
913
- ```
914
-
915
- ---
916
-
917
- ### Example 3: Resumable Event Processing
918
-
919
- **Scenario:** Worker consumes events, checkpoints position, resumes after crash.
920
-
921
- ```typescript
922
- import { DurableCursor } from "./streams/effect/cursor";
923
- import { DurableCursorLive } from "./streams/effect/layers";
924
-
925
- const program = Effect.gen(function* () {
926
- const cursorService = yield* DurableCursor;
927
-
928
- const cursor = yield* cursorService.create({
929
- stream: "projects/foo/events",
930
- checkpoint: "workers/event-processor",
931
- batchSize: 50,
932
- });
933
-
934
- console.log("Starting from position:", yield* cursor.getPosition());
935
-
936
- for await (const msg of cursor.consume()) {
937
- console.log("Processing event:", msg.value);
938
-
939
- // Simulate processing
940
- yield* Effect.sleep("100 millis");
941
-
942
- // Checkpoint position
943
- yield* msg.commit();
944
- }
945
- });
946
-
947
- Effect.runPromise(program.pipe(Effect.provide(DurableCursorLive)));
948
-
949
- // If this crashes, next run starts from last committed position
950
- ```
951
-
952
- ---
953
-
954
- ## Diagrams
955
-
956
- ### Message Flow Between Agents
957
-
958
- ```
959
- ┌───────────┐ ┌──────────────┐ ┌───────────┐
960
- │ Agent A │ │ Event Stream │ │ Agent B │
961
- └─────┬─────┘ └──────┬───────┘ └─────┬─────┘
962
- │ │ │
963
- │ 1. Send message │ │
964
- ├────────────────────────────────>│ │
965
- │ { to: "agent-b", │ 2. Append message_sent event │
966
- │ payload: {...} } │ (id=42, seq=100) │
967
- │ │ │
968
- │ │ 3. Update message view │
969
- │ │ INSERT INTO messages (...) │
970
- │ │ │
971
- │ │ │
972
- │ │ 4. Agent B consumes │
973
- │ │<─────────────────────────────────┤
974
- │ │ DurableCursor.consume() │
975
- │ │ afterSequence=99 │
976
- │ │ │
977
- │ │ 5. Return events │
978
- │ ├─────────────────────────────────>│
979
- │ │ [{ seq=100, type=msg_sent }] │
980
- │ │ │
981
- │ │ 6. Process message │
982
- │ │<─────────────────────────────────┤
983
- │ │ msg.commit() │
984
- │ │ │
985
- │ │ 7. Checkpoint position=100 │
986
- │ │<─────────────────────────────────┤
987
- │ │ │
988
- ```
989
-
990
- ---
991
-
992
- ### File Reservation Protocol (CAS Lock)
993
-
994
- ```
995
- ┌──────────┐ ┌──────────┐
996
- │ Agent A │ │ Agent B │
997
- └────┬─────┘ └────┬─────┘
998
- │ │
999
- │ 1. Reserve src/auth.ts │
1000
- ├─────────────────────────────┐ │
1001
- │ │ │
1002
- │ 2. DurableLock.acquire() │ │
1003
- │ INSERT INTO locks │ │
1004
- │ VALUES ('src/auth.ts', │ │
1005
- │ 'AgentA', │ │
1006
- │ seq=0) │ │
1007
- │ → SUCCESS │ │
1008
- │ │ │
1009
- │ 3. Edit src/auth.ts │ │
1010
- │ │ │
1011
- │ │ 4. Reserve src/auth.ts
1012
- │ │ ├──────────────────────┐
1013
- │ │ │ │
1014
- │ │ │ 5. TRY INSERT │
1015
- │ │ │ → CONFLICT │
1016
- │ │ │ │
1017
- │ │ │ 6. TRY UPDATE │
1018
- │ │ │ WHERE expires_at │
1019
- │ │ │ < now OR │
1020
- │ │ │ holder='AgentB' │
1021
- │ │ │ → 0 rows │
1022
- │ │ │ │
1023
- │ │ │ 7. Retry with │
1024
- │ │ │ exponential │
1025
- │ │ │ backoff... │
1026
- │ │ │ │
1027
- │ 8. Release lock │ │ │
1028
- ├─────────────────────────────┤ │ │
1029
- │ │ │ │
1030
- │ 9. DELETE FROM locks │ │ 10. Retry succeeds │
1031
- │ WHERE resource=... │ │ UPDATE (0 rows │
1032
- │ AND holder='AgentA' │ │ before, now │
1033
- │ │ │ lock is free) │
1034
- │ │ │ → SUCCESS │
1035
- │ │ │ │
1036
- ```
1037
-
1038
- ---
1039
-
1040
- ### Ask Pattern (Request/Response)
1041
-
1042
- ```mermaid
1043
- sequenceDiagram
1044
- participant A as Agent A
1045
- participant D as DurableDeferred
1046
- participant M as Mailbox
1047
- participant B as Agent B
1048
-
1049
- A->>D: create({ ttl: 60s })
1050
- D-->>A: url: "deferred:abc123"
1051
- A->>M: send(to: B, { payload, replyTo: url })
1052
- M->>B: Envelope delivered
1053
- B->>B: Process request
1054
- B->>D: resolve(url, response)
1055
- D-->>A: Response (unblocks)
1056
- ```
1057
-
1058
- ---
1059
-
1060
- ### Event Stream Structure
1061
-
1062
- ```
1063
- events table (append-only log):
1064
-
1065
- ┌────┬──────────┬──────────────┬───────────────┬─────────────────────────────┐
1066
- │ id │ sequence │ type │ timestamp │ payload │
1067
- ├────┼──────────┼──────────────┼───────────────┼─────────────────────────────┤
1068
- │ 1 │ 1 │ agent_reg... │ 1704000000000 │ { agent: "AgentA", ... } │
1069
- │ 2 │ 2 │ message_sent │ 1704000001000 │ { from: "A", to: ["B"] } │
1070
- │ 3 │ 3 │ file_reserv. │ 1704000002000 │ { agent: "A", paths: [...]} │
1071
- │ 4 │ 4 │ message_read │ 1704000003000 │ { message_id: 2, agent: "B"}│
1072
- │ 5 │ 5 │ file_releas. │ 1704000004000 │ { agent: "A", paths: [...]} │
1073
- └────┴──────────┴──────────────┴───────────────┴─────────────────────────────┘
1074
-
1075
-
1076
- │ DurableCursor reads from here
1077
- │ Checkpoints last sequence number
1078
-
1079
- ┌───────────────────┐
1080
- │ cursors table │
1081
- ├───────────────────┤
1082
- │ checkpoint: "..." │
1083
- │ position: 4 │ ← Resume from seq=5
1084
- └───────────────────┘
1085
- ```
1086
-
1087
- ---
1088
-
1089
- ## Credits and Inspiration
1090
-
1091
- - **[Kyle Matthews](https://twitter.com/kylemathews)** (Founder/CPO @ [Electric SQL](https://electric-sql.com/)) - The Durable Streams protocol and the insight that composable primitives can build powerful actor systems. [Original tweet](https://x.com/kylemathews/status/1999896667030700098)
1092
- - **[Agent Mail](https://github.com/sst/opencode)** - Multi-agent coordination layer for OpenCode (SST). Swarm Mail's API surface is heavily inspired by Agent Mail's design.
1093
- - **[Electric SQL](https://electric-sql.com/)** - Real-time sync engine for Postgres. The cursor pattern and positioned consumption ideas come from Electric's sync protocol.
1094
- - **[Effect-TS](https://effect.website/)** - Functional effect system powering the implementation. Effect's composability and type safety make the primitives ergonomic.
1095
-
1096
- ---
1097
-
1098
- ## What's Next?
1099
-
1100
- Swarm Mail is **production-ready** but has room for optimization:
1101
-
1102
- ### Performance Improvements
1103
-
1104
- - [ ] **LISTEN/NOTIFY** - replace deferred polling with Postgres LISTEN/NOTIFY for instant wakeup
1105
- - [ ] **Connection pooling** - reuse PGLite connections instead of opening new ones
1106
- - [ ] **Batch inserts** - batch multiple events into one transaction
1107
- - [ ] **Index tuning** - add covering indexes for hot queries
1108
-
1109
- ### Features
1110
-
1111
- - [ ] **Message priorities** - priority queue for urgent messages
1112
- - [ ] **Dead letter queue** - failed messages go to DLQ for retry
1113
- - [ ] **Message TTL** - auto-expire old messages
1114
- - [ ] **Broadcast channels** - pub/sub for topic-based routing
1115
- - [ ] **Saga pattern** - distributed transactions with compensations
1116
-
1117
- ### Developer Experience
1118
-
1119
- - [ ] **DevTools UI** - web UI for inspecting events, cursors, locks
1120
- - [ ] **CLI tools** - `swarm-mail inspect`, `swarm-mail replay`
1121
- - [ ] **Metrics** - Prometheus metrics for message latency, lock contention
1122
- - [ ] **Tracing** - OpenTelemetry spans for distributed tracing
1123
-
1124
- ---
1125
-
1126
- ## Summary
1127
-
1128
- Swarm Mail is an **embedded, event-sourced actor system** for multi-agent coordination. It provides:
1129
-
1130
- - ✅ **Durable Streams primitives** - DurableCursor, DurableDeferred, DurableLock, DurableMailbox
1131
- - ✅ **Composable patterns** - Ask pattern (RPC), file reservations (CAS locks)
1132
- - ✅ **Effect-TS integration** - type-safe, composable, dependency-injected
1133
- - ✅ **Local-first** - embedded PGLite, zero network dependencies
1134
- - ✅ **Event sourcing** - full audit trail, time travel debugging
1135
- - ✅ **Resumable** - checkpointed cursors for exactly-once processing
1136
-
1137
- **Use it when you need multi-agent coordination without external servers.**
1138
-
1139
- ---
1140
-
1141
- ```
1142
- * * 🐝 * *
1143
- * * * * *
1144
- 🐝 SHIP IT 🐝
1145
- * * * * *
1146
- * * 🐝 * *
1147
- ```