opencode-swarm-plugin 0.22.0 → 0.23.0

package/docs/planning/ADR-005-devtools-observability.md

@@ -0,0 +1,202 @@
+# ADR-005: DevTools + Observability
+
+## Status
+
+Proposed
+
+## Context
+
+Swarm Mail currently has no visibility:
+
+- No UI to inspect events, messages, locks
+- No metrics on latency, queue depth, throughput
+- No distributed tracing across agents
+- Hard to debug coordination issues
+
+Need both developer tools (UI + CLI) and production observability (metrics + tracing).
+
+## Decision
+
+Build layered observability:
+
+### 1. DevTools UI (SvelteKit)
+
+**Stack:**
+
+- SvelteKit for SSR + static export
+- Vite for dev server + build
+- Server-Sent Events (SSE) for real-time updates
+- Embeddable static build
+
+**Features:**
+
+- Event stream viewer (filterable, searchable)
+- Message inbox/outbox per agent
+- File reservation timeline
+- Saga instance tracker (future)
+
+**Build:**
+
+```bash
+cd apps/devtools
+bun run build  # Static export to apps/devtools/build
+```
+
+**Embed in plugin:**
+
+```typescript
+// Serve static UI at /_swarm/devtools
+const server = serve({
+  port: 4000,
+  fetch: (req) => {
+    if (req.url.startsWith("/_swarm/devtools")) {
+      return serveStatic("apps/devtools/build");
+    }
+  },
+});
+```
+
+### 2. CLI (@effect/cli)
+
+**Commands:**
+
+```bash
+swarm events [--project <key>] [--type <type>] [--tail]
+swarm messages [--agent <name>] [--unread]
+swarm locks [--agent <name>]
+swarm replay --from <sequence> [--to <sequence>]
+swarm metrics
+```
+
+**Implementation:**
+
+```typescript
+import { Command } from "@effect/cli";
+
+const eventsCommand = Command.make(
+  "events",
+  {
+    project: Options.string("project").optional,
+    type: Options.string("type").optional,
+    tail: Options.boolean("tail"),
+  },
+  ({ project, type, tail }) => {
+    // Query events table, optionally --tail with live query
+  },
+);
+```
+
+### 3. Metrics (Prometheus)
+
+**Histograms:**
+
+- `swarm_message_latency_seconds` - Send to receive time
+- `swarm_lock_contention_seconds` - Time waiting for lock
+- `swarm_queue_depth` - Unread messages per agent
+
+**Counters:**
+
+- `swarm_events_total{type}` - Events by type
+- `swarm_messages_sent_total{sender, recipient}`
+- `swarm_locks_acquired_total{agent}`
+
+**Example:**
+
+```typescript
+import { Registry, Histogram } from 'prom-client'
+
+const messageLat ency = new Histogram({
+  name: 'swarm_message_latency_seconds',
+  help: 'Message delivery latency',
+  buckets: [0.01, 0.05, 0.1, 0.5, 1.0, 5.0]
+})
+
+// Record latency
+const start = Date.now()
+await sendMessage(msg)
+const latency = (Date.now() - start) / 1000
+messageLatency.observe(latency)
+```
+
+### 4. Distributed Tracing (OpenTelemetry)
+
+**Integration:**
+
+```typescript
+import { @effect/opentelemetry } from '@effect/opentelemetry'
+import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node'
+
+const provider = new NodeTracerProvider()
+const tracer = provider.getTracer('swarm-mail')
+
+// Trace message send
+const span = tracer.startSpan('sendMessage', {
+  attributes: {
+    'swarm.sender': 'AgentA',
+    'swarm.recipient': 'AgentB',
+    'swarm.thread_id': 'bd-123'
+  }
+})
+
+await sendMessage(msg)
+span.end()
+```
+
+**Trace Propagation:**
+
+- Add trace_id to message metadata
+- Worker agents continue traces from parents
+- Visualize full swarm execution flow
+
+## Consequences
+
+### Easier
+
+- **Visibility** - See all events, messages, locks in real-time
+- **Debugging** - Trace issues across agents via distributed tracing
+- **Performance** - Identify slow operations via histograms
+- **Operations** - CLI for prod debugging without UI
+
+### More Difficult
+
+- **Maintenance** - Another app to maintain (DevTools UI)
+- **Bundle size** - Metrics/tracing deps increase plugin size
+- **Performance overhead** - Instrumentation adds latency
+- **Configuration** - Metrics exporters, trace backends
+
+## Implementation Notes
+
+### Phase 1: CLI (Week 1)
+
+- Add @effect/cli dependency
+- Implement events, messages, locks commands
+- Test with real swarm sessions
+
+### Phase 2: DevTools UI (Week 2-3)
+
+- Scaffold SvelteKit app
+- Build event stream viewer
+- Add SSE endpoint for real-time updates
+- Static export + embed in plugin
+
+### Phase 3: Metrics (Week 4)
+
+- Add prom-client dependency
+- Instrument send/receive latency
+- Add queue depth gauge
+- Expose /metrics endpoint
+
+### Phase 4: Tracing (Week 5)
+
+- Add @effect/opentelemetry
+- Instrument message send/receive
+- Propagate trace context
+- Test with Jaeger/Zipkin
+
+### Success Criteria
+
+- [ ] CLI can tail events in real-time
+- [ ] DevTools UI shows live message stream
+- [ ] Metrics exposed at /metrics endpoint
+- [ ] Traces visible in Jaeger UI
+- [ ] Documentation for all observability tools

package/docs/planning/ROADMAP.md

@@ -0,0 +1,368 @@
+# Implementation Roadmap
+
+## Overview
+
+This roadmap implements all 5 ADRs in phased releases over 12 weeks. Each phase delivers incremental value and can ship independently.
+
+## Phase 0: Preparation (Week 0)
+
+**Goal:** Set up monorepo infrastructure and validate all ADRs
+
+**Tasks:**
+
+- [ ] Install Turborepo + configure turbo.json
+- [ ] Set up Changesets for versioning
+- [ ] Configure dependency-cruiser for circular dep detection
+- [ ] Create packages/swarm-mail and packages/opencode-swarm-plugin directories
+- [ ] Set up TypeScript project references
+- [ ] Configure CI/CD with Turborepo caching
+
+**Deliverables:**
+
+- Working monorepo build (`turbo run build`)
+- Published ADRs in docs/planning/
+- CI pipeline running tests for both packages
+
+**Success Criteria:**
+
+- `bun run build` builds both packages in correct order
+- No circular dependencies detected
+- CI completes in <2 minutes (with caching)
+
+---
+
+## Phase 1: Package Extraction (Weeks 1-2)
+
+**Goal:** Extract swarm-mail package and publish to npm
+
+**ADRs:** ADR-001, ADR-002
+
+**Tasks:**
+
+- [ ] Move src/streams/\* to packages/swarm-mail/src/streams/
+- [ ] Move agent-mail.ts, swarm-mail.ts to swarm-mail
+- [ ] Update imports in opencode-swarm-plugin to use swarm-mail
+- [ ] Migrate integration tests
+- [ ] Write swarm-mail README with examples
+- [ ] Add deprecation warnings in opencode-swarm-plugin
+- [ ] Publish swarm-mail@0.1.0 to npm
+
+**Deliverables:**
+
+- swarm-mail published on npm
+- opencode-swarm-plugin depends on swarm-mail
+- Migration guide for existing users
+- TypeDoc API documentation
+
+**Success Criteria:**
+
+- All tests pass in both packages
+- swarm-mail works in standalone project
+- No circular dependencies
+- Published tarball <500KB
+
+---
+
+## Phase 2: Performance Optimizations (Weeks 3-4)
+
+**Goal:** Replace polling with live queries and add batch operations
+
+**ADRs:** ADR-003
+
+**Tasks:**
+
+- [ ] Create live query wrapper (src/streams/live-query.ts)
+- [ ] Add subscription cleanup tracking
+- [ ] Replace polling in watchInbox()
+- [ ] Replace polling in watchEvents()
+- [ ] Add batch message send API
+- [ ] Add batch event append API
+- [ ] Write integration tests for live queries
+- [ ] Run performance benchmarks (latency, CPU, memory)
+
+**Deliverables:**
+
+- Live queries for inbox, events, file reservations
+- Batch APIs for messages and events
+- Performance benchmarks showing improvements
+- Feature flag for gradual rollout
+
+**Success Criteria:**
+
+- Notification latency <50ms (99th percentile)
+- CPU usage <1% in idle state
+- Batch operations 10x faster than individual
+- Memory usage increase <20%
+
+**Metrics (Before → After):**
+
+- Latency: 250-500ms → <10ms (25-50x improvement)
+- CPU: 5-10% → <1% (5-10x reduction)
+- Queries/sec: 2-4 → 0 (eliminated)
+
+---
+
+## Phase 3: Message Queue Features (Weeks 5-7)
+
+**Goal:** Add priority queues, DLQ, TTL, pub/sub
+
+**ADRs:** ADR-004
+
+**Tasks:**
+
+**Week 5: Priority Queues + DLQ**
+
+- [ ] Add priority column to messages table
+- [ ] Update getInbox() to ORDER BY priority DESC
+- [ ] Create failed_messages table
+- [ ] Implement retry logic with exponential backoff
+- [ ] Add DLQ viewer to CLI
+
+**Week 6: TTL + Pub/Sub**
+
+- [ ] Add expires_at column to messages
+- [ ] Implement background TTL cleanup job
+- [ ] Add topic column to messages
+- [ ] Implement subscribeToTopic() using live queries
+- [ ] Support wildcard topic subscriptions
+
+**Week 7: Testing + Documentation**
+
+- [ ] Write integration tests for all features
+- [ ] Add examples to README
+- [ ] Document retry/DLQ behavior
+- [ ] Document pub/sub patterns
+
+**Deliverables:**
+
+- Priority queues (4 levels: 0=urgent, 3=low)
+- DLQ with retry tracking
+- TTL with background cleanup
+- Pub/sub with wildcard topics
+
+**Success Criteria:**
+
+- Priority messages processed first
+- Failed messages retry 3x before DLQ
+- Expired messages cleaned up within 5 minutes
+- Topic subscriptions work with wildcards
+
+---
+
+## Phase 4: DevTools + CLI (Weeks 8-10)
+
+**Goal:** Build DevTools UI and CLI for observability
+
+**ADRs:** ADR-005
+
+**Tasks:**
+
+**Week 8: CLI**
+
+- [ ] Add @effect/cli dependency
+- [ ] Implement `swarm events` command
+- [ ] Implement `swarm messages` command
+- [ ] Implement `swarm locks` command
+- [ ] Implement `swarm replay` command
+- [ ] Add `--tail` mode for real-time updates
+
+**Week 9: DevTools UI**
+
+- [ ] Scaffold SvelteKit app in apps/devtools
+- [ ] Build event stream viewer
+- [ ] Build message inbox/outbox viewer
+- [ ] Build file reservation timeline
+- [ ] Add SSE endpoint for real-time updates
+
+**Week 10: Integration + Polish**
+
+- [ ] Static export of DevTools UI
+- [ ] Embed UI in plugin (serve at /\_swarm/devtools)
+- [ ] Add screenshots to README
+- [ ] Write user guide
+
+**Deliverables:**
+
+- CLI with 5 commands (events, messages, locks, replay, metrics)
+- DevTools UI (embeddable SvelteKit app)
+- Real-time updates via SSE
+- User guide with screenshots
+
+**Success Criteria:**
+
+- CLI can tail events in real-time
+- DevTools UI shows live message stream
+- UI works offline (static export)
+- Documentation covers all CLI commands
+
+---
+
+## Phase 5: Metrics + Tracing (Weeks 11-12)
+
+**Goal:** Add Prometheus metrics and OpenTelemetry tracing
+
+**ADRs:** ADR-005
+
+**Tasks:**
+
+**Week 11: Metrics**
+
+- [ ] Add prom-client dependency
+- [ ] Instrument message send/receive latency
+- [ ] Add lock contention histogram
+- [ ] Add queue depth gauge
+- [ ] Expose /metrics endpoint
+- [ ] Add Grafana dashboard template
+
+**Week 12: Tracing**
+
+- [ ] Add @effect/opentelemetry dependency
+- [ ] Instrument message send/receive spans
+- [ ] Propagate trace context in messages
+- [ ] Add trace_id to message metadata
+- [ ] Test with Jaeger/Zipkin
+- [ ] Write tracing guide
+
+**Deliverables:**
+
+- Prometheus metrics at /metrics endpoint
+- OpenTelemetry tracing integration
+- Grafana dashboard template
+- Tracing guide with Jaeger setup
+
+**Success Criteria:**
+
+- Metrics exposed and scrapeable by Prometheus
+- Traces visible in Jaeger UI
+- Trace propagation across agents works
+- Documentation for all observability tools
+
+---
+
+## Phase 6: Saga Pattern (Future)
+
+**Goal:** Implement saga orchestration for long-running workflows
+
+**ADRs:** ADR-004 (Phase 5)
+
+**Status:** Deferred to v2.0
+
+**Tasks:**
+
+- [ ] Create saga_instances and saga_steps tables
+- [ ] Add saga coordinator logic
+- [ ] Implement compensation pattern
+- [ ] Add saga viewer to DevTools UI
+- [ ] Write saga pattern examples
+
+**Deliverables:**
+
+- Saga orchestration pattern
+- Compensation (undo) support
+- Saga viewer in DevTools
+- 3+ example saga workflows
+
+---
+
+## Release Schedule
+
+| Version   | Phase | Features                       | ETA     |
+| --------- | ----- | ------------------------------ | ------- |
+| **0.1.0** | 1     | swarm-mail package extraction | Week 2  |
+| **0.2.0** | 2     | Live queries, batch operations | Week 4  |
+| **0.3.0** | 3     | Priority, DLQ, TTL, pub/sub    | Week 7  |
+| **0.4.0** | 4     | DevTools UI + CLI              | Week 10 |
+| **0.5.0** | 5     | Metrics + tracing              | Week 12 |
+| **1.0.0** | All   | Stable release                 | Week 13 |
+| **2.0.0** | 6     | Saga pattern (future)          | TBD     |
+
+---
+
+## Dependencies Between Phases
+
+```
+Phase 0 (Monorepo)
+  └──> Phase 1 (Package Extraction)
+         └──> Phase 2 (Performance)
+                ├──> Phase 3 (Queue Features)
+                └──> Phase 4 (DevTools)
+                       └──> Phase 5 (Metrics/Tracing)
+                              └──> Phase 6 (Sagas, future)
+```
+
+**Critical Path:** Phases 0→1→2 are sequential. Phases 3-5 can partially overlap after Phase 2.
+
+---
+
+## Risk Mitigation
+
+| Risk                                     | Phase | Mitigation                                                |
+| ---------------------------------------- | ----- | --------------------------------------------------------- |
+| Breaking changes during extraction       | 1     | Feature branch, comprehensive tests, migration guide      |
+| Performance regression with live queries | 2     | Feature flag, benchmark before/after, fallback to polling |
+| Complexity of saga pattern               | 6     | Defer to v2.0, gather user feedback first                 |
+| DevTools UI maintenance burden           | 4     | Keep UI minimal, focus on CLI for power users             |
+| Metrics overhead                         | 5     | Make metrics opt-in, minimal instrumentation              |
+
+---
+
+## Success Metrics
+
+**Phase 1 (Package Extraction):**
+
+- swarm-mail used in 3+ external projects within 3 months
+- Zero breaking changes reported by users
+
+**Phase 2 (Performance):**
+
+- 25x faster notification latency (500ms → <10ms)
+- 5x lower CPU usage (10% → <2%)
+
+**Phase 3 (Queue Features):**
+
+- 95% of messages processed within priority SLA
+- <1% messages fail to DLQ
+
+**Phase 4 (DevTools):**
+
+- 80% of developers use DevTools UI for debugging
+- CLI used in 50%+ of support cases
+
+**Phase 5 (Metrics/Tracing):**
+
+- Metrics dashboard used in production monitoring
+- Distributed traces reduce debugging time by 50%
+
+---
+
+## Post-1.0 Backlog (v2.0+)
+
+**Saga Pattern** (ADR-004 Phase 5)
+
+- Long-running multi-agent workflows
+- Compensation (undo) support
+- Saga state visualization
+
+**Advanced Pub/Sub**
+
+- Message routing rules
+- Filter expressions (SQL-like WHERE clauses)
+- At-least-once vs exactly-once delivery guarantees
+
+**Multi-Project Support**
+
+- Cross-project message routing
+- Project-level isolation
+- Shared infra for mono repos
+
+**Performance Tier 2**
+
+- Connection pooling for multi-DB scenarios
+- Message batching optimizations
+- SKIP LOCKED for exactly-once semantics
+
+**Security**
+
+- Message encryption at rest
+- Agent authentication/authorization
+- Audit logging

package/package.json

@@ -1,10 +1,10 @@
 {
   "name": "opencode-swarm-plugin",
-  "version": "0.22.0",
+  "version": "0.23.0",
   "description": "Multi-agent swarm coordination for OpenCode with learning capabilities, beads integration, and Agent Mail",
   "type": "module",
-  "main": "dist/index.js",
-  "types": "dist/index.d.ts",
+  "main": "./dist/index.js",
+  "types": "./dist/index.d.ts",
   "bin": {
     "swarm": "./bin/swarm.ts"
   },
@@ -14,37 +14,25 @@
       "types": "./dist/index.d.ts"
     },
     "./plugin": {
-      "import": "./dist/plugin.js"
+      "import": "./dist/plugin.js",
+      "types": "./dist/plugin.d.ts"
     }
   },
   "scripts": {
-    "build": "bun build ./src/index.ts --outdir ./dist --target node --external @electric-sql/pglite && bun build ./src/plugin.ts --outfile ./dist/plugin.js --target node --external @electric-sql/pglite",
+    "build": "bun build ./src/index.ts --outdir ./dist --target node --external @electric-sql/pglite --external swarm-mail && bun build ./src/plugin.ts --outfile ./dist/plugin.js --target node --external @electric-sql/pglite --external swarm-mail && tsc",
     "dev": "bun --watch src/index.ts",
-    "test": "bun test src/schemas src/skills.test.ts src/streams/index.test.ts src/streams/events.test.ts src/streams/projections.test.ts src/streams/migrations.test.ts src/streams/debug.test.ts src/streams/agent-mail.test.ts src/streams/effect/deferred.test.ts src/streams/effect/mailbox.test.ts",
-    "test:watch": "bun test --watch src/schemas src/skills.test.ts",
-    "test:integration": "vitest run --config vitest.integration.config.ts",
-    "test:swarm": "bun test src/swarm-mail.integration.test.ts",
-    "test:all": "bun run test && bun run test:swarm",
+    "test": "bun test --timeout 10000 src/anti-patterns.test.ts src/mandate-promotion.test.ts src/mandate-storage.test.ts src/output-guardrails.test.ts src/pattern-maturity.test.ts src/skills.test.ts src/structured.test.ts src/schemas/",
+    "test:integration": "bun test --timeout 60000 src/*.integration.test.ts",
+    "test:all": "bun test --timeout 60000 src/",
+    "test:watch": "bun test --watch src/",
     "typecheck": "tsc --noEmit",
-    "clean": "rm -rf dist",
-    "eval:dev": "evalite watch evals/",
-    "eval:run": "evalite run evals/",
-    "eval:ci": "evalite run evals/ --threshold 80",
-    "release": "npm run build && npm version patch && git push && npm run publish:otp",
-    "release:minor": "npm run build && npm version minor && git push && npm run publish:otp",
-    "release:major": "npm run build && npm version major && git push && npm run publish:otp",
-    "publish:otp": "bash -c 'source .env && npm publish --otp=$(op item get $NPM_1P_ITEM --otp)'"
+    "publish:pkg": "npm publish --access public --provenance"
   },
   "dependencies": {
-    "@clack/prompts": "^0.11.0",
-    "@effect/schema": "^0.75.5",
-    "@electric-sql/pglite": "0.3.14",
     "@opencode-ai/plugin": "^1.0.134",
-    "effect": "^3.19.12",
     "gray-matter": "^4.0.3",
     "ioredis": "^5.4.1",
-    "minimatch": "^10.1.1",
-    "nanoid": "^5.1.6",
+    "swarm-mail": "workspace:*",
     "zod": "4.1.8"
   },
   "devDependencies": {
@@ -53,6 +41,7 @@
     "ai": "6.0.0-beta.150",
     "bun-types": "^1.3.4",
     "evalite": "^1.0.0-beta.10",
+    "turbo": "^2.6.3",
     "typescript": "^5.7.0",
     "vitest": "^4.0.15"
   },

package/src/agent-mail.ts

@@ -31,7 +31,7 @@ import { tool } from "@opencode-ai/plugin";
 import { z } from "zod";
 import { isToolAvailable, warnMissingTool } from "./tool-availability";
 import { getRateLimiter, type RateLimiter } from "./rate-limiter";
-import type { MailSessionState } from "./streams/events";
+import type { MailSessionState } from "swarm-mail";
 
 // ============================================================================
 // Configuration

package/src/beads.ts

@@ -104,8 +104,7 @@ import {
   type BeadCreateArgs,
   type EpicCreateResult,
 } from "./schemas";
-import { createEvent } from "./streams/events";
-import { appendEvent } from "./streams/store";
+import { createEvent, appendEvent } from "swarm-mail";
 
 /**
  * Custom error for bead operations

package/src/index.ts

@@ -320,12 +320,12 @@ export {
 } from "./swarm-mail";
 
 /**
- * Re-export shared types from streams/events
+ * Re-export shared types from swarm-mail package
  *
  * Includes:
  * - MailSessionState - Shared session state type for Agent Mail and Swarm Mail
  */
-export { type MailSessionState } from "./streams/events";
+export { type MailSessionState } from "swarm-mail";
 
 /**
  * Re-export structured module