agent-inbox 0.1.8 → 0.1.9

package/docs/PLAN.md

@@ -1,545 +0,0 @@
-# Agent Inbox — Implementation Plan
-
-## Project Structure
-
-```
-agent-inbox/
-├── package.json
-├── tsconfig.json
-├── vitest.config.ts
-├── DESIGN.md
-├── src/
-│   ├── index.ts                    # Entry point — starts IPC server + MAP connection + MCP server
-│   ├── types.ts                    # Core types: Agent, Message, Recipient, Conversation, Turn, Thread, MessageContent
-│   │
-│   ├── storage/
-│   │   ├── interface.ts            # Storage interface
-│   │   └── memory.ts              # InMemoryStorage implementation
-│   │
-│   ├── router/
-│   │   └── message-router.ts      # Message routing logic: resolve recipients, local vs remote delivery
-│   │
-│   ├── traceability/
-│   │   └── traceability.ts        # Auto-create conversations/turns/threads from messaging events
-│   │
-│   ├── ipc/
-│   │   └── ipc-server.ts          # UNIX socket server (NDJSON protocol, same pattern as sidecar)
-│   │
-│   ├── map/
-│   │   └── map-client.ts          # MAP SDK connection wrapper (connect, send, receive, lifecycle)
-│   │
-│   └── mcp/
-│       └── mcp-server.ts          # MCP tools: send_message, check_inbox, read_thread, list_agents
-│
-└── test/
-    ├── storage.test.ts
-    ├── message-router.test.ts
-    ├── traceability.test.ts
-    ├── ipc-server.test.ts
-    └── mcp-server.test.ts
-```
-
-## Implementation Steps
-
-### Step 1: Project Scaffolding
-
-Set up the TypeScript project with dependencies.
-
-**package.json**:
-- `typescript`, `vitest` (dev)
-- `@multi-agent-protocol/sdk` (peer, optional — same as claude-code-swarm)
-- `@modelcontextprotocol/sdk` (MCP server)
-- `ulid` (ID generation)
-
-**tsconfig.json**: ESM output, strict mode, Node 18+ target.
-
-### Step 2: Core Types (`src/types.ts`)
-
-Define all data model types from the design doc:
-- `Agent` — agent_id, display_name, program, model, scope, status, metadata, timestamps
-- `Message` — id, scope, sender_id, recipients, subject, content, thread_tag, in_reply_to, importance, metadata, created_at
-- `Recipient` — agent_id, kind (to/cc/bcc), delivered_at, read_at, ack_at
-- `MessageContent` — union type: text, data, event, reference, x-*
-- `Conversation`, `Turn`, `Thread` — traceability entities
-- `IpcCommand` / `IpcResponse` — UNIX socket protocol shapes (matching sidecar conventions)
-
-### Step 3: Storage Interface + InMemoryStorage
-
-**`src/storage/interface.ts`**:
-```typescript
-interface Storage {
-  getAgent(agentId: string): Agent | undefined;
-  putAgent(agent: Agent): void;
-  listAgents(scope: string): Agent[];
-
-  getMessage(id: string): Message | undefined;
-  putMessage(message: Message): Message;
-  getInbox(agentId: string, opts?: { unreadOnly?: boolean }): Message[];
-  getThread(threadTag: string, scope: string): Message[];
-  searchMessages(query: string, scope: string): Message[];
-
-  getConversation(id: string): Conversation | undefined;
-  putConversation(conversation: Conversation): Conversation;
-  listConversations(scope: string): Conversation[];
-  addTurn(turn: Turn): void;
-  getTurns(conversationId: string): Turn[];
-}
-```
-
-**`src/storage/memory.ts`**: Maps and arrays. `getInbox` filters by recipient agent_id + unread. `getThread` filters by thread_tag + scope. `searchMessages` does simple substring matching.
-
-**Tests**: CRUD operations, inbox filtering, thread retrieval.
-
-### Step 4: Message Router (`src/router/message-router.ts`)
-
-Core routing logic:
-- `resolveRecipients(to)` — resolve target(s) from agent ID, scope, or qualified address
-- `normalizeContent(payload)` — wrap plain strings as `{ type: "text", text: ... }`, pass structured content through
-- `routeMessage(command)` — create Message record, resolve recipients, determine local vs remote delivery
-- `isLocal(agentId)` — check if agent is in local registry
-
-For Phase 1, all delivery is local (single MAP server). Federation routing deferred to Phase 3.
-
-**Tests**: Recipient resolution, content normalization, local routing.
-
-### Step 5: Traceability Layer (`src/traceability/traceability.ts`)
-
-Auto-creates traceability records from messaging events:
-- Message with `conversationId` → add Turn to that Conversation
-- Message with `thread_tag`, no `conversationId` → find or create Conversation for `thread_tag + scope`, add Turn
-- Message with neither → add Turn to catch-all Conversation for scope
-- Reply (`in_reply_to`) → create Thread if not exists, link Turn
-
-Uses an internal EventEmitter — the message router emits `message.created` events, the traceability layer subscribes.
-
-**Tests**: Auto-conversation creation, thread creation from replies, catch-all grouping.
-
-### Step 6: IPC Server (`src/ipc/ipc-server.ts`)
-
-UNIX socket server using the same NDJSON protocol as the sidecar. This is how the claude-code-swarm plugin communicates with Agent Inbox.
-
-**Socket path**: Configurable, default `~/.claude/agent-inbox/inbox.sock` (or per-session).
-
-**Commands handled**:
-
-| Command | Action |
-|---------|--------|
-| `{ action: "send", from, to, payload, threadTag?, inReplyTo?, meta? }` | Route message via message-router |
-| `{ action: "emit", event, meta? }` | Broadcast to scope via MAP connection |
-| `{ action: "notify", event: { type: "agent.spawn", agent: {...} } }` | Update agent registry |
-| `{ action: "notify", event: { type: "agent.done", agentId: "..." } }` | Mark agent offline |
-| `{ action: "ping" }` | Health check → `{ ok: true, pid }` |
-
-**Response**: `{ ok: true, messageId?: string }` or `{ ok: false, error: string }`
-
-Same NDJSON-over-UNIX-socket pattern as sidecar-server.mjs: parse newline-delimited JSON, dispatch command, respond with JSON + newline.
-
-**Tests**: Socket connection, command dispatch, response format, concurrent clients.
-
-### Step 7: MAP Client (`src/map/map-client.ts`)
-
-Wrapper around `@multi-agent-protocol/sdk` for Agent Inbox's MAP connection.
-
-- `connect(serverUrl, opts)` — connect to MAP server as `agent-inbox` role
-- `send(target, message)` — deliver message to specific agent via MAP
-- `onMessage(handler)` — receive incoming messages, store in inbox, write to inbox.jsonl
-- `disconnect()` — clean shutdown
-
-The MAP client is **optional** — Agent Inbox can run without a MAP server in standalone/testing mode (IPC + MCP only, no MAP transport).
-
-When connected, incoming MAP messages are:
-1. Stored via `storage.putMessage()`
-2. Processed by traceability layer
-3. Written to `inbox.jsonl` for Claude context injection (same format as sidecar)
-
-**Tests**: Mock MAP connection, message receive flow, standalone mode.
-
-### Step 8: MCP Tools Server (`src/mcp/mcp-server.ts`)
-
-MCP server exposing agent-facing tools via `@modelcontextprotocol/sdk`.
-
-**Phase 1 tools**:
-
-| Tool | Parameters | Returns |
-|------|-----------|---------|
-| `send_message` | `to, body?, content?, from?, threadTag?, inReplyTo?, importance?, subject?` | `{ ok, messageId }` |
-| `check_inbox` | `agentId, unreadOnly?, limit?` | `{ count, messages: [...] }` |
-| `read_thread` | `threadTag, scope?` | `{ threadTag, count, messages: [...] }` |
-| `list_agents` | `scope?, includeFederated?` | `{ count, agents: [...] }` |
-
-Registration is handled automatically (auto-register on first `check_inbox`). Replies use `send_message` with `inReplyTo`. Acknowledgement is implicit on read. Each tool delegates to the message router and storage. The MCP server runs as a stdio transport (standard for Claude Code MCP tools).
-
-**Tests**: Tool invocation, parameter validation, response format.
-
-### Step 9: Entry Point (`src/index.ts`)
-
-Wire everything together:
-
-```typescript
-// 1. Create storage
-const storage = new InMemoryStorage();
-
-// 2. Create internal event emitter
-const events = new EventEmitter();
-
-// 3. Create traceability layer (subscribes to events)
-const traceability = new TraceabilityLayer(storage, events);
-
-// 4. Create message router
-const router = new MessageRouter(storage, events);
-
-// 5. Optionally connect to MAP server
-const mapClient = config.map?.server
-  ? await MapClient.connect(config.map.server, config.map)
-  : null;
-
-// 6. Start IPC server (UNIX socket)
-const ipcServer = new IpcServer(socketPath, router, storage, mapClient);
-await ipcServer.start();
-
-// 7. Start MCP server (stdio)
-const mcpServer = new McpServer(router, storage);
-await mcpServer.start();
-```
-
-### Step 10: Integration Testing
-
-End-to-end tests:
-1. **IPC flow**: Send command via UNIX socket → message stored → appears in inbox
-2. **MCP flow**: Call `send_message` tool → message routed → check_inbox returns it
-3. **MAP flow** (with mock server): Receive MAP message → stored in inbox → written to inbox.jsonl
-4. **Traceability**: Send messages with thread_tags → conversations auto-created
-5. **Lifecycle notifications**: Plugin sends spawn/done notify → agent registry updated
-6. **Graceful degradation**: MAP unavailable → IPC still works, MCP still works
-
-## Dependency Graph
-
-```
-types.ts ← storage/interface.ts ← storage/memory.ts
-                ↑
-types.ts ← router/message-router.ts (depends on storage, events)
-                ↑
-types.ts ← traceability/traceability.ts (depends on storage, events)
-                ↑
-           ipc/ipc-server.ts (depends on router, storage, map-client)
-           map/map-client.ts (depends on storage, events, MAP SDK)
-           mcp/mcp-server.ts (depends on router, storage)
-                ↑
-           index.ts (wires everything)
-```
-
-Build order: types → storage → router → traceability → ipc/map/mcp → index
-
-## Phase 2: Traceability + MAP Mail
-
-- SQLiteStorage implementation (persistent storage with FTS5)
-- MAP JSON-RPC endpoint (`mail/*` methods)
-- Full-text search via SQLite
-- `map/subscribe` real-time event streaming
-- Full traceability: conversations, turns, threads with auto-grouping
-
-## Phase 3: Federation + Multi-Server
-
-Federation uses MAP's native `federation/connect` and `federation/route` protocol. Agent Inbox acts as a **gateway agent** in MAP's federation model.
-
-### Project Structure Additions
-
-```
-agent-inbox/
-├── src/
-│   ├── federation/
-│   │   ├── connection-manager.ts   # Manage MAP + peer connections, federation/connect handshake
-│   │   ├── routing-engine.ts       # Configurable routing: table / broadcast / hierarchical
-│   │   ├── delivery-queue.ts       # Queue messages for offline peers (memory or SQLite)
-│   │   ├── address.ts              # Parse/resolve agent-id@system-id addresses
-│   │   └── trust.ts                # Trust policies: allow-list, scope perms, auth (stubs)
-│   │
-│   ├── registry/
-│   │   └── warm-registry.ts        # Agent registry with active/away/expired lifecycle + TTL
-│   │
-│   ├── map/
-│   │   └── map-client.ts           # Updated: system ID resolution from MAP systemInfo
-│   │
-│   └── types.ts                    # Updated: federation types
-│
-└── test/
-    ├── federation/
-    │   ├── connection-manager.test.ts
-    │   ├── routing-engine.test.ts
-    │   ├── delivery-queue.test.ts
-    │   ├── address.test.ts
-    │   └── trust.test.ts
-    └── registry/
-        └── warm-registry.test.ts
-```
-
-### Step 11: Core Types — Federation Extensions (`src/types.ts`)
-
-New types for federation:
-
-```typescript
-// System identity
-interface SystemId {
-  id: string;                          // resolved system ID
-  source: 'config' | 'map' | 'auto';  // how it was resolved
-}
-
-// Federated addressing
-interface FederatedAddress {
-  agent?: string;    // target agent ID
-  system?: string;   // target system ID
-  scope?: string;    // optional scope qualifier
-}
-
-// Agent registry with warm model
-type AgentStatus = 'active' | 'away' | 'expired';
-
-interface AgentRegistryConfig {
-  gracePeriodMs: number;        // disconnect → expired (default: 60000)
-  retainExpiredMs: number;      // keep expired entries (default: 3600000)
-  requeueOnReconnect: boolean;  // flush queued messages on reconnect (default: true)
-}
-
-// Federation peer configuration
-interface FederationPeerConfig {
-  systemId: string;              // human-friendly alias
-  url: string;                   // MAP server URL
-  auth?: FederationAuth;         // auth for federation/connect
-  exposure?: ExposurePolicy;     // what we expose to this peer
-}
-
-// Federation link state
-interface FederationLink {
-  peerId: string;
-  sessionId: string;
-  status: 'connected' | 'disconnected' | 'authenticating';
-  exposure: ExposurePolicy;
-}
-
-// Exposure control (MAP model)
-type ExposureLevel = 'none' | 'gateway' | 'tagged' | 'all';
-
-interface ExposurePolicy {
-  agents: ExposureLevel;
-  scopes?: string[];             // for 'tagged' level
-  events?: ExposureLevel;
-}
-
-// Routing configuration
-interface FederationRoutingConfig {
-  strategy: 'table' | 'broadcast' | 'hierarchical';
-  tableTTL?: number;             // routing entry TTL (default: 300000)
-  refreshOnMiss?: boolean;       // query peers on cache miss (default: true)
-  broadcastTimeout?: number;     // ms for broadcast strategy (default: 5000)
-  upstream?: string[];           // for hierarchical strategy
-}
-
-// Delivery queue configuration
-interface DeliveryQueueConfig {
-  persistence: 'memory' | 'sqlite';
-  maxTTL: number;                // default: 86400000 (24h)
-  maxQueueSize: number;          // per destination (default: 10000)
-  retryStrategy: 'exponential' | 'fixed';
-  retryBaseInterval: number;     // ms (default: 1000)
-  retryMaxAttempts: number;      // 0 = unlimited until TTL
-  flushOnReconnect: boolean;     // default: true
-  overflow: 'drop-oldest' | 'drop-newest' | 'reject-new';
-}
-
-// Trust policies
-interface FederationTrustPolicy {
-  allowedServers: string[];                     // implemented first
-  scopePermissions: Record<string, string[]>;   // stub
-  requireAuth: boolean;                         // stub
-  authMethod?: 'bearer' | 'api-key' | 'mtls' | 'did:wba';
-  authTokens?: Record<string, string>;          // stub
-}
-```
-
-### Step 12: Address Resolution (`src/federation/address.ts`)
-
-Parse and resolve federated addresses:
-
-- `parseAddress("agent-beta")` → `{ agent: "agent-beta" }` (local)
-- `parseAddress("agent-beta@backend-team")` → `{ agent: "agent-beta", system: "backend-team" }` (federated)
-- `parseAddress("@backend-team")` → `{ system: "backend-team" }` (broadcast to system)
-
-Resolution order:
-1. No `@` → local registry lookup (same scope first, then all scopes)
-2. With `@` → connection manager resolves system ID to peer connection
-3. With `to_scope` → scope-qualified lookup (local or remote)
-
-**Tests**: Parse formats, resolution order, unknown agent/system handling.
-
-### Step 13: Warm Agent Registry (`src/registry/warm-registry.ts`)
-
-Wraps the storage layer with TTL-based lifecycle:
-
-- `register(agentId, ...)` — set status `active`. Reject if ID taken by active/away agent (first-wins).
-- `disconnect(agentId)` — set status `away`, start grace period timer
-- `reconnect(agentId)` — set status back to `active`, cancel grace timer, flush queued messages
-- `expire(agentId)` — set status `expired` (called by timer after grace period)
-- `cleanup()` — remove entries past `retainExpiredMs`
-- `isRoutable(agentId)` → true for `active` or `away`
-- `getStatus(agentId)` → `active` | `away` | `expired` | `unknown`
-
-**Tests**: Lifecycle transitions, grace period expiry, reconnect flush, ID conflict rejection.
-
-### Step 14: Connection Manager (`src/federation/connection-manager.ts`)
-
-Manages MAP connections and federation peer links:
-
-- `connect(serverUrl, opts)` → connect to a MAP server (local agent traffic)
-- `federate(peer: FederationPeerConfig)` → establish `federation/connect` with auth + exposure exchange
-- `resolve(address: FederatedAddress)` → `{ connection, peerId }` or undefined
-- `route(message)` → determine correct connection + deliver via `federation/route`
-- `peers()` → list all federation links and their status
-- `disconnect(peerId)` → tear down a specific peer link
-
-System ID tiered resolution on startup:
-1. Explicit config (`INBOX_SYSTEM_ID`)
-2. Derived from MAP server `systemInfo.name` (from `map/connect` response)
-3. Auto-generated `inbox-{random}` (persisted to file for restart stability)
-
-**Tests**: Multi-connection management, federation handshake (mocked), peer lifecycle.
-
-### Step 15: Routing Engine (`src/federation/routing-engine.ts`)
-
-Configurable routing strategy:
-
-**Table strategy** (default):
-- Populate routing table from `federation/connect` exposure data
-- Entries: `{ agentId → peerId, lastSeen, status }`
-- TTL-based expiry
-- On cache miss + `refreshOnMiss`: query peers before failing
-
-**Broadcast strategy**:
-- Forward to all connected peers
-- First responder wins (within `broadcastTimeout`)
-
-**Hierarchical strategy**:
-- Check local routing table first
-- On miss, delegate to configured `upstream` hub(s)
-
-All strategies share the same interface: `resolveRoute(address) → peerId | null`
-
-**Tests**: Table population/expiry, broadcast fanout, hierarchical delegation, strategy switching.
-
-### Step 16: Delivery Queue (`src/federation/delivery-queue.ts`)
-
-Queue messages for offline/unreachable peers:
-
-- `enqueue(peerId, message)` — add to queue (check overflow policy)
-- `flush(peerId)` — drain queue on reconnect (if `flushOnReconnect`)
-- `tick()` — process retries (exponential or fixed backoff), expire past TTL
-- `size(peerId)` → current queue depth
-
-Supports `memory` or `sqlite` persistence backends.
-
-**Tests**: Enqueue/flush, TTL expiry, overflow policies, retry backoff timing.
-
-### Step 17: Trust Policies (`src/federation/trust.ts`)
-
-Layered trust enforcement:
-
-- **Layer 1 (implemented)**: `allowedServers` — reject `federation/connect` from unknown systems
-- **Layer 2 (stub)**: `scopePermissions` — always allow (returns true)
-- **Layer 3 (stub)**: `requireAuth` — always pass (returns true)
-
-Interface:
-- `canConnect(systemId)` → boolean
-- `canRoute(fromSystem, toScope)` → boolean
-- `validateAuth(systemId, credentials)` → boolean
-
-**Tests**: Allow-list enforcement, stub passthrough behavior.
-
-### Step 18: Integration — Wire Federation into Existing Code
-
-Update existing components:
-
-**`src/router/message-router.ts`**:
-- `resolveRecipients()` now calls `parseAddress()` for `@system` format
-- `routeMessage()` checks `isLocal()` first, then delegates to connection manager for remote
-- Remote delivery goes through routing engine → connection manager → `federation/route`
-
-**`src/map/map-client.ts`**:
-- Extract `systemInfo.name` from `map/connect` response for system ID resolution
-- Support multiple connections (local MAP + federation peers)
-
-**`src/index.ts`**:
-- Load federation config from env/config file
-- Initialize warm registry, connection manager, routing engine, delivery queue, trust policies
-- Wire into existing message router
-
-**`src/mcp/mcp-server.ts`**:
-- `send_message` `to` parameter accepts `agent@system` format
-- `list_agents` can optionally include federated agents (from routing table)
-
-### Step 19: Federation Integration Testing
-
-End-to-end tests:
-
-1. **Federated send**: Agent A on system 1 → `send_message(to="AgentC@system2")` → routed via federation/route → delivered
-2. **Hub relay**: System A → Hub → System B routing
-3. **Warm registry**: Agent disconnects → messages queue → agent reconnects → messages flush
-4. **Offline peer**: Peer disconnects → messages queue → peer reconnects → queue flushes
-5. **Trust enforcement**: Unknown system tries to federate → rejected by allow-list
-6. **Routing strategies**: Table miss + refresh, broadcast fanout, hierarchical delegation
-7. **Address parsing**: All address formats resolve correctly
-8. **System ID resolution**: Config > MAP systemInfo > auto-generated, persisted across restarts
-
-### Step 20: ConnectionManager Transport Layer
-
-Wire real MAP SDK connections into ConnectionManager so federation routing
-actually sends/receives messages over the wire.
-
-**Design decisions:**
-- **SDK injection (option B)**: `index.ts` imports the MAP SDK once (via MapClient's
-  existing dynamic import path), then passes the `AgentConnection` class to
-  ConnectionManager. No duplicate dynamic imports.
-- **Incoming messages delegate to router**: When a federated peer sends a message,
-  ConnectionManager receives it via `conn.onMessage()` and delegates to
-  `MessageRouter.routeMessage()` to store and distribute locally. Clean boundary.
-- **MapClient stays single-connection**: Only handles the local MAP server.
-  ConnectionManager owns all federation peer connections.
-
-**Changes:**
-
-**`src/federation/connection-manager.ts`**:
-- New constructor param: `sdkConnectFn` — `(url, opts) => Promise<MapConnection>`
-  (the bound `AgentConnection.connect` from the SDK). Optional — when null,
-  federation stays event-based (no transport, for testing).
-- New private field: `connections: Map<string, MapConnection>` — real SDK connections
-  per federated peer.
-- `federate()` updated: after trust check + creating FederationLink, calls
-  `sdkConnectFn(peer.url, { name: systemId, role: "gateway", ... })` to open
-  a real connection. Registers `conn.onMessage()` handler that delegates to
-  the injected message handler callback.
-- `route()` updated: instead of emitting `federation.route` event, calls
-  `this.connections.get(peerId).send(target, payload)`.
-- `disconnect()` updated: calls `conn.disconnect()` on the real connection.
-- New constructor param: `onIncomingMessage` callback — called when a peer
-  connection receives a message. `index.ts` wires this to `router.routeMessage()`.
-
-**`src/index.ts`**:
-- After MapClient successfully imports the SDK, extract `AgentConnection.connect`
-  and pass it to ConnectionManager.
-- Pass `onIncomingMessage` callback that calls `router.routeMessage()` with the
-  incoming message data, setting the sender to `sender@peerId` format.
-
-**`src/map/map-client.ts`**:
-- Expose `getAgentConnectionClass()` — returns the dynamically imported
-  `AgentConnection` class (or undefined if SDK not available). Called by
-  `index.ts` to inject into ConnectionManager.
-
-**Testing:**
-- Unit tests: mock the `sdkConnectFn` to return a fake connection object.
-  Verify `federate()` opens connections, `route()` calls `conn.send()`,
-  incoming messages are forwarded via callback.
-- Integration tests: extend existing suite with transport-level scenarios.
-
-## Deferred (Future)
-
-- **agentic-mesh integration**: Optional transport layer alternative to direct MAP federation. Kept in plans but not in Phase 3 scope.
-- **Federation discovery**: Automatic peer discovery via `.well-known` or `did:wba`. Phase 3 uses explicit config only.
-- **Transitive routing**: System A → Hub → System B (hub relays without explicit peer link between A and B). Partially supported by hierarchical routing strategy but trust implications need design.