@hienlh/ppm 0.8.87 → 0.8.89

package/docs/streaming-input-guide.md

@@ -0,0 +1,267 @@
+# Streaming Input Migration Quick Reference (v0.8.55+)
+
+## What Changed?
+
+**Before (v0.8.54):** Each message triggered a new SDK query
+```
+Message 1 → SDK subprocess spawn → generate response → close
+Message 2 → SDK subprocess spawn → generate response → close
+(Slow, context resets between messages)
+```
+
+**After (v0.8.55):** Single persistent streaming session
+```
+Session created → AsyncGenerator streaming input opened
+Message 1 → Push into generator → process events
+Message 2 → Push into same generator → continue streaming
+(Fast, continuous context, no SDK restarts)
+```
+
+## Key Concepts
+
+### Session State (BE-Owned)
+The backend maintains a `SessionEntry` per chat session:
+- Tracks connected clients (can be zero if FE disconnected)
+- Maintains streaming phase (idle, connecting, thinking, streaming)
+- Buffers events for reconnection sync
+- Auto-cleans after 5 minutes of FE inactivity
+
+### Message Priority (v0.8.55+)
+```typescript
+// Send message with priority
+ws.send({
+  type: "message",
+  content: "Debug this code",
+  priority: "now"  // "now" | "next" | "later"
+})
+```
+- **"now"** — Abort current query, restart with this message
+- **"next"** — Queue after current, run next
+- **"later"** — Append to queue, run last
+
+### Event Buffering on Reconnect
+When FE WS reconnects after disconnect:
+1. BE sends `session_state` with current phase + pending approval
+2. BE sends `turn_events` with all buffered events since last connection
+3. FE rebuilds chat UI state from buffered events
+4. No message loss (unless session cleaned up after 5min)
+
+## Common Patterns
+
+### Frontend: Send Message
+```typescript
+// In useChat hook or message input handler
+ws.send(JSON.stringify({
+  type: "message",
+  content: userInput,
+  priority: "now",  // Optional
+  images: [{ id: "img1", data: "base64..." }]  // Optional
+}));
+```
+
+### Frontend: Handle Reconnection
+```typescript
+function handleReconnect() {
+  // 1. WS open fires
+  // 2. Server sends session_state
+  const sessionState = JSON.parse(msg);
+  // 3. Server sends turn_events
+  const turnEvents = JSON.parse(msg);
+
+  // 4. FE rebuilds state from buffered events
+  turnEvents.events.forEach(event => {
+    chatStore.addEvent(event);
+  });
+
+  // 5. FE is now synced with BE
+}
+```
+
+### Backend: Session Lifecycle
+```typescript
+// 1. FE connects
+open(ws) {
+  const entry = activeSessions.get(sessionId);
+  if (!entry) {
+    // Create new session entry
+    activeSessions.set(sessionId, {
+      phase: "idle",
+      clients: new Set([ws]),
+      turnEvents: []
+    });
+  } else {
+    // Reconnect: clear cleanup timer, add client
+    entry.clients.add(ws);
+  }
+}
+
+// 2. FE sends message
+message(ws, data) {
+  const parsed = JSON.parse(data);
+  if (parsed.type === "message") {
+    // Abort current if streaming, wait for cleanup
+    if (entry.phase !== "idle") {
+      entry.abort.abort();
+      await entry.streamPromise;
+    }
+    // Start new streaming loop (detached)
+    entry.streamPromise = runStreamLoop(...);
+  }
+}
+
+// 3. Streaming loop runs independently
+async function runStreamLoop() {
+  for await (const event of chatService.sendMessage(...)) {
+    bufferAndBroadcast(sessionId, event);  // To all connected clients
+  }
+  setPhase(sessionId, "idle");  // Back to idle when done
+  if (entry.clients.size === 0) {
+    startCleanupTimer(sessionId);  // 5-min cleanup
+  }
+}
+
+// 4. FE disconnects
+close(ws) {
+  entry.clients.delete(ws);
+  // Stream continues! (BE owns the connection)
+  // Timer started if no more clients
+}
+```
+
+## Phase State Machine
+
+```
+     ┌─ initializing  (setup, session resume)
+     ↓
+  idle ←→ connecting  (waiting for first SDK event, heartbeat)
+     ↑         ↓
+     │    ┌──→ thinking  (extended thinking)
+     │    ↓  ↓
+     └─── streaming  (text/tool_use content)
+          ↑     ↓
+          └─────┘ (dynamic switch)
+```
+
+**Transitions:**
+- Heartbeat: `connecting` → (5s elapsed updates) → `thinking` (when content arrives)
+- Content: `thinking` → `streaming` (first text event)
+- Dynamic: `streaming` ↔ `thinking` (based on event types)
+- Done: Any → `idle` (stream complete, ready for next message)
+
+## WebSocket Messages (v0.8.55+)
+
+### Client → Server
+```typescript
+// Send message
+{ type: "message"; content: string; priority?: string; images?: {...}[] }
+
+// Approve tool
+{ type: "approval_response"; requestId: string; approved: boolean }
+
+// Cancel current
+{ type: "cancel" }
+
+// Handshake after open
+{ type: "ready" }
+```
+
+### Server → Client
+```typescript
+// Content
+{ type: "text"; content: string }
+{ type: "thinking"; content: string }
+
+// Tool execution
+{ type: "tool_use"; tool: string; input: unknown }
+{ type: "tool_result"; output: string; isError?: boolean }
+
+// User approval request
+{ type: "approval_request"; requestId: string; tool: string; input: unknown }
+
+// Session state (sent on open/ready)
+{ type: "session_state"; sessionId: string; phase: SessionPhase; pendingApproval: {...} | null }
+
+// Buffered events (on reconnect)
+{ type: "turn_events"; events: unknown[] }
+
+// Metadata
+{ type: "account_info"; accountId: string; accountLabel: string }
+{ type: "phase_changed"; phase: SessionPhase; elapsed?: number }
+{ type: "title_updated"; title: string }
+
+// Completion
+{ type: "done"; sessionId: string; contextWindowPct?: number }
+
+// Error
+{ type: "error"; message: string }
+
+// Keepalive
+{ type: "ping" }
+```
+
+## Benefits
+
+| Aspect | Before (v0.8.54) | After (v0.8.55) |
+|--------|------------------|-----------------|
+| **SDK Restarts** | Per message | Once per session |
+| **Context** | Resets between messages | Persistent |
+| **Startup Time** | 2-5s per message | Instant follow-ups |
+| **Reconnection** | Message loss | Event buffering ensures sync |
+| **Concurrency** | N/A | Multiple clients per session |
+| **Tool Approvals** | Restarts query | Integrated in stream |
+
+## Troubleshooting
+
+### Session Cleaned Up (No Longer Exists)
+**Cause:** FE disconnected for >5 minutes
+**Solution:** Create new session, FE reconnects with new sessionId
+
+### Events Missing After Reconnect
+**Cause:** Server-side event buffer (10k event limit) overflowed
+**Solution:** Flush buffer periodically or increase limit if needed
+
+### Phase Stuck in "Connecting"
+**Cause:** SDK subprocess not responding (120s timeout)
+**Solution:** Check environment (ANTHROPIC_API_KEY, network), see error message for hints
+
+### Multiple Clients Out of Sync
+**Cause:** Broadcast failed for one client, others ahead
+**Solution:** Evicted client will reconnect and re-sync from buffered events
+
+## Debugging
+
+### Enable Logging
+```bash
+# Check server logs for session lifecycle
+[chat] session=abc123 phase → connecting
+[chat] session=abc123 first SDK event after 1250ms: type=text
+[chat] session=abc123 stream completed (45 events)
+[chat] session=abc123 phase → idle
+```
+
+### Check Session State
+```typescript
+// On WS message handler
+console.log(`Session entry:`, activeSessions.get(sessionId));
+// Outputs: { phase, clients.size, pendingApprovalEvent, turnEvents.length }
+```
+
+### Monitor Reconnections
+```typescript
+// In WS open handler
+console.log(`FE reconnected (phase=${existing.phase}, clients=${existing.clients.size})`);
+// Tells you: active streaming, how many clients connected
+```
+
+## Performance Notes
+
+- **No SDK overhead:** Persistent streaming eliminates subprocess spawn overhead
+- **Event buffering:** Clients see all events after reconnect (max 10k events per turn)
+- **Memory:** Session entries cleaned after 5min (bounded memory usage)
+- **Latency:** Follow-up messages start immediately (no SDK init)
+
+---
+
+**For detailed architecture:** See `docs/system-architecture.md` → "Chat Streaming Flow" section
+**For API types:** See `src/types/api.ts` and `src/types/chat.ts`
+**For implementation:** See `src/server/ws/chat.ts` and `src/providers/claude-agent-sdk.ts`

package/docs/system-architecture.md

@@ -977,6 +977,157 @@ ppm db data <name> <table>   # Show table data (paginated)
 
 ---
 
+## MCP Server Management
+
+### Overview
+MCP (Model Context Protocol) servers extend Claude with custom tools and resources. PPM manages MCP server configurations via Settings UI, storing them in SQLite and passing them to the Claude Agent SDK.
+
+**Features:**
+- **Add/Edit/Delete** MCP servers via Settings UI
+- **Auto-import** from `~/.claude.json` on first access (convenience, no forced import)
+- **Three transport types:** stdio, HTTP, SSE
+- **Validation** on name and config before storage
+- **SDK integration:** Servers passed to `query()` as `mcpServers` object, tools auto-allowed via `mcp__*` wildcard
+
+### Storage Schema
+
+```sql
+CREATE TABLE mcp_servers (
+  name TEXT PRIMARY KEY,
+  transport TEXT NOT NULL DEFAULT 'stdio',  -- 'stdio' | 'http' | 'sse'
+  config TEXT NOT NULL,                     -- JSON: McpServerConfig
+  created_at TEXT DEFAULT (datetime('now')),
+  updated_at TEXT DEFAULT (datetime('now'))
+);
+```
+
+**Config Format (JSON):**
+```json
+{
+  "type": "stdio",
+  "command": "path/to/server",
+  "args": ["--flag"],
+  "env": { "VAR": "value" }
+}
+```
+
+Or HTTP/SSE:
+```json
+{
+  "type": "http",
+  "url": "http://localhost:3000",
+  "headers": { "Authorization": "Bearer token" }
+}
+```
+
+### REST API
+
+**Endpoints** (`src/server/routes/mcp.ts`):
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| **GET** | `/api/settings/mcp` | List all servers; auto-import on first access |
+| **GET** | `/api/settings/mcp/:name` | Get single server config |
+| **POST** | `/api/settings/mcp` | Add new server (validates name + config) |
+| **PUT** | `/api/settings/mcp/:name` | Update existing server |
+| **DELETE** | `/api/settings/mcp/:name` | Remove server |
+| **GET** | `/api/settings/mcp/import/preview` | Preview servers in `~/.claude.json` |
+| **POST** | `/api/settings/mcp/import` | Bulk import from `~/.claude.json` |
+
+**Add Server Example:**
+```bash
+POST /api/settings/mcp
+Content-Type: application/json
+
+{
+  "name": "file-server",
+  "config": {
+    "type": "stdio",
+    "command": "/usr/local/bin/file-server",
+    "args": ["--port", "8000"]
+  }
+}
+```
+
+### Service Layer
+
+**McpConfigService** (`src/services/mcp-config.service.ts`):
+- `list()` — Record<name, McpServerConfig> (SDK-compatible format)
+- `listWithMeta()` — Array with metadata (for UI)
+- `get(name)` — Single server config
+- `set(name, config)` — Add or update (upsert)
+- `remove(name)` — Delete server
+- `exists(name)` — Check if name exists
+- `bulkImport(servers)` — Transactional import from `~/.claude.json`, skips existing/invalid
+
+**Validation:**
+- `validateMcpName(name)` — alphanumeric + hyphens/underscores, max 50 chars
+- `validateMcpConfig(config)` — type-specific checks (command for stdio, url for http/sse)
+
+### Frontend Integration
+
+**UI Components:**
+- `MCP Settings Section` (`src/web/components/settings/mcp-settings-section.tsx`) — Tab in Settings UI
+- `MCP Server Dialog` (`src/web/components/settings/mcp-server-dialog.tsx`) — Add/Edit modal
+- `API client` (`src/web/lib/api-mcp.ts`) — Fetch/mutate operations
+
+**Workflow:**
+1. User opens Settings → MCP tab
+2. **GET** `/api/settings/mcp` (auto-imports on first access)
+3. Display list with transport badge + actions (edit, delete)
+4. Click "Add" → Dialog with name + transport selector + config fields
+5. **POST** to `/api/settings/mcp` or **PUT** to update
+6. On success, list refreshes
+
+### SDK Integration
+
+**Claude Agent SDK Provider** (`src/providers/claude-agent-sdk.ts`):
+```typescript
+// Line ~574
+const mcpServers = mcpConfigService.list();
+const hasMcp = Object.keys(mcpServers).length > 0;
+
+// Line ~589: Pass to query() if servers exist
+const mcpTools = ["mcp__*"];
+const queryConfig = {
+  // ... other options
+  ...(hasMcp && { mcpServers }),
+  allowedTools: [...otherTools, ...mcpTools],
+};
+
+const query = new Query(messages, queryConfig);
+```
+
+**Tool Allow List:**
+- All MCP tools automatically allowed via wildcard `mcp__*`
+- MCP server connection failures don't block chat (logged as warning)
+
+### Import Flow
+
+**Auto-import on first access:**
+1. GET `/api/settings/mcp` called
+2. If table is empty, read `~/.claude.json`
+3. If `mcpServers` key exists, bulk import (validate + skip duplicates)
+4. Return populated list
+
+**Manual import:**
+1. GET `/api/settings/mcp/import/preview` — show what's available
+2. POST `/api/settings/mcp/import` — import validated servers
+3. Returns `{ imported: N, skipped: M }`
+
+### Error Handling
+
+| Scenario | Response |
+|----------|----------|
+| Invalid name (non-alphanumeric) | 400 Bad Request |
+| Invalid config (missing required fields) | 400 Bad Request |
+| Duplicate name | 409 Conflict |
+| Server not found (GET/:name, PUT/:name, DELETE/:name) | 404 Not Found |
+| `~/.claude.json` not found (import) | 404 Not Found |
+| Corrupt config JSON (recovery) | Log warning, skip entry, continue |
+
+---
+
 ## Deployment Architecture
 
 ### Single-Machine Deployment (Current)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hienlh/ppm",
-  "version": "0.8.87",
+  "version": "0.8.89",
   "description": "Personal Project Manager — mobile-first web IDE with AI assistance",
   "author": "hienlh",
   "license": "MIT",