@forloop-cc/forloop-opencode-plugin-planner 2.2.1

package/docs/superpowers/specs/2026-05-03-sprint-isolated-directory-structure-design.md

@@ -0,0 +1,149 @@
+# Sprint-Isolated Directory Structure Design
+
+## Problem
+
+The ForLoop planner agent runs on both local machines and AWS Lambda. Currently, all sprint files (knowledge, plans, tasks) are stored in a flat `~/.forloop/` directory without sprint isolation. This causes file collisions when switching sprints and makes it impossible to have clean per-sprint working directories.
+
+## Requirement
+
+All sprint-specific files must be stored under `~/.forloop/sprint-{sprint_id}/`. The manifest stays at `~/.forloop/manifest.json` (shared across sprints) to track which sprint is currently active.
+
+## Target Directory Structure
+
+```
+/tmp/home/.forloop/
+├── manifest.json                    # Shared, tracks activeSprintId
+├── sync/
+│   └── manifest.json                # Sync tracking metadata
+└── sprint-14/                       # Sprint-specific root
+    ├── knowledge/
+    │   └── knowledge-auth-jwt-20260410-093015.md
+    ├── plan/
+    │   └── plan-14-20260410-093015.md
+    └── task/
+        └── task-14-20260410-100000.md
+```
+
+## Manifest Format (Version 2)
+
+```json
+{
+  "version": 2,
+  "activeSprintId": 14,
+  "sprints": {
+    "14": {
+      "sprintDir": "sprint-14",
+      "plan": {
+        "file": "sprint-14/plan/plan-14-20260410-093015.md",
+        "createdAt": "2026-04-10T09:30:15Z"
+      },
+      "tasks": {
+        "file": "sprint-14/task/task-14-20260410-100000.md",
+        "storyIds": ["123", "124", "125"]
+      }
+    }
+  }
+}
+```
+
+## Path Resolution Logic
+
+### Core Function: `resolveForloopRoot(contextDir, sprintId?)`
+
+```
+1. If sprintId provided or resolvable (env/branch/manifest):
+   → $HOME/.forloop/sprint-{sprintId}/
+
+2. If no sprintId available:
+   → $HOME/.forloop/ (root only, for manifest access)
+
+3. Manifest always at:
+   → $HOME/.forloop/manifest.json
+```
+
+## Files to Update
+
+### 1. `plugins/capabilities/context-resolver.ts`
+- Add `getSprintDir(sprintId)` → returns `"sprint-{id}"`
+- Add `resolveSprintRoot(sprintId?)` → returns `$HOME/.forloop/sprint-{id}/` or `$HOME/.forloop/`
+- `getActiveSprintIdFromManifest()` stays unchanged (reads root manifest)
+- Export new functions for use by sync-tools and skills
+
+### 2. `plugins/tools/sync-tools.ts`
+- `resolveForloopRoot()`: Use sprint-aware resolution. Falls back to `$HOME/.forloop/` if no sprint
+- `ensureForloopStructure()`: Create `sprint-{id}/knowledge/plan/task/` instead of flat dirs
+- `getSyncManifestPath()`: Move to `$HOME/.forloop/sync/manifest.json` (shared, not per-sprint)
+- File mapping: `knowledge/X.md` → `sprint-{id}/knowledge/X.md`
+
+### 3. `plugins/hooks/shell-env.ts`
+- No changes needed (doesn't reference .forloop paths directly)
+
+### 4. `agent/forLoopPlanner.md`
+- Update all path references:
+  - `~/.forloop/plan/` → `~/.forloop/sprint-{id}/plan/`
+  - `~/.forloop/knowledge/` → `~/.forloop/sprint-{id}/knowledge/`
+  - `~/.forloop/task/` → `~/.forloop/sprint-{id}/task/`
+  - `~/.forloop/manifest.json` → stays (shared)
+- Update permission block: `/tmp/home/.forloop/sprint-*/**`
+- Update examples and workflows to use sprint subdir
+
+### 5. `agent/forLoopStoryEvaluator.md`
+- Update permission block to include sprint subdirs
+- No functional changes (evaluator doesn't write files)
+
+### 6. `skills/forloop-context/SKILL.md`
+- Step 1: Resolve to `~/.forloop/sprint-{id}/` after sprint is known
+- Step 2: Create `sprint-{id}/` structure instead of flat `.forloop/knowledge/plan/task/`
+- Update all example paths and commands
+- Update manifest examples to v2 format
+
+### 7. `skills/knowledge-management/SKILL.md`
+- `.forloop/knowledge/` → `.forloop/sprint-{id}/knowledge/`
+- Update file creation paths and S3 upload examples
+- Update quality gates and anti-patterns
+
+### 8. `skills/plan-documentation/SKILL.md`
+- `.forloop/plan/` → `.forloop/sprint-{id}/plan/`
+- Update mkdir, file write, manifest examples
+- Update all workflow examples
+
+### 9. `skills/task-tracking/SKILL.md`
+- `.forloop/task/` → `.forloop/sprint-{id}/task/`
+- Update file write, manifest examples
+- Update S3 folder mapping section (local paths change, S3 paths stay the same)
+
+### 10. `skills/aivy-documents-sync/SKILL.md`
+- Update all path references to sprint subdirs
+- Update sync examples
+
+### 11. `skills/sprint-planning/SKILL.md`
+- Update context loading paths
+- Update manifest and file path references
+
+### 12. Tests
+- All unit tests with path assertions need updating
+- Mock file paths should use sprint subdir format
+- Integration tests verify correct directory creation
+
+## Backward Compatibility
+
+- On first access, check if old flat structure exists (`~/.forloop/knowledge/`, `~/.forloop/plan/`, etc.)
+- If old structure exists:
+  - If manifest has an active sprint, create `sprint-{id}/` and move old files into it
+  - If no active sprint, leave old files in place; they will be orphaned when new sprint is selected
+- Manifest version bump from 1 → 2 signals new structure
+- The `sprintDir` field is added to manifest sprint entries; missing `sprintDir` implies v1 (flat structure) and triggers migration
+
+## Error Handling
+
+- No sprint ID available: agent works at `~/.forloop/` root only (manifest access)
+- Sprint directory missing: auto-create on first file operation
+- Old v1 manifest: upgrade to v2 by adding `sprintDir` field, preserve existing data, migrate files if present
+- Corrupted sprint dir name in manifest: fall back to `sprint-{activeSprintId}` convention
+
+## Testing Strategy
+
+1. Unit tests: Verify `resolveSprintRoot()` with various inputs
+2. Unit tests: Verify `ensureForloopStructure()` creates correct dirs
+3. Integration: Create sprint, verify directory structure
+4. Integration: Switch sprints, verify isolation

package/docs/superpowers/specs/2026-05-19-opencode-message-recording-design.md

@@ -0,0 +1,468 @@
+# OpenCode Message Recording Feature
+
+**Date:** 2026-05-19
+**Status:** Design
+**Version:** 1.0
+
+## Overview
+
+Record the full opencode conversation (user messages + AI responses) to the ForLoop database. User messages are captured via the `chat.message` hook, assistant responses are assembled from streaming `message.part.updated` events, and deletions are handled via `message.removed`. Recording is active only when a working sprint is confirmed via `~/.forloop/manifest.json`.
+
+## Motivation
+
+The ForLoop web client already records messages from the `planner.chat` workflow. However, conversations happening inside the opencode IDE plugin are not captured. This feature bridges that gap, ensuring all sprint-related conversations — whether from the web client or the opencode IDE — are stored in one place for future features like conversation summarization.
+
+## Architecture
+
+```
+opencode runtime
+    │
+    ├── "chat.message" hook ──────────┐   (user messages with full text via parts)
+    │                                  │
+    ├── event: "message.part.updated" ─┤   (assistant text parts stream)
+    ├── event: "message.updated" ──────┤   (signal: assistant message complete)
+    ├── event: "message.removed" ──────┤   (message deletion)
+    │                                  ▼
+    │                    plugins/hooks/message-recorder.ts
+    │                              │
+    │                    reads ~/.forloop/manifest.json
+    │                              │
+    │                    if activeSprintId exists:
+    │                              │
+    │                    ForLoopAPIClient.recordMessage()
+    │                    ForLoopAPIClient.removeMessage()
+    │                              │
+    │                              ▼
+    │              POST /api/opencode/messages
+    │              DELETE /api/opencode/messages/:messageId
+    │                              │
+    └──────────────────────────────┘
+                                   ▼
+                          server_lambda (ForLoop API)
+                                   │
+                          Conversation table (Prisma)
+```
+
+### Hook Responsibilities
+
+| Hook | Event | Action |
+|------|-------|--------|
+| `chat.message` | User sends a message | Extract text from `output.parts`, POST user message row |
+| `event` | `message.part.updated` (type="text") | Buffer text by messageID into `textBuffer` map |
+| `event` | `message.updated` (role="assistant") | Flush buffered text, POST assistant response update |
+| `event` | `message.removed` | DELETE row from Conversation table |
+
+### Why Not `message.updated` Alone for Text?
+
+The `EventMessageUpdated` event provides a `Message` info object (UserMessage | AssistantMessage), which contains metadata (id, role, sessionID, agent, model, time) but does **not** include the message text body. Text content flows through `TextPart` objects delivered via `message.part.updated` events. For user messages, the `chat.message` hook provides parts directly. For assistant responses, we buffer streaming text parts until the message is complete.
+
+## Data Model
+
+### Reusing the Conversation Table
+
+The existing `Conversation` table stores paired user message + agent response:
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `id` | Int (PK, auto) | Primary key |
+| `sprintId` | Int | Sprint context |
+| `conversationId` | String | Composite ID for grouping |
+| `timestamp` | DateTime | When recorded |
+| `userMessage` | String | User's message |
+| `agentResponse` | String | Agent's response |
+| `metadata` | String? | JSON blob |
+
+Message flow mapped to the paired model:
+
+1. **User message** (`chat.message` hook) → INSERT new row with `userMessage` filled, `agentResponse` empty
+2. **Assistant parts** (`message.part.updated` events) → buffered in-memory by messageID
+3. **Assistant complete** (`message.updated` with role "assistant") → UPDATE last row for `conversationId`, setting `agentResponse` from buffer
+4. **Message removed** (`message.removed`) → DELETE the row for that messageID
+
+### conversationId Format
+
+Consistent `sprint:{id}:agent:{agent}:` prefix with source-identifying suffix:
+
+| Source | Format | Example |
+|--------|--------|---------|
+| Web client (existing) | `sprint:{id}:agent:{agent}:web:user:{userId}` | `sprint:14:agent:forLoopPlanner:web:user:3` |
+| opencode plugin (new) | `sprint:{id}:agent:{agent}:opencode:{sessionId}` | `sprint:14:agent:forLoopPlanner:opencode:ses_abc123` |
+
+Agent keys are normalized to lowercase-kebab casing (e.g., `"forLoopPlanner"` → `"forLoopPlanner"`) to match the web client convention and enable cross-source queries.
+
+### metadata Structure
+
+```json
+{
+  "source": "opencode",
+  "model": { "providerID": "anthropic", "modelID": "claude-sonnet-4" },
+  "messageId": "msg_xyz",
+  "sessionId": "ses_abc123",
+  "recordedAt": 1712345678000
+}
+```
+
+## API Contract
+
+### POST `/api/opencode/messages`
+
+Auth: `x-api-token` header with `agent:query` scope.
+
+**Request:**
+```json
+{
+  "sprintId": 14,
+  "conversationId": "sprint:14:agent:forLoopPlanner:opencode:ses_abc123",
+  "messageId": "msg_xyz",
+  "role": "user",
+  "content": "Help me plan sprint 14",
+  "agent": "forLoopPlanner",
+  "model": { "providerID": "anthropic", "modelID": "claude-sonnet-4" },
+  "timestamp": 1712345678000
+}
+```
+
+**Fields:**
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `sprintId` | number | Yes | Sprint context |
+| `conversationId` | string | Yes | Composite conversation identifier |
+| `messageId` | string | Yes | opencode message ID |
+| `role` | `"user"` \| `"assistant"` | Yes | Message role |
+| `content` | string | Yes | Full message text |
+| `agent` | string | No | opencode agent name (lowercase-kebab) |
+| `model` | object | No | `{ providerID, modelID }` |
+| `timestamp` | number | No | Unix ms timestamp |
+
+**Response (200):**
+```json
+{ "status": "ok", "id": 42 }
+```
+
+**Behavior by role:**
+
+| Role | Action |
+|------|--------|
+| `"user"` | INSERT new row: `conversationId`, `sprintId`, `userMessage`, `metadata` |
+| `"assistant"` | UPDATE last row for `conversationId`: set `agentResponse`, update `metadata` |
+
+### DELETE `/api/opencode/messages/:messageId`
+
+Auth: `x-api-token` header with `agent:query` scope.
+
+**Request body:**
+```json
+{
+  "sprintId": 14,
+  "sessionId": "ses_abc123"
+}
+```
+
+**Response (200):**
+```json
+{ "status": "ok" }
+```
+
+## Plugin Implementation
+
+### New File: `plugins/hooks/message-recorder.ts`
+
+Two exports:
+
+1. `createChatMessageHook(client)` — the `chat.message` hook for user messages
+2. `createEventHook(client)` — the `event` hook for parts/removals
+
+```typescript
+import { ForLoopAPIClient } from '../capabilities/api-client';
+import fs from 'fs';
+import path from 'path';
+
+// Normalize agent key: "forLoopPlanner" -> "forLoopPlanner"
+function normalizeAgentKey(agent: string): string {
+  return agent
+    .replace(/([a-z])([A-Z])/g, '$1-$2')
+    .replace(/([A-Z]+)([A-Z][a-z])/g, '$1-$2')
+    .toLowerCase();
+}
+
+function buildConversationId(sprintId: number, agent: string, sessionId: string): string {
+  return `sprint:${sprintId}:agent:${normalizeAgentKey(agent)}:opencode:${sessionId}`;
+}
+
+function readActiveSprintId(): number | null {
+  try {
+    const home = process.env.HOME || '/tmp/home';
+    const manifestPath = path.join(home, '.forloop', 'manifest.json');
+    if (!fs.existsSync(manifestPath)) return null;
+    const raw = fs.readFileSync(manifestPath, 'utf8');
+    const parsed = JSON.parse(raw);
+    const id = parsed?.activeSprintId;
+    return Number.isFinite(id) ? Number(id) : null;
+  } catch {
+    return null;
+  }
+}
+
+// Extract text from parts array (from chat.message hook)
+function extractTextFromParts(parts: any[]): string {
+  return parts
+    .filter((p: any) => p.type === 'text')
+    .map((p: any) => p.text || '')
+    .join('');
+}
+
+// ─── chat.message hook: captures user messages ───
+
+export function createChatMessageHook(client: ForLoopAPIClient) {
+  return async (input: any, output: any) => {
+    const sprintId = readActiveSprintId();
+    if (!sprintId) return;
+
+    const { sessionID, agent, model, messageID } = input;
+    const { message, parts } = output;
+    const content = extractTextFromParts(parts);
+    if (!content) return;
+
+    const conversationId = buildConversationId(sprintId, agent || 'unknown', sessionID);
+
+    try {
+      await client.recordMessage({
+        sprintId,
+        sessionId: sessionID,
+        messageId: messageID || message.id,
+        conversationId,
+        role: 'user',
+        content,
+        agent: agent ? normalizeAgentKey(agent) : undefined,
+        model,
+        timestamp: message.time?.created ?? Date.now(),
+      });
+    } catch (err) {
+      console.warn('[ForLoop] Failed to record user message:', (err as Error).message);
+    }
+  };
+}
+
+// ─── event hook: captures assistant text + removals ───
+
+export function createEventHook(client: ForLoopAPIClient) {
+  const textBuffer = new Map<string, string>(); // messageID -> accumulated text
+
+  return async ({ event }: { event: { type: string; properties: any } }) => {
+    const sprintId = readActiveSprintId();
+    if (!sprintId) return;
+
+    switch (event.type) {
+
+      case 'message.part.updated': {
+        const part = event.properties?.part;
+        if (!part || part.type !== 'text') return;
+
+        const current = textBuffer.get(part.messageID) || '';
+        // part.text is the full accumulated text for streaming parts
+        textBuffer.set(part.messageID, part.text || current);
+        break;
+      }
+
+      case 'message.updated': {
+        const info = event.properties?.info;
+        if (!info || info.role !== 'assistant') return;
+
+        const bufferedText = textBuffer.get(info.id);
+        if (!bufferedText) return;
+        // NOTE: do NOT delete from textBuffer — parts may still arrive after
+        // this event. Each subsequent message.updated will flush again with
+        // the latest text. Buffer is cleaned up on message.removed only.
+
+        const conversationId = buildConversationId(
+          sprintId,
+          info.agent || 'unknown',
+          info.sessionID
+        );
+
+        try {
+          await client.recordMessage({
+            sprintId,
+            sessionId: info.sessionID,
+            messageId: info.id,
+            conversationId,
+            role: 'assistant',
+            content: bufferedText,
+            agent: info.agent ? normalizeAgentKey(info.agent) : undefined,
+            model: info.model,
+            timestamp: info.time?.created ?? Date.now(),
+          });
+        } catch (err) {
+          console.warn('[ForLoop] Failed to record assistant message:', (err as Error).message);
+        }
+        break;
+      }
+
+      case 'message.removed': {
+        const { sessionID, messageID } = event.properties || {};
+        if (!sessionID || !messageID) return;
+
+        textBuffer.delete(messageID); // clean up buffer too
+
+        try {
+          await client.removeMessage(messageID, sprintId, sessionID);
+        } catch (err) {
+          console.warn('[ForLoop] Failed to remove message:', (err as Error).message);
+        }
+        break;
+      }
+    }
+  };
+}
+```
+
+### Modified File: `plugins/forloop-plugin.ts`
+
+Wire both hooks:
+
+```diff
++ import { createChatMessageHook, createEventHook } from './hooks/message-recorder';
+
+  export default async (ctx: any) => {
+    // ... existing init ...
+
+    return {
+      tool: { ... },
++     event: createEventHook(client),
++     'chat.message': createChatMessageHook(client),
+      'shell.env': injectEnvironment,
+    };
+  };
+```
+
+### Modified File: `plugins/capabilities/api-client.ts`
+
+Two new methods on `ForLoopAPIClient`:
+
+```typescript
+async recordMessage(data: {
+  sprintId: number;
+  sessionId: string;
+  messageId: string;
+  conversationId: string;
+  role: 'user' | 'assistant';
+  content: string;
+  agent?: string;
+  model?: { providerID: string; modelID: string };
+  timestamp: number;
+}): Promise<any> {
+  return this.request('/api/opencode/messages', {
+    method: 'POST',
+    body: JSON.stringify(data),
+  });
+}
+
+async removeMessage(messageId: string, sprintId: number, sessionId: string): Promise<void> {
+  await this.request(`/api/opencode/messages/${encodeURIComponent(messageId)}`, {
+    method: 'DELETE',
+    body: JSON.stringify({ sprintId, sessionId }),
+  });
+}
+```
+
+## Server Implementation (server_lambda)
+
+### New Routes
+
+Add to `opencodeRoutes.ts`:
+
+```typescript
+router.post('/messages', authenticateApiToken, requireScopes(['agent:query']), recordMessage);
+router.delete('/messages/:messageId', authenticateApiToken, requireScopes(['agent:query']), removeMessage);
+```
+
+### New Controller Methods (opencodeController.ts)
+
+**`recordMessage`:**
+1. Validate required fields (`sprintId`, `messageId`, `conversationId`, `role`, `content`)
+2. If `role === "user"`:
+   - INSERT into `Conversation` with `userMessage = content`, `agentResponse = ''`, `metadata = {...}`
+3. If `role === "assistant"`:
+   - UPDATE `Conversation SET agentResponse = content, metadata = metadata || {...}` WHERE `conversationId = :id` AND `id = (SELECT MAX(id) FROM Conversation WHERE conversationId = :id)`
+4. Return `{ status: "ok", id: <row id> }`
+
+**`removeMessage`:**
+1. Parse `sprintId` and `sessionId` from body
+2. Query for row matching the messageId in metadata JSON
+3. `DELETE FROM Conversation WHERE id = :id`
+4. Return `{ status: "ok" }`
+
+## Error Handling
+
+| Scenario | Plugin Behavior |
+|----------|----------------|
+| No manifest / no active sprint | Silent skip (return immediately) |
+| Non-text part type | No-op |
+| Non-assistant message.updated | No-op |
+| No buffered text for assistant | Skip (return immediately) |
+| API unreachable / network error | Log warning, drop (no retry) |
+| Server returns 4xx | Log warning, drop |
+| Server returns 5xx | Log warning, drop |
+
+**Key principle:** All hooks are fire-and-forget. They must never throw — a failing message recorder must not disrupt the opencode session.
+
+## Text Buffer Lifecycle
+
+The in-memory `textBuffer` map accumulates text by messageID. Each `message.part.updated` with type "text" updates the buffer with the full accumulated `part.text`. Each `message.updated` with role "assistant" flushes the current buffer value to the API (overwriting the `agentResponse` column). The buffer is NOT deleted after flush — parts may arrive after `message.updated`, and each subsequent flush simply overwrites with the latest text. The last write always wins, ensuring the complete text is persisted.
+
+Cleanup happens only on:
+- `message.removed`: buffer entry deleted
+- Plugin unload / process exit: buffer is garbage collected
+
+## Feature Toggle
+
+Always active when `activeSprintId` is present in `~/.forloop/manifest.json`. No explicit toggle — if there's no sprint, there's nothing to record.
+
+## Testing
+
+### Unit Tests: `test/suites/unit/message-recorder.test.ts`
+
+| Test | Description |
+|------|-------------|
+| `chat.message` hook | Extracts text from parts, calls `recordMessage` with user role |
+| `chat.message` hook — no sprint | No API calls |
+| `chat.message` hook — no text | No API calls |
+| `event` — `message.part.updated` (text) | Buffers text by messageID |
+| `event` — `message.part.updated` (non-text) | No-op |
+| `event` — `message.updated` (assistant) | Flushes buffer, calls `recordMessage` |
+| `event` — `message.updated` (user) | No-op (user handled by `chat.message`) |
+| `event` — `message.updated` (no buffer) | No-op |
+| `event` — `message.removed` | Calls `removeMessage`, cleans buffer |
+| No manifest | No API calls for any event |
+| API error | Does not throw, logs warning |
+| Agent key normalization | `"forLoopPlanner"` → `"forLoopPlanner"` |
+| conversationId format | Matches `sprint:{id}:agent:{agent}:opencode:{sessionId}` |
+| Buffer persists after flush | Subsequent `message.updated` flushes again with latest text |
+| textBuffer cleanup after remove | Buffer entry is deleted |
+
+### Integration Tests: `test/suites/integration/message-api.test.ts`
+
+| Test | Description |
+|------|-------------|
+| POST user message | INSERT row, returns id |
+| POST assistant message | UPDATE existing row |
+| DELETE message | Removes row |
+| Invalid sprintId | Returns 400 |
+| No token | Returns 401 |
+
+## Conversation Summarization Compatibility
+
+The `conversationId` format difference (`web` vs `opencode`) does not affect summarization. The summarizer queries by `sprintId + conversationId` — the ID is a grouping key. The format difference is actually useful metadata for identifying the source. The shared `sprint:{id}:agent:{agent}:` prefix enables cross-source queries (e.g., "all conversations with forLoopPlanner").
+
+## Files Changed
+
+| File | Change |
+|------|--------|
+| `plugins/hooks/message-recorder.ts` | **New** — `createChatMessageHook` + `createEventHook` |
+| `plugins/forloop-plugin.ts` | **Modified** — wire `event` and `chat.message` hooks |
+| `plugins/capabilities/api-client.ts` | **Modified** — add `recordMessage`, `removeMessage` methods |
+| `server_lambda/src/routes/opencodeRoutes.ts` | **Modified** — add message routes |
+| `server_lambda/src/controllers/opencodeController.ts` | **Modified** — add controller methods |
+| `test/suites/unit/message-recorder.test.ts` | **New** — unit tests |
+| `test/suites/integration/message-api.test.ts` | **New** — integration tests |