@senad-d/observme 0.1.0

package/ObservMe-Production-Docs/pi-session-format.md

@@ -0,0 +1,427 @@
+# Session File Format
+
+Sessions are stored as JSONL (JSON Lines) files. Each line is a JSON object with a `type` field. Session entries form a tree structure via `id`/`parentId` fields, enabling in-place branching without creating new files.
+
+## File Location
+
+```
+~/.pi/agent/sessions/--<path>--/<timestamp>_<uuid>.jsonl
+```
+
+Where `<path>` is the working directory with `/` replaced by `-`.
+
+## Deleting Sessions
+
+Sessions can be removed by deleting their `.jsonl` files under `~/.pi/agent/sessions/`.
+
+Pi also supports deleting sessions interactively from `/resume` (select a session and press `Ctrl+D`, then confirm). When available, pi uses the `trash` CLI to avoid permanent deletion.
+
+## Session Version
+
+Sessions have a version field in the header:
+
+- **Version 1**: Linear entry sequence (legacy, auto-migrated on load)
+- **Version 2**: Tree structure with `id`/`parentId` linking
+- **Version 3**: Renamed `hookMessage` role to `custom` (extensions unification)
+
+Existing sessions are automatically migrated to the current version (v3) when loaded.
+
+## Source Files
+
+Source on GitHub ([pi-mono](https://github.com/earendil-works/pi-mono)):
+- [`packages/coding-agent/src/core/session-manager.ts`](https://github.com/earendil-works/pi-mono/blob/main/packages/coding-agent/src/core/session-manager.ts) - Session entry types and SessionManager
+- [`packages/coding-agent/src/core/messages.ts`](https://github.com/earendil-works/pi-mono/blob/main/packages/coding-agent/src/core/messages.ts) - Extended message types (BashExecutionMessage, CustomMessage, etc.)
+- [`packages/ai/src/types.ts`](https://github.com/earendil-works/pi-mono/blob/main/packages/ai/src/types.ts) - Base message types (UserMessage, AssistantMessage, ToolResultMessage)
+- [`packages/agent/src/types.ts`](https://github.com/earendil-works/pi-mono/blob/main/packages/agent/src/types.ts) - AgentMessage union type
+
+For TypeScript definitions in your project, inspect `node_modules/@earendil-works/pi-coding-agent/dist/` and `node_modules/@earendil-works/pi-ai/dist/`.
+
+## Message Types
+
+Session entries contain `AgentMessage` objects. Understanding these types is essential for parsing sessions and writing extensions.
+
+### Content Blocks
+
+Messages contain arrays of typed content blocks:
+
+```typescript
+interface TextContent {
+  type: "text";
+  text: string;
+}
+
+interface ImageContent {
+  type: "image";
+  data: string;      // base64 encoded
+  mimeType: string;  // e.g., "image/jpeg", "image/png"
+}
+
+interface ThinkingContent {
+  type: "thinking";
+  thinking: string;
+  thinkingSignature?: string;
+  redacted?: boolean;
+}
+
+interface ToolCall {
+  type: "toolCall";
+  id: string;
+  name: string;
+  arguments: Record<string, any>;
+  thoughtSignature?: string;
+}
+```
+
+### Base Message Types (from pi-ai)
+
+```typescript
+interface UserMessage {
+  role: "user";
+  content: string | (TextContent | ImageContent)[];
+  timestamp: number;  // Unix ms
+}
+
+interface AssistantMessage {
+  role: "assistant";
+  content: (TextContent | ThinkingContent | ToolCall)[];
+  api: string;
+  provider: string;
+  model: string;
+  responseModel?: string;
+  responseId?: string;
+  diagnostics?: any[];
+  usage: Usage;
+  stopReason: "stop" | "length" | "toolUse" | "error" | "aborted";
+  errorMessage?: string;
+  timestamp: number;
+}
+
+interface ToolResultMessage {
+  role: "toolResult";
+  toolCallId: string;
+  toolName: string;
+  content: (TextContent | ImageContent)[];
+  details?: any;      // Tool-specific metadata
+  isError: boolean;
+  timestamp: number;
+}
+
+interface Usage {
+  input: number;
+  output: number;
+  cacheRead: number;
+  cacheWrite: number;
+  cacheWrite1h?: number;
+  reasoning?: number;
+  totalTokens: number;
+  cost: {
+    input: number;
+    output: number;
+    cacheRead: number;
+    cacheWrite: number;
+    total: number;
+  };
+}
+```
+
+### Extended Message Types (from pi-coding-agent)
+
+```typescript
+interface BashExecutionMessage {
+  role: "bashExecution";
+  command: string;
+  output: string;
+  exitCode: number | undefined;
+  cancelled: boolean;
+  truncated: boolean;
+  fullOutputPath?: string;
+  excludeFromContext?: boolean;  // true for !! prefix commands
+  timestamp: number;
+}
+
+interface CustomMessage {
+  role: "custom";
+  customType: string;            // Extension identifier
+  content: string | (TextContent | ImageContent)[];
+  display: boolean;              // Show in TUI
+  details?: any;                 // Extension-specific metadata
+  timestamp: number;
+}
+
+interface BranchSummaryMessage {
+  role: "branchSummary";
+  summary: string;
+  fromId: string;                // Entry we branched from
+  timestamp: number;
+}
+
+interface CompactionSummaryMessage {
+  role: "compactionSummary";
+  summary: string;
+  tokensBefore: number;
+  timestamp: number;
+}
+```
+
+### AgentMessage Union
+
+```typescript
+type AgentMessage =
+  | UserMessage
+  | AssistantMessage
+  | ToolResultMessage
+  | BashExecutionMessage
+  | CustomMessage
+  | BranchSummaryMessage
+  | CompactionSummaryMessage;
+```
+
+## Entry Base
+
+All entries (except `SessionHeader`) extend `SessionEntryBase`:
+
+```typescript
+interface SessionEntryBase {
+  type: string;
+  id: string;           // 8-char hex ID
+  parentId: string | null;  // Parent entry ID (null for first entry)
+  timestamp: string;    // ISO timestamp
+}
+```
+
+## Entry Types
+
+### SessionHeader
+
+First line of the file. Metadata only, not part of the tree (no `id`/`parentId`).
+
+```json
+{"type":"session","version":3,"id":"uuid","timestamp":"2024-12-03T14:00:00.000Z","cwd":"/path/to/project"}
+```
+
+For sessions with a parent (created via `/fork`, `/clone`, or `newSession({ parentSession })`):
+
+```json
+{"type":"session","version":3,"id":"uuid","timestamp":"2024-12-03T14:00:00.000Z","cwd":"/path/to/project","parentSession":"/path/to/original/session.jsonl"}
+```
+
+### SessionMessageEntry
+
+A message in the conversation. The `message` field contains an `AgentMessage`.
+
+```json
+{"type":"message","id":"a1b2c3d4","parentId":null,"timestamp":"2024-12-03T14:00:01.000Z","message":{"role":"user","content":"Hello"}}
+{"type":"message","id":"b2c3d4e5","parentId":"a1b2c3d4","timestamp":"2024-12-03T14:00:02.000Z","message":{"role":"assistant","content":[{"type":"text","text":"Hi!"}],"provider":"anthropic","model":"claude-sonnet-4-5","usage":{...},"stopReason":"stop"}}
+{"type":"message","id":"c3d4e5f6","parentId":"b2c3d4e5","timestamp":"2024-12-03T14:00:03.000Z","message":{"role":"toolResult","toolCallId":"call_123","toolName":"bash","content":[{"type":"text","text":"output"}],"isError":false}}
+```
+
+### ModelChangeEntry
+
+Emitted when the user switches models mid-session.
+
+```json
+{"type":"model_change","id":"d4e5f6a7","parentId":"c3d4e5f6","timestamp":"2024-12-03T14:05:00.000Z","provider":"openai","modelId":"gpt-4o"}
+```
+
+### ThinkingLevelChangeEntry
+
+Emitted when the user changes the thinking/reasoning level.
+
+```json
+{"type":"thinking_level_change","id":"e5f6a7b8","parentId":"d4e5f6a7","timestamp":"2024-12-03T14:06:00.000Z","thinkingLevel":"high"}
+```
+
+### CompactionEntry
+
+Created when context is compacted. Stores a summary of earlier messages.
+
+```json
+{"type":"compaction","id":"f6a7b8c9","parentId":"e5f6a7b8","timestamp":"2024-12-03T14:10:00.000Z","summary":"User discussed X, Y, Z...","firstKeptEntryId":"c3d4e5f6","tokensBefore":50000}
+```
+
+Optional fields:
+- `details`: Implementation-specific data (e.g., `{ readFiles: string[], modifiedFiles: string[] }` for default, or custom data for extensions)
+- `fromHook`: `true` if generated by an extension, `false`/`undefined` if pi-generated (legacy field name)
+
+### BranchSummaryEntry
+
+Created when switching branches via `/tree` with an LLM generated summary of the left branch up to the common ancestor. Captures context from the abandoned path.
+
+```json
+{"type":"branch_summary","id":"a7b8c9d0","parentId":"a1b2c3d4","timestamp":"2024-12-03T14:15:00.000Z","fromId":"f6a7b8c9","summary":"Branch explored approach A..."}
+```
+
+Optional fields:
+- `details`: File tracking data (`{ readFiles: string[], modifiedFiles: string[] }`) for default, or custom data for extensions
+- `fromHook`: `true` if generated by an extension, `false`/`undefined` if pi-generated (legacy field name)
+
+### CustomEntry
+
+Extension state persistence. Does NOT participate in LLM context.
+
+```json
+{"type":"custom","id":"b8c9d0e1","parentId":"a7b8c9d0","timestamp":"2024-12-03T14:20:00.000Z","customType":"my-extension","data":{"count":42}}
+```
+
+Use `customType` to identify your extension's entries on reload. Custom entries are for extension state and do not have a public custom-entry renderer API. If an extension needs visible custom content, use `custom_message` plus `pi.registerMessageRenderer(customType, renderer)`.
+
+### CustomMessageEntry
+
+Extension-injected messages that DO participate in LLM context.
+
+```json
+{"type":"custom_message","id":"c9d0e1f2","parentId":"b8c9d0e1","timestamp":"2024-12-03T14:25:00.000Z","customType":"my-extension","content":"Injected context...","display":true}
+```
+
+Fields:
+- `content`: String or `(TextContent | ImageContent)[]` (same as UserMessage)
+- `display`: `true` = show in TUI with distinct styling, `false` = hidden
+- `details`: Optional extension-specific metadata (not sent to LLM)
+
+### LabelEntry
+
+User-defined bookmark/marker on an entry.
+
+```json
+{"type":"label","id":"d0e1f2a3","parentId":"c9d0e1f2","timestamp":"2024-12-03T14:30:00.000Z","targetId":"a1b2c3d4","label":"checkpoint-1"}
+```
+
+Set `label` to `undefined` to clear a label.
+
+### SessionInfoEntry
+
+Session metadata (e.g., user-defined display name). Set via `/name`, `--name` / `-n`, or `pi.setSessionName()` in extensions.
+
+```json
+{"type":"session_info","id":"e1f2a3b4","parentId":"d0e1f2a3","timestamp":"2024-12-03T14:35:00.000Z","name":"Refactor auth module"}
+```
+
+The session name is displayed in the session selector (`/resume`) instead of the first message when set.
+
+## Tree Structure
+
+Entries form a tree:
+- First entry has `parentId: null`
+- Each subsequent entry points to its parent via `parentId`
+- Branching creates new children from an earlier entry
+- The "leaf" is the current position in the tree
+
+```
+[user msg] ─── [assistant] ─── [user msg] ─── [assistant] ─┬─ [user msg] ← current leaf
+                                                            │
+                                                            └─ [branch_summary] ─── [user msg] ← alternate branch
+```
+
+## Context Building
+
+`getBranch()` walks from the current leaf to the root and returns the active branch entries in path order, including non-message entries.
+
+`buildSessionContext()` walks the active path and produces the message list for the LLM while honoring compaction:
+
+1. Collects all entries on the active path.
+2. Extracts current model and thinking level settings from the path.
+3. If a `CompactionEntry` is on the path:
+   - Emits the compaction summary first.
+   - Then includes messages from `firstKeptEntryId` to the compaction.
+   - Then includes messages after the compaction.
+4. Converts selected entries to messages:
+   - `message` -> stored `AgentMessage`
+   - `compaction` -> `compactionSummary`
+   - `branch_summary` -> `branchSummary`
+   - `custom_message` -> `CustomMessage`
+   - `custom` -> no context message
+
+## Parsing Example
+
+```typescript
+import { readFileSync } from "fs";
+
+const lines = readFileSync("session.jsonl", "utf8").trim().split("\n");
+
+for (const line of lines) {
+  const entry = JSON.parse(line);
+
+  switch (entry.type) {
+    case "session":
+      console.log(`Session v${entry.version ?? 1}: ${entry.id}`);
+      break;
+    case "message":
+      console.log(`[${entry.id}] ${entry.message.role}: ${JSON.stringify(entry.message.content)}`);
+      break;
+    case "compaction":
+      console.log(`[${entry.id}] Compaction: ${entry.tokensBefore} tokens summarized`);
+      break;
+    case "branch_summary":
+      console.log(`[${entry.id}] Branch from ${entry.fromId}`);
+      break;
+    case "custom":
+      console.log(`[${entry.id}] Custom (${entry.customType}): ${JSON.stringify(entry.data)}`);
+      break;
+    case "custom_message":
+      console.log(`[${entry.id}] Extension message (${entry.customType}): ${entry.content}`);
+      break;
+    case "label":
+      console.log(`[${entry.id}] Label "${entry.label}" on ${entry.targetId}`);
+      break;
+    case "model_change":
+      console.log(`[${entry.id}] Model: ${entry.provider}/${entry.modelId}`);
+      break;
+    case "thinking_level_change":
+      console.log(`[${entry.id}] Thinking: ${entry.thinkingLevel}`);
+      break;
+  }
+}
+```
+
+## SessionManager API
+
+Key methods for working with sessions programmatically.
+
+### Static Creation Methods
+- `SessionManager.create(cwd, sessionDir?)` - New session
+- `SessionManager.open(path, sessionDir?)` - Open existing session file
+- `SessionManager.continueRecent(cwd, sessionDir?)` - Continue most recent or create new
+- `SessionManager.inMemory(cwd?)` - No file persistence
+- `SessionManager.forkFrom(sourcePath, targetCwd, sessionDir?)` - Fork session from another project
+
+### Static Listing Methods
+- `SessionManager.list(cwd, sessionDir?, onProgress?)` - List sessions for a directory
+- `SessionManager.listAll(onProgress?)` - List all sessions across all projects
+
+### Instance Methods - Session Management
+- `newSession(options?)` - Start a new session (options: `{ parentSession?: string }`)
+- `setSessionFile(path)` - Switch to a different session file
+- `createBranchedSession(leafId)` - Extract branch to new session file
+
+### Instance Methods - Appending (all return entry ID)
+- `appendMessage(message)` - Add message
+- `appendThinkingLevelChange(level)` - Record thinking change
+- `appendModelChange(provider, modelId)` - Record model change
+- `appendCompaction(summary, firstKeptEntryId, tokensBefore, details?, fromHook?)` - Add compaction
+- `appendCustomEntry(customType, data?)` - Extension state (not in context)
+- `appendSessionInfo(name)` - Set session display name
+- `appendCustomMessageEntry(customType, content, display, details?)` - Extension message (in context)
+- `appendLabelChange(targetId, label)` - Set/clear label
+
+### Instance Methods - Tree Navigation
+- `getLeafId()` - Current position
+- `getLeafEntry()` - Get current leaf entry
+- `getEntry(id)` - Get entry by ID
+- `getBranch(fromId?)` - Walk from entry to root
+- `getTree()` - Get full tree structure
+- `getChildren(parentId)` - Get direct children
+- `getLabel(id)` - Get label for entry
+- `branch(entryId)` - Move leaf to earlier entry
+- `resetLeaf()` - Reset leaf to null (before any entries)
+- `branchWithSummary(entryId, summary, details?, fromHook?)` - Branch with context summary
+
+### Instance Methods - Context & Info
+- `buildSessionContext()` - Get messages, thinkingLevel, and model for LLM
+- `getEntries()` - All entries (excluding header)
+- `getHeader()` - Session header metadata
+- `getSessionName()` - Get display name from latest session_info entry
+- `getCwd()` - Working directory
+- `getSessionDir()` - Session storage directory
+- `getSessionId()` - Session UUID
+- `getSessionFile()` - Session file path (undefined for in-memory)
+- `isPersisted()` - Whether session is saved to disk