@xalia/agent 0.6.8 → 0.6.10

package/.env.development

@@ -0,0 +1,6 @@
+LLM_API_KEY_MAP={"openrouter":"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","together":"5928479bc38fd315acc8359ba42587f8efc804e01b06eb02eb5ee97e044afaa1","openai":"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"}
+
+# Internal API secret for sending emails
+SEND_EMAIL_API_SECRET=73f52358f5741e2467d0ca13733b03f34ed334589c878c28c442f8a08abf85a6
+
+XALIA_URL=http://localhost:3000

package/.env.test

@@ -0,0 +1,7 @@
+# Test environment configuration
+# Copy API keys from .env.development and add test-specific settings
+
+LLM_API_KEY_MAP={"openrouter":"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","together":"5928479bc38fd315acc8359ba42587f8efc804e01b06eb02eb5ee97e044afaa1","openai":"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"}
+
+# Disable LLM-generated titles for deterministic Playwright tests
+DISABLE_LLM_TITLES=true

package/README.md

@@ -25,6 +25,17 @@ Example of running agent can be found in [test script](../mcppro/scripts/test_sc
 
 ## Architecture
 
+For a detailed explanation of the context system architecture, including how context is formed, compressed, and managed during agent execution, see:
+
+**📚 [Context System Architecture Documentation](../context_system.md)**
+
+This document covers:
+- Context window structure and composition
+- System prompt fragment injection
+- Automatic context compression
+- Session file handling
+- Transaction-based message flow
+
 ### Core Components
 
 #### Agent (`src/agent/`)

package/context_system.md

@@ -0,0 +1,498 @@
+# Agent Context System
+
+## Overview
+
+This document describes how the agent's context system works, including context formation, compression, fragment injection, and message flow during agent execution.
+
+## High-Level Architecture
+
+The context system uses a **layered architecture** with:
+1. **Fragment-based system prompts** for modularity
+2. **Transaction-based updates** for atomicity
+3. **Automatic compression** for long conversations
+4. **Dynamic context injection** for tools and files
+
+### Context Window Structure
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                    CONTEXT WINDOW (MessageParam[])              │
+├─────────────────────────────────────────────────────────────────┤
+│ Index 0: SYSTEM MESSAGE (Always present, dynamically generated) │
+├─────────────────────────────────────────────────────────────────┤
+│ Index 1+: CONVERSATION MESSAGES                                 │
+│   - May start with checkpoint message (if compressed)           │
+│   - User, Assistant, and Tool messages                          │
+│   - Workspace message at end (if ContextManagerWithWorkspace)   │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+## Core Components
+
+### 1. System Prompt Composition (Index 0)
+
+The system message is **never stored in the database**. It's dynamically composed from three sources:
+
+```
+SystemPromptProvider.computePrompt():
+├── Global Prompt (static, shared by all agents)
+├── Agent Prompt (per-agent, from AgentProfile)
+└── Fragments (dynamic, tool-specific context)
+    ├── "file_manager" → Available files list
+    ├── "participants" → User UUID to name mapping
+    └── [Custom fragments...]
+
+Composition: global + "\n" + agent + "\n" + fragments.join("\n")
+```
+
+The full prompt is lazily created and cached on every call to `getLLMContext()`.
+
+**Key files:**
+- `agent/src/agent/promptProvider.ts` - System prompt composition
+- `agent/src/agent/context.ts` - Base context manager
+
+### 2. Conversation Messages (Index 1+)
+
+Messages are stored in the database and loaded on session start.  All messages are stored, whether or not the conversation has been compressed.  Compressed conversations also have "checkpoints" (which include a message index).
+In this way, the (compressed) LLM context can be recreated and historical messages can still be sent to clients.
+
+The LLM structure varies depending on compression state:
+
+#### Without Compression (< 80 messages)
+
+```
+Index 0: System Message
+Index 1: User Message
+Index 2: Assistant Message (with tool_calls)
+Index 3: Tool Message (results, with content deduplication)
+Index 4: Assistant Message
+...
+Index N: Workspace Message (optional, special position)
+```
+
+#### With Compression (≥ 80 messages)
+```
+Index 0: System Message
+Index 1: Checkpoint Message (summarizes old messages)
+Index 2+: Recent messages (20-30 most recent)
+Index N: Workspace Message (if applicable)
+```
+
+**Key files:**
+- `agent/src/agent/compressingContextManager.ts` - Compression logic
+- `agent/src/agent/contextWithWorkspace.ts` - Workspace message handling
+- `agent/src/chat/server/chatContextManager.ts` - Chat-specific coordination
+
+## Context Formation During Agent Session
+
+### Initial Setup
+
+```typescript
+// chatContextManager.ts:229-269
+constructor(
+  systemPrompt: string,
+  sessionMessages: SessionMessage[],
+  checkpoint: SessionCheckpoint | undefined
+) {
+  // Resolve messages with checkpoint (if compressed)
+  const { messages: llmMessages } = resolveConversationWithCheckpoint(
+    sessionMessages,
+    checkpoint
+  );
+
+  // Create compressing context manager
+  this.llmContext = new CompressingContextManager(
+    systemPrompt,
+    llmMessages,
+    getLLM
+  );
+}
+```
+
+### Transaction-Based Message Flow
+
+```
+User sends messages
+  ↓
+startTx(userMessages) → Creates ChatContextTransaction
+  ↓
+  Transaction context = [
+    System (with fragments: files, participants),
+    ...Committed messages,
+    ...New user messages,
+    Workspace message (if set)
+  ]
+  ↓
+Agent executes (tool calls, reasoning, responses)
+  ↓
+commit(tx) → Appends agent messages to committed history
+  ↓
+checkCompression() → Triggers if > 80 messages
+```
+
+**Key files:**
+- `agent/src/agent/agent.ts:472-479` - Transaction pattern
+- `agent/src/chat/server/chatContextManager.ts` - Transaction implementation
+
+## Context Compression
+
+### Trigger Mechanism
+
+Compression activates when committed messages exceed **80 messages** (configurable via `COMPRESSION_TRIGGER_NUM_MESSAGES`).
+
+### Compression Process
+
+```typescript
+// compressingContextManager.ts:99-128
+async compress(): Promise<string> {
+  // Step 1: Extract messages to compress
+  const messagesToCompress = this.leadingMessages(numToCompress);
+
+  // Step 2: Create summary with specialized LLM
+  const llm = await this.getLLM();
+  const summary = await createSummary(llm, messagesToCompress);
+
+  // Step 3: Replace with checkpoint message
+  const checkpointMessage = {
+    role: "user",
+    content: "We are continuing an earlier conversation. " +
+             "The remainder of this message is a summary: " + summary
+  };
+  this.replaceLeadingMessages(numToCompress, checkpointMessage);
+
+  // Step 4: Save checkpoint to database
+  await checkpointWriter.createCheckpoint({
+    summary,
+    num_messages_compressed: numToCompress,
+    session_uuid: sessionUUID
+  });
+
+  return summary;
+}
+```
+
+The compression agent uses a specialized prompt:
+```
+"You are a context summarizer, creating MINIMAL conversation digests
+which can be used to CONTINUE the conversation at a later date.
+TOKEN EFFICIENCY is a HIGH PRIORITY. Summaries will only be seen by
+LLMs and should USE ANY AND ALL ABBREVIATIONS to keep them as CONCISE
+as possible, while PRESERVING IMPORTANT DETAILS of the CONVERSATION."
+```
+
+**Result:** Token savings of ~50-70% while preserving conversational context.
+
+**Key files:**
+- `agent/src/agent/compressingContextManager.ts` - Compression implementation
+- `agent/src/chat/server/chatContextManager.ts:535-548` - Trigger logic
+
+## Tool-Specific Context Injection
+
+The system uses **prompt fragments** to inject dynamic context. Fragments are identified by string IDs and can be updated independently.
+
+### Fragment Injection Pattern
+
+```typescript
+// Lazy-update pattern (chatContextManager.ts:272-281)
+getLLMContext(): MessageParam[] {
+  if (this.fileManagerDescriptionsDirty) {  // Check dirty flag
+    // Regenerate the file list prompt
+    const prompt = createSessionFilesManagerPrompt(this.fileManager);
+
+    // Update the system prompt fragment
+    this.llmContext.setPromptFragment("file_manager", prompt);
+
+    this.fileManagerDescriptionsDirty = false;  // Clear flag
+  }
+
+  return this.llmContext.getLLMContext();
+}
+```
+
+### Key Injection Points
+
+#### A. File Manager Context
+
+```typescript
+// sessionFileManager.ts:271-287
+function createSessionFilesManagerPrompt(fm: ISessionFileManager): string {
+  const files = fm.listFiles();
+  if (files.length === 0) return "";
+
+  let prompt =
+    "Files can be read/written as required. Create new files conservatively, " +
+    "usually when complex content is requested. Use base64 for binary (image," +
+    "pdf), ascii for text formats (html,markdown). " +
+    "Available files:\nname,type,summary\n";
+
+  for (const f of files) {
+    prompt += `${f.name},${f.mime_type},${f.summary || ""}\n`;
+  }
+  return prompt;
+}
+```
+
+**Event-driven updates:**
+```typescript
+// When files change, mark dirty
+onFileChanged(_entry: SessionFileEntry): void {
+  this.fileManagerDescriptionsDirty = true;
+}
+```
+
+#### B. Participants Context
+
+```typescript
+// openSession.ts:1532-1547
+private updateParticipantsPrompt(): void {
+  const prompt =
+    "The following map is from user_uuids to real names. In user " +
+    "messages, real names will be used...\n" +
+    participants.map(([uuid, name]) => `${uuid}: ${name}`).join("\n");
+
+  this.contextManager.setPromptFragment("participants", prompt);
+}
+```
+
+#### C. MCP Tool Definitions
+
+MCP tools are passed as a separate parameter to the LLM, not as system prompt fragments:
+
+```typescript
+// agent.ts:245-254
+async chatCompletion(context: MessageParam[]) {
+  const mcpTools = this.mcpServerManager.getOpenAITools();
+  const enabledTools = this.tools.concat(mcpTools);
+
+  return await this.llm.getConversationResponse(
+    context,
+    enabledTools,  // ← Tool schemas passed here
+    eventHandler
+  );
+}
+```
+
+**Key files:**
+- `agent/src/agent/promptProvider.ts` - Fragment management
+- `agent/src/chat/server/sessionFileManager.ts` - File manager integration
+- `agent/src/chat/server/openSession.ts` - Participants management
+
+## Session Files Handling
+
+Session files use an **event-driven architecture** with **lazy-update prompt injection**.
+
+### File Storage
+
+Two implementations:
+- **MemoryFileManager**: In-memory storage for testing
+- **ChatSessionFileManager**: Database-backed with two-tier caching
+  - Metadata cache: `fileMap: Map<string, SessionFileDescriptor>`
+  - Content cache: `fileDataCache: Map<string, string>`
+
+### Event Flow
+
+```
+User/Agent modifies file
+  ↓
+SessionFileManager.putFileContent() / deleteFile()
+  ↓
+Update internal state (fileMap, DB)
+  ↓
+Trigger event handlers
+  ↓
+ChatContextManager.onFileChanged() / onFileDeleted()
+  ↓
+Set fileManagerDescriptionsDirty = true
+  ↓
+Next getLLMContext() call regenerates prompt fragment
+```
+
+### Content Deduplication
+
+File tool calls use a special optimization to prevent context bloat:
+
+```typescript
+// sessionFileManager.ts:361-369
+const getFileContentFn: ToolHandler = async (_, args) => {
+  const { name } = parseName(args);
+  const response = await fileManager.getFileContent(name);
+
+  // Replace full data URL with session file reference
+  const overwriteResponse = fileManager.getSessionFileRelativeUrl(name);
+
+  return { response, overwriteResponse };
+};
+```
+
+**Result:**
+- **Initial response**: `data:image/png;base64,iVBORw0KGg...` (large)
+- **Overwritten to**: `file+session://./chart.png` (compact)
+
+**Key files:**
+- `agent/src/chat/server/sessionFileManager.ts` - File management
+- `agent/src/chat/server/chatContextManager.ts:271-315` - File event handling
+
+## Workspace Message
+
+The workspace message is a **special trailing user message** that provides persistent context across all agent responses in a session.
+
+### Special Handling
+
+```typescript
+// contextWithWorkspace.ts:142-149
+setWorkspace(userMessage: UserMessageParam | undefined) {
+  if (!userMessage) {
+    userMessage = { role: "user", content: "" };
+  }
+  super.popMessage();        // Remove old workspace
+  super.pushMessage(userMessage);  // Add new workspace
+}
+```
+
+### Transaction Behavior
+
+During transactions, the workspace message is:
+1. Moved from the end of committed messages
+2. Positioned **before** agent responses
+3. Restored to the end after commit
+
+```
+Committed: [U1, A1, U2, A2, Workspace]
+Transaction: [U1, A1, U2, A2, NewUser, Workspace, NewAgent]
+After commit: [U1, A1, U2, A2, NewUser, NewAgent, Workspace]
+```
+
+**Key files:**
+- `agent/src/agent/contextWithWorkspace.ts` - Workspace implementation
+
+## Dynamic Todo/Task Tracking
+
+**Result: NOT IMPLEMENTED**
+
+The current agent does not have a dynamic todo/task tracking system during execution. There is no equivalent to Claude Code's TodoWrite tool.
+
+What exists:
+- Message history tracking
+- Tool call result tracking
+- File metadata tracking
+- Checkpoint tracking
+
+This could be a future enhancement.
+
+## Complete Data Flow
+
+```
+┌────────────────────────────────────────────────────────────┐
+│ 1. User Message Received                                   │
+└────────────────────────────────────────────────────────────┘
+  ↓
+┌────────────────────────────────────────────────────────────┐
+│ 2. ChatContextManager.processUserMessage()                │
+│    → Assigns message_idx                                   │
+└────────────────────────────────────────────────────────────┘
+  ↓
+┌────────────────────────────────────────────────────────────┐
+│ 3. ChatContextManager.startAgentResponse()                │
+│    → Creates ChatContextTransaction                        │
+│    → Injects: user messages + workspace + file list       │
+│                + participants                              │
+└────────────────────────────────────────────────────────────┘
+  ↓
+┌────────────────────────────────────────────────────────────┐
+│ 4. Agent.userMessagesRaw()                                │
+│    → Loop: LLM call → Tool calls → LLM call → ...         │
+└────────────────────────────────────────────────────────────┘
+  ↓
+┌────────────────────────────────────────────────────────────┐
+│ 5. ChatContextManager.endAgentResponse()                  │
+│    → Commits transaction                                   │
+│    → Checks compression trigger                            │
+└────────────────────────────────────────────────────────────┘
+  ↓ (if > 80 messages)
+┌────────────────────────────────────────────────────────────┐
+│ 6. Background: CompressingContextManager.compress()       │
+│    → Summarizes old messages                               │
+│    → Creates checkpoint                                    │
+│    → Saves to DB                                           │
+└────────────────────────────────────────────────────────────┘
+```
+
+## Architecture Layers
+
+```
+┌─────────────────────────────────────────────┐
+│  Agent (agent.ts)                           │
+│  - Execution loop                           │
+│  - Tool orchestration                       │
+└─────────────────┬───────────────────────────┘
+                  │
+┌─────────────────▼───────────────────────────┐
+│  ChatContextManager (chatContextManager.ts) │
+│  - Transaction management                   │
+│  - Compression triggering                   │
+│  - Fragment coordination                    │
+└─────────────────┬───────────────────────────┘
+                  │
+        ┌─────────┴─────────┐
+        ▼                   ▼
+┌───────────────────┐  ┌────────────────────────┐
+│ CompressingContext│  │ SessionFileManager     │
+│ Manager           │  │ (sessionFileManager.ts)│
+│ (compressing...ts)│  │ - File tracking        │
+│ - Summarization   │  │ - Prompt generation    │
+└───────┬───────────┘  └────────────────────────┘
+        │
+        ▼
+┌──────────────────────┐
+│ ContextWithWorkspace │
+│ (contextWith...ts)   │
+│ - Workspace msg      │
+└──────────┬───────────┘
+           │
+           ▼
+┌──────────────────────┐
+│ ContextManager       │
+│ (context.ts)         │
+│ - Base messages      │
+│ - SystemPromptProvider│
+└──────────────────────┘
+```
+
+## Key Design Principles
+
+1. **System message is never stored** - Regenerated on every context request
+2. **Fragments use lazy evaluation** - Only recomputed when dirty flags are set
+3. **Workspace has special handling** - Always at end, moved during transactions
+4. **Content is deduplicated** - Large data URLs replaced with compact references
+5. **Compression is one-way** - Old messages summarized, originals kept in DB for audit
+6. **Checkpoints are user messages** - Special prefix indicates it's a summary
+7. **Transactions ensure atomicity** - All-or-nothing commits with rollback
+
+## Performance Optimizations
+
+- **Lazy fragment updates**: Only regenerate when dirty
+- **Content deduplication**: File contents replaced with references
+- **Automatic compression**: Keeps context window manageable
+- **Two-tier file caching**: Metadata + content caches
+- **Event-driven updates**: Minimizes unnecessary recomputation
+
+## References
+
+### Core Context Files
+
+| Function | File |
+|----------|------|
+| Base context | [`src/agent/context.ts`](src/agent/context.ts) |
+| Workspace support | [`src/agent/contextWithWorkspace.ts`](src/agent/contextWithWorkspace.ts) |
+| Compression logic | [`src/agent/compressingContextManager.ts`](src/agent/compressingContextManager.ts) |
+| System prompt fragments | [`src/agent/promptProvider.ts`](src/agent/promptProvider.ts) |
+| Agent execution | [`src/agent/agent.ts`](src/agent/agent.ts) |
+
+### Chat System Files
+
+| Function | File |
+|----------|------|
+| Chat coordination | [`src/chat/server/chatContextManager.ts`](src/chat/server/chatContextManager.ts) |
+| File context | [`src/chat/server/sessionFileManager.ts`](src/chat/server/sessionFileManager.ts) |
+| Participants context | [`src/chat/server/openSession.ts`](src/chat/server/openSession.ts) |
+| Message transformation | [`src/chat/server/conversation.ts`](src/chat/server/conversation.ts) |