@falai/agent 1.1.2 → 1.2.0

package/docs/api/overview.md

@@ -12,10 +12,14 @@ Complete API documentation for `@falai/agent`. This framework provides a strongl
   - [RoutingEngine](#routingengine)
   - [ResponseEngine](#responseengine)
   - [PromptComposer](#promptcomposer)
+  - [StreamingToolExecutor](#streamingtoolexecutor)
+  - [CompactionEngine](#compactionengine)
+  - [PromptSectionCache](#promptsectioncache)
 
 - [AI Providers](#ai-providers)
 - [Persistence Adapters](#persistence-adapters)
 - [Types & Interfaces](#types--interfaces)
+  - [EnhancedTool](#enhancedtool)
 - [Utilities](#utilities)
 
 ---
@@ -602,6 +606,119 @@ build(): Promise<string>
 
 
 
+---
+
+### StreamingToolExecutor
+
+Executes tools as they arrive from the LLM stream with concurrency control, abort handling, and ordered result yielding.
+
+#### Constructor
+
+```typescript
+new StreamingToolExecutor<TContext, TData>(
+  toolContext: ToolContext<TContext, TData>,
+  options?: {
+    maxParallel?: number;   // default: 10
+    signal?: AbortSignal;
+  }
+)
+```
+
+#### Methods
+
+```typescript
+addTool(toolCall: ToolCallRequest, tool: EnhancedTool<TContext, TData>): void
+```
+
+Queue a tool for execution. Concurrency safety is evaluated once at queue time.
+
+```typescript
+getCompletedResults(): Generator<ToolExecutionUpdate<TData>>
+```
+
+Synchronous generator yielding available results in request order.
+
+```typescript
+getRemainingResults(): AsyncGenerator<ToolExecutionUpdate<TData>>
+```
+
+Async generator yielding all results, waiting for pending tools.
+
+```typescript
+discard(): void
+getUpdatedContext(): TContext
+hasUnfinishedTools(): boolean
+```
+
+See [Streaming Execution Guide](../core/tools/streaming-execution.md) for detailed usage.
+
+---
+
+### CompactionEngine
+
+Manages conversation history size through multi-layered compaction strategies.
+
+#### Static Methods
+
+```typescript
+CompactionEngine.estimateTokens(history: HistoryItem[]): number
+CompactionEngine.applyToolResultBudget(history: HistoryItem[], maxCharsPerResult: number): HistoryItem[]
+CompactionEngine.validateOptions(options: CompactionOptions): void
+CompactionEngine.checkAndCompact(history: HistoryItem[], options: CompactionOptions): Promise<CompactionResult>
+```
+
+See [Context Compaction Guide](../guides/context-compaction.md) for detailed usage.
+
+---
+
+### PromptSectionCache
+
+Memoizes static prompt sections across turns, recomputing only dynamic sections per-turn. Integrates with `PromptComposer` for optimized prompt generation.
+
+#### Constructor
+
+```typescript
+new PromptSectionCache(config?: PromptCacheConfig)
+```
+
+#### Configuration
+
+```typescript
+interface PromptCacheConfig {
+  enabled?: boolean;       // default: true
+  volatileKeys?: string[]; // keys that always recompute
+}
+```
+
+#### Methods
+
+```typescript
+register(key: string, type: PromptSectionType, compute: () => string | null | Promise<string | null>): void
+```
+
+Register a section as `'static'` (cached) or `'dynamic'` (recomputed every turn).
+
+```typescript
+get(key: string): Promise<string | null>
+```
+
+Get a section's value, using cache for static sections.
+
+```typescript
+resolveAll(): Promise<(string | null)[]>
+```
+
+Resolve all sections in registration order.
+
+```typescript
+invalidate(key: string): void
+invalidateAll(): void
+```
+
+Invalidate a specific section or all sections.
+
+See [Prompt Optimization Guide](../guides/prompt-optimization.md) for detailed usage.
+
 ---
 
 ## AI Providers
@@ -1077,6 +1194,29 @@ type ToolHandler<TContext, TArgs extends unknown[], TResult, TData> = (
 }>;
 ```
 
+### EnhancedTool
+
+Extends `Tool` with optional metadata for concurrency, permissions, validation, and result budgeting. See [EnhancedTool Reference](../core/tools/enhanced-tool.md) for full documentation.
+
+```typescript
+interface EnhancedTool<TContext, TData, TResult> extends Tool<TContext, TData, TResult> {
+  isConcurrencySafe?(input?: Record<string, unknown>): boolean;
+  isReadOnly?(input?: Record<string, unknown>): boolean;
+  isDestructive?(input?: Record<string, unknown>): boolean;
+  interruptBehavior?(): 'cancel' | 'block';
+  maxResultSizeChars?: number;
+  validateInput?(input: Record<string, unknown>, context: ToolContext<TContext, TData>): Promise<ToolValidationResult> | ToolValidationResult;
+  checkPermissions?(input: Record<string, unknown>, context: ToolContext<TContext, TData>): Promise<ToolPermissionResult> | ToolPermissionResult;
+}
+
+interface ToolValidationResult { valid: boolean; error?: string; correctedInput?: Record<string, unknown>; }
+interface ToolPermissionResult { allowed: boolean; reason?: string; canOverride?: boolean; }
+interface ToolCallRequest { id: string; toolName: string; arguments: Record<string, unknown>; }
+interface ToolExecutionUpdate<TData> { toolCallId: string; result?: ToolExecutionResult; progress?: string; contextUpdate?: Record<string, unknown>; dataUpdate?: Partial<TData>; }
+interface CompactionOptions { maxTokens: number; compactionThreshold: number; preserveRecentCount: number; maxToolResultChars: number; provider: AiProvider; }
+interface CompactionResult<TData> { history: HistoryItem[]; strategy: 'none' | 'tool_result_budget' | 'micro_compact' | 'auto_compact'; estimatedTokens: number; messagesCompacted: number; summary?: string; }
+```
+
 ### AI Provider Types
 
 ```typescript

package/docs/core/tools/enhanced-tool.md

@@ -0,0 +1,186 @@
+# EnhancedTool Interface
+
+`EnhancedTool` extends the existing `Tool` interface with optional metadata for concurrency control, permission gating, input validation, and result size management. All additional methods are optional — plain `Tool` objects remain fully compatible.
+
+## Interface
+
+```typescript
+interface EnhancedTool<TContext = any, TData = any, TResult = any>
+  extends Tool<TContext, TData, TResult> {
+
+  // Concurrency & safety
+  isConcurrencySafe?(input?: Record<string, unknown>): boolean;
+  isReadOnly?(input?: Record<string, unknown>): boolean;
+  isDestructive?(input?: Record<string, unknown>): boolean;
+
+  // Execution control
+  interruptBehavior?(): 'cancel' | 'block';
+  maxResultSizeChars?: number;
+
+  // Validation & permissions
+  validateInput?(
+    input: Record<string, unknown>,
+    context: ToolContext<TContext, TData>
+  ): Promise<ToolValidationResult> | ToolValidationResult;
+
+  checkPermissions?(
+    input: Record<string, unknown>,
+    context: ToolContext<TContext, TData>
+  ): Promise<ToolPermissionResult> | ToolPermissionResult;
+}
+```
+
+## Methods & Properties
+
+### isConcurrencySafe
+
+Returns `true` if this tool can safely run in parallel with other concurrency-safe tools. The `StreamingToolExecutor` evaluates this once at queue time and caches the result.
+
+Default (when absent): `false` — the tool runs serially.
+
+```typescript
+const listFiles: EnhancedTool = {
+  id: "list-files",
+  name: "list_files",
+  description: "List files in a directory",
+  handler: async (ctx, args) => { /* ... */ },
+  isConcurrencySafe: () => true,
+};
+```
+
+The method receives the tool's input arguments, so concurrency safety can be input-dependent:
+
+```typescript
+isConcurrencySafe: (input) => {
+  // Safe for read paths, not safe for write paths
+  return input?.mode === "read";
+},
+```
+
+### isReadOnly / isDestructive
+
+Informational metadata. `isReadOnly` indicates the tool has no side effects; `isDestructive` indicates irreversible operations. Both default to `false` when absent.
+
+```typescript
+isReadOnly: () => true,
+isDestructive: () => false,
+```
+
+### interruptBehavior
+
+Controls how the tool responds to abort signals (sibling failure or parent cancellation):
+
+- `'cancel'` — immediately abort the tool
+- `'block'` — allow the tool to finish (default when absent)
+
+```typescript
+interruptBehavior: () => "cancel",
+```
+
+### maxResultSizeChars
+
+Maximum characters for the tool result. Results exceeding this limit are truncated with a notice like `[Truncated: 12000 chars total, showing first 5000]`.
+
+```typescript
+maxResultSizeChars: 50_000,
+```
+
+### validateInput
+
+Called before the tool handler. If it returns `{ valid: false }`, the handler is never invoked and a validation error is returned instead.
+
+```typescript
+validateInput: async (input, ctx) => {
+  if (!input.resourceId || typeof input.resourceId !== "string") {
+    return { valid: false, error: "resourceId must be a non-empty string" };
+  }
+  return { valid: true };
+},
+```
+
+The return type:
+
+```typescript
+interface ToolValidationResult {
+  valid: boolean;
+  error?: string;
+  correctedInput?: Record<string, unknown>;
+}
+```
+
+### checkPermissions
+
+Called before the tool handler (after validation). If it returns `{ allowed: false }`, the handler is never invoked and a permission error is returned.
+
+```typescript
+checkPermissions: async (input, ctx) => {
+  const role = (ctx.context as any)?.userRole;
+  if (role !== "admin") {
+    return { allowed: false, reason: "Only admins can delete resources", canOverride: false };
+  }
+  return { allowed: true };
+},
+```
+
+The return type:
+
+```typescript
+interface ToolPermissionResult {
+  allowed: boolean;
+  reason?: string;
+  canOverride?: boolean;
+}
+```
+
+## Full Example
+
+```typescript
+const deleteTool: EnhancedTool = {
+  id: "delete-resource",
+  name: "delete_resource",
+  description: "Delete a resource permanently",
+  parameters: {
+    type: "object",
+    properties: { resourceId: { type: "string" } },
+    required: ["resourceId"],
+  },
+  handler: async (ctx, args) => {
+    await deleteResource(args?.resourceId as string);
+    return { success: true };
+  },
+
+  isConcurrencySafe: () => false,
+  isReadOnly: () => false,
+  isDestructive: () => true,
+  interruptBehavior: () => "block",
+  maxResultSizeChars: 500,
+
+  validateInput: async (input) => {
+    if (!input.resourceId || typeof input.resourceId !== "string") {
+      return { valid: false, error: "resourceId must be a non-empty string" };
+    }
+    return { valid: true };
+  },
+
+  checkPermissions: async (input, ctx) => {
+    const role = (ctx.context as any)?.userRole;
+    if (role !== "admin") {
+      return { allowed: false, reason: "Only admins can delete resources" };
+    }
+    return { allowed: true };
+  },
+};
+```
+
+## Backward Compatibility
+
+Plain `Tool` objects without any `EnhancedTool` methods work exactly as before. The framework applies these defaults:
+
+| Property | Default |
+|---|---|
+| `isConcurrencySafe` | `false` |
+| `isReadOnly` | `false` |
+| `isDestructive` | `false` |
+| `interruptBehavior` | `'block'` |
+| `validateInput` | skipped |
+| `checkPermissions` | skipped |

package/docs/core/tools/streaming-execution.md

@@ -0,0 +1,161 @@
+# Streaming Tool Execution
+
+The `StreamingToolExecutor` executes tools as they arrive from the LLM stream rather than waiting for the full response. It provides concurrency control, abort handling, and ordered result yielding.
+
+## Overview
+
+- Tools begin executing immediately as they are parsed from the LLM stream
+- Read-only (concurrency-safe) tools run in parallel
+- Write (non-concurrency-safe) tools run serially with exclusive access
+- Results are always yielded in the original request order
+- Progress messages bypass ordering and are delivered immediately
+
+## Concurrency Control
+
+The executor enforces a strict invariant at all times:
+
+> Either **all** executing tools have `isConcurrencySafe === true`, **or** exactly **one** tool is executing with `isConcurrencySafe === false`.
+
+Tools without the `isConcurrencySafe` method default to `false` (serial execution), preserving backward compatibility with plain `Tool` objects.
+
+A configurable `maxParallel` limit (default: 10) caps the number of concurrently executing tools regardless of concurrency safety.
+
+### Example: Mixed Read/Write Tools
+
+```typescript
+import { Agent, EnhancedTool } from "@falai/agent";
+
+const readFile: EnhancedTool = {
+  id: "read-file",
+  name: "read_file",
+  description: "Read a file from disk",
+  parameters: {
+    type: "object",
+    properties: { path: { type: "string" } },
+    required: ["path"],
+  },
+  handler: async (ctx, args) => {
+    const content = await fs.readFile(args?.path as string, "utf-8");
+    return { data: content, success: true };
+  },
+  isConcurrencySafe: () => true,   // safe to run in parallel
+  isReadOnly: () => true,
+  maxResultSizeChars: 50_000,
+};
+
+const writeFile: EnhancedTool = {
+  id: "write-file",
+  name: "write_file",
+  description: "Write content to a file",
+  parameters: {
+    type: "object",
+    properties: {
+      path: { type: "string" },
+      content: { type: "string" },
+    },
+    required: ["path", "content"],
+  },
+  handler: async (ctx, args) => {
+    await fs.writeFile(args?.path as string, args?.content as string);
+    return { success: true };
+  },
+  isConcurrencySafe: () => false,  // must run exclusively
+  interruptBehavior: () => "block",
+};
+
+const agent = new Agent({
+  name: "CodeAssistant",
+  provider: anthropicProvider,
+  tools: [readFile, writeFile],
+});
+```
+
+When the LLM requests `read_file` three times followed by `write_file`, the three reads execute in parallel. Once all reads complete, the write executes alone.
+
+## Abort Behavior
+
+### Sibling Abort
+
+When a tool in a concurrent batch fails, all sibling tools in the same batch receive an abort signal. Each tool's `interruptBehavior` determines the response:
+
+- `'cancel'` — tool is immediately aborted
+- `'block'` (default) — tool is allowed to finish
+
+### Parent AbortSignal
+
+A parent `AbortSignal` can be passed via `StreamingToolExecutorOptions`. When it fires:
+
+1. Tools with `interruptBehavior() === 'cancel'` are aborted immediately
+2. Tools with `interruptBehavior() === 'block'` complete normally
+3. No new queued tools are started
+
+```typescript
+const controller = new AbortController();
+
+// Pass signal through agent options or directly to the executor
+for await (const chunk of agent.respondStream({
+  history,
+  signal: controller.signal,
+})) {
+  process.stdout.write(chunk.delta);
+}
+
+// Cancel from user action
+controller.abort();
+```
+
+## Progress Reporting
+
+Tools can emit progress messages during execution. These are yielded immediately to the caller without being buffered behind result ordering.
+
+```typescript
+for await (const chunk of agent.respondStream({ history })) {
+  if (chunk.toolExecution?.progress) {
+    console.log(`[progress] ${chunk.toolExecution.toolCallId}: ${chunk.toolExecution.progress}`);
+  }
+  if (chunk.toolExecution?.result) {
+    console.log(`[result] ${chunk.toolExecution.toolCallId}: done`);
+  }
+  process.stdout.write(chunk.delta);
+}
+```
+
+## Result Ordering
+
+Results are always yielded in the same order as the original tool call requests, regardless of actual completion order. If tool B finishes before tool A, tool B's result is buffered until tool A's result is yielded first.
+
+## API Reference
+
+### Constructor
+
+```typescript
+new StreamingToolExecutor<TContext, TData>(
+  toolContext: ToolContext<TContext, TData>,
+  options?: {
+    maxParallel?: number;   // default: 10
+    signal?: AbortSignal;   // parent abort signal
+  }
+)
+```
+
+### Methods
+
+| Method | Description |
+|---|---|
+| `addTool(toolCall, tool)` | Queue a tool for execution. Concurrency safety is evaluated once at queue time. |
+| `getCompletedResults()` | Synchronous generator yielding available results in request order. |
+| `getRemainingResults()` | Async generator yielding all results (waits for pending tools). |
+| `discard()` | Stop processing new queued tools. Running tools continue per their `interruptBehavior`. |
+| `getUpdatedContext()` | Return accumulated context updates from completed tools. |
+| `hasUnfinishedTools()` | `true` if any tools are still queued or executing. |
+
+### Default Behaviors for Plain `Tool` Objects
+
+| Property | Default |
+|---|---|
+| `isConcurrencySafe` | `false` |
+| `isReadOnly` | `false` |
+| `isDestructive` | `false` |
+| `interruptBehavior` | `'block'` |
+
+Plain `Tool` objects work without modification — they execute serially and are allowed to complete on abort.

package/docs/guides/context-compaction.md

@@ -0,0 +1,96 @@
+# Context Compaction
+
+The `CompactionEngine` automatically manages conversation history size when approaching token limits. It applies multi-layered strategies in order of cost, from cheap truncation to LLM-powered summarization.
+
+## Compaction Strategies
+
+Strategies are applied in order until the history fits within the token budget:
+
+| Strategy | Cost | Description |
+|---|---|---|
+| `none` | Free | History is under threshold — no action taken |
+| `tool_result_budget` | Free | Truncate oversized tool results with a notice |
+| `micro_compact` | Free | Collapse whitespace in verbose tool outputs |
+| `auto_compact` | LLM call | Summarize old messages via the configured AI provider |
+
+If the LLM summarization fails, the engine falls back to aggressive truncation (removing oldest messages) and logs a warning. The next compaction attempt will retry summarization.
+
+## Configuration
+
+Compaction is configured at the agent level via the `compaction` option:
+
+```typescript
+import { Agent } from "@falai/agent";
+
+const agent = new Agent({
+  name: "LongConversationAgent",
+  provider: anthropicProvider,
+  compaction: {
+    maxTokens: 100_000,
+    compactionThreshold: 0.8,    // trigger at 80% of budget
+    preserveRecentCount: 10,     // always keep last 10 messages
+    maxToolResultChars: 5_000,   // truncate tool results over 5k chars
+    provider: anthropicProvider, // provider for LLM summarization
+  },
+});
+```
+
+### CompactionOptions
+
+| Option | Type | Constraint | Description |
+|---|---|---|---|
+| `maxTokens` | `number` | > 0 | Maximum token budget for the conversation |
+| `compactionThreshold` | `number` | 0.5 – 0.95 | Ratio at which compaction triggers |
+| `preserveRecentCount` | `number` | ≥ 2 | Recent messages that are never modified |
+| `maxToolResultChars` | `number` | > 0 | Per-tool-result character limit before truncation |
+| `provider` | `AiProvider` | — | Provider used for LLM summarization |
+
+Invalid options throw at construction time.
+
+## How It Works
+
+When the `SessionManager` detects that estimated tokens exceed `maxTokens * compactionThreshold`, the `CompactionEngine` runs:
+
+1. **Token estimation** — character-based heuristic (~4 chars/token), no external tokenizer needed
+2. **Tool result budget** — truncate any tool result exceeding `maxToolResultChars`, append a notice like `[Truncated: 12000 chars total, showing first 5000]`
+3. **Micro-compact** — collapse whitespace in tool outputs for the compactable portion of history
+4. **Auto-compact** — summarize old messages via the AI provider, replacing them with a `[Conversation Summary]` system message
+
+The last `preserveRecentCount` messages are never modified or removed by any strategy.
+
+## Manual Compaction
+
+You can also use the `CompactionEngine` directly:
+
+```typescript
+import { CompactionEngine } from "@falai/agent";
+
+const result = await CompactionEngine.checkAndCompact(history, {
+  maxTokens: 100_000,
+  compactionThreshold: 0.8,
+  preserveRecentCount: 10,
+  maxToolResultChars: 5_000,
+  provider: anthropicProvider,
+});
+
+console.log(result.strategy);        // 'none' | 'tool_result_budget' | 'micro_compact' | 'auto_compact'
+console.log(result.estimatedTokens); // tokens after compaction
+console.log(result.messagesCompacted);
+```
+
+### Standalone Utilities
+
+```typescript
+// Estimate tokens for a history
+const tokens = CompactionEngine.estimateTokens(history);
+
+// Truncate tool results only
+const budgeted = CompactionEngine.applyToolResultBudget(history, 5_000);
+```
+
+## Key Properties
+
+- **Idempotent** — compacting already-compacted history with the same options produces the same result
+- **Deterministic estimation** — `estimateTokens` always returns the same value for the same input
+- **Preservation guarantee** — the last `preserveRecentCount` messages are never touched
+- **Graceful degradation** — LLM failure falls back to truncation, never crashes