recursive-llm-ts 4.5.0 → 4.7.0

package/README.md

@@ -14,6 +14,7 @@ TypeScript/JavaScript package for [Recursive Language Models (RLM)](https://gith
 
 **Performance & Resilience**
 - **Pure Go Backend** - 50x faster startup, 3x less memory vs Python
+- **Context Overflow Recovery** - Automatic detection and 6 reduction strategies (mapreduce, truncate, chunked, tfidf, textrank, refine)
 - **Caching** - Exact-match caching with in-memory and file-based backends
 - **Retry & Fallback** - Exponential backoff, jitter, and multi-provider fallback chains
 - **AbortController** - Cancel any operation mid-flight
@@ -31,7 +32,7 @@ TypeScript/JavaScript package for [Recursive Language Models (RLM)](https://gith
 - **Meta-Agent Mode** - Automatically optimize queries for better results
 - **Observability** - OpenTelemetry tracing, Langfuse integration, and debug logging
 - **File Storage** - Process local directories or S3/MinIO/LocalStack buckets as LLM context
-- **120+ Tests** - Comprehensive Vitest test suite
+- **150+ Tests** - Comprehensive Vitest + Go test suites
 
 ## Installation
 
@@ -281,12 +282,18 @@ const results = await rlm.batchCompletion([
 Rich error hierarchy with actionable information:
 
 ```typescript
-import { RLMRateLimitError, RLMValidationError, RLMTimeoutError } from 'recursive-llm-ts';
+import {
+  RLMRateLimitError, RLMValidationError,
+  RLMTimeoutError, RLMContextOverflowError
+} from 'recursive-llm-ts';
 
 try {
   const result = await rlm.completion(query, context);
 } catch (err) {
-  if (err instanceof RLMRateLimitError) {
+  if (err instanceof RLMContextOverflowError) {
+    console.log(`Context overflow: ${err.requestTokens} tokens > ${err.modelLimit} limit`);
+    // Enable context_overflow config to auto-recover from this
+  } else if (err instanceof RLMRateLimitError) {
     console.log(`Rate limited. Retry after: ${err.retryAfter}s`);
   } else if (err instanceof RLMValidationError) {
     console.log(`Schema mismatch:`, err.zodErrors);
@@ -297,6 +304,57 @@ try {
 }
 ```
 
+### Context Overflow Handling
+
+Automatically detect and recover from context window overflows. When your input exceeds the model's token limit, RLM catches the error and applies a reduction strategy to fit the context within bounds.
+
+```typescript
+const rlm = new RLM('gpt-4o-mini', {
+  api_key: process.env.OPENAI_API_KEY,
+  context_overflow: {
+    enabled: true,           // Enable overflow recovery (default: true)
+    strategy: 'tfidf',       // Reduction strategy (see table below)
+    max_model_tokens: 32768, // Override auto-detected limit (optional)
+    safety_margin: 0.15,     // Reserve 15% for prompts/overhead (default: 0.15)
+    max_reduction_attempts: 3, // Max retry attempts (default: 3)
+  }
+});
+
+// Process a document that may exceed the model's context window
+const result = await rlm.completion(
+  'Summarize the key findings',
+  veryLargeDocument  // If too large, auto-reduces and retries
+);
+```
+
+**Builder API:**
+```typescript
+const rlm = RLM.builder('gpt-4o-mini')
+  .apiKey(process.env.OPENAI_API_KEY!)
+  .withContextOverflow({ strategy: 'textrank', max_model_tokens: 32768 })
+  .build();
+```
+
+**Strategy Comparison:**
+
+| Strategy | API Calls | Speed | Quality | Best For |
+|----------|-----------|-------|---------|----------|
+| `mapreduce` | Many (parallel) | Medium | High | General-purpose, large documents |
+| `truncate` | 0 | Fastest | Low | Quick-and-dirty, when beginning of doc matters |
+| `chunked` | Many (sequential) | Slow | High | Detailed extraction from specific sections |
+| `tfidf` | 0 | Fast | Medium | Fast first pass, keyword-rich documents |
+| `textrank` | 0 | Fast | Medium-High | Documents with clear sentence structure |
+| `refine` | Many (sequential) | Slow | Highest | When quality matters most, iterative refinement |
+
+**Strategy Details:**
+
+- **`mapreduce`** (default) - Splits context into chunks, summarizes each in parallel via LLM calls, then merges summaries. Good balance of quality and speed.
+- **`truncate`** - Drops tokens from the end to fit the budget. Zero API calls, but loses information. Best when the beginning of the document is most important.
+- **`chunked`** - Processes chunks sequentially, extracting relevant content from each. Higher quality than mapreduce for targeted extraction.
+- **`tfidf`** - Pure Go, zero API calls. Uses TF-IDF scoring to select the most informative sentences. Preserves original document order. Great for a fast, no-cost first pass.
+- **`textrank`** - Pure Go, zero API calls. Graph-based sentence ranking using PageRank over cosine-similarity of TF-IDF vectors. Better at identifying structurally important sentences than plain TF-IDF.
+- **`refine`** - Sequential iterative refinement. Processes chunks one at a time, building and refining an answer progressively. Highest quality but slowest, as each chunk sees the accumulated context.
+
 ### Config Validation
 
 Catch configuration issues at construction time:
@@ -739,7 +797,10 @@ interface RLMConfig {
   temperature?: number;          // Sampling temperature
   max_tokens?: number;           // Maximum tokens in response
 
-  // New in v5: Caching, retry, fallback
+  // Context overflow recovery
+  context_overflow?: ContextOverflowConfig;
+
+  // Caching, retry, fallback
   cache?: CacheConfig;           // Cache configuration
   retry?: RetryConfig;           // Retry configuration
   fallback?: FallbackConfig;     // Fallback model configuration
@@ -768,6 +829,14 @@ interface FallbackConfig {
   strategy?: 'sequential';      // Fallback strategy
 }
 
+interface ContextOverflowConfig {
+  enabled?: boolean;             // Enable overflow recovery (default: true)
+  max_model_tokens?: number;     // Override auto-detected model limit (0 = auto-detect)
+  strategy?: 'mapreduce' | 'truncate' | 'chunked' | 'tfidf' | 'textrank' | 'refine';
+  safety_margin?: number;        // Fraction to reserve for overhead (default: 0.15)
+  max_reduction_attempts?: number; // Max reduction retries (default: 3)
+}
+
 interface MetaAgentConfig {
   enabled: boolean;              // Enable the meta-agent
   model?: string;                // Model for query optimization (defaults to main model)
@@ -848,6 +917,7 @@ class RLMTimeoutError extends RLMError { elapsed; limit; }
 class RLMProviderError extends RLMError { statusCode; provider; }
 class RLMBinaryError extends RLMError { binaryPath; }
 class RLMConfigError extends RLMError { field; value; }
+class RLMContextOverflowError extends RLMError { modelLimit; requestTokens; }
 class RLMSchemaError extends RLMError { path; constraint; }
 class RLMAbortError extends RLMError {}
 ```

package/dist/bridge-interface.d.ts

@@ -36,6 +36,18 @@ export interface TraceEvent {
     span_id?: string;
     parent_id?: string;
 }
+export interface ContextOverflowConfig {
+    /** Enable automatic context overflow recovery (default: true) */
+    enabled?: boolean;
+    /** Override detected model token limit (0 = auto-detect from API errors) */
+    max_model_tokens?: number;
+    /** Strategy: 'mapreduce' (default), 'truncate', 'chunked', 'tfidf', 'textrank', or 'refine' */
+    strategy?: 'mapreduce' | 'truncate' | 'chunked' | 'tfidf' | 'textrank' | 'refine';
+    /** Fraction of token budget to reserve for prompts/overhead (default: 0.15) */
+    safety_margin?: number;
+    /** Maximum reduction attempts before giving up (default: 3) */
+    max_reduction_attempts?: number;
+}
 export interface RLMConfig {
     recursive_model?: string;
     api_base?: string;
@@ -46,12 +58,14 @@ export interface RLMConfig {
     go_binary_path?: string;
     meta_agent?: MetaAgentConfig;
     observability?: ObservabilityConfig;
+    context_overflow?: ContextOverflowConfig;
     debug?: boolean;
     api_version?: string;
     timeout?: number;
     temperature?: number;
     max_tokens?: number;
     structured?: any;
+    [key: string]: any;
 }
 export interface FileStorageConfig {
     /** Storage type: 'local' or 's3' */

package/dist/errors.d.ts

@@ -89,6 +89,16 @@ export declare class RLMSchemaError extends RLMError {
         constraint: string;
     });
 }
+/** Thrown when the request exceeds the model's context window. */
+export declare class RLMContextOverflowError extends RLMError {
+    readonly modelLimit: number;
+    readonly requestTokens: number;
+    constructor(opts: {
+        message: string;
+        modelLimit: number;
+        requestTokens: number;
+    });
+}
 /** Thrown when an operation is aborted via AbortController. */
 export declare class RLMAbortError extends RLMError {
     constructor(message?: string);

package/dist/errors.js

@@ -8,7 +8,7 @@
  * - `suggestion` – human-readable remediation hint
  */
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.RLMAbortError = exports.RLMSchemaError = exports.RLMConfigError = exports.RLMBinaryError = exports.RLMProviderError = exports.RLMTimeoutError = exports.RLMRateLimitError = exports.RLMValidationError = exports.RLMError = void 0;
+exports.RLMAbortError = exports.RLMContextOverflowError = exports.RLMSchemaError = exports.RLMConfigError = exports.RLMBinaryError = exports.RLMProviderError = exports.RLMTimeoutError = exports.RLMRateLimitError = exports.RLMValidationError = exports.RLMError = void 0;
 exports.classifyError = classifyError;
 // ─── Base Error ──────────────────────────────────────────────────────────────
 class RLMError extends Error {
@@ -132,6 +132,22 @@ class RLMSchemaError extends RLMError {
     }
 }
 exports.RLMSchemaError = RLMSchemaError;
+// ─── Context Overflow ─────────────────────────────────────────────────────────
+/** Thrown when the request exceeds the model's context window. */
+class RLMContextOverflowError extends RLMError {
+    constructor(opts) {
+        super(opts.message, {
+            code: 'CONTEXT_OVERFLOW',
+            retryable: true,
+            suggestion: `Request has ${opts.requestTokens} tokens but model limit is ${opts.modelLimit}. ` +
+                'Enable context_overflow handling or reduce your context size.',
+        });
+        this.name = 'RLMContextOverflowError';
+        this.modelLimit = opts.modelLimit;
+        this.requestTokens = opts.requestTokens;
+    }
+}
+exports.RLMContextOverflowError = RLMContextOverflowError;
 // ─── Abort ───────────────────────────────────────────────────────────────────
 /** Thrown when an operation is aborted via AbortController. */
 class RLMAbortError extends RLMError {
@@ -153,6 +169,14 @@ exports.RLMAbortError = RLMAbortError;
 function classifyError(err, context) {
     var _a;
     const msg = typeof err === 'string' ? err : err.message;
+    // Context overflow
+    if (msg.toLowerCase().includes('maximum context length') || msg.toLowerCase().includes('context_length_exceeded') || msg.toLowerCase().includes('too many input tokens')) {
+        const limitMatch = msg.match(/maximum context length is (\d[\d,]*)/i);
+        const requestMatch = msg.match(/(?:has|requested) (\d[\d,]*)\s*(?:input )?tokens/i);
+        const modelLimit = limitMatch ? parseInt(limitMatch[1].replace(/,/g, ''), 10) : 0;
+        const requestTokens = requestMatch ? parseInt(requestMatch[1].replace(/,/g, ''), 10) : 0;
+        return new RLMContextOverflowError({ message: msg, modelLimit, requestTokens });
+    }
     // Rate limit
     if (msg.includes('429') || msg.toLowerCase().includes('rate limit') || msg.toLowerCase().includes('too many requests')) {
         const retryMatch = msg.match(/retry.after[:\s]+(\d+)/i);

package/dist/index.d.ts

@@ -1,9 +1,9 @@
 export { RLM, RLMBuilder, RLMCompletionResult, RLMResultFormatter } from './rlm';
-export { RLMConfig, RLMResult, RLMStats, MetaAgentConfig, ObservabilityConfig, TraceEvent, FileStorageConfig } from './bridge-interface';
+export { RLMConfig, RLMResult, RLMStats, MetaAgentConfig, ObservabilityConfig, TraceEvent, FileStorageConfig, ContextOverflowConfig } from './bridge-interface';
 export { BridgeType } from './bridge-factory';
 export { StructuredRLMResult, SubTask, CoordinatorConfig, SchemaDecomposition } from './structured-types';
 export { RLMExtendedConfig, ValidationResult, ValidationIssue, ValidationLevel, validateConfig, assertValidConfig } from './config';
-export { RLMError, RLMValidationError, RLMRateLimitError, RLMTimeoutError, RLMProviderError, RLMBinaryError, RLMConfigError, RLMSchemaError, RLMAbortError, classifyError, } from './errors';
+export { RLMError, RLMValidationError, RLMRateLimitError, RLMTimeoutError, RLMProviderError, RLMBinaryError, RLMConfigError, RLMSchemaError, RLMContextOverflowError, RLMAbortError, classifyError, } from './errors';
 export { RLMStream, StreamOptions, StreamChunk, StreamChunkType, TextStreamChunk, PartialObjectStreamChunk, UsageStreamChunk, ErrorStreamChunk, DoneStreamChunk, createSimulatedStream, } from './streaming';
 export { RLMCache, CacheConfig, CacheStats, CacheProvider, MemoryCache, FileCache } from './cache';
 export { RetryConfig, FallbackConfig, withRetry, withFallback } from './retry';

package/dist/index.js

@@ -1,6 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.buildFileContext = exports.S3StorageError = exports.S3FileStorage = exports.LocalFileStorage = exports.FileContextBuilder = exports.RLMAgentCoordinator = exports.RLMEventEmitter = exports.withFallback = exports.withRetry = exports.FileCache = exports.MemoryCache = exports.RLMCache = exports.createSimulatedStream = exports.RLMStream = exports.classifyError = exports.RLMAbortError = exports.RLMSchemaError = exports.RLMConfigError = exports.RLMBinaryError = exports.RLMProviderError = exports.RLMTimeoutError = exports.RLMRateLimitError = exports.RLMValidationError = exports.RLMError = exports.assertValidConfig = exports.validateConfig = exports.RLMResultFormatter = exports.RLMBuilder = exports.RLM = void 0;
+exports.buildFileContext = exports.S3StorageError = exports.S3FileStorage = exports.LocalFileStorage = exports.FileContextBuilder = exports.RLMAgentCoordinator = exports.RLMEventEmitter = exports.withFallback = exports.withRetry = exports.FileCache = exports.MemoryCache = exports.RLMCache = exports.createSimulatedStream = exports.RLMStream = exports.classifyError = exports.RLMAbortError = exports.RLMContextOverflowError = exports.RLMSchemaError = exports.RLMConfigError = exports.RLMBinaryError = exports.RLMProviderError = exports.RLMTimeoutError = exports.RLMRateLimitError = exports.RLMValidationError = exports.RLMError = exports.assertValidConfig = exports.validateConfig = exports.RLMResultFormatter = exports.RLMBuilder = exports.RLM = void 0;
 // ─── Core ────────────────────────────────────────────────────────────────────
 var rlm_1 = require("./rlm");
 Object.defineProperty(exports, "RLM", { enumerable: true, get: function () { return rlm_1.RLM; } });
@@ -19,6 +19,7 @@ Object.defineProperty(exports, "RLMProviderError", { enumerable: true, get: func
 Object.defineProperty(exports, "RLMBinaryError", { enumerable: true, get: function () { return errors_1.RLMBinaryError; } });
 Object.defineProperty(exports, "RLMConfigError", { enumerable: true, get: function () { return errors_1.RLMConfigError; } });
 Object.defineProperty(exports, "RLMSchemaError", { enumerable: true, get: function () { return errors_1.RLMSchemaError; } });
+Object.defineProperty(exports, "RLMContextOverflowError", { enumerable: true, get: function () { return errors_1.RLMContextOverflowError; } });
 Object.defineProperty(exports, "RLMAbortError", { enumerable: true, get: function () { return errors_1.RLMAbortError; } });
 Object.defineProperty(exports, "classifyError", { enumerable: true, get: function () { return errors_1.classifyError; } });
 // ─── Streaming ───────────────────────────────────────────────────────────────

package/dist/rlm.d.ts

@@ -13,7 +13,7 @@
  * console.log(result.result);
  * ```
  */
-import { RLMConfig, RLMResult, RLMStats, TraceEvent, FileStorageConfig } from './bridge-interface';
+import { RLMConfig, RLMResult, RLMStats, TraceEvent, FileStorageConfig, ContextOverflowConfig } from './bridge-interface';
 import { BridgeType } from './bridge-factory';
 import { z } from 'zod';
 import { StructuredRLMResult } from './structured-types';
@@ -89,6 +89,8 @@ export declare class RLMBuilder {
     withFallback(models: string[]): this;
     /** Set the bridge type */
     bridge(type: BridgeType): this;
+    /** Configure context overflow recovery */
+    withContextOverflow(config?: ContextOverflowConfig): this;
     /** Set the Go binary path */
     binaryPath(path: string): this;
     /** Add LiteLLM passthrough parameters */

package/dist/rlm.js

@@ -170,6 +170,11 @@ class RLMBuilder {
         this.bridgeType = type;
         return this;
     }
+    /** Configure context overflow recovery */
+    withContextOverflow(config) {
+        this.config.context_overflow = Object.assign({ enabled: true }, config);
+        return this;
+    }
     /** Set the Go binary path */
     binaryPath(path) {
         this.config.go_binary_path = path;

package/go/README.md

@@ -159,6 +159,11 @@ All fields in `config` are optional and have defaults:
 | `max_iterations` | int | 30 | Maximum REPL iterations per call |
 | `temperature` | float | 0.7 | LLM temperature (0-2) |
 | `timeout` | int | 60 | HTTP timeout in seconds |
+| `context_overflow.enabled` | bool | true | Enable context overflow recovery |
+| `context_overflow.strategy` | string | `mapreduce` | Reduction strategy: mapreduce, truncate, chunked, tfidf, textrank, refine |
+| `context_overflow.max_model_tokens` | int | 0 (auto) | Override detected model token limit |
+| `context_overflow.safety_margin` | float | 0.15 | Fraction reserved for prompt overhead |
+| `context_overflow.max_reduction_attempts` | int | 3 | Max retry attempts |
 
 Any other fields in `config` are passed as extra parameters to the LLM API.
 
@@ -258,7 +263,10 @@ rlm/                         # Public package (importable)
 ├── prompt.go                # System prompt builder
 ├── repl.go                  # JavaScript REPL (goja)
 ├── openai.go                # OpenAI API client
-└── errors.go                # Error types
+├── errors.go                # Error types
+├── context_overflow.go      # Context overflow detection + 6 reduction strategies
+├── tfidf.go                 # TF-IDF extractive compression (pure Go)
+└── textrank.go              # TextRank graph-based ranking with PageRank
 ```
 
 ## Error Handling