genai-lite 0.3.2 → 0.4.0

package/README.md

@@ -1,10 +1,11 @@
 # genai-lite
 
-A lightweight, portable Node.js/TypeScript library providing a unified interface for interacting with multiple Generative AI providers (OpenAI, Anthropic, Google Gemini, Mistral, and more).
+A lightweight, portable Node.js/TypeScript library providing a unified interface for interacting with multiple Generative AI providers—both cloud-based (OpenAI, Anthropic, Google Gemini, Mistral) and local (llama.cpp).
 
 ## Features
 
 - 🔌 **Unified API** - Single interface for multiple AI providers
+- 🏠 **Local & Cloud Models** - Run models locally with llama.cpp or use cloud APIs
 - 🔐 **Flexible API Key Management** - Bring your own key storage solution
 - 📦 **Zero Electron Dependencies** - Works in any Node.js environment
 - 🎯 **TypeScript First** - Full type safety and IntelliSense support
@@ -21,13 +22,14 @@ npm install genai-lite
 
 ## Quick Start
 
+### Cloud Providers (OpenAI, Anthropic, Gemini, Mistral)
+
 ```typescript
 import { LLMService, fromEnvironment } from 'genai-lite';
 
 // Create service with environment variable API key provider
 const llmService = new LLMService(fromEnvironment);
 
-// Option 1: Direct message sending
 const response = await llmService.sendMessage({
   providerId: 'openai',
   modelId: 'gpt-4.1-mini',
@@ -37,26 +39,47 @@ const response = await llmService.sendMessage({
   ]
 });
 
-// Option 2: Create messages from template (recommended for complex prompts)
-const { messages } = await llmService.createMessages({
-  template: '<SYSTEM>You are a helpful assistant.</SYSTEM><USER>Hello, how are you?</USER>',
-  providerId: 'openai',
-  modelId: 'gpt-4.1-mini'
-});
+if (response.object === 'chat.completion') {
+  console.log(response.choices[0].message.content);
+} else {
+  console.error('Error:', response.error.message);
+}
+```
 
-const response2 = await llmService.sendMessage({
-  providerId: 'openai',
-  modelId: 'gpt-4.1-mini',
-  messages
+### Local Models (llama.cpp)
+
+```typescript
+import { LLMService } from 'genai-lite';
+
+// Start llama.cpp server first: llama-server -m /path/to/model.gguf --port 8080
+const llmService = new LLMService(async () => 'not-needed');
+
+const response = await llmService.sendMessage({
+  providerId: 'llamacpp',
+  modelId: 'llama-3-8b-instruct',  // Must match your loaded model
+  messages: [
+    { role: 'system', content: 'You are a helpful assistant.' },
+    { role: 'user', content: 'Explain quantum computing briefly.' }
+  ]
 });
 
 if (response.object === 'chat.completion') {
   console.log(response.choices[0].message.content);
-} else {
-  console.error('Error:', response.error.message);
 }
 ```
 
+See the [llama.cpp Integration](#llamacpp-integration) section for setup details.
+
+## Example Application
+
+For a complete, production-ready example showcasing all genai-lite capabilities, see the **[chat-demo](examples/chat-demo)** interactive web application. The demo includes:
+- Multi-provider chat interface with all supported providers
+- Template rendering and model presets
+- llama.cpp utilities (tokenization, embeddings, health checks)
+- Settings persistence, export/import features
+
+The chat-demo serves as both a comprehensive showcase and a quick-test environment for library changes.
+
 ## API Key Management
 
 genai-lite uses a flexible API key provider pattern. You can use the built-in environment variable provider or create your own:
@@ -124,6 +147,64 @@ const llmService = new LLMService(myKeyProvider);
 - `codestral-2501` - Specialized for code generation
 - `devstral-small-2505` - Compact development-focused model
 
+### llama.cpp (Local Models)
+
+Run models locally via [llama.cpp](https://github.com/ggml-org/llama.cpp) server. Model IDs can be any name—they're not validated since you load your own GGUF models.
+
+**Example models:**
+- `llama-3-8b-instruct` - Llama 3 8B Instruct
+- `llama-3-70b-instruct` - Llama 3 70B Instruct
+- `mistral-7b-instruct` - Mistral 7B Instruct
+- `my-custom-model` - Any custom model you've loaded
+
+**Setup:**
+
+1. Start llama.cpp server with your model:
+```bash
+llama-server -m /path/to/model.gguf --port 8080
+```
+
+2. Use with genai-lite (no API key needed):
+```typescript
+import { LLMService } from 'genai-lite';
+
+// API key can be any string for llama.cpp
+const service = new LLMService(async () => 'not-needed');
+
+const response = await service.sendMessage({
+  providerId: 'llamacpp',
+  modelId: 'llama-3-8b-instruct', // Must match your loaded model name
+  messages: [{ role: 'user', content: 'Hello!' }]
+});
+```
+
+3. Configure server URL via environment variable:
+```bash
+export LLAMACPP_API_BASE_URL=http://localhost:8080
+```
+
+**Advanced features** - Access non-LLM endpoints:
+
+```typescript
+import { LlamaCppServerClient } from 'genai-lite';
+
+const client = new LlamaCppServerClient('http://localhost:8080');
+
+// Check server health
+const health = await client.getHealth();
+
+// Tokenize text
+const { tokens } = await client.tokenize('Hello world');
+
+// Generate embeddings
+const { embedding } = await client.createEmbedding('Some text');
+
+// Code completion
+const result = await client.infill('def hello():\n', '\nprint("done")');
+```
+
+See the [llama.cpp Integration](#llamacpp-integration) section for details.
+
 ### Models with Reasoning Support
 
 Some models include advanced reasoning/thinking capabilities that enhance their problem-solving abilities:
@@ -268,7 +349,7 @@ The `onMissing` property controls what happens when the expected thinking tag is
 
 - `'ignore'`: Silently continue without the tag
 - `'warn'`: Log a warning but continue processing
-- `'error'`: Return an error response
+- `'error'`: Return an error response with the original response preserved in `partialResponse`
 - `'auto'` (default): Intelligently decide based on the model's native reasoning capabilities
 
 **How `'auto'` Mode Works:**
@@ -290,6 +371,7 @@ const response = await llmService.sendMessage({
   }
 });
 // Result: ERROR if <thinking> tag is missing (strict enforcement)
+// The response is still accessible via errorResponse.partialResponse
 
 // With native reasoning models (e.g., Claude with reasoning enabled)
 const response = await llmService.sendMessage({
@@ -654,6 +736,10 @@ if (response.object === 'error') {
       break;
     case 'validation_error':
       console.error('Invalid request:', response.error.message);
+      // For validation errors, the response may still be available
+      if (response.partialResponse) {
+        console.log('Partial response:', response.partialResponse.choices[0].message.content);
+      }
       break;
     default:
       console.error('Error:', response.error.message);
@@ -661,6 +747,261 @@ if (response.object === 'error') {
 }
 ```
 
+## llama.cpp Integration
+
+`genai-lite` provides comprehensive support for running local LLMs via [llama.cpp](https://github.com/ggml-org/llama.cpp) server, enabling completely offline AI capabilities with the same unified interface.
+
+### Why llama.cpp?
+
+- **Privacy**: All model inference runs locally on your hardware
+- **Cost**: No API costs after initial model download
+- **Control**: Use any GGUF model from Hugging Face
+- **Performance**: Optimized C++ implementation with hardware acceleration
+
+### Setup
+
+#### 1. Install llama.cpp
+
+```bash
+# Clone and build llama.cpp
+git clone https://github.com/ggml-org/llama.cpp
+cd llama.cpp
+make
+
+# Or download pre-built binaries from releases
+```
+
+#### 2. Download a Model
+
+Get GGUF models from Hugging Face, for example:
+- [Meta-Llama-3.1-8B-Instruct-GGUF](https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF)
+- [Mistral-7B-Instruct-v0.3-GGUF](https://huggingface.co/bartowski/Mistral-7B-Instruct-v0.3-GGUF)
+
+#### 3. Start the Server
+
+```bash
+# Basic usage
+llama-server -m /path/to/model.gguf --port 8080
+
+# With more options
+llama-server -m /path/to/model.gguf \
+  --port 8080 \
+  -c 4096 \           # Context size
+  -np 4 \             # Parallel requests
+  --threads 8         # CPU threads
+```
+
+### Basic Usage
+
+```typescript
+import { LLMService } from 'genai-lite';
+
+// llama.cpp doesn't need API keys
+const service = new LLMService(async () => 'not-needed');
+
+const response = await service.sendMessage({
+  providerId: 'llamacpp',
+  modelId: 'llama-3-8b-instruct',  // Arbitrary name matching your model
+  messages: [
+    { role: 'system', content: 'You are a helpful assistant.' },
+    { role: 'user', content: 'Explain quantum computing in simple terms.' }
+  ],
+  settings: {
+    temperature: 0.7,
+    maxTokens: 500
+  }
+});
+
+if (response.object === 'chat.completion') {
+  console.log(response.choices[0].message.content);
+}
+```
+
+### Configuration
+
+#### Environment Variable
+
+Set the server URL via environment variable (default: `http://localhost:8080`):
+
+```bash
+export LLAMACPP_API_BASE_URL=http://localhost:8080
+```
+
+#### Multiple Servers
+
+Register multiple llama.cpp instances for different models:
+
+```typescript
+import { LLMService, LlamaCppClientAdapter } from 'genai-lite';
+
+const service = new LLMService(async () => 'not-needed');
+
+// Register adapters for different servers/models
+service.registerAdapter(
+  'llamacpp-small',
+  new LlamaCppClientAdapter({ baseURL: 'http://localhost:8080' })
+);
+
+service.registerAdapter(
+  'llamacpp-large',
+  new LlamaCppClientAdapter({ baseURL: 'http://localhost:8081' })
+);
+
+// Use them
+const response = await service.sendMessage({
+  providerId: 'llamacpp-small',
+  modelId: 'llama-3-8b',
+  messages: [{ role: 'user', content: 'Hello!' }]
+});
+```
+
+#### Health Checking
+
+Enable automatic health checks before requests:
+
+```typescript
+import { LlamaCppClientAdapter } from 'genai-lite';
+
+const adapter = new LlamaCppClientAdapter({
+  baseURL: 'http://localhost:8080',
+  checkHealth: true  // Check server status before each request
+});
+
+service.registerAdapter('llamacpp', adapter);
+```
+
+### Advanced Features
+
+#### Server Management
+
+The `LlamaCppServerClient` class provides access to all llama.cpp server endpoints:
+
+```typescript
+import { LlamaCppServerClient } from 'genai-lite';
+
+const client = new LlamaCppServerClient('http://localhost:8080');
+
+// Health monitoring
+const health = await client.getHealth();
+console.log(health.status); // 'ok', 'loading', or 'error'
+
+// Server properties
+const props = await client.getProps();
+console.log(props.total_slots); // Number of available slots
+
+// Performance metrics (if enabled)
+const metrics = await client.getMetrics();
+```
+
+#### Tokenization
+
+```typescript
+const client = new LlamaCppServerClient('http://localhost:8080');
+
+// Tokenize text
+const { tokens } = await client.tokenize('Hello, world!');
+console.log(tokens); // [123, 456, 789]
+
+// Count tokens before sending to LLM
+const prompt = 'Long text...';
+const { tokens: promptTokens } = await client.tokenize(prompt);
+if (promptTokens.length > 4000) {
+  console.log('Prompt too long, truncating...');
+}
+
+// Detokenize back to text
+const { content } = await client.detokenize([123, 456, 789]);
+console.log(content); // 'Hello, world!'
+```
+
+#### Text Embeddings
+
+```typescript
+const client = new LlamaCppServerClient('http://localhost:8080');
+
+// Generate embeddings for semantic search
+const { embedding } = await client.createEmbedding('Search query text');
+console.log(embedding.length); // e.g., 768 dimensions
+
+// With images (for multimodal models)
+const { embedding: multimodalEmbed } = await client.createEmbedding(
+  'Describe this image',
+  'base64_image_data_here'
+);
+```
+
+#### Code Infilling
+
+Perfect for code completion in IDEs:
+
+```typescript
+const client = new LlamaCppServerClient('http://localhost:8080');
+
+const result = await client.infill(
+  'def calculate_fibonacci(n):\n    ',  // Prefix (before cursor)
+  '\n    return result'                   // Suffix (after cursor)
+);
+
+console.log(result.content);
+// Output: "if n <= 1:\n        return n\n    result = calculate_fibonacci(n-1) + calculate_fibonacci(n-2)"
+```
+
+### Error Handling
+
+```typescript
+const response = await service.sendMessage({
+  providerId: 'llamacpp',
+  modelId: 'my-model',
+  messages: [{ role: 'user', content: 'Hello' }]
+});
+
+if (response.object === 'error') {
+  switch (response.error.code) {
+    case 'NETWORK_ERROR':
+      console.error('Server not running or unreachable');
+      break;
+    case 'PROVIDER_ERROR':
+      console.error('Server error:', response.error.message);
+      break;
+    default:
+      console.error('Unknown error:', response.error);
+  }
+}
+```
+
+### Best Practices
+
+1. **Model Naming**: Use descriptive model IDs (e.g., `llama-3-8b-instruct`) since llama.cpp accepts any name
+2. **Context Size**: Set appropriate context (`-c` flag) when starting the server
+3. **Parallel Requests**: Configure slots (`-np`) based on your hardware
+4. **Health Monitoring**: Enable `checkHealth` for production to detect server issues early
+5. **Resource Management**: Monitor memory usage; large models need significant RAM
+
+### Troubleshooting
+
+**Server not responding:**
+```bash
+# Check if server is running
+curl http://localhost:8080/health
+
+# Should return: {"status":"ok"}
+```
+
+**Model loading errors:**
+```bash
+# Increase memory or reduce context size
+llama-server -m model.gguf --port 8080 -c 2048
+```
+
+**Slow responses:**
+```bash
+# Use quantized models (smaller but faster)
+# e.g., Q4_K_M, Q5_K_M instead of F16
+
+# Increase threads
+llama-server -m model.gguf --threads 16
+```
+
 ## Using with Electron
 
 `genai-lite` is designed to work seamlessly within an Electron application's main process, especially when paired with a secure storage solution like `genai-key-storage-lite`.
@@ -720,6 +1061,26 @@ import type {
   CreateMessagesResult,
   TemplateMetadata
 } from 'genai-lite';
+
+// llama.cpp integration types and classes
+import {
+  LlamaCppClientAdapter,
+  LlamaCppServerClient,
+  createFallbackModelInfo
+} from 'genai-lite';
+
+import type {
+  LlamaCppClientConfig,
+  LlamaCppHealthResponse,
+  LlamaCppTokenizeResponse,
+  LlamaCppDetokenizeResponse,
+  LlamaCppEmbeddingResponse,
+  LlamaCppInfillResponse,
+  LlamaCppPropsResponse,
+  LlamaCppMetricsResponse,
+  LlamaCppSlot,
+  LlamaCppSlotsResponse
+} from 'genai-lite';
 ```
 
 ## Utilities
@@ -1101,6 +1462,10 @@ These utilities enable:
 - **Template Reusability**: Define templates once, use with different variables
 - **Type Safety**: Full TypeScript support with LLMMessage types
 
+## Examples
+
+See the **[chat-demo](examples/chat-demo)** application for a complete working example that demonstrates all library features in a production-ready React + Express application.
+
 ## Contributing
 
 Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.

package/dist/index.d.ts

@@ -5,7 +5,12 @@ export type { ModelPreset } from "./types/presets";
 export * from "./llm/types";
 export * from "./llm/clients/types";
 export { fromEnvironment } from "./providers/fromEnvironment";
+export { LlamaCppClientAdapter } from "./llm/clients/LlamaCppClientAdapter";
+export { LlamaCppServerClient } from "./llm/clients/LlamaCppServerClient";
+export type { LlamaCppClientConfig, } from "./llm/clients/LlamaCppClientAdapter";
+export type { LlamaCppHealthResponse, LlamaCppTokenizeResponse, LlamaCppDetokenizeResponse, LlamaCppEmbeddingResponse, LlamaCppInfillResponse, LlamaCppPropsResponse, LlamaCppMetricsResponse, LlamaCppSlot, LlamaCppSlotsResponse, } from "./llm/clients/LlamaCppServerClient";
 export { renderTemplate } from "./prompting/template";
 export { countTokens, getSmartPreview, extractRandomVariables } from "./prompting/content";
 export { parseStructuredContent, parseRoleTags, extractInitialTaggedContent, parseTemplateWithMetadata } from "./prompting/parser";
 export type { TemplateMetadata } from "./prompting/parser";
+export { createFallbackModelInfo } from "./llm/config";

package/dist/index.js

@@ -14,7 +14,7 @@ var __exportStar = (this && this.__exportStar) || function(m, exports) {
     for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
 };
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.parseTemplateWithMetadata = exports.extractInitialTaggedContent = exports.parseRoleTags = exports.parseStructuredContent = exports.extractRandomVariables = exports.getSmartPreview = exports.countTokens = exports.renderTemplate = exports.fromEnvironment = exports.LLMService = void 0;
+exports.createFallbackModelInfo = exports.parseTemplateWithMetadata = exports.extractInitialTaggedContent = exports.parseRoleTags = exports.parseStructuredContent = exports.extractRandomVariables = exports.getSmartPreview = exports.countTokens = exports.renderTemplate = exports.LlamaCppServerClient = exports.LlamaCppClientAdapter = exports.fromEnvironment = exports.LLMService = void 0;
 // --- LLM Service ---
 var LLMService_1 = require("./llm/LLMService");
 Object.defineProperty(exports, "LLMService", { enumerable: true, get: function () { return LLMService_1.LLMService; } });
@@ -25,6 +25,11 @@ __exportStar(require("./llm/clients/types"), exports);
 // --- API Key Providers ---
 var fromEnvironment_1 = require("./providers/fromEnvironment");
 Object.defineProperty(exports, "fromEnvironment", { enumerable: true, get: function () { return fromEnvironment_1.fromEnvironment; } });
+// --- llama.cpp Integration ---
+var LlamaCppClientAdapter_1 = require("./llm/clients/LlamaCppClientAdapter");
+Object.defineProperty(exports, "LlamaCppClientAdapter", { enumerable: true, get: function () { return LlamaCppClientAdapter_1.LlamaCppClientAdapter; } });
+var LlamaCppServerClient_1 = require("./llm/clients/LlamaCppServerClient");
+Object.defineProperty(exports, "LlamaCppServerClient", { enumerable: true, get: function () { return LlamaCppServerClient_1.LlamaCppServerClient; } });
 // --- Utilities ---
 var template_1 = require("./prompting/template");
 Object.defineProperty(exports, "renderTemplate", { enumerable: true, get: function () { return template_1.renderTemplate; } });
@@ -37,3 +42,5 @@ Object.defineProperty(exports, "parseStructuredContent", { enumerable: true, get
 Object.defineProperty(exports, "parseRoleTags", { enumerable: true, get: function () { return parser_1.parseRoleTags; } });
 Object.defineProperty(exports, "extractInitialTaggedContent", { enumerable: true, get: function () { return parser_1.extractInitialTaggedContent; } });
 Object.defineProperty(exports, "parseTemplateWithMetadata", { enumerable: true, get: function () { return parser_1.parseTemplateWithMetadata; } });
+var config_1 = require("./llm/config");
+Object.defineProperty(exports, "createFallbackModelInfo", { enumerable: true, get: function () { return config_1.createFallbackModelInfo; } });

package/dist/llm/LLMService.js

@@ -193,6 +193,14 @@ class LLMService {
                                         type: "validation_error",
                                     },
                                     object: "error",
+                                    partialResponse: {
+                                        id: result.id,
+                                        provider: result.provider,
+                                        model: result.model,
+                                        created: result.created,
+                                        choices: result.choices,
+                                        usage: result.usage
+                                    }
                                 };
                             }
                             else if (effectiveOnMissing === 'warn') {

package/dist/llm/LLMService.test.js

@@ -44,17 +44,34 @@ describe('LLMService', () => {
                 expect(errorResponse.error.code).toBe('UNSUPPORTED_PROVIDER');
                 expect(errorResponse.error.message).toContain('Unsupported provider');
             });
-            it('should return validation error for unsupported model', async () => {
+            it('should succeed with fallback for unknown model', async () => {
                 const request = {
-                    providerId: 'openai',
+                    providerId: 'mock', // Use mock provider to avoid real API calls
                     modelId: 'unsupported-model',
                     messages: [{ role: 'user', content: 'Hello' }]
                 };
                 const response = await service.sendMessage(request);
-                expect(response.object).toBe('error');
-                const errorResponse = response;
-                expect(errorResponse.error.code).toBe('UNSUPPORTED_MODEL');
-                expect(errorResponse.error.message).toContain('Unsupported model');
+                // Should succeed with mock response (not error) even for unknown model
+                expect(response.object).toBe('chat.completion');
+            });
+            it('should silently work with flexible providers unknown models (no warning)', async () => {
+                const warnings = [];
+                const consoleWarnSpy = jest.spyOn(console, 'warn').mockImplementation((msg) => {
+                    warnings.push(msg);
+                });
+                // Test with mock provider (which has allowUnknownModels: true)
+                const request = {
+                    providerId: 'mock',
+                    modelId: 'totally-unknown-model-xyz',
+                    messages: [{ role: 'user', content: 'Testing flexible provider' }]
+                };
+                const response = await service.sendMessage(request);
+                // Should succeed with mock response
+                expect(response.object).toBe('chat.completion');
+                // Should NOT warn about unknown model (filter out adapter constructor warnings)
+                const unknownModelWarnings = warnings.filter(w => !w.includes('No adapter constructor'));
+                expect(unknownModelWarnings.length).toBe(0); // No warnings for flexible providers
+                consoleWarnSpy.mockRestore();
             });
             it('should return validation error for empty messages', async () => {
                 const request = {
@@ -160,8 +177,8 @@ describe('LLMService', () => {
                 // Second request to same provider
                 request.messages = [{ role: 'user', content: 'Second request' }];
                 await service.sendMessage(request);
-                // API key provider should be called for each request with mock provider
-                expect(mockApiKeyProvider).toHaveBeenCalledTimes(0); // Mock provider doesn't need API keys
+                // API key provider should be called once per unique provider (mock provider now registered)
+                expect(mockApiKeyProvider).toHaveBeenCalledTimes(2);
             });
         });
         describe('settings management', () => {
@@ -325,11 +342,13 @@ describe('LLMService', () => {
     describe('getProviders', () => {
         it('should return all supported providers', async () => {
             const providers = await service.getProviders();
-            expect(providers).toHaveLength(4);
+            expect(providers).toHaveLength(6);
             expect(providers.find(p => p.id === 'openai')).toBeDefined();
             expect(providers.find(p => p.id === 'anthropic')).toBeDefined();
             expect(providers.find(p => p.id === 'gemini')).toBeDefined();
             expect(providers.find(p => p.id === 'mistral')).toBeDefined();
+            expect(providers.find(p => p.id === 'llamacpp')).toBeDefined();
+            expect(providers.find(p => p.id === 'mock')).toBeDefined();
         });
         it('should include provider metadata', async () => {
             const providers = await service.getProviders();
@@ -488,6 +507,9 @@ describe('LLMService', () => {
                 expect(errorResponse.error.type).toBe('validation_error');
                 expect(errorResponse.error.message).toContain('response was expected to start with a <thinking> tag');
                 expect(errorResponse.error.message).toContain('does not have native reasoning active');
+                // Check that partial response is included
+                expect(errorResponse.partialResponse).toBeDefined();
+                expect(errorResponse.partialResponse.choices[0].message.content).toBe('Response without thinking tag.');
             });
             it('should handle missing tag for non-reasoning model with warn', async () => {
                 const consoleSpy = jest.spyOn(console, 'warn').mockImplementation();
@@ -509,6 +531,27 @@ describe('LLMService', () => {
                 expect(consoleSpy).toHaveBeenCalledWith(expect.stringContaining('Expected <thinking> tag was not found'));
                 consoleSpy.mockRestore();
             });
+            it('should handle missing tag with explicit error mode', async () => {
+                const request = {
+                    providerId: 'mistral',
+                    modelId: 'codestral-2501',
+                    messages: [{ role: 'user', content: 'test_thinking:Response without thinking tag.' }],
+                    settings: {
+                        thinkingExtraction: {
+                            enabled: true,
+                            onMissing: 'error' // Explicitly set to error
+                        }
+                    }
+                };
+                const response = await service.sendMessage(request);
+                expect(response.object).toBe('error');
+                const errorResponse = response;
+                expect(errorResponse.error.code).toBe('MISSING_EXPECTED_TAG');
+                expect(errorResponse.error.message).toContain('response was expected to start with a <thinking> tag');
+                // Check that partial response is included
+                expect(errorResponse.partialResponse).toBeDefined();
+                expect(errorResponse.partialResponse.choices[0].message.content).toBe('Response without thinking tag.');
+            });
             it('should handle missing tag for non-reasoning model with ignore', async () => {
                 const request = {
                     providerId: 'mistral',
@@ -543,6 +586,8 @@ describe('LLMService', () => {
                 expect(response.object).toBe('error');
                 const errorResponse = response;
                 expect(errorResponse.error.message).toContain('expected to start with a <reasoning> tag');
+                expect(errorResponse.partialResponse).toBeDefined();
+                expect(errorResponse.partialResponse.choices[0].message.content).toBe('Response without custom tag.');
             });
             describe('auto mode with native reasoning detection', () => {
                 it('should enforce thinking tags for non-reasoning models by default', async () => {
@@ -564,6 +609,8 @@ describe('LLMService', () => {
                     const errorResponse = response;
                     expect(errorResponse.error.code).toBe('MISSING_EXPECTED_TAG');
                     expect(errorResponse.error.message).toContain('does not have native reasoning active');
+                    expect(errorResponse.partialResponse).toBeDefined();
+                    expect(errorResponse.partialResponse.choices[0].message.content).toBe('Response without thinking tag.');
                 });
                 it('should respect explicit reasoning.enabled: false even for models with enabledByDefault', async () => {
                     // This is the key test for the fix
@@ -584,6 +631,7 @@ describe('LLMService', () => {
                     expect(response.object).toBe('error');
                     const errorResponse = response;
                     expect(errorResponse.error.code).toBe('MISSING_EXPECTED_TAG');
+                    expect(errorResponse.partialResponse).toBeDefined();
                 });
             });
         });