@llumiverse/core 0.23.0-dev.20251121 → 0.24.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,300 +1,223 @@
1
- # Llumiverse - Universal LLM Connectors for Node.js
1
+ # Llumiverse - The Universal, Lightweight LLM Driver
2
2
 
3
3
  ![Build](https://github.com/vertesia/llumiverse/actions/workflows/build-and-test.yml/badge.svg)
4
4
  [![npm version](https://badge.fury.io/js/%40llumiverse%2Fcore.svg)](https://badge.fury.io/js/%40llumiverse%2Fcore)
5
5
  [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0)
6
6
 
7
- LLumiverse is a universal interface for interacting with Large Language Models, for the Typescript/Javascript ecosystem. It provides a lightweight modular library for interacting with various LLM models and execution platforms.
7
+ **Llumiverse** is the unified connectivity layer for the Large Language Model ecosystem. It provides a robust, modular, and strongly-typed interface for interacting with almost any AI provider in Node.js, Bun, and TypeScript environments.
8
8
 
9
- It solely focuses on abstracting LLMs and their execution platforms, and does not provide prompt templating, or RAG, or chains, letting you pick the best tool for the job.
9
+ > **Think of it as the "JDBC" or "ODBC" for LLMs.**
10
10
 
11
- The following LLM platforms are supported in the current version:
11
+ It solely focuses on **abstracting the connection and execution protocol**, letting you switch providers with zero code changes. It does **not** enforce prompt templating, rigid chain structures, or "magic" orchestration logic. You build the application; we handle the plumbing.
12
12
 
13
- | Provider | Completion | Chat | Model Listing | Multimodal | Fine-Tuning |
14
- | --- | :-: | :-: | :-: | :-: | :-: |
15
- | AWS Bedrock | ✅ | ✅ | ✅ | ✅ | ✅ |
16
- | Azure OpenAI | ✅ | ✅ | ✅ | ✅ | ✅ |
17
- | Google Vertex AI | ✅ | ✅ | ✅ | ✅ | By Request |
18
- | Groq | ✅ | ✅ | ✅ | N/A | N/A |
19
- | HuggingFace Inference Endpoints | ✅ | ✅ | N/A | N/A | N/A |
20
- | IBM WatsonX | ✅ | ✅ | ✅ | N/A | By Request |
21
- | Mistral AI | ✅ | ✅ | ✅ | N/A | By Request |
22
- | OpenAI | ✅ | ✅ | ✅ | ✅ | ✅ |
23
- | Replicate | ✅ | ✅ | ✅ | N/A | ✅ |
24
- | Together AI| ✅ | ✅ | ✅ | N/A | By Request |
13
+ ### Why Llumiverse?
25
14
 
26
- New capabilities and platform can easily be added by creating a new driver for the platform.
15
+ * **🚫 No Vendor Lock-in:** Switch from OpenAI to Vertex AI to Bedrock in minutes.
16
+ * **🧩 Modular & Lightweight:** Install only what you need. Zero bloat.
17
+ * **🛡️ Type-Safe:** First-class TypeScript support with normalized interfaces for completion, streaming, and tool use.
18
+ * **⚡ Universal Support:** Works in Node.js, serverless functions, and Bun.
27
19
 
20
+ ---
28
21
 
29
- ## Requirements
30
-
31
- * node v22+, or bun 1.0+
22
+ ### Supported Platforms
32
23
 
33
- ## Installation
24
+ | Provider | Completion | Chat | Streaming | Multimodal | Tool Calling | Embeddings |
25
+ | :--- | :-: | :-: | :-: | :-: | :-: | :-: |
26
+ | **AWS Bedrock** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
27
+ | **Azure Foundry** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
28
+ | **Azure OpenAI** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
29
+ | **Google Vertex AI** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
30
+ | **Groq** | ✅ | ✅ | ✅ | N/A | ✅ | N/A |
31
+ | **HuggingFace** | ✅ | ✅ | ✅ | N/A | N/A | N/A |
32
+ | **IBM WatsonX** | ✅ | ✅ | ✅ | N/A | N/A | ✅ |
33
+ | **Mistral AI** | ✅ | ✅ | ✅ | N/A | ✅ | ✅ |
34
+ | **OpenAI** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
35
+ | **OpenAI Compatible** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
36
+ | **Replicate** | ✅ | ✅ | ✅ | N/A | N/A | N/A |
37
+ | **Together AI** | ✅ | ✅ | ✅ | N/A | N/A | N/A |
38
+ | **xAI (Grok)** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
34
39
 
35
- 1. If you want to use llumiverse to execute prompt completion on various supported providers then install `@llumiverse/core` and `@llumiverse/drivers`
40
+ ---
36
41
 
37
- ```
38
- npm install @llumiverse/core @llumiverse/drivers
39
- ```
42
+ ### Llumiverse vs. The Rest
40
43
 
41
- 2. If you only want to use typescript types or other structures from llumiverse you only need to install `@llumiverse/core`
44
+ **Vs. LangChain / LlamaIndex:**
45
+ These are **application frameworks** that provide high-level abstractions for chains, memory, and RAG. Llumiverse is a **driver** library. It operates at a lower level.
46
+ * *Use Llumiverse if:* You want full control over your prompt logic, you want to build your own custom framework, or you simply need a clean, unified way to call an API without the weight of a massive dependency tree.
47
+ * *Use LangChain if:* You want pre-built "chains" and are okay with higher complexity and abstraction overhead.
42
48
 
43
- ```
44
- npm install @llumiverse/core
45
- ```
46
-
47
- 3. If you want to develop a new llumiverse driver for an unsupported LLM provider you only need to install `@llumiverse/core`
48
-
49
- ```
50
- npm install @llumiverse/core
51
- ```
49
+ **Vs. Vercel AI SDK:**
50
+ Vercel's SDK is fantastic for frontend/full-stack React/Next.js applications.
51
+ * *Use Llumiverse if:* You are building backend services, background workers, CLI tools, or generic Node.js applications where frontend-specific hooks aren't relevant. Llumiverse is environment-agnostic.
52
52
 
53
- ## Usage
53
+ ---
54
54
 
55
- First, you need to instantiate a driver instance for the target LLM platform you want to interact too. Each driver accepts its own set of parameters when instantiating.
55
+ ## Requirements
56
56
 
57
- ### OpenAI driver
57
+ * Node.js 22+
58
+ * Bun 1.0+ (experimental support)
58
59
 
59
- ```javascript
60
- import { OpenAIDriver } from "@llumiverse/drivers";
60
+ ## Installation
61
61
 
62
- // create an instance of the OpenAI driver
63
- const driver = new OpenAIDriver({
64
- apiKey: "YOUR_OPENAI_API_KEY"
65
- });
62
+ **For most use cases (recommended):**
63
+ ```bash
64
+ npm install @llumiverse/core @llumiverse/drivers
66
65
  ```
67
66
 
68
- ### Bedrock driver
69
-
70
- In this example, we will instantiate the Bedrock driver using credentials from the Shared Credentials File (i.e. ~/.aws/credentials).
71
- Learn more on how to [setup AWS credentials in node](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/setting-credentials-node.html).
67
+ * `@llumiverse/core`: The interfaces, types, and base abstractions.
68
+ * `@llumiverse/drivers`: The concrete implementations for all supported providers.
72
69
 
73
- ```javascript
74
- import { BedrockDriver } from "@llumiverse/drivers";
70
+ ## Quick Start
75
71
 
76
- const driver = new BedrockDriver({
77
- region: 'us-west-2'
78
- });
79
- ```
72
+ ### 1. Initialize a Driver
80
73
 
81
- ### VertexAI driver
74
+ Each driver takes normalized options but may require specific credentials (API keys, regions, project IDs).
82
75
 
83
- For the following example to work you need to define a `GOOGLE_APPLICATION_CREDENTIALS` environment variable.
76
+ ```typescript
77
+ // OpenAI
78
+ import { OpenAIDriver } from "@llumiverse/drivers";
79
+ const driver = new OpenAIDriver({ apiKey: process.env.OPENAI_API_KEY });
84
80
 
85
- ```javascript
81
+ // Google Vertex AI
86
82
  import { VertexAIDriver } from "@llumiverse/drivers";
87
83
  const driver = new VertexAIDriver({
88
- project: 'YOUR_GCLOUD_PROJECT',
84
+ project: process.env.GOOGLE_PROJECT_ID,
89
85
  region: 'us-central1'
90
86
  });
91
- ```
92
-
93
- ### Replicate driver
94
-
95
- ```javascript
96
- import { ReplicateDriver } from "@llumiverse/drivers";
97
-
98
- const driver = new ReplicateDriver({
99
- apiKey: "YOUR_REPLICATE_API_KEY"
100
- });
101
- ```
102
-
103
- ### TogetherAI driver
104
-
105
- ```javascript
106
- import { TogetherAIDriver } from "@llumiverse/drivers";
107
-
108
- const driver = new TogetherAIDriver({
109
- apiKey: "YOUR_TOGETHER_AI_API_KEY"
110
- });
111
- ```
112
-
113
- ### HuggingFace driver
114
-
115
- ```javascript
116
- import { HuggingFaceIEDriver } from "@llumiverse/drivers";
117
-
118
- const driver = new HuggingFaceIEDriver({
119
- apiKey: "YOUR_HUGGINGFACE_API_KEY",
120
- endpoint_url: "YOUR_HUGGINGFACE_ENDPOINT",
121
- });
122
- ```
123
-
124
- ### Listing available models
125
-
126
- Once you instantiated a driver you can list the available models. Some drivers accept an argument for the `listModel` method to search for matching models. Some drivers like for example `replicate` are listing a preselected set of models. To list other models you need to perform a search by giving a text query as an argument.
127
-
128
- In the following example, we are assuming that we have already instantiated a driver, which is available as the `driver` variable.
129
-
130
-
131
- ```javascript
132
- import { AIModel } from "@llumiverse/core";
133
-
134
- // instantiate the desired driver
135
- const driver = createDriverInstance();
136
87
 
137
- // list available models on the target LLM. (some drivers may require a search parameter to discover more models)
138
- const models: AIModel[] = await driver.listModels();
88
+ // AWS Bedrock (uses local AWS credentials automatically)
89
+ import { BedrockDriver } from "@llumiverse/drivers";
90
+ const driver = new BedrockDriver({ region: 'us-west-2' });
139
91
 
140
- console.log('# Available Models:');
141
- for (const model of models) {
142
- console.log(`${model.name} [${model.id}]`);
143
- }
92
+ // ... and many more!
144
93
  ```
145
94
 
146
- ### Execute a prompt
147
-
148
- To execute a prompt we need to create a prompt in the LLumiverse format and pass it to the driver `execute` method.
149
-
150
- The prompt format is very similar to the OpenAI prompt format. It is an array of messages with a `content` and a `role` property. The roles can be any of `"user" | "system" | "assistant" | "safety"`.
151
-
152
- The `safety` role is similar to `system` but has a greater precedence over the other messages. Thus, it will override any `user` or `system` message that is saying something contrary to the `safety` message.
153
-
154
- In order to execute a prompt we also need to specify a target model, given a model ID which is known by the target LLM. We may also specify execution options like `temperature`, `max_tokens` etc.
155
-
156
- In the following example, we are again assuming that we have already instantiated a driver, which is available as the `driver` variable.
157
-
158
- Also, we are assuming the model ID we want to target is available as the `model` variable. To get a list of the existing models (and their IDs) you can list the model as we shown in the previous example
159
-
160
- Here is an example of model IDs depending on the driver type:
161
- * OpenAI: `gpt-3.5-turbo`
162
- * Bedrock: `arn:aws:bedrock:us-west-2::foundation-model/cohere.command-text-v14`
163
- * VertexAI: `text-bison`
164
- * Replicate: `meta/llama-2-70b-chat:02e509c789964a7ea8736978a43525956ef40397be9033abf9fd2badfe68c9e3`
165
- * TogetherAI: `mistralai/Mistral-7B-instruct-v0.1`
166
- * HuggingFace: `aws-mistral-7b-instruct-v0-1-015`
167
-
168
-
169
- ```javascript
170
- import { PromptRole, PromptSegment } from "@llumiverse/core";
95
+ ### 2. Execute a Prompt (Standard)
171
96
 
97
+ The execution API is normalized. You send a `PromptSegment[]` and get a `Completion` object, regardless of the provider.
172
98
 
173
- // instantiate the desired driver
174
- const driver = createDriverInstance();
175
- const model = "the-model-id"; // change with your desired model ID
99
+ ```typescript
100
+ import { PromptRole } from "@llumiverse/core";
176
101
 
177
- // create the prompt.
178
- const prompt: PromptSegment[] = [
102
+ // 1. Create a generic prompt
103
+ const prompt = [
104
+ {
105
+ role: PromptRole.system,
106
+ content: "You are a helpful coding assistant."
107
+ },
179
108
  {
180
109
  role: PromptRole.user,
181
- content: 'Please, write a short story about winter in Paris, in no more than 512 characters.'
110
+ content: "Write a hello world in Python."
182
111
  }
183
- ]
112
+ ];
184
113
 
185
- // execute a model and wait for the response
186
- console.log(`\n# Executing prompt on ${model} model: ${prompt}`);
114
+ // 2. Execute against ANY driver
187
115
  const response = await driver.execute(prompt, {
188
- model,
189
- temperature: 0.6,
116
+ model: "gpt-4o", // or "gemini-1.5-pro", "anthropic.claude-3-sonnet-20240229-v1:0", etc.
117
+ temperature: 0.7,
190
118
  max_tokens: 1024
191
119
  });
192
120
 
193
- console.log('\n# LLM response:', response.result)
194
- console.log('# Response took', response.execution_time, 'ms')
195
- console.log('# Token usage:', response.token_usage);
121
+ console.log(response.result[0].value);
196
122
  ```
197
123
 
124
+ ### 3. Stream a Response
198
125
 
199
- ### Execute a prompt in streaming mode
200
-
201
- In this example, we will execute a prompt and will stream the result to display it on the console as it is returned by the target LLM platform.
202
-
203
- **Note** that some models don't support streaming. In that case, the driver will simulate a streaming using a single chunk of text corresponding to the entire response.
204
-
205
-
206
- ```javascript
207
- import { PromptRole, PromptSegment } from "@llumiverse/core";
208
-
209
- // instantiate the desired driver
210
- const driver = createDriverInstance();
211
- const model = "the-model-id"; // change with your desired model ID
212
-
213
- // create the prompt.
214
- const prompt: PromptSegment[] = [
215
- {
216
- role: PromptRole.user,
217
- content: 'Please, write a short story about winter in Paris, in no more than 512 characters.'
218
- }
219
- ]
126
+ Streaming is standardized into an `AsyncIterable`.
220
127
 
221
- // execute the prompt in streaming mode
222
- console.log(`\n# Executing prompt on model ${model} in streaming mode: ${prompt}`);
128
+ ```typescript
223
129
  const stream = await driver.stream(prompt, {
224
- model,
225
- temperature: 0.6,
226
- max_tokens: 1024
130
+ model: "gpt-4o",
131
+ temperature: 0.7
227
132
  });
228
133
 
229
- // print the streaming response as it comes
230
134
  for await (const chunk of stream) {
231
135
  process.stdout.write(chunk);
232
136
  }
233
137
 
234
- // when the response stream is consumed we can get the final response using stream.completion field.
235
- const streamingResponse = stream.completion!;
236
-
237
- console.log('\n# LLM response:', streamingResponse.result)
238
- console.log('# Response took', streamingResponse.execution_time, 'ms')
239
- console.log('# Token usage:', streamingResponse.token_usage);
138
+ // Access the full assembled response afterwards if needed
139
+ console.log("\nFull response:", stream.completion.result[0].value);
240
140
  ```
241
141
 
242
- ### Generate embeddings
243
-
244
- LLumiverse drivers expose a method to generate vector embeddings for a given text.
245
- Drivers supporting embeddings as of v0.10.0 are `bedrock`, `openai`, `vertexai`.
246
- If embeddings are not yet supported the generateEmbeddings method will throws an error.
142
+ ### 4. Generate Embeddings
247
143
 
248
- Here is an example on using the `vertexai` driver. For the example to work you need to define a `GOOGLE_APPLICATION_CREDENTIALS` env variable to be able to access your gcloud project
144
+ Embeddings are normalized to a simple `{ values: number[] }` structure.
249
145
 
250
- ```javascript
251
- import { VertexAIDriver } from "@llumiverse/drivers";
252
-
253
- const driver = new VertexAIDriver({
254
- project: 'your-project-id',
255
- region: 'us-central1' // your zone
146
+ ```typescript
147
+ const embedding = await driver.generateEmbeddings({
148
+ content: "Llumiverse is awesome."
256
149
  });
257
150
 
258
- const r = await vertex.generateEmbeddings({ content: "Hello world!" });
259
-
260
- // print the vector
261
- console.log('Embeddings: ', v.values);
151
+ console.log(embedding.values); // [0.012, -0.34, ...]
262
152
  ```
263
153
 
264
- The result object contains the vector as the `values` property, the `model` used to generate the embeddings and an optional `token_count` which if defined is the token count of the input text.
265
- Depending on the driver, the result may contain additional properties.
154
+ ## Advanced Features
266
155
 
267
- Also you can specify a specific model to be used or pass other driver supported parameter.
156
+ ### Tool Calling (Function Calling)
268
157
 
269
- **Example:**
158
+ Llumiverse normalizes tool definitions across providers (OpenAI, Bedrock, Vertex, etc.), allowing you to define tools once and use them anywhere.
270
159
 
271
- ```javascript
272
- import { VertexAIDriver } from "@llumiverse/drivers";
160
+ ```typescript
161
+ import { ToolDefinition } from "@llumiverse/core";
273
162
 
274
- const driver = new VertexAIDriver({
275
- project: 'your-project-id',
276
- region: 'us-central1' // your zone
277
- });
163
+ // 1. Define the tool
164
+ const getWeatherTool: ToolDefinition = {
165
+ name: "get_weather",
166
+ description: "Get the current weather in a given location",
167
+ input_schema: {
168
+ type: "object",
169
+ properties: {
170
+ location: { type: "string" },
171
+ unit: { type: "string", enum: ["celsius", "fahrenheit"] }
172
+ },
173
+ required: ["location"]
174
+ }
175
+ };
278
176
 
279
- const r = await vertex.generateEmbeddings({
280
- content: "Hello world!",
281
- model: "textembedding-gecko@002",
282
- task_type: "SEMANTIC_SIMILARITY"
177
+ // 2. Execute with tools
178
+ const response = await driver.execute(prompt, {
179
+ model: "gpt-4o",
180
+ tools: [getWeatherTool]
283
181
  });
284
182
 
285
- // print the vector
286
- console.log('Embeddings: ', v.values);
183
+ // 3. Handle the tool use
184
+ if (response.tool_use) {
185
+ for (const tool of response.tool_use) {
186
+ console.log(`Executing ${tool.tool_name} with args:`, tool.tool_input);
187
+ // ... call your actual function ...
188
+ }
189
+ }
287
190
  ```
288
191
 
289
- The `task_type` parameter is specific to the [textembedding-gecko model](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings).
192
+ ### Structured Outputs (JSON Schemas)
290
193
 
194
+ Enforce a specific JSON structure for the response. Llumiverse handles the schema conversion for the underlying provider (e.g., using `response_format` for OpenAI or equivalent techniques for others).
291
195
 
292
- ## Contributing
196
+ ```typescript
197
+ const schema = {
198
+ type: "object",
199
+ properties: {
200
+ sentiment: { type: "string", enum: ["positive", "neutral", "negative"] },
201
+ confidence: { type: "number" }
202
+ },
203
+ required: ["sentiment", "confidence"]
204
+ };
205
+
206
+ const response = await driver.execute(prompt, {
207
+ model: "gpt-4o",
208
+ result_schema: schema
209
+ });
293
210
 
294
- Contributions are welcome!
295
- Please see [CONTRIBUTING.md](https://github.com/vertesia/llumiverse/blob/main/CONTRIBUTING.md) for more details.
211
+ // The result is automatically parsed
212
+ const data = response.result[0].value;
213
+ console.log(data.sentiment); // "positive"
214
+ ```
215
+
216
+ ## Contributing
296
217
 
218
+ We welcome contributions! Whether it's a new driver, a bug fix, or a docs improvement.
219
+ Please see [CONTRIBUTING.md](https://github.com/vertesia/llumiverse/blob/main/CONTRIBUTING.md) for details.
297
220
 
298
221
  ## License
299
222
 
300
- Llumiverse is licensed under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). Feel free to use it accordingly.
223
+ Apache 2.0
@@ -18,6 +18,7 @@ class DefaultCompletionStream {
18
18
  this.completion = undefined;
19
19
  this.chunks = 0;
20
20
  const accumulatedResults = []; // Accumulate CompletionResult[] from chunks
21
+ const accumulatedToolUse = new Map(); // Accumulate tool_use by id
21
22
  this.driver.logger.debug(`[${this.driver.provider}] Streaming Execution of ${this.options.model} with prompt`);
22
23
  const start = Date.now();
23
24
  let finish_reason = undefined;
@@ -42,6 +43,43 @@ class DefaultCompletionStream {
42
43
  promptTokens = Math.max(promptTokens, chunk.token_usage.prompt ?? 0);
43
44
  resultTokens = Math.max(resultTokens ?? 0, chunk.token_usage.result ?? 0);
44
45
  }
46
+ // Accumulate tool_use from chunks
47
+ // Note: During streaming, tool_input comes as string chunks that need concatenation
48
+ if (chunk.tool_use && chunk.tool_use.length > 0) {
49
+ for (const tool of chunk.tool_use) {
50
+ const existing = accumulatedToolUse.get(tool.id);
51
+ if (existing) {
52
+ // Merge tool input (for streaming where arguments come as string pieces)
53
+ if (tool.tool_input !== null && tool.tool_input !== undefined) {
54
+ const existingInput = existing.tool_input;
55
+ const newInput = tool.tool_input;
56
+ if (typeof existingInput === 'string' && typeof newInput === 'string') {
57
+ // Concatenate string arguments
58
+ existing.tool_input = existingInput + newInput;
59
+ }
60
+ else if (existingInput && typeof existingInput === 'object' && newInput && typeof newInput === 'object') {
61
+ // Merge objects
62
+ existing.tool_input = { ...existingInput, ...newInput };
63
+ }
64
+ else {
65
+ existing.tool_input = tool.tool_input;
66
+ }
67
+ }
68
+ // Update tool name if provided (might come in later chunk)
69
+ if (tool.tool_name) {
70
+ existing.tool_name = tool.tool_name;
71
+ }
72
+ // Update actual ID if provided (OpenAI sends id only in first chunk)
73
+ if (tool._actual_id) {
74
+ existing._actual_id = tool._actual_id;
75
+ }
76
+ }
77
+ else {
78
+ // New tool call
79
+ accumulatedToolUse.set(tool.id, { ...tool });
80
+ }
81
+ }
82
+ }
45
83
  if (Array.isArray(chunk.result) && chunk.result.length > 0) {
46
84
  // Process each result in the chunk, combining consecutive text/JSON
47
85
  for (const result of chunk.result) {
@@ -115,6 +153,27 @@ class DefaultCompletionStream {
115
153
  // Allows for checking for truthy-ness on token_usage, rather than it's internals. For testing and downstream usage.
116
154
  const tokens = resultTokens ?
117
155
  { prompt: promptTokens, result: resultTokens, total: resultTokens + promptTokens, } : undefined;
156
+ // Convert accumulated tool_use Map to array
157
+ const toolUseArray = accumulatedToolUse.size > 0 ? Array.from(accumulatedToolUse.values()) : undefined;
158
+ // Finalize tool calls: restore actual IDs and parse JSON arguments
159
+ if (toolUseArray) {
160
+ for (const tool of toolUseArray) {
161
+ // Restore actual ID from OpenAI (was stored in _actual_id during streaming)
162
+ if (tool._actual_id) {
163
+ tool.id = tool._actual_id;
164
+ delete tool._actual_id;
165
+ }
166
+ // Parse tool_input strings as JSON if needed (streaming sends arguments as string chunks)
167
+ if (typeof tool.tool_input === 'string') {
168
+ try {
169
+ tool.tool_input = JSON.parse(tool.tool_input);
170
+ }
171
+ catch {
172
+ // Keep as string if not valid JSON
173
+ }
174
+ }
175
+ }
176
+ }
118
177
  this.completion = {
119
178
  result: accumulatedResults, // Return the accumulated CompletionResult[] instead of text
120
179
  prompt: this.prompt,
@@ -122,7 +181,13 @@ class DefaultCompletionStream {
122
181
  token_usage: tokens,
123
182
  finish_reason: finish_reason,
124
183
  chunks: this.chunks,
184
+ tool_use: toolUseArray,
125
185
  };
186
+ // Build conversation context for multi-turn support
187
+ const conversation = this.driver.buildStreamingConversation(this.prompt, accumulatedResults, toolUseArray, this.options);
188
+ if (conversation !== undefined) {
189
+ this.completion.conversation = conversation;
190
+ }
126
191
  try {
127
192
  if (this.completion) {
128
193
  this.driver.validateResult(this.completion, this.options);
@@ -1 +1 @@
1
- {"version":3,"file":"CompletionStream.js","sourceRoot":"","sources":["../../src/CompletionStream.ts"],"names":[],"mappings":";;;AAGA,MAAa,uBAAuB;IAKb;IACR;IACA;IALX,MAAM,CAAS,CAAC,0DAA0D;IAC1E,UAAU,CAAyC;IAEnD,YAAmB,MAA8C,EACtD,MAAe,EACf,OAAyB;QAFjB,WAAM,GAAN,MAAM,CAAwC;QACtD,WAAM,GAAN,MAAM,CAAS;QACf,YAAO,GAAP,OAAO,CAAkB;QAChC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC;IACpB,CAAC;IAED,KAAK,CAAC,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC;QACzB,cAAc;QACd,IAAI,CAAC,UAAU,GAAG,SAAS,CAAC;QAC5B,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC;QAChB,MAAM,kBAAkB,GAAU,EAAE,CAAC,CAAC,4CAA4C;QAElF,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,KAAK,CACpB,IAAI,IAAI,CAAC,MAAM,CAAC,QAAQ,4BAA4B,IAAI,CAAC,OAAO,CAAC,KAAK,cAAc,CACvF,CAAC;QAEF,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;QACzB,IAAI,aAAa,GAAuB,SAAS,CAAC;QAClD,IAAI,YAAY,GAAW,CAAC,CAAC;QAC7B,IAAI,YAAY,GAAuB,SAAS,CAAC;QAEjD,IAAI,CAAC;YACD,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,MAAM,CAAC,2BAA2B,CAAC,IAAI,CAAC,MAAM,EAAE,IAAI,CAAC,OAAO,CAAC,CAAC;YACxF,IAAI,KAAK,EAAE,MAAM,KAAK,IAAI,MAAM,EAAE,CAAC;gBAC/B,IAAI,KAAK,EAAE,CAAC;oBACR,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;wBAC5B,IAAI,CAAC,MAAM,EAAE,CAAC;wBACd,MAAM,KAAK,CAAC;oBAChB,CAAC;yBAAM,CAAC;wBACJ,IAAI,KAAK,CAAC,aAAa,EAAE,CAAC,CAA2B,iDAAiD;4BAClG,aAAa,GAAG,KAAK,CAAC,aAAa,CAAC,CAAa,yEAAyE;wBAC9H,CAAC;wBACD,IAAI,KAAK,CAAC,WAAW,EAAE,CAAC;4BACpB,gDAAgD;4BAChD,qCAAqC;4BACrC,8EAA8E;4BAC9E,YAAY,GAAG,IAAI,CAAC,GAAG,CAAC,YAAY,EAAE,KAAK,CAAC,WAAW,CAAC,MAAM,IAAI,CAAC,CAAC,CAAC;4BACrE,YAAY,GAAG,IAAI,CAAC,GAAG,CAAC,YAAY,IAAI,CAAC,EAAE,KAAK,CAAC,WAAW,CAAC,MAAM,IAAI,CAAC,CAAC,CAAC;wBAC9E,CAAC;wBACD,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,MAAM,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;4BACzD,oEAAoE;4BACpE,KAAK,MAAM,MAAM,IAAI,KAAK,CAAC,MAAM,EAAE,CAAC;gCAChC,2DAA2D;gCAC3D,MAAM,UAAU,GAAG,kBAAkB,CAAC,kBAAkB,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;gCAErE,IAAI,UAAU;oCACV,CAAC,CAAC,UAAU,CAAC,IAAI,KAAK,MAAM,IAAI,MAAM,CAAC,IAAI,KAAK,MAAM,CAAC;wCACnD,CAAC,UAAU,CAAC,IAAI,KAAK,MAAM,IAAI,MAAM,CAAC,IAAI,KAAK,MAAM,CAAC,CAAC,EAAE,CAAC;oCAC9D,2CAA2C;oCAC3C,IAAI,MAAM,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;wCACzB,UAAU,CAAC,KAAK,IAAI,MAAM,CAAC,KAAK,CAAC;oCACrC,CAAC;yCAAM,IAAI,MAAM,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;wCAChC,gDAAgD;wCAChD,IAAI,CAAC;4CACD,MAAM,UAAU,GAAG,UAAU,CAAC,KAAK,CAAC;4CACpC,MAAM,aAAa,GAAG,MAAM,CAAC,KAAK,CAAC;4CACnC,IAAI,UAAU,KAAK,IAAI,IAAI,OAAO,UAAU,KAAK,QAAQ;gDACrD,aAAa,KAAK,IAAI,IAAI,OAAO,aAAa,KAAK,QAAQ,EAAE,CAAC;gDAC9D,MAAM,QAAQ,GAAG,EAAE,GAAG,UAAU,EAAE,GAAG,aAAa,EAAE,CAAC;gDACrD,UAAU,CAAC,KAAK,GAAG,QAAQ,CAAC;4CAChC,CAAC;iDAAM,CAAC;gDACJ,oDAAoD;gDACpD,MAAM,OAAO,GAAG,OAAO,UAAU,KAAK,QAAQ,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,UAAU,CAAC,CAAC;gDACzF,MAAM,UAAU,GAAG,OAAO,aAAa,KAAK,QAAQ,CAAC,CAAC,CAAC,aAAa,CAAC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,aAAa,CAAC,CAAC;gDACrG,UAAU,CAAC,KAAK,GAAG,OAAO,GAAG,UAAU,CAAC;4CAC5C,CAAC;wCACL,CAAC;wCAAC,MAAM,CAAC;4CACL,6DAA6D;4CAC7D,UAAU,CAAC,KAAK,GAAG,MAAM,CAAC,UAAU,CAAC,KAAK,CAAC,GAAG,MAAM,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;wCACvE,CAAC;oCACL,CAAC;gCACL,CAAC;qCAAM,CAAC;oCACJ,oBAAoB;oCACpB,kBAAkB,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;gCACpC,CAAC;4BACL,CAAC;4BAED,qDAAqD;4BACrD,wCAAwC;4BACxC,MAAM,UAAU,GAAG,KAAK,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE;gCACpC,QAAQ,CAAC,CAAC,IAAI,EAAE,CAAC;oCACb,KAAK,MAAM;wCACP,OAAO,CAAC,CAAC,KAAK,CAAC;oCACnB,KAAK,MAAM;wCACP,OAAO,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;oCACnC,KAAK,OAAO;wCACR,iDAAiD;wCACjD,MAAM,cAAc,GAAG,OAAO,CAAC,CAAC,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;wCACzG,OAAO,aAAa,cAAc,QAAQ,CAAC;oCAC/C;wCACI,OAAO,MAAM,CAAE,CAAS,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC;gCAC9C,CAAC;4BACL,CAAC,CAAC,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;4BAEZ,IAAI,UAAU,EAAE,CAAC;gCACb,IAAI,CAAC,MAAM,EAAE,CAAC;gCACd,MAAM,UAAU,CAAC;4BACrB,CAAC;wBACL,CAAC;oBACL,CAAC;gBACL,CAAC;YACL,CAAC;QACL,CAAC;QAAC,OAAO,KAAU,EAAE,CAAC;YAClB,KAAK,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC;YAC3B,MAAM,KAAK,CAAC;QAChB,CAAC;QAED,2FAA2F;QAC3F,oHAAoH;QACpH,MAAM,MAAM,GAAoC,YAAY,CAAC,CAAC;YAC1D,EAAE,MAAM,EAAE,YAAY,EAAE,MAAM,EAAE,YAAY,EAAE,KAAK,EAAE,YAAY,GAAG,YAAY,GAAG,CAAC,CAAC,CAAC,SAAS,CAAA;QAEnG,IAAI,CAAC,UAAU,GAAG;YACd,MAAM,EAAE,kBAAkB,EAAE,4DAA4D;YACxF,MAAM,EAAE,IAAI,CAAC,MAAM;YACnB,cAAc,EAAE,IAAI,CAAC,GAAG,EAAE,GAAG,KAAK;YAClC,WAAW,EAAE,MAAM;YACnB,aAAa,EAAE,aAAa;YAC5B,MAAM,EAAE,IAAI,CAAC,MAAM;SACtB,CAAA;QAED,IAAI,CAAC;YACD,IAAI,IAAI,CAAC,UAAU,EAAE,CAAC;gBAClB,IAAI,CAAC,MAAM,CAAC,cAAc,CAAC,IAAI,CAAC,UAAU,EAAE,IAAI,CAAC,OAAO,CAAC,CAAC;YAC9D,CAAC;QACL,CAAC;QAAC,OAAO,KAAU,EAAE,CAAC;YAClB,KAAK,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC;YAC3B,MAAM,KAAK,CAAC;QAChB,CAAC;IACL,CAAC;CAEJ;AAxID,0DAwIC;AAED,MAAa,wBAAwB;IAId;IACR;IACA;IAJX,UAAU,CAAyC;IAEnD,YAAmB,MAA8C,EACtD,MAAe,EACf,OAAyB;QAFjB,WAAM,GAAN,MAAM,CAAwC;QACtD,WAAM,GAAN,MAAM,CAAS;QACf,YAAO,GAAP,OAAO,CAAkB;IACpC,CAAC;IAED,KAAK,CAAC,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC;QACzB,cAAc;QACd,IAAI,CAAC,UAAU,GAAG,SAAS,CAAC;QAC5B,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,KAAK,CACpB,IAAI,IAAI,CAAC,MAAM,CAAC,QAAQ,kEAAkE,CAC7F,CAAC;QACF,IAAI,CAAC;YACD,MAAM,UAAU,GAAG,MAAM,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC,MAAM,EAAE,IAAI,CAAC,OAAO,CAAC,CAAC;YACzE,kFAAkF;YAClF,MAAM,OAAO,GAAG,UAAU,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE;gBACtC,QAAQ,CAAC,CAAC,IAAI,EAAE,CAAC;oBACb,KAAK,MAAM;wBACP,OAAO,CAAC,CAAC,KAAK,CAAC;oBACnB,KAAK,MAAM;wBACP,OAAO,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;oBACnC,KAAK,OAAO;wBACR,iDAAiD;wBACjD,MAAM,cAAc,GAAG,OAAO,CAAC,CAAC,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;wBACzG,OAAO,WAAW,cAAc,MAAM,CAAC;oBAC3C;wBACI,OAAO,MAAM,CAAE,CAAS,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC;gBAC9C,CAAC;YACL,CAAC,CAAC,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;YACZ,MAAM,OAAO,CAAC;YACd,IAAI,CAAC,UAAU,GAAG,UAAU,CAAC,CAAC,mEAAmE;QACrG,CAAC;QAAC,OAAO,KAAU,EAAE,CAAC;YAClB,KAAK,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC;YAC3B,MAAM,KAAK,CAAC;QAChB,CAAC;IACL,CAAC;CACJ;AAvCD,4DAuCC"}
1
+ {"version":3,"file":"CompletionStream.js","sourceRoot":"","sources":["../../src/CompletionStream.ts"],"names":[],"mappings":";;;AAGA,MAAa,uBAAuB;IAKb;IACR;IACA;IALX,MAAM,CAAS,CAAC,0DAA0D;IAC1E,UAAU,CAAyC;IAEnD,YAAmB,MAA8C,EACtD,MAAe,EACf,OAAyB;QAFjB,WAAM,GAAN,MAAM,CAAwC;QACtD,WAAM,GAAN,MAAM,CAAS;QACf,YAAO,GAAP,OAAO,CAAkB;QAChC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC;IACpB,CAAC;IAED,KAAK,CAAC,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC;QACzB,cAAc;QACd,IAAI,CAAC,UAAU,GAAG,SAAS,CAAC;QAC5B,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC;QAChB,MAAM,kBAAkB,GAAU,EAAE,CAAC,CAAC,4CAA4C;QAClF,MAAM,kBAAkB,GAAyB,IAAI,GAAG,EAAE,CAAC,CAAC,4BAA4B;QAExF,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,KAAK,CACpB,IAAI,IAAI,CAAC,MAAM,CAAC,QAAQ,4BAA4B,IAAI,CAAC,OAAO,CAAC,KAAK,cAAc,CACvF,CAAC;QAEF,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;QACzB,IAAI,aAAa,GAAuB,SAAS,CAAC;QAClD,IAAI,YAAY,GAAW,CAAC,CAAC;QAC7B,IAAI,YAAY,GAAuB,SAAS,CAAC;QAEjD,IAAI,CAAC;YACD,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,MAAM,CAAC,2BAA2B,CAAC,IAAI,CAAC,MAAM,EAAE,IAAI,CAAC,OAAO,CAAC,CAAC;YACxF,IAAI,KAAK,EAAE,MAAM,KAAK,IAAI,MAAM,EAAE,CAAC;gBAC/B,IAAI,KAAK,EAAE,CAAC;oBACR,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;wBAC5B,IAAI,CAAC,MAAM,EAAE,CAAC;wBACd,MAAM,KAAK,CAAC;oBAChB,CAAC;yBAAM,CAAC;wBACJ,IAAI,KAAK,CAAC,aAAa,EAAE,CAAC,CAA2B,iDAAiD;4BAClG,aAAa,GAAG,KAAK,CAAC,aAAa,CAAC,CAAa,yEAAyE;wBAC9H,CAAC;wBACD,IAAI,KAAK,CAAC,WAAW,EAAE,CAAC;4BACpB,gDAAgD;4BAChD,qCAAqC;4BACrC,8EAA8E;4BAC9E,YAAY,GAAG,IAAI,CAAC,GAAG,CAAC,YAAY,EAAE,KAAK,CAAC,WAAW,CAAC,MAAM,IAAI,CAAC,CAAC,CAAC;4BACrE,YAAY,GAAG,IAAI,CAAC,GAAG,CAAC,YAAY,IAAI,CAAC,EAAE,KAAK,CAAC,WAAW,CAAC,MAAM,IAAI,CAAC,CAAC,CAAC;wBAC9E,CAAC;wBACD,kCAAkC;wBAClC,oFAAoF;wBACpF,IAAI,KAAK,CAAC,QAAQ,IAAI,KAAK,CAAC,QAAQ,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;4BAC9C,KAAK,MAAM,IAAI,IAAI,KAAK,CAAC,QAAQ,EAAE,CAAC;gCAChC,MAAM,QAAQ,GAAG,kBAAkB,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;gCACjD,IAAI,QAAQ,EAAE,CAAC;oCACX,yEAAyE;oCACzE,IAAI,IAAI,CAAC,UAAU,KAAK,IAAI,IAAI,IAAI,CAAC,UAAU,KAAK,SAAS,EAAE,CAAC;wCAC5D,MAAM,aAAa,GAAG,QAAQ,CAAC,UAAqB,CAAC;wCACrD,MAAM,QAAQ,GAAG,IAAI,CAAC,UAAqB,CAAC;wCAC5C,IAAI,OAAO,aAAa,KAAK,QAAQ,IAAI,OAAO,QAAQ,KAAK,QAAQ,EAAE,CAAC;4CACpE,+BAA+B;4CAC9B,QAAgB,CAAC,UAAU,GAAG,aAAa,GAAG,QAAQ,CAAC;wCAC5D,CAAC;6CAAM,IAAI,aAAa,IAAI,OAAO,aAAa,KAAK,QAAQ,IAAI,QAAQ,IAAI,OAAO,QAAQ,KAAK,QAAQ,EAAE,CAAC;4CACxG,gBAAgB;4CAChB,QAAQ,CAAC,UAAU,GAAG,EAAE,GAAI,aAAwB,EAAE,GAAI,QAAmB,EAAS,CAAC;wCAC3F,CAAC;6CAAM,CAAC;4CACJ,QAAQ,CAAC,UAAU,GAAG,IAAI,CAAC,UAAU,CAAC;wCAC1C,CAAC;oCACL,CAAC;oCACD,2DAA2D;oCAC3D,IAAI,IAAI,CAAC,SAAS,EAAE,CAAC;wCACjB,QAAQ,CAAC,SAAS,GAAG,IAAI,CAAC,SAAS,CAAC;oCACxC,CAAC;oCACD,qEAAqE;oCACrE,IAAK,IAAY,CAAC,UAAU,EAAE,CAAC;wCAC1B,QAAgB,CAAC,UAAU,GAAI,IAAY,CAAC,UAAU,CAAC;oCAC5D,CAAC;gCACL,CAAC;qCAAM,CAAC;oCACJ,gBAAgB;oCAChB,kBAAkB,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,EAAE,EAAE,GAAG,IAAI,EAAE,CAAC,CAAC;gCACjD,CAAC;4BACL,CAAC;wBACL,CAAC;wBACD,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,MAAM,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;4BACzD,oEAAoE;4BACpE,KAAK,MAAM,MAAM,IAAI,KAAK,CAAC,MAAM,EAAE,CAAC;gCAChC,2DAA2D;gCAC3D,MAAM,UAAU,GAAG,kBAAkB,CAAC,kBAAkB,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;gCAErE,IAAI,UAAU;oCACV,CAAC,CAAC,UAAU,CAAC,IAAI,KAAK,MAAM,IAAI,MAAM,CAAC,IAAI,KAAK,MAAM,CAAC;wCACnD,CAAC,UAAU,CAAC,IAAI,KAAK,MAAM,IAAI,MAAM,CAAC,IAAI,KAAK,MAAM,CAAC,CAAC,EAAE,CAAC;oCAC9D,2CAA2C;oCAC3C,IAAI,MAAM,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;wCACzB,UAAU,CAAC,KAAK,IAAI,MAAM,CAAC,KAAK,CAAC;oCACrC,CAAC;yCAAM,IAAI,MAAM,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;wCAChC,gDAAgD;wCAChD,IAAI,CAAC;4CACD,MAAM,UAAU,GAAG,UAAU,CAAC,KAAK,CAAC;4CACpC,MAAM,aAAa,GAAG,MAAM,CAAC,KAAK,CAAC;4CACnC,IAAI,UAAU,KAAK,IAAI,IAAI,OAAO,UAAU,KAAK,QAAQ;gDACrD,aAAa,KAAK,IAAI,IAAI,OAAO,aAAa,KAAK,QAAQ,EAAE,CAAC;gDAC9D,MAAM,QAAQ,GAAG,EAAE,GAAG,UAAU,EAAE,GAAG,aAAa,EAAE,CAAC;gDACrD,UAAU,CAAC,KAAK,GAAG,QAAQ,CAAC;4CAChC,CAAC;iDAAM,CAAC;gDACJ,oDAAoD;gDACpD,MAAM,OAAO,GAAG,OAAO,UAAU,KAAK,QAAQ,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,UAAU,CAAC,CAAC;gDACzF,MAAM,UAAU,GAAG,OAAO,aAAa,KAAK,QAAQ,CAAC,CAAC,CAAC,aAAa,CAAC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,aAAa,CAAC,CAAC;gDACrG,UAAU,CAAC,KAAK,GAAG,OAAO,GAAG,UAAU,CAAC;4CAC5C,CAAC;wCACL,CAAC;wCAAC,MAAM,CAAC;4CACL,6DAA6D;4CAC7D,UAAU,CAAC,KAAK,GAAG,MAAM,CAAC,UAAU,CAAC,KAAK,CAAC,GAAG,MAAM,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;wCACvE,CAAC;oCACL,CAAC;gCACL,CAAC;qCAAM,CAAC;oCACJ,oBAAoB;oCACpB,kBAAkB,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;gCACpC,CAAC;4BACL,CAAC;4BAED,qDAAqD;4BACrD,wCAAwC;4BACxC,MAAM,UAAU,GAAG,KAAK,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE;gCACpC,QAAQ,CAAC,CAAC,IAAI,EAAE,CAAC;oCACb,KAAK,MAAM;wCACP,OAAO,CAAC,CAAC,KAAK,CAAC;oCACnB,KAAK,MAAM;wCACP,OAAO,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;oCACnC,KAAK,OAAO;wCACR,iDAAiD;wCACjD,MAAM,cAAc,GAAG,OAAO,CAAC,CAAC,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;wCACzG,OAAO,aAAa,cAAc,QAAQ,CAAC;oCAC/C;wCACI,OAAO,MAAM,CAAE,CAAS,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC;gCAC9C,CAAC;4BACL,CAAC,CAAC,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;4BAEZ,IAAI,UAAU,EAAE,CAAC;gCACb,IAAI,CAAC,MAAM,EAAE,CAAC;gCACd,MAAM,UAAU,CAAC;4BACrB,CAAC;wBACL,CAAC;oBACL,CAAC;gBACL,CAAC;YACL,CAAC;QACL,CAAC;QAAC,OAAO,KAAU,EAAE,CAAC;YAClB,KAAK,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC;YAC3B,MAAM,KAAK,CAAC;QAChB,CAAC;QAED,2FAA2F;QAC3F,oHAAoH;QACpH,MAAM,MAAM,GAAoC,YAAY,CAAC,CAAC;YAC1D,EAAE,MAAM,EAAE,YAAY,EAAE,MAAM,EAAE,YAAY,EAAE,KAAK,EAAE,YAAY,GAAG,YAAY,GAAG,CAAC,CAAC,CAAC,SAAS,CAAA;QAEnG,4CAA4C;QAC5C,MAAM,YAAY,GAAG,kBAAkB,CAAC,IAAI,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,kBAAkB,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;QAEvG,mEAAmE;QACnE,IAAI,YAAY,EAAE,CAAC;YACf,KAAK,MAAM,IAAI,IAAI,YAAY,EAAE,CAAC;gBAC9B,4EAA4E;gBAC5E,IAAK,IAAY,CAAC,UAAU,EAAE,CAAC;oBAC3B,IAAI,CAAC,EAAE,GAAI,IAAY,CAAC,UAAU,CAAC;oBACnC,OAAQ,IAAY,CAAC,UAAU,CAAC;gBACpC,CAAC;gBACD,0FAA0F;gBAC1F,IAAI,OAAO,IAAI,CAAC,UAAU,KAAK,QAAQ,EAAE,CAAC;oBACtC,IAAI,CAAC;wBACD,IAAI,CAAC,UAAU,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;oBAClD,CAAC;oBAAC,MAAM,CAAC;wBACL,mCAAmC;oBACvC,CAAC;gBACL,CAAC;YACL,CAAC;QACL,CAAC;QAED,IAAI,CAAC,UAAU,GAAG;YACd,MAAM,EAAE,kBAAkB,EAAE,4DAA4D;YACxF,MAAM,EAAE,IAAI,CAAC,MAAM;YACnB,cAAc,EAAE,IAAI,CAAC,GAAG,EAAE,GAAG,KAAK;YAClC,WAAW,EAAE,MAAM;YACnB,aAAa,EAAE,aAAa;YAC5B,MAAM,EAAE,IAAI,CAAC,MAAM;YACnB,QAAQ,EAAE,YAAY;SACzB,CAAA;QAED,oDAAoD;QACpD,MAAM,YAAY,GAAG,IAAI,CAAC,MAAM,CAAC,0BAA0B,CACvD,IAAI,CAAC,MAAM,EACX,kBAAkB,EAClB,YAAY,EACZ,IAAI,CAAC,OAAO,CACf,CAAC;QACF,IAAI,YAAY,KAAK,SAAS,EAAE,CAAC;YAC7B,IAAI,CAAC,UAAU,CAAC,YAAY,GAAG,YAAY,CAAC;QAChD,CAAC;QAED,IAAI,CAAC;YACD,IAAI,IAAI,CAAC,UAAU,EAAE,CAAC;gBAClB,IAAI,CAAC,MAAM,CAAC,cAAc,CAAC,IAAI,CAAC,UAAU,EAAE,IAAI,CAAC,OAAO,CAAC,CAAC;YAC9D,CAAC;QACL,CAAC;QAAC,OAAO,KAAU,EAAE,CAAC;YAClB,KAAK,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC;YAC3B,MAAM,KAAK,CAAC;QAChB,CAAC;IACL,CAAC;CAEJ;AA7MD,0DA6MC;AAED,MAAa,wBAAwB;IAId;IACR;IACA;IAJX,UAAU,CAAyC;IAEnD,YAAmB,MAA8C,EACtD,MAAe,EACf,OAAyB;QAFjB,WAAM,GAAN,MAAM,CAAwC;QACtD,WAAM,GAAN,MAAM,CAAS;QACf,YAAO,GAAP,OAAO,CAAkB;IACpC,CAAC;IAED,KAAK,CAAC,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC;QACzB,cAAc;QACd,IAAI,CAAC,UAAU,GAAG,SAAS,CAAC;QAC5B,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,KAAK,CACpB,IAAI,IAAI,CAAC,MAAM,CAAC,QAAQ,kEAAkE,CAC7F,CAAC;QACF,IAAI,CAAC;YACD,MAAM,UAAU,GAAG,MAAM,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC,MAAM,EAAE,IAAI,CAAC,OAAO,CAAC,CAAC;YACzE,kFAAkF;YAClF,MAAM,OAAO,GAAG,UAAU,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE;gBACtC,QAAQ,CAAC,CAAC,IAAI,EAAE,CAAC;oBACb,KAAK,MAAM;wBACP,OAAO,CAAC,CAAC,KAAK,CAAC;oBACnB,KAAK,MAAM;wBACP,OAAO,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;oBACnC,KAAK,OAAO;wBACR,iDAAiD;wBACjD,MAAM,cAAc,GAAG,OAAO,CAAC,CAAC,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;wBACzG,OAAO,WAAW,cAAc,MAAM,CAAC;oBAC3C;wBACI,OAAO,MAAM,CAAE,CAAS,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC;gBAC9C,CAAC;YACL,CAAC,CAAC,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;YACZ,MAAM,OAAO,CAAC;YACd,IAAI,CAAC,UAAU,GAAG,UAAU,CAAC,CAAC,mEAAmE;QACrG,CAAC;QAAC,OAAO,KAAU,EAAE,CAAC;YAClB,KAAK,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC;YAC3B,MAAM,KAAK,CAAC;QAChB,CAAC;IACL,CAAC;CACJ;AAvCD,4DAuCC"}
package/lib/cjs/Driver.js CHANGED
@@ -102,20 +102,17 @@ class AbstractDriver {
102
102
  });
103
103
  }
104
104
  async _execute(prompt, options) {
105
- this.logger.debug(`[${this.provider}] Executing prompt on ${options.model}`);
106
105
  try {
107
106
  const start = Date.now();
108
107
  let result;
109
- switch (options.output_modality) {
110
- case common_1.Modalities.text:
111
- result = await this.requestTextCompletion(prompt, options);
112
- this.validateResult(result, options);
113
- break;
114
- case common_1.Modalities.image:
115
- result = await this.requestImageGeneration(prompt, options);
116
- break;
117
- default:
118
- throw new Error(`Unsupported modality: ${options['output_modality'] ?? "No modality specified"}`);
108
+ if (this.isImageModel(options.model)) {
109
+ this.logger.debug(`[${this.provider}] Executing prompt on ${options.model}, image pathway.`);
110
+ result = await this.requestImageGeneration(prompt, options);
111
+ }
112
+ else {
113
+ this.logger.debug(`[${this.provider}] Executing prompt on ${options.model}, text pathway.`);
114
+ result = await this.requestTextCompletion(prompt, options);
115
+ this.validateResult(result, options);
119
116
  }
120
117
  const execution_time = Date.now() - start;
121
118
  return { ...result, prompt, execution_time };
@@ -125,6 +122,9 @@ class AbstractDriver {
125
122
  throw error;
126
123
  }
127
124
  }
125
+ isImageModel(_model) {
126
+ return false;
127
+ }
128
128
  // by default no stream is supported. we block and we return all at once
129
129
  async stream(segments, options) {
130
130
  const prompt = await this.createPrompt(segments, options);
@@ -132,6 +132,9 @@ class AbstractDriver {
132
132
  if (options.output_modality === common_1.Modalities.text && canStream) {
133
133
  return new CompletionStream_js_1.DefaultCompletionStream(this, prompt, options);
134
134
  }
135
+ else if (this.isImageModel(options.model)) {
136
+ return new CompletionStream_js_1.FallbackCompletionStream(this, prompt, options);
137
+ }
135
138
  else {
136
139
  return new CompletionStream_js_1.FallbackCompletionStream(this, prompt, options);
137
140
  }
@@ -168,6 +171,20 @@ class AbstractDriver {
168
171
  async listTrainableModels() {
169
172
  return [];
170
173
  }
174
+ /**
175
+ * Build the conversation context after streaming completion.
176
+ * Override this in driver implementations that support multi-turn conversations.
177
+ *
178
+ * @param prompt - The prompt that was sent (includes prior conversation context)
179
+ * @param result - The completion results from the streamed response
180
+ * @param toolUse - The tool calls from the streamed response (if any)
181
+ * @param options - The execution options
182
+ * @returns The updated conversation context, or undefined if not supported
183
+ */
184
+ buildStreamingConversation(_prompt, _result, _toolUse, _options) {
185
+ // Default implementation returns undefined - drivers can override
186
+ return undefined;
187
+ }
171
188
  async requestImageGeneration(_prompt, _options) {
172
189
  throw new Error("Image generation not implemented.");
173
190
  //Cannot be made abstract, as abstract methods are required in the derived class