modelfusion 0.125.0 → 0.127.0

package/CHANGELOG.md

@@ -1,5 +1,66 @@
 # Changelog
 
+## v0.127.0 - 2024-01-15
+
+### Changed
+
+- **breaking change**: `streamStructure` returns an async iterable over deep partial objects. If you need to get the fully validated final result, you can use the `fullResponse: true` option and await the `structurePromise` value. Example:
+
+  ```ts
+  const { structureStream, structurePromise } = await streamStructure({
+    model: ollama
+      .ChatTextGenerator({
+        model: "openhermes2.5-mistral",
+        maxGenerationTokens: 1024,
+        temperature: 0,
+      })
+      .asStructureGenerationModel(jsonStructurePrompt.text()),
+
+    schema: zodSchema(
+      z.object({
+        characters: z.array(
+          z.object({
+            name: z.string(),
+            class: z
+              .string()
+              .describe("Character class, e.g. warrior, mage, or thief."),
+            description: z.string(),
+          })
+        ),
+      })
+    ),
+
+    prompt:
+      "Generate 3 character descriptions for a fantasy role playing game.",
+
+    fullResponse: true,
+  });
+
+  for await (const partialStructure of structureStream) {
+    console.clear();
+    console.log(partialStructure);
+  }
+
+  const structure = await structurePromise;
+
+  console.clear();
+  console.log("FINAL STRUCTURE");
+  console.log(structure);
+  ```
+
+- **breaking change**: Renamed `text` value in `streamText` with `fullResponse: true` to `textPromise`.
+
+### Fixed
+
+- Ollama streaming.
+- Ollama structure generation and streaming.
+
+## v0.126.0 - 2024-01-15
+
+### Changed
+
+- **breaking change**: rename `useTool` to `runTool` and `useTools` to `runTools` to avoid confusion with React hooks.
+
 ## v0.125.0 - 2024-01-14
 
 ### Added

package/README.md

@@ -184,14 +184,9 @@ const structureStream = await streamStructure({
   prompt: "Generate 3 character descriptions for a fantasy role playing game.",
 });
 
-for await (const part of structureStream) {
-  if (!part.isComplete) {
-    const unknownPartialStructure = part.value;
-    console.log("partial value", unknownPartialStructure);
-  } else {
-    const fullyTypedStructure = part.value;
-    console.log("final value", fullyTypedStructure);
-  }
+for await (const partialStructure of structureStream) {
+  console.clear();
+  console.log(partialStructure);
 }
 ```
 
@@ -372,12 +367,12 @@ Tools are functions (and associated metadata) that can be executed by an AI mode
 
 ModelFusion offers several tools out-of-the-box: [Math.js](https://modelfusion.dev/guide/tools/available-tools/mathjs), [MediaWiki Search](https://modelfusion.dev/guide/tools/available-tools/mediawiki-search), [SerpAPI](https://modelfusion.dev/guide/tools/available-tools/serpapi), [Google Custom Search](https://modelfusion.dev/guide/tools/available-tools/google-custom-search). You can also create [custom tools](https://modelfusion.dev/guide/tools).
 
-#### [useTool](https://modelfusion.dev/guide/tools/use-tool)
+#### [runTool](https://modelfusion.dev/guide/tools/run-tool)
 
-With `useTool`, you can ask a tool-compatible language model (e.g. OpenAI chat) to invoke a single tool. `useTool` first generates a tool call and then executes the tool with the arguments.
+With `runTool`, you can ask a tool-compatible language model (e.g. OpenAI chat) to invoke a single tool. `runTool` first generates a tool call and then executes the tool with the arguments.
 
 ```ts
-const { tool, toolCall, args, ok, result } = await useTool({
+const { tool, toolCall, args, ok, result } = await runTool({
   model: openai.ChatTextGenerator({ model: "gpt-3.5-turbo" }),
   too: calculator,
   prompt: [openai.ChatMessage.user("What's fourteen times twelve?")],
@@ -390,12 +385,12 @@ console.log(`Ok:`, ok);
 console.log(`Result or Error:`, result);
 ```
 
-#### [useTools](https://modelfusion.dev/guide/tools/use-tools)
+#### [runTools](https://modelfusion.dev/guide/tools/run-tools)
 
-With `useTools`, you can ask a language model to generate several tool calls as well as text. The model will choose which tools (if any) should be called with which arguments. Both the text and the tool calls are optional. This function executes the tools.
+With `runTools`, you can ask a language model to generate several tool calls as well as text. The model will choose which tools (if any) should be called with which arguments. Both the text and the tool calls are optional. This function executes the tools.
 
 ```ts
-const { text, toolResults } = await useTools({
+const { text, toolResults } = await runTools({
   model: openai.ChatTextGenerator({ model: "gpt-3.5-turbo" }),
   tools: [calculator /* ... */],
   prompt: [openai.ChatMessage.user("What's fourteen times twelve?")],
@@ -404,7 +399,7 @@ const { text, toolResults } = await useTools({
 
 #### [Agent Loop](https://modelfusion.dev/guide/tools/agent-loop)
 
-You can use `useTools` to implement an agent loop that responds to user messages and executes tools. [Learn more](https://modelfusion.dev/guide/tools/agent-loop).
+You can use `runTools` to implement an agent loop that responds to user messages and executes tools. [Learn more](https://modelfusion.dev/guide/tools/agent-loop).
 
 ### [Vector Indices](https://modelfusion.dev/guide/vector-index)
 
@@ -580,8 +575,8 @@ modelfusion.setLogFormat("detailed-object"); // log full events
   - [Embed Value](https://modelfusion.dev/guide/function/embed)
   - [Classify Value](https://modelfusion.dev/guide/function/classify)
 - [Tools](https://modelfusion.dev/guide/tools)
-  - [Use Tool](https://modelfusion.dev/guide/tools/use-tool)
-  - [Use Tools](https://modelfusion.dev/guide/tools/use-tools)
+  - [Run Tool](https://modelfusion.dev/guide/tools/run-tool)
+  - [Run Tools](https://modelfusion.dev/guide/tools/run-tools)
   - [Agent Loop](https://modelfusion.dev/guide/tools/agent-loop)
   - [Available Tools](https://modelfusion.dev/guide/tools/available-tools/)
   - [Custom Tools](https://modelfusion.dev/guide/tools/custom-tools)

package/core/FunctionEvent.d.ts

@@ -1,8 +1,8 @@
 import { ModelCallFinishedEvent, ModelCallStartedEvent } from "../model-function/ModelCallEvent.js";
 import { RetrieveFinishedEvent, RetrieveStartedEvent } from "../retriever/RetrieveEvent.js";
 import { ExecuteToolFinishedEvent, ExecuteToolStartedEvent } from "../tool/execute-tool/ExecuteToolEvent.js";
-import { UseToolFinishedEvent, UseToolStartedEvent } from "../tool/use-tool/UseToolEvent.js";
-import { useToolsFinishedEvent, useToolsStartedEvent } from "../tool/use-tools/UseToolsEvent.js";
+import { runToolFinishedEvent, runToolStartedEvent } from "../tool/run-tool/RunToolEvent.js";
+import { runToolsFinishedEvent, runToolsStartedEvent } from "../tool/run-tools/RunToolsEvent.js";
 import { UpsertIntoVectorIndexFinishedEvent, UpsertIntoVectorIndexStartedEvent } from "../vector-index/UpsertIntoVectorIndexEvent.js";
 import { ExecuteFunctionFinishedEvent, ExecuteFunctionStartedEvent } from "./ExecuteFunctionEvent.js";
 import { ExtensionFunctionFinishedEvent, ExtensionFunctionStartedEvent } from "./ExtensionFunctionEvent.js";
@@ -82,4 +82,4 @@ export interface BaseFunctionFinishedEvent extends BaseFunctionEvent {
      */
     result: BaseFunctionFinishedEventResult;
 }
-export type FunctionEvent = ExecuteFunctionStartedEvent | ExecuteFunctionFinishedEvent | ExecuteToolStartedEvent | ExecuteToolFinishedEvent | ExtensionFunctionStartedEvent | ExtensionFunctionFinishedEvent | ModelCallStartedEvent | ModelCallFinishedEvent | RetrieveStartedEvent | RetrieveFinishedEvent | UpsertIntoVectorIndexStartedEvent | UpsertIntoVectorIndexFinishedEvent | UseToolStartedEvent | UseToolFinishedEvent | useToolsStartedEvent | useToolsFinishedEvent;
+export type FunctionEvent = ExecuteFunctionStartedEvent | ExecuteFunctionFinishedEvent | ExecuteToolStartedEvent | ExecuteToolFinishedEvent | ExtensionFunctionStartedEvent | ExtensionFunctionFinishedEvent | ModelCallStartedEvent | ModelCallFinishedEvent | RetrieveStartedEvent | RetrieveFinishedEvent | UpsertIntoVectorIndexStartedEvent | UpsertIntoVectorIndexFinishedEvent | runToolStartedEvent | runToolFinishedEvent | runToolsStartedEvent | runToolsFinishedEvent;

package/model-function/generate-structure/StructureFromTextGenerationModel.cjs

@@ -31,7 +31,7 @@ class StructureFromTextGenerationModel {
     }
     getModelWithJsonOutput(schema) {
         if (this.template.withJsonOutput != null) {
-            return this.template.withJsonOutput?.({
+            return this.template.withJsonOutput({
                 model: this.model,
                 schema,
             });
@@ -40,7 +40,7 @@ class StructureFromTextGenerationModel {
     }
     async doGenerateStructure(schema, prompt, options) {
         const { rawResponse, text } = await (0, generateText_js_1.generateText)({
-            model: this.model,
+            model: this.getModelWithJsonOutput(schema),
             prompt: this.template.createPrompt(prompt, schema),
             fullResponse: true,
             ...options,

package/model-function/generate-structure/StructureFromTextGenerationModel.js

@@ -28,7 +28,7 @@ export class StructureFromTextGenerationModel {
     }
     getModelWithJsonOutput(schema) {
         if (this.template.withJsonOutput != null) {
-            return this.template.withJsonOutput?.({
+            return this.template.withJsonOutput({
                 model: this.model,
                 schema,
             });
@@ -37,7 +37,7 @@ export class StructureFromTextGenerationModel {
     }
     async doGenerateStructure(schema, prompt, options) {
         const { rawResponse, text } = await generateText({
-            model: this.model,
+            model: this.getModelWithJsonOutput(schema),
             prompt: this.template.createPrompt(prompt, schema),
             fullResponse: true,
             ...options,

package/model-function/generate-structure/StructureFromTextStreamingModel.cjs

@@ -11,7 +11,7 @@ class StructureFromTextStreamingModel extends StructureFromTextGenerationModel_j
     }
     async doStreamStructure(schema, prompt, options) {
         const textStream = await (0, streamText_js_1.streamText)({
-            model: this.model,
+            model: this.getModelWithJsonOutput(schema),
             prompt: this.template.createPrompt(prompt, schema),
             ...options,
         });

package/model-function/generate-structure/StructureFromTextStreamingModel.js

@@ -8,7 +8,7 @@ export class StructureFromTextStreamingModel extends StructureFromTextGeneration
     }
     async doStreamStructure(schema, prompt, options) {
         const textStream = await streamText({
-            model: this.model,
+            model: this.getModelWithJsonOutput(schema),
             prompt: this.template.createPrompt(prompt, schema),
             ...options,
         });

package/model-function/generate-structure/streamStructure.cjs

@@ -10,6 +10,12 @@ async function streamStructure({ model, schema, prompt, fullResponse, ...options
         : prompt;
     let accumulatedText = "";
     let lastStructure;
+    let resolveStructure;
+    let rejectStructure;
+    const structurePromise = new Promise((resolve, reject) => {
+        resolveStructure = resolve;
+        rejectStructure = reject;
+    });
     const callResponse = await (0, executeStreamCall_js_1.executeStreamCall)({
         functionType: "stream-structure",
         input: {
@@ -29,29 +35,25 @@ async function streamStructure({ model, schema, prompt, fullResponse, ...options
             // only send a new part into the stream when the partial structure has changed:
             if (!(0, isDeepEqualData_js_1.isDeepEqualData)(lastStructure, latestStructure)) {
                 lastStructure = latestStructure;
-                return {
-                    isComplete: false,
-                    value: lastStructure,
-                };
+                return lastStructure;
             }
             return undefined;
         },
-        processFinished: () => {
+        onDone: () => {
             // process the final result (full type validation):
             const parseResult = schema.validate(lastStructure);
-            if (!parseResult.success) {
-                reportError(parseResult.error);
-                throw parseResult.error;
+            if (parseResult.success) {
+                resolveStructure(parseResult.data);
+            }
+            else {
+                rejectStructure(parseResult.error);
             }
-            return {
-                isComplete: true,
-                value: parseResult.data,
-            };
         },
     });
     return fullResponse
         ? {
             structureStream: callResponse.value,
+            structurePromise,
             metadata: callResponse.metadata,
         }
         : callResponse.value;

package/model-function/generate-structure/streamStructure.d.ts

@@ -1,25 +1,12 @@
+import type { PartialDeep } from "type-fest";
 import { FunctionOptions } from "../../core/FunctionOptions.js";
 import { JsonSchemaProducer } from "../../core/schema/JsonSchemaProducer.js";
 import { Schema } from "../../core/schema/Schema.js";
 import { ModelCallMetadata } from "../ModelCallMetadata.js";
 import { StructureStreamingModel } from "./StructureGenerationModel.js";
-export type StructureStreamPart<STRUCTURE> = {
-    isComplete: false;
-    value: unknown;
-} | {
-    isComplete: true;
-    value: STRUCTURE;
-};
 /**
  * Generate and stream an object for a prompt and a structure definition.
  *
- * The final object is typed according to the structure definition.
- * The partial objects are of unknown type,
- * but are supposed to be partial version of the final object
- * (unless the underlying model returns invalid data).
- *
- * The structure definition is used as part of the final prompt.
- *
  * For the OpenAI chat model, this generates and parses a function call with a single function.
  *
  * @see https://modelfusion.dev/guide/function/generate-structure
@@ -44,15 +31,8 @@ export type StructureStreamPart<STRUCTURE> = {
  *   ]
  * });
  *
- * for await (const part of structureStream) {
- *   if (!part.isComplete) {
- *     const unknownPartialStructure = part.value;
- *     // use your own logic to handle partial structures, e.g. with Zod .deepPartial()
- *     // it depends on your application at which points you want to act on the partial structures
- *   } else {
- *     const fullyTypedStructure = part.value;
- *     // ...
- *   }
+ * for await (const partialStructure of structureStream) {
+ *   // ...
  * }
  *
  * @param {StructureStreamingModel<PROMPT>} structureGenerator - The model to use for streaming
@@ -60,26 +40,28 @@ export type StructureStreamPart<STRUCTURE> = {
  * @param {PROMPT | ((schema: Schema<STRUCTURE>) => PROMPT)} prompt
  * The prompt to be used.
  * You can also pass a function that takes the schema as an argument and returns the prompt.
- * @param {FunctionOptions} [options] - Optional function options
  *
  * @returns {AsyncIterableResultPromise<StructureStreamPart<STRUCTURE>>}
  * The async iterable result promise.
- * Each part of the stream is either a partial structure or the final structure.
- * It contains a isComplete flag to indicate whether the structure is complete,
- * and a value that is either the partial structure or the final structure.
+ * Each part of the stream is a partial structure.
  */
 export declare function streamStructure<STRUCTURE, PROMPT>(args: {
     model: StructureStreamingModel<PROMPT>;
     schema: Schema<STRUCTURE> & JsonSchemaProducer;
     prompt: PROMPT | ((schema: Schema<STRUCTURE>) => PROMPT);
     fullResponse?: false;
-} & FunctionOptions): Promise<AsyncIterable<StructureStreamPart<STRUCTURE>>>;
+} & FunctionOptions): Promise<AsyncIterable<PartialDeep<STRUCTURE, {
+    recurseIntoArrays: true;
+}>>>;
 export declare function streamStructure<STRUCTURE, PROMPT>(args: {
     model: StructureStreamingModel<PROMPT>;
     schema: Schema<STRUCTURE> & JsonSchemaProducer;
     prompt: PROMPT | ((schema: Schema<STRUCTURE>) => PROMPT);
     fullResponse: true;
 } & FunctionOptions): Promise<{
-    structureStream: AsyncIterable<StructureStreamPart<STRUCTURE>>;
+    structureStream: AsyncIterable<PartialDeep<STRUCTURE, {
+        recurseIntoArrays: true;
+    }>>;
+    structurePromise: PromiseLike<STRUCTURE>;
     metadata: Omit<ModelCallMetadata, "durationInMs" | "finishTimestamp">;
 }>;

package/model-function/generate-structure/streamStructure.js

@@ -7,6 +7,12 @@ export async function streamStructure({ model, schema, prompt, fullResponse, ...
         : prompt;
     let accumulatedText = "";
     let lastStructure;
+    let resolveStructure;
+    let rejectStructure;
+    const structurePromise = new Promise((resolve, reject) => {
+        resolveStructure = resolve;
+        rejectStructure = reject;
+    });
     const callResponse = await executeStreamCall({
         functionType: "stream-structure",
         input: {
@@ -26,29 +32,25 @@ export async function streamStructure({ model, schema, prompt, fullResponse, ...
             // only send a new part into the stream when the partial structure has changed:
             if (!isDeepEqualData(lastStructure, latestStructure)) {
                 lastStructure = latestStructure;
-                return {
-                    isComplete: false,
-                    value: lastStructure,
-                };
+                return lastStructure;
             }
             return undefined;
         },
-        processFinished: () => {
+        onDone: () => {
             // process the final result (full type validation):
             const parseResult = schema.validate(lastStructure);
-            if (!parseResult.success) {
-                reportError(parseResult.error);
-                throw parseResult.error;
+            if (parseResult.success) {
+                resolveStructure(parseResult.data);
+            }
+            else {
+                rejectStructure(parseResult.error);
             }
-            return {
-                isComplete: true,
-                value: parseResult.data,
-            };
         },
     });
     return fullResponse
         ? {
             structureStream: callResponse.value,
+            structurePromise,
             metadata: callResponse.metadata,
         }
         : callResponse.value;

package/model-function/generate-text/streamText.cjs

@@ -46,7 +46,7 @@ async function streamText({ model, prompt, fullResponse, ...options }) {
     return fullResponse
         ? {
             textStream: callResponse.value,
-            text: textPromise,
+            textPromise,
             metadata: callResponse.metadata,
         }
         : callResponse.value;

package/model-function/generate-text/streamText.d.ts

@@ -37,6 +37,6 @@ export declare function streamText<PROMPT>(args: {
     fullResponse: true;
 } & FunctionOptions): Promise<{
     textStream: AsyncIterable<string>;
-    text: PromiseLike<string>;
+    textPromise: PromiseLike<string>;
     metadata: Omit<ModelCallMetadata, "durationInMs" | "finishTimestamp">;
 }>;

package/model-function/generate-text/streamText.js

@@ -43,7 +43,7 @@ export async function streamText({ model, prompt, fullResponse, ...options }) {
     return fullResponse
         ? {
             textStream: callResponse.value,
-            text: textPromise,
+            textPromise,
             metadata: callResponse.metadata,
         }
         : callResponse.value;

package/model-provider/mistral/MistralChatModel.test.cjs

@@ -44,7 +44,7 @@ describe("streamText", () => {
             ]);
         });
         it("should return text", async () => {
-            const { text } = await (0, streamText_js_1.streamText)({
+            const { textPromise } = await (0, streamText_js_1.streamText)({
                 model: new MistralChatModel_js_1.MistralChatModel({
                     api: new MistralApiConfiguration_js_1.MistralApiConfiguration({ apiKey: "test-key" }),
                     model: "mistral-tiny",
@@ -52,7 +52,7 @@ describe("streamText", () => {
                 prompt: "hello",
                 fullResponse: true,
             });
-            expect(await text).toStrictEqual("Hello, world!");
+            expect(await textPromise).toStrictEqual("Hello, world!");
         });
     });
 });

package/model-provider/mistral/MistralChatModel.test.js

@@ -42,7 +42,7 @@ describe("streamText", () => {
             ]);
         });
         it("should return text", async () => {
-            const { text } = await streamText({
+            const { textPromise } = await streamText({
                 model: new MistralChatModel({
                     api: new MistralApiConfiguration({ apiKey: "test-key" }),
                     model: "mistral-tiny",
@@ -50,7 +50,7 @@ describe("streamText", () => {
                 prompt: "hello",
                 fullResponse: true,
             });
-            expect(await text).toStrictEqual("Hello, world!");
+            expect(await textPromise).toStrictEqual("Hello, world!");
         });
     });
 });

package/model-provider/ollama/OllamaChatModel.cjs

@@ -231,7 +231,7 @@ const ollamaChatStreamChunkSchema = zod_1.z.discriminatedUnion("done", [
         created_at: zod_1.z.string(),
         total_duration: zod_1.z.number(),
         load_duration: zod_1.z.number().optional(),
-        prompt_eval_count: zod_1.z.number(),
+        prompt_eval_count: zod_1.z.number().optional(),
         prompt_eval_duration: zod_1.z.number().optional(),
         eval_count: zod_1.z.number(),
         eval_duration: zod_1.z.number(),

package/model-provider/ollama/OllamaChatModel.d.ts

@@ -94,10 +94,10 @@ export declare class OllamaChatModel extends AbstractModel<OllamaChatModelSettin
         done: true;
         created_at: string;
         total_duration: number;
-        prompt_eval_count: number;
         eval_count: number;
         eval_duration: number;
         load_duration?: number | undefined;
+        prompt_eval_count?: number | undefined;
         prompt_eval_duration?: number | undefined;
     }>>>;
     extractTextDelta(delta: unknown): string | undefined;
@@ -197,7 +197,7 @@ declare const ollamaChatStreamChunkSchema: z.ZodDiscriminatedUnion<"done", [z.Zo
     created_at: z.ZodString;
     total_duration: z.ZodNumber;
     load_duration: z.ZodOptional<z.ZodNumber>;
-    prompt_eval_count: z.ZodNumber;
+    prompt_eval_count: z.ZodOptional<z.ZodNumber>;
     prompt_eval_duration: z.ZodOptional<z.ZodNumber>;
     eval_count: z.ZodNumber;
     eval_duration: z.ZodNumber;
@@ -206,20 +206,20 @@ declare const ollamaChatStreamChunkSchema: z.ZodDiscriminatedUnion<"done", [z.Zo
     done: true;
     created_at: string;
     total_duration: number;
-    prompt_eval_count: number;
     eval_count: number;
     eval_duration: number;
     load_duration?: number | undefined;
+    prompt_eval_count?: number | undefined;
     prompt_eval_duration?: number | undefined;
 }, {
     model: string;
     done: true;
     created_at: string;
     total_duration: number;
-    prompt_eval_count: number;
     eval_count: number;
     eval_duration: number;
     load_duration?: number | undefined;
+    prompt_eval_count?: number | undefined;
     prompt_eval_duration?: number | undefined;
 }>]>;
 export type OllamaChatStreamChunk = z.infer<typeof ollamaChatStreamChunkSchema>;
@@ -274,10 +274,10 @@ export declare const OllamaChatResponseFormat: {
             done: true;
             created_at: string;
             total_duration: number;
-            prompt_eval_count: number;
             eval_count: number;
             eval_duration: number;
             load_duration?: number | undefined;
+            prompt_eval_count?: number | undefined;
             prompt_eval_duration?: number | undefined;
         }>>>;
     };

package/model-provider/ollama/OllamaChatModel.js

@@ -227,7 +227,7 @@ const ollamaChatStreamChunkSchema = z.discriminatedUnion("done", [
         created_at: z.string(),
         total_duration: z.number(),
         load_duration: z.number().optional(),
-        prompt_eval_count: z.number(),
+        prompt_eval_count: z.number().optional(),
         prompt_eval_duration: z.number().optional(),
         eval_count: z.number(),
         eval_duration: z.number(),

package/model-provider/ollama/OllamaCompletionModel.cjs

@@ -180,7 +180,7 @@ class OllamaCompletionModel extends AbstractModel_js_1.AbstractModel {
         return this.settings.promptTemplate ?? OllamaCompletionPrompt_js_1.Text;
     }
     withJsonOutput() {
-        return this;
+        return this.withSettings({ format: "json" });
     }
     withTextPrompt() {
         return this.withPromptTemplate(this.promptTemplateProvider.text());
@@ -235,7 +235,7 @@ const ollamaCompletionStreamChunkSchema = zod_1.z.discriminatedUnion("done", [
         load_duration: zod_1.z.number().optional(),
         sample_count: zod_1.z.number().optional(),
         sample_duration: zod_1.z.number().optional(),
-        prompt_eval_count: zod_1.z.number(),
+        prompt_eval_count: zod_1.z.number().optional(),
         prompt_eval_duration: zod_1.z.number().optional(),
         eval_count: zod_1.z.number(),
         eval_duration: zod_1.z.number(),

package/model-provider/ollama/OllamaCompletionModel.d.ts

@@ -127,12 +127,12 @@ export declare class OllamaCompletionModel<CONTEXT_WINDOW_SIZE extends number |
         done: true;
         created_at: string;
         total_duration: number;
-        prompt_eval_count: number;
         eval_count: number;
         eval_duration: number;
         load_duration?: number | undefined;
         sample_count?: number | undefined;
         sample_duration?: number | undefined;
+        prompt_eval_count?: number | undefined;
         prompt_eval_duration?: number | undefined;
         context?: number[] | undefined;
     }>>>;
@@ -209,7 +209,7 @@ declare const ollamaCompletionStreamChunkSchema: z.ZodDiscriminatedUnion<"done",
     load_duration: z.ZodOptional<z.ZodNumber>;
     sample_count: z.ZodOptional<z.ZodNumber>;
     sample_duration: z.ZodOptional<z.ZodNumber>;
-    prompt_eval_count: z.ZodNumber;
+    prompt_eval_count: z.ZodOptional<z.ZodNumber>;
     prompt_eval_duration: z.ZodOptional<z.ZodNumber>;
     eval_count: z.ZodNumber;
     eval_duration: z.ZodNumber;
@@ -219,12 +219,12 @@ declare const ollamaCompletionStreamChunkSchema: z.ZodDiscriminatedUnion<"done",
     done: true;
     created_at: string;
     total_duration: number;
-    prompt_eval_count: number;
     eval_count: number;
     eval_duration: number;
     load_duration?: number | undefined;
     sample_count?: number | undefined;
     sample_duration?: number | undefined;
+    prompt_eval_count?: number | undefined;
     prompt_eval_duration?: number | undefined;
     context?: number[] | undefined;
 }, {
@@ -232,12 +232,12 @@ declare const ollamaCompletionStreamChunkSchema: z.ZodDiscriminatedUnion<"done",
     done: true;
     created_at: string;
     total_duration: number;
-    prompt_eval_count: number;
     eval_count: number;
     eval_duration: number;
     load_duration?: number | undefined;
     sample_count?: number | undefined;
     sample_duration?: number | undefined;
+    prompt_eval_count?: number | undefined;
     prompt_eval_duration?: number | undefined;
     context?: number[] | undefined;
 }>]>;
@@ -288,12 +288,12 @@ export declare const OllamaCompletionResponseFormat: {
             done: true;
             created_at: string;
             total_duration: number;
-            prompt_eval_count: number;
             eval_count: number;
             eval_duration: number;
             load_duration?: number | undefined;
             sample_count?: number | undefined;
             sample_duration?: number | undefined;
+            prompt_eval_count?: number | undefined;
             prompt_eval_duration?: number | undefined;
             context?: number[] | undefined;
         }>>>;

package/model-provider/ollama/OllamaCompletionModel.js

@@ -177,7 +177,7 @@ export class OllamaCompletionModel extends AbstractModel {
         return this.settings.promptTemplate ?? Text;
     }
     withJsonOutput() {
-        return this;
+        return this.withSettings({ format: "json" });
     }
     withTextPrompt() {
         return this.withPromptTemplate(this.promptTemplateProvider.text());
@@ -231,7 +231,7 @@ const ollamaCompletionStreamChunkSchema = z.discriminatedUnion("done", [
         load_duration: z.number().optional(),
         sample_count: z.number().optional(),
         sample_duration: z.number().optional(),
-        prompt_eval_count: z.number(),
+        prompt_eval_count: z.number().optional(),
         prompt_eval_duration: z.number().optional(),
         eval_count: z.number(),
         eval_duration: z.number(),

package/model-provider/ollama/OllamaCompletionModel.test.cjs

@@ -124,13 +124,11 @@ describe("streamStructure", () => {
             schema: (0, ZodSchema_js_1.zodSchema)(zod_1.z.object({ name: zod_1.z.string() })),
             prompt: "generate a name",
         });
-        // note: space moved to last chunk bc of trimming
         expect(await (0, arrayFromAsync_js_1.arrayFromAsync)(stream)).toStrictEqual([
-            { isComplete: false, value: {} },
-            { isComplete: false, value: { name: "" } },
-            { isComplete: false, value: { name: "M" } },
-            { isComplete: false, value: { name: "Mike" } },
-            { isComplete: true, value: { name: "Mike" } },
+            {},
+            { name: "" },
+            { name: "M" },
+            { name: "Mike" },
         ]);
     });
 });

package/model-provider/ollama/OllamaCompletionModel.test.js

@@ -122,13 +122,11 @@ describe("streamStructure", () => {
             schema: zodSchema(z.object({ name: z.string() })),
             prompt: "generate a name",
         });
-        // note: space moved to last chunk bc of trimming
         expect(await arrayFromAsync(stream)).toStrictEqual([
-            { isComplete: false, value: {} },
-            { isComplete: false, value: { name: "" } },
-            { isComplete: false, value: { name: "M" } },
-            { isComplete: false, value: { name: "Mike" } },
-            { isComplete: true, value: { name: "Mike" } },
+            {},
+            { name: "" },
+            { name: "M" },
+            { name: "Mike" },
         ]);
     });
 });

package/model-provider/openai/OpenAIChatModel.test.cjs

@@ -89,13 +89,11 @@ describe("streamStructure", () => {
             schema: (0, ZodSchema_js_1.zodSchema)(zod_1.z.object({ name: zod_1.z.string() })),
             prompt: "generate a name",
         });
-        // note: space moved to last chunk bc of trimming
         expect(await (0, arrayFromAsync_js_1.arrayFromAsync)(stream)).toStrictEqual([
-            { isComplete: false, value: {} },
-            { isComplete: false, value: { name: "" } },
-            { isComplete: false, value: { name: "M" } },
-            { isComplete: false, value: { name: "Mike" } },
-            { isComplete: true, value: { name: "Mike" } },
+            {},
+            { name: "" },
+            { name: "M" },
+            { name: "Mike" },
         ]);
     });
 });

package/model-provider/openai/OpenAIChatModel.test.js

@@ -87,13 +87,11 @@ describe("streamStructure", () => {
             schema: zodSchema(z.object({ name: z.string() })),
             prompt: "generate a name",
         });
-        // note: space moved to last chunk bc of trimming
         expect(await arrayFromAsync(stream)).toStrictEqual([
-            { isComplete: false, value: {} },
-            { isComplete: false, value: { name: "" } },
-            { isComplete: false, value: { name: "M" } },
-            { isComplete: false, value: { name: "Mike" } },
-            { isComplete: true, value: { name: "Mike" } },
+            {},
+            { name: "" },
+            { name: "M" },
+            { name: "Mike" },
         ]);
     });
 });

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "modelfusion",
   "description": "The TypeScript library for building AI applications.",
-  "version": "0.125.0",
+  "version": "0.127.0",
   "author": "Lars Grammel",
   "license": "MIT",
   "keywords": [
@@ -80,6 +80,7 @@
     "@vitest/ui": "1.1.0",
     "eslint": "^8.45.0",
     "eslint-config-prettier": "9.1.0",
-    "msw": "2.0.11"
+    "msw": "2.0.11",
+    "type-fest": "4.9.0"
   }
 }

package/tool/index.cjs

@@ -27,5 +27,5 @@ __exportStar(require("./WebSearchTool.cjs"), exports);
 __exportStar(require("./execute-tool/index.cjs"), exports);
 __exportStar(require("./generate-tool-call/index.cjs"), exports);
 __exportStar(require("./generate-tool-calls/index.cjs"), exports);
-__exportStar(require("./use-tool/index.cjs"), exports);
-__exportStar(require("./use-tools/index.cjs"), exports);
+__exportStar(require("./run-tool/index.cjs"), exports);
+__exportStar(require("./run-tools/index.cjs"), exports);

package/tool/index.d.ts

@@ -11,5 +11,5 @@ export * from "./WebSearchTool.js";
 export * from "./execute-tool/index.js";
 export * from "./generate-tool-call/index.js";
 export * from "./generate-tool-calls/index.js";
-export * from "./use-tool/index.js";
-export * from "./use-tools/index.js";
+export * from "./run-tool/index.js";
+export * from "./run-tools/index.js";

package/tool/index.js

@@ -11,5 +11,5 @@ export * from "./WebSearchTool.js";
 export * from "./execute-tool/index.js";
 export * from "./generate-tool-call/index.js";
 export * from "./generate-tool-calls/index.js";
-export * from "./use-tool/index.js";
-export * from "./use-tools/index.js";
+export * from "./run-tool/index.js";
+export * from "./run-tools/index.js";

package/tool/run-tool/RunToolEvent.d.ts

@@ -0,0 +1,7 @@
+import { BaseFunctionFinishedEvent, BaseFunctionStartedEvent } from "../../core/FunctionEvent.js";
+export interface runToolStartedEvent extends BaseFunctionStartedEvent {
+    functionType: "run-tool";
+}
+export interface runToolFinishedEvent extends BaseFunctionFinishedEvent {
+    functionType: "run-tool";
+}

package/tool/{use-tool → run-tool}/index.cjs

@@ -14,5 +14,5 @@ var __exportStar = (this && this.__exportStar) || function(m, exports) {
     for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
 };
 Object.defineProperty(exports, "__esModule", { value: true });
-__exportStar(require("./UseToolEvent.cjs"), exports);
-__exportStar(require("./useTool.cjs"), exports);
+__exportStar(require("./RunToolEvent.cjs"), exports);
+__exportStar(require("./runTool.cjs"), exports);

package/tool/run-tool/index.d.ts

@@ -0,0 +1,2 @@
+export * from "./RunToolEvent.js";
+export * from "./runTool.js";

package/tool/run-tool/index.js

@@ -0,0 +1,2 @@
+export * from "./RunToolEvent.js";
+export * from "./runTool.js";

package/tool/{use-tool/useTool.cjs → run-tool/runTool.cjs}

@@ -1,11 +1,11 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.useTool = void 0;
+exports.runTool = void 0;
 const executeFunctionCall_js_1 = require("../../core/executeFunctionCall.cjs");
 const safeExecuteToolCall_js_1 = require("../execute-tool/safeExecuteToolCall.cjs");
 const generateToolCall_js_1 = require("../generate-tool-call/generateToolCall.cjs");
 /**
- * `useTool` uses `generateToolCall` to generate parameters for a tool and
+ * `runTool` uses `generateToolCall` to generate parameters for a tool and
  * then executes the tool with the parameters using `executeTool`.
  *
  * @returns The result contains the name of the tool (`tool` property),
@@ -15,7 +15,7 @@ const generateToolCall_js_1 = require("../generate-tool-call/generateToolCall.cj
  * @see {@link generateToolCall}
  * @see {@link executeTool}
  */
-async function useTool({ model, tool, prompt, ...options }) {
+async function runTool({ model, tool, prompt, ...options }) {
     // Note: PROMPT must not be a function.
     const expandedPrompt = typeof prompt === "function"
         ? prompt(tool)
@@ -23,8 +23,8 @@ async function useTool({ model, tool, prompt, ...options }) {
     return (0, executeFunctionCall_js_1.executeFunctionCall)({
         options,
         input: expandedPrompt,
-        functionType: "use-tool",
+        functionType: "run-tool",
         execute: async (options) => (0, safeExecuteToolCall_js_1.safeExecuteToolCall)(tool, await (0, generateToolCall_js_1.generateToolCall)({ model, tool, prompt: expandedPrompt, ...options }), options),
     });
 }
-exports.useTool = useTool;
+exports.runTool = runTool;

package/tool/{use-tool/useTool.d.ts → run-tool/runTool.d.ts}

@@ -3,7 +3,7 @@ import { Tool } from "../Tool.js";
 import { ToolCallResult } from "../ToolCallResult.js";
 import { ToolCallGenerationModel, ToolCallGenerationModelSettings } from "../generate-tool-call/ToolCallGenerationModel.js";
 /**
- * `useTool` uses `generateToolCall` to generate parameters for a tool and
+ * `runTool` uses `generateToolCall` to generate parameters for a tool and
  * then executes the tool with the parameters using `executeTool`.
  *
  * @returns The result contains the name of the tool (`tool` property),
@@ -13,7 +13,7 @@ import { ToolCallGenerationModel, ToolCallGenerationModelSettings } from "../gen
  * @see {@link generateToolCall}
  * @see {@link executeTool}
  */
-export declare function useTool<PROMPT, TOOL extends Tool<string, any, any>>({ model, tool, prompt, ...options }: {
+export declare function runTool<PROMPT, TOOL extends Tool<string, any, any>>({ model, tool, prompt, ...options }: {
     model: ToolCallGenerationModel<PROMPT, ToolCallGenerationModelSettings>;
     tool: TOOL;
     prompt: PROMPT | ((tool: TOOL) => PROMPT);

package/tool/{use-tool/useTool.js → run-tool/runTool.js}

@@ -2,7 +2,7 @@ import { executeFunctionCall } from "../../core/executeFunctionCall.js";
 import { safeExecuteToolCall } from "../execute-tool/safeExecuteToolCall.js";
 import { generateToolCall } from "../generate-tool-call/generateToolCall.js";
 /**
- * `useTool` uses `generateToolCall` to generate parameters for a tool and
+ * `runTool` uses `generateToolCall` to generate parameters for a tool and
  * then executes the tool with the parameters using `executeTool`.
  *
  * @returns The result contains the name of the tool (`tool` property),
@@ -12,7 +12,7 @@ import { generateToolCall } from "../generate-tool-call/generateToolCall.js";
  * @see {@link generateToolCall}
  * @see {@link executeTool}
  */
-export async function useTool({ model, tool, prompt, ...options }) {
+export async function runTool({ model, tool, prompt, ...options }) {
     // Note: PROMPT must not be a function.
     const expandedPrompt = typeof prompt === "function"
         ? prompt(tool)
@@ -20,7 +20,7 @@ export async function useTool({ model, tool, prompt, ...options }) {
     return executeFunctionCall({
         options,
         input: expandedPrompt,
-        functionType: "use-tool",
+        functionType: "run-tool",
         execute: async (options) => safeExecuteToolCall(tool, await generateToolCall({ model, tool, prompt: expandedPrompt, ...options }), options),
     });
 }

package/tool/run-tools/RunToolsEvent.d.ts

@@ -0,0 +1,7 @@
+import { BaseFunctionFinishedEvent, BaseFunctionStartedEvent } from "../../core/FunctionEvent.js";
+export interface runToolsStartedEvent extends BaseFunctionStartedEvent {
+    functionType: "run-tools";
+}
+export interface runToolsFinishedEvent extends BaseFunctionFinishedEvent {
+    functionType: "run-tools";
+}

package/tool/{use-tools → run-tools}/index.cjs

@@ -14,5 +14,5 @@ var __exportStar = (this && this.__exportStar) || function(m, exports) {
     for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
 };
 Object.defineProperty(exports, "__esModule", { value: true });
-__exportStar(require("./UseToolsEvent.cjs"), exports);
-__exportStar(require("./useTools.cjs"), exports);
+__exportStar(require("./RunToolsEvent.cjs"), exports);
+__exportStar(require("./runTools.cjs"), exports);

package/tool/run-tools/index.d.ts

@@ -0,0 +1,2 @@
+export * from "./RunToolsEvent.js";
+export * from "./runTools.js";

package/tool/run-tools/index.js

@@ -0,0 +1,2 @@
+export * from "./RunToolsEvent.js";
+export * from "./runTools.js";

package/tool/{use-tools/useTools.cjs → run-tools/runTools.cjs}

@@ -1,11 +1,11 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.useTools = void 0;
+exports.runTools = void 0;
 const executeFunctionCall_js_1 = require("../../core/executeFunctionCall.cjs");
 const ToolCallError_js_1 = require("../ToolCallError.cjs");
 const safeExecuteToolCall_js_1 = require("../execute-tool/safeExecuteToolCall.cjs");
 const generateToolCalls_js_1 = require("../generate-tool-calls/generateToolCalls.cjs");
-async function useTools({ model, tools, prompt, ...options }) {
+async function runTools({ model, tools, prompt, ...options }) {
     // Note: PROMPT must not be a function.
     const expandedPrompt = typeof prompt === "function"
         ? prompt(tools)
@@ -13,7 +13,7 @@ async function useTools({ model, tools, prompt, ...options }) {
     return (0, executeFunctionCall_js_1.executeFunctionCall)({
         options,
         input: expandedPrompt,
-        functionType: "use-tools",
+        functionType: "run-tools",
         execute: async (options) => {
             const modelResponse = await (0, generateToolCalls_js_1.generateToolCalls)({
                 model,
@@ -51,4 +51,4 @@ async function useTools({ model, tools, prompt, ...options }) {
         },
     });
 }
-exports.useTools = useTools;
+exports.runTools = runTools;

package/tool/{use-tools/useTools.d.ts → run-tools/runTools.d.ts}

@@ -11,7 +11,7 @@ type ToToolCallUnion<T> = {
     [KEY in StringKeys<T>]: T[KEY] extends Tool<any, infer PARAMETERS, infer OUTPUT> ? ToolCallResult<KEY, PARAMETERS, OUTPUT> : never;
 }[StringKeys<T>];
 type ToOutputValue<TOOLS extends ToolArray<Tool<any, any, any>[]>> = ToToolCallUnion<ToToolMap<TOOLS>>;
-export declare function useTools<PROMPT, TOOLS extends Array<Tool<any, any, any>>>({ model, tools, prompt, ...options }: {
+export declare function runTools<PROMPT, TOOLS extends Array<Tool<any, any, any>>>({ model, tools, prompt, ...options }: {
     model: ToolCallsGenerationModel<PROMPT, ToolCallsGenerationModelSettings>;
     tools: TOOLS;
     prompt: PROMPT | ((tools: TOOLS) => PROMPT);

package/tool/{use-tools/useTools.js → run-tools/runTools.js}

@@ -2,7 +2,7 @@ import { executeFunctionCall } from "../../core/executeFunctionCall.js";
 import { ToolCallError } from "../ToolCallError.js";
 import { safeExecuteToolCall } from "../execute-tool/safeExecuteToolCall.js";
 import { generateToolCalls } from "../generate-tool-calls/generateToolCalls.js";
-export async function useTools({ model, tools, prompt, ...options }) {
+export async function runTools({ model, tools, prompt, ...options }) {
     // Note: PROMPT must not be a function.
     const expandedPrompt = typeof prompt === "function"
         ? prompt(tools)
@@ -10,7 +10,7 @@ export async function useTools({ model, tools, prompt, ...options }) {
     return executeFunctionCall({
         options,
         input: expandedPrompt,
-        functionType: "use-tools",
+        functionType: "run-tools",
         execute: async (options) => {
             const modelResponse = await generateToolCalls({
                 model,

package/tool/use-tool/UseToolEvent.d.ts

@@ -1,7 +0,0 @@
-import { BaseFunctionFinishedEvent, BaseFunctionStartedEvent } from "../../core/FunctionEvent.js";
-export interface UseToolStartedEvent extends BaseFunctionStartedEvent {
-    functionType: "use-tool";
-}
-export interface UseToolFinishedEvent extends BaseFunctionFinishedEvent {
-    functionType: "use-tool";
-}

package/tool/use-tool/index.d.ts

@@ -1,2 +0,0 @@
-export * from "./UseToolEvent.js";
-export * from "./useTool.js";

package/tool/use-tool/index.js

@@ -1,2 +0,0 @@
-export * from "./UseToolEvent.js";
-export * from "./useTool.js";

package/tool/use-tools/UseToolsEvent.d.ts

@@ -1,7 +0,0 @@
-import { BaseFunctionFinishedEvent, BaseFunctionStartedEvent } from "../../core/FunctionEvent.js";
-export interface useToolsStartedEvent extends BaseFunctionStartedEvent {
-    functionType: "use-tools";
-}
-export interface useToolsFinishedEvent extends BaseFunctionFinishedEvent {
-    functionType: "use-tools";
-}

package/tool/use-tools/index.d.ts

@@ -1,2 +0,0 @@
-export * from "./UseToolsEvent.js";
-export * from "./useTools.js";

package/tool/use-tools/index.js

@@ -1,2 +0,0 @@
-export * from "./UseToolsEvent.js";
-export * from "./useTools.js";