modelfusion 0.49.0 → 0.50.0

package/README.md

@@ -15,13 +15,14 @@
 
 ## Introduction
 
-ModelFusion is a library for building AI apps, chatbots, and agents. It provides abstractions for AI models, vector indices, and tools.
+ModelFusion is a library for building AI applications, chatbots, and agents. Here are the main features:
 
-- **Multimodal Support**: Beyond just LLMs, ModelFusion encompasses a diverse array of models including text generation, text-to-speech, speech-to-text, and image generation, allowing you to build multi-modal AI applications with ease.
-- **Flexibility and control**: AI application development can be complex and unique to each project. With ModelFusion, you have complete control over the prompts and model settings, and you can access the raw responses from the models quickly to build what you need.
-- **Type inference and validation**: ModelFusion uses TypeScript to infer types wherever possible and to validate model responses. By default, [Zod](https://github.com/colinhacks/zod) is used for type validation, but you can also use other libraries.
-- **Guards**: ModelFusion provides a guard function that you can use to implement retry on error, redacting and changing reponses, etc.
-- **Integrated support features**: Essential features like **observability**, logging, retries, throttling, tracing, and error handling are built-in, helping you focus more on building your application.
+- **Multimodal**: ModelFusion supports a wide range of models including text generation, image generation, text-to-speech, speech-to-text, and embedding models.
+- **Streaming**: ModelFusion supports streaming for many generation models, e.g. text streaming, structure streaming, and full duplex speech streaming.
+- **Utility functions**: ModelFusion provides functionality for tools and tool usage, vector indices, and guards functions.
+- **Type inference and validation**: ModelFusion infers TypeScript types wherever possible and to validates model responses.
+- **Observability and logging**: ModelFusion provides an observer framework and out-of-the-box logging support.
+- **Resilience and Robustness**: ModelFusion ensures seamless operation through automatic retries, throttling, and error handling mechanisms.
 
 ## Quick Install
 
@@ -71,6 +72,84 @@ for await (const textFragment of textStream) {
 
 Providers: [OpenAI](https://modelfusion.dev/integration/model-provider/openai), [Anthropic](https://modelfusion.dev/integration/model-provider/anthropic), [Cohere](https://modelfusion.dev/integration/model-provider/cohere), [Llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp)
 
+### [Generate Image](https://modelfusion.dev/guide/function/generate-image)
+
+Generate an image from a prompt.
+
+```ts
+const image = await generateImage(
+  new OpenAIImageGenerationModel({ size: "512x512" }),
+  "the wicked witch of the west in the style of early 19th century painting"
+);
+```
+
+Providers: [OpenAI (Dall·E)](https://modelfusion.dev/integration/model-provider/openai), [Stability AI](https://modelfusion.dev/integration/model-provider/stability), [Automatic1111](https://modelfusion.dev/integration/model-provider/automatic1111)
+
+### [Generate Speech](https://modelfusion.dev/guide/function/generate-speech)
+
+Synthesize speech (audio) from text. Also called TTS (text-to-speech).
+
+#### generateSpeech
+
+`generateSpeech` synthesizes speech from text.
+
+```ts
+// `speech` is a Buffer with MP3 audio data
+const speech = await generateSpeech(
+  new LmntSpeechModel({
+    voice: "034b632b-df71-46c8-b440-86a42ffc3cf3", // Henry
+  }),
+  "Good evening, ladies and gentlemen! Exciting news on the airwaves tonight " +
+    "as The Rolling Stones unveil 'Hackney Diamonds,' their first collection of " +
+    "fresh tunes in nearly twenty years, featuring the illustrious Lady Gaga, the " +
+    "magical Stevie Wonder, and the final beats from the late Charlie Watts."
+);
+```
+
+Providers: [Eleven Labs](https://modelfusion.dev/integration/model-provider/elevenlabs), [LMNT](https://modelfusion.dev/integration/model-provider/lmnt)
+
+#### streamSpeech
+
+`generateSpeech` generates a stream of speech chunks from text or from a text stream. Depending on the model, this can be fully duplex.
+
+```ts
+const textStream = await streamText(/* ... */);
+
+const speechStream = await streamSpeech(
+  new ElevenLabsSpeechModel({
+    voice: "pNInz6obpgDQGcFmaJgB", // Adam
+    model: "eleven_monolingual_v1",
+    voiceSettings: { stability: 1, similarityBoost: 0.35 },
+    generationConfig: {
+      chunkLengthSchedule: [50, 90, 120, 150, 200],
+    },
+  }),
+  textStream
+);
+
+for await (const part of speechStream) {
+  // each part is a Buffer with MP3 audio data
+}
+```
+
+Providers: [Eleven Labs](https://modelfusion.dev/integration/model-provider/elevenlabs)
+
+### [Generate Transcription](https://modelfusion.dev/guide/function/generate-transcription)
+
+Transcribe speech (audio) data into text. Also called speech-to-text (STT).
+
+```ts
+const transcription = await generateTranscription(
+  new OpenAITranscriptionModel({ model: "whisper-1" }),
+  {
+    type: "mp3",
+    data: await fs.promises.readFile("data/test.mp3"),
+  }
+);
+```
+
+Providers: [OpenAI (Whisper)](https://modelfusion.dev/integration/model-provider/openai)
+
 ### [Generate Structure](https://modelfusion.dev/guide/function/generate-structure#generatestructure)
 
 Generate typed objects using a language model and a schema.
@@ -189,149 +268,6 @@ const { structure, value, text } = await generateStructureOrText(
 
 Providers: [OpenAI](https://modelfusion.dev/integration/model-provider/openai)
 
-### [Tools](https://modelfusion.dev/guide/tools)
-
-Tools are functions that can be executed by an AI model. They are useful for building chatbots and agents.
-
-Predefined tools: [SerpAPI](https://modelfusion.dev/integration/tool/serpapi), [Google Custom Search](https://modelfusion.dev/integration/tool/google-custom-search)
-
-#### Create Tool
-
-A tool is a function with a name, a description, and a schema for the input parameters.
-
-```ts
-const calculator = new Tool({
-  name: "calculator",
-  description: "Execute a calculation",
-
-  inputSchema: new ZodSchema(
-    z.object({
-      a: z.number().describe("The first number."),
-      b: z.number().describe("The second number."),
-      operator: z
-        .enum(["+", "-", "*", "/"])
-        .describe("The operator (+, -, *, /)."),
-    })
-  ),
-
-  execute: async ({ a, b, operator }) => {
-    switch (operator) {
-      case "+":
-        return a + b;
-      case "-":
-        return a - b;
-      case "*":
-        return a * b;
-      case "/":
-        return a / b;
-      default:
-        throw new Error(`Unknown operator: ${operator}`);
-    }
-  },
-});
-```
-
-#### useTool
-
-The model determines the parameters for the tool from the prompt and then executes it.
-
-```ts
-const { tool, parameters, result } = await useTool(
-  new OpenAIChatModel({ model: "gpt-3.5-turbo" }),
-  calculator,
-  [OpenAIChatMessage.user("What's fourteen times twelve?")]
-);
-```
-
-#### useToolOrGenerateText
-
-The model determines which tool to use and its parameters from the prompt and then executes it.
-Text is generated as a fallback.
-
-```ts
-const { tool, parameters, result, text } = await useToolOrGenerateText(
-  new OpenAIChatModel({ model: "gpt-3.5-turbo" }),
-  [calculator /* and other tools... */],
-  [OpenAIChatMessage.user("What's fourteen times twelve?")]
-);
-```
-
-### [Generate Transcription](https://modelfusion.dev/guide/function/generate-transcription)
-
-Transcribe speech (audio) data into text. Also called speech-to-text (STT).
-
-```ts
-const transcription = await generateTranscription(
-  new OpenAITranscriptionModel({ model: "whisper-1" }),
-  {
-    type: "mp3",
-    data: await fs.promises.readFile("data/test.mp3"),
-  }
-);
-```
-
-Providers: [OpenAI (Whisper)](https://modelfusion.dev/integration/model-provider/openai)
-
-### [Generate Speech](https://modelfusion.dev/guide/function/generate-speech)
-
-Synthesize speech (audio) from text. Also called TTS (text-to-speech).
-
-Providers: [Eleven Labs](https://modelfusion.dev/integration/model-provider/elevenlabs), [LMNT](https://modelfusion.dev/integration/model-provider/lmnt)
-
-#### generateSpeech
-
-`generateSpeech` synthesizes speech from text.
-
-```ts
-// `speech` is a Buffer with MP3 audio data
-const speech = await generateSpeech(
-  new LmntSpeechModel({
-    voice: "034b632b-df71-46c8-b440-86a42ffc3cf3", // Henry
-  }),
-  "Good evening, ladies and gentlemen! Exciting news on the airwaves tonight " +
-    "as The Rolling Stones unveil 'Hackney Diamonds,' their first collection of " +
-    "fresh tunes in nearly twenty years, featuring the illustrious Lady Gaga, the " +
-    "magical Stevie Wonder, and the final beats from the late Charlie Watts."
-);
-```
-
-#### streamSpeech
-
-`generateSpeech` generates a stream of speech chunks from text or from a text stream. Depending on the model, this can be fully duplex.
-
-```ts
-const textStream = await streamText(/* ... */);
-
-const speechStream = await streamSpeech(
-  new ElevenLabsSpeechModel({
-    voice: "pNInz6obpgDQGcFmaJgB", // Adam
-    model: "eleven_monolingual_v1",
-    voiceSettings: { stability: 1, similarityBoost: 0.35 },
-    generationConfig: {
-      chunkLengthSchedule: [50, 90, 120, 150, 200],
-    },
-  }),
-  textStream
-);
-
-for await (const part of speechStream) {
-  // each part is a Buffer with MP3 audio data
-}
-```
-
-### [Generate Image](https://modelfusion.dev/guide/function/generate-image)
-
-Generate an image from a prompt.
-
-```ts
-const image = await generateImage(
-  new OpenAIImageGenerationModel({ size: "512x512" }),
-  "the wicked witch of the west in the style of early 19th century painting"
-);
-```
-
-Providers: [OpenAI (Dall·E)](https://modelfusion.dev/integration/model-provider/openai), [Stability AI](https://modelfusion.dev/integration/model-provider/stability), [Automatic1111](https://modelfusion.dev/integration/model-provider/automatic1111)
-
 ### [Embed Value](https://modelfusion.dev/guide/function/embed)
 
 Create embeddings for text and other values. Embeddings are vectors that represent the essence of the values in the context of the model.
@@ -381,7 +317,7 @@ Guard functions can be used to implement retry on error, redacting and changing
 
 ```ts
 const result = await guard(
-  (input) =>
+  (input, options) =>
     generateStructure(
       new OpenAIChatModel({
         // ...
@@ -389,7 +325,8 @@ const result = await guard(
       new ZodStructureDefinition({
         // ...
       }),
-      input
+      input,
+      options
     ),
   [
     // ...
@@ -408,7 +345,74 @@ const result = await guard(
 );
 ```
 
-### [Upserting and Retrieving Objects from Vector Indices](https://modelfusion.dev/guide/vector-index)
+### [Tools](https://modelfusion.dev/guide/tools)
+
+Tools are functions that can be executed by an AI model. They are useful for building chatbots and agents.
+
+Predefined tools: [SerpAPI](https://modelfusion.dev/integration/tool/serpapi), [Google Custom Search](https://modelfusion.dev/integration/tool/google-custom-search)
+
+#### Create Tool
+
+A tool is a function with a name, a description, and a schema for the input parameters.
+
+```ts
+const calculator = new Tool({
+  name: "calculator",
+  description: "Execute a calculation",
+
+  inputSchema: new ZodSchema(
+    z.object({
+      a: z.number().describe("The first number."),
+      b: z.number().describe("The second number."),
+      operator: z
+        .enum(["+", "-", "*", "/"])
+        .describe("The operator (+, -, *, /)."),
+    })
+  ),
+
+  execute: async ({ a, b, operator }) => {
+    switch (operator) {
+      case "+":
+        return a + b;
+      case "-":
+        return a - b;
+      case "*":
+        return a * b;
+      case "/":
+        return a / b;
+      default:
+        throw new Error(`Unknown operator: ${operator}`);
+    }
+  },
+});
+```
+
+#### useTool
+
+The model determines the parameters for the tool from the prompt and then executes it.
+
+```ts
+const { tool, parameters, result } = await useTool(
+  new OpenAIChatModel({ model: "gpt-3.5-turbo" }),
+  calculator,
+  [OpenAIChatMessage.user("What's fourteen times twelve?")]
+);
+```
+
+#### useToolOrGenerateText
+
+The model determines which tool to use and its parameters from the prompt and then executes it.
+Text is generated as a fallback.
+
+```ts
+const { tool, parameters, result, text } = await useToolOrGenerateText(
+  new OpenAIChatModel({ model: "gpt-3.5-turbo" }),
+  [calculator /* and other tools... */],
+  [OpenAIChatMessage.user("What's fourteen times twelve?")]
+);
+```
+
+### [Vector Indices](https://modelfusion.dev/guide/vector-index)
 
 ```ts
 const texts = [
@@ -529,9 +533,15 @@ for (const choice of (response as OpenAICompletionResponse).choices) {
 }
 ```
 
-### Observability
+### Logging and Observability
+
+ModelFusion provides an [observer framework](https://modelfusion.dev/guide/util/observer) and [out-of-the-box logging support](https://modelfusion.dev/guide/util/logging). You can easily trace runs and call hierarchies, and you can add your own observers.
 
-Integrations: [Helicone](https://modelfusion.dev/integration/observability/helicone)
+#### Global Logging Example
+
+```ts
+setGlobalFunctionLogging("detailed-object"); // log full events
+```
 
 ## Documentation
 
@@ -654,3 +664,11 @@ Generate text on a Cloudflare Worker using ModelFusion and OpenAI.
 ### [Contributing Guide](https://github.com/lgrammel/modelfusion/blob/main/CONTRIBUTING.md)
 
 Read the [ModelFusion contributing guide](https://github.com/lgrammel/modelfusion/blob/main/CONTRIBUTING.md) to learn about the development process, how to propose bugfixes and improvements, and how to build and test your changes.
+
+```
+
+```
+
+```
+
+```

package/core/FunctionEvent.d.ts

@@ -1,11 +1,19 @@
+import { GuardFinishedEvent, GuardStartedEvent } from "../guard/GuardEvent.js";
 import { ModelCallFinishedEvent, ModelCallStartedEvent } from "../model-function/ModelCallEvent.js";
 import { RetrieveFinishedEvent, RetrieveStartedEvent } from "../retriever/RetrieveEvent.js";
 import { ExecuteToolFinishedEvent, ExecuteToolStartedEvent } from "../tool/ExecuteToolEvent.js";
+import { UseToolFinishedEvent, UseToolStartedEvent } from "../tool/UseToolEvent.js";
+import { UseToolOrGenerateTextFinishedEvent, UseToolOrGenerateTextStartedEvent } from "../tool/UseToolOrGenerateTextEvent.js";
+import { UpsertIntoVectorIndexFinishedEvent, UpsertIntoVectorIndexStartedEvent } from "../vector-index/UpsertIntoVectorIndexEvent.js";
 export interface BaseFunctionEvent {
     /**
      * Unique identifier for the function call.
      */
     callId: string | undefined;
+    /**
+     * Unique identifier of the call id of the parent function.
+     */
+    parentCallId?: string | undefined;
     /**
      * Optional unique identifier for the function.
      */
@@ -73,4 +81,4 @@ export interface BaseFunctionFinishedEvent extends BaseFunctionEvent {
      */
     result: BaseFunctionFinishedEventResult;
 }
-export type FunctionEvent = ModelCallStartedEvent | ExecuteToolStartedEvent | RetrieveStartedEvent | ModelCallFinishedEvent | ExecuteToolFinishedEvent | RetrieveFinishedEvent;
+export type FunctionEvent = ExecuteToolStartedEvent | ExecuteToolFinishedEvent | GuardStartedEvent | GuardFinishedEvent | ModelCallStartedEvent | ModelCallFinishedEvent | RetrieveStartedEvent | RetrieveFinishedEvent | UpsertIntoVectorIndexStartedEvent | UpsertIntoVectorIndexFinishedEvent | UseToolStartedEvent | UseToolFinishedEvent | UseToolOrGenerateTextStartedEvent | UseToolOrGenerateTextFinishedEvent;

package/core/FunctionOptions.d.ts

@@ -21,6 +21,10 @@ export type FunctionOptions = {
      * Optional run as part of which this function is called.
      */
     run?: Run;
+    /**
+     * Unique identifier of the call id of the parent function.
+     */
+    parentCallId?: string | undefined;
 };
 /**
  * The logging to use for the function. Logs are sent to the console.

package/core/executeFunctionCall.cjs

@@ -0,0 +1,85 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.executeFunctionCall = void 0;
+const nanoid_1 = require("nanoid");
+const FunctionEventSource_js_1 = require("./FunctionEventSource.cjs");
+const GlobalFunctionLogging_js_1 = require("./GlobalFunctionLogging.cjs");
+const GlobalFunctionObservers_js_1 = require("./GlobalFunctionObservers.cjs");
+const AbortError_js_1 = require("./api/AbortError.cjs");
+const getFunctionCallLogger_js_1 = require("./getFunctionCallLogger.cjs");
+const getRun_js_1 = require("./getRun.cjs");
+const DurationMeasurement_js_1 = require("../util/DurationMeasurement.cjs");
+const runSafe_js_1 = require("../util/runSafe.cjs");
+async function executeFunctionCall({ options, input, functionType, execute, inputPropertyName = "input", outputPropertyName = "value", }) {
+    const run = await (0, getRun_js_1.getRun)(options?.run);
+    const eventSource = new FunctionEventSource_js_1.FunctionEventSource({
+        observers: [
+            ...(0, getFunctionCallLogger_js_1.getFunctionCallLogger)(options?.logging ?? (0, GlobalFunctionLogging_js_1.getGlobalFunctionLogging)()),
+            ...(0, GlobalFunctionObservers_js_1.getGlobalFunctionObservers)(),
+            ...(run?.functionObserver != null ? [run.functionObserver] : []),
+            ...(options?.observers ?? []),
+        ],
+        errorHandler: run?.errorHandler,
+    });
+    const durationMeasurement = (0, DurationMeasurement_js_1.startDurationMeasurement)();
+    const startMetadata = {
+        functionType,
+        callId: `call-${(0, nanoid_1.nanoid)()}`,
+        parentCallId: options?.parentCallId,
+        runId: run?.runId,
+        sessionId: run?.sessionId,
+        userId: run?.userId,
+        functionId: options?.functionId,
+        [inputPropertyName]: input,
+        timestamp: durationMeasurement.startDate,
+        startTimestamp: durationMeasurement.startDate,
+    };
+    eventSource.notify({
+        eventType: "started",
+        ...startMetadata,
+    });
+    const result = await (0, runSafe_js_1.runSafe)(() => execute({
+        functionId: options?.functionId,
+        logging: options?.logging,
+        observers: options?.observers,
+        run,
+        parentCallId: startMetadata.callId,
+    }));
+    const finishMetadata = {
+        eventType: "finished",
+        ...startMetadata,
+        finishTimestamp: new Date(),
+        durationInMs: durationMeasurement.durationInMs,
+    };
+    if (!result.ok) {
+        if (result.isAborted) {
+            eventSource.notify({
+                ...finishMetadata,
+                eventType: "finished",
+                result: {
+                    status: "abort",
+                },
+            });
+            throw new AbortError_js_1.AbortError();
+        }
+        eventSource.notify({
+            ...finishMetadata,
+            eventType: "finished",
+            result: {
+                status: "error",
+                error: result.error,
+            },
+        });
+        throw result.error;
+    }
+    eventSource.notify({
+        ...finishMetadata,
+        eventType: "finished",
+        result: {
+            status: "success",
+            [outputPropertyName]: result.value,
+        },
+    });
+    return result.value;
+}
+exports.executeFunctionCall = executeFunctionCall;

package/core/executeFunctionCall.d.ts

@@ -0,0 +1,10 @@
+import { FunctionOptions } from "./FunctionOptions.js";
+import { FunctionEvent } from "./FunctionEvent.js";
+export declare function executeFunctionCall<VALUE>({ options, input, functionType, execute, inputPropertyName, outputPropertyName, }: {
+    options?: FunctionOptions;
+    input: unknown;
+    functionType: FunctionEvent["functionType"];
+    execute: (options?: FunctionOptions) => PromiseLike<VALUE>;
+    inputPropertyName?: string;
+    outputPropertyName?: string;
+}): Promise<VALUE>;

package/core/executeFunctionCall.js

@@ -0,0 +1,81 @@
+import { nanoid as createId } from "nanoid";
+import { FunctionEventSource } from "./FunctionEventSource.js";
+import { getGlobalFunctionLogging } from "./GlobalFunctionLogging.js";
+import { getGlobalFunctionObservers } from "./GlobalFunctionObservers.js";
+import { AbortError } from "./api/AbortError.js";
+import { getFunctionCallLogger } from "./getFunctionCallLogger.js";
+import { getRun } from "./getRun.js";
+import { startDurationMeasurement } from "../util/DurationMeasurement.js";
+import { runSafe } from "../util/runSafe.js";
+export async function executeFunctionCall({ options, input, functionType, execute, inputPropertyName = "input", outputPropertyName = "value", }) {
+    const run = await getRun(options?.run);
+    const eventSource = new FunctionEventSource({
+        observers: [
+            ...getFunctionCallLogger(options?.logging ?? getGlobalFunctionLogging()),
+            ...getGlobalFunctionObservers(),
+            ...(run?.functionObserver != null ? [run.functionObserver] : []),
+            ...(options?.observers ?? []),
+        ],
+        errorHandler: run?.errorHandler,
+    });
+    const durationMeasurement = startDurationMeasurement();
+    const startMetadata = {
+        functionType,
+        callId: `call-${createId()}`,
+        parentCallId: options?.parentCallId,
+        runId: run?.runId,
+        sessionId: run?.sessionId,
+        userId: run?.userId,
+        functionId: options?.functionId,
+        [inputPropertyName]: input,
+        timestamp: durationMeasurement.startDate,
+        startTimestamp: durationMeasurement.startDate,
+    };
+    eventSource.notify({
+        eventType: "started",
+        ...startMetadata,
+    });
+    const result = await runSafe(() => execute({
+        functionId: options?.functionId,
+        logging: options?.logging,
+        observers: options?.observers,
+        run,
+        parentCallId: startMetadata.callId,
+    }));
+    const finishMetadata = {
+        eventType: "finished",
+        ...startMetadata,
+        finishTimestamp: new Date(),
+        durationInMs: durationMeasurement.durationInMs,
+    };
+    if (!result.ok) {
+        if (result.isAborted) {
+            eventSource.notify({
+                ...finishMetadata,
+                eventType: "finished",
+                result: {
+                    status: "abort",
+                },
+            });
+            throw new AbortError();
+        }
+        eventSource.notify({
+            ...finishMetadata,
+            eventType: "finished",
+            result: {
+                status: "error",
+                error: result.error,
+            },
+        });
+        throw result.error;
+    }
+    eventSource.notify({
+        ...finishMetadata,
+        eventType: "finished",
+        result: {
+            status: "success",
+            [outputPropertyName]: result.value,
+        },
+    });
+    return result.value;
+}

package/guard/GuardEvent.cjs

@@ -0,0 +1,2 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });

package/guard/GuardEvent.d.ts

@@ -0,0 +1,7 @@
+import { BaseFunctionFinishedEvent, BaseFunctionStartedEvent } from "../core/FunctionEvent.js";
+export interface GuardStartedEvent extends BaseFunctionStartedEvent {
+    functionType: "guard";
+}
+export interface GuardFinishedEvent extends BaseFunctionFinishedEvent {
+    functionType: "guard";
+}

package/guard/GuardEvent.js

@@ -0,0 +1 @@
+export {};