modelfusion 0.41.3 → 0.43.0

package/README.md

@@ -19,8 +19,9 @@ ModelFusion is a library for building AI apps, chatbots, and agents. It provides
 
 - **Multimodal Support**: Beyond just LLMs, ModelFusion encompasses a diverse array of models including text generation, text-to-speech, speech-to-text, and image generation, allowing you to build multifaceted AI applications with ease.
 - **Flexibility and control**: AI application development can be complex and unique to each project. With ModelFusion, you have complete control over the prompts and model settings, and you can access the raw responses from the models quickly to build what you need.
-- **Type inference and validation**: ModelFusion uses TypeScript and [Zod](https://github.com/colinhacks/zod) to infer types wherever possible and to validate model responses.
-- **Integrated support features**: Essential features like logging, retries, throttling, tracing, and error handling are built-in, helping you focus more on building your application.
+- **Type inference and validation**: ModelFusion uses TypeScript to infer types wherever possible and to validate model responses. By default, [Zod](https://github.com/colinhacks/zod) is used for type validation, but you can also use other libraries.
+- **Guards**: ModelFusion provides a guard function that you can use to implement retry on error, redacting and changing reponses, etc.
+- **Integrated support features**: Essential features like **observability**, logging, retries, throttling, tracing, and error handling are built-in, helping you focus more on building your application.
 
 ## Quick Install
 
@@ -51,7 +52,7 @@ const text = await generateText(
 );
 ```
 
-Providers: [OpenAI](https://modelfusion.dev/integration/model-provider/openai), [Cohere](https://modelfusion.dev/integration/model-provider/cohere), [Llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp), [Hugging Face](https://modelfusion.dev/integration/model-provider/huggingface)
+Providers: [OpenAI](https://modelfusion.dev/integration/model-provider/openai), [Anthropic](https://modelfusion.dev/integration/model-provider/anthropic), [Cohere](https://modelfusion.dev/integration/model-provider/cohere), [Llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp), [Hugging Face](https://modelfusion.dev/integration/model-provider/huggingface)
 
 #### streamText
 
@@ -68,7 +69,7 @@ for await (const textFragment of textStream) {
 }
 ```
 
-Providers: [OpenAI](https://modelfusion.dev/integration/model-provider/openai), [Cohere](https://modelfusion.dev/integration/model-provider/cohere), [Llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp)
+Providers: [OpenAI](https://modelfusion.dev/integration/model-provider/openai), [Anthropic](https://modelfusion.dev/integration/model-provider/anthropic), [Cohere](https://modelfusion.dev/integration/model-provider/cohere), [Llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp)
 
 #### Prompt Format
 
@@ -104,6 +105,7 @@ const textStream = await streamText(
 | Prompt Format | Instruction Prompt | Chat Prompt |
 | ------------- | ------------------ | ----------- |
 | OpenAI Chat   | ✅                 | ✅          |
+| Anthropic     | ✅                 | ✅          |
 | Llama 2       | ✅                 | ✅          |
 | Alpaca        | ✅                 | ❌          |
 | Vicuna        | ❌                 | ✅          |
@@ -453,6 +455,43 @@ const retrievedTexts = await retrieve(
 
 Available Vector Stores: [Memory](https://modelfusion.dev/integration/vector-index/memory), [Pinecone](https://modelfusion.dev/integration/vector-index/pinecone)
 
+### Guards
+
+[Guards](https://github.com/lgrammel/modelfusion/tree/main/examples/basic/src/guard) can be used to implement retry on error, redacting and changing reponses, etc.
+
+#### Retry structure parsing on error:
+
+```ts
+const result = await guard(
+  (input) =>
+    generateStructure(
+      new OpenAIChatModel({
+        // ...
+      }),
+      new ZodStructureDefinition({
+        // ...
+      }),
+      input
+    ),
+  [
+    // ...
+  ],
+  [
+    fixStructure({
+      modifyInputForRetry: async ({ input, error }) => [
+        ...input,
+        OpenAIChatMessage.functionCall(null, {
+          name: error.structureName,
+          arguments: error.valueText,
+        }),
+        OpenAIChatMessage.user(error.message),
+        OpenAIChatMessage.user("Please fix the error and try again."),
+      ],
+    }),
+  ]
+);
+```
+
 ### Observability
 
 Integrations: [Helicone](https://modelfusion.dev/integration/observability/helicone)
@@ -494,10 +533,18 @@ Integrations: [Helicone](https://modelfusion.dev/integration/observability/helic
 
 Examples for almost all of the individual functions and objects. Highly recommended to get started.
 
+### [StoryTeller](https://github.com/lgrammel/storyteller)
+
+> _multi-modal_, _structure streaming_, _image generation_, _text to speech_, _speech to text_, _text generation_, _structure generation_, _embeddings_
+
+StoryTeller is an exploratory web application that creates short audio stories for pre-school kids.
+
 ### [Chatbot (Terminal)](https://github.com/lgrammel/modelfusion/tree/main/examples/chatbot-terminal)
 
 > _Terminal app_, _chat_, _llama.cpp_
 
+A chat with an AI assistant, implemented as a terminal app.
+
 ### [Chatbot (Next.JS)](https://github.com/lgrammel/modelfusion/tree/main/examples/chatbot-next-js)
 
 > _Next.js app_, _OpenAI GPT-3.5-turbo_, _streaming_, _abort handling_
@@ -551,3 +598,7 @@ Extracts information about a topic from a PDF and writes a tweet in your own sty
 ### [Contributing Guide](https://github.com/lgrammel/modelfusion/blob/main/CONTRIBUTING.md)
 
 Read the [ModelFusion contributing guide](https://github.com/lgrammel/modelfusion/blob/main/CONTRIBUTING.md) to learn about the development process, how to propose bugfixes and improvements, and how to build and test your changes.
+
+```
+
+```

package/guard/fixStructure.cjs

@@ -0,0 +1,21 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.fixStructure = void 0;
+const StructureParseError_js_1 = require("../model-function/generate-structure/StructureParseError.cjs");
+const StructureValidationError_js_1 = require("../model-function/generate-structure/StructureValidationError.cjs");
+const fixStructure = ({ modifyInputForRetry }) => async (result) => {
+    if (result.type === "error" &&
+        (result.error instanceof StructureValidationError_js_1.StructureValidationError ||
+            result.error instanceof StructureParseError_js_1.StructureParseError)) {
+        return {
+            action: "retry",
+            input: await modifyInputForRetry({
+                type: "error",
+                input: result.input,
+                error: result.error,
+            }),
+        };
+    }
+    return undefined;
+};
+exports.fixStructure = fixStructure;

package/guard/fixStructure.d.ts

@@ -0,0 +1,10 @@
+import { StructureParseError } from "../model-function/generate-structure/StructureParseError.js";
+import { StructureValidationError } from "../model-function/generate-structure/StructureValidationError.js";
+import { Guard } from "./guard.js";
+export declare const fixStructure: <INPUT, OUTPUT>(options: {
+    modifyInputForRetry: (options: {
+        type: "error";
+        input: INPUT;
+        error: StructureValidationError | StructureParseError;
+    }) => PromiseLike<INPUT>;
+}) => Guard<INPUT, OUTPUT>;

package/guard/fixStructure.js

@@ -0,0 +1,17 @@
+import { StructureParseError } from "../model-function/generate-structure/StructureParseError.js";
+import { StructureValidationError } from "../model-function/generate-structure/StructureValidationError.js";
+export const fixStructure = ({ modifyInputForRetry }) => async (result) => {
+    if (result.type === "error" &&
+        (result.error instanceof StructureValidationError ||
+            result.error instanceof StructureParseError)) {
+        return {
+            action: "retry",
+            input: await modifyInputForRetry({
+                type: "error",
+                input: result.input,
+                error: result.error,
+            }),
+        };
+    }
+    return undefined;
+};

package/guard/guard.cjs

@@ -0,0 +1,69 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.guard = void 0;
+async function guard(execute, input, guards, options) {
+    const maxRetries = options?.maxRetries ?? 1;
+    let attempts = 0;
+    while (attempts <= maxRetries) {
+        let result;
+        try {
+            result = {
+                type: "value",
+                input,
+                output: await execute(input),
+            };
+        }
+        catch (error) {
+            result = {
+                type: "error",
+                input,
+                error,
+            };
+        }
+        let isValid = true;
+        for (const guard of guards) {
+            const guardResult = await guard(result);
+            if (guardResult === undefined) {
+                continue;
+            }
+            switch (guardResult.action) {
+                case "passThrough": {
+                    break;
+                }
+                case "retry": {
+                    input = guardResult.input;
+                    isValid = false;
+                    break;
+                }
+                case "return": {
+                    result = {
+                        type: "value",
+                        input,
+                        output: guardResult.output,
+                    };
+                    break;
+                }
+                case "throwError": {
+                    result = {
+                        type: "error",
+                        input,
+                        error: guardResult.error,
+                    };
+                    break;
+                }
+            }
+        }
+        if (isValid) {
+            if (result.type === "value") {
+                return result.output;
+            }
+            else {
+                throw result.error;
+            }
+        }
+        attempts++;
+    }
+    throw new Error(`Maximum retry attempts of ${maxRetries} reached ` +
+        `without producing a valid output or handling an error after ${attempts} attempts.`);
+}
+exports.guard = guard;

package/guard/guard.d.ts

@@ -0,0 +1,28 @@
+type OutputResult<INPUT, OUTPUT> = {
+    type: "value";
+    input: INPUT;
+    output: OUTPUT;
+    error?: undefined;
+} | {
+    type: "error";
+    input: INPUT;
+    output?: undefined;
+    error: unknown;
+};
+export type OutputValidator<INPUT, OUTPUT> = ({ type, input, output, error, }: OutputResult<INPUT, OUTPUT>) => PromiseLike<boolean>;
+export type Guard<INPUT, OUTPUT> = ({ type, input, output, error, }: OutputResult<INPUT, OUTPUT>) => PromiseLike<{
+    action: "retry";
+    input: INPUT;
+} | {
+    action: "return";
+    output: OUTPUT;
+} | {
+    action: "throwError";
+    error: unknown;
+} | {
+    action: "passThrough";
+} | undefined>;
+export declare function guard<INPUT, OUTPUT>(execute: (input: INPUT) => PromiseLike<OUTPUT>, input: INPUT, guards: Array<Guard<INPUT, OUTPUT>>, options?: {
+    maxRetries: number;
+}): Promise<OUTPUT | undefined>;
+export {};

package/guard/guard.js

@@ -0,0 +1,65 @@
+export async function guard(execute, input, guards, options) {
+    const maxRetries = options?.maxRetries ?? 1;
+    let attempts = 0;
+    while (attempts <= maxRetries) {
+        let result;
+        try {
+            result = {
+                type: "value",
+                input,
+                output: await execute(input),
+            };
+        }
+        catch (error) {
+            result = {
+                type: "error",
+                input,
+                error,
+            };
+        }
+        let isValid = true;
+        for (const guard of guards) {
+            const guardResult = await guard(result);
+            if (guardResult === undefined) {
+                continue;
+            }
+            switch (guardResult.action) {
+                case "passThrough": {
+                    break;
+                }
+                case "retry": {
+                    input = guardResult.input;
+                    isValid = false;
+                    break;
+                }
+                case "return": {
+                    result = {
+                        type: "value",
+                        input,
+                        output: guardResult.output,
+                    };
+                    break;
+                }
+                case "throwError": {
+                    result = {
+                        type: "error",
+                        input,
+                        error: guardResult.error,
+                    };
+                    break;
+                }
+            }
+        }
+        if (isValid) {
+            if (result.type === "value") {
+                return result.output;
+            }
+            else {
+                throw result.error;
+            }
+        }
+        attempts++;
+    }
+    throw new Error(`Maximum retry attempts of ${maxRetries} reached ` +
+        `without producing a valid output or handling an error after ${attempts} attempts.`);
+}

package/guard/index.cjs

@@ -0,0 +1,18 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __exportStar = (this && this.__exportStar) || function(m, exports) {
+    for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+__exportStar(require("./fixStructure.cjs"), exports);
+__exportStar(require("./guard.cjs"), exports);

package/guard/index.d.ts

@@ -0,0 +1,2 @@
+export * from "./fixStructure.js";
+export * from "./guard.js";

package/guard/index.js

@@ -0,0 +1,2 @@
+export * from "./fixStructure.js";
+export * from "./guard.js";

package/index.cjs

@@ -18,6 +18,7 @@ __exportStar(require("./composed-function/index.cjs"), exports);
 __exportStar(require("./core/index.cjs"), exports);
 __exportStar(require("./cost/index.cjs"), exports);
 __exportStar(require("./event-source/index.cjs"), exports);
+__exportStar(require("./guard/index.cjs"), exports);
 __exportStar(require("./model-function/index.cjs"), exports);
 __exportStar(require("./model-provider/index.cjs"), exports);
 __exportStar(require("./observability/index.cjs"), exports);

package/index.d.ts

@@ -2,6 +2,7 @@ export * from "./composed-function/index.js";
 export * from "./core/index.js";
 export * from "./cost/index.js";
 export * from "./event-source/index.js";
+export * from "./guard/index.js";
 export * from "./model-function/index.js";
 export * from "./model-provider/index.js";
 export * from "./observability/index.js";

package/index.js

@@ -2,6 +2,7 @@ export * from "./composed-function/index.js";
 export * from "./core/index.js";
 export * from "./cost/index.js";
 export * from "./event-source/index.js";
+export * from "./guard/index.js";
 export * from "./model-function/index.js";
 export * from "./model-provider/index.js";
 export * from "./observability/index.js";

package/model-function/generate-structure/StructureFromTextGenerationModel.cjs

@@ -2,6 +2,7 @@
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.StructureFromTextGenerationModel = void 0;
 const generateText_js_1 = require("../generate-text/generateText.cjs");
+const StructureParseError_js_1 = require("./StructureParseError.cjs");
 class StructureFromTextGenerationModel {
     constructor({ model, format, }) {
         Object.defineProperty(this, "model", {
@@ -30,10 +31,20 @@ class StructureFromTextGenerationModel {
     }
     async doGenerateStructure(structure, prompt, options) {
         const { response, value } = await (0, generateText_js_1.generateText)(this.model, this.format.createPrompt(prompt, structure), options).asFullResponse();
-        return {
-            response,
-            structure: this.format.extractStructure(value),
-        };
+        try {
+            return {
+                response,
+                value: this.format.extractStructure(value),
+                valueText: value,
+            };
+        }
+        catch (error) {
+            throw new StructureParseError_js_1.StructureParseError({
+                structureName: structure.name,
+                valueText: value,
+                cause: error,
+            });
+        }
     }
     withSettings(additionalSettings) {
         return new StructureFromTextGenerationModel({

package/model-function/generate-structure/StructureFromTextGenerationModel.d.ts

@@ -18,7 +18,8 @@ export declare class StructureFromTextGenerationModel<PROMPT, MODEL extends Text
     get settingsForEvent(): Partial<MODEL["settings"]>;
     doGenerateStructure(structure: StructureDefinition<string, unknown>, prompt: PROMPT, options?: FunctionOptions): Promise<{
         response: unknown;
-        structure: unknown;
+        value: unknown;
+        valueText: string;
     }>;
     withSettings(additionalSettings: Partial<MODEL["settings"]>): this;
 }

package/model-function/generate-structure/StructureFromTextGenerationModel.js

@@ -1,4 +1,5 @@
 import { generateText } from "../generate-text/generateText.js";
+import { StructureParseError } from "./StructureParseError.js";
 export class StructureFromTextGenerationModel {
     constructor({ model, format, }) {
         Object.defineProperty(this, "model", {
@@ -27,10 +28,20 @@ export class StructureFromTextGenerationModel {
     }
     async doGenerateStructure(structure, prompt, options) {
         const { response, value } = await generateText(this.model, this.format.createPrompt(prompt, structure), options).asFullResponse();
-        return {
-            response,
-            structure: this.format.extractStructure(value),
-        };
+        try {
+            return {
+                response,
+                value: this.format.extractStructure(value),
+                valueText: value,
+            };
+        }
+        catch (error) {
+            throw new StructureParseError({
+                structureName: structure.name,
+                valueText: value,
+                cause: error,
+            });
+        }
     }
     withSettings(additionalSettings) {
         return new StructureFromTextGenerationModel({

package/model-function/generate-structure/StructureGenerationModel.d.ts

@@ -7,7 +7,8 @@ export interface StructureGenerationModelSettings extends ModelSettings {
 export interface StructureGenerationModel<PROMPT, SETTINGS extends StructureGenerationModelSettings = StructureGenerationModelSettings> extends Model<SETTINGS> {
     doGenerateStructure(structure: StructureDefinition<string, unknown>, prompt: PROMPT, options?: FunctionOptions): PromiseLike<{
         response: unknown;
-        structure: unknown;
+        valueText: string;
+        value: unknown;
         usage?: {
             promptTokens: number;
             completionTokens: number;

package/model-function/generate-structure/StructureOrTextGenerationModel.d.ts

@@ -9,10 +9,12 @@ export interface StructureOrTextGenerationModel<PROMPT, SETTINGS extends Structu
         structureAndText: {
             structure: null;
             value: null;
+            valueText: null;
             text: string;
         } | {
             structure: string;
             value: unknown;
+            valueText: string;
             text: string | null;
         };
         usage?: {

package/model-function/generate-structure/StructureParseError.cjs

@@ -0,0 +1,34 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.StructureParseError = void 0;
+const getErrorMessage_js_1 = require("../../util/getErrorMessage.cjs");
+class StructureParseError extends Error {
+    constructor({ structureName, valueText, cause, }) {
+        super(`Structure parsing failed for '${structureName}'. ` +
+            `Value: ${valueText}.\n` +
+            `Error message: ${(0, getErrorMessage_js_1.getErrorMessage)(cause)}`);
+        Object.defineProperty(this, "structureName", {
+            enumerable: true,
+            configurable: true,
+            writable: true,
+            value: void 0
+        });
+        Object.defineProperty(this, "cause", {
+            enumerable: true,
+            configurable: true,
+            writable: true,
+            value: void 0
+        });
+        Object.defineProperty(this, "valueText", {
+            enumerable: true,
+            configurable: true,
+            writable: true,
+            value: void 0
+        });
+        this.name = "StructureParseError";
+        this.structureName = structureName;
+        this.cause = cause;
+        this.valueText = valueText;
+    }
+}
+exports.StructureParseError = StructureParseError;

package/model-function/generate-structure/StructureParseError.d.ts

@@ -0,0 +1,10 @@
+export declare class StructureParseError extends Error {
+    readonly structureName: string;
+    readonly cause: unknown;
+    readonly valueText: string;
+    constructor({ structureName, valueText, cause, }: {
+        structureName: string;
+        valueText: string;
+        cause: unknown;
+    });
+}

package/model-function/generate-structure/StructureParseError.js

@@ -0,0 +1,30 @@
+import { getErrorMessage } from "../../util/getErrorMessage.js";
+export class StructureParseError extends Error {
+    constructor({ structureName, valueText, cause, }) {
+        super(`Structure parsing failed for '${structureName}'. ` +
+            `Value: ${valueText}.\n` +
+            `Error message: ${getErrorMessage(cause)}`);
+        Object.defineProperty(this, "structureName", {
+            enumerable: true,
+            configurable: true,
+            writable: true,
+            value: void 0
+        });
+        Object.defineProperty(this, "cause", {
+            enumerable: true,
+            configurable: true,
+            writable: true,
+            value: void 0
+        });
+        Object.defineProperty(this, "valueText", {
+            enumerable: true,
+            configurable: true,
+            writable: true,
+            value: void 0
+        });
+        this.name = "StructureParseError";
+        this.structureName = structureName;
+        this.cause = cause;
+        this.valueText = valueText;
+    }
+}

package/model-function/generate-structure/StructureValidationError.cjs

@@ -3,9 +3,9 @@ Object.defineProperty(exports, "__esModule", { value: true });
 exports.StructureValidationError = void 0;
 const getErrorMessage_js_1 = require("../../util/getErrorMessage.cjs");
 class StructureValidationError extends Error {
-    constructor({ structureName, value, cause, }) {
-        super(`Structure validation error for '${structureName}'. ` +
-            `Value: ${JSON.stringify(value)}.\n` +
+    constructor({ structureName, value, valueText, cause, }) {
+        super(`Structure validation failed for '${structureName}'. ` +
+            `Value: ${valueText}.\n` +
             `Error message: ${(0, getErrorMessage_js_1.getErrorMessage)(cause)}`);
         Object.defineProperty(this, "structureName", {
             enumerable: true,
@@ -19,6 +19,12 @@ class StructureValidationError extends Error {
             writable: true,
             value: void 0
         });
+        Object.defineProperty(this, "valueText", {
+            enumerable: true,
+            configurable: true,
+            writable: true,
+            value: void 0
+        });
         Object.defineProperty(this, "value", {
             enumerable: true,
             configurable: true,
@@ -29,6 +35,7 @@ class StructureValidationError extends Error {
         this.structureName = structureName;
         this.cause = cause;
         this.value = value;
+        this.valueText = valueText;
     }
 }
 exports.StructureValidationError = StructureValidationError;

package/model-function/generate-structure/StructureValidationError.d.ts

@@ -1,10 +1,12 @@
 export declare class StructureValidationError extends Error {
     readonly structureName: string;
     readonly cause: unknown;
+    readonly valueText: string;
     readonly value: unknown;
-    constructor({ structureName, value, cause, }: {
+    constructor({ structureName, value, valueText, cause, }: {
         structureName: string;
         value: unknown;
+        valueText: string;
         cause: unknown;
     });
 }

package/model-function/generate-structure/StructureValidationError.js

@@ -1,8 +1,8 @@
 import { getErrorMessage } from "../../util/getErrorMessage.js";
 export class StructureValidationError extends Error {
-    constructor({ structureName, value, cause, }) {
-        super(`Structure validation error for '${structureName}'. ` +
-            `Value: ${JSON.stringify(value)}.\n` +
+    constructor({ structureName, value, valueText, cause, }) {
+        super(`Structure validation failed for '${structureName}'. ` +
+            `Value: ${valueText}.\n` +
             `Error message: ${getErrorMessage(cause)}`);
         Object.defineProperty(this, "structureName", {
             enumerable: true,
@@ -16,6 +16,12 @@ export class StructureValidationError extends Error {
             writable: true,
             value: void 0
         });
+        Object.defineProperty(this, "valueText", {
+            enumerable: true,
+            configurable: true,
+            writable: true,
+            value: void 0
+        });
         Object.defineProperty(this, "value", {
             enumerable: true,
             configurable: true,
@@ -26,5 +32,6 @@ export class StructureValidationError extends Error {
         this.structureName = structureName;
         this.cause = cause;
         this.value = value;
+        this.valueText = valueText;
     }
 }

package/model-function/generate-structure/generateStructure.cjs

@@ -15,11 +15,12 @@ function generateStructure(model, structureDefinition, prompt, options) {
         options,
         generateResponse: async (options) => {
             const result = await model.doGenerateStructure(structureDefinition, expandedPrompt, options);
-            const structure = result.structure;
+            const structure = result.value;
             const parseResult = structureDefinition.schema.validate(structure);
             if (!parseResult.success) {
                 throw new StructureValidationError_js_1.StructureValidationError({
                     structureName: structureDefinition.name,
+                    valueText: result.valueText,
                     value: structure,
                     cause: parseResult.error,
                 });