modelfusion 0.104.0 → 0.106.0

package/CHANGELOG.md

@@ -1,5 +1,65 @@
 # Changelog
 
+## v0.106.0 - 2023-12-28
+
+Experimental features that are unlikely to become stable before v1.0 have been moved to a separate `modelfusion-experimental` package.
+
+### Removed
+
+- Cost calculation
+- `guard` function
+- Browser and server features (incl. flow)
+- `summarizeRecursively` function
+
+## v0.105.0 - 2023-12-26
+
+### Added
+
+- Tool call support for chat prompts. Assistant messages can contain tool calls, and tool messages can contain tool call results. Tool calls can be used to implement e.g. agents:
+
+  ```ts
+  const chat: ChatPrompt = {
+    system: "You are ...",
+    messages: [ChatMessage.user({ text: instruction })],
+  };
+
+  while (true) {
+    const { text, toolResults } = await useToolsOrGenerateText(
+      openai
+        .ChatTextGenerator({ model: "gpt-4-1106-preview" })
+        .withChatPrompt(),
+      tools, // array of tools
+      chat
+    );
+
+    // add the assistant and tool messages to the chat:
+    chat.messages.push(
+      ChatMessage.assistant({ text, toolResults }),
+      ChatMessage.tool({ toolResults })
+    );
+
+    if (toolResults == null) {
+      return; // no more actions, break loop
+    }
+
+    // ... (handle tool results)
+  }
+  ```
+
+- `streamText` returns a `text` promise when invoked with `fullResponse: true`. After the streaming has finished, the promise resolves with the full text.
+
+  ```ts
+  const { text, textStream } = await streamText(
+    openai.ChatTextGenerator({ model: "gpt-3.5-turbo" }).withTextPrompt(),
+    "Write a short story about a robot learning to love:",
+    { fullResponse: true }
+  );
+
+  // ... (handle streaming)
+
+  console.log(await text); // full text
+  ```
+
 ## v0.104.0 - 2023-12-24
 
 ### Changed

package/README.md

@@ -12,16 +12,13 @@
 
 ## Introduction
 
-**ModelFusion** is a TypeScript library for building AI applications, chatbots, and agents.
+**ModelFusion** is an abstraction layer for integrating AI models into JavaScript and TypeScript applications, unifying the API for common operations such as text streaming, structure generation, and tool usage. It provides features to support production environments, including observability hooks, logging, and automatic retries. You can use ModelFusion to build AI applications, chatbots, and agents.
 
 - **Vendor-neutral**: ModelFusion is a non-commercial open source project that is community-driven. You can use it with any supported provider.
 - **Multi-modal**: ModelFusion supports a wide range of models including text generation, image generation, vision, text-to-speech, speech-to-text, and embedding models.
-- **Streaming**: ModelFusion supports streaming for many generation models, e.g. text streaming, structure streaming, and full duplex speech streaming.
-- **Utility functions**: ModelFusion provides functionality for tools and tool usage, vector indices, and guards functions.
 - **Type inference and validation**: ModelFusion infers TypeScript types wherever possible and to validates model responses.
 - **Observability and logging**: ModelFusion provides an observer framework and out-of-the-box logging support.
 - **Resilience and Robustness**: ModelFusion ensures seamless operation through automatic retries, throttling, and error handling mechanisms.
-- **Server**: ModelFusion provides a Fastify plugin that exposes a ModelFusion flow as a REST endpoint that uses server-sent events.
 
 > [!NOTE]
 > ModelFusion is in its initial development phase. The main API is now mostly stable, but until version 1.0 there may be breaking changes. Feedback and suggestions are welcome.
@@ -649,7 +646,7 @@ import { modelfusion } from "modelfusion";
 modelfusion.setLogFormat("detailed-object"); // log full events
 ```
 
-### [Server](https://modelfusion.dev/guide/server/)
+### [Server](https://modelfusion.dev/guide/experimental/server/)
 
 > [!WARNING]
 > ModelFusion Server is in its initial development phase and not feature-complete. The API is experimental and breaking changes are likely. Feedback and suggestions are welcome.
@@ -668,7 +665,7 @@ import {
   FileSystemAssetStorage,
   FileSystemLogger,
   modelFusionFastifyPlugin,
-} from "modelfusion/fastify-server"; // '/fastify-server' import path
+} from "modelfusion-experimental/fastify-server"; // '/fastify-server' import path
 
 // configurable logging for all runs using ModelFusion observability:
 const logger = new FileSystemLogger({
@@ -693,7 +690,7 @@ fastify.register(modelFusionFastifyPlugin, {
 Using `invokeFlow`, you can easily connect your client to a ModelFusion flow endpoint:
 
 ```ts
-import { invokeFlow } from "modelfusion/browser"; // '/browser' import path
+import { invokeFlow } from "modelfusion-experimental/browser"; // '/browser' import path
 
 invokeFlow({
   url: `${BASE_URL}/myFlow`,
@@ -727,14 +724,12 @@ invokeFlow({
   - [Generate structure or text](https://modelfusion.dev/guide/function/generate-structure-or-text)
   - [Tokenize Text](https://modelfusion.dev/guide/function/tokenize-text)
   - [Embed Value](https://modelfusion.dev/guide/function/embed)
-- [Guards](https://modelfusion.dev/guide/guard)
 - [Tools](https://modelfusion.dev/guide/tools)
 - [Vector Indices](https://modelfusion.dev/guide/vector-index)
   - [Upsert](https://modelfusion.dev/guide/vector-index/upsert)
   - [Retrieve](https://modelfusion.dev/guide/vector-index/retrieve)
 - [Text Chunks](https://modelfusion.dev/guide/text-chunk/)
   - [Split Text](https://modelfusion.dev/guide/text-chunk/split)
-- [Server](https://modelfusion.dev/guide/server/)
 - [Utilities](https://modelfusion.dev/guide/util/)
   - [API Configuration](https://modelfusion.dev/guide/util/api-configuration)
     - [Retry strategies](https://modelfusion.dev/guide/util/api-configuration/retry)
@@ -743,7 +738,10 @@ invokeFlow({
   - [Observers](https://modelfusion.dev/guide/util/observer)
   - [Runs](https://modelfusion.dev/guide/util/run)
   - [Abort signals](https://modelfusion.dev/guide/util/abort)
-  - [Cost calculation](https://modelfusion.dev/guide/util/cost-calculation)
+- [Experimental](https://modelfusion.dev/guide/experimental/)
+  - [Guards](https://modelfusion.dev/guide/experimental/guard)
+  - [Server](https://modelfusion.dev/guide/experimental/server/)
+  - [Cost calculation](https://modelfusion.dev/guide/experimental/cost-calculation)
 - [Troubleshooting](https://modelfusion.dev/guide/troubleshooting)
   - [Bundling](https://modelfusion.dev/guide/troubleshooting/bundling)
 

package/core/DefaultRun.cjs

@@ -2,7 +2,6 @@
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.DefaultRun = void 0;
 const nanoid_1 = require("nanoid");
-const SuccessfulModelCall_js_1 = require("../model-function/SuccessfulModelCall.cjs");
 const FunctionEventSource_js_1 = require("./FunctionEventSource.cjs");
 class DefaultRun {
     constructor({ runId = `run-${(0, nanoid_1.nanoid)()}`, sessionId, userId, abortSignal, observers, errorHandler, } = {}) {
@@ -69,8 +68,5 @@ class DefaultRun {
             errorHandler: this.errorHandler.bind(this),
         });
     }
-    get successfulModelCalls() {
-        return (0, SuccessfulModelCall_js_1.extractSuccessfulModelCalls)(this.events);
-    }
 }
 exports.DefaultRun = DefaultRun;

package/core/DefaultRun.d.ts

@@ -1,4 +1,3 @@
-import { SuccessfulModelCall } from "../model-function/SuccessfulModelCall.js";
 import { ErrorHandler } from "../util/ErrorHandler.js";
 import { FunctionEvent } from "./FunctionEvent.js";
 import { FunctionObserver } from "./FunctionObserver.js";
@@ -22,5 +21,4 @@ export declare class DefaultRun implements Run {
     readonly functionObserver: {
         onFunctionEvent: (event: FunctionEvent) => void;
     };
-    get successfulModelCalls(): Array<SuccessfulModelCall>;
 }

package/core/DefaultRun.js

@@ -1,5 +1,4 @@
 import { nanoid as createId } from "nanoid";
-import { extractSuccessfulModelCalls, } from "../model-function/SuccessfulModelCall.js";
 import { FunctionEventSource } from "./FunctionEventSource.js";
 export class DefaultRun {
     constructor({ runId = `run-${createId()}`, sessionId, userId, abortSignal, observers, errorHandler, } = {}) {
@@ -66,7 +65,4 @@ export class DefaultRun {
             errorHandler: this.errorHandler.bind(this),
         });
     }
-    get successfulModelCalls() {
-        return extractSuccessfulModelCalls(this.events);
-    }
 }

package/core/ExtensionFunctionEvent.d.ts

@@ -0,0 +1,11 @@
+import { BaseFunctionFinishedEvent, BaseFunctionStartedEvent } from "./FunctionEvent.js";
+export interface ExtensionFunctionStartedEvent extends BaseFunctionStartedEvent {
+    functionType: "extension";
+    extension: string;
+    data: unknown;
+}
+export interface ExtensionFunctionFinishedEvent extends BaseFunctionFinishedEvent {
+    functionType: "extension";
+    extension: string;
+    data: unknown;
+}

package/core/FunctionEvent.d.ts

@@ -1,4 +1,4 @@
-import { GuardFinishedEvent, GuardStartedEvent } from "../guard/GuardEvent.js";
+import { ExtensionFunctionStartedEvent, ExtensionFunctionFinishedEvent } from "./ExtensionFunctionEvent.js";
 import { ModelCallFinishedEvent, ModelCallStartedEvent } from "../model-function/ModelCallEvent.js";
 import { RetrieveFinishedEvent, RetrieveStartedEvent } from "../retriever/RetrieveEvent.js";
 import { ExecuteToolFinishedEvent, ExecuteToolStartedEvent } from "../tool/execute-tool/ExecuteToolEvent.js";
@@ -82,4 +82,4 @@ export interface BaseFunctionFinishedEvent extends BaseFunctionEvent {
      */
     result: BaseFunctionFinishedEventResult;
 }
-export type FunctionEvent = ExecuteFunctionStartedEvent | ExecuteFunctionFinishedEvent | ExecuteToolStartedEvent | ExecuteToolFinishedEvent | GuardStartedEvent | GuardFinishedEvent | ModelCallStartedEvent | ModelCallFinishedEvent | RetrieveStartedEvent | RetrieveFinishedEvent | UpsertIntoVectorIndexStartedEvent | UpsertIntoVectorIndexFinishedEvent | UseToolStartedEvent | UseToolFinishedEvent | UseToolsOrGenerateTextStartedEvent | UseToolsOrGenerateTextFinishedEvent;
+export type FunctionEvent = ExecuteFunctionStartedEvent | ExecuteFunctionFinishedEvent | ExecuteToolStartedEvent | ExecuteToolFinishedEvent | ExtensionFunctionStartedEvent | ExtensionFunctionFinishedEvent | ModelCallStartedEvent | ModelCallFinishedEvent | RetrieveStartedEvent | RetrieveFinishedEvent | UpsertIntoVectorIndexStartedEvent | UpsertIntoVectorIndexFinishedEvent | UseToolStartedEvent | UseToolFinishedEvent | UseToolsOrGenerateTextStartedEvent | UseToolsOrGenerateTextFinishedEvent;

package/extension/index.cjs

@@ -1,15 +1,34 @@
 "use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __exportStar = (this && this.__exportStar) || function(m, exports) {
+    for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
+};
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.AbstractModel = exports.createTextResponseHandler = exports.createJsonResponseHandler = exports.createAudioMpegResponseHandler = exports.postToApi = exports.postJsonToApi = exports.loadApiKey = exports.callWithRetryAndThrottle = void 0;
+exports.parseEventSourceStream = exports.AbstractModel = exports.executeFunctionCall = exports.postToApi = exports.postJsonToApi = exports.createTextResponseHandler = exports.createJsonResponseHandler = exports.createAudioMpegResponseHandler = exports.loadApiKey = exports.callWithRetryAndThrottle = void 0;
+__exportStar(require("../core/ExtensionFunctionEvent.cjs"), exports);
 var callWithRetryAndThrottle_js_1 = require("../core/api/callWithRetryAndThrottle.cjs");
 Object.defineProperty(exports, "callWithRetryAndThrottle", { enumerable: true, get: function () { return callWithRetryAndThrottle_js_1.callWithRetryAndThrottle; } });
 var loadApiKey_js_1 = require("../core/api/loadApiKey.cjs");
 Object.defineProperty(exports, "loadApiKey", { enumerable: true, get: function () { return loadApiKey_js_1.loadApiKey; } });
 var postToApi_js_1 = require("../core/api/postToApi.cjs");
-Object.defineProperty(exports, "postJsonToApi", { enumerable: true, get: function () { return postToApi_js_1.postJsonToApi; } });
-Object.defineProperty(exports, "postToApi", { enumerable: true, get: function () { return postToApi_js_1.postToApi; } });
 Object.defineProperty(exports, "createAudioMpegResponseHandler", { enumerable: true, get: function () { return postToApi_js_1.createAudioMpegResponseHandler; } });
 Object.defineProperty(exports, "createJsonResponseHandler", { enumerable: true, get: function () { return postToApi_js_1.createJsonResponseHandler; } });
 Object.defineProperty(exports, "createTextResponseHandler", { enumerable: true, get: function () { return postToApi_js_1.createTextResponseHandler; } });
+Object.defineProperty(exports, "postJsonToApi", { enumerable: true, get: function () { return postToApi_js_1.postJsonToApi; } });
+Object.defineProperty(exports, "postToApi", { enumerable: true, get: function () { return postToApi_js_1.postToApi; } });
+var executeFunctionCall_js_1 = require("../core/executeFunctionCall.cjs");
+Object.defineProperty(exports, "executeFunctionCall", { enumerable: true, get: function () { return executeFunctionCall_js_1.executeFunctionCall; } });
 var AbstractModel_js_1 = require("../model-function/AbstractModel.cjs");
 Object.defineProperty(exports, "AbstractModel", { enumerable: true, get: function () { return AbstractModel_js_1.AbstractModel; } });
+var parseEventSourceStream_js_1 = require("../util/streaming/parseEventSourceStream.cjs");
+Object.defineProperty(exports, "parseEventSourceStream", { enumerable: true, get: function () { return parseEventSourceStream_js_1.parseEventSourceStream; } });

package/extension/index.d.ts

@@ -1,4 +1,8 @@
+export * from "../core/ExtensionFunctionEvent.js";
 export { callWithRetryAndThrottle } from "../core/api/callWithRetryAndThrottle.js";
 export { loadApiKey } from "../core/api/loadApiKey.js";
-export { postJsonToApi, postToApi, createAudioMpegResponseHandler, createJsonResponseHandler, createTextResponseHandler, ResponseHandler, } from "../core/api/postToApi.js";
+export { ResponseHandler, createAudioMpegResponseHandler, createJsonResponseHandler, createTextResponseHandler, postJsonToApi, postToApi, } from "../core/api/postToApi.js";
+export { executeFunctionCall } from "../core/executeFunctionCall.js";
 export { AbstractModel } from "../model-function/AbstractModel.js";
+export { ErrorHandler } from "../util/ErrorHandler.js";
+export { parseEventSourceStream } from "../util/streaming/parseEventSourceStream.js";

package/extension/index.js

@@ -1,4 +1,7 @@
+export * from "../core/ExtensionFunctionEvent.js";
 export { callWithRetryAndThrottle } from "../core/api/callWithRetryAndThrottle.js";
 export { loadApiKey } from "../core/api/loadApiKey.js";
-export { postJsonToApi, postToApi, createAudioMpegResponseHandler, createJsonResponseHandler, createTextResponseHandler, } from "../core/api/postToApi.js";
+export { createAudioMpegResponseHandler, createJsonResponseHandler, createTextResponseHandler, postJsonToApi, postToApi, } from "../core/api/postToApi.js";
+export { executeFunctionCall } from "../core/executeFunctionCall.js";
 export { AbstractModel } from "../model-function/AbstractModel.js";
+export { parseEventSourceStream } from "../util/streaming/parseEventSourceStream.js";

package/index.cjs

@@ -14,10 +14,7 @@ var __exportStar = (this && this.__exportStar) || function(m, exports) {
     for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
 };
 Object.defineProperty(exports, "__esModule", { value: true });
-__exportStar(require("./composed-function/index.cjs"), exports);
 __exportStar(require("./core/index.cjs"), exports);
-__exportStar(require("./cost/index.cjs"), exports);
-__exportStar(require("./guard/index.cjs"), exports);
 __exportStar(require("./model-function/index.cjs"), exports);
 __exportStar(require("./model-provider/index.cjs"), exports);
 __exportStar(require("./observability/index.cjs"), exports);

package/index.d.ts

@@ -1,7 +1,4 @@
-export * from "./composed-function/index.js";
 export * from "./core/index.js";
-export * from "./cost/index.js";
-export * from "./guard/index.js";
 export * from "./model-function/index.js";
 export * from "./model-provider/index.js";
 export * from "./observability/index.js";

package/index.js

@@ -1,7 +1,4 @@
-export * from "./composed-function/index.js";
 export * from "./core/index.js";
-export * from "./cost/index.js";
-export * from "./guard/index.js";
 export * from "./model-function/index.js";
 export * from "./model-provider/index.js";
 export * from "./observability/index.js";

package/model-function/Delta.d.ts

@@ -1,7 +1,6 @@
 export type Delta<T> = {
     type: "delta";
-    fullDelta: unknown;
-    valueDelta: T;
+    deltaValue: T;
 } | {
     type: "error";
     error: unknown;

package/model-function/executeStreamCall.cjs

@@ -11,7 +11,7 @@ const getRun_js_1 = require("../core/getRun.cjs");
 const AsyncQueue_js_1 = require("../util/AsyncQueue.cjs");
 const DurationMeasurement_js_1 = require("../util/DurationMeasurement.cjs");
 const runSafe_js_1 = require("../util/runSafe.cjs");
-async function executeStreamCall({ model, options, input, functionType, startStream, processDelta, processFinished, getResult, }) {
+async function executeStreamCall({ model, options, input, functionType, startStream, processDelta, processFinished, onDone, }) {
     const run = await (0, getRun_js_1.getRun)(options?.run);
     const settings = model.settings;
     const eventSource = new FunctionEventSource_js_1.FunctionEventSource({
@@ -121,6 +121,7 @@ async function executeStreamCall({ model, options, input, functionType, startStr
                     responseQueue.error(loopResult.error);
                     return; // error is handled through queue
                 }
+                onDone?.();
                 const finishMetadata = {
                     eventType: "finished",
                     ...startMetadata,
@@ -131,7 +132,6 @@ async function executeStreamCall({ model, options, input, functionType, startStr
                     ...finishMetadata,
                     result: {
                         status: "success",
-                        ...getResult(),
                     },
                 });
             }
@@ -140,7 +140,9 @@ async function executeStreamCall({ model, options, input, functionType, startStr
                 responseQueue.close();
             }
         })();
-        return responseQueue;
+        return {
+            stream: responseQueue,
+        };
     });
     if (!result.ok) {
         const finishMetadata = {
@@ -170,7 +172,7 @@ async function executeStreamCall({ model, options, input, functionType, startStr
         throw result.error;
     }
     return {
-        value: result.value,
+        value: result.value.stream,
         metadata: startMetadata,
     };
 }

package/model-function/executeStreamCall.d.ts

@@ -3,7 +3,7 @@ import { Delta } from "./Delta.js";
 import { Model, ModelSettings } from "./Model.js";
 import { ModelCallStartedEvent } from "./ModelCallEvent.js";
 import { ModelCallMetadata } from "./ModelCallMetadata.js";
-export declare function executeStreamCall<DELTA_VALUE, VALUE, MODEL extends Model<ModelSettings>>({ model, options, input, functionType, startStream, processDelta, processFinished, getResult, }: {
+export declare function executeStreamCall<DELTA_VALUE, VALUE, MODEL extends Model<ModelSettings>>({ model, options, input, functionType, startStream, processDelta, processFinished, onDone, }: {
     model: MODEL;
     options?: FunctionOptions;
     input: unknown;
@@ -13,7 +13,7 @@ export declare function executeStreamCall<DELTA_VALUE, VALUE, MODEL extends Mode
         type: "delta";
     }) => VALUE | undefined;
     processFinished?: () => VALUE | undefined;
-    getResult: () => Record<string, unknown>;
+    onDone?: () => void;
 }): Promise<{
     value: AsyncIterable<VALUE>;
     metadata: Omit<ModelCallMetadata, "durationInMs" | "finishTimestamp">;

package/model-function/executeStreamCall.js

@@ -8,7 +8,7 @@ import { getRun } from "../core/getRun.js";
 import { AsyncQueue } from "../util/AsyncQueue.js";
 import { startDurationMeasurement } from "../util/DurationMeasurement.js";
 import { runSafe } from "../util/runSafe.js";
-export async function executeStreamCall({ model, options, input, functionType, startStream, processDelta, processFinished, getResult, }) {
+export async function executeStreamCall({ model, options, input, functionType, startStream, processDelta, processFinished, onDone, }) {
     const run = await getRun(options?.run);
     const settings = model.settings;
     const eventSource = new FunctionEventSource({
@@ -118,6 +118,7 @@ export async function executeStreamCall({ model, options, input, functionType, s
                     responseQueue.error(loopResult.error);
                     return; // error is handled through queue
                 }
+                onDone?.();
                 const finishMetadata = {
                     eventType: "finished",
                     ...startMetadata,
@@ -128,7 +129,6 @@ export async function executeStreamCall({ model, options, input, functionType, s
                     ...finishMetadata,
                     result: {
                         status: "success",
-                        ...getResult(),
                     },
                 });
             }
@@ -137,7 +137,9 @@ export async function executeStreamCall({ model, options, input, functionType, s
                 responseQueue.close();
             }
         })();
-        return responseQueue;
+        return {
+            stream: responseQueue,
+        };
     });
     if (!result.ok) {
         const finishMetadata = {
@@ -167,7 +169,7 @@ export async function executeStreamCall({ model, options, input, functionType, s
         throw result.error;
     }
     return {
-        value: result.value,
+        value: result.value.stream,
         metadata: startMetadata,
     };
 }

package/model-function/generate-speech/streamSpeech.cjs

@@ -21,8 +21,7 @@ async function streamSpeech(model, text, options) {
         model,
         options,
         startStream: async (options) => model.doGenerateSpeechStreamDuplex(textStream, options),
-        processDelta: (delta) => delta.valueDelta,
-        getResult: () => ({}),
+        processDelta: (delta) => delta.deltaValue,
     });
     return options?.fullResponse
         ? {

package/model-function/generate-speech/streamSpeech.js

@@ -18,8 +18,7 @@ export async function streamSpeech(model, text, options) {
         model,
         options,
         startStream: async (options) => model.doGenerateSpeechStreamDuplex(textStream, options),
-        processDelta: (delta) => delta.valueDelta,
-        getResult: () => ({}),
+        processDelta: (delta) => delta.deltaValue,
     });
     return options?.fullResponse
         ? {

package/model-function/generate-structure/StructureFromTextStreamingModel.cjs

@@ -11,35 +11,6 @@ class StructureFromTextStreamingModel extends StructureFromTextGenerationModel_j
     constructor(options) {
         super(options);
     }
-    async doStreamStructure(schema, prompt, options) {
-        const textStream = await (0, streamText_js_1.streamText)(this.model, this.template.createPrompt(prompt, schema), options);
-        const queue = new AsyncQueue_js_1.AsyncQueue();
-        // run async on purpose:
-        (async () => {
-            try {
-                let fullText = "";
-                for await (const deltaText of textStream) {
-                    fullText += deltaText;
-                    const deltaStructure = (0, parsePartialJson_js_1.parsePartialJson)(fullText);
-                    // only publish parsable structures
-                    if (deltaStructure != null) {
-                        queue.push({
-                            type: "delta",
-                            fullDelta: fullText,
-                            valueDelta: deltaStructure,
-                        });
-                    }
-                }
-            }
-            catch (error) {
-                queue.push({ type: "error", error });
-            }
-            finally {
-                queue.close();
-            }
-        })();
-        return queue;
-    }
     async doGenerateStructure(schema, prompt, options) {
         const { response, text } = await (0, generateText_js_1.generateText)(this.model, this.template.createPrompt(prompt, schema), {
             ...options,
@@ -59,6 +30,31 @@ class StructureFromTextStreamingModel extends StructureFromTextGenerationModel_j
             });
         }
     }
+    async doStreamStructure(schema, prompt, options) {
+        const textStream = await (0, streamText_js_1.streamText)(this.model, this.template.createPrompt(prompt, schema), options);
+        const queue = new AsyncQueue_js_1.AsyncQueue();
+        // run async on purpose:
+        (async () => {
+            try {
+                for await (const deltaText of textStream) {
+                    queue.push({ type: "delta", deltaValue: deltaText });
+                }
+            }
+            catch (error) {
+                queue.push({ type: "error", error });
+            }
+            finally {
+                queue.close();
+            }
+        })();
+        return queue;
+    }
+    extractStructureTextDelta(delta) {
+        return delta;
+    }
+    parseAccumulatedStructureText(accumulatedText) {
+        return (0, parsePartialJson_js_1.parsePartialJson)(accumulatedText);
+    }
     withSettings(additionalSettings) {
         return new StructureFromTextStreamingModel({
             model: this.model.withSettings(additionalSettings),

package/model-function/generate-structure/StructureFromTextStreamingModel.d.ts

@@ -12,11 +12,13 @@ export declare class StructureFromTextStreamingModel<SOURCE_PROMPT, TARGET_PROMP
         model: MODEL;
         template: StructureFromTextPromptTemplate<SOURCE_PROMPT, TARGET_PROMPT>;
     });
-    doStreamStructure(schema: Schema<unknown> & JsonSchemaProducer, prompt: SOURCE_PROMPT, options?: FunctionOptions): Promise<AsyncQueue<Delta<unknown>>>;
     doGenerateStructure(schema: Schema<unknown> & JsonSchemaProducer, prompt: SOURCE_PROMPT, options?: FunctionOptions): Promise<{
         response: unknown;
         value: unknown;
         valueText: string;
     }>;
+    doStreamStructure(schema: Schema<unknown> & JsonSchemaProducer, prompt: SOURCE_PROMPT, options?: FunctionOptions): Promise<AsyncQueue<Delta<string>>>;
+    extractStructureTextDelta(delta: unknown): string;
+    parseAccumulatedStructureText(accumulatedText: string): unknown;
     withSettings(additionalSettings: Partial<MODEL["settings"]>): this;
 }

package/model-function/generate-structure/StructureFromTextStreamingModel.js

@@ -8,35 +8,6 @@ export class StructureFromTextStreamingModel extends StructureFromTextGeneration
     constructor(options) {
         super(options);
     }
-    async doStreamStructure(schema, prompt, options) {
-        const textStream = await streamText(this.model, this.template.createPrompt(prompt, schema), options);
-        const queue = new AsyncQueue();
-        // run async on purpose:
-        (async () => {
-            try {
-                let fullText = "";
-                for await (const deltaText of textStream) {
-                    fullText += deltaText;
-                    const deltaStructure = parsePartialJson(fullText);
-                    // only publish parsable structures
-                    if (deltaStructure != null) {
-                        queue.push({
-                            type: "delta",
-                            fullDelta: fullText,
-                            valueDelta: deltaStructure,
-                        });
-                    }
-                }
-            }
-            catch (error) {
-                queue.push({ type: "error", error });
-            }
-            finally {
-                queue.close();
-            }
-        })();
-        return queue;
-    }
     async doGenerateStructure(schema, prompt, options) {
         const { response, text } = await generateText(this.model, this.template.createPrompt(prompt, schema), {
             ...options,
@@ -56,6 +27,31 @@ export class StructureFromTextStreamingModel extends StructureFromTextGeneration
             });
         }
     }
+    async doStreamStructure(schema, prompt, options) {
+        const textStream = await streamText(this.model, this.template.createPrompt(prompt, schema), options);
+        const queue = new AsyncQueue();
+        // run async on purpose:
+        (async () => {
+            try {
+                for await (const deltaText of textStream) {
+                    queue.push({ type: "delta", deltaValue: deltaText });
+                }
+            }
+            catch (error) {
+                queue.push({ type: "error", error });
+            }
+            finally {
+                queue.close();
+            }
+        })();
+        return queue;
+    }
+    extractStructureTextDelta(delta) {
+        return delta;
+    }
+    parseAccumulatedStructureText(accumulatedText) {
+        return parsePartialJson(accumulatedText);
+    }
     withSettings(additionalSettings) {
         return new StructureFromTextStreamingModel({
             model: this.model.withSettings(additionalSettings),

package/model-function/generate-structure/StructureGenerationModel.d.ts

@@ -19,4 +19,6 @@ export interface StructureGenerationModel<PROMPT, SETTINGS extends StructureGene
 }
 export interface StructureStreamingModel<PROMPT, SETTINGS extends StructureGenerationModelSettings = StructureGenerationModelSettings> extends StructureGenerationModel<PROMPT, SETTINGS> {
     doStreamStructure(schema: Schema<unknown> & JsonSchemaProducer, prompt: PROMPT, options?: FunctionOptions): PromiseLike<AsyncIterable<Delta<unknown>>>;
+    extractStructureTextDelta(delta: unknown): string | undefined;
+    parseAccumulatedStructureText(accumulatedText: string): unknown;
 }