modelfusion 0.105.0 → 0.107.0

package/CHANGELOG.md

@@ -1,5 +1,31 @@
 # Changelog
 
+## v0.107.0 - 2023-12-29
+
+### Added
+
+- Mistral instruct prompt template
+
+### Changed
+
+- **breaking change**: Renamed `LlamaCppTextGenerationModel` to `LlamaCppCompletionModel`.
+
+### Fixed
+
+- Updated `LlamaCppCompletionModel` to the latest llama.cpp version.
+- Fixed formatting of system prompt for chats in Llama2 2 prompt template.
+
+## v0.106.0 - 2023-12-28
+
+Experimental features that are unlikely to become stable before v1.0 have been moved to a separate `modelfusion-experimental` package.
+
+### Removed
+
+- Cost calculation
+- `guard` function
+- Browser and server features (incl. flow)
+- `summarizeRecursively` function
+
 ## v0.105.0 - 2023-12-26
 
 ### Added

package/README.md

@@ -12,19 +12,17 @@
 
 ## Introduction
 
-**ModelFusion** is a TypeScript library for building AI applications, chatbots, and agents.
+**ModelFusion** is an abstraction layer for integrating AI models into JavaScript and TypeScript applications, unifying the API for common operations such as text streaming, structure generation, and tool usage. It provides features to support production environments, including observability hooks, logging, and automatic retries. You can use ModelFusion to build AI applications, chatbots, and agents.
 
 - **Vendor-neutral**: ModelFusion is a non-commercial open source project that is community-driven. You can use it with any supported provider.
 - **Multi-modal**: ModelFusion supports a wide range of models including text generation, image generation, vision, text-to-speech, speech-to-text, and embedding models.
-- **Streaming**: ModelFusion supports streaming for many generation models, e.g. text streaming, structure streaming, and full duplex speech streaming.
-- **Utility functions**: ModelFusion provides functionality for tools and tool usage, vector indices, and guards functions.
-- **Type inference and validation**: ModelFusion infers TypeScript types wherever possible and to validates model responses.
+- **Type inference and validation**: ModelFusion infers TypeScript types wherever possible and validates model responses.
 - **Observability and logging**: ModelFusion provides an observer framework and out-of-the-box logging support.
-- **Resilience and Robustness**: ModelFusion ensures seamless operation through automatic retries, throttling, and error handling mechanisms.
-- **Server**: ModelFusion provides a Fastify plugin that exposes a ModelFusion flow as a REST endpoint that uses server-sent events.
+- **Resilience and robustness**: ModelFusion ensures seamless operation through automatic retries, throttling, and error handling mechanisms.
+- **Built for production**: ModelFusion is fully tree-shakeable, can be used in serverless environments, and only uses a minimal set of dependencies.
 
 > [!NOTE]
-> ModelFusion is in its initial development phase. The main API is now mostly stable, but until version 1.0 there may be breaking changes. Feedback and suggestions are welcome.
+> ModelFusion is getting closer to a stable v1, which is expected in Q1/2024. The main API is now mostly stable, but until version 1.0 there may be breaking changes. Feedback and suggestions are welcome.
 
 ## Quick Install
 
@@ -47,8 +45,8 @@ You can provide API keys for the different [integrations](https://modelfusion.de
 
 ### [Generate Text](https://modelfusion.dev/guide/function/generate-text)
 
-Generate text using a language model and a prompt. You can stream the text if it is supported by the model. You can use images for multi-modal prompting if the model supports it (e.g. with [llama.cpp](https://modelfusion.dev/guide/)).
-You can use [prompt templates](https://modelfusion.dev/guide/function/generate-text#prompt-format) to change the prompt template of a model.
+Generate text using a language model and a prompt. You can stream the text if it is supported by the model. You can use images for multi-modal prompting if the model supports it (e.g. with [llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp)).
+You can use [prompt templates](https://modelfusion.dev/guide/function/generate-text#prompt-template) to change the prompt template of a model.
 
 #### generateText
 
@@ -61,7 +59,7 @@ const text = await generateText(
 );
 ```
 
-Providers: [OpenAI](https://modelfusion.dev/integration/model-provider/openai), [OpenAI compatible](https://modelfusion.dev/integration/model-provider/openaicompatible), [Llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp), [Ollama](https://modelfusion.dev/integration/model-provider/ollama), [Llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp), [Mistral](https://modelfusion.dev/integration/model-provider/mistral), [Hugging Face](https://modelfusion.dev/integration/model-provider/huggingface), [Cohere](https://modelfusion.dev/integration/model-provider/cohere), [Anthropic](https://modelfusion.dev/integration/model-provider/anthropic)
+Providers: [OpenAI](https://modelfusion.dev/integration/model-provider/openai), [OpenAI compatible](https://modelfusion.dev/integration/model-provider/openaicompatible), [Llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp), [Ollama](https://modelfusion.dev/integration/model-provider/ollama), [Mistral](https://modelfusion.dev/integration/model-provider/mistral), [Hugging Face](https://modelfusion.dev/integration/model-provider/huggingface), [Cohere](https://modelfusion.dev/integration/model-provider/cohere), [Anthropic](https://modelfusion.dev/integration/model-provider/anthropic)
 
 #### streamText
 
@@ -326,53 +324,11 @@ const reconstructedText = await tokenizer.detokenize(tokens);
 
 Providers: [OpenAI](https://modelfusion.dev/integration/model-provider/openai), [Llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp), [Cohere](https://modelfusion.dev/integration/model-provider/cohere)
 
-### [Guards](https://modelfusion.dev/guide/guard)
-
-Guard functions can be used to implement retry on error, redacting and changing reponses, etc.
-
-#### Retry structure parsing on error
-
-```ts
-const result = await guard(
-  (input, options) =>
-    generateStructure(
-      openai
-        .ChatTextGenerator({
-          // ...
-        })
-        .asFunctionCallStructureGenerationModel({
-          fnName: "myFunction",
-        }),
-      zodSchema({
-        // ...
-      }),
-      input,
-      options
-    ),
-  [
-    // ...
-  ],
-  fixStructure({
-    modifyInputForRetry: async ({ input, error }) => [
-      ...input,
-      openai.ChatMessage.assistant(null, {
-        functionCall: {
-          name: "sentiment",
-          arguments: JSON.stringify(error.valueText),
-        },
-      }),
-      openai.ChatMessage.user(error.message),
-      openai.ChatMessage.user("Please fix the error and try again."),
-    ],
-  })
-);
-```
-
 ### [Tools](https://modelfusion.dev/guide/tools)
 
 Tools are functions that can be executed by an AI model. They are useful for building chatbots and agents.
 
-Predefined tools: [SerpAPI](https://modelfusion.dev/integration/tool/serpapi), [Google Custom Search](https://modelfusion.dev/integration/tool/google-custom-search)
+Predefined tools: [Math.js](https://modelfusion.dev/guide/tools/predefined-tools/mathjs), [SerpAPI](https://modelfusion.dev/guide/tools/predefined-tools/serpapi), [Google Custom Search](https://modelfusion.dev/guide/tools/predefined-tools/google-custom-search)
 
 #### [Creating Tools](https://modelfusion.dev/guide/tools/create-tools)
 
@@ -649,7 +605,7 @@ import { modelfusion } from "modelfusion";
 modelfusion.setLogFormat("detailed-object"); // log full events
 ```
 
-### [Server](https://modelfusion.dev/guide/server/)
+### [Server](https://modelfusion.dev/guide/experimental/server/)
 
 > [!WARNING]
 > ModelFusion Server is in its initial development phase and not feature-complete. The API is experimental and breaking changes are likely. Feedback and suggestions are welcome.
@@ -668,7 +624,7 @@ import {
   FileSystemAssetStorage,
   FileSystemLogger,
   modelFusionFastifyPlugin,
-} from "modelfusion/fastify-server"; // '/fastify-server' import path
+} from "modelfusion-experimental/fastify-server"; // '/fastify-server' import path
 
 // configurable logging for all runs using ModelFusion observability:
 const logger = new FileSystemLogger({
@@ -693,7 +649,7 @@ fastify.register(modelFusionFastifyPlugin, {
 Using `invokeFlow`, you can easily connect your client to a ModelFusion flow endpoint:
 
 ```ts
-import { invokeFlow } from "modelfusion/browser"; // '/browser' import path
+import { invokeFlow } from "modelfusion-experimental/browser"; // '/browser' import path
 
 invokeFlow({
   url: `${BASE_URL}/myFlow`,
@@ -727,14 +683,12 @@ invokeFlow({
   - [Generate structure or text](https://modelfusion.dev/guide/function/generate-structure-or-text)
   - [Tokenize Text](https://modelfusion.dev/guide/function/tokenize-text)
   - [Embed Value](https://modelfusion.dev/guide/function/embed)
-- [Guards](https://modelfusion.dev/guide/guard)
 - [Tools](https://modelfusion.dev/guide/tools)
 - [Vector Indices](https://modelfusion.dev/guide/vector-index)
   - [Upsert](https://modelfusion.dev/guide/vector-index/upsert)
   - [Retrieve](https://modelfusion.dev/guide/vector-index/retrieve)
 - [Text Chunks](https://modelfusion.dev/guide/text-chunk/)
   - [Split Text](https://modelfusion.dev/guide/text-chunk/split)
-- [Server](https://modelfusion.dev/guide/server/)
 - [Utilities](https://modelfusion.dev/guide/util/)
   - [API Configuration](https://modelfusion.dev/guide/util/api-configuration)
     - [Retry strategies](https://modelfusion.dev/guide/util/api-configuration/retry)
@@ -743,7 +697,10 @@ invokeFlow({
   - [Observers](https://modelfusion.dev/guide/util/observer)
   - [Runs](https://modelfusion.dev/guide/util/run)
   - [Abort signals](https://modelfusion.dev/guide/util/abort)
-  - [Cost calculation](https://modelfusion.dev/guide/util/cost-calculation)
+- [Experimental](https://modelfusion.dev/guide/experimental/)
+  - [Guards](https://modelfusion.dev/guide/experimental/guard)
+  - [Server](https://modelfusion.dev/guide/experimental/server/)
+  - [Cost calculation](https://modelfusion.dev/guide/experimental/cost-calculation)
 - [Troubleshooting](https://modelfusion.dev/guide/troubleshooting)
   - [Bundling](https://modelfusion.dev/guide/troubleshooting/bundling)
 

package/core/DefaultRun.cjs

@@ -2,7 +2,6 @@
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.DefaultRun = void 0;
 const nanoid_1 = require("nanoid");
-const SuccessfulModelCall_js_1 = require("../model-function/SuccessfulModelCall.cjs");
 const FunctionEventSource_js_1 = require("./FunctionEventSource.cjs");
 class DefaultRun {
     constructor({ runId = `run-${(0, nanoid_1.nanoid)()}`, sessionId, userId, abortSignal, observers, errorHandler, } = {}) {
@@ -69,8 +68,5 @@ class DefaultRun {
             errorHandler: this.errorHandler.bind(this),
         });
     }
-    get successfulModelCalls() {
-        return (0, SuccessfulModelCall_js_1.extractSuccessfulModelCalls)(this.events);
-    }
 }
 exports.DefaultRun = DefaultRun;

package/core/DefaultRun.d.ts

@@ -1,4 +1,3 @@
-import { SuccessfulModelCall } from "../model-function/SuccessfulModelCall.js";
 import { ErrorHandler } from "../util/ErrorHandler.js";
 import { FunctionEvent } from "./FunctionEvent.js";
 import { FunctionObserver } from "./FunctionObserver.js";
@@ -22,5 +21,4 @@ export declare class DefaultRun implements Run {
     readonly functionObserver: {
         onFunctionEvent: (event: FunctionEvent) => void;
     };
-    get successfulModelCalls(): Array<SuccessfulModelCall>;
 }

package/core/DefaultRun.js

@@ -1,5 +1,4 @@
 import { nanoid as createId } from "nanoid";
-import { extractSuccessfulModelCalls, } from "../model-function/SuccessfulModelCall.js";
 import { FunctionEventSource } from "./FunctionEventSource.js";
 export class DefaultRun {
     constructor({ runId = `run-${createId()}`, sessionId, userId, abortSignal, observers, errorHandler, } = {}) {
@@ -66,7 +65,4 @@ export class DefaultRun {
             errorHandler: this.errorHandler.bind(this),
         });
     }
-    get successfulModelCalls() {
-        return extractSuccessfulModelCalls(this.events);
-    }
 }

package/core/ExtensionFunctionEvent.d.ts

@@ -0,0 +1,11 @@
+import { BaseFunctionFinishedEvent, BaseFunctionStartedEvent } from "./FunctionEvent.js";
+export interface ExtensionFunctionStartedEvent extends BaseFunctionStartedEvent {
+    functionType: "extension";
+    extension: string;
+    data: unknown;
+}
+export interface ExtensionFunctionFinishedEvent extends BaseFunctionFinishedEvent {
+    functionType: "extension";
+    extension: string;
+    data: unknown;
+}

package/core/FunctionEvent.d.ts

@@ -1,4 +1,4 @@
-import { GuardFinishedEvent, GuardStartedEvent } from "../guard/GuardEvent.js";
+import { ExtensionFunctionStartedEvent, ExtensionFunctionFinishedEvent } from "./ExtensionFunctionEvent.js";
 import { ModelCallFinishedEvent, ModelCallStartedEvent } from "../model-function/ModelCallEvent.js";
 import { RetrieveFinishedEvent, RetrieveStartedEvent } from "../retriever/RetrieveEvent.js";
 import { ExecuteToolFinishedEvent, ExecuteToolStartedEvent } from "../tool/execute-tool/ExecuteToolEvent.js";
@@ -82,4 +82,4 @@ export interface BaseFunctionFinishedEvent extends BaseFunctionEvent {
      */
     result: BaseFunctionFinishedEventResult;
 }
-export type FunctionEvent = ExecuteFunctionStartedEvent | ExecuteFunctionFinishedEvent | ExecuteToolStartedEvent | ExecuteToolFinishedEvent | GuardStartedEvent | GuardFinishedEvent | ModelCallStartedEvent | ModelCallFinishedEvent | RetrieveStartedEvent | RetrieveFinishedEvent | UpsertIntoVectorIndexStartedEvent | UpsertIntoVectorIndexFinishedEvent | UseToolStartedEvent | UseToolFinishedEvent | UseToolsOrGenerateTextStartedEvent | UseToolsOrGenerateTextFinishedEvent;
+export type FunctionEvent = ExecuteFunctionStartedEvent | ExecuteFunctionFinishedEvent | ExecuteToolStartedEvent | ExecuteToolFinishedEvent | ExtensionFunctionStartedEvent | ExtensionFunctionFinishedEvent | ModelCallStartedEvent | ModelCallFinishedEvent | RetrieveStartedEvent | RetrieveFinishedEvent | UpsertIntoVectorIndexStartedEvent | UpsertIntoVectorIndexFinishedEvent | UseToolStartedEvent | UseToolFinishedEvent | UseToolsOrGenerateTextStartedEvent | UseToolsOrGenerateTextFinishedEvent;

package/extension/index.cjs

@@ -1,15 +1,34 @@
 "use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __exportStar = (this && this.__exportStar) || function(m, exports) {
+    for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
+};
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.AbstractModel = exports.createTextResponseHandler = exports.createJsonResponseHandler = exports.createAudioMpegResponseHandler = exports.postToApi = exports.postJsonToApi = exports.loadApiKey = exports.callWithRetryAndThrottle = void 0;
+exports.parseEventSourceStream = exports.AbstractModel = exports.executeFunctionCall = exports.postToApi = exports.postJsonToApi = exports.createTextResponseHandler = exports.createJsonResponseHandler = exports.createAudioMpegResponseHandler = exports.loadApiKey = exports.callWithRetryAndThrottle = void 0;
+__exportStar(require("../core/ExtensionFunctionEvent.cjs"), exports);
 var callWithRetryAndThrottle_js_1 = require("../core/api/callWithRetryAndThrottle.cjs");
 Object.defineProperty(exports, "callWithRetryAndThrottle", { enumerable: true, get: function () { return callWithRetryAndThrottle_js_1.callWithRetryAndThrottle; } });
 var loadApiKey_js_1 = require("../core/api/loadApiKey.cjs");
 Object.defineProperty(exports, "loadApiKey", { enumerable: true, get: function () { return loadApiKey_js_1.loadApiKey; } });
 var postToApi_js_1 = require("../core/api/postToApi.cjs");
-Object.defineProperty(exports, "postJsonToApi", { enumerable: true, get: function () { return postToApi_js_1.postJsonToApi; } });
-Object.defineProperty(exports, "postToApi", { enumerable: true, get: function () { return postToApi_js_1.postToApi; } });
 Object.defineProperty(exports, "createAudioMpegResponseHandler", { enumerable: true, get: function () { return postToApi_js_1.createAudioMpegResponseHandler; } });
 Object.defineProperty(exports, "createJsonResponseHandler", { enumerable: true, get: function () { return postToApi_js_1.createJsonResponseHandler; } });
 Object.defineProperty(exports, "createTextResponseHandler", { enumerable: true, get: function () { return postToApi_js_1.createTextResponseHandler; } });
+Object.defineProperty(exports, "postJsonToApi", { enumerable: true, get: function () { return postToApi_js_1.postJsonToApi; } });
+Object.defineProperty(exports, "postToApi", { enumerable: true, get: function () { return postToApi_js_1.postToApi; } });
+var executeFunctionCall_js_1 = require("../core/executeFunctionCall.cjs");
+Object.defineProperty(exports, "executeFunctionCall", { enumerable: true, get: function () { return executeFunctionCall_js_1.executeFunctionCall; } });
 var AbstractModel_js_1 = require("../model-function/AbstractModel.cjs");
 Object.defineProperty(exports, "AbstractModel", { enumerable: true, get: function () { return AbstractModel_js_1.AbstractModel; } });
+var parseEventSourceStream_js_1 = require("../util/streaming/parseEventSourceStream.cjs");
+Object.defineProperty(exports, "parseEventSourceStream", { enumerable: true, get: function () { return parseEventSourceStream_js_1.parseEventSourceStream; } });

package/extension/index.d.ts

@@ -1,4 +1,8 @@
+export * from "../core/ExtensionFunctionEvent.js";
 export { callWithRetryAndThrottle } from "../core/api/callWithRetryAndThrottle.js";
 export { loadApiKey } from "../core/api/loadApiKey.js";
-export { postJsonToApi, postToApi, createAudioMpegResponseHandler, createJsonResponseHandler, createTextResponseHandler, ResponseHandler, } from "../core/api/postToApi.js";
+export { ResponseHandler, createAudioMpegResponseHandler, createJsonResponseHandler, createTextResponseHandler, postJsonToApi, postToApi, } from "../core/api/postToApi.js";
+export { executeFunctionCall } from "../core/executeFunctionCall.js";
 export { AbstractModel } from "../model-function/AbstractModel.js";
+export { ErrorHandler } from "../util/ErrorHandler.js";
+export { parseEventSourceStream } from "../util/streaming/parseEventSourceStream.js";

package/extension/index.js

@@ -1,4 +1,7 @@
+export * from "../core/ExtensionFunctionEvent.js";
 export { callWithRetryAndThrottle } from "../core/api/callWithRetryAndThrottle.js";
 export { loadApiKey } from "../core/api/loadApiKey.js";
-export { postJsonToApi, postToApi, createAudioMpegResponseHandler, createJsonResponseHandler, createTextResponseHandler, } from "../core/api/postToApi.js";
+export { createAudioMpegResponseHandler, createJsonResponseHandler, createTextResponseHandler, postJsonToApi, postToApi, } from "../core/api/postToApi.js";
+export { executeFunctionCall } from "../core/executeFunctionCall.js";
 export { AbstractModel } from "../model-function/AbstractModel.js";
+export { parseEventSourceStream } from "../util/streaming/parseEventSourceStream.js";

package/index.cjs

@@ -14,10 +14,7 @@ var __exportStar = (this && this.__exportStar) || function(m, exports) {
     for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
 };
 Object.defineProperty(exports, "__esModule", { value: true });
-__exportStar(require("./composed-function/index.cjs"), exports);
 __exportStar(require("./core/index.cjs"), exports);
-__exportStar(require("./cost/index.cjs"), exports);
-__exportStar(require("./guard/index.cjs"), exports);
 __exportStar(require("./model-function/index.cjs"), exports);
 __exportStar(require("./model-provider/index.cjs"), exports);
 __exportStar(require("./observability/index.cjs"), exports);

package/index.d.ts

@@ -1,7 +1,4 @@
-export * from "./composed-function/index.js";
 export * from "./core/index.js";
-export * from "./cost/index.js";
-export * from "./guard/index.js";
 export * from "./model-function/index.js";
 export * from "./model-provider/index.js";
 export * from "./observability/index.js";

package/index.js

@@ -1,7 +1,4 @@
-export * from "./composed-function/index.js";
 export * from "./core/index.js";
-export * from "./cost/index.js";
-export * from "./guard/index.js";
 export * from "./model-function/index.js";
 export * from "./model-provider/index.js";
 export * from "./observability/index.js";

package/model-function/generate-structure/jsonStructurePrompt.cjs

@@ -2,10 +2,46 @@
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.jsonStructurePrompt = void 0;
 const parseJSON_js_1 = require("../../core/schema/parseJSON.cjs");
-function jsonStructurePrompt(createPrompt) {
-    return {
-        createPrompt,
-        extractStructure: (response) => (0, parseJSON_js_1.parseJSON)({ text: response }),
-    };
+const DEFAULT_SCHEMA_PREFIX = "JSON schema:";
+const DEFAULT_SCHEMA_SUFFIX = "\nYou MUST answer with a JSON object matches the above schema.";
+exports.jsonStructurePrompt = {
+    custom(createPrompt) {
+        return { createPrompt, extractStructure };
+    },
+    text({ schemaPrefix, schemaSuffix, } = {}) {
+        return {
+            createPrompt: (prompt, schema) => ({
+                system: createSystemPrompt({ schema, schemaPrefix, schemaSuffix }),
+                instruction: prompt,
+            }),
+            extractStructure,
+        };
+    },
+    instruction({ schemaPrefix, schemaSuffix, } = {}) {
+        return {
+            createPrompt: (prompt, schema) => ({
+                system: createSystemPrompt({
+                    originalSystemPrompt: prompt.system,
+                    schema,
+                    schemaPrefix,
+                    schemaSuffix,
+                }),
+                instruction: prompt.instruction,
+            }),
+            extractStructure,
+        };
+    },
+};
+function createSystemPrompt({ originalSystemPrompt, schema, schemaPrefix = DEFAULT_SCHEMA_PREFIX, schemaSuffix = DEFAULT_SCHEMA_SUFFIX, }) {
+    return [
+        originalSystemPrompt,
+        schemaPrefix,
+        JSON.stringify(schema.getJsonSchema()),
+        schemaSuffix,
+    ]
+        .filter(Boolean)
+        .join("\n");
+}
+function extractStructure(response) {
+    return (0, parseJSON_js_1.parseJSON)({ text: response });
 }
-exports.jsonStructurePrompt = jsonStructurePrompt;

package/model-function/generate-structure/jsonStructurePrompt.d.ts

@@ -1,4 +1,15 @@
 import { JsonSchemaProducer } from "../../core/schema/JsonSchemaProducer.js";
 import { Schema } from "../../core/schema/Schema.js";
+import { InstructionPrompt } from "../../model-function/generate-text/prompt-template/InstructionPrompt.js";
 import { StructureFromTextPromptTemplate } from "./StructureFromTextPromptTemplate.js";
-export declare function jsonStructurePrompt<SOURCE_PROMPT, TARGET_PROMPT>(createPrompt: (prompt: SOURCE_PROMPT, schema: Schema<unknown> & JsonSchemaProducer) => TARGET_PROMPT): StructureFromTextPromptTemplate<SOURCE_PROMPT, TARGET_PROMPT>;
+export declare const jsonStructurePrompt: {
+    custom<SOURCE_PROMPT, TARGET_PROMPT>(createPrompt: (prompt: SOURCE_PROMPT, schema: Schema<unknown> & JsonSchemaProducer) => TARGET_PROMPT): StructureFromTextPromptTemplate<SOURCE_PROMPT, TARGET_PROMPT>;
+    text({ schemaPrefix, schemaSuffix, }?: {
+        schemaPrefix?: string | undefined;
+        schemaSuffix?: string | undefined;
+    }): StructureFromTextPromptTemplate<string, InstructionPrompt>;
+    instruction({ schemaPrefix, schemaSuffix, }?: {
+        schemaPrefix?: string | undefined;
+        schemaSuffix?: string | undefined;
+    }): StructureFromTextPromptTemplate<InstructionPrompt, InstructionPrompt>;
+};

package/model-function/generate-structure/jsonStructurePrompt.js

@@ -1,7 +1,44 @@
 import { parseJSON } from "../../core/schema/parseJSON.js";
-export function jsonStructurePrompt(createPrompt) {
-    return {
-        createPrompt,
-        extractStructure: (response) => parseJSON({ text: response }),
-    };
+const DEFAULT_SCHEMA_PREFIX = "JSON schema:";
+const DEFAULT_SCHEMA_SUFFIX = "\nYou MUST answer with a JSON object matches the above schema.";
+export const jsonStructurePrompt = {
+    custom(createPrompt) {
+        return { createPrompt, extractStructure };
+    },
+    text({ schemaPrefix, schemaSuffix, } = {}) {
+        return {
+            createPrompt: (prompt, schema) => ({
+                system: createSystemPrompt({ schema, schemaPrefix, schemaSuffix }),
+                instruction: prompt,
+            }),
+            extractStructure,
+        };
+    },
+    instruction({ schemaPrefix, schemaSuffix, } = {}) {
+        return {
+            createPrompt: (prompt, schema) => ({
+                system: createSystemPrompt({
+                    originalSystemPrompt: prompt.system,
+                    schema,
+                    schemaPrefix,
+                    schemaSuffix,
+                }),
+                instruction: prompt.instruction,
+            }),
+            extractStructure,
+        };
+    },
+};
+function createSystemPrompt({ originalSystemPrompt, schema, schemaPrefix = DEFAULT_SCHEMA_PREFIX, schemaSuffix = DEFAULT_SCHEMA_SUFFIX, }) {
+    return [
+        originalSystemPrompt,
+        schemaPrefix,
+        JSON.stringify(schema.getJsonSchema()),
+        schemaSuffix,
+    ]
+        .filter(Boolean)
+        .join("\n");
+}
+function extractStructure(response) {
+    return parseJSON({ text: response });
 }

package/model-function/generate-text/PromptTemplateTextGenerationModel.d.ts

@@ -1,6 +1,7 @@
 import { FunctionOptions } from "../../core/FunctionOptions.js";
 import { TextGenerationToolCallModel, ToolCallPromptTemplate } from "../../tool/generate-tool-call/TextGenerationToolCallModel.js";
-import { TextGenerationToolCallsOrGenerateTextModel, ToolCallsOrGenerateTextPromptTemplate } from "../../tool/generate-tool-calls-or-text/TextGenerationToolCallsOrGenerateTextModel.js";
+import { TextGenerationToolCallsOrGenerateTextModel } from "../../tool/generate-tool-calls-or-text/TextGenerationToolCallsOrGenerateTextModel.js";
+import { ToolCallsOrGenerateTextPromptTemplate } from "../../tool/generate-tool-calls-or-text/ToolCallsOrGenerateTextPromptTemplate.js";
 import { StructureFromTextGenerationModel } from "../generate-structure/StructureFromTextGenerationModel.js";
 import { StructureFromTextPromptTemplate } from "../generate-structure/StructureFromTextPromptTemplate.js";
 import { TextGenerationModel, TextGenerationModelSettings } from "./TextGenerationModel.js";

package/model-function/generate-text/PromptTemplateTextGenerationModel.js

@@ -1,5 +1,5 @@
 import { TextGenerationToolCallModel, } from "../../tool/generate-tool-call/TextGenerationToolCallModel.js";
-import { TextGenerationToolCallsOrGenerateTextModel, } from "../../tool/generate-tool-calls-or-text/TextGenerationToolCallsOrGenerateTextModel.js";
+import { TextGenerationToolCallsOrGenerateTextModel } from "../../tool/generate-tool-calls-or-text/TextGenerationToolCallsOrGenerateTextModel.js";
 import { StructureFromTextGenerationModel } from "../generate-structure/StructureFromTextGenerationModel.js";
 export class PromptTemplateTextGenerationModel {
     constructor({ model, promptTemplate, }) {

package/model-function/generate-text/prompt-template/ChatMLPromptTemplate.test.cjs

@@ -46,4 +46,15 @@ describe("chat prompt", () => {
         });
         expect(prompt).toMatchSnapshot();
     });
+    it("should format prompt with system message and user-assistant-user messages", () => {
+        const prompt = (0, ChatMLPromptTemplate_js_1.chat)().format({
+            system: "you are a chatbot",
+            messages: [
+                { role: "user", content: "1st user message" },
+                { role: "assistant", content: "assistant message" },
+                { role: "user", content: "2nd user message" },
+            ],
+        });
+        expect(prompt).toMatchSnapshot();
+    });
 });

package/model-function/generate-text/prompt-template/ChatMLPromptTemplate.test.js

@@ -44,4 +44,15 @@ describe("chat prompt", () => {
         });
         expect(prompt).toMatchSnapshot();
     });
+    it("should format prompt with system message and user-assistant-user messages", () => {
+        const prompt = chat().format({
+            system: "you are a chatbot",
+            messages: [
+                { role: "user", content: "1st user message" },
+                { role: "assistant", content: "assistant message" },
+                { role: "user", content: "2nd user message" },
+            ],
+        });
+        expect(prompt).toMatchSnapshot();
+    });
 });

package/model-function/generate-text/prompt-template/Llama2PromptTemplate.cjs

@@ -15,7 +15,7 @@ const END_SYSTEM = "\n<</SYS>>\n\n";
  *
  * Llama 2 prompt template:
  * ```
- * <s>[INST]{ instruction } [/INST]
+ * <s>[INST] { instruction } [/INST]
  * ```
  *
  * @see https://www.philschmid.de/llama-2#how-to-prompt-llama-2-chat
@@ -24,7 +24,7 @@ function text() {
     return {
         stopSequences: [END_SEGMENT],
         format(prompt) {
-            return `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt}${END_INSTRUCTION}\n`;
+            return `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt}${END_INSTRUCTION}`;
         },
     };
 }
@@ -72,12 +72,14 @@ function chat() {
     return {
         format(prompt) {
             validateLlama2Prompt(prompt);
-            let text = prompt.system != null
-                ? // Separate section for system message to simplify implementation
-                    // (this is slightly different from the original instructions):
-                    `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${BEGIN_SYSTEM}${prompt.system}${END_SYSTEM}${END_INSTRUCTION}${END_SEGMENT}`
-                : "";
-            for (const { role, content } of prompt.messages) {
+            // get content of the first message (validated to be a user message)
+            const content = prompt.messages[0].content;
+            let text = `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt.system != null
+                ? `${BEGIN_SYSTEM}${prompt.system}${END_SYSTEM}`
+                : ""}${content}${END_INSTRUCTION}`;
+            // process remaining messages
+            for (let i = 1; i < prompt.messages.length; i++) {
+                const { role, content } = prompt.messages[i];
                 switch (role) {
                     case "user": {
                         const textContent = (0, ContentPart_js_1.validateContentIsString)(content, prompt);

package/model-function/generate-text/prompt-template/Llama2PromptTemplate.d.ts

@@ -6,7 +6,7 @@ import { InstructionPrompt } from "./InstructionPrompt.js";
  *
  * Llama 2 prompt template:
  * ```
- * <s>[INST]{ instruction } [/INST]
+ * <s>[INST] { instruction } [/INST]
  * ```
  *
  * @see https://www.philschmid.de/llama-2#how-to-prompt-llama-2-chat

package/model-function/generate-text/prompt-template/Llama2PromptTemplate.js

@@ -12,7 +12,7 @@ const END_SYSTEM = "\n<</SYS>>\n\n";
  *
  * Llama 2 prompt template:
  * ```
- * <s>[INST]{ instruction } [/INST]
+ * <s>[INST] { instruction } [/INST]
  * ```
  *
  * @see https://www.philschmid.de/llama-2#how-to-prompt-llama-2-chat
@@ -21,7 +21,7 @@ export function text() {
     return {
         stopSequences: [END_SEGMENT],
         format(prompt) {
-            return `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt}${END_INSTRUCTION}\n`;
+            return `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt}${END_INSTRUCTION}`;
         },
     };
 }
@@ -67,12 +67,14 @@ export function chat() {
     return {
         format(prompt) {
             validateLlama2Prompt(prompt);
-            let text = prompt.system != null
-                ? // Separate section for system message to simplify implementation
-                    // (this is slightly different from the original instructions):
-                    `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${BEGIN_SYSTEM}${prompt.system}${END_SYSTEM}${END_INSTRUCTION}${END_SEGMENT}`
-                : "";
-            for (const { role, content } of prompt.messages) {
+            // get content of the first message (validated to be a user message)
+            const content = prompt.messages[0].content;
+            let text = `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt.system != null
+                ? `${BEGIN_SYSTEM}${prompt.system}${END_SYSTEM}`
+                : ""}${content}${END_INSTRUCTION}`;
+            // process remaining messages
+            for (let i = 1; i < prompt.messages.length; i++) {
+                const { role, content } = prompt.messages[i];
                 switch (role) {
                     case "user": {
                         const textContent = validateContentIsString(content, prompt);

package/model-function/generate-text/prompt-template/Llama2PromptTemplate.test.cjs

@@ -46,4 +46,15 @@ describe("chat prompt", () => {
         });
         expect(prompt).toMatchSnapshot();
     });
+    it("should format prompt with system message and user-assistant-user messages", () => {
+        const prompt = (0, Llama2PromptTemplate_js_1.chat)().format({
+            system: "you are a chatbot",
+            messages: [
+                { role: "user", content: "1st user message" },
+                { role: "assistant", content: "assistant message" },
+                { role: "user", content: "2nd user message" },
+            ],
+        });
+        expect(prompt).toMatchSnapshot();
+    });
 });