modelfusion 0.106.0 → 0.108.0

package/CHANGELOG.md

@@ -1,5 +1,64 @@
 # Changelog
 
+## v0.108.0 - 2023-12-30
+
+### Added
+
+- [Open AI compatible completion model](https://modelfusion.dev/integration/model-provider/openaicompatible/). It e.g. works with Fireworks AI.
+- Together AI API configuration (for Open AI compatible chat models):
+
+  ```ts
+  import {
+    TogetherAIApiConfiguration,
+    openaicompatible,
+    streamText,
+  } from "modelfusion";
+
+  const textStream = await streamText(
+    openaicompatible
+      .ChatTextGenerator({
+        api: new TogetherAIApiConfiguration(),
+        model: "mistralai/Mixtral-8x7B-Instruct-v0.1",
+      })
+      .withTextPrompt(),
+
+    "Write a story about a robot learning to love"
+  );
+  ```
+
+- Updated Llama.cpp model settings. GBNF grammars can be passed into the `grammar` setting:
+
+  ```ts
+  const text = await generateText(
+    llamacpp
+      .TextGenerator({
+        maxGenerationTokens: 512,
+        temperature: 0,
+        // simple list grammar:
+        grammar: `root ::= ("- " item)+
+  item ::= [^\\n]+ "\\n"`,
+      })
+      .withTextPromptTemplate(MistralInstructPrompt.text()),
+
+    "List 5 ingredients for a lasagna:\n\n"
+  );
+  ```
+
+## v0.107.0 - 2023-12-29
+
+### Added
+
+- Mistral instruct prompt template
+
+### Changed
+
+- **breaking change**: Renamed `LlamaCppTextGenerationModel` to `LlamaCppCompletionModel`.
+
+### Fixed
+
+- Updated `LlamaCppCompletionModel` to the latest llama.cpp version.
+- Fixed formatting of system prompt for chats in Llama2 2 prompt template.
+
 ## v0.106.0 - 2023-12-28
 
 Experimental features that are unlikely to become stable before v1.0 have been moved to a separate `modelfusion-experimental` package.

package/README.md

@@ -16,12 +16,13 @@
 
 - **Vendor-neutral**: ModelFusion is a non-commercial open source project that is community-driven. You can use it with any supported provider.
 - **Multi-modal**: ModelFusion supports a wide range of models including text generation, image generation, vision, text-to-speech, speech-to-text, and embedding models.
-- **Type inference and validation**: ModelFusion infers TypeScript types wherever possible and to validates model responses.
+- **Type inference and validation**: ModelFusion infers TypeScript types wherever possible and validates model responses.
 - **Observability and logging**: ModelFusion provides an observer framework and out-of-the-box logging support.
-- **Resilience and Robustness**: ModelFusion ensures seamless operation through automatic retries, throttling, and error handling mechanisms.
+- **Resilience and robustness**: ModelFusion ensures seamless operation through automatic retries, throttling, and error handling mechanisms.
+- **Built for production**: ModelFusion is fully tree-shakeable, can be used in serverless environments, and only uses a minimal set of dependencies.
 
 > [!NOTE]
-> ModelFusion is in its initial development phase. The main API is now mostly stable, but until version 1.0 there may be breaking changes. Feedback and suggestions are welcome.
+> ModelFusion is getting closer to a stable v1, which is expected in Q1/2024. The main API is now mostly stable, but until version 1.0 there may be breaking changes. Feedback and suggestions are welcome.
 
 ## Quick Install
 
@@ -44,8 +45,8 @@ You can provide API keys for the different [integrations](https://modelfusion.de
 
 ### [Generate Text](https://modelfusion.dev/guide/function/generate-text)
 
-Generate text using a language model and a prompt. You can stream the text if it is supported by the model. You can use images for multi-modal prompting if the model supports it (e.g. with [llama.cpp](https://modelfusion.dev/guide/)).
-You can use [prompt templates](https://modelfusion.dev/guide/function/generate-text#prompt-format) to change the prompt template of a model.
+Generate text using a language model and a prompt. You can stream the text if it is supported by the model. You can use images for multi-modal prompting if the model supports it (e.g. with [llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp)).
+You can use [prompt templates](https://modelfusion.dev/guide/function/generate-text#prompt-template) to change the prompt template of a model.
 
 #### generateText
 
@@ -58,7 +59,7 @@ const text = await generateText(
 );
 ```
 
-Providers: [OpenAI](https://modelfusion.dev/integration/model-provider/openai), [OpenAI compatible](https://modelfusion.dev/integration/model-provider/openaicompatible), [Llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp), [Ollama](https://modelfusion.dev/integration/model-provider/ollama), [Llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp), [Mistral](https://modelfusion.dev/integration/model-provider/mistral), [Hugging Face](https://modelfusion.dev/integration/model-provider/huggingface), [Cohere](https://modelfusion.dev/integration/model-provider/cohere), [Anthropic](https://modelfusion.dev/integration/model-provider/anthropic)
+Providers: [OpenAI](https://modelfusion.dev/integration/model-provider/openai), [OpenAI compatible](https://modelfusion.dev/integration/model-provider/openaicompatible), [Llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp), [Ollama](https://modelfusion.dev/integration/model-provider/ollama), [Mistral](https://modelfusion.dev/integration/model-provider/mistral), [Hugging Face](https://modelfusion.dev/integration/model-provider/huggingface), [Cohere](https://modelfusion.dev/integration/model-provider/cohere), [Anthropic](https://modelfusion.dev/integration/model-provider/anthropic)
 
 #### streamText
 
@@ -323,53 +324,11 @@ const reconstructedText = await tokenizer.detokenize(tokens);
 
 Providers: [OpenAI](https://modelfusion.dev/integration/model-provider/openai), [Llama.cpp](https://modelfusion.dev/integration/model-provider/llamacpp), [Cohere](https://modelfusion.dev/integration/model-provider/cohere)
 
-### [Guards](https://modelfusion.dev/guide/guard)
-
-Guard functions can be used to implement retry on error, redacting and changing reponses, etc.
-
-#### Retry structure parsing on error
-
-```ts
-const result = await guard(
-  (input, options) =>
-    generateStructure(
-      openai
-        .ChatTextGenerator({
-          // ...
-        })
-        .asFunctionCallStructureGenerationModel({
-          fnName: "myFunction",
-        }),
-      zodSchema({
-        // ...
-      }),
-      input,
-      options
-    ),
-  [
-    // ...
-  ],
-  fixStructure({
-    modifyInputForRetry: async ({ input, error }) => [
-      ...input,
-      openai.ChatMessage.assistant(null, {
-        functionCall: {
-          name: "sentiment",
-          arguments: JSON.stringify(error.valueText),
-        },
-      }),
-      openai.ChatMessage.user(error.message),
-      openai.ChatMessage.user("Please fix the error and try again."),
-    ],
-  })
-);
-```
-
 ### [Tools](https://modelfusion.dev/guide/tools)
 
 Tools are functions that can be executed by an AI model. They are useful for building chatbots and agents.
 
-Predefined tools: [SerpAPI](https://modelfusion.dev/integration/tool/serpapi), [Google Custom Search](https://modelfusion.dev/integration/tool/google-custom-search)
+Predefined tools: [Math.js](https://modelfusion.dev/guide/tools/predefined-tools/mathjs), [SerpAPI](https://modelfusion.dev/guide/tools/predefined-tools/serpapi), [Google Custom Search](https://modelfusion.dev/guide/tools/predefined-tools/google-custom-search)
 
 #### [Creating Tools](https://modelfusion.dev/guide/tools/create-tools)
 
@@ -579,16 +538,17 @@ const textStream = await streamText(
 );
 ```
 
-| Prompt Template | Text Prompt | Instruction Prompt | Chat Prompt |
-| --------------- | ----------- | ------------------ | ----------- |
-| OpenAI Chat     | ✅          | ✅                 | ✅          |
-| Anthropic       | ✅          | ✅                 | ✅          |
-| Llama 2         | ✅          | ✅                 | ✅          |
-| ChatML          | ✅          | ✅                 | ✅          |
-| NeuralChat      | ✅          | ✅                 | ✅          |
-| Alpaca          | ✅          | ✅                 | ❌          |
-| Vicuna          | ❌          | ❌                 | ✅          |
-| Generic Text    | ✅          | ✅                 | ✅          |
+| Prompt Template  | Text Prompt | Instruction Prompt | Chat Prompt |
+| ---------------- | ----------- | ------------------ | ----------- |
+| OpenAI Chat      | ✅          | ✅                 | ✅          |
+| Anthropic        | ✅          | ✅                 | ✅          |
+| Llama 2          | ✅          | ✅                 | ✅          |
+| ChatML           | ✅          | ✅                 | ✅          |
+| NeuralChat       | ✅          | ✅                 | ✅          |
+| Mistral Instruct | ✅          | ✅                 | ✅          |
+| Alpaca           | ✅          | ✅                 | ❌          |
+| Vicuna           | ❌          | ❌                 | ✅          |
+| Generic Text     | ✅          | ✅                 | ✅          |
 
 ### [Image Generation Prompt Templates](https://modelfusion.dev/guide/function/generate-image/prompt-format)
 

package/model-function/generate-text/prompt-template/ChatMLPromptTemplate.test.cjs

@@ -46,4 +46,15 @@ describe("chat prompt", () => {
         });
         expect(prompt).toMatchSnapshot();
     });
+    it("should format prompt with system message and user-assistant-user messages", () => {
+        const prompt = (0, ChatMLPromptTemplate_js_1.chat)().format({
+            system: "you are a chatbot",
+            messages: [
+                { role: "user", content: "1st user message" },
+                { role: "assistant", content: "assistant message" },
+                { role: "user", content: "2nd user message" },
+            ],
+        });
+        expect(prompt).toMatchSnapshot();
+    });
 });

package/model-function/generate-text/prompt-template/ChatMLPromptTemplate.test.js

@@ -44,4 +44,15 @@ describe("chat prompt", () => {
         });
         expect(prompt).toMatchSnapshot();
     });
+    it("should format prompt with system message and user-assistant-user messages", () => {
+        const prompt = chat().format({
+            system: "you are a chatbot",
+            messages: [
+                { role: "user", content: "1st user message" },
+                { role: "assistant", content: "assistant message" },
+                { role: "user", content: "2nd user message" },
+            ],
+        });
+        expect(prompt).toMatchSnapshot();
+    });
 });

package/model-function/generate-text/prompt-template/Llama2PromptTemplate.cjs

@@ -24,7 +24,7 @@ function text() {
     return {
         stopSequences: [END_SEGMENT],
         format(prompt) {
-            return `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt}${END_INSTRUCTION}\n`;
+            return `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt}${END_INSTRUCTION}`;
         },
     };
 }
@@ -72,12 +72,14 @@ function chat() {
     return {
         format(prompt) {
             validateLlama2Prompt(prompt);
-            let text = prompt.system != null
-                ? // Separate section for system message to simplify implementation
-                    // (this is slightly different from the original instructions):
-                    `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${BEGIN_SYSTEM}${prompt.system}${END_SYSTEM}${END_INSTRUCTION}${END_SEGMENT}`
-                : "";
-            for (const { role, content } of prompt.messages) {
+            // get content of the first message (validated to be a user message)
+            const content = prompt.messages[0].content;
+            let text = `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt.system != null
+                ? `${BEGIN_SYSTEM}${prompt.system}${END_SYSTEM}`
+                : ""}${content}${END_INSTRUCTION}`;
+            // process remaining messages
+            for (let i = 1; i < prompt.messages.length; i++) {
+                const { role, content } = prompt.messages[i];
                 switch (role) {
                     case "user": {
                         const textContent = (0, ContentPart_js_1.validateContentIsString)(content, prompt);

package/model-function/generate-text/prompt-template/Llama2PromptTemplate.js

@@ -21,7 +21,7 @@ export function text() {
     return {
         stopSequences: [END_SEGMENT],
         format(prompt) {
-            return `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt}${END_INSTRUCTION}\n`;
+            return `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt}${END_INSTRUCTION}`;
         },
     };
 }
@@ -67,12 +67,14 @@ export function chat() {
     return {
         format(prompt) {
             validateLlama2Prompt(prompt);
-            let text = prompt.system != null
-                ? // Separate section for system message to simplify implementation
-                    // (this is slightly different from the original instructions):
-                    `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${BEGIN_SYSTEM}${prompt.system}${END_SYSTEM}${END_INSTRUCTION}${END_SEGMENT}`
-                : "";
-            for (const { role, content } of prompt.messages) {
+            // get content of the first message (validated to be a user message)
+            const content = prompt.messages[0].content;
+            let text = `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt.system != null
+                ? `${BEGIN_SYSTEM}${prompt.system}${END_SYSTEM}`
+                : ""}${content}${END_INSTRUCTION}`;
+            // process remaining messages
+            for (let i = 1; i < prompt.messages.length; i++) {
+                const { role, content } = prompt.messages[i];
                 switch (role) {
                     case "user": {
                         const textContent = validateContentIsString(content, prompt);

package/model-function/generate-text/prompt-template/Llama2PromptTemplate.test.cjs

@@ -46,4 +46,15 @@ describe("chat prompt", () => {
         });
         expect(prompt).toMatchSnapshot();
     });
+    it("should format prompt with system message and user-assistant-user messages", () => {
+        const prompt = (0, Llama2PromptTemplate_js_1.chat)().format({
+            system: "you are a chatbot",
+            messages: [
+                { role: "user", content: "1st user message" },
+                { role: "assistant", content: "assistant message" },
+                { role: "user", content: "2nd user message" },
+            ],
+        });
+        expect(prompt).toMatchSnapshot();
+    });
 });

package/model-function/generate-text/prompt-template/Llama2PromptTemplate.test.js

@@ -44,4 +44,15 @@ describe("chat prompt", () => {
         });
         expect(prompt).toMatchSnapshot();
     });
+    it("should format prompt with system message and user-assistant-user messages", () => {
+        const prompt = chat().format({
+            system: "you are a chatbot",
+            messages: [
+                { role: "user", content: "1st user message" },
+                { role: "assistant", content: "assistant message" },
+                { role: "user", content: "2nd user message" },
+            ],
+        });
+        expect(prompt).toMatchSnapshot();
+    });
 });

package/model-function/generate-text/prompt-template/MistralInstructPromptTemplate.cjs

@@ -0,0 +1,150 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.validateMistralPrompt = exports.chat = exports.instruction = exports.text = void 0;
+const ContentPart_js_1 = require("./ContentPart.cjs");
+const InvalidPromptError_js_1 = require("./InvalidPromptError.cjs");
+const BEGIN_SEGMENT = "<s>";
+const END_SEGMENT = "</s>";
+const BEGIN_INSTRUCTION = "[INST] ";
+const END_INSTRUCTION = " [/INST] ";
+/**
+ * Formats a text prompt as a Mistral instruct prompt.
+ *
+ * Mistral prompt template:
+ * ```
+ * <s>[INST] { instruction } [/INST]
+ * ```
+ *
+ * @see https://docs.mistral.ai/models/#chat-template
+ */
+function text() {
+    return {
+        stopSequences: [END_SEGMENT],
+        format(prompt) {
+            return `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt}${END_INSTRUCTION}`;
+        },
+    };
+}
+exports.text = text;
+/**
+ * Formats an instruction prompt as a Mistral instruct prompt.
+ *
+ * Note that Mistral does not support system prompts. We emulate them.
+ *
+ * Mistral prompt template when system prompt is set:
+ * ```
+ * <s>[INST] ${ system prompt } [/INST] </s>[INST] ${instruction} [/INST] ${ response prefix }
+ * ```
+ *
+ * Mistral prompt template when there is no system prompt:
+ * ```
+ * <s>[INST] ${ instruction } [/INST] ${ response prefix }
+ * ```
+ *
+ * @see https://docs.mistral.ai/models/#chat-template
+ */
+function instruction() {
+    return {
+        stopSequences: [END_SEGMENT],
+        format(prompt) {
+            const instruction = (0, ContentPart_js_1.validateContentIsString)(prompt.instruction, prompt);
+            if (prompt.system != null) {
+                return `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt.system}${END_INSTRUCTION}${END_SEGMENT}${BEGIN_INSTRUCTION}${instruction}${END_INSTRUCTION}${prompt.responsePrefix ?? ""}`;
+            }
+            return `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${instruction}${END_INSTRUCTION}${prompt.responsePrefix ?? ""}`;
+        },
+    };
+}
+exports.instruction = instruction;
+/**
+ * Formats a chat prompt as a Mistral instruct prompt.
+ *
+ * Note that Mistral does not support system prompts. We emulate them.
+ *
+ * Mistral prompt template when system prompt is set:
+ * ```
+ * <s>[INST] ${ system prompt } [/INST] </s> [INST] ${ user msg 1 } [/INST] ${ model response 1 } [INST] ${ user msg 2 } [/INST] ${ model response 2 } [INST] ${ user msg 3 } [/INST]
+ * ```
+ *
+ * Mistral prompt template when there is no system prompt:
+ * ```
+ * <s>[INST] ${ user msg 1 } [/INST] ${ model response 1 } </s>[INST] ${ user msg 2 } [/INST] ${ model response 2 } [INST] ${ user msg 3 } [/INST]
+ * ```
+ *
+ * @see https://docs.mistral.ai/models/#chat-template
+ */
+function chat() {
+    return {
+        format(prompt) {
+            validateMistralPrompt(prompt);
+            let text = "";
+            let i = 0;
+            // handle the special first segment
+            if (prompt.system != null) {
+                text += `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt.system}${END_INSTRUCTION}${END_SEGMENT}`;
+            }
+            else {
+                // get content of the first message (validated to be a user message)
+                text = `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt.messages[0].content}${END_INSTRUCTION}`;
+                // process 2nd message (validated to be an assistant message)
+                if (prompt.messages.length > 1) {
+                    text += `${prompt.messages[1].content}${END_SEGMENT}`;
+                }
+                i = 2;
+            }
+            // process remaining messages
+            for (; i < prompt.messages.length; i++) {
+                const { role, content } = prompt.messages[i];
+                switch (role) {
+                    case "user": {
+                        const textContent = (0, ContentPart_js_1.validateContentIsString)(content, prompt);
+                        text += `${BEGIN_INSTRUCTION}${textContent}${END_INSTRUCTION}`;
+                        break;
+                    }
+                    case "assistant": {
+                        text += (0, ContentPart_js_1.validateContentIsString)(content, prompt);
+                        break;
+                    }
+                    case "tool": {
+                        throw new InvalidPromptError_js_1.InvalidPromptError("Tool messages are not supported.", prompt);
+                    }
+                    default: {
+                        const _exhaustiveCheck = role;
+                        throw new Error(`Unsupported role: ${_exhaustiveCheck}`);
+                    }
+                }
+            }
+            return text;
+        },
+        stopSequences: [END_SEGMENT],
+    };
+}
+exports.chat = chat;
+/**
+ * Checks if a Mistral chat prompt is valid. Throws a {@link ChatPromptValidationError} if it's not.
+ *
+ * - The first message of the chat must be a user message.
+ * - Then it must be alternating between an assistant message and a user message.
+ * - The last message must always be a user message (when submitting to a model).
+ *
+ * The type checking is done at runtime when you submit a chat prompt to a model with a prompt template.
+ *
+ * @throws {@link ChatPromptValidationError}
+ */
+function validateMistralPrompt(chatPrompt) {
+    const messages = chatPrompt.messages;
+    if (messages.length < 1) {
+        throw new InvalidPromptError_js_1.InvalidPromptError("ChatPrompt should have at least one message.", chatPrompt);
+    }
+    for (let i = 0; i < messages.length; i++) {
+        const expectedRole = i % 2 === 0 ? "user" : "assistant";
+        const role = messages[i].role;
+        if (role !== expectedRole) {
+            throw new InvalidPromptError_js_1.InvalidPromptError(`Message at index ${i} should have role '${expectedRole}', but has role '${role}'.`, chatPrompt);
+        }
+    }
+    if (messages.length % 2 === 0) {
+        throw new InvalidPromptError_js_1.InvalidPromptError("The last message must be a user message.", chatPrompt);
+    }
+}
+exports.validateMistralPrompt = validateMistralPrompt;

package/model-function/generate-text/prompt-template/MistralInstructPromptTemplate.d.ts

@@ -0,0 +1,62 @@
+import { TextGenerationPromptTemplate } from "../TextGenerationPromptTemplate.js";
+import { ChatPrompt } from "./ChatPrompt.js";
+import { InstructionPrompt } from "./InstructionPrompt.js";
+/**
+ * Formats a text prompt as a Mistral instruct prompt.
+ *
+ * Mistral prompt template:
+ * ```
+ * <s>[INST] { instruction } [/INST]
+ * ```
+ *
+ * @see https://docs.mistral.ai/models/#chat-template
+ */
+export declare function text(): TextGenerationPromptTemplate<string, string>;
+/**
+ * Formats an instruction prompt as a Mistral instruct prompt.
+ *
+ * Note that Mistral does not support system prompts. We emulate them.
+ *
+ * Mistral prompt template when system prompt is set:
+ * ```
+ * <s>[INST] ${ system prompt } [/INST] </s>[INST] ${instruction} [/INST] ${ response prefix }
+ * ```
+ *
+ * Mistral prompt template when there is no system prompt:
+ * ```
+ * <s>[INST] ${ instruction } [/INST] ${ response prefix }
+ * ```
+ *
+ * @see https://docs.mistral.ai/models/#chat-template
+ */
+export declare function instruction(): TextGenerationPromptTemplate<InstructionPrompt, string>;
+/**
+ * Formats a chat prompt as a Mistral instruct prompt.
+ *
+ * Note that Mistral does not support system prompts. We emulate them.
+ *
+ * Mistral prompt template when system prompt is set:
+ * ```
+ * <s>[INST] ${ system prompt } [/INST] </s> [INST] ${ user msg 1 } [/INST] ${ model response 1 } [INST] ${ user msg 2 } [/INST] ${ model response 2 } [INST] ${ user msg 3 } [/INST]
+ * ```
+ *
+ * Mistral prompt template when there is no system prompt:
+ * ```
+ * <s>[INST] ${ user msg 1 } [/INST] ${ model response 1 } </s>[INST] ${ user msg 2 } [/INST] ${ model response 2 } [INST] ${ user msg 3 } [/INST]
+ * ```
+ *
+ * @see https://docs.mistral.ai/models/#chat-template
+ */
+export declare function chat(): TextGenerationPromptTemplate<ChatPrompt, string>;
+/**
+ * Checks if a Mistral chat prompt is valid. Throws a {@link ChatPromptValidationError} if it's not.
+ *
+ * - The first message of the chat must be a user message.
+ * - Then it must be alternating between an assistant message and a user message.
+ * - The last message must always be a user message (when submitting to a model).
+ *
+ * The type checking is done at runtime when you submit a chat prompt to a model with a prompt template.
+ *
+ * @throws {@link ChatPromptValidationError}
+ */
+export declare function validateMistralPrompt(chatPrompt: ChatPrompt): void;

package/model-function/generate-text/prompt-template/MistralInstructPromptTemplate.js

@@ -0,0 +1,143 @@
+import { validateContentIsString } from "./ContentPart.js";
+import { InvalidPromptError } from "./InvalidPromptError.js";
+const BEGIN_SEGMENT = "<s>";
+const END_SEGMENT = "</s>";
+const BEGIN_INSTRUCTION = "[INST] ";
+const END_INSTRUCTION = " [/INST] ";
+/**
+ * Formats a text prompt as a Mistral instruct prompt.
+ *
+ * Mistral prompt template:
+ * ```
+ * <s>[INST] { instruction } [/INST]
+ * ```
+ *
+ * @see https://docs.mistral.ai/models/#chat-template
+ */
+export function text() {
+    return {
+        stopSequences: [END_SEGMENT],
+        format(prompt) {
+            return `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt}${END_INSTRUCTION}`;
+        },
+    };
+}
+/**
+ * Formats an instruction prompt as a Mistral instruct prompt.
+ *
+ * Note that Mistral does not support system prompts. We emulate them.
+ *
+ * Mistral prompt template when system prompt is set:
+ * ```
+ * <s>[INST] ${ system prompt } [/INST] </s>[INST] ${instruction} [/INST] ${ response prefix }
+ * ```
+ *
+ * Mistral prompt template when there is no system prompt:
+ * ```
+ * <s>[INST] ${ instruction } [/INST] ${ response prefix }
+ * ```
+ *
+ * @see https://docs.mistral.ai/models/#chat-template
+ */
+export function instruction() {
+    return {
+        stopSequences: [END_SEGMENT],
+        format(prompt) {
+            const instruction = validateContentIsString(prompt.instruction, prompt);
+            if (prompt.system != null) {
+                return `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt.system}${END_INSTRUCTION}${END_SEGMENT}${BEGIN_INSTRUCTION}${instruction}${END_INSTRUCTION}${prompt.responsePrefix ?? ""}`;
+            }
+            return `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${instruction}${END_INSTRUCTION}${prompt.responsePrefix ?? ""}`;
+        },
+    };
+}
+/**
+ * Formats a chat prompt as a Mistral instruct prompt.
+ *
+ * Note that Mistral does not support system prompts. We emulate them.
+ *
+ * Mistral prompt template when system prompt is set:
+ * ```
+ * <s>[INST] ${ system prompt } [/INST] </s> [INST] ${ user msg 1 } [/INST] ${ model response 1 } [INST] ${ user msg 2 } [/INST] ${ model response 2 } [INST] ${ user msg 3 } [/INST]
+ * ```
+ *
+ * Mistral prompt template when there is no system prompt:
+ * ```
+ * <s>[INST] ${ user msg 1 } [/INST] ${ model response 1 } </s>[INST] ${ user msg 2 } [/INST] ${ model response 2 } [INST] ${ user msg 3 } [/INST]
+ * ```
+ *
+ * @see https://docs.mistral.ai/models/#chat-template
+ */
+export function chat() {
+    return {
+        format(prompt) {
+            validateMistralPrompt(prompt);
+            let text = "";
+            let i = 0;
+            // handle the special first segment
+            if (prompt.system != null) {
+                text += `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt.system}${END_INSTRUCTION}${END_SEGMENT}`;
+            }
+            else {
+                // get content of the first message (validated to be a user message)
+                text = `${BEGIN_SEGMENT}${BEGIN_INSTRUCTION}${prompt.messages[0].content}${END_INSTRUCTION}`;
+                // process 2nd message (validated to be an assistant message)
+                if (prompt.messages.length > 1) {
+                    text += `${prompt.messages[1].content}${END_SEGMENT}`;
+                }
+                i = 2;
+            }
+            // process remaining messages
+            for (; i < prompt.messages.length; i++) {
+                const { role, content } = prompt.messages[i];
+                switch (role) {
+                    case "user": {
+                        const textContent = validateContentIsString(content, prompt);
+                        text += `${BEGIN_INSTRUCTION}${textContent}${END_INSTRUCTION}`;
+                        break;
+                    }
+                    case "assistant": {
+                        text += validateContentIsString(content, prompt);
+                        break;
+                    }
+                    case "tool": {
+                        throw new InvalidPromptError("Tool messages are not supported.", prompt);
+                    }
+                    default: {
+                        const _exhaustiveCheck = role;
+                        throw new Error(`Unsupported role: ${_exhaustiveCheck}`);
+                    }
+                }
+            }
+            return text;
+        },
+        stopSequences: [END_SEGMENT],
+    };
+}
+/**
+ * Checks if a Mistral chat prompt is valid. Throws a {@link ChatPromptValidationError} if it's not.
+ *
+ * - The first message of the chat must be a user message.
+ * - Then it must be alternating between an assistant message and a user message.
+ * - The last message must always be a user message (when submitting to a model).
+ *
+ * The type checking is done at runtime when you submit a chat prompt to a model with a prompt template.
+ *
+ * @throws {@link ChatPromptValidationError}
+ */
+export function validateMistralPrompt(chatPrompt) {
+    const messages = chatPrompt.messages;
+    if (messages.length < 1) {
+        throw new InvalidPromptError("ChatPrompt should have at least one message.", chatPrompt);
+    }
+    for (let i = 0; i < messages.length; i++) {
+        const expectedRole = i % 2 === 0 ? "user" : "assistant";
+        const role = messages[i].role;
+        if (role !== expectedRole) {
+            throw new InvalidPromptError(`Message at index ${i} should have role '${expectedRole}', but has role '${role}'.`, chatPrompt);
+        }
+    }
+    if (messages.length % 2 === 0) {
+        throw new InvalidPromptError("The last message must be a user message.", chatPrompt);
+    }
+}