modelmix 1.0.2 → 1.1.2

package/README.md

@@ -1,14 +1,15 @@
-# 🧬 ModelMix: Integrate Multiple AI Language Models with Ease 🚀
+# 🧬 ModelMix: Unified API for Diverse AI Language Models
 
 **ModelMix** is a versatile module that enables seamless integration of various language models from different providers through a unified interface. With ModelMix, you can effortlessly manage and utilize multiple AI models while controlling parallel requests to avoid provider restrictions. 
 
-## Features ✨
+## ✨ Features 
 
 - **Unified Interface**: Interact with multiple AI models through a single, coherent API.
 - **Request Control**: Manage the number of parallel requests to adhere to provider limitations.
 - **Flexible Integration**: Easily integrate popular models like OpenAI and Anthropic, as well as custom models such as Perplexity.
+- **History Tracking:** Automatically logs the conversation history with model responses, allowing you to limit the number of historical messages with max_history.
 
-## Installation 📦
+## 📦 Installation
 
 First, install the ModelMix package:
 
@@ -22,7 +23,7 @@ You'll also need to install the respective SDKs for each model provider you plan
 npm install openai @anthropic-ai/sdk dotenv
 ```
 
-## Usage 🛠️
+## 🛠️ Usage
 
 Here's a quick example to get you started:
 
@@ -37,14 +38,22 @@ Here's a quick example to get you started:
 
     ```javascript
     import 'dotenv/config'
-    const env = process.env;
-
     import OpenAI from 'openai';
     import Anthropic from '@anthropic-ai/sdk';
 
     import { ModelMix, OpenAIModel, AnthropicModel, CustomModel } from 'model-mix';
 
-    const driver = new ModelMix({ system: "You are ALF from Melmac.", max_tokens: 200 });
+    const env = process.env;
+
+    const driver = new ModelMix({
+        options: {
+            max_tokens: 200,
+        },
+        config: {
+            system: "You are ALF from Melmac.",
+            max_history: 2
+        }
+    });
 
     driver.attach(new OpenAIModel(new OpenAI({ apiKey: env.OPENAI_API_KEY })));
     driver.attach(new AnthropicModel(new Anthropic({ apiKey: env.ANTHROPIC_API_KEY })));
@@ -52,7 +61,8 @@ Here's a quick example to get you started:
         config: {
             url: 'https://api.perplexity.ai/chat/completions',
             bearer: env.PPLX_API_KEY,
-            prefix: ["pplx", "llama", "mixtral"]
+            prefix: ["pplx", "llama", "mixtral"],
+            system: "You are my personal assistant."
         }
     }));
     ```
@@ -60,75 +70,87 @@ Here's a quick example to get you started:
 3. **Generate responses from different models**:
 
     ```javascript
-    const question = 'Have you ever eaten a cat?';
+    const claude = await driver.create('claude-3-sonnet-20240229', { temperature: 0.5 });
+    await claude.addImage("./watson.png")
+    const imageDescription = await claude.addText("describe the image").message();
+    console.log(imageDescription);
+
+    const gpt = await driver.create('gpt-4o', { temperature: 0.5 });
+    const question = await gpt.addText("Have you ever eaten a cat?").message();
+    console.log(question);
+
+    const pplx = await driver.create('pplx-70b-online', { max_tokens: 500 });
+    await pplx.addText('How much is ETH trading in USD?');
+    const news = await pplx.addText('What are the 3 most recent Ethereum news?').message();
+    console.log(news);
+    ```
 
-    console.log("OpenAI - gpt-4o");
-    const txtGPT = await driver.create(question, 'gpt-4o', { max_tokens: 100 });
-    console.log(txtGPT);
+4. Find the files for this example at: [/ModelMix/demo](https://github.com/clasen/ModelMix/tree/master/demo).
 
-    console.log("-------\n");
 
-    console.log("Anthropic - claude-3-sonnet-20240229");
-    const txtClaude = await driver.create(question, 'claude-3-sonnet-20240229', { temperature: 0.5 });
-    console.log(txtClaude);
+## 📚 ModelMix Class Overview
 
-    console.log("-------\n");
+#### ModelMix
 
-    console.log("Perplexity - pplx-70b-online");
-    const txtPPLX = await driver.create(question, 'pplx-70b-online');
-    console.log(txtPPLX);
-    ```
+**Constructor**
 
-## ModelMix Class Overview 📚
+```javascript
+new ModelMix(args = { options: {}, config: {} })
+```
 
-### `ModelMix` Class
+- **args**: Configuration object with `options` and `config` properties.
 
-#### Constructor
-- **options**: An optional configuration object for setting default values.
+**Methods**
 
-#### Methods
-- **`attach(modelInstance)`**: Attach a new model instance to the ModelMix.
-  - **modelInstance**: An instance of a model class (e.g., `OpenAIModel`, `AnthropicModel`, `CustomModel`).
+- `attach(modelInstance)`: Attaches a model instance to the `ModelMix`.
+- `create(modelKey, overOptions = {})`: Creates a new `MessageHandler` for the specified model.
 
-- **`create(prompt, modelKey, options = {})`**: Create a request to a specific model.
-  - **prompt**: The input prompt to send to the model.
-  - **modelKey**: The model key to identify which model to use.
-  - **options**: Optional configuration overrides for the request.
+#### MessageHandler
 
-- **`processQueue(modelEntry)`**: Process the request queue for a specific model instance.
-  - **modelEntry**: The model instance whose queue will be processed.
+- **mix**: Instance of `ModelMix`.
+- **modelEntry**: Model entry object.
+- **options**: Options for the message handler.
+- **config**: Configuration for the message handler.
 
-### `OpenAIModel` Class
+**Methods**
 
-#### Constructor
-- **openai**: An instance of the OpenAI client.
-- **args**: Configuration and options for the model.
+- `new()`: Initializes a new message handler instance.
+- `addText(text, config = { role: "user" })`: Adds a text message.
+- `addImage(filePath, config = { role: "user" })`: Adds an image message from a file path.
+- `message()`: Sends the message and returns the response.
+- `raw()`: Sends the message and returns the raw response data.
 
-#### Methods
-- **`create(prompt, options = {})`**: Send a prompt to the OpenAI model and return the response.
-  - **prompt**: The input prompt to send to the model.
-  - **options**: Optional configuration overrides for the request.
+#### OpenAIModel
 
-### `AnthropicModel` Class
+**Constructor**
 
-#### Constructor
-- **anthropic**: An instance of the Anthropic client.
-- **args**: Configuration and options for the model.
+```javascript
+new OpenAIModel(openai, args = { options: {}, config: {} })
+```
+
+- **openai**: Instance of the OpenAI client.
+- **args**: Configuration object with `options` and `config` properties.
+
+#### AnthropicModel
+
+**Constructor**
 
-#### Methods
-- **`create(prompt, options = {})`**: Send a prompt to the Anthropic model and return the response.
-  - **prompt**: The input prompt to send to the model.
-  - **options**: Optional configuration overrides for the request.
+```javascript
+new AnthropicModel(anthropic, args = { options: {}, config: {} })
+```
+
+- **anthropic**: Instance of the Anthropic client.
+- **args**: Configuration object with `options` and `config` properties.
 
-### `CustomModel` Class
+#### CustomModel
 
-#### Constructor
-- **args**: Configuration and options for the model.
+**Constructor**
+
+```javascript
+new CustomModel(args = { config: {}, options: {} })
+```
 
-#### Methods
-- **`create(prompt, options = {})`**: Send a prompt to a custom model and return the response.
-  - **prompt**: The input prompt to send to the model.
-  - **options**: Optional configuration overrides for the request.
+- **args**: Configuration object with `config` and `options` properties.
 
 ## 🤝 Contributing
 

package/demo/demo.mjs

@@ -1,13 +1,22 @@
 
 import 'dotenv/config'
-const env = process.env;
-
 import OpenAI from 'openai';
 import Anthropic from '@anthropic-ai/sdk';
 
 import { ModelMix, OpenAIModel, AnthropicModel, CustomModel } from '../index.js';
 
-const driver = new ModelMix({ system: "You are ALF from Melmac.", max_tokens: 200 });
+const env = process.env;
+
+
+const driver = new ModelMix({
+    options: {
+        max_tokens: 200,
+    },
+    config: {
+        system: "You are ALF from Melmac.",
+        max_history: 2
+    }
+});
 
 driver.attach(new OpenAIModel(new OpenAI({ apiKey: env.OPENAI_API_KEY })));
 driver.attach(new AnthropicModel(new Anthropic({ apiKey: env.ANTHROPIC_API_KEY })));
@@ -15,35 +24,21 @@ driver.attach(new CustomModel({
     config: {
         url: 'https://api.perplexity.ai/chat/completions',
         bearer: env.PPLX_API_KEY,
-        prefix: ["pplx", "llama", "mixtral"]
+        prefix: ["pplx", "llama", "mixtral"],
+        system: "You are my personal assistant."
     }
 }));
 
-const question = 'Have you ever eaten a cat?';
-
-console.log("OpenAI - gpt-4o");
-const txtGPT = await driver.create(question, 'gpt-4o', { max_tokens: 100 });
-console.log(txtGPT);
-/*
-OpenAI - gpt-4o
-No way, I have not eaten a cat, at least not since I crash-landed on Earth! I've learned a lot about how pets are cherished members of the family here. So while cats were considered a delicacy on Melmac, I've since adapted to Earth customs and have a new appreciation for them, even though I still might joke about it from time to time! Have you got any snacks lying around that aren’t feline?
-*/
-
-console.log("-------\n");
-
-console.log("Anthropic - claude-3-sonnet-20240229");
-const txtClaude = await driver.create(question, 'claude-3-sonnet-20240229', { temperature: 0.5 });
-console.log(txtClaude);
-/*
-Anthropic - claude-3-sonnet-20240229
-No, I'm an AI assistant created by Anthropic to be helpful, harmless, and honest. I don't actually eat anything since I'm an artificial intelligence without a physical body.
-*/
-
-console.log("-------\n");
-
-console.log("Perplexity - pplx-70b-online");
-const txtPPLX = await driver.create(question, 'pplx-70b-online');
-console.log(txtPPLX);
-/*
-As ALF from Melmac, I can assure you that I have not personally eaten a cat. Cats are not consumed in my homeworld, as our diet is not Earth-based. On Earth, however, there are instances where cat meat has been consumed in various cultures, historically and in times of necessity. The search results mention that cat meat is considered food in some countries, such as parts of Asia, and there have been reports of it being consumed during famine periods in places like Italy. It's important to note that the consumption of cat meat is often frowned upon and has faced opposition, particularly in regions where cats are popular as pets.
-*/
+const claude = await driver.create('claude-3-sonnet-20240229', { temperature: 0.5 });
+await claude.addImage("./watson.png")
+const imageDescription = await claude.addText("describe the image").message();
+console.log(imageDescription);
+
+const gpt = await driver.create('gpt-4o', { temperature: 0.5 });
+const question = await gpt.addText("Have you ever eaten a cat?").message();
+console.log(question);
+
+const pplx = await driver.create('pplx-70b-online', { max_tokens: 500 });
+await pplx.addText('How much is ETH trading in USD?');
+const news = await pplx.addText('What are the 3 most recent Ethereum news?').message();
+console.log(news);

package/index.js

@@ -1,16 +1,23 @@
 const axios = require('axios');
+const fs = require('fs').promises;
+const mime = require('mime-types');
 
 class ModelMix {
-    constructor(options = {}) {
+    constructor(args = { options: {}, config: {} }) {
         this.models = {};
         this.defaultOptions = {
             model: 'gpt-4o',
             max_tokens: 2000,
             temperature: 1,
-            system: 'You are an assistant.',
             top_p: 1,
-            ...options
+            ...args.options
         };
+
+        this.config = {
+            system: 'You are an assistant.',
+            max_history: 5, // Default max history
+            ...args.config
+        }
     }
 
     attach(modelInstance) {
@@ -20,7 +27,7 @@ class ModelMix {
         modelInstance.active_requests = 0;
     }
 
-    async create(prompt, modelKey, options = {}) {
+    async create(modelKey, overOptions = {}) {
         const modelEntry = Object.values(this.models).find(entry =>
             entry.config.prefix.some(p => modelKey.startsWith(p))
         );
@@ -29,12 +36,10 @@ class ModelMix {
             throw new Error(`Model with prefix matching ${modelKey} is not attached.`);
         }
 
-        const config = { ...this.defaultOptions, ...modelEntry.options, ...options, model: modelKey };
+        const options = { ...this.defaultOptions, ...modelEntry.options, ...overOptions, model: modelKey };
+        const config = { ...this.config, ...modelEntry.config };
 
-        return new Promise((resolve, reject) => {
-            modelEntry.queue.push({ prompt, config, resolve, reject });
-            this.processQueue(modelEntry);
-        });
+        return new MessageHandler(this, modelEntry, options, config);
     }
 
     async processQueue(modelEntry) {
@@ -50,7 +55,7 @@ class ModelMix {
         modelEntry.active_requests++;
 
         try {
-            const result = await modelEntry.create(nextTask.prompt, nextTask.config);
+            const result = await modelEntry.create(nextTask.args);
             nextTask.resolve(result);
         } catch (error) {
             nextTask.reject(error);
@@ -61,6 +66,110 @@ class ModelMix {
     }
 }
 
+class MessageHandler {
+    constructor(mix, modelEntry, options, config) {
+        this.mix = mix;
+        this.modelEntry = modelEntry;
+        this.options = options;
+        this.config = config;
+        this.messages = [];
+    }
+
+    new() {
+        this.messages = [];
+        return this;
+    }
+
+    addText(text, config = { role: "user" }) {
+        const content = [{
+            type: "text",
+            text
+        }];
+
+        this.messages.push({ ...config, content });
+        return this;
+    }
+
+    async addImage(filePath, config = { role: "user" }) {
+        try {
+            const imageBuffer = await fs.readFile(filePath);
+            const mimeType = mime.lookup(filePath);
+
+            if (!mimeType || !mimeType.startsWith('image/')) {
+                throw new Error('Invalid image file type');
+            }
+
+            const data = imageBuffer.toString('base64');
+
+            const imageMessage = {
+                ...config,
+                content: [
+                    {
+                        type: "image",
+                        "source": {
+                            type: "base64",
+                            media_type: mimeType,
+                            data
+                        }
+                    }
+                ]
+            };
+
+            this.messages.push(imageMessage);
+        } catch (error) {
+            console.error('Error reading the image file:', error);
+        }
+
+        return this;
+    }
+
+    async message() {
+        const response = await this.execute();
+        return response.message;
+    }
+
+    async raw() {
+        const data = await this.execute();
+        return data.response;
+    }
+
+    groupByRoles(messages) {
+        return messages.reduce((acc, message) => {
+            const existingRole = acc.find(item => item.role === message.role);
+            if (existingRole) {
+                existingRole.content = existingRole.content.concat(message.content);
+            } else {
+                acc.push({ role: message.role, content: Array.isArray(message.content) ? message.content : [message.content] });
+            }
+            return acc;
+        }, []);
+    }
+
+    async execute() {
+
+        this.messages = this.groupByRoles(this.messages);
+
+        if (this.messages.length === 0) { // Only system message is present
+            throw new Error("No user messages have been added. Use addMessage(prompt) to add a message.");
+        }
+
+        this.messages = this.messages.slice(-this.config.max_history);
+        this.options.messages = this.messages;
+
+        return new Promise((resolve, reject) => {
+            this.modelEntry.queue.push({
+                args: { options: this.options, config: this.config },
+                resolve: (result) => {
+                    this.messages.push({ role: "assistant", content: result.message });
+                    resolve(result);
+                },
+                reject
+            });
+            this.mix.processQueue(this.modelEntry);
+        });
+    }
+}
+
 class OpenAIModel {
     constructor(openai, args = { options: {}, config: {} }) {
         this.openai = openai;
@@ -79,17 +188,32 @@ class OpenAIModel {
         }
     }
 
-    async create(prompt, options = {}) {
-        options.messages = [
-            { role: "system", content: options.system },
-            { role: "user", content: prompt }
-        ];
-
-        delete options.system;
+    async create(args = { options: {}, config: {} }) {
 
-        const response = await this.openai.chat.completions.create(options);
+        args.options.messages = [{ role: 'system', content: args.config.system }, ...args.options.messages || []];
+        args.options.messages = this.convertMessages(args.options.messages);
+        const response = await this.openai.chat.completions.create(args.options);
+        return { response, message: response.choices[0].message.content };
+    }
 
-        return response.choices[0].message.content;
+    convertMessages(messages) {
+        return messages.map(message => {
+            if (message.role === 'user' && message.content instanceof Array) {
+                message.content = message.content.map(content => {
+                    if (content.type === 'image') {
+                        const { type, media_type, data } = content.source;
+                        return {
+                            type: 'image_url',
+                            image_url: {
+                                url: `data:${media_type};${type},${data}`
+                            }
+                        };
+                    }
+                    return content;
+                });
+            }
+            return message;
+        });
     }
 }
 
@@ -108,19 +232,14 @@ class AnthropicModel {
         }
     }
 
-    async create(prompt, options = {}) {
-        options.messages = [
-            { role: "user", content: prompt }
-        ];
+    async create(args = { config: {}, options: {} }) {
 
-        const response = await this.anthropic.messages.create(options);
+        args.options.system = args.config.system;
 
+        const response = await this.anthropic.messages.create(args.options);
         const responseText = response.content[0].text;
 
-        // Add a short delay to avoid hitting rate limits
-        await new Promise(resolve => setTimeout(resolve, 1000));
-
-        return responseText.trim();
+        return { response, message: responseText.trim() };
     }
 }
 
@@ -144,19 +263,10 @@ class CustomModel {
         };
     }
 
-    async create(prompt, options = {}) {
-        const mergedOptions = {
-            ...this.options,
-            ...options,
-            messages: [
-                { role: "system", content: options.system || "" },
-                { role: "user", content: prompt }
-            ]
-        };
-
-        delete mergedOptions.system;
-
-        const response = await axios.post(this.config.url, mergedOptions, {
+    async create(args = { config: {}, options: {} }) {
+        args.options.messages = [{ role: 'system', content: args.config.system }, ...args.options.messages || []];
+        
+        const response = await axios.post(this.config.url, args.options, {
             headers: {
                 'accept': 'application/json',
                 'authorization': `Bearer ${this.config.bearer}`,
@@ -164,8 +274,8 @@ class CustomModel {
             }
         });
 
-        return response.data.choices[0].message.content;
+        return { response: response.data, message: response.data.choices[0].message.content };
     }
 }
 
-module.exports = { OpenAIModel, AnthropicModel, CustomModel, ModelMix };
+module.exports = { OpenAIModel, AnthropicModel, CustomModel, ModelMix };

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "modelmix",
-  "version": "1.0.2",
-  "description": "🧬 ModelMix - Integrate AI Models with Ease.",
+  "version": "1.1.2",
+  "description": "🧬 ModelMix - Unified API for Diverse AI Language Models.",
   "main": "index.js",
   "repository": {
     "type": "git",