modelmix 1.2.0 → 2.0.0

package/README.md

@@ -1,13 +1,13 @@
 # 🧬 ModelMix: Unified API for Diverse AI Language Models
 
-**ModelMix** is a versatile module that enables seamless integration of various language models from different providers through a unified interface. With ModelMix, you can effortlessly manage and utilize multiple AI models while controlling parallel requests to avoid provider restrictions. 
+**ModelMix** is a versatile module that enables seamless integration of various language models from different providers through a unified interface. With ModelMix, you can effortlessly manage and utilize multiple AI models while controlling parallel requests to avoid provider restrictions.
 
-## ✨ Features 
+## ✨ Features
 
 - **Unified Interface**: Interact with multiple AI models through a single, coherent API.
-- **Request Control**: Manage the number of parallel requests to adhere to provider limitations.
-- **Flexible Integration**: Easily integrate popular models like OpenAI and Anthropic, as well as custom models such as Perplexity.
-- **History Tracking:** Automatically logs the conversation history with model responses, allowing you to limit the number of historical messages with max_history.
+- **Request Control**: Manage the number of parallel requests to adhere to provider limitations (`max_request`).
+- **Flexible Integration**: Easily integrate popular models like OpenAI, Anthropic, Perplexity, Ollama, or custom models.
+- **History Tracking**: Automatically logs the conversation history with model responses, allowing you to limit the number of historical messages with `max_history`.
 
 ## 📦 Installation
 
@@ -17,10 +17,10 @@ First, install the ModelMix package:
 npm install modelmix
 ```
 
-You'll also need to install the respective SDKs for each model provider you plan to use:
+Also, install dotenv to manage environment variables:
 
 ```bash
-npm install openai @anthropic-ai/sdk dotenv
+npm install dotenv
 ```
 
 ## 🛠️ Usage
@@ -37,80 +37,94 @@ Here's a quick example to get you started:
 2. **Create and configure your models**:
 
     ```javascript
-    import 'dotenv/config'
-    import OpenAI from 'openai';
-    import Anthropic from '@anthropic-ai/sdk';
-
-    import { ModelMix, OpenAIModel, AnthropicModel, CustomModel } from 'modelmix';
+    import 'dotenv/config';
+    import { ModelMix, MixOpenAI, MixAnthropic, MixPerplexity, MixOllama } from 'modelmix';
 
     const env = process.env;
 
-    const driver = new ModelMix({
+    const mmix = new ModelMix({
         options: {
             max_tokens: 200,
         },
         config: {
             system: "You are ALF from Melmac.",
-            max_history: 2
+            max_history: 2,
+            max_request: 1,
         }
     });
 
-    driver.attach(new OpenAIModel(new OpenAI({ apiKey: env.OPENAI_API_KEY })));
-    driver.attach(new AnthropicModel(new Anthropic({ apiKey: env.ANTHROPIC_API_KEY })));
-    driver.attach(new CustomModel({
+    mmix.attach(new MixOpenAI({ config: { apiKey: env.OPENAI_API_KEY } }));
+    mmix.attach(new MixAnthropic({ config: { apiKey: env.ANTHROPIC_API_KEY } }));
+    mmix.attach(new MixPerplexity({
         config: {
-            url: 'https://api.perplexity.ai/chat/completions',
-            bearer: env.PPLX_API_KEY,
-            prefix: ["pplx", "llama", "mixtral"],
+            apiKey: env.PPLX_API_KEY
+        },
+        options: {
             system: "You are my personal assistant."
         }
     }));
+    mmix.attach(new MixOllama({
+        config: {
+            url: 'http://localhost:11434/api/chat',
+            prefix: ['llava'],
+        },
+        options: {
+            temperature: 0.5,
+        }
+    }));
     ```
 
 3. **Generate responses from different models**:
 
     ```javascript
-    const claude = await driver.create('claude-3-sonnet-20240229', { temperature: 0.5 });
-    await claude.addImage("./watson.png")
-    const imageDescription = await claude.addText("describe the image").message();
+    console.log("\n" + '--------| gpt-4o |--------');
+    const gpt = mmix.create('gpt-4o', { temperature: 0.5 }).addText("Have you ever eaten a cat?");
+    console.log(await gpt.message());
+
+    console.log("\n" + '--------| claude-3-sonnet-20240229 |--------');
+    const claude = mmix.create('claude-3-sonnet-20240229', { temperature: 0.5 });
+    claude.addImage("./watson.png");
+    const imageDescription = await claude.addText("Describe the image").message();
     console.log(imageDescription);
 
-    const gpt = await driver.create('gpt-4o', { temperature: 0.5 });
-    const question = await gpt.addText("Have you ever eaten a cat?").message();
-    console.log(question);
-
-    const pplx = await driver.create('pplx-70b-online', { max_tokens: 500 });
-    await pplx.addText('How much is ETH trading in USD?');
+    console.log("\n" + '--------| pplx-70b-online |--------');
+    const pplx = mmix.create('pplx-70b-online', { max_tokens: 500 });
+    pplx.addText('How much is ETH trading in USD?');
     const news = await pplx.addText('What are the 3 most recent Ethereum news?').message();
     console.log(news);
-    ```
 
+    console.log("\n" + '--------| ollama (llava:latest) |--------');
+    await mmix.create('llava:latest')
+        .addImage("./watson.png")
+        .addText("What is the predominant color?")
+        .stream((data) => { console.log(data.message); });
+    ```
 4. Find the files for this example at: [/ModelMix/demo](https://github.com/clasen/ModelMix/tree/master/demo).
-
-
+   
 ## 📚 ModelMix Class Overview
 
 ```javascript
 new ModelMix(args = { options: {}, config: {} })
 ```
+
 - **args**: Configuration object with `options` and `config` properties.
   - **options**: This object contains default options that are applied to all models. These options can be overridden when creating a specific model instance. Examples of default options include:
     - `max_tokens`: Sets the maximum number of tokens to generate, e.g., 2000.
     - `temperature`: Controls the randomness of the model's output, e.g., 1.
     - `top_p`: Controls the diversity of the output, e.g., 1.
-    - ...
+    - ...(Additional default options can be added as needed)
   - **config**: This object contains configuration settings that control the behavior of the `ModelMix` instance. These settings can also be overridden for specific model instances. Examples of configuration settings include:
     - `system`: Sets the default system message for the model, e.g., "You are an assistant."
     - `max_history`: Limits the number of historical messages to retain, e.g., 5.
-    - `max_request`: Limits the number of parallel request.
-    - ...
+    - `max_request`: Limits the number of parallel requests, e.g., 1.
+    - ...(Additional configuration parameters can be added as needed)
 
 **Methods**
 
 - `attach(modelInstance)`: Attaches a model instance to the `ModelMix`.
 - `create(modelKey, overOptions = {})`: Creates a new `MessageHandler` for the specified model.
 
-#### MessageHandler
+### MessageHandler Class Overview
 
 **Methods**
 
@@ -119,42 +133,69 @@ new ModelMix(args = { options: {}, config: {} })
 - `addImage(filePath, config = { role: "user" })`: Adds an image message from a file path.
 - `message()`: Sends the message and returns the response.
 - `raw()`: Sends the message and returns the raw response data.
+- `stream(callback)`: Sends the message and streams the response, invoking the callback with each streamed part.
 
-#### OpenAIModel
+### MixCustom Class Overview
 
 ```javascript
-new OpenAIModel(openai, args = { options: {}, config: {} })
+new MixCustom(args = { config: {}, options: {}, headers: {} })
 ```
 
-- **openai**: Instance of the OpenAI client.
-- **args**: Configuration object with `options` and `config` properties.
+- **args**: Configuration object with `config`, `options`, and `headers` properties.
+  - **config**:
+    - `url`: The endpoint URL to which the model sends requests.
+    - `prefix`: An array of strings used as a prefix for requests.
+    - ...(Additional configuration parameters can be added as needed)
+  - **options**: This object contains default options that are applied to all models. These options can be overridden when creating a specific model instance. Examples of default options include:
+    - `max_tokens`: Sets the maximum number of tokens to generate, e.g., 2000.
+    - `temperature`: Controls the randomness of the model's output, e.g., 1.
+    - `top_p`: Controls the diversity of the output, e.g., 1.
+    - ...(Additional default options can be added as needed)
+  - **headers**:
+    - `authorization`: The authorization header, typically including a Bearer token for API access.
+    - `x-api-key`: A custom header for API key if needed.
+    - ...(Additional headers can be added as needed)
 
-#### AnthropicModel
+### MixOpenAI Class Overview
 
 ```javascript
-new AnthropicModel(anthropic, args = { options: {}, config: {} })
+new MixOpenAI(args = { config: {}, options: {} })
 ```
 
-- **anthropic**: Instance of the Anthropic client.
-- **args**: Configuration object with `options` and `config` properties.
+- **args**: Configuration object with `config` and `options` properties.
+  - **config**: Specific configuration settings for OpenAI, including the `apiKey`.
+  - **options**: Default options for OpenAI model instances.
 
-#### CustomModel
+### MixAnthropic Class Overview
 
 ```javascript
-new CustomModel(args = { config: {}, options: {} })
+new MixAnthropic(args = { config: {}, options: {} })
 ```
 
 - **args**: Configuration object with `config` and `options` properties.
-  - **config**: 
-    - `url`: 
-    - `bearer`: 
-    - `prefix`: 
-    - ...
-  - **options**: This object contains default options that are applied to all models. These options can be overridden when creating a specific model instance. Examples of default options include:
-    - `max_tokens`: Sets the maximum number of tokens to generate, e.g., 2000.
-    - `temperature`: Controls the randomness of the model's output, e.g., 1.
-    - `top_p`: Controls the diversity of the output, e.g., 1.
-    - ...
+  - **config**: Specific configuration settings for Anthropic, including the `apiKey`.
+  - **options**: Default options for Anthropic model instances.
+
+### MixPerplexity Class Overview
+
+```javascript
+new MixPerplexity(args = { config: {}, options: {} })
+```
+
+- **args**: Configuration object with `config` and `options` properties.
+  - **config**: Specific configuration settings for Perplexity, including the `apiKey`.
+  - **options**: Default options for Perplexity model instances.
+
+### MixOllama Class Overview
+
+```javascript
+new MixOllama(args = { config: {}, options: {} })
+```
+
+- **args**: Configuration object with `config` and `options` properties.
+  - **config**: Specific configuration settings for Ollama.
+    - `url`: The endpoint URL to which the model sends requests.
+  - **options**: Default options for Ollama model instances.
 
 ## 🤝 Contributing
 

package/demo/custom.mjs

@@ -0,0 +1,28 @@
+import 'dotenv/config'
+
+import { ModelMix, MixCustom } from '../index.js';
+
+const env = process.env;
+
+const mmix = new ModelMix({
+    options: {
+        max_tokens: 200,
+    },
+    config: {
+        system: 'You are ALF from Melmac.',
+        max_history: 2
+    }
+});
+
+mmix.attach(new MixCustom({
+    config: {
+        url: 'https://api.perplexity.ai/chat/completions',
+        prefix: ["pplx", "llama", "mixtral"],
+    },
+    headers: {
+        'authorization': `Bearer ${env.PPLX_API_KEY}`
+    }
+}));
+
+const r = await mmix.create('pplx-70b-online').addText('do you like cats?').message()
+console.log(r)

package/demo/demo.mjs

@@ -1,9 +1,5 @@
-
 import 'dotenv/config'
-import OpenAI from 'openai';
-import Anthropic from '@anthropic-ai/sdk';
-
-import { ModelMix, OpenAIModel, AnthropicModel, CustomModel } from '../index.js';
+import { ModelMix, MixOpenAI, MixAnthropic, MixPerplexity, MixOllama } from '../index.js';
 
 const env = process.env;
 
@@ -12,35 +8,50 @@ const mmix = new ModelMix({
         max_tokens: 200,
     },
     config: {
-        system: "You are ALF from Melmac.",
-        max_history: 2
+        system: 'You are ALF from Melmac.',
+        max_history: 2,
+        max_request: 1,
     }
 });
 
-mmix.attach(new OpenAIModel(new OpenAI({ apiKey: env.OPENAI_API_KEY })));
-mmix.attach(new AnthropicModel(new Anthropic({ apiKey: env.ANTHROPIC_API_KEY })));
-mmix.attach(new CustomModel({
+
+mmix.attach(new MixOpenAI({ config: { apiKey: env.OPENAI_API_KEY } }));
+mmix.attach(new MixAnthropic({ config: { apiKey: env.ANTHROPIC_API_KEY } }));
+mmix.attach(new MixPerplexity({
+    config: {
+        apiKey: env.PPLX_API_KEY
+    },
+    system: 'You are my personal assistant.'
+}));
+mmix.attach(new MixOllama({
     config: {
-        url: 'https://api.perplexity.ai/chat/completions',
-        bearer: env.PPLX_API_KEY,
-        prefix: ["pplx", "llama", "mixtral"],
-        system: "You are my personal assistant."
+        url: 'http://localhost:11434/api/chat',
+        prefix: ['llava'],
+    },
+    options: {
+        temperature: 0.5,
     }
 }));
 
+
 console.log("\n" + '--------| gpt-4o |--------');
-const gpt = await mmix.create('gpt-4o', { temperature: 0.5 });
-const question = await gpt.addText("Have you ever eaten a cat?").message();
-console.log(question);
+const gpt = mmix.create('gpt-4o', { temperature: 0.5 }).addText("Have you ever eaten a cat?");
+console.log(await gpt.message());
 
 console.log("\n" + '--------| claude-3-sonnet-20240229 |--------');
-const claude = await mmix.create('claude-3-sonnet-20240229', { temperature: 0.5 });
-await claude.addImage("./watson.png")
-const imageDescription = await claude.addText("describe the image").message();
+const claude = mmix.create('claude-3-sonnet-20240229', { temperature: 0.5 });
+claude.addImage('./watson.png');
+const imageDescription = await claude.addText('describe the image').message();
 console.log(imageDescription);
 
 console.log("\n" + '--------| pplx-70b-online |--------');
-const pplx = await mmix.create('pplx-70b-online', { max_tokens: 500 });
-await pplx.addText('How much is ETH trading in USD?');
+const pplx = mmix.create('pplx-70b-online', { max_tokens: 500 });
+pplx.addText('How much is ETH trading in USD?');
 const news = await pplx.addText('What are the 3 most recent Ethereum news?').message();
 console.log(news);
+
+console.log("\n" + '--------| ollama (llava:latest) |--------');
+await mmix.create('llava:latest')
+    .addImage('./watson.png')
+    .addText('what is the predominant color?')
+    .stream((data) => { console.log(data.message); });

package/demo/node_modules/.package-lock.json

@@ -84,6 +84,27 @@
         "node": ">= 0.8"
       }
     },
+    "node_modules/debug": {
+      "version": "4.3.4",
+      "resolved": "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz",
+      "integrity": "sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ==",
+      "dependencies": {
+        "ms": "2.1.2"
+      },
+      "engines": {
+        "node": ">=6.0"
+      },
+      "peerDependenciesMeta": {
+        "supports-color": {
+          "optional": true
+        }
+      }
+    },
+    "node_modules/debug/node_modules/ms": {
+      "version": "2.1.2",
+      "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz",
+      "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w=="
+    },
     "node_modules/delayed-stream": {
       "version": "1.0.0",
       "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
@@ -157,6 +178,14 @@
         "ms": "^2.0.0"
       }
     },
+    "node_modules/lemonlog": {
+      "version": "1.1.2",
+      "resolved": "https://registry.npmjs.org/lemonlog/-/lemonlog-1.1.2.tgz",
+      "integrity": "sha512-ztAM2mVrr9+CLoKvXKCJeGb6jNNDZUnATlcIouFuj7ymyNKXXZhx4gXyQ5lsW6oLx5KXKb/vLclXVdcqJ5sHVA==",
+      "dependencies": {
+        "debug": "^4.3.4"
+      }
+    },
     "node_modules/mime-db": {
       "version": "1.52.0",
       "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",

package/demo/node_modules/debug/LICENSE

@@ -0,0 +1,20 @@
+(The MIT License)
+
+Copyright (c) 2014-2017 TJ Holowaychuk <tj@vision-media.ca>
+Copyright (c) 2018-2021 Josh Junon
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software
+and associated documentation files (the 'Software'), to deal in the Software without restriction,
+including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
+and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so,
+subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial
+portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT
+LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+