@elizaos/plugin-local-ai 1.0.0-beta.8 → 1.0.1

package/README.md

@@ -7,82 +7,39 @@ This plugin provides local AI model capabilities through the ElizaOS platform, s
 Add the plugin to your character configuration:
 
 ```json
-"plugins": ["@elizaos/plugin-local-ai"]
+"plugins": ["@elizaos-plugins/plugin-local-ai"]
 ```
 
 ## Configuration
 
-The plugin requires these environment variables (can be set in .env file or character settings):
-
-```json
-"settings": {
-  "USE_LOCAL_AI": true,
-  "USE_STUDIOLM_TEXT_MODELS": false,
-  "USE_OLLAMA_TEXT_MODELS": false,
-
-  "OLLAMA_API_ENDPOINT": "http://localhost:11434",
-  "OLLAMA_MODEL": "deepseek-r1-distill-qwen-7b",
-  "USE_OLLAMA_EMBEDDING": false,
-  "OLLAMA_EMBEDDING_MODEL": "",
-  "OLLAMA_SMALL_MODEL": "deepseek-r1:1.5b",
-  "OLLAMA_MEDIUM_MODEL": "deepseek-r1:7b",
-  "OLLAMA_LARGE_MODEL": "deepseek-r1:7b",
-
-  "STUDIOLM_SERVER_URL": "http://localhost:1234",
-  "STUDIOLM_SMALL_MODEL": "lmstudio-community/deepseek-r1-distill-qwen-1.5b",
-  "STUDIOLM_MEDIUM_MODEL": "deepseek-r1-distill-qwen-7b",
-  "STUDIOLM_EMBEDDING_MODEL": false
-}
-```
+The plugin is configured using environment variables (typically set in a `.env` file or via your deployment settings):
 
 Or in `.env` file:
 
 ```env
-# Local AI Configuration
-USE_LOCAL_AI=true
-USE_STUDIOLM_TEXT_MODELS=false
-USE_OLLAMA_TEXT_MODELS=false
-
-# Ollama Configuration
-OLLAMA_API_ENDPOINT=http://localhost:11434
-OLLAMA_MODEL=deepseek-r1-distill-qwen-7b
-USE_OLLAMA_EMBEDDING=false
-OLLAMA_EMBEDDING_MODEL=
-OLLAMA_SMALL_MODEL=deepseek-r1:1.5b
-OLLAMA_MEDIUM_MODEL=deepseek-r1:7b
-OLLAMA_LARGE_MODEL=deepseek-r1:7b
-
-# StudioLM Configuration
-STUDIOLM_SERVER_URL=http://localhost:1234
-STUDIOLM_SMALL_MODEL=lmstudio-community/deepseek-r1-distill-qwen-1.5b
-STUDIOLM_MEDIUM_MODEL=deepseek-r1-distill-qwen-7b
-STUDIOLM_EMBEDDING_MODEL=false
-```
-
-### Configuration Options
+# Optional: Specify a custom directory for models (GGUF files)
+# MODELS_DIR=/path/to/your/models
 
-#### Text Model Source (Choose One)
+# Optional: Specify a custom directory for caching other components (tokenizers, etc.)
+# CACHE_DIR=/path/to/your/cache
 
-- `USE_STUDIOLM_TEXT_MODELS`: Enable StudioLM text models
-- `USE_OLLAMA_TEXT_MODELS`: Enable Ollama text models
-  Note: Only one text model source can be enabled at a time
+# Optional: Specify filenames for the text generation and embedding models within the models directory
+# LOCAL_SMALL_MODEL=my-custom-small-model.gguf
+# LOCAL_LARGE_MODEL=my-custom-large-model.gguf
+# LOCAL_EMBEDDING_MODEL=my-custom-embedding-model.gguf
 
-#### Ollama Settings
-
-- `OLLAMA_API_ENDPOINT`: Ollama API endpoint (default: http://localhost:11434)
-- `OLLAMA_MODEL`: Default model for general use
-- `USE_OLLAMA_EMBEDDING`: Enable Ollama for embeddings
-- `OLLAMA_EMBEDDING_MODEL`: Model for embeddings when enabled
-- `OLLAMA_SMALL_MODEL`: Model for lighter tasks
-- `OLLAMA_MEDIUM_MODEL`: Model for standard tasks
-- `OLLAMA_LARGE_MODEL`: Model for complex tasks
+# Optional: Fallback dimension size for embeddings if generation fails. Defaults to the model's default (e.g., 384).
+# LOCAL_EMBEDDING_DIMENSIONS=384
+```
 
-#### StudioLM Settings
+### Configuration Options
 
-- `STUDIOLM_SERVER_URL`: StudioLM API endpoint (default: http://localhost:1234)
-- `STUDIOLM_SMALL_MODEL`: Model for lighter tasks
-- `STUDIOLM_MEDIUM_MODEL`: Model for standard tasks
-- `STUDIOLM_EMBEDDING_MODEL`: Model for embeddings (or false to disable)
+- `MODELS_DIR` (Optional): Specifies a custom directory for storing model files (GGUF format). If not set, defaults to `~/.eliza/models`.
+- `CACHE_DIR` (Optional): Specifies a custom directory for caching other components like tokenizers. If not set, defaults to `~/.eliza/cache`.
+- `LOCAL_SMALL_MODEL` (Optional): Specifies the filename for the small text generation model (e.g., `DeepHermes-3-Llama-3-3B-Preview-q4.gguf`) located in the models directory.
+- `LOCAL_LARGE_MODEL` (Optional): Specifies the filename for the large text generation model (e.g., `DeepHermes-3-Llama-3-8B-q4.gguf`) located in the models directory.
+- `LOCAL_EMBEDDING_MODEL` (Optional): Specifies the filename for the text embedding model (e.g., `bge-small-en-v1.5.Q4_K_M.gguf`) located in the models directory.
+- `LOCAL_EMBEDDING_DIMENSIONS` (Optional): Defines the expected dimension size for text embeddings. This is primarily used as a fallback dimension if the embedding model fails to generate an embedding. If not set, it defaults to the embedding model's native dimension size (e.g., 384 for `bge-small-en-v1.5.Q4_K_M.gguf`).
 
 ## Features
 
@@ -90,10 +47,35 @@ The plugin provides these model classes:
 
 - `TEXT_SMALL`: Fast, efficient text generation using smaller models
 - `TEXT_LARGE`: More capable text generation using larger models
+- `TEXT_EMBEDDING`: Generates text embeddings locally.
 - `IMAGE_DESCRIPTION`: Local image analysis using Florence-2 vision model
 - `TEXT_TO_SPEECH`: Local text-to-speech synthesis
 - `TRANSCRIPTION`: Local audio transcription using Whisper
 
+### Text Generation
+
+```typescript
+// Using small model
+const smallResponse = await runtime.useModel(ModelType.TEXT_SMALL, {
+  prompt: 'Generate a short response',
+  stopSequences: [],
+});
+
+// Using large model
+const largeResponse = await runtime.useModel(ModelType.TEXT_LARGE, {
+  prompt: 'Generate a detailed response',
+  stopSequences: [],
+});
+```
+
+### Text Embedding
+
+```typescript
+const embedding = await runtime.useModel(ModelType.TEXT_EMBEDDING, {
+  text: 'Text to get embedding for',
+});
+```
+
 ### Image Analysis
 
 ```typescript
@@ -114,39 +96,3 @@ const audioStream = await runtime.useModel(ModelType.TEXT_TO_SPEECH, 'Text to co
 ```typescript
 const transcription = await runtime.useModel(ModelType.TRANSCRIPTION, audioBuffer);
 ```
-
-### Text Generation
-
-```typescript
-// Using small model
-const smallResponse = await runtime.useModel(ModelType.TEXT_SMALL, {
-  context: 'Generate a short response',
-  stopSequences: [],
-});
-
-// Using large model
-const largeResponse = await runtime.useModel(ModelType.TEXT_LARGE, {
-  context: 'Generate a detailed response',
-  stopSequences: [],
-});
-```
-
-## Model Sources
-
-### 1. StudioLM (LM Studio)
-
-- Local inference server for running various open models
-- Supports chat completion API similar to OpenAI
-- Configure with `USE_STUDIOLM_TEXT_MODELS=true`
-- Supports both small and medium-sized models
-- Optional embedding model support
-
-### 2. Ollama
-
-- Local model server with optimized inference
-- Supports various open models in GGUF format
-- Configure with `USE_OLLAMA_TEXT_MODELS=true`
-- Supports small, medium, and large models
-- Optional embedding model support
-
-Note: The plugin validates that only one text model source is enabled at a time to prevent conflicts.

package/dist/index.d.ts

@@ -0,0 +1,9 @@
+import { Plugin } from '@elizaos/core';
+
+/**
+ * Plugin that provides functionality for local AI using LLaMA models.
+ * @type {Plugin}
+ */
+declare const localAiPlugin: Plugin;
+
+export { localAiPlugin as default, localAiPlugin };