@easbot/local-model-sdk 0.1.3

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 houjallen
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.en.md

@@ -0,0 +1,282 @@
+# Local Model SDK
+
+Local model SDK based on Transformers.js + ONNX, supporting Language Model (text generation) and Embedding (text embedding) capabilities, fully compatible with AI SDK specifications.
+
+## Features
+
+- ✅ **Language Model Support**: Text generation powered by Transformers.js
+- ✅ **Embedding Support**: High-quality text embedding vector generation
+- ✅ **Streaming Generation**: Real-time streaming text output
+- ✅ **Batch Processing**: Efficient batch embedding processing
+- ✅ **Model Caching**: Automatic model file cache management
+- ✅ **AI SDK Compatible**: Fully compatible with AI SDK v2 specification
+- ✅ **TypeScript**: Complete type definition support
+- ✅ **Offline Operation**: No network required, fully local
+
+## Installation
+
+```bash
+pnpm add @easbot/local-model-sdk
+```
+
+## Quick Start
+
+### Language Model - Text Generation
+
+```typescript
+import { LocalModelProvider } from '@easbot/local-model-sdk';
+import { generateText } from 'ai';
+
+// Create Provider
+const provider = new LocalModelProvider({
+  cacheDir: '.cache/models', // Optional, defaults to ~/.cache/easbot/models
+});
+
+// Create Language Model
+const model = provider.languageModel('qwen2.5-0.5b-instruct');
+
+// Generate text
+const result = await generateText({
+  model,
+  prompt: 'What is the capital of France?',
+});
+
+console.log(result.text);
+```
+
+### Language Model - Streaming
+
+```typescript
+import { streamText } from 'ai';
+
+const result = streamText({
+  model,
+  prompt: 'Write a short story about a robot.',
+});
+
+// Output chunk by chunk
+for await (const chunk of result.textStream) {
+  process.stdout.write(chunk);
+}
+```
+
+### Embedding - Text Embedding
+
+```typescript
+import { embed, embedMany } from 'ai';
+
+// Single text embedding
+const result = await embed({
+  model: provider.textEmbeddingModel('bge-small-zh-v1.5'),
+  value: 'Hello world',
+});
+
+console.log(result.embedding); // number[]
+
+// Batch text embedding
+const batchResult = await embedMany({
+  model: provider.textEmbeddingModel('bge-small-zh-v1.5'),
+  values: ['Text 1', 'Text 2', 'Text 3'],
+});
+
+console.log(batchResult.embeddings); // number[][]
+```
+
+## Supported Models
+
+### Language Models
+
+| Model ID | Description | Parameters |
+|----------|-------------|------------|
+| `qwen3-0.6b` | Qwen3 0.6B Base | 0.6B |
+| `qwen2.5-0.5b-instruct` | Qwen2.5 0.5B Instruct | 0.5B |
+| `qwen2.5-1.5b-instruct` | Qwen2.5 1.5B Instruct | 1.5B |
+| `qwen2.5-3b-instruct` | Qwen2.5 3B Instruct | 3B |
+
+### Embedding Models
+
+| Model ID | Description | Dimensions |
+|----------|-------------|------------|
+| `bge-small-zh-v1.5` | BGE Small Chinese | 512 |
+| `bge-base-zh-v1.5` | BGE Base Chinese | 768 |
+| `bge-large-zh-v1.5` | BGE Large Chinese | 1024 |
+
+## API Documentation
+
+### LocalModelProvider
+
+Provider class for creating and managing model instances.
+
+```typescript
+const provider = new LocalModelProvider({
+  cacheDir: '.cache/models', // Optional
+});
+```
+
+#### Methods
+
+- `languageModel(modelId: string): LocalLanguageModel` - Create Language Model instance
+- `textEmbeddingModel(modelId: string): LocalEmbeddingModel` - Create Embedding Model instance
+- `getLoadedModels(): ModelInfo[]` - Get list of loaded models
+- `dispose(): Promise<void>` - Unload all models and release resources
+
+### LocalLanguageModel
+
+Implements AI SDK's `LanguageModelV2` interface.
+
+```typescript
+const model = provider.languageModel('qwen2.5-0.5b-instruct');
+
+// Non-streaming generation
+const result = await generateText({
+  model,
+  prompt: 'Your prompt here',
+  temperature: 0.7,
+  maxTokens: 100,
+});
+
+// Streaming generation
+const stream = streamText({
+  model,
+  prompt: 'Your prompt here',
+});
+```
+
+### LocalEmbeddingModel
+
+Implements AI SDK's `EmbeddingModelV2` interface.
+
+```typescript
+const model = provider.textEmbeddingModel('bge-small-zh-v1.5');
+
+// Single text
+const result = await embed({
+  model,
+  value: 'Your text here',
+});
+
+// Batch texts
+const batchResult = await embedMany({
+  model,
+  values: ['Text 1', 'Text 2', 'Text 3'],
+});
+```
+
+## Configuration Options
+
+### Provider Configuration
+
+```typescript
+interface LocalModelProviderConfig {
+  /** Model cache directory, defaults to ~/.cache/easbot/models */
+  cacheDir?: string;
+}
+```
+
+### Generation Options
+
+```typescript
+// Passed through AI SDK's generateText/streamText
+{
+  temperature?: number;  // 0.0 - 2.0, default 0.7
+  maxTokens?: number;    // Maximum tokens to generate
+  topP?: number;         // 0.0 - 1.0, default 0.9
+  topK?: number;         // Default 50
+  abortSignal?: AbortSignal; // Cancellation signal
+}
+```
+
+## Error Handling
+
+SDK provides detailed error types:
+
+```typescript
+import { LocalModelError, ErrorCode } from '@easbot/local-model-sdk';
+
+try {
+  const result = await generateText({ model, prompt: 'Test' });
+} catch (error) {
+  if (error instanceof LocalModelError) {
+    console.error(`Error [${error.code}]: ${error.message}`);
+  }
+}
+```
+
+### Error Codes
+
+- `MODEL_NOT_FOUND` - Model not found
+- `MODEL_LOAD_ERROR` - Model loading failed
+- `MODEL_NOT_INITIALIZED` - Model not initialized
+- `GENERATION_ERROR` - Generation error
+- `EMBEDDING_ERROR` - Embedding error
+- `INVALID_INPUT` - Invalid input
+- `ABORTED` - Operation cancelled
+
+## Performance Optimization
+
+### Model Reuse
+
+Provider automatically reuses loaded models:
+
+```typescript
+// First call loads the model
+const model1 = provider.languageModel('qwen2.5-0.5b-instruct');
+
+// Second call reuses the loaded model
+const model2 = provider.languageModel('qwen2.5-0.5b-instruct');
+```
+
+### Batch Embedding
+
+Use `embedMany` for batch processing with better performance:
+
+```typescript
+// Recommended: Batch processing
+const result = await embedMany({
+  model,
+  values: ['Text 1', 'Text 2', 'Text 3'],
+});
+
+// Not recommended: Individual processing
+for (const text of texts) {
+  await embed({ model, value: text });
+}
+```
+
+### Resource Management
+
+Release resources when done:
+
+```typescript
+// Unload all models
+await provider.dispose();
+```
+
+## Examples
+
+See `examples/` directory for more examples:
+
+- `basic-usage.ts` - Basic usage
+- `streaming.ts` - Streaming generation
+- `embedding.ts` - Text embedding
+- `batch-embedding.ts` - Batch embedding
+- `model-switching.ts` - Model switching
+- `error-handling.ts` - Error handling
+
+## Model Download
+
+Models are automatically downloaded from Hugging Face on first use. For offline usage, see [Model Download Guide](./docs/model-download-guide.md).
+
+## Tech Stack
+
+- [Transformers.js](https://huggingface.co/docs/transformers.js) - Transformers in browser and Node.js
+- [ONNX Runtime](https://onnxruntime.ai/) - High-performance inference engine
+- [AI SDK](https://sdk.vercel.ai/) - Vercel AI SDK v2
+
+## License
+
+MIT
+
+## Related Projects
+
+- [@easbot/ollama-sdk](../ollama-sdk) - Ollama Model SDK

package/README.md

@@ -0,0 +1,282 @@
+# Local Model SDK
+
+基于 Transformers.js + ONNX 的本地模型 SDK，支持 Language Model（文本生成）和 Embedding（文本嵌入）功能，完全兼容 AI SDK 规范。
+
+## 特性
+
+- ✅ **Language Model 支持**：基于 Transformers.js 的文本生成能力
+- ✅ **Embedding 支持**：高质量的文本嵌入向量生成
+- ✅ **流式生成**：支持实时流式文本输出
+- ✅ **批量处理**：高效的批量嵌入处理
+- ✅ **模型缓存**：自动管理模型文件缓存
+- ✅ **AI SDK 兼容**：完全兼容 AI SDK v2 规范
+- ✅ **TypeScript**：完整的类型定义支持
+- ✅ **离线运行**：无需网络连接，完全本地化
+
+## 安装
+
+```bash
+pnpm add @easbot/local-model-sdk
+```
+
+## 快速开始
+
+### Language Model - 文本生成
+
+```typescript
+import { LocalModelProvider } from '@easbot/local-model-sdk';
+import { generateText } from 'ai';
+
+// 创建 Provider
+const provider = new LocalModelProvider({
+  cacheDir: '.cache/models', // 可选，默认为 ~/.cache/easbot/models
+});
+
+// 创建 Language Model
+const model = provider.languageModel('qwen2.5-0.5b-instruct');
+
+// 生成文本
+const result = await generateText({
+  model,
+  prompt: 'What is the capital of France?',
+});
+
+console.log(result.text);
+```
+
+### Language Model - 流式生成
+
+```typescript
+import { streamText } from 'ai';
+
+const result = streamText({
+  model,
+  prompt: 'Write a short story about a robot.',
+});
+
+// 逐块输出
+for await (const chunk of result.textStream) {
+  process.stdout.write(chunk);
+}
+```
+
+### Embedding - 文本嵌入
+
+```typescript
+import { embed, embedMany } from 'ai';
+
+// 单个文本嵌入
+const result = await embed({
+  model: provider.textEmbeddingModel('bge-small-zh-v1.5'),
+  value: 'Hello world',
+});
+
+console.log(result.embedding); // number[]
+
+// 批量文本嵌入
+const batchResult = await embedMany({
+  model: provider.textEmbeddingModel('bge-small-zh-v1.5'),
+  values: ['Text 1', 'Text 2', 'Text 3'],
+});
+
+console.log(batchResult.embeddings); // number[][]
+```
+
+## 支持的模型
+
+### Language Models
+
+| 模型 ID | 描述 | 参数量 |
+|---------|------|--------|
+| `qwen3-0.6b` | Qwen3 0.6B 基础模型 | 0.6B |
+| `qwen2.5-0.5b-instruct` | Qwen2.5 0.5B 指令模型 | 0.5B |
+| `qwen2.5-1.5b-instruct` | Qwen2.5 1.5B 指令模型 | 1.5B |
+| `qwen2.5-3b-instruct` | Qwen2.5 3B 指令模型 | 3B |
+
+### Embedding Models
+
+| 模型 ID | 描述 | 维度 |
+|---------|------|------|
+| `bge-small-zh-v1.5` | BGE Small 中文模型 | 512 |
+| `bge-base-zh-v1.5` | BGE Base 中文模型 | 768 |
+| `bge-large-zh-v1.5` | BGE Large 中文模型 | 1024 |
+
+## API 文档
+
+### LocalModelProvider
+
+Provider 类，用于创建和管理模型实例。
+
+```typescript
+const provider = new LocalModelProvider({
+  cacheDir: '.cache/models', // 可选
+});
+```
+
+#### 方法
+
+- `languageModel(modelId: string): LocalLanguageModel` - 创建 Language Model 实例
+- `textEmbeddingModel(modelId: string): LocalEmbeddingModel` - 创建 Embedding Model 实例
+- `getLoadedModels(): ModelInfo[]` - 获取已加载的模型列表
+- `dispose(): Promise<void>` - 卸载所有模型，释放资源
+
+### LocalLanguageModel
+
+实现 AI SDK 的 `LanguageModelV2` 接口。
+
+```typescript
+const model = provider.languageModel('qwen2.5-0.5b-instruct');
+
+// 非流式生成
+const result = await generateText({
+  model,
+  prompt: 'Your prompt here',
+  temperature: 0.7,
+  maxTokens: 100,
+});
+
+// 流式生成
+const stream = streamText({
+  model,
+  prompt: 'Your prompt here',
+});
+```
+
+### LocalEmbeddingModel
+
+实现 AI SDK 的 `EmbeddingModelV2` 接口。
+
+```typescript
+const model = provider.textEmbeddingModel('bge-small-zh-v1.5');
+
+// 单个文本
+const result = await embed({
+  model,
+  value: 'Your text here',
+});
+
+// 批量文本
+const batchResult = await embedMany({
+  model,
+  values: ['Text 1', 'Text 2', 'Text 3'],
+});
+```
+
+## 配置选项
+
+### Provider 配置
+
+```typescript
+interface LocalModelProviderConfig {
+  /** 模型缓存目录，默认为 ~/.cache/easbot/models */
+  cacheDir?: string;
+}
+```
+
+### 生成选项
+
+```typescript
+// 通过 AI SDK 的 generateText/streamText 传递
+{
+  temperature?: number;  // 0.0 - 2.0，默认 0.7
+  maxTokens?: number;    // 最大生成 token 数
+  topP?: number;         // 0.0 - 1.0，默认 0.9
+  topK?: number;         // 默认 50
+  abortSignal?: AbortSignal; // 取消信号
+}
+```
+
+## 错误处理
+
+SDK 提供详细的错误类型：
+
+```typescript
+import { LocalModelError, ErrorCode } from '@easbot/local-model-sdk';
+
+try {
+  const result = await generateText({ model, prompt: 'Test' });
+} catch (error) {
+  if (error instanceof LocalModelError) {
+    console.error(`Error [${error.code}]: ${error.message}`);
+  }
+}
+```
+
+### 错误代码
+
+- `MODEL_NOT_FOUND` - 模型未找到
+- `MODEL_LOAD_ERROR` - 模型加载失败
+- `MODEL_NOT_INITIALIZED` - 模型未初始化
+- `GENERATION_ERROR` - 生成错误
+- `EMBEDDING_ERROR` - 嵌入错误
+- `INVALID_INPUT` - 无效输入
+- `ABORTED` - 操作已取消
+
+## 性能优化
+
+### 模型复用
+
+Provider 会自动复用已加载的模型：
+
+```typescript
+// 第一次调用会加载模型
+const model1 = provider.languageModel('qwen2.5-0.5b-instruct');
+
+// 第二次调用会复用已加载的模型
+const model2 = provider.languageModel('qwen2.5-0.5b-instruct');
+```
+
+### 批量嵌入
+
+使用 `embedMany` 进行批量处理，性能更优：
+
+```typescript
+// 推荐：批量处理
+const result = await embedMany({
+  model,
+  values: ['Text 1', 'Text 2', 'Text 3'],
+});
+
+// 不推荐：逐个处理
+for (const text of texts) {
+  await embed({ model, value: text });
+}
+```
+
+### 资源管理
+
+使用完毕后释放资源：
+
+```typescript
+// 卸载所有模型
+await provider.dispose();
+```
+
+## 示例
+
+查看 `examples/` 目录获取更多示例：
+
+- `basic-usage.ts` - 基本用法
+- `streaming.ts` - 流式生成
+- `embedding.ts` - 文本嵌入
+- `batch-embedding.ts` - 批量嵌入
+- `model-switching.ts` - 模型切换
+- `error-handling.ts` - 错误处理
+
+## 模型下载
+
+首次使用时，模型会自动从 Hugging Face 下载。如需离线使用，请参考 [模型下载指南](./docs/model-download-guide.md)。
+
+## 技术栈
+
+- [Transformers.js](https://huggingface.co/docs/transformers.js) - 浏览器和 Node.js 中的 Transformers
+- [ONNX Runtime](https://onnxruntime.ai/) - 高性能推理引擎
+- [AI SDK](https://sdk.vercel.ai/) - Vercel AI SDK v2
+
+## 许可证
+
+MIT
+
+## 相关项目
+
+- [@easbot/ollama-sdk](../ollama-sdk) - Ollama 模型 SDK