@aigne/lmstudio 1.0.0

package/CHANGELOG.md

@@ -0,0 +1,18 @@
+# Changelog
+
+## 1.0.0 (2025-08-28)
+
+
+### Features
+
+* add lmstudio model adapter ([#406](https://github.com/AIGNE-io/aigne-framework/issues/406)) ([6610993](https://github.com/AIGNE-io/aigne-framework/commit/6610993cb500b1fac2bf5d17f40f351d4c897bd7))
+
+
+### Dependencies
+
+* The following workspace dependencies were updated
+  * dependencies
+    * @aigne/openai bumped to 0.13.2
+  * devDependencies
+    * @aigne/core bumped to 1.57.0
+    * @aigne/test-utils bumped to 0.5.38

package/LICENSE.md

@@ -0,0 +1,93 @@
+Elastic License 2.0
+
+URL: https://www.elastic.co/licensing/elastic-license
+
+## Acceptance
+
+By using the software, you agree to all of the terms and conditions below.
+
+## Copyright License
+
+The licensor grants you a non-exclusive, royalty-free, worldwide,
+non-sublicensable, non-transferable license to use, copy, distribute, make
+available, and prepare derivative works of the software, in each case subject to
+the limitations and conditions below.
+
+## Limitations
+
+You may not provide the software to third parties as a hosted or managed
+service, where the service provides users with access to any substantial set of
+the features or functionality of the software.
+
+You may not move, change, disable, or circumvent the license key functionality
+in the software, and you may not remove or obscure any functionality in the
+software that is protected by the license key.
+
+You may not alter, remove, or obscure any licensing, copyright, or other notices
+of the licensor in the software. Any use of the licensor’s trademarks is subject
+to applicable law.
+
+## Patents
+
+The licensor grants you a license, under any patent claims the licensor can
+license, or becomes able to license, to make, have made, use, sell, offer for
+sale, import and have imported the software, in each case subject to the
+limitations and conditions in this license. This license does not cover any
+patent claims that you cause to be infringed by modifications or additions to
+the software. If you or your company make any written claim that the software
+infringes or contributes to infringement of any patent, your patent license for
+the software granted under these terms ends immediately. If your company makes
+such a claim, your patent license ends immediately for work on behalf of your
+company.
+
+## Notices
+
+You must ensure that anyone who gets a copy of any part of the software from you
+also gets a copy of these terms.
+
+If you modify the software, you must include in any modified copies of the
+software prominent notices stating that you have modified the software.
+
+## No Other Rights
+
+These terms do not imply any licenses other than those expressly granted in
+these terms.
+
+## Termination
+
+If you use the software in violation of these terms, such use is not licensed,
+and your licenses will automatically terminate. If the licensor provides you
+with a notice of your violation, and you cease all violation of this license no
+later than 30 days after you receive that notice, your licenses will be
+reinstated retroactively. However, if you violate these terms after such
+reinstatement, any additional violation of these terms will cause your licenses
+to terminate automatically and permanently.
+
+## No Liability
+
+*As far as the law allows, the software comes as is, without any warranty or
+condition, and the licensor will not be liable to you for any damages arising
+out of these terms or the use or nature of the software, under any kind of
+legal claim.*
+
+## Definitions
+
+The **licensor** is the entity offering these terms, and the **software** is the
+software the licensor makes available under these terms, including any portion
+of it.
+
+**you** refers to the individual or entity agreeing to these terms.
+
+**your company** is any legal entity, sole proprietorship, or other kind of
+organization that you work for, plus all organizations that have control over,
+are under the control of, or are under common control with that
+organization. **control** means ownership of substantially all the assets of an
+entity, or the power to direct its management and policies by vote, contract, or
+otherwise. Control can be direct or indirect.
+
+**your licenses** are all the licenses granted to you for the software under
+these terms.
+
+**use** means anything you do with the software requiring one of your licenses.
+
+**trademark** means trademarks, service marks, and similar rights.

package/README.md

@@ -0,0 +1,235 @@
+# @aigne/lmstudio
+
+AIGNE LM Studio model adapter for integrating with locally hosted AI models via LM Studio.
+
+## Overview
+
+This model adapter provides a seamless integration with LM Studio's OpenAI-compatible API, allowing you to run local Large Language Models (LLMs) through the AIGNE framework. LM Studio provides a user-friendly interface for running local AI models with an OpenAI-compatible API.
+
+## Installation
+
+```bash
+npm install @aigne/lmstudio
+# or
+pnpm add @aigne/lmstudio
+# or
+yarn add @aigne/lmstudio
+```
+
+## Prerequisites
+
+1. **Install LM Studio**: Download and install LM Studio from [https://lmstudio.ai/](https://lmstudio.ai/)
+2. **Download Models**: Use LM Studio's interface to download local models (e.g., Llama 3.2, Mistral, etc.)
+3. **Start Local Server**: Launch the local server from LM Studio's "Local Server" tab
+
+## Quick Start
+
+```typescript
+import { LMStudioChatModel } from "@aigne/lmstudio";
+
+// Create a new LM Studio chat model
+const model = new LMStudioChatModel({
+  baseURL: "http://localhost:1234/v1", // Default LM Studio server URL
+  model: "llama-3.2-3b-instruct", // Model name as shown in LM Studio
+  modelOptions: {
+    temperature: 0.7,
+    maxTokens: 2048,
+  },
+});
+
+// Basic usage
+const response = await model.invoke({
+  messages: [
+    { role: "user", content: "What is the capital of France?" }
+  ],
+});
+
+console.log(response.text); // "The capital of France is Paris."
+```
+
+## Configuration
+
+### Environment Variables
+
+You can configure the LM Studio connection using environment variables:
+
+```bash
+# LM Studio server URL (default: http://localhost:1234/v1)
+LM_STUDIO_BASE_URL=http://localhost:1234/v1
+
+# API Key (not required for local LM Studio, defaults to "not-required")
+# Only set this if you have configured authentication in LM Studio
+# LM_STUDIO_API_KEY=your-key-if-needed
+```
+
+**Note:** LM Studio typically runs locally without authentication. The API key is set to a placeholder value "not-required" by default.
+
+### Model Options
+
+```typescript
+const model = new LMStudioChatModel({
+  baseURL: "http://localhost:1234/v1",
+  model: "llama-3.2-3b-instruct",
+  // apiKey: "not-required", // Optional, not required for local LM Studio
+  modelOptions: {
+    temperature: 0.7,     // Controls randomness (0.0 to 2.0)
+    maxTokens: 2048,      // Maximum tokens in response
+    topP: 0.9,           // Nucleus sampling
+    frequencyPenalty: 0,  // Frequency penalty
+    presencePenalty: 0,   // Presence penalty
+  },
+});
+```
+
+## Features
+
+### Streaming Support
+
+```typescript
+const model = new LMStudioChatModel();
+
+const stream = await model.invoke(
+  {
+    messages: [{ role: "user", content: "Tell me a short story" }],
+  },
+  { streaming: true }
+);
+
+for await (const chunk of stream) {
+  if (chunk.type === "delta" && chunk.delta.text) {
+    process.stdout.write(chunk.delta.text.text);
+  }
+}
+```
+
+### Tool/Function Calling
+
+LM Studio supports OpenAI-compatible function calling:
+
+```typescript
+const tools = [
+  {
+    type: "function" as const,
+    function: {
+      name: "get_weather",
+      description: "Get current weather information",
+      parameters: {
+        type: "object",
+        properties: {
+          location: {
+            type: "string",
+            description: "The city and state, e.g. San Francisco, CA",
+          },
+        },
+        required: ["location"],
+      },
+    },
+  },
+];
+
+const response = await model.invoke({
+  messages: [
+    { role: "user", content: "What's the weather like in New York?" }
+  ],
+  tools,
+});
+
+if (response.toolCalls?.length) {
+  console.log("Tool calls:", response.toolCalls);
+}
+```
+
+### Structured Output
+
+Generate structured JSON responses:
+
+```typescript
+const responseFormat = {
+  type: "json_schema" as const,
+  json_schema: {
+    name: "weather_response",
+    schema: {
+      type: "object",
+      properties: {
+        location: { type: "string" },
+        temperature: { type: "number" },
+        description: { type: "string" },
+      },
+      required: ["location", "temperature", "description"],
+    },
+  },
+};
+
+const response = await model.invoke({
+  messages: [
+    { role: "user", content: "Get weather for Paris in JSON format" }
+  ],
+  responseFormat,
+});
+
+console.log(response.json); // Parsed JSON object
+```
+
+## Supported Models
+
+LM Studio supports a wide variety of models. Popular choices include:
+
+- **Llama 3.2** (3B, 8B, 70B variants)
+- **Llama 3.1** (8B, 70B, 405B variants)
+- **Mistral** (7B, 8x7B variants)
+- **CodeLlama** (7B, 13B, 34B variants)
+- **Qwen** (various sizes)
+- **Phi-3** (mini, small, medium variants)
+
+The model name should match exactly what appears in your LM Studio interface.
+
+## Error Handling
+
+```typescript
+import { LMStudioChatModel } from "@aigne/lmstudio";
+
+const model = new LMStudioChatModel();
+
+try {
+  const response = await model.invoke({
+    messages: [{ role: "user", content: "Hello!" }],
+  });
+  console.log(response.text);
+} catch (error) {
+  if (error.code === "ECONNREFUSED") {
+    console.error("LM Studio server is not running. Please start the local server.");
+  } else {
+    console.error("Error:", error.message);
+  }
+}
+```
+
+## Performance Tips
+
+1. **Model Selection**: Smaller models (3B-8B parameters) are faster but less capable
+2. **Context Length**: Be mindful of context window limits for your chosen model
+3. **Temperature**: Lower values (0.1-0.3) for factual tasks, higher (0.7-1.0) for creative tasks
+4. **Batch Processing**: Process multiple requests concurrently when possible
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Connection Refused**: Ensure LM Studio's local server is running
+2. **Model Not Found**: Verify the model name matches what's shown in LM Studio
+3. **Out of Memory**: Try using a smaller model or reduce context length
+4. **Slow Responses**: Consider using GPU acceleration if available
+
+## License
+
+This project is licensed under the Elastic License 2.0 - see the [LICENSE](../../LICENSE) file for details.
+
+## Contributing
+
+Contributions are welcome! Please read our [contributing guidelines](../../CONTRIBUTING.md) first.
+
+## Support
+
+- [GitHub Issues](https://github.com/AIGNE-io/aigne-framework/issues)
+- [Documentation](https://www.arcblock.io/docs/aigne-framework)
+- [LM Studio Documentation](https://lmstudio.ai/docs)

package/lib/cjs/index.d.ts

@@ -0,0 +1 @@
+export * from "./lmstudio-chat-model.js";

package/lib/cjs/index.js

@@ -0,0 +1,17 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __exportStar = (this && this.__exportStar) || function(m, exports) {
+    for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+__exportStar(require("./lmstudio-chat-model.js"), exports);

package/lib/cjs/lmstudio-chat-model.d.ts

@@ -0,0 +1,24 @@
+import { OpenAIChatModel, type OpenAIChatModelOptions } from "@aigne/openai";
+/**
+ * Implementation of the ChatModel interface for LM Studio
+ *
+ * This model allows you to run local LLMs through LM Studio,
+ * with an OpenAI-compatible API interface.
+ *
+ * Default model: 'llama-3.2-3b-instruct'
+ *
+ * @example
+ * Here's how to create and use an LM Studio chat model:
+ * {@includeCode ../test/lmstudio-chat-model.test.ts#example-lmstudio-chat-model}
+ *
+ * @example
+ * Here's an example with streaming response:
+ * {@includeCode ../test/lmstudio-chat-model.test.ts#example-lmstudio-chat-model-streaming}
+ */
+export declare class LMStudioChatModel extends OpenAIChatModel {
+    constructor(options?: OpenAIChatModelOptions);
+    protected apiKeyEnvName: string;
+    protected apiKeyDefault: string;
+    protected supportsNativeStructuredOutputs: boolean;
+    protected supportsTemperature: boolean;
+}

package/lib/cjs/lmstudio-chat-model.js

@@ -0,0 +1,36 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.LMStudioChatModel = void 0;
+const openai_1 = require("@aigne/openai");
+const LM_STUDIO_DEFAULT_BASE_URL = "http://localhost:1234/v1";
+const LM_STUDIO_DEFAULT_CHAT_MODEL = "llama-3.2-3b-instruct";
+/**
+ * Implementation of the ChatModel interface for LM Studio
+ *
+ * This model allows you to run local LLMs through LM Studio,
+ * with an OpenAI-compatible API interface.
+ *
+ * Default model: 'llama-3.2-3b-instruct'
+ *
+ * @example
+ * Here's how to create and use an LM Studio chat model:
+ * {@includeCode ../test/lmstudio-chat-model.test.ts#example-lmstudio-chat-model}
+ *
+ * @example
+ * Here's an example with streaming response:
+ * {@includeCode ../test/lmstudio-chat-model.test.ts#example-lmstudio-chat-model-streaming}
+ */
+class LMStudioChatModel extends openai_1.OpenAIChatModel {
+    constructor(options) {
+        super({
+            ...options,
+            model: options?.model || LM_STUDIO_DEFAULT_CHAT_MODEL,
+            baseURL: options?.baseURL || process.env.LM_STUDIO_BASE_URL || LM_STUDIO_DEFAULT_BASE_URL,
+        });
+    }
+    apiKeyEnvName = "LM_STUDIO_API_KEY";
+    apiKeyDefault = "not-required";
+    supportsNativeStructuredOutputs = false;
+    supportsTemperature = true;
+}
+exports.LMStudioChatModel = LMStudioChatModel;

package/lib/cjs/package.json

@@ -0,0 +1,3 @@
+{
+  "type": "commonjs"
+}

package/lib/dts/index.d.ts

@@ -0,0 +1 @@
+export * from "./lmstudio-chat-model.js";

package/lib/dts/lmstudio-chat-model.d.ts

@@ -0,0 +1,24 @@
+import { OpenAIChatModel, type OpenAIChatModelOptions } from "@aigne/openai";
+/**
+ * Implementation of the ChatModel interface for LM Studio
+ *
+ * This model allows you to run local LLMs through LM Studio,
+ * with an OpenAI-compatible API interface.
+ *
+ * Default model: 'llama-3.2-3b-instruct'
+ *
+ * @example
+ * Here's how to create and use an LM Studio chat model:
+ * {@includeCode ../test/lmstudio-chat-model.test.ts#example-lmstudio-chat-model}
+ *
+ * @example
+ * Here's an example with streaming response:
+ * {@includeCode ../test/lmstudio-chat-model.test.ts#example-lmstudio-chat-model-streaming}
+ */
+export declare class LMStudioChatModel extends OpenAIChatModel {
+    constructor(options?: OpenAIChatModelOptions);
+    protected apiKeyEnvName: string;
+    protected apiKeyDefault: string;
+    protected supportsNativeStructuredOutputs: boolean;
+    protected supportsTemperature: boolean;
+}

package/lib/esm/index.d.ts

@@ -0,0 +1 @@
+export * from "./lmstudio-chat-model.js";

package/lib/esm/index.js

@@ -0,0 +1 @@
+export * from "./lmstudio-chat-model.js";

package/lib/esm/lmstudio-chat-model.d.ts

@@ -0,0 +1,24 @@
+import { OpenAIChatModel, type OpenAIChatModelOptions } from "@aigne/openai";
+/**
+ * Implementation of the ChatModel interface for LM Studio
+ *
+ * This model allows you to run local LLMs through LM Studio,
+ * with an OpenAI-compatible API interface.
+ *
+ * Default model: 'llama-3.2-3b-instruct'
+ *
+ * @example
+ * Here's how to create and use an LM Studio chat model:
+ * {@includeCode ../test/lmstudio-chat-model.test.ts#example-lmstudio-chat-model}
+ *
+ * @example
+ * Here's an example with streaming response:
+ * {@includeCode ../test/lmstudio-chat-model.test.ts#example-lmstudio-chat-model-streaming}
+ */
+export declare class LMStudioChatModel extends OpenAIChatModel {
+    constructor(options?: OpenAIChatModelOptions);
+    protected apiKeyEnvName: string;
+    protected apiKeyDefault: string;
+    protected supportsNativeStructuredOutputs: boolean;
+    protected supportsTemperature: boolean;
+}

package/lib/esm/lmstudio-chat-model.js

@@ -0,0 +1,32 @@
+import { OpenAIChatModel } from "@aigne/openai";
+const LM_STUDIO_DEFAULT_BASE_URL = "http://localhost:1234/v1";
+const LM_STUDIO_DEFAULT_CHAT_MODEL = "llama-3.2-3b-instruct";
+/**
+ * Implementation of the ChatModel interface for LM Studio
+ *
+ * This model allows you to run local LLMs through LM Studio,
+ * with an OpenAI-compatible API interface.
+ *
+ * Default model: 'llama-3.2-3b-instruct'
+ *
+ * @example
+ * Here's how to create and use an LM Studio chat model:
+ * {@includeCode ../test/lmstudio-chat-model.test.ts#example-lmstudio-chat-model}
+ *
+ * @example
+ * Here's an example with streaming response:
+ * {@includeCode ../test/lmstudio-chat-model.test.ts#example-lmstudio-chat-model-streaming}
+ */
+export class LMStudioChatModel extends OpenAIChatModel {
+    constructor(options) {
+        super({
+            ...options,
+            model: options?.model || LM_STUDIO_DEFAULT_CHAT_MODEL,
+            baseURL: options?.baseURL || process.env.LM_STUDIO_BASE_URL || LM_STUDIO_DEFAULT_BASE_URL,
+        });
+    }
+    apiKeyEnvName = "LM_STUDIO_API_KEY";
+    apiKeyDefault = "not-required";
+    supportsNativeStructuredOutputs = false;
+    supportsTemperature = true;
+}

package/lib/esm/package.json

@@ -0,0 +1,3 @@
+{
+  "type": "module"
+}

package/package.json

@@ -0,0 +1,54 @@
+{
+  "name": "@aigne/lmstudio",
+  "version": "1.0.0",
+  "description": "AIGNE LM Studio model adapter for integrating with locally hosted AI models via LM Studio",
+  "publishConfig": {
+    "access": "public"
+  },
+  "author": "Arcblock <blocklet@arcblock.io> https://github.com/blocklet",
+  "homepage": "https://github.com/AIGNE-io/aigne-framework",
+  "license": "Elastic-2.0",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/AIGNE-io/aigne-framework"
+  },
+  "files": [
+    "lib/cjs",
+    "lib/dts",
+    "lib/esm",
+    "LICENSE",
+    "README.md",
+    "CHANGELOG.md"
+  ],
+  "type": "module",
+  "main": "./lib/cjs/index.js",
+  "module": "./lib/esm/index.js",
+  "types": "./lib/dts/index.d.ts",
+  "exports": {
+    ".": {
+      "import": "./lib/esm/index.js",
+      "require": "./lib/cjs/index.js",
+      "types": "./lib/dts/index.d.ts"
+    }
+  },
+  "dependencies": {
+    "@aigne/openai": "^0.13.2"
+  },
+  "devDependencies": {
+    "@types/bun": "^1.2.18",
+    "@types/node": "^24.0.12",
+    "npm-run-all": "^4.1.5",
+    "rimraf": "^6.0.1",
+    "typescript": "^5.8.3",
+    "@aigne/core": "^1.57.0",
+    "@aigne/test-utils": "^0.5.38"
+  },
+  "scripts": {
+    "lint": "tsc --noEmit",
+    "build": "tsc --build scripts/tsconfig.build.json",
+    "clean": "rimraf lib test/coverage",
+    "test": "bun test",
+    "test:coverage": "bun test --coverage --coverage-reporter=lcov --coverage-reporter=text",
+    "postbuild": "node ../../scripts/post-build-lib.mjs"
+  }
+}