inference-server 1.0.0-beta.24 → 1.0.0-beta.26

package/README.md

@@ -23,21 +23,21 @@ The project includes a model resource pool, an inference queue and a HTTP API se
 Example with minimal configuration:
 
 ```ts basic.ts
-import { ModelServer } from 'inference-server'
+import { InferenceServer } from 'inference-server'
 
-const modelServer = new ModelServer({
+const localModels = new InferenceServer({
   log: 'info', // default is 'warn'
   models: {
     'my-model': { // Identifiers can use a-zA-Z0-9_:\-\.
       // Required are `task`, `engine`, `url` and/or `file`.
       task: 'text-completion', // text-completion models can be used for chat and text generation tasks
       engine: 'node-llama-cpp', // each engine comes with a peer dep. `npm install node-llama-cpp@3`
-      url: 'https://huggingface.co/HuggingFaceTB/smollm-135M-instruct-v0.2-Q8_0-GGUF/blob/main/smollm-135m-instruct-add-basics-q8_0.gguf',
+      url: 'https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF/blob/main/smollm2-1.7b-instruct-q4_k_m.gguf',
     },
   },
 })
-await modelServer.start()
-const result = await modelServer.processChatCompletionTask({
+await localModels.start()
+const result = await localModels.processChatCompletionTask({
   model: 'my-model',
   messages: [
     {
@@ -47,23 +47,24 @@ const result = await modelServer.processChatCompletionTask({
   ],
 })
 console.debug(result)
-modelServer.stop()
+localModels.stop()
 ```
 
-Or, to start an OAI compatible HTTP server with two concurrent instances of the same model:
+Or, to launch an OpenAI-compatible HTTP server with two concurrent instances of the same model:
+
 
 ```ts http-api.ts
-import { startHTTPServer } from 'inference-server'
+import { InferenceServer, createExpressServer } from 'inference-server'
 import OpenAI from 'openai'
 
-const server = await startHTTPServer({
-  listen: { port: 3000 }, // apart from `listen` options are identical to ModelServer
+const localModels = new InferenceServer({
+  log: 'info', // default is 'warn'
   concurrency: 2, // two inference processes may run at the same time
   models: {
     'smollm': {
       task: 'text-completion',
       engine: 'node-llama-cpp',
-      url: 'https://huggingface.co/HuggingFaceTB/smollm-135M-instruct-v0.2-Q8_0-GGUF/blob/main/smollm-135m-instruct-add-basics-q8_0.gguf',
+      url: 'https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF/blob/main/smollm2-1.7b-instruct-q4_k_m.gguf',
       maxInstances: 2, // two instances of this model may be loaded into memory
       device: {
         cpuThreads: 4, // limit cpu threads so we dont occupy all cores
@@ -71,6 +72,9 @@ const server = await startHTTPServer({
     },
   },
 })
+localModels.start()
+const httpServer = createExpressServer(localModels)
+httpServer.listen(3000)
 
 const client = new OpenAI({
   baseURL: 'http://localhost:3000/openai/v1/',
@@ -88,9 +92,12 @@ for await (const chunk of completion) {
     process.stdout.write(chunk.choices[0].delta.content)
   }
 }
-server.stop()
+localModels.stop()
+httpServer.close()
 ```
 
+In addition to the OpenAI endpoints, the server created by `createExpressServer` will also display some internal state at `/`. The endpoint implementations are also exported from the package and can be used with any http server.
+
 More usage examples:
 - Using all available options / model options API doc [./examples/all-options](./examples/all-options.js).
 - Custom engines [./tests/engines/experiments.test.ts](./tests/engines/experiments.test.ts).
@@ -127,6 +134,9 @@ Only available when using node-llama-cpp and a model that supports function call
 ##### Huge node_modules when installing all engines
 CUDA binaries are distributed with each engine seperately, which leads to an extra 0.5-1GB of disk use. Unfortunately there is nothing I can do about that.
 
+##### Can't use larger models with transformers.js
+Models using the ONNX external data format are currently ([not supported in node.js+transformers.js](https://github.com/huggingface/transformers.js/blob/e129c47c65a049173f35e6263fd8d9f660dfc1a7/src/models.js#L240-L242)).
+
 #### TODO / Roadmap
 
 Not in any particular order:
@@ -139,20 +149,25 @@ Not in any particular order:
 - [x] Add stable-diffusion engine
 - [x] Implement more transformer.js tasks
 - [x] [CLI](https://github.com/iimez/inference-server/discussions/7)
+- [x] Rename ModelServer -> InferenceServer and refactor task types
+- [x] Support for transformer.js text-completion and chat-completion tasks
 - [ ] Add some light jsdoc for server/pool/store methods
+- [ ] Should support adding a json schema grammar on completion requests (not just a grammar name)
 - [ ] utilize node-llama-cpp's support to reuse LlamaModel weights with multiple contexts (across instances)
-- [ ] Support transformer.js for text-completion tasks ([not yet supported in Node.js](https://github.com/huggingface/transformers.js/blob/e129c47c65a049173f35e6263fd8d9f660dfc1a7/src/models.js#L240-L242))
+- [ ] Add `n` parameter support to node-llama-cpp+transformers chat completions
 - [ ] Infill completion support https://github.com/withcatai/node-llama-cpp/blob/beta/src/evaluator/LlamaCompletion.ts#L322-L336
 - [ ] Support HF auth and support HF-like model repositories on other domains
-- [ ] Find a way to type available custom engines (and their options?)
 - [ ] Rework GPU+device usage / lock (Support multiple models on gpu in cases where its possible)
 - [ ] Add engine interfaces for resource use (and estimates, see https://github.com/ggerganov/llama.cpp/issues/4315 and https://github.com/withcatai/node-llama-cpp/blob/beta/src/gguf/insights/utils/resolveContextContextSizeOption.ts)
 - [ ] Allow configuring a pools max memory usage
+- [ ] Support logits with transformers and node-llama-cpp
 - [ ] Test deno/bun support
 - [ ] Add image generation endpoint in oai api
 - [ ] Add transcript endpoint in oai api
-- [ ] Add `n` parameter support to node-llama-cpp chat completions
+- [ ] Add createSpeech endpoint in oai api
+- [ ] Read more model metadata and expose it in API+CLI
 - [ ] Replace express with tinyhttp
+- [ ] Find a way to type available custom engines (and their options?)
 
 ### Contributing
 
@@ -163,7 +178,6 @@ If you are using this package - let me know where [you would like this to go](ht
 - Create a separate HTTP API thats independent of the OpenAI spec and stateful. See [discussion](https://github.com/iimez/inference-server/discussions/8).
 - Add a clientside library (React hooks?) for use of above API.
 - Provide a Docker image. And maybe a Prometheus endpoint.
-- Logprobs support for node-llama-cpp.
 
 #### Currently not the Goals
 

package/dist/api/openai/handlers/chat.d.ts

@@ -1,3 +1,3 @@
 import type { IncomingMessage, ServerResponse } from 'node:http';
-import type { ModelServer } from '../../../server.js';
-export declare function createChatCompletionHandler(modelServer: ModelServer): (req: IncomingMessage, res: ServerResponse) => Promise<void>;
+import type { InferenceServer } from '../../../server.js';
+export declare function createChatCompletionHandler(inferenceServer: InferenceServer): (req: IncomingMessage, res: ServerResponse) => Promise<void>;

package/dist/api/openai/handlers/chat.js

@@ -87,7 +87,7 @@ function createResponseMessageContent(content) {
     }
     return text;
 }
-export function createChatCompletionHandler(modelServer) {
+export function createChatCompletionHandler(inferenceServer) {
     return async (req, res) => {
         let args;
         try {
@@ -106,7 +106,7 @@ export function createChatCompletionHandler(modelServer) {
             res.end(JSON.stringify({ error: 'Invalid request (need at least model and messages)' }));
             return;
         }
-        if (!modelServer.modelExists(args.model)) {
+        if (!inferenceServer.modelExists(args.model)) {
             res.writeHead(400, { 'Content-Type': 'application/json' });
             res.end(JSON.stringify({ error: 'Model does not exist' }));
             return;
@@ -151,7 +151,7 @@ export function createChatCompletionHandler(modelServer) {
                     completionGrammar = 'json';
                 }
             }
-            let completionTools;
+            let toolDefinitions;
             if (args.tools) {
                 const functionTools = args.tools
                     .filter((tool) => tool.type === 'function')
@@ -163,11 +163,11 @@ export function createChatCompletionHandler(modelServer) {
                     };
                 });
                 if (functionTools.length) {
-                    if (!completionTools) {
-                        completionTools = {};
+                    if (!toolDefinitions) {
+                        toolDefinitions = {};
                     }
                     for (const tool of functionTools) {
-                        completionTools[tool.name] = {
+                        toolDefinitions[tool.name] = {
                             description: tool.description,
                             parameters: tool.parameters,
                         };
@@ -179,7 +179,7 @@ export function createChatCompletionHandler(modelServer) {
                 model: args.model,
                 messages,
                 temperature: args.temperature ? args.temperature : undefined,
-                stream: args.stream ? Boolean(args.stream) : false,
+                // stream: args.stream ? Boolean(args.stream) : false,
                 maxTokens: args.max_tokens ? args.max_tokens : undefined,
                 seed: args.seed ? args.seed : undefined,
                 stop,
@@ -192,7 +192,7 @@ export function createChatCompletionHandler(modelServer) {
                 topP: args.top_p ? args.top_p : undefined,
                 tokenBias: args.logit_bias ? args.logit_bias : undefined,
                 grammar: completionGrammar,
-                tools: completionTools,
+                tools: toolDefinitions ? { definitions: toolDefinitions } : undefined,
                 // additional non-spec params
                 repeatPenaltyNum: args.repeat_penalty_num
                     ? args.repeat_penalty_num
@@ -200,11 +200,12 @@ export function createChatCompletionHandler(modelServer) {
                 minP: args.min_p ? args.min_p : undefined,
                 topK: args.top_k ? args.top_k : undefined,
             });
-            const { instance, release } = await modelServer.requestInstance(completionReq, controller.signal);
+            const { instance, release } = await inferenceServer.requestInstance(completionReq, controller.signal);
             if (ssePing) {
                 clearInterval(ssePing);
             }
-            const task = instance.processChatCompletionTask(completionReq, {
+            const task = instance.processChatCompletionTask({
+                ...completionReq,
                 signal: controller.signal,
                 onChunk: (chunk) => {
                     if (args.stream) {

package/dist/api/openai/handlers/chat.js.map

@@ -1 +1 @@
-{"version":3,"file":"chat.js","sourceRoot":"","sources":["../../../../src/api/openai/handlers/chat.ts"],"names":[],"mappings":"AAWA,OAAO,EAAE,oBAAoB,EAAE,MAAM,sCAAsC,CAAA;AAC3E,OAAO,EAAE,eAAe,EAAE,MAAM,sBAAsB,CAAA;AACtD,OAAO,EAAE,gBAAgB,EAAE,MAAM,2BAA2B,CAAA;AAC5D,OAAO,EAAE,eAAe,EAAE,cAAc,EAAE,MAAM,aAAa,CAAA;AAiB7D,KAAK,UAAU,uBAAuB,CACrC,QAAsC;IAEtC,MAAM,gBAAgB,GAAmC,EAAE,CAAA;IAC3D,MAAM,cAAc,GAAkB,EAAE,CAAA;IAExC,KAAK,MAAM,OAAO,IAAI,QAAQ,EAAE,CAAC;QAChC,MAAM,IAAI,GAAG,cAAc,CAAC,OAAO,CAAC,IAAI,CAAC,CAAA;QACzC,MAAM,aAAa,GAAQ;YAC1B,IAAI;YACJ,OAAO,EAAE,EAAE;SACX,CAAA;QACD,IAAI,IAAI,KAAK,MAAM,IAAI,cAAc,IAAI,OAAO,EAAE,CAAC;YAClD,aAAa,CAAC,MAAM,GAAG,OAAO,CAAC,YAAY,CAAA;QAC5C,CAAC;QAED,IAAI,OAAO,OAAO,CAAC,OAAO,KAAK,QAAQ,EAAE,CAAC;YACzC,aAAa,CAAC,OAAO,CAAC,IAAI,CAAC;gBAC1B,IAAI,EAAE,MAAM;gBACZ,IAAI,EAAE,OAAO,CAAC,OAAO;aACrB,CAAC,CAAA;QACH,CAAC;aAAM,IAAI,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC;YAC3C,KAAK,MAAM,IAAI,IAAI,OAAO,CAAC,OAAO,EAAE,CAAC;gBAEpC,IAAI,OAAO,IAAI,KAAK,QAAQ,EAAE,CAAC;oBAC9B,aAAa,CAAC,OAAO,CAAC,IAAI,CAAC;wBAC1B,IAAI,EAAE,MAAM;wBACZ,IAAI,EAAE,IAAI;qBACV,CAAC,CAAA;gBACH,CAAC;qBAAM,IAAI,IAAI,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;oBACjC,aAAa,CAAC,OAAO,CAAC,IAAI,CAAC;wBAC1B,IAAI,EAAE,MAAM;wBACZ,IAAI,EAAE,IAAI,CAAC,IAAI;qBACf,CAAC,CAAA;gBACH,CAAC;qBAAM,IAAI,IAAI,CAAC,IAAI,KAAK,WAAW,EAAE,CAAC;oBACtC,IAAI,CAAC,gBAAgB,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,EAAE,CAAC;wBAC3C,gBAAgB,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,GAAG,gBAAgB,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,CAAA;oBAC5E,CAAC;oBACD,MAAM,OAAO,GAAgC;wBAC5C,IAAI,EAAE,OAAO;qBACb,CAAA;oBACD,aAAa,CAAC,OAAO,CAAC,IAAI,CAAC,OAAO,CAAC,CAAA;oBACnC,gBAAgB,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,KAAK,EAAE,EAAE;wBACnD,OAAO,CAAC,KAAK,GAAG,KAAK,CAAA;oBACtB,CAAC,CAAC,CAAA;gBACH,CAAC;qBAAM,IAAI,IAAI,CAAC,IAAI,KAAK,aAAa,EAAE,CAAC;oBACxC,aAAa,CAAC,OAAO,CAAC,IAAI,CAAC;wBAC1B,IAAI,EAAE,OAAO;wBACb,KAAK,EAAE,IAAI,CAAC,WAAW;qBACvB,CAAC,CAAA;gBACH,CAAC;qBAAM,IAAI,IAAI,CAAC,IAAI,KAAK,SAAS,EAAE,CAAC;oBACpC,aAAa,CAAC,OAAO,CAAC,IAAI,CAAC;wBAC1B,IAAI,EAAE,MAAM;wBACZ,IAAI,EAAE,IAAI,CAAC,OAAO;qBAClB,CAAC,CAAA;gBACH,CAAC;YACF,CAAC;QACF,CAAC;aAAM,CAAC;YACP,MAAM,IAAI,KAAK,CAAC,yBAAyB,CAAC,CAAA;QAC3C,CAAC;QAED,cAAc,CAAC,IAAI,CAAC,aAAa,CAAC,CAAA;IACnC,CAAC;IAED,MAAM,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,gBAAgB,CAAC,CAAC,CAAA;IAElD,OAAO,cAAc,CAAA;AAEtB,CAAC;AAED,SAAS,4BAA4B,CACpC,OAAsC;IAEtC,IAAI,CAAC,OAAO,EAAE,CAAC;QACd,OAAO,IAAI,CAAA;IACZ,CAAC;IACD,IAAI,OAAO,OAAO,KAAK,QAAQ,EAAE,CAAC;QACjC,OAAO,OAAO,CAAA;IACf,CAAC;IACD,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC;QAC7B,MAAM,IAAI,KAAK,CAAC,kCAAkC,CAAC,CAAA;IACpD,CAAC;IAED,IAAI,IAAI,GAAG,EAAE,CAAA;IACb,KAAK,MAAM,IAAI,IAAI,OAAO,EAAE,CAAC;QAC5B,IAAI,IAAI,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;YAC1B,IAAI,IAAI,IAAI,CAAC,IAAI,CAAA;QAClB,CAAC;QACD,kEAAkE;IACnE,CAAC;IACD,OAAO,IAAI,CAAA;AACZ,CAAC;AAED,MAAM,UAAU,2BAA2B,CAAC,WAAwB;IACnE,OAAO,KAAK,EAAE,GAAoB,EAAE,GAAmB,EAAE,EAAE;QAC1D,IAAI,IAAgC,CAAA;QAEpC,IAAI,CAAC;YACJ,MAAM,IAAI,GAAG,MAAM,oBAAoB,CAAC,GAAG,CAAC,CAAA;YAC5C,IAAI,GAAG,IAAI,CAAA;QACZ,CAAC;QAAC,OAAO,CAAC,EAAE,CAAC;YACZ,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAA;YAChB,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,iBAAiB,EAAE,CAAC,CAAC,CAAA;YACrD,OAAM;QACP,CAAC;QAED,8BAA8B;QAC9B,IAAI,CAAC,IAAI,CAAC,KAAK,IAAI,CAAC,IAAI,CAAC,QAAQ,EAAE,CAAC;YACnC,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,oDAAoD,EAAE,CAAC,CAAC,CAAA;YACxF,OAAM;QACP,CAAC;QAED,IAAI,CAAC,WAAW,CAAC,WAAW,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;YAC1C,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,sBAAsB,EAAE,CAAC,CAAC,CAAA;YAC1D,OAAM;QACP,CAAC;QAED,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAA;QACxC,GAAG,CAAC,EAAE,CAAC,OAAO,EAAE,GAAG,EAAE;YACpB,OAAO,CAAC,KAAK,CAAC,0BAA0B,CAAC,CAAA;YACzC,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QACF,GAAG,CAAC,EAAE,CAAC,KAAK,EAAE,GAAG,EAAE;YAClB,OAAO,CAAC,KAAK,CAAC,yBAAyB,CAAC,CAAA;YACxC,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QACF,GAAG,CAAC,EAAE,CAAC,SAAS,EAAE,GAAG,EAAE;YACtB,OAAO,CAAC,KAAK,CAAC,2BAA2B,CAAC,CAAA;YAC1C,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QACF,GAAG,CAAC,EAAE,CAAC,OAAO,EAAE,GAAG,EAAE;YACpB,OAAO,CAAC,KAAK,CAAC,cAAc,CAAC,CAAA;YAC7B,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QAEF,IAAI,CAAC;YACJ,IAAI,OAAmC,CAAA;YACvC,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;gBACjB,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE;oBAClB,cAAc,EAAE,mBAAmB;oBACnC,eAAe,EAAE,UAAU;oBAC3B,UAAU,EAAE,YAAY;iBACxB,CAAC,CAAA;gBACF,GAAG,CAAC,YAAY,EAAE,CAAA;gBAClB,OAAO,GAAG,WAAW,CAAC,GAAG,EAAE;oBAC1B,GAAG,CAAC,KAAK,CAAC,WAAW,CAAC,CAAA;gBACvB,CAAC,EAAE,KAAK,CAAC,CAAA;YACV,CAAC;YAED,IAAI,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS,CAAA;YAC5C,IAAI,OAAO,IAAI,KAAK,QAAQ,EAAE,CAAC;gBAC9B,IAAI,GAAG,CAAC,IAAI,CAAC,CAAA;YACd,CAAC;YAED,IAAI,iBAAqC,CAAA;YACzC,IAAI,IAAI,CAAC,eAAe,EAAE,CAAC;gBAC1B,IAAI,IAAI,CAAC,eAAe,CAAC,IAAI,KAAK,aAAa,EAAE,CAAC;oBACjD,iBAAiB,GAAG,MAAM,CAAA;gBAC3B,CAAC;YACF,CAAC;YAED,IAAI,eAEQ,CAAA;YAEZ,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;gBAChB,MAAM,aAAa,GAAG,IAAI,CAAC,KAAK;qBAC9B,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,KAAK,UAAU,CAAC;qBAC1C,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE;oBACb,OAAO;wBACN,IAAI,EAAE,IAAI,CAAC,QAAQ,CAAC,IAAI;wBACxB,WAAW,EAAE,IAAI,CAAC,QAAQ,CAAC,WAAW;wBACtC,UAAU,EAAE,IAAI,CAAC,QAAQ,CAAC,UAAU;qBACpC,CAAA;gBACF,CAAC,CAAC,CAAA;gBACH,IAAI,aAAa,CAAC,MAAM,EAAE,CAAC;oBAC1B,IAAI,CAAC,eAAe,EAAE,CAAC;wBACtB,eAAe,GAAG,EAAE,CAAA;oBACrB,CAAC;oBACD,KAAK,MAAM,IAAI,IAAI,aAAa,EAAE,CAAC;wBAClC,eAAe,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG;4BAC5B,WAAW,EAAE,IAAI,CAAC,WAAW;4BAC7B,UAAU,EAAE,IAAI,CAAC,UAAU;yBACT,CAAA;oBACpB,CAAC;gBACF,CAAC;YACF,CAAC;YAED,MAAM,QAAQ,GAAG,MAAM,uBAAuB,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAA;YAC7D,MAAM,aAAa,GAAG,eAAe,CAAwB;gBAC5D,KAAK,EAAE,IAAI,CAAC,KAAK;gBACjB,QAAQ;gBACR,WAAW,EAAE,IAAI,CAAC,WAAW,CAAC,CAAC,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC,CAAC,SAAS;gBAC5D,MAAM,EAAE,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,KAAK;gBAClD,SAAS,EAAE,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,SAAS;gBACxD,IAAI,EAAE,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS;gBACvC,IAAI;gBACJ,gBAAgB,EAAE,IAAI,CAAC,iBAAiB;oBACvC,CAAC,CAAC,IAAI,CAAC,iBAAiB;oBACxB,CAAC,CAAC,SAAS;gBACZ,eAAe,EAAE,IAAI,CAAC,gBAAgB;oBACrC,CAAC,CAAC,IAAI,CAAC,gBAAgB;oBACvB,CAAC,CAAC,SAAS;gBACZ,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS;gBACzC,SAAS,EAAE,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,SAAS;gBACxD,OAAO,EAAE,iBAAiB;gBAC1B,KAAK,EAAE,eAAe;gBACtB,6BAA6B;gBAC7B,gBAAgB,EAAE,IAAI,CAAC,kBAAkB;oBACxC,CAAC,CAAC,IAAI,CAAC,kBAAkB;oBACzB,CAAC,CAAC,SAAS;gBACZ,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS;gBACzC,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS;aACzC,CAAC,CAAA;YACF,MAAM,EAAE,QAAQ,EAAE,OAAO,EAAE,GAAG,MAAM,WAAW,CAAC,eAAe,CAC9D,aAAa,EACb,UAAU,CAAC,MAAM,CACjB,CAAA;YAED,IAAI,OAAO,EAAE,CAAC;gBACb,aAAa,CAAC,OAAO,CAAC,CAAA;YACvB,CAAC;YACD,MAAM,IAAI,GAAG,QAAQ,CAAC,yBAAyB,CAAC,aAAa,EAAE;gBAC9D,MAAM,EAAE,UAAU,CAAC,MAAM;gBACzB,OAAO,EAAE,CAAC,KAAK,EAAE,EAAE;oBAClB,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;wBACjB,MAAM,SAAS,GAA8B;4BAC5C,EAAE,EAAE,IAAI,CAAC,EAAE;4BACX,MAAM,EAAE,uBAAuB;4BAC/B,KAAK,EAAE,IAAI,CAAC,KAAK;4BACjB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC;4BACpD,OAAO,EAAE;gCACR;oCACC,KAAK,EAAE,CAAC;oCACR,KAAK,EAAE;wCACN,IAAI,EAAE,WAAW;wCACjB,OAAO,EAAE,KAAK,CAAC,IAAI;qCACnB;oCACD,QAAQ,EAAE,IAAI;oCACd,aAAa,EAAE,IAAI;iCACnB;6BACD;yBACD,CAAA;wBACD,GAAG,CAAC,KAAK,CAAC,SAAS,IAAI,CAAC,SAAS,CAAC,SAAS,CAAC,MAAM,CAAC,CAAA;oBACpD,CAAC;gBACF,CAAC;aACD,CAAC,CAAA;YAEF,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,MAAM,CAAA;YAEhC,OAAO,EAAE,CAAA;YAET,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;gBACjB,IAAI,MAAM,CAAC,YAAY,KAAK,WAAW,EAAE,CAAC;oBACzC,kDAAkD;oBAClD,sFAAsF;oBACtF,MAAM,qBAAqB,GAA8B;wBACxD,EAAE,EAAE,IAAI,CAAC,EAAE;wBACX,MAAM,EAAE,uBAAuB;wBAC/B,KAAK,EAAE,IAAI,CAAC,KAAK;wBACjB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC;wBACpD,OAAO,EAAE;4BACR;gCACC,KAAK,EAAE,CAAC;gCACR,KAAK,EAAE;oCACN,IAAI,EAAE,WAAW;oCACjB,OAAO,EAAE,IAAI;iCACb;gCACD,QAAQ,EAAE,IAAI;gCACd,aAAa,EAAE,MAAM,CAAC,YAAY;oCACjC,CAAC,CAAC,eAAe,CAAC,MAAM,CAAC,YAAY,CAAC;oCACtC,CAAC,CAAC,MAAM;6BACT;yBACD;qBACD,CAAA;oBAED,MAAM,SAAS,GACd,MAAM,CAAC,OAAO,CAAC,SAAU,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,KAAK,EAAE,EAAE;wBAC7C,OAAO;4BACN,KAAK;4BACL,EAAE,EAAE,IAAI,CAAC,EAAE;4BACX,IAAI,EAAE,UAAU;4BAChB,QAAQ,EAAE;gCACT,IAAI,EAAE,IAAI,CAAC,IAAI;gCACf,SAAS,EAAE,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,UAAU,CAAC;6BAC1C;yBACD,CAAA;oBACF,CAAC,CAAC,CAAA;oBACH,qBAAqB,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,UAAU,GAAG,SAAS,CAAA;oBAC7D,GAAG,CAAC,KAAK,CAAC,SAAS,IAAI,CAAC,SAAS,CAAC,qBAAqB,CAAC,MAAM,CAAC,CAAA;gBAChE,CAAC;gBACD,IAAI,IAAI,CAAC,cAAc,EAAE,aAAa,EAAE,CAAC;oBACxC,MAAM,UAAU,GAA8B;wBAC7C,EAAE,EAAE,IAAI,CAAC,EAAE;wBACX,MAAM,EAAE,uBAAuB;wBAC/B,KAAK,EAAE,IAAI,CAAC,KAAK;wBACjB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC;wBACpD,kBAAkB,EAAE,QAAQ,CAAC,WAAW;wBACxC,OAAO,EAAE;4BACR;gCACC,KAAK,EAAE,CAAC;gCACR,KAAK,EAAE,EAAE;gCACT,QAAQ,EAAE,IAAI;gCACd,aAAa,EAAE,MAAM,CAAC,YAAY;oCACjC,CAAC,CAAC,eAAe,CAAC,MAAM,CAAC,YAAY,CAAC;oCACtC,CAAC,CAAC,MAAM;6BACT;yBACD;wBACD,KAAK,EAAE;4BACN,aAAa,EAAE,MAAM,CAAC,YAAY;4BAClC,iBAAiB,EAAE,MAAM,CAAC,gBAAgB;4BAC1C,YAAY,EAAE,MAAM,CAAC,aAAa;yBAClC;qBACD,CAAA;oBACD,GAAG,CAAC,KAAK,CAAC,SAAS,IAAI,CAAC,SAAS,CAAC,UAAU,CAAC,MAAM,CAAC,CAAA;gBACrD,CAAC;gBACD,GAAG,CAAC,KAAK,CAAC,cAAc,CAAC,CAAA;gBACzB,GAAG,CAAC,GAAG,EAAE,CAAA;YACV,CAAC;iBAAM,CAAC;gBACP,MAAM,QAAQ,GAA0B;oBACvC,EAAE,EAAE,IAAI,CAAC,EAAE;oBACX,KAAK,EAAE,IAAI,CAAC,KAAK;oBACjB,MAAM,EAAE,iBAAiB;oBACzB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC;oBACpD,kBAAkB,EAAE,QAAQ,CAAC,WAAW;oBACxC,OAAO,EAAE;wBACR;4BACC,KAAK,EAAE,CAAC;4BACR,OAAO,EAAE;gCACR,IAAI,EAAE,WAAW;gCACjB,OAAO,EAAE,4BAA4B,CAAC,MAAM,CAAC,OAAO,CAAC,OAAO,CAAC;gCAC7D,OAAO,EAAE,IAAI;6BACb;4BACD,QAAQ,EAAE,IAAI;4BACd,aAAa,EAAE,MAAM,CAAC,YAAY;gCACjC,CAAC,CAAC,eAAe,CAAC,MAAM,CAAC,YAAY,CAAC;gCACtC,CAAC,CAAC,MAAM;yBACT;qBACD;oBACD,KAAK,EAAE;wBACN,aAAa,EAAE,MAAM,CAAC,YAAY;wBAClC,iBAAiB,EAAE,MAAM,CAAC,gBAAgB;wBAC1C,YAAY,EAAE,MAAM,CAAC,aAAa;qBAClC;iBACD,CAAA;gBACD,IACC,WAAW,IAAI,MAAM,CAAC,OAAO;oBAC7B,MAAM,CAAC,OAAO,CAAC,SAAS,EAAE,MAAM,EAC/B,CAAC;oBACF,QAAQ,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,UAAU;wBACrC,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE;4BACrC,OAAO;gCACN,EAAE,EAAE,IAAI,CAAC,EAAE;gCACX,IAAI,EAAE,UAAU;gCAChB,QAAQ,EAAE;oCACT,IAAI,EAAE,IAAI,CAAC,IAAI;oCACf,SAAS,EAAE,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,UAAU,CAAC;iCAC1C;6BACD,CAAA;wBACF,CAAC,CAAC,CAAA;gBACJ,CAAC;gBACD,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;gBAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAA;YAC3C,CAAC;QACF,CAAC;QAAC,OAAO,CAAC,EAAE,CAAC;YACZ,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAA;YAChB,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;gBACjB,GAAG,CAAC,KAAK,CAAC,eAAe,CAAC,CAAA;YAC3B,CAAC;iBAAM,CAAC;gBACP,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;gBAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,uBAAuB,EAAE,CAAC,CAAC,CAAA;YAC5D,CAAC;QACF,CAAC;IACF,CAAC,CAAA;AACF,CAAC"}
+{"version":3,"file":"chat.js","sourceRoot":"","sources":["../../../../src/api/openai/handlers/chat.ts"],"names":[],"mappings":"AAWA,OAAO,EAAE,oBAAoB,EAAE,MAAM,sCAAsC,CAAA;AAC3E,OAAO,EAAE,eAAe,EAAE,MAAM,sBAAsB,CAAA;AACtD,OAAO,EAAE,gBAAgB,EAAE,MAAM,2BAA2B,CAAA;AAC5D,OAAO,EAAE,eAAe,EAAE,cAAc,EAAE,MAAM,aAAa,CAAA;AAiB7D,KAAK,UAAU,uBAAuB,CACrC,QAAsC;IAEtC,MAAM,gBAAgB,GAAmC,EAAE,CAAA;IAC3D,MAAM,cAAc,GAAkB,EAAE,CAAA;IAExC,KAAK,MAAM,OAAO,IAAI,QAAQ,EAAE,CAAC;QAChC,MAAM,IAAI,GAAG,cAAc,CAAC,OAAO,CAAC,IAAI,CAAC,CAAA;QACzC,MAAM,aAAa,GAAQ;YAC1B,IAAI;YACJ,OAAO,EAAE,EAAE;SACX,CAAA;QACD,IAAI,IAAI,KAAK,MAAM,IAAI,cAAc,IAAI,OAAO,EAAE,CAAC;YAClD,aAAa,CAAC,MAAM,GAAG,OAAO,CAAC,YAAY,CAAA;QAC5C,CAAC;QAED,IAAI,OAAO,OAAO,CAAC,OAAO,KAAK,QAAQ,EAAE,CAAC;YACzC,aAAa,CAAC,OAAO,CAAC,IAAI,CAAC;gBAC1B,IAAI,EAAE,MAAM;gBACZ,IAAI,EAAE,OAAO,CAAC,OAAO;aACrB,CAAC,CAAA;QACH,CAAC;aAAM,IAAI,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC;YAC3C,KAAK,MAAM,IAAI,IAAI,OAAO,CAAC,OAAO,EAAE,CAAC;gBAEpC,IAAI,OAAO,IAAI,KAAK,QAAQ,EAAE,CAAC;oBAC9B,aAAa,CAAC,OAAO,CAAC,IAAI,CAAC;wBAC1B,IAAI,EAAE,MAAM;wBACZ,IAAI,EAAE,IAAI;qBACV,CAAC,CAAA;gBACH,CAAC;qBAAM,IAAI,IAAI,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;oBACjC,aAAa,CAAC,OAAO,CAAC,IAAI,CAAC;wBAC1B,IAAI,EAAE,MAAM;wBACZ,IAAI,EAAE,IAAI,CAAC,IAAI;qBACf,CAAC,CAAA;gBACH,CAAC;qBAAM,IAAI,IAAI,CAAC,IAAI,KAAK,WAAW,EAAE,CAAC;oBACtC,IAAI,CAAC,gBAAgB,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,EAAE,CAAC;wBAC3C,gBAAgB,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,GAAG,gBAAgB,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,CAAA;oBAC5E,CAAC;oBACD,MAAM,OAAO,GAAgC;wBAC5C,IAAI,EAAE,OAAO;qBACb,CAAA;oBACD,aAAa,CAAC,OAAO,CAAC,IAAI,CAAC,OAAO,CAAC,CAAA;oBACnC,gBAAgB,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,KAAK,EAAE,EAAE;wBACnD,OAAO,CAAC,KAAK,GAAG,KAAK,CAAA;oBACtB,CAAC,CAAC,CAAA;gBACH,CAAC;qBAAM,IAAI,IAAI,CAAC,IAAI,KAAK,aAAa,EAAE,CAAC;oBACxC,aAAa,CAAC,OAAO,CAAC,IAAI,CAAC;wBAC1B,IAAI,EAAE,OAAO;wBACb,KAAK,EAAE,IAAI,CAAC,WAAW;qBACvB,CAAC,CAAA;gBACH,CAAC;qBAAM,IAAI,IAAI,CAAC,IAAI,KAAK,SAAS,EAAE,CAAC;oBACpC,aAAa,CAAC,OAAO,CAAC,IAAI,CAAC;wBAC1B,IAAI,EAAE,MAAM;wBACZ,IAAI,EAAE,IAAI,CAAC,OAAO;qBAClB,CAAC,CAAA;gBACH,CAAC;YACF,CAAC;QACF,CAAC;aAAM,CAAC;YACP,MAAM,IAAI,KAAK,CAAC,yBAAyB,CAAC,CAAA;QAC3C,CAAC;QAED,cAAc,CAAC,IAAI,CAAC,aAAa,CAAC,CAAA;IACnC,CAAC;IAED,MAAM,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,gBAAgB,CAAC,CAAC,CAAA;IAElD,OAAO,cAAc,CAAA;AAEtB,CAAC;AAED,SAAS,4BAA4B,CACpC,OAAsC;IAEtC,IAAI,CAAC,OAAO,EAAE,CAAC;QACd,OAAO,IAAI,CAAA;IACZ,CAAC;IACD,IAAI,OAAO,OAAO,KAAK,QAAQ,EAAE,CAAC;QACjC,OAAO,OAAO,CAAA;IACf,CAAC;IACD,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC;QAC7B,MAAM,IAAI,KAAK,CAAC,kCAAkC,CAAC,CAAA;IACpD,CAAC;IAED,IAAI,IAAI,GAAG,EAAE,CAAA;IACb,KAAK,MAAM,IAAI,IAAI,OAAO,EAAE,CAAC;QAC5B,IAAI,IAAI,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;YAC1B,IAAI,IAAI,IAAI,CAAC,IAAI,CAAA;QAClB,CAAC;QACD,kEAAkE;IACnE,CAAC;IACD,OAAO,IAAI,CAAA;AACZ,CAAC;AAED,MAAM,UAAU,2BAA2B,CAAC,eAAgC;IAC3E,OAAO,KAAK,EAAE,GAAoB,EAAE,GAAmB,EAAE,EAAE;QAC1D,IAAI,IAAgC,CAAA;QAEpC,IAAI,CAAC;YACJ,MAAM,IAAI,GAAG,MAAM,oBAAoB,CAAC,GAAG,CAAC,CAAA;YAC5C,IAAI,GAAG,IAAI,CAAA;QACZ,CAAC;QAAC,OAAO,CAAC,EAAE,CAAC;YACZ,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAA;YAChB,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,iBAAiB,EAAE,CAAC,CAAC,CAAA;YACrD,OAAM;QACP,CAAC;QAED,8BAA8B;QAC9B,IAAI,CAAC,IAAI,CAAC,KAAK,IAAI,CAAC,IAAI,CAAC,QAAQ,EAAE,CAAC;YACnC,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,oDAAoD,EAAE,CAAC,CAAC,CAAA;YACxF,OAAM;QACP,CAAC;QAED,IAAI,CAAC,eAAe,CAAC,WAAW,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;YAC9C,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,sBAAsB,EAAE,CAAC,CAAC,CAAA;YAC1D,OAAM;QACP,CAAC;QAED,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAA;QACxC,GAAG,CAAC,EAAE,CAAC,OAAO,EAAE,GAAG,EAAE;YACpB,OAAO,CAAC,KAAK,CAAC,0BAA0B,CAAC,CAAA;YACzC,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QACF,GAAG,CAAC,EAAE,CAAC,KAAK,EAAE,GAAG,EAAE;YAClB,OAAO,CAAC,KAAK,CAAC,yBAAyB,CAAC,CAAA;YACxC,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QACF,GAAG,CAAC,EAAE,CAAC,SAAS,EAAE,GAAG,EAAE;YACtB,OAAO,CAAC,KAAK,CAAC,2BAA2B,CAAC,CAAA;YAC1C,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QACF,GAAG,CAAC,EAAE,CAAC,OAAO,EAAE,GAAG,EAAE;YACpB,OAAO,CAAC,KAAK,CAAC,cAAc,CAAC,CAAA;YAC7B,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QAEF,IAAI,CAAC;YACJ,IAAI,OAAmC,CAAA;YACvC,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;gBACjB,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE;oBAClB,cAAc,EAAE,mBAAmB;oBACnC,eAAe,EAAE,UAAU;oBAC3B,UAAU,EAAE,YAAY;iBACxB,CAAC,CAAA;gBACF,GAAG,CAAC,YAAY,EAAE,CAAA;gBAClB,OAAO,GAAG,WAAW,CAAC,GAAG,EAAE;oBAC1B,GAAG,CAAC,KAAK,CAAC,WAAW,CAAC,CAAA;gBACvB,CAAC,EAAE,KAAK,CAAC,CAAA;YACV,CAAC;YAED,IAAI,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS,CAAA;YAC5C,IAAI,OAAO,IAAI,KAAK,QAAQ,EAAE,CAAC;gBAC9B,IAAI,GAAG,CAAC,IAAI,CAAC,CAAA;YACd,CAAC;YAED,IAAI,iBAAqC,CAAA;YACzC,IAAI,IAAI,CAAC,eAAe,EAAE,CAAC;gBAC1B,IAAI,IAAI,CAAC,eAAe,CAAC,IAAI,KAAK,aAAa,EAAE,CAAC;oBACjD,iBAAiB,GAAG,MAAM,CAAA;gBAC3B,CAAC;YACF,CAAC;YAED,IAAI,eAEQ,CAAA;YAEZ,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;gBAChB,MAAM,aAAa,GAAG,IAAI,CAAC,KAAK;qBAC9B,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,KAAK,UAAU,CAAC;qBAC1C,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE;oBACb,OAAO;wBACN,IAAI,EAAE,IAAI,CAAC,QAAQ,CAAC,IAAI;wBACxB,WAAW,EAAE,IAAI,CAAC,QAAQ,CAAC,WAAW;wBACtC,UAAU,EAAE,IAAI,CAAC,QAAQ,CAAC,UAAU;qBACpC,CAAA;gBACF,CAAC,CAAC,CAAA;gBACH,IAAI,aAAa,CAAC,MAAM,EAAE,CAAC;oBAC1B,IAAI,CAAC,eAAe,EAAE,CAAC;wBACtB,eAAe,GAAG,EAAE,CAAA;oBACrB,CAAC;oBACD,KAAK,MAAM,IAAI,IAAI,aAAa,EAAE,CAAC;wBAClC,eAAe,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG;4BAC5B,WAAW,EAAE,IAAI,CAAC,WAAW;4BAC7B,UAAU,EAAE,IAAI,CAAC,UAAU;yBACT,CAAA;oBACpB,CAAC;gBACF,CAAC;YACF,CAAC;YAED,MAAM,QAAQ,GAAG,MAAM,uBAAuB,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAA;YAC7D,MAAM,aAAa,GAAG,eAAe,CAAuB;gBAC3D,KAAK,EAAE,IAAI,CAAC,KAAK;gBACjB,QAAQ;gBACR,WAAW,EAAE,IAAI,CAAC,WAAW,CAAC,CAAC,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC,CAAC,SAAS;gBAC5D,sDAAsD;gBACtD,SAAS,EAAE,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,SAAS;gBACxD,IAAI,EAAE,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS;gBACvC,IAAI;gBACJ,gBAAgB,EAAE,IAAI,CAAC,iBAAiB;oBACvC,CAAC,CAAC,IAAI,CAAC,iBAAiB;oBACxB,CAAC,CAAC,SAAS;gBACZ,eAAe,EAAE,IAAI,CAAC,gBAAgB;oBACrC,CAAC,CAAC,IAAI,CAAC,gBAAgB;oBACvB,CAAC,CAAC,SAAS;gBACZ,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS;gBACzC,SAAS,EAAE,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,SAAS;gBACxD,OAAO,EAAE,iBAAiB;gBAC1B,KAAK,EAAE,eAAe,CAAC,CAAC,CAAC,EAAE,WAAW,EAAE,eAAe,EAAE,CAAC,CAAC,CAAC,SAAS;gBACrE,6BAA6B;gBAC7B,gBAAgB,EAAE,IAAI,CAAC,kBAAkB;oBACxC,CAAC,CAAC,IAAI,CAAC,kBAAkB;oBACzB,CAAC,CAAC,SAAS;gBACZ,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS;gBACzC,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS;aAEzC,CAAC,CAAA;YACF,MAAM,EAAE,QAAQ,EAAE,OAAO,EAAE,GAAG,MAAM,eAAe,CAAC,eAAe,CAClE,aAAa,EACb,UAAU,CAAC,MAAM,CACjB,CAAA;YAED,IAAI,OAAO,EAAE,CAAC;gBACb,aAAa,CAAC,OAAO,CAAC,CAAA;YACvB,CAAC;YACD,MAAM,IAAI,GAAG,QAAQ,CAAC,yBAAyB,CAAC;gBAC/C,GAAG,aAAa;gBAChB,MAAM,EAAE,UAAU,CAAC,MAAM;gBACzB,OAAO,EAAE,CAAC,KAAK,EAAE,EAAE;oBAClB,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;wBACjB,MAAM,SAAS,GAA8B;4BAC5C,EAAE,EAAE,IAAI,CAAC,EAAE;4BACX,MAAM,EAAE,uBAAuB;4BAC/B,KAAK,EAAE,IAAI,CAAC,KAAK;4BACjB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC;4BACpD,OAAO,EAAE;gCACR;oCACC,KAAK,EAAE,CAAC;oCACR,KAAK,EAAE;wCACN,IAAI,EAAE,WAAW;wCACjB,OAAO,EAAE,KAAK,CAAC,IAAI;qCACnB;oCACD,QAAQ,EAAE,IAAI;oCACd,aAAa,EAAE,IAAI;iCACnB;6BACD;yBACD,CAAA;wBACD,GAAG,CAAC,KAAK,CAAC,SAAS,IAAI,CAAC,SAAS,CAAC,SAAS,CAAC,MAAM,CAAC,CAAA;oBACpD,CAAC;gBACF,CAAC;aACD,CAAC,CAAA;YACF,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,MAAM,CAAA;YAChC,OAAO,EAAE,CAAA;YAET,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;gBACjB,IAAI,MAAM,CAAC,YAAY,KAAK,WAAW,EAAE,CAAC;oBACzC,kDAAkD;oBAClD,sFAAsF;oBACtF,MAAM,qBAAqB,GAA8B;wBACxD,EAAE,EAAE,IAAI,CAAC,EAAE;wBACX,MAAM,EAAE,uBAAuB;wBAC/B,KAAK,EAAE,IAAI,CAAC,KAAK;wBACjB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC;wBACpD,OAAO,EAAE;4BACR;gCACC,KAAK,EAAE,CAAC;gCACR,KAAK,EAAE;oCACN,IAAI,EAAE,WAAW;oCACjB,OAAO,EAAE,IAAI;iCACb;gCACD,QAAQ,EAAE,IAAI;gCACd,aAAa,EAAE,MAAM,CAAC,YAAY;oCACjC,CAAC,CAAC,eAAe,CAAC,MAAM,CAAC,YAAY,CAAC;oCACtC,CAAC,CAAC,MAAM;6BACT;yBACD;qBACD,CAAA;oBAED,MAAM,SAAS,GACd,MAAM,CAAC,OAAO,CAAC,SAAU,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,KAAK,EAAE,EAAE;wBAC7C,OAAO;4BACN,KAAK;4BACL,EAAE,EAAE,IAAI,CAAC,EAAE;4BACX,IAAI,EAAE,UAAU;4BAChB,QAAQ,EAAE;gCACT,IAAI,EAAE,IAAI,CAAC,IAAI;gCACf,SAAS,EAAE,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,UAAU,CAAC;6BAC1C;yBACD,CAAA;oBACF,CAAC,CAAC,CAAA;oBACH,qBAAqB,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,UAAU,GAAG,SAAS,CAAA;oBAC7D,GAAG,CAAC,KAAK,CAAC,SAAS,IAAI,CAAC,SAAS,CAAC,qBAAqB,CAAC,MAAM,CAAC,CAAA;gBAChE,CAAC;gBACD,IAAI,IAAI,CAAC,cAAc,EAAE,aAAa,EAAE,CAAC;oBACxC,MAAM,UAAU,GAA8B;wBAC7C,EAAE,EAAE,IAAI,CAAC,EAAE;wBACX,MAAM,EAAE,uBAAuB;wBAC/B,KAAK,EAAE,IAAI,CAAC,KAAK;wBACjB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC;wBACpD,kBAAkB,EAAE,QAAQ,CAAC,WAAW;wBACxC,OAAO,EAAE;4BACR;gCACC,KAAK,EAAE,CAAC;gCACR,KAAK,EAAE,EAAE;gCACT,QAAQ,EAAE,IAAI;gCACd,aAAa,EAAE,MAAM,CAAC,YAAY;oCACjC,CAAC,CAAC,eAAe,CAAC,MAAM,CAAC,YAAY,CAAC;oCACtC,CAAC,CAAC,MAAM;6BACT;yBACD;wBACD,KAAK,EAAE;4BACN,aAAa,EAAE,MAAM,CAAC,YAAY;4BAClC,iBAAiB,EAAE,MAAM,CAAC,gBAAgB;4BAC1C,YAAY,EAAE,MAAM,CAAC,aAAa;yBAClC;qBACD,CAAA;oBACD,GAAG,CAAC,KAAK,CAAC,SAAS,IAAI,CAAC,SAAS,CAAC,UAAU,CAAC,MAAM,CAAC,CAAA;gBACrD,CAAC;gBACD,GAAG,CAAC,KAAK,CAAC,cAAc,CAAC,CAAA;gBACzB,GAAG,CAAC,GAAG,EAAE,CAAA;YACV,CAAC;iBAAM,CAAC;gBACP,MAAM,QAAQ,GAA0B;oBACvC,EAAE,EAAE,IAAI,CAAC,EAAE;oBACX,KAAK,EAAE,IAAI,CAAC,KAAK;oBACjB,MAAM,EAAE,iBAAiB;oBACzB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC;oBACpD,kBAAkB,EAAE,QAAQ,CAAC,WAAW;oBACxC,OAAO,EAAE;wBACR;4BACC,KAAK,EAAE,CAAC;4BACR,OAAO,EAAE;gCACR,IAAI,EAAE,WAAW;gCACjB,OAAO,EAAE,4BAA4B,CAAC,MAAM,CAAC,OAAO,CAAC,OAAO,CAAC;gCAC7D,OAAO,EAAE,IAAI;6BACb;4BACD,QAAQ,EAAE,IAAI;4BACd,aAAa,EAAE,MAAM,CAAC,YAAY;gCACjC,CAAC,CAAC,eAAe,CAAC,MAAM,CAAC,YAAY,CAAC;gCACtC,CAAC,CAAC,MAAM;yBACT;qBACD;oBACD,KAAK,EAAE;wBACN,aAAa,EAAE,MAAM,CAAC,YAAY;wBAClC,iBAAiB,EAAE,MAAM,CAAC,gBAAgB;wBAC1C,YAAY,EAAE,MAAM,CAAC,aAAa;qBAClC;iBACD,CAAA;gBACD,IACC,WAAW,IAAI,MAAM,CAAC,OAAO;oBAC7B,MAAM,CAAC,OAAO,CAAC,SAAS,EAAE,MAAM,EAC/B,CAAC;oBACF,QAAQ,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,UAAU;wBACrC,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE;4BACrC,OAAO;gCACN,EAAE,EAAE,IAAI,CAAC,EAAE;gCACX,IAAI,EAAE,UAAU;gCAChB,QAAQ,EAAE;oCACT,IAAI,EAAE,IAAI,CAAC,IAAI;oCACf,SAAS,EAAE,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,UAAU,CAAC;iCAC1C;6BACD,CAAA;wBACF,CAAC,CAAC,CAAA;gBACJ,CAAC;gBACD,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;gBAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAA;YAC3C,CAAC;QACF,CAAC;QAAC,OAAO,CAAC,EAAE,CAAC;YACZ,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAA;YAChB,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;gBACjB,GAAG,CAAC,KAAK,CAAC,eAAe,CAAC,CAAA;YAC3B,CAAC;iBAAM,CAAC;gBACP,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;gBAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,uBAAuB,EAAE,CAAC,CAAC,CAAA;YAC5D,CAAC;QACF,CAAC;IACF,CAAC,CAAA;AACF,CAAC"}

package/dist/api/openai/handlers/completions.d.ts

@@ -1,3 +1,3 @@
 import type { IncomingMessage, ServerResponse } from 'node:http';
-import type { ModelServer } from '../../../server.js';
-export declare function createCompletionHandler(modelServer: ModelServer): (req: IncomingMessage, res: ServerResponse) => Promise<void>;
+import type { InferenceServer } from '../../../server.js';
+export declare function createCompletionHandler(inferenceServer: InferenceServer): (req: IncomingMessage, res: ServerResponse) => Promise<void>;

package/dist/api/openai/handlers/completions.js

@@ -1,7 +1,7 @@
 import { parseJSONRequestBody } from '../../../api/parseJSONRequestBody.js';
 import { omitEmptyValues } from '../../../lib/util.js';
 import { finishReasonMap } from '../enums.js';
-export function createCompletionHandler(modelServer) {
+export function createCompletionHandler(inferenceServer) {
     return async (req, res) => {
         let args;
         try {
@@ -20,7 +20,7 @@ export function createCompletionHandler(modelServer) {
             res.end(JSON.stringify({ error: 'Invalid request' }));
             return;
         }
-        if (!modelServer.modelExists(args.model)) {
+        if (!inferenceServer.modelExists(args.model)) {
             res.writeHead(400, { 'Content-Type': 'application/json' });
             res.end(JSON.stringify({ error: 'Invalid model' }));
             return;
@@ -55,7 +55,6 @@ export function createCompletionHandler(modelServer) {
                 model: args.model,
                 prompt: args.prompt,
                 temperature: args.temperature ? args.temperature : undefined,
-                stream: args.stream ? Boolean(args.stream) : false,
                 maxTokens: args.max_tokens ? args.max_tokens : undefined,
                 seed: args.seed ? args.seed : undefined,
                 stop,
@@ -74,8 +73,9 @@ export function createCompletionHandler(modelServer) {
                 minP: args.min_p ? args.min_p : undefined,
                 topK: args.top_k ? args.top_k : undefined,
             });
-            const { instance, release } = await modelServer.requestInstance(completionReq, controller.signal);
-            const task = instance.processTextCompletionTask(completionReq, {
+            const { instance, release } = await inferenceServer.requestInstance(completionReq, controller.signal);
+            const task = instance.processTextCompletionTask({
+                ...completionReq,
                 signal: controller.signal,
                 onChunk: (chunk) => {
                     if (args.stream) {

package/dist/api/openai/handlers/completions.js.map

@@ -1 +1 @@
-{"version":3,"file":"completions.js","sourceRoot":"","sources":["../../../../src/api/openai/handlers/completions.ts"],"names":[],"mappings":"AAIA,OAAO,EAAE,oBAAoB,EAAE,MAAM,sCAAsC,CAAA;AAC3E,OAAO,EAAE,eAAe,EAAE,MAAM,sBAAsB,CAAA;AACtD,OAAO,EAAE,eAAe,EAAE,MAAM,aAAa,CAAA;AAiB7C,MAAM,UAAU,uBAAuB,CAAC,WAAwB;IAC/D,OAAO,KAAK,EAAE,GAAoB,EAAE,GAAmB,EAAE,EAAE;QAC1D,IAAI,IAA4B,CAAA;QAEhC,IAAI,CAAC;YACJ,MAAM,IAAI,GAAG,MAAM,oBAAoB,CAAC,GAAG,CAAC,CAAA;YAC5C,IAAI,GAAG,IAAI,CAAA;QACZ,CAAC;QAAC,OAAO,CAAC,EAAE,CAAC;YACZ,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAA;YAChB,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,iBAAiB,EAAE,CAAC,CAAC,CAAA;YACrD,OAAM;QACP,CAAC;QAED,8BAA8B;QAC9B,IAAI,CAAC,IAAI,CAAC,KAAK,IAAI,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC;YACjC,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,iBAAiB,EAAE,CAAC,CAAC,CAAA;YACrD,OAAM;QACP,CAAC;QACD,IAAI,CAAC,WAAW,CAAC,WAAW,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;YAC1C,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,eAAe,EAAE,CAAC,CAAC,CAAA;YACnD,OAAM;QACP,CAAC;QAED,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAA;QACxC,GAAG,CAAC,EAAE,CAAC,OAAO,EAAE,GAAG,EAAE;YACpB,OAAO,CAAC,KAAK,CAAC,0BAA0B,CAAC,CAAA;YACzC,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QACF,GAAG,CAAC,EAAE,CAAC,KAAK,EAAE,GAAG,EAAE;YAClB,OAAO,CAAC,KAAK,CAAC,yBAAyB,CAAC,CAAA;YACxC,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QAEF,IAAI,CAAC;YACJ,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;gBACjB,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE;oBAClB,cAAc,EAAE,mBAAmB;oBACnC,eAAe,EAAE,UAAU;oBAC3B,UAAU,EAAE,YAAY;iBACxB,CAAC,CAAA;gBACF,GAAG,CAAC,YAAY,EAAE,CAAA;YACnB,CAAC;YAED,IAAI,MAAM,GAAG,IAAI,CAAC,MAAM,CAAA;YAExB,IAAI,OAAO,MAAM,KAAK,QAAQ,EAAE,CAAC;gBAChC,MAAM,IAAI,KAAK,CAAC,yBAAyB,CAAC,CAAA;YAC3C,CAAC;YAED,IAAI,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS,CAAA;YAC5C,IAAI,OAAO,IAAI,KAAK,QAAQ,EAAE,CAAC;gBAC9B,IAAI,GAAG,CAAC,IAAI,CAAC,CAAA;YACd,CAAC;YAED,MAAM,aAAa,GAAG,eAAe,CAAwB;gBAC5D,KAAK,EAAE,IAAI,CAAC,KAAK;gBACjB,MAAM,EAAE,IAAI,CAAC,MAAgB;gBAC7B,WAAW,EAAE,IAAI,CAAC,WAAW,CAAC,CAAC,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC,CAAC,SAAS;gBAC5D,MAAM,EAAE,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,KAAK;gBAClD,SAAS,EAAE,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,SAAS;gBACxD,IAAI,EAAE,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS;gBACvC,IAAI;gBACJ,gBAAgB,EAAE,IAAI,CAAC,iBAAiB;oBACvC,CAAC,CAAC,IAAI,CAAC,iBAAiB;oBACxB,CAAC,CAAC,SAAS;gBACZ,eAAe,EAAE,IAAI,CAAC,gBAAgB;oBACrC,CAAC,CAAC,IAAI,CAAC,gBAAgB;oBACvB,CAAC,CAAC,SAAS;gBACZ,SAAS,EAAE,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,SAAS;gBACxD,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS;gBACzC,6BAA6B;gBAC7B,gBAAgB,EAAE,IAAI,CAAC,kBAAkB;oBACxC,CAAC,CAAC,IAAI,CAAC,kBAAkB;oBACzB,CAAC,CAAC,SAAS;gBACZ,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS;gBACzC,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS;aACzC,CAAC,CAAA;YAEF,MAAM,EAAE,QAAQ,EAAE,OAAO,EAAE,GAAG,MAAM,WAAW,CAAC,eAAe,CAC9D,aAAa,EACb,UAAU,CAAC,MAAM,CACjB,CAAA;YACD,MAAM,IAAI,GAAG,QAAQ,CAAC,yBAAyB,CAAC,aAAa,EAAE;gBAC9D,MAAM,EAAE,UAAU,CAAC,MAAM;gBACzB,OAAO,EAAE,CAAC,KAAK,EAAE,EAAE;oBAClB,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;wBACjB,MAAM,SAAS,GAA0B;4BACxC,EAAE,EAAE,IAAI,CAAC,EAAE;4BACX,KAAK,EAAE,IAAI,CAAC,KAAK;4BACjB,MAAM,EAAE,iBAAiB;4BACzB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC;4BACpD,OAAO,EAAE;gCACR;oCACC,KAAK,EAAE,CAAC;oCACR,IAAI,EAAE,KAAK,CAAC,IAAI;oCAChB,QAAQ,EAAE,IAAI;oCACd,6DAA6D;oCAC7D,aAAa,EAAE,IAAI;iCACnB;6BACD;yBACD,CAAA;wBACD,GAAG,CAAC,KAAK,CAAC,SAAS,IAAI,CAAC,SAAS,CAAC,SAAS,CAAC,MAAM,CAAC,CAAA;oBACpD,CAAC;gBACF,CAAC;aACD,CAAC,CAAA;YACF,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,MAAM,CAAA;YAChC,OAAO,EAAE,CAAA;YAET,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;gBACjB,IAAI,IAAI,CAAC,cAAc,EAAE,aAAa,EAAE,CAAC;oBACxC,MAAM,UAAU,GAA0B;wBACzC,EAAE,EAAE,IAAI,CAAC,EAAE;wBACX,KAAK,EAAE,IAAI,CAAC,KAAK;wBACjB,MAAM,EAAE,iBAAiB;wBACzB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC;wBACpD,OAAO,EAAE;4BACR;gCACC,KAAK,EAAE,CAAC;gCACR,IAAI,EAAE,EAAE;gCACR,QAAQ,EAAE,IAAI;gCACd,aAAa;gCACb,aAAa,EAAE,MAAM,CAAC,YAAY;oCACjC,CAAC,CAAC,eAAe,CAAC,MAAM,CAAC,YAAY,CAAC;oCACtC,CAAC,CAAC,MAAM;6BACT;yBACD;qBACD,CAAA;oBACD,GAAG,CAAC,KAAK,CACR,SAAS,IAAI,CAAC,SAAS,CAAC,UAAU,CAAC,MAAM,CACzC,CAAA;gBACF,CAAC;gBACD,GAAG,CAAC,KAAK,CAAC,cAAc,CAAC,CAAA;gBACzB,GAAG,CAAC,GAAG,EAAE,CAAA;YACV,CAAC;iBAAM,CAAC;gBACP,MAAM,QAAQ,GAAkC;oBAC/C,EAAE,EAAE,IAAI,CAAC,EAAE;oBACX,KAAK,EAAE,IAAI,CAAC,KAAK;oBACjB,MAAM,EAAE,iBAAiB;oBACzB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC;oBACpD,kBAAkB,EAAE,QAAQ,CAAC,WAAW;oBACxC,OAAO,EAAE;wBACR;4BACC,KAAK,EAAE,CAAC;4BACR,IAAI,EAAE,MAAM,CAAC,IAAI;4BACjB,QAAQ,EAAE,IAAI;4BACd,aAAa;4BACb,aAAa,EAAE,MAAM,CAAC,YAAY;gCACjC,CAAC,CAAC,eAAe,CAAC,MAAM,CAAC,YAAY,CAAC;gCACtC,CAAC,CAAC,MAAM;yBACT;qBACD;oBACD,KAAK,EAAE;wBACN,aAAa,EAAE,MAAM,CAAC,YAAY;wBAClC,iBAAiB,EAAE,MAAM,CAAC,gBAAgB;wBAC1C,YAAY,EAAE,MAAM,CAAC,aAAa;qBAClC;iBACD,CAAA;gBACD,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;gBAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAA;YAC3C,CAAC;QACF,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACd,OAAO,CAAC,KAAK,CAAC,GAAG,CAAC,CAAA;YAClB,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;gBACjB,GAAG,CAAC,KAAK,CAAC,eAAe,CAAC,CAAA;YAC3B,CAAC;iBAAM,CAAC;gBACP,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;gBAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,uBAAuB,EAAE,CAAC,CAAC,CAAA;YAC5D,CAAC;QACF,CAAC;IACF,CAAC,CAAA;AACF,CAAC"}
+{"version":3,"file":"completions.js","sourceRoot":"","sources":["../../../../src/api/openai/handlers/completions.ts"],"names":[],"mappings":"AAIA,OAAO,EAAE,oBAAoB,EAAE,MAAM,sCAAsC,CAAA;AAC3E,OAAO,EAAE,eAAe,EAAE,MAAM,sBAAsB,CAAA;AACtD,OAAO,EAAE,eAAe,EAAE,MAAM,aAAa,CAAA;AAiB7C,MAAM,UAAU,uBAAuB,CAAC,eAAgC;IACvE,OAAO,KAAK,EAAE,GAAoB,EAAE,GAAmB,EAAE,EAAE;QAC1D,IAAI,IAA4B,CAAA;QAEhC,IAAI,CAAC;YACJ,MAAM,IAAI,GAAG,MAAM,oBAAoB,CAAC,GAAG,CAAC,CAAA;YAC5C,IAAI,GAAG,IAAI,CAAA;QACZ,CAAC;QAAC,OAAO,CAAC,EAAE,CAAC;YACZ,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAA;YAChB,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,iBAAiB,EAAE,CAAC,CAAC,CAAA;YACrD,OAAM;QACP,CAAC;QAED,8BAA8B;QAC9B,IAAI,CAAC,IAAI,CAAC,KAAK,IAAI,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC;YACjC,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,iBAAiB,EAAE,CAAC,CAAC,CAAA;YACrD,OAAM;QACP,CAAC;QACD,IAAI,CAAC,eAAe,CAAC,WAAW,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;YAC9C,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,eAAe,EAAE,CAAC,CAAC,CAAA;YACnD,OAAM;QACP,CAAC;QAED,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAA;QACxC,GAAG,CAAC,EAAE,CAAC,OAAO,EAAE,GAAG,EAAE;YACpB,OAAO,CAAC,KAAK,CAAC,0BAA0B,CAAC,CAAA;YACzC,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QACF,GAAG,CAAC,EAAE,CAAC,KAAK,EAAE,GAAG,EAAE;YAClB,OAAO,CAAC,KAAK,CAAC,yBAAyB,CAAC,CAAA;YACxC,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QAEF,IAAI,CAAC;YACJ,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;gBACjB,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE;oBAClB,cAAc,EAAE,mBAAmB;oBACnC,eAAe,EAAE,UAAU;oBAC3B,UAAU,EAAE,YAAY;iBACxB,CAAC,CAAA;gBACF,GAAG,CAAC,YAAY,EAAE,CAAA;YACnB,CAAC;YAED,IAAI,MAAM,GAAG,IAAI,CAAC,MAAM,CAAA;YAExB,IAAI,OAAO,MAAM,KAAK,QAAQ,EAAE,CAAC;gBAChC,MAAM,IAAI,KAAK,CAAC,yBAAyB,CAAC,CAAA;YAC3C,CAAC;YAED,IAAI,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS,CAAA;YAC5C,IAAI,OAAO,IAAI,KAAK,QAAQ,EAAE,CAAC;gBAC9B,IAAI,GAAG,CAAC,IAAI,CAAC,CAAA;YACd,CAAC;YAED,MAAM,aAAa,GAAG,eAAe,CAAuB;gBAC3D,KAAK,EAAE,IAAI,CAAC,KAAK;gBACjB,MAAM,EAAE,IAAI,CAAC,MAAgB;gBAC7B,WAAW,EAAE,IAAI,CAAC,WAAW,CAAC,CAAC,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC,CAAC,SAAS;gBAC5D,SAAS,EAAE,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,SAAS;gBACxD,IAAI,EAAE,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS;gBACvC,IAAI;gBACJ,gBAAgB,EAAE,IAAI,CAAC,iBAAiB;oBACvC,CAAC,CAAC,IAAI,CAAC,iBAAiB;oBACxB,CAAC,CAAC,SAAS;gBACZ,eAAe,EAAE,IAAI,CAAC,gBAAgB;oBACrC,CAAC,CAAC,IAAI,CAAC,gBAAgB;oBACvB,CAAC,CAAC,SAAS;gBACZ,SAAS,EAAE,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,SAAS;gBACxD,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS;gBACzC,6BAA6B;gBAC7B,gBAAgB,EAAE,IAAI,CAAC,kBAAkB;oBACxC,CAAC,CAAC,IAAI,CAAC,kBAAkB;oBACzB,CAAC,CAAC,SAAS;gBACZ,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS;gBACzC,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS;aACzC,CAAC,CAAA;YAEF,MAAM,EAAE,QAAQ,EAAE,OAAO,EAAE,GAAG,MAAM,eAAe,CAAC,eAAe,CAClE,aAAa,EACb,UAAU,CAAC,MAAM,CACjB,CAAA;YACD,MAAM,IAAI,GAAG,QAAQ,CAAC,yBAAyB,CAAC;gBAC/C,GAAG,aAAa;gBAChB,MAAM,EAAE,UAAU,CAAC,MAAM;gBACzB,OAAO,EAAE,CAAC,KAAK,EAAE,EAAE;oBAClB,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;wBACjB,MAAM,SAAS,GAA0B;4BACxC,EAAE,EAAE,IAAI,CAAC,EAAE;4BACX,KAAK,EAAE,IAAI,CAAC,KAAK;4BACjB,MAAM,EAAE,iBAAiB;4BACzB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC;4BACpD,OAAO,EAAE;gCACR;oCACC,KAAK,EAAE,CAAC;oCACR,IAAI,EAAE,KAAK,CAAC,IAAI;oCAChB,QAAQ,EAAE,IAAI;oCACd,6DAA6D;oCAC7D,aAAa,EAAE,IAAI;iCACnB;6BACD;yBACD,CAAA;wBACD,GAAG,CAAC,KAAK,CAAC,SAAS,IAAI,CAAC,SAAS,CAAC,SAAS,CAAC,MAAM,CAAC,CAAA;oBACpD,CAAC;gBACF,CAAC;aACD,CAAC,CAAA;YACF,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,MAAM,CAAA;YAChC,OAAO,EAAE,CAAA;YAET,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;gBACjB,IAAI,IAAI,CAAC,cAAc,EAAE,aAAa,EAAE,CAAC;oBACxC,MAAM,UAAU,GAA0B;wBACzC,EAAE,EAAE,IAAI,CAAC,EAAE;wBACX,KAAK,EAAE,IAAI,CAAC,KAAK;wBACjB,MAAM,EAAE,iBAAiB;wBACzB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC;wBACpD,OAAO,EAAE;4BACR;gCACC,KAAK,EAAE,CAAC;gCACR,IAAI,EAAE,EAAE;gCACR,QAAQ,EAAE,IAAI;gCACd,aAAa;gCACb,aAAa,EAAE,MAAM,CAAC,YAAY;oCACjC,CAAC,CAAC,eAAe,CAAC,MAAM,CAAC,YAAY,CAAC;oCACtC,CAAC,CAAC,MAAM;6BACT;yBACD;qBACD,CAAA;oBACD,GAAG,CAAC,KAAK,CACR,SAAS,IAAI,CAAC,SAAS,CAAC,UAAU,CAAC,MAAM,CACzC,CAAA;gBACF,CAAC;gBACD,GAAG,CAAC,KAAK,CAAC,cAAc,CAAC,CAAA;gBACzB,GAAG,CAAC,GAAG,EAAE,CAAA;YACV,CAAC;iBAAM,CAAC;gBACP,MAAM,QAAQ,GAAkC;oBAC/C,EAAE,EAAE,IAAI,CAAC,EAAE;oBACX,KAAK,EAAE,IAAI,CAAC,KAAK;oBACjB,MAAM,EAAE,iBAAiB;oBACzB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC;oBACpD,kBAAkB,EAAE,QAAQ,CAAC,WAAW;oBACxC,OAAO,EAAE;wBACR;4BACC,KAAK,EAAE,CAAC;4BACR,IAAI,EAAE,MAAM,CAAC,IAAI;4BACjB,QAAQ,EAAE,IAAI;4BACd,aAAa;4BACb,aAAa,EAAE,MAAM,CAAC,YAAY;gCACjC,CAAC,CAAC,eAAe,CAAC,MAAM,CAAC,YAAY,CAAC;gCACtC,CAAC,CAAC,MAAM;yBACT;qBACD;oBACD,KAAK,EAAE;wBACN,aAAa,EAAE,MAAM,CAAC,YAAY;wBAClC,iBAAiB,EAAE,MAAM,CAAC,gBAAgB;wBAC1C,YAAY,EAAE,MAAM,CAAC,aAAa;qBAClC;iBACD,CAAA;gBACD,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;gBAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAA;YAC3C,CAAC;QACF,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACd,OAAO,CAAC,KAAK,CAAC,GAAG,CAAC,CAAA;YAClB,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;gBACjB,GAAG,CAAC,KAAK,CAAC,eAAe,CAAC,CAAA;YAC3B,CAAC;iBAAM,CAAC;gBACP,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;gBAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,uBAAuB,EAAE,CAAC,CAAC,CAAA;YAC5D,CAAC;QACF,CAAC;IACF,CAAC,CAAA;AACF,CAAC"}

package/dist/api/openai/handlers/embeddings.d.ts

@@ -1,3 +1,3 @@
 import { IncomingMessage, ServerResponse } from 'node:http';
-import { ModelServer } from '../../../server.js';
-export declare function createEmbeddingsHandler(modelServer: ModelServer): (req: IncomingMessage, res: ServerResponse) => Promise<void>;
+import { InferenceServer } from '../../../server.js';
+export declare function createEmbeddingsHandler(inferenceServer: InferenceServer): (req: IncomingMessage, res: ServerResponse) => Promise<void>;

package/dist/api/openai/handlers/embeddings.js

@@ -1,6 +1,6 @@
 import { parseJSONRequestBody } from '../../../api/parseJSONRequestBody.js';
 import { omitEmptyValues } from '../../../lib/util.js';
-export function createEmbeddingsHandler(modelServer) {
+export function createEmbeddingsHandler(inferenceServer) {
     return async (req, res) => {
         let args;
         try {
@@ -19,7 +19,7 @@ export function createEmbeddingsHandler(modelServer) {
             res.end(JSON.stringify({ error: 'Invalid request' }));
             return;
         }
-        if (!modelServer.modelExists(args.model)) {
+        if (!inferenceServer.modelExists(args.model)) {
             res.writeHead(400, { 'Content-Type': 'application/json' });
             res.end(JSON.stringify({ error: 'Invalid model' }));
             return;
@@ -42,7 +42,7 @@ export function createEmbeddingsHandler(modelServer) {
                 model: args.model,
                 input: args.input,
             });
-            const { instance, release } = await modelServer.requestInstance(embeddingsReq, controller.signal);
+            const { instance, release } = await inferenceServer.requestInstance(embeddingsReq, controller.signal);
             const task = instance.processEmbeddingTask(embeddingsReq);
             const result = await task.result;
             release();

package/dist/api/openai/handlers/embeddings.js.map

@@ -1 +1 @@
-{"version":3,"file":"embeddings.js","sourceRoot":"","sources":["../../../../src/api/openai/handlers/embeddings.ts"],"names":[],"mappings":"AAGA,OAAO,EAAE,oBAAoB,EAAE,MAAM,sCAAsC,CAAA;AAC3E,OAAO,EAAE,eAAe,EAAE,MAAM,sBAAsB,CAAA;AAQtD,MAAM,UAAU,uBAAuB,CAAC,WAAwB;IAC/D,OAAO,KAAK,EAAE,GAAoB,EAAE,GAAmB,EAAE,EAAE;QAC1D,IAAI,IAA4B,CAAA;QAEhC,IAAI,CAAC;YACJ,MAAM,IAAI,GAAG,MAAM,oBAAoB,CAAC,GAAG,CAAC,CAAA;YAC5C,IAAI,GAAG,IAAI,CAAA;QACZ,CAAC;QAAC,OAAO,CAAC,EAAE,CAAC;YACZ,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAA;YAChB,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,iBAAiB,EAAE,CAAC,CAAC,CAAA;YACrD,OAAM;QACP,CAAC;QAED,8BAA8B;QAC9B,IAAI,CAAC,IAAI,CAAC,KAAK,IAAI,CAAC,IAAI,CAAC,KAAK,EAAE,CAAC;YAChC,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,iBAAiB,EAAE,CAAC,CAAC,CAAA;YACrD,OAAM;QACP,CAAC;QACD,IAAI,CAAC,WAAW,CAAC,WAAW,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;YAC1C,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,eAAe,EAAE,CAAC,CAAC,CAAA;YACnD,OAAM;QACP,CAAC;QAED,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAA;QACxC,GAAG,CAAC,EAAE,CAAC,OAAO,EAAE,GAAG,EAAE;YACpB,OAAO,CAAC,KAAK,CAAC,0BAA0B,CAAC,CAAA;YACzC,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QACF,GAAG,CAAC,EAAE,CAAC,KAAK,EAAE,GAAG,EAAE;YAClB,OAAO,CAAC,KAAK,CAAC,yBAAyB,CAAC,CAAA;YACxC,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QAEF,IAAI,CAAC;YAGJ,IAAI,KAAK,GAAG,IAAI,CAAC,KAAK,CAAA;YAEtB,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;gBAC/B,MAAM,IAAI,KAAK,CAAC,wBAAwB,CAAC,CAAA;YAC1C,CAAC;YAED,MAAM,aAAa,GAAG,eAAe,CAAmB;gBACvD,KAAK,EAAE,IAAI,CAAC,KAAK;gBACjB,KAAK,EAAE,IAAI,CAAC,KAAe;aAC3B,CAAC,CAAA;YAEF,MAAM,EAAE,QAAQ,EAAE,OAAO,EAAE,GAAG,MAAM,WAAW,CAAC,eAAe,CAC9D,aAAa,EACb,UAAU,CAAC,MAAM,CACjB,CAAA;YACD,MAAM,IAAI,GAAG,QAAQ,CAAC,oBAAoB,CAAC,aAAa,CAAC,CAAA;YACzD,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,MAAM,CAAA;YAChC,OAAO,EAAE,CAAA;YAET,MAAM,QAAQ,GAAmC;gBAChD,KAAK,EAAE,QAAQ,CAAC,OAAO;gBACvB,MAAM,EAAE,MAAM;gBACd,IAAI,EAAE,MAAM,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,SAAS,EAAE,KAAK,EAAE,EAAE,CAAC,CAAC;oBAClD,SAAS,EAAE,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC;oBAChC,KAAK;oBACL,MAAM,EAAE,WAAW;iBACnB,CAAC,CAAC;gBACH,KAAK,EAAE;oBACN,aAAa,EAAE,MAAM,CAAC,WAAW;oBACjC,YAAY,EAAE,MAAM,CAAC,WAAW;iBAChC;aACD,CAAA;YACD,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAA;QAE3C,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACd,OAAO,CAAC,KAAK,CAAC,GAAG,CAAC,CAAA;YAClB,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,uBAAuB,EAAE,CAAC,CAAC,CAAA;QAC5D,CAAC;IACF,CAAC,CAAA;AACF,CAAC"}
+{"version":3,"file":"embeddings.js","sourceRoot":"","sources":["../../../../src/api/openai/handlers/embeddings.ts"],"names":[],"mappings":"AAGA,OAAO,EAAE,oBAAoB,EAAE,MAAM,sCAAsC,CAAA;AAC3E,OAAO,EAAE,eAAe,EAAE,MAAM,sBAAsB,CAAA;AAQtD,MAAM,UAAU,uBAAuB,CAAC,eAAgC;IACvE,OAAO,KAAK,EAAE,GAAoB,EAAE,GAAmB,EAAE,EAAE;QAC1D,IAAI,IAA4B,CAAA;QAEhC,IAAI,CAAC;YACJ,MAAM,IAAI,GAAG,MAAM,oBAAoB,CAAC,GAAG,CAAC,CAAA;YAC5C,IAAI,GAAG,IAAI,CAAA;QACZ,CAAC;QAAC,OAAO,CAAC,EAAE,CAAC;YACZ,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAA;YAChB,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,iBAAiB,EAAE,CAAC,CAAC,CAAA;YACrD,OAAM;QACP,CAAC;QAED,8BAA8B;QAC9B,IAAI,CAAC,IAAI,CAAC,KAAK,IAAI,CAAC,IAAI,CAAC,KAAK,EAAE,CAAC;YAChC,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,iBAAiB,EAAE,CAAC,CAAC,CAAA;YACrD,OAAM;QACP,CAAC;QACD,IAAI,CAAC,eAAe,CAAC,WAAW,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;YAC9C,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,eAAe,EAAE,CAAC,CAAC,CAAA;YACnD,OAAM;QACP,CAAC;QAED,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAA;QACxC,GAAG,CAAC,EAAE,CAAC,OAAO,EAAE,GAAG,EAAE;YACpB,OAAO,CAAC,KAAK,CAAC,0BAA0B,CAAC,CAAA;YACzC,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QACF,GAAG,CAAC,EAAE,CAAC,KAAK,EAAE,GAAG,EAAE;YAClB,OAAO,CAAC,KAAK,CAAC,yBAAyB,CAAC,CAAA;YACxC,UAAU,CAAC,KAAK,EAAE,CAAA;QACnB,CAAC,CAAC,CAAA;QAEF,IAAI,CAAC;YAGJ,IAAI,KAAK,GAAG,IAAI,CAAC,KAAK,CAAA;YAEtB,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;gBAC/B,MAAM,IAAI,KAAK,CAAC,wBAAwB,CAAC,CAAA;YAC1C,CAAC;YAED,MAAM,aAAa,GAAG,eAAe,CAAkB;gBACtD,KAAK,EAAE,IAAI,CAAC,KAAK;gBACjB,KAAK,EAAE,IAAI,CAAC,KAAe;aAC3B,CAAC,CAAA;YAEF,MAAM,EAAE,QAAQ,EAAE,OAAO,EAAE,GAAG,MAAM,eAAe,CAAC,eAAe,CAClE,aAAa,EACb,UAAU,CAAC,MAAM,CACjB,CAAA;YACD,MAAM,IAAI,GAAG,QAAQ,CAAC,oBAAoB,CAAC,aAAa,CAAC,CAAA;YACzD,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,MAAM,CAAA;YAChC,OAAO,EAAE,CAAA;YAET,MAAM,QAAQ,GAAmC;gBAChD,KAAK,EAAE,QAAQ,CAAC,OAAO;gBACvB,MAAM,EAAE,MAAM;gBACd,IAAI,EAAE,MAAM,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,SAAS,EAAE,KAAK,EAAE,EAAE,CAAC,CAAC;oBAClD,SAAS,EAAE,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC;oBAChC,KAAK;oBACL,MAAM,EAAE,WAAW;iBACnB,CAAC,CAAC;gBACH,KAAK,EAAE;oBACN,aAAa,EAAE,MAAM,CAAC,WAAW;oBACjC,YAAY,EAAE,MAAM,CAAC,WAAW;iBAChC;aACD,CAAA;YACD,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAA;QAE3C,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACd,OAAO,CAAC,KAAK,CAAC,GAAG,CAAC,CAAA;YAClB,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;YAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,uBAAuB,EAAE,CAAC,CAAC,CAAA;QAC5D,CAAC;IACF,CAAC,CAAA;AACF,CAAC"}

package/dist/api/openai/handlers/models.d.ts

@@ -1,3 +1,3 @@
 import type { IncomingMessage, ServerResponse } from 'node:http';
-import type { ModelServer } from '../../../server';
-export declare function createModelsHandler(modelServer: ModelServer): (req: IncomingMessage, res: ServerResponse) => Promise<void>;
+import type { InferenceServer } from '../../../server';
+export declare function createModelsHandler(inferenceServer: InferenceServer): (req: IncomingMessage, res: ServerResponse) => Promise<void>;

package/dist/api/openai/handlers/models.js

@@ -1,8 +1,8 @@
 // handler for v1/models
 // https://platform.openai.com/docs/api-reference/models/list
-export function createModelsHandler(modelServer) {
+export function createModelsHandler(inferenceServer) {
     return async (req, res) => {
-        const models = modelServer.store.getStatus();
+        const models = inferenceServer.store.getStatus();
         const data = Object.entries(models).map(([id, info]) => {
             return {
                 object: 'model',

package/dist/api/openai/handlers/models.js.map

@@ -1 +1 @@
-{"version":3,"file":"models.js","sourceRoot":"","sources":["../../../../src/api/openai/handlers/models.ts"],"names":[],"mappings":"AAIA,wBAAwB;AACxB,6DAA6D;AAC7D,MAAM,UAAU,mBAAmB,CAAC,WAAwB;IAC3D,OAAO,KAAK,EAAE,GAAoB,EAAE,GAAmB,EAAE,EAAE;QAE1D,MAAM,MAAM,GAAG,WAAW,CAAC,KAAK,CAAC,SAAS,EAAE,CAAA;QAC5C,MAAM,IAAI,GAAmB,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,CACtD,CAAC,CAAC,EAAE,EAAE,IAAI,CAAC,EAAE,EAAE;YACd,OAAO;gBACN,MAAM,EAAE,OAAO;gBACf,EAAE;gBACF,OAAO,EAAE,CAAC;gBACV,QAAQ,EAAE,IAAI,CAAC,MAAM;aACrB,CAAA;QACF,CAAC,CACD,CAAA;QAED,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;QAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,MAAM,EAAE,MAAM,EAAE,IAAI,EAAE,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAA;IAC3D,CAAC,CAAA;AACF,CAAC"}
+{"version":3,"file":"models.js","sourceRoot":"","sources":["../../../../src/api/openai/handlers/models.ts"],"names":[],"mappings":"AAIA,wBAAwB;AACxB,6DAA6D;AAC7D,MAAM,UAAU,mBAAmB,CAAC,eAAgC;IACnE,OAAO,KAAK,EAAE,GAAoB,EAAE,GAAmB,EAAE,EAAE;QAE1D,MAAM,MAAM,GAAG,eAAe,CAAC,KAAK,CAAC,SAAS,EAAE,CAAA;QAChD,MAAM,IAAI,GAAmB,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,CACtD,CAAC,CAAC,EAAE,EAAE,IAAI,CAAC,EAAE,EAAE;YACd,OAAO;gBACN,MAAM,EAAE,OAAO;gBACf,EAAE;gBACF,OAAO,EAAE,CAAC;gBACV,QAAQ,EAAE,IAAI,CAAC,MAAM;aACrB,CAAA;QACF,CAAC,CACD,CAAA;QAED,GAAG,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,CAAA;QAC1D,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,MAAM,EAAE,MAAM,EAAE,IAAI,EAAE,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAA;IAC3D,CAAC,CAAA;AACF,CAAC"}

package/dist/api/openai/index.d.ts

@@ -1,5 +1,5 @@
-import type { ModelServer } from '../../server.js';
-export declare function createOpenAIRequestHandlers(modelServer: ModelServer): {
+import type { InferenceServer } from '../../server.js';
+export declare function createOpenAIRequestHandlers(inferenceServer: InferenceServer): {
     chatCompletions: (req: import("http").IncomingMessage, res: import("http").ServerResponse) => Promise<void>;
     completions: (req: import("http").IncomingMessage, res: import("http").ServerResponse) => Promise<void>;
     models: (req: import("http").IncomingMessage, res: import("http").ServerResponse) => Promise<void>;

package/dist/api/openai/index.js

@@ -3,12 +3,12 @@ import { createCompletionHandler } from './handlers/completions.js';
 import { createModelsHandler } from './handlers/models.js';
 import { createEmbeddingsHandler } from './handlers/embeddings.js';
 // See OpenAI API specs at https://github.com/openai/openai-openapi/blob/master/openapi.yaml
-export function createOpenAIRequestHandlers(modelServer) {
+export function createOpenAIRequestHandlers(inferenceServer) {
     return {
-        chatCompletions: createChatCompletionHandler(modelServer),
-        completions: createCompletionHandler(modelServer),
-        models: createModelsHandler(modelServer),
-        embeddings: createEmbeddingsHandler(modelServer),
+        chatCompletions: createChatCompletionHandler(inferenceServer),
+        completions: createCompletionHandler(inferenceServer),
+        models: createModelsHandler(inferenceServer),
+        embeddings: createEmbeddingsHandler(inferenceServer),
     };
 }
 //# sourceMappingURL=index.js.map

package/dist/api/openai/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../../../src/api/openai/index.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,2BAA2B,EAAE,MAAM,oBAAoB,CAAA;AAChE,OAAO,EAAE,uBAAuB,EAAE,MAAM,2BAA2B,CAAA;AACnE,OAAO,EAAE,mBAAmB,EAAE,MAAM,sBAAsB,CAAA;AAC1D,OAAO,EAAE,uBAAuB,EAAE,MAAM,0BAA0B,CAAA;AAGlE,4FAA4F;AAC5F,MAAM,UAAU,2BAA2B,CAAC,WAAwB;IACnE,OAAO;QACN,eAAe,EAAE,2BAA2B,CAAC,WAAW,CAAC;QACzD,WAAW,EAAE,uBAAuB,CAAC,WAAW,CAAC;QACjD,MAAM,EAAE,mBAAmB,CAAC,WAAW,CAAC;QACxC,UAAU,EAAE,uBAAuB,CAAC,WAAW,CAAC;KAChD,CAAA;AACF,CAAC"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../../../src/api/openai/index.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,2BAA2B,EAAE,MAAM,oBAAoB,CAAA;AAChE,OAAO,EAAE,uBAAuB,EAAE,MAAM,2BAA2B,CAAA;AACnE,OAAO,EAAE,mBAAmB,EAAE,MAAM,sBAAsB,CAAA;AAC1D,OAAO,EAAE,uBAAuB,EAAE,MAAM,0BAA0B,CAAA;AAGlE,4FAA4F;AAC5F,MAAM,UAAU,2BAA2B,CAAC,eAAgC;IAC3E,OAAO;QACN,eAAe,EAAE,2BAA2B,CAAC,eAAe,CAAC;QAC7D,WAAW,EAAE,uBAAuB,CAAC,eAAe,CAAC;QACrD,MAAM,EAAE,mBAAmB,CAAC,eAAe,CAAC;QAC5C,UAAU,EAAE,uBAAuB,CAAC,eAAe,CAAC;KACpD,CAAA;AACF,CAAC"}

package/dist/cli/lib/loadConfig.d.ts

@@ -1,5 +1,5 @@
-import { ModelServerOptions } from '../../server.js';
+import type { InferenceServerOptions } from '../../server.js';
 export declare function loadConfig(customPath?: string): Promise<{
     path: string;
-    options: ModelServerOptions;
+    options: InferenceServerOptions;
 } | null>;

package/dist/cli/lib/loadConfig.js

@@ -1,4 +1,4 @@
-import fs from 'node:fs/promises';
+import { promises as fs } from 'node:fs';
 import path from 'node:path';
 import url from 'node:url';
 export async function loadConfig(customPath) {
@@ -23,9 +23,9 @@ export async function loadConfig(customPath) {
             // For package.json, we need to check for the 'infs' or 'inference-server' key
             if (configName === 'package.json') {
                 const packageJson = options;
-                const modelServerOptions = packageJson.infs || packageJson['inference-server'];
-                if (modelServerOptions) {
-                    return { path: configPath, options: modelServerOptions };
+                const inferenceServerOptions = packageJson.infs || packageJson['inference-server'];
+                if (inferenceServerOptions) {
+                    return { path: configPath, options: inferenceServerOptions };
                 }
                 continue;
             }

package/dist/cli/lib/loadConfig.js.map

@@ -1 +1 @@
-{"version":3,"file":"loadConfig.js","sourceRoot":"","sources":["../../../src/cli/lib/loadConfig.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,MAAM,kBAAkB,CAAA;AACjC,OAAO,IAAI,MAAM,WAAW,CAAA;AAC5B,OAAO,GAAG,MAAM,UAAU,CAAA;AAG1B,MAAM,CAAC,KAAK,UAAU,UAAU,CAC/B,UAAmB;IAEnB,yCAAyC;IACzC,IAAI,UAAU,EAAE,CAAC;QAChB,IAAI,CAAC;YACJ,MAAM,UAAU,GAAG,IAAI,CAAC,OAAO,CAAC,UAAU,CAAC,CAAA;YAC3C,MAAM,OAAO,GAAG,MAAM,mBAAmB,CAAC,UAAU,CAAC,CAAA;YACrD,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,OAAO,EAAE,CAAA;QACrC,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YAChB,MAAM,IAAI,KAAK,CAAC,8BAA8B,UAAU,KAAM,KAAe,CAAC,OAAO,EAAE,CAAC,CAAA;QACzF,CAAC;IACF,CAAC;IAED,yBAAyB;IACzB,MAAM,WAAW,GAAG,CAAC,gBAAgB,EAAE,iBAAiB,EAAE,kBAAkB,EAAE,cAAc,CAAC,CAAA;IAE7F,gCAAgC;IAChC,KAAK,MAAM,UAAU,IAAI,WAAW,EAAE,CAAC;QACtC,IAAI,CAAC;YACJ,MAAM,UAAU,GAAG,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,GAAG,EAAE,EAAE,UAAU,CAAC,CAAA;YAC1D,MAAM,OAAO,GAAG,MAAM,mBAAmB,CAAC,UAAU,CAAC,CAAA;YAErD,8EAA8E;YAC9E,IAAI,UAAU,KAAK,cAAc,EAAE,CAAC;gBACnC,MAAM,WAAW,GAAG,OAAc,CAAA;gBAClC,MAAM,kBAAkB,GAAG,WAAW,CAAC,IAAI,IAAI,WAAW,CAAC,kBAAkB,CAAC,CAAA;gBAC9E,IAAI,kBAAkB,EAAE,CAAC;oBACxB,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,OAAO,EAAE,kBAAkB,EAAE,CAAA;gBACzD,CAAC;gBACD,SAAQ;YACT,CAAC;YAED,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,OAAO,EAAE,CAAA;QACrC,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YAChB,kCAAkC;YAClC,SAAQ;QACT,CAAC;IACF,CAAC;IAID,OAAO,IAAI,CAAA;AACZ,CAAC;AAED,KAAK,UAAU,mBAAmB,CAAC,UAAkB;IACpD,kCAAkC;IAClC,IAAI,UAAU,CAAC,QAAQ,CAAC,OAAO,CAAC,EAAE,CAAC;QAClC,MAAM,OAAO,GAAG,MAAM,EAAE,CAAC,QAAQ,CAAC,UAAU,EAAE,OAAO,CAAC,CAAA;QACtD,OAAO,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAA;IAC3B,CAAC;SAAM,CAAC;QACP,6CAA6C;QAC7C,MAAM,QAAQ,GAAG,MAAM,MAAM,CAAC,GAAG,CAAC,aAAa,CAAC,UAAU,CAAC,CAAC,IAAI,CAAC,CAAA;QACjE,OAAO,QAAQ,CAAC,MAAM,IAAI,QAAQ,CAAC,OAAO,CAAA;IAC3C,CAAC;AACF,CAAC"}
+{"version":3,"file":"loadConfig.js","sourceRoot":"","sources":["../../../src/cli/lib/loadConfig.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,IAAI,EAAE,EAAE,MAAM,SAAS,CAAA;AACxC,OAAO,IAAI,MAAM,WAAW,CAAA;AAC5B,OAAO,GAAG,MAAM,UAAU,CAAA;AAG1B,MAAM,CAAC,KAAK,UAAU,UAAU,CAAC,UAAmB;IACnD,yCAAyC;IACzC,IAAI,UAAU,EAAE,CAAC;QAChB,IAAI,CAAC;YACJ,MAAM,UAAU,GAAG,IAAI,CAAC,OAAO,CAAC,UAAU,CAAC,CAAA;YAC3C,MAAM,OAAO,GAAG,MAAM,mBAAmB,CAAC,UAAU,CAAC,CAAA;YACrD,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,OAAO,EAAE,CAAA;QACrC,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YAChB,MAAM,IAAI,KAAK,CAAC,8BAA8B,UAAU,KAAM,KAAe,CAAC,OAAO,EAAE,CAAC,CAAA;QACzF,CAAC;IACF,CAAC;IAED,yBAAyB;IACzB,MAAM,WAAW,GAAG,CAAC,gBAAgB,EAAE,iBAAiB,EAAE,kBAAkB,EAAE,cAAc,CAAC,CAAA;IAE7F,gCAAgC;IAChC,KAAK,MAAM,UAAU,IAAI,WAAW,EAAE,CAAC;QACtC,IAAI,CAAC;YACJ,MAAM,UAAU,GAAG,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,GAAG,EAAE,EAAE,UAAU,CAAC,CAAA;YAC1D,MAAM,OAAO,GAAG,MAAM,mBAAmB,CAAC,UAAU,CAAC,CAAA;YAErD,8EAA8E;YAC9E,IAAI,UAAU,KAAK,cAAc,EAAE,CAAC;gBACnC,MAAM,WAAW,GAAG,OAAc,CAAA;gBAClC,MAAM,sBAAsB,GAAG,WAAW,CAAC,IAAI,IAAI,WAAW,CAAC,kBAAkB,CAAC,CAAA;gBAClF,IAAI,sBAAsB,EAAE,CAAC;oBAC5B,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,OAAO,EAAE,sBAAsB,EAAE,CAAA;gBAC7D,CAAC;gBACD,SAAQ;YACT,CAAC;YAED,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,OAAO,EAAE,CAAA;QACrC,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YAChB,kCAAkC;YAClC,SAAQ;QACT,CAAC;IACF,CAAC;IAED,OAAO,IAAI,CAAA;AACZ,CAAC;AAED,KAAK,UAAU,mBAAmB,CAAC,UAAkB;IACpD,kCAAkC;IAClC,IAAI,UAAU,CAAC,QAAQ,CAAC,OAAO,CAAC,EAAE,CAAC;QAClC,MAAM,OAAO,GAAG,MAAM,EAAE,CAAC,QAAQ,CAAC,UAAU,EAAE,OAAO,CAAC,CAAA;QACtD,OAAO,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAA;IAC3B,CAAC;SAAM,CAAC;QACP,6CAA6C;QAC7C,MAAM,QAAQ,GAAG,MAAM,MAAM,CAAC,GAAG,CAAC,aAAa,CAAC,UAAU,CAAC,CAAC,IAAI,CAAC,CAAA;QACjE,OAAO,QAAQ,CAAC,MAAM,IAAI,QAAQ,CAAC,OAAO,CAAA;IAC3C,CAAC;AACF,CAAC"}

package/dist/cli/lib/parseModelPathInfo.d.ts

@@ -0,0 +1,8 @@
+interface ModelPathInfo {
+    domain?: string;
+    org?: string;
+    name: string;
+    branch?: string;
+}
+export declare function parseModelPathInfo(modelPath: string): ModelPathInfo;
+export {};

package/dist/cli/lib/parseModelPathInfo.js

@@ -0,0 +1,29 @@
+import path from 'node:path';
+export function parseModelPathInfo(modelPath) {
+    const parts = modelPath.split(path.sep).filter(Boolean); // Remove empty parts
+    if (parts.length === 0) {
+        throw new Error('Invalid model path: empty path');
+    }
+    // Special handling for huggingface.co paths with branch
+    if (parts[0] === 'huggingface.co') {
+        const lastPart = parts[parts.length - 1];
+        if (lastPart.includes('-')) {
+            const lastMinusIndex = lastPart.lastIndexOf('-');
+            const repoName = lastPart.slice(0, lastMinusIndex);
+            const branch = lastPart.slice(lastMinusIndex + 1);
+            return {
+                domain: parts[0],
+                org: parts[1],
+                name: repoName,
+                branch: branch,
+            };
+        }
+    }
+    const name = parts[parts.length - 1];
+    const domain = parts[0];
+    return {
+        domain,
+        name,
+    };
+}
+//# sourceMappingURL=parseModelPathInfo.js.map

package/dist/cli/lib/parseModelPathInfo.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"parseModelPathInfo.js","sourceRoot":"","sources":["../../../src/cli/lib/parseModelPathInfo.ts"],"names":[],"mappings":"AAAA,OAAO,IAAI,MAAM,WAAW,CAAA;AAS5B,MAAM,UAAU,kBAAkB,CAAC,SAAiB;IACnD,MAAM,KAAK,GAAG,SAAS,CAAC,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAA,CAAC,qBAAqB;IAC7E,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACxB,MAAM,IAAI,KAAK,CAAC,gCAAgC,CAAC,CAAA;IAClD,CAAC;IAED,wDAAwD;IACxD,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,gBAAgB,EAAE,CAAC;QACnC,MAAM,QAAQ,GAAG,KAAK,CAAC,KAAK,CAAC,MAAM,GAAG,CAAC,CAAC,CAAA;QACxC,IAAI,QAAQ,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;YAC5B,MAAM,cAAc,GAAG,QAAQ,CAAC,WAAW,CAAC,GAAG,CAAC,CAAA;YAChD,MAAM,QAAQ,GAAG,QAAQ,CAAC,KAAK,CAAC,CAAC,EAAE,cAAc,CAAC,CAAA;YAClD,MAAM,MAAM,GAAG,QAAQ,CAAC,KAAK,CAAC,cAAc,GAAG,CAAC,CAAC,CAAA;YACjD,OAAO;gBACN,MAAM,EAAE,KAAK,CAAC,CAAC,CAAC;gBAChB,GAAG,EAAE,KAAK,CAAC,CAAC,CAAC;gBACb,IAAI,EAAE,QAAQ;gBACd,MAAM,EAAE,MAAM;aACd,CAAA;QACF,CAAC;IACF,CAAC;IACD,MAAM,IAAI,GAAG,KAAK,CAAC,KAAK,CAAC,MAAM,GAAG,CAAC,CAAC,CAAA;IACpC,MAAM,MAAM,GAAG,KAAK,CAAC,CAAC,CAAC,CAAA;IACvB,OAAO;QACN,MAAM;QACN,IAAI;KACJ,CAAA;AACF,CAAC"}

package/dist/cli/prepareCommand.js.map

@@ -1 +1 @@
-{"version":3,"file":"prepareCommand.js","sourceRoot":"","sources":["../../src/cli/prepareCommand.ts"],"names":[],"mappings":"AACA,OAAO,IAAI,MAAM,WAAW,CAAA;AAC5B,OAAO,KAAK,MAAM,OAAO,CAAA;AAEzB,OAAO,EAAE,UAAU,EAAE,MAAM,mBAAmB,CAAA;AAE9C,OAAO,EAAE,kBAAkB,EAAE,MAAM,2BAA2B,CAAA;AAC9D,OAAO,EAAE,oBAAoB,EAAE,MAAM,sCAAsC,CAAA;AAC3E,OAAO,EAAE,wBAAwB,EAAE,MAAM,0CAA0C,CAAA;AACnF,OAAO,EAAE,eAAe,EAAE,MAAM,iCAAiC,CAAA;AACjE,OAAO,EAAE,SAAS,EAAE,MAAM,wBAAwB,CAAA;AAClD,OAAO,EAAE,UAAU,EAAE,MAAM,gCAAgC,CAAA;AAO3D,KAAK,UAAU,gBAAgB,CAAC,OAA2B,EAAE,WAAoB;IAChF,IAAI,eAAe,GAAG,eAAe,CAAC,QAAQ,CAAC,CAAA;IAC/C,IAAI,OAAO,CAAC,SAAS,EAAE,CAAC;QACvB,eAAe,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,SAAS,EAAE,QAAQ,CAAC,CAAA;IACzD,CAAC;IACD,MAAM,kBAAkB,GAAoC,EAAE,CAAA;IAC9D,MAAM,WAAW,GAA6C,EAAE,CAAA;IAChE,KAAK,MAAM,OAAO,IAAI,OAAO,CAAC,MAAM,EAAE,CAAC;QACtC,MAAM,YAAY,GAAG,OAAO,CAAC,MAAM,CAAC,OAAO,CAAC,CAAA;QAC5C,MAAM,SAAS,GAAG,kBAAkB,CAAC,QAAQ,CAAC,YAAY,CAAC,MAAM,CAAC,CAAA;QAClE,IAAI,SAAS,EAAE,CAAC;YACf,MAAM,mBAAmB,GAAG,YAAmC,CAAA;YAC/D,gFAAgF;YAChF,oBAAoB,CAAC,OAAO,EAAE,mBAAmB,CAAC,CAAA;YAClD,kBAAkB,CAAC,OAAO,CAAC,GAAG;gBAC7B,EAAE,EAAE,OAAO;gBACX,YAAY,EAAE,CAAC;gBACf,YAAY,EAAE,CAAC;gBACf,eAAe;gBACf,OAAO,EAAE,UAAU;gBACnB,QAAQ,EAAE,wBAAwB,CAAC;oBAClC,GAAG,EAAE,mBAAmB,CAAC,GAAG;oBAC5B,QAAQ,EAAE,mBAAmB,CAAC,QAAQ;oBACtC,eAAe;iBACf,CAAC;gBACF,GAAG,mBAAmB;aACtB,CAAA;QACF,CAAC;IACF,CAAC;IACD,MAAM,aAAa,GAAG,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,OAAO,IAAI,EAAE,CAAC,CAAA;IACxD,MAAM,aAAa,GAAgC,EAAE,CAAA;IACrD,KAAK,MAAM,GAAG,IAAI,WAAW,EAAE,CAAC;QAC/B,MAAM,SAAS,GAAG,kBAAkB,CAAC,QAAQ,CAAC,GAAG,CAAC,MAAM,CAAC,CAAA;QACzD,MAAM,QAAQ,GAAG,aAAa,CAAC,QAAQ,CAAC,GAAG,CAAC,MAAM,CAAC,CAAA;QACnD,IAAI,CAAC,SAAS,IAAI,CAAC,QAAQ,EAAE,CAAC;YAC7B,MAAM,IAAI,KAAK,CAAC,WAAW,GAAG,CAAC,MAAM,oBAAoB,GAAG,CAAC,KAAK,kBAAkB,CAAC,CAAA;QACtF,CAAC;QACD,IAAI,QAAQ,EAAE,CAAC;YACd,aAAa,CAAC,GAAG,CAAC,MAAM,CAAC,GAAG,OAAO,CAAC,OAAQ,CAAC,GAAG,CAAC,MAAM,CAAC,CAAA;QACzD,CAAC;IACF,CAAC;IACD,MAAM,KAAK,GAAG,IAAI,UAAU,CAAC;QAC5B,GAAG,EAAE,SAAS,CAAC,IAAI;QACnB,kBAAkB,EAAE,WAAW;QAC/B,MAAM,EAAE,kBAAkB;QAC1B,eAAe;KACf,CAAC,CAAA;IAEF,MAAM,cAAc,GAAG,EAAE,CAAA;IACzB,KAAK,MAAM,GAAG,IAAI,kBAAkB,EAAE,CAAC;QACtC,sBAAsB;QACtB,MAAM,gBAAgB,GAAG,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,IAAI,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,KAAK,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,MAAM,KAAK,GAAG,CAAC,CAAA;QAC1G,IAAI,CAAC,gBAAgB,EAAE,CAAC;YACvB,SAAQ;QACT,CAAC;QACD,cAAc,CAAC,IAAI,CAClB,IAAI,OAAO,CAAC,KAAK,EAAE,OAAO,EAAE,MAAM,EAAE,EAAE;YACrC,IAAI,CAAC;gBACJ,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,cAAc,GAAG,YAAY,CAAC,CAAA;gBAC1D,aAAa,CAAC,GAAG,CAAC,GAAG,MAAM,CAAA;gBAC3B,OAAO,CAAC;oBACP,GAAG;oBACH,MAAM;iBACN,CAAC,CAAA;YACH,CAAC;YAAC,OAAO,GAAG,EAAE,CAAC;gBACd,MAAM,CAAC,GAAG,CAAC,CAAA;YACZ,CAAC;QACF,CAAC,CAAC,CACF,CAAA;IACF,CAAC;IACD,MAAM,OAAO,CAAC,GAAG,CAAC,cAAc,CAAC,CAAA;IACjC,MAAM,KAAK,CAAC,IAAI,CAAC,aAAa,CAAC,CAAA;AAChC,CAAC;AAED,KAAK,UAAU,aAAa,CAAC,UAAmB,EAAE,WAAoB;IACrE,IAAI,CAAC;QACJ,MAAM,MAAM,GAAG,MAAM,UAAU,CAAC,UAAU,CAAC,CAAA;QAC3C,IAAI,CAAC,MAAM,EAAE,CAAC;YACb,MAAM,IAAI,KAAK,CACd,0KAA0K,CAC1K,CAAA;QACF,CAAC;QAED,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI,CAAC,sBAAsB,UAAU,EAAE,CAAC,CAAC,CAAA;QAC3D,MAAM,gBAAgB,CAAC,MAAM,CAAC,OAAO,EAAE,WAAW,CAAC,CAAA;QAEnD,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,KAAK,CAAC,+BAA+B,CAAC,CAAC,CAAA;IAC1D,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QAChB,OAAO,CAAC,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,UAAW,KAAe,CAAC,OAAO,EAAE,CAAC,CAAC,CAAA;QAC9D,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAA;IAChB,CAAC;AACF,CAAC;AAED,MAAM,CAAC,MAAM,cAAc,GAA0C;IACpE,OAAO,EAAE,sBAAsB;IAC/B,OAAO,EAAE,CAAC,MAAM,EAAE,UAAU,CAAC;IAC7B,QAAQ,EAAE,yCAAyC;IACnD,OAAO,EAAE,CAAC,KAAK,EAAE,EAAE;QAClB,OAAO,KAAK;aACV,UAAU,CAAC,YAAY,EAAE;YACzB,IAAI,EAAE,QAAQ;YACd,QAAQ,EAAE,kDAAkD;SAC5D,CAAC;aACD,MAAM,CAAC,aAAa,EAAE;YACtB,KAAK,EAAE,GAAG;YACV,IAAI,EAAE,QAAQ;YACd,QAAQ,EAAE,0CAA0C;YACpD,OAAO,EAAE,CAAC;SACV,CAAC,CAAA;IACJ,CAAC;IACD,OAAO,EAAE,KAAK,EAAE,IAAI,EAAE,EAAE;QACvB,MAAM,aAAa,CAAC,IAAI,CAAC,UAAU,EAAE,IAAI,CAAC,WAAW,CAAC,CAAA;IACvD,CAAC;CACD,CAAA"}
+{"version":3,"file":"prepareCommand.js","sourceRoot":"","sources":["../../src/cli/prepareCommand.ts"],"names":[],"mappings":"AACA,OAAO,IAAI,MAAM,WAAW,CAAA;AAC5B,OAAO,KAAK,MAAM,OAAO,CAAA;AAEzB,OAAO,EAAE,UAAU,EAAE,MAAM,mBAAmB,CAAA;AAE9C,OAAO,EAAE,kBAAkB,EAAE,MAAM,2BAA2B,CAAA;AAC9D,OAAO,EAAE,oBAAoB,EAAE,MAAM,sCAAsC,CAAA;AAC3E,OAAO,EAAE,wBAAwB,EAAE,MAAM,0CAA0C,CAAA;AACnF,OAAO,EAAE,eAAe,EAAE,MAAM,iCAAiC,CAAA;AACjE,OAAO,EAAE,SAAS,EAAE,MAAM,wBAAwB,CAAA;AAClD,OAAO,EAAE,UAAU,EAAE,MAAM,gCAAgC,CAAA;AAO3D,KAAK,UAAU,gBAAgB,CAAC,OAA+B,EAAE,WAAoB;IACpF,IAAI,eAAe,GAAG,eAAe,CAAC,QAAQ,CAAC,CAAA;IAC/C,IAAI,OAAO,CAAC,SAAS,EAAE,CAAC;QACvB,eAAe,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,SAAS,EAAE,QAAQ,CAAC,CAAA;IACzD,CAAC;IACD,MAAM,kBAAkB,GAAoC,EAAE,CAAA;IAC9D,MAAM,WAAW,GAA6C,EAAE,CAAA;IAChE,KAAK,MAAM,OAAO,IAAI,OAAO,CAAC,MAAM,EAAE,CAAC;QACtC,MAAM,YAAY,GAAG,OAAO,CAAC,MAAM,CAAC,OAAO,CAAC,CAAA;QAC5C,MAAM,SAAS,GAAG,kBAAkB,CAAC,QAAQ,CAAC,YAAY,CAAC,MAAM,CAAC,CAAA;QAClE,IAAI,SAAS,EAAE,CAAC;YACf,MAAM,mBAAmB,GAAG,YAAmC,CAAA;YAC/D,gFAAgF;YAChF,oBAAoB,CAAC,OAAO,EAAE,mBAAmB,CAAC,CAAA;YAClD,kBAAkB,CAAC,OAAO,CAAC,GAAG;gBAC7B,EAAE,EAAE,OAAO;gBACX,YAAY,EAAE,CAAC;gBACf,YAAY,EAAE,CAAC;gBACf,eAAe;gBACf,OAAO,EAAE,UAAU;gBACnB,QAAQ,EAAE,wBAAwB,CAAC;oBAClC,GAAG,EAAE,mBAAmB,CAAC,GAAG;oBAC5B,QAAQ,EAAE,mBAAmB,CAAC,QAAQ;oBACtC,eAAe;iBACf,CAAC;gBACF,GAAG,mBAAmB;aACtB,CAAA;QACF,CAAC;IACF,CAAC;IACD,MAAM,aAAa,GAAG,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,OAAO,IAAI,EAAE,CAAC,CAAA;IACxD,MAAM,aAAa,GAAgC,EAAE,CAAA;IACrD,KAAK,MAAM,GAAG,IAAI,WAAW,EAAE,CAAC;QAC/B,MAAM,SAAS,GAAG,kBAAkB,CAAC,QAAQ,CAAC,GAAG,CAAC,MAAM,CAAC,CAAA;QACzD,MAAM,QAAQ,GAAG,aAAa,CAAC,QAAQ,CAAC,GAAG,CAAC,MAAM,CAAC,CAAA;QACnD,IAAI,CAAC,SAAS,IAAI,CAAC,QAAQ,EAAE,CAAC;YAC7B,MAAM,IAAI,KAAK,CAAC,WAAW,GAAG,CAAC,MAAM,oBAAoB,GAAG,CAAC,KAAK,kBAAkB,CAAC,CAAA;QACtF,CAAC;QACD,IAAI,QAAQ,EAAE,CAAC;YACd,aAAa,CAAC,GAAG,CAAC,MAAM,CAAC,GAAG,OAAO,CAAC,OAAQ,CAAC,GAAG,CAAC,MAAM,CAAC,CAAA;QACzD,CAAC;IACF,CAAC;IACD,MAAM,KAAK,GAAG,IAAI,UAAU,CAAC;QAC5B,GAAG,EAAE,SAAS,CAAC,IAAI;QACnB,kBAAkB,EAAE,WAAW;QAC/B,MAAM,EAAE,kBAAkB;QAC1B,eAAe;KACf,CAAC,CAAA;IAEF,MAAM,cAAc,GAAG,EAAE,CAAA;IACzB,KAAK,MAAM,GAAG,IAAI,kBAAkB,EAAE,CAAC;QACtC,sBAAsB;QACtB,MAAM,gBAAgB,GAAG,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,IAAI,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,KAAK,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,MAAM,KAAK,GAAG,CAAC,CAAA;QAC1G,IAAI,CAAC,gBAAgB,EAAE,CAAC;YACvB,SAAQ;QACT,CAAC;QACD,cAAc,CAAC,IAAI,CAClB,IAAI,OAAO,CAAC,KAAK,EAAE,OAAO,EAAE,MAAM,EAAE,EAAE;YACrC,IAAI,CAAC;gBACJ,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,cAAc,GAAG,YAAY,CAAC,CAAA;gBAC1D,aAAa,CAAC,GAAG,CAAC,GAAG,MAAM,CAAA;gBAC3B,OAAO,CAAC;oBACP,GAAG;oBACH,MAAM;iBACN,CAAC,CAAA;YACH,CAAC;YAAC,OAAO,GAAG,EAAE,CAAC;gBACd,MAAM,CAAC,GAAG,CAAC,CAAA;YACZ,CAAC;QACF,CAAC,CAAC,CACF,CAAA;IACF,CAAC;IACD,MAAM,OAAO,CAAC,GAAG,CAAC,cAAc,CAAC,CAAA;IACjC,MAAM,KAAK,CAAC,IAAI,CAAC,aAAa,CAAC,CAAA;AAChC,CAAC;AAED,KAAK,UAAU,aAAa,CAAC,UAAmB,EAAE,WAAoB;IACrE,IAAI,CAAC;QACJ,MAAM,MAAM,GAAG,MAAM,UAAU,CAAC,UAAU,CAAC,CAAA;QAC3C,IAAI,CAAC,MAAM,EAAE,CAAC;YACb,MAAM,IAAI,KAAK,CACd,0KAA0K,CAC1K,CAAA;QACF,CAAC;QAED,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI,CAAC,sBAAsB,UAAU,EAAE,CAAC,CAAC,CAAA;QAC3D,MAAM,gBAAgB,CAAC,MAAM,CAAC,OAAO,EAAE,WAAW,CAAC,CAAA;QAEnD,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,KAAK,CAAC,+BAA+B,CAAC,CAAC,CAAA;IAC1D,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QAChB,OAAO,CAAC,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,UAAW,KAAe,CAAC,OAAO,EAAE,CAAC,CAAC,CAAA;QAC9D,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAA;IAChB,CAAC;AACF,CAAC;AAED,MAAM,CAAC,MAAM,cAAc,GAA0C;IACpE,OAAO,EAAE,sBAAsB;IAC/B,OAAO,EAAE,CAAC,MAAM,EAAE,UAAU,CAAC;IAC7B,QAAQ,EAAE,yCAAyC;IACnD,OAAO,EAAE,CAAC,KAAK,EAAE,EAAE;QAClB,OAAO,KAAK;aACV,UAAU,CAAC,YAAY,EAAE;YACzB,IAAI,EAAE,QAAQ;YACd,QAAQ,EAAE,kDAAkD;SAC5D,CAAC;aACD,MAAM,CAAC,aAAa,EAAE;YACtB,KAAK,EAAE,GAAG;YACV,IAAI,EAAE,QAAQ;YACd,QAAQ,EAAE,0CAA0C;YACpD,OAAO,EAAE,CAAC;SACV,CAAC,CAAA;IACJ,CAAC;IACD,OAAO,EAAE,KAAK,EAAE,IAAI,EAAE,EAAE;QACvB,MAAM,aAAa,CAAC,IAAI,CAAC,UAAU,EAAE,IAAI,CAAC,WAAW,CAAC,CAAA;IACvD,CAAC;CACD,CAAA"}

package/dist/cli/removeCommand.js

@@ -80,27 +80,32 @@ async function removeModels(pattern, skipPrompt) {
     });
     const matches = matchCachedModels(cacheInfo, pattern);
     if (matches.length === 0) {
-        console.log('No models found matching the pattern');
+        console.error(chalk.red(`No models found matching the given pattern`));
         return;
     }
     if (!skipPrompt) {
         const treeLines = renderTreeView(matches);
         console.log(treeLines.join('\n'));
         const { totalSize, fileCount } = calculateTreeAggs(matches);
-        console.log(chalk.cyan(`Total size: ${prettyBytes(totalSize)} bytes`));
-        const confirmed = await promptConfirmation(`Delete ${fileCount} files?`);
+        // console.log(chalk.cyan(`Total size: ${prettyBytes(totalSize)} bytes`))
+        const fileText = fileCount === 1 ? `one file` : `${fileCount} files`;
+        const modelText = matches.length === 1 ? chalk.bold(matches[0].relPath) : `${matches.length} models`;
+        console.log(`\nThis will remove ${fileText} freeing ${prettyBytes(totalSize)} total.`);
+        const confirmed = await promptConfirmation(`Delete ${modelText} from disk?`);
         if (!confirmed) {
             console.log('Aborted');
             return;
         }
     }
     try {
-        console.log('Deleting files...');
+        // const subject = matches.length === 1 ? 'model' : 'models'
+        const modelText = matches.length === 1 ? matches[0].name : `${matches.length} models`;
+        console.log(`Deleting ${modelText} ...`);
         await deleteFiles(matches);
-        console.log(chalk.green('Deleted files'));
+        console.log(chalk.green('Done'));
     }
     catch (error) {
-        console.error(chalk.red(`Error deleting files: ${error.message}`));
+        console.error(chalk.red(`Error during removal: ${error.message}`));
     }
 }
 export const removeCommand = {