@archimonde12/llm-proxy 1.0.0

package/LICENSE

@@ -0,0 +1,20 @@
+MIT License
+
+Copyright (c) 2026
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, as long as the following conditions are met:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,272 @@
+# llm-proxy
+
+**llm-proxy** is a lightweight, high-performance gateway designed to unify multiple LLM backends (Ollama, vLLM, OpenAI-compatible servers, etc.) into a single, standardized OpenAI-compatible API endpoint.
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Node.js Version](https://img.shields.io/badge/node-%3E%3D20-brightgreen)](https://nodejs.org/)
+[![Package Manager](https://img.shields.io/badge/package-pnpm-ff6302.svg)](https://pnpm.io/)
+[![TypeScript](https://img.shields.io/badge/TypeScript-%233178c6.svg?style=flat&logo=typescript&logoColor=white)](https://www.typescriptlang.org/)
+[![Fastify](https://img.shields.io/badge/Fastify-black?logo=fastify)](https://www.fastify.io/)
+[![Build Status](https://img.shields.io/badge/build%20passing-%234caf50.svg)](#)
+
+---
+
+## Key features
+
+*   **Unified interface**: Access different backends (Ollama, vLLM, etc.) using the standard OpenAI SDK and payload format.
+*   **Multi-backend support**: Seamlessly route requests to various providers via a simple `models.json` configuration.
+*   **Dynamic configuration**: Update your model list and backend URLs on-the-fly via the **Admin API** without restarting the server.
+*   **Built-in observability**:
+    *   **Prometheus metrics**: Monitor request counts, latency, and token usage.
+    *   **Request history**: Track recent requests via an in-memory buffer.
+*   **Web UI**: A built-in dashboard to manage configuration, environment variables, and monitoring (when `ui/dist` is present).
+*   **Streaming**: Full support for Server-Sent Events (SSE) for real-time chat completions.
+*   **Lightweight and fast**: Built with **Fastify** and **TypeScript** for minimal overhead and maximum throughput.
+
+---
+
+## Quick start
+
+### Prerequisites
+
+*   [Node.js](https://nodejs.org/) (**v20+ required**)
+*   [pnpm](https://pnpm.io/)
+
+### Installation
+
+**From npm (global CLI)** — no clone required; the published package includes `dist/` and `ui/dist` (dashboard at `/ui` when you run the server).
+
+```bash
+npm install -g @archimonde12/llm-proxy
+```
+
+Requires **Node.js 20+** (see `engines` in `package.json`). After installing, run `llm-proxy --help` or `llm-proxy start`.
+
+**Without global install (npx):**
+
+```bash
+npx @archimonde12/llm-proxy --help
+```
+
+Configuration file resolution is described under [Configuration](#configuration) (important when using a global install from arbitrary working directories).
+
+**From source** (development):
+
+```bash
+# Clone the repository (replace OWNER with your fork or upstream)
+git clone https://github.com/OWNER/llm-proxy.git
+cd llm-proxy
+
+# Install dependencies
+pnpm install
+```
+
+### Running the server
+
+**Development** (TypeScript; no production UI bundle required for API-only work):
+
+```bash
+pnpm dev
+```
+
+In **development** (`pnpm dev`), the default bind is **0.0.0.0:8787** (override with `HOST` and `PORT`). In **`llm-proxy start`** / **`pnpm start`**, the default bind is **127.0.0.1:8787** unless you set `HOST`, `PORT`, or pass `--host` / `--port`.
+
+**Production** (compiled server and built web UI):
+
+```bash
+npm run build   # or: pnpm build — Vite UI bundle + TypeScript; produces ui/dist and dist/
+pnpm start      # or: npm start — runs node dist/index.js
+```
+
+A full build always runs the UI step first, so `ui/dist` is present afterward. Maintainers can use **npm** or **pnpm** for `build` / `start`; the build script does not invoke `pnpm` internally.
+
+### CLI (`llm-proxy`)
+
+With **`npm install -g @archimonde12/llm-proxy`**, use the `llm-proxy` command on your `PATH`.
+
+From a **source** tree after `npm run build` / `pnpm build`:
+
+```bash
+pnpm cli -- <subcommand> [options]
+# equivalent:
+node dist/cli/bin.js <subcommand> [options]
+```
+
+| Command | Purpose |
+| :--- | :--- |
+| `init` | Create a starter `models.json` (wizard when interactive; use `-y` for defaults). Options: `--file <path>`, `-y` / `--yes`. |
+| `start` | Start the HTTP server. Options: `--models <path>`, `--port <port>` (default **8787**), `--host <host>` (default **127.0.0.1**). |
+| `status` | Call public **`GET /healthz`**. Options: `--url <baseUrl>` (default `http://127.0.0.1:8787`). |
+| `doctor` | Validate config, check listen port, optionally ping upstreams. Options: `--models <path>`, `--host`, `--port`, `--deep`. |
+| `config validate` | Validate `models.json` against the schema. Options: `--file <path>` (default `./models.json`). |
+
+### Environment variables (server / CLI)
+
+| Variable | Used by | Description |
+| :--- | :--- | :--- |
+| `MODELS_PATH` | `pnpm dev`, `start`, `doctor` | Path to `models.json`. For `start` and `doctor`, only used when `--models` is omitted; `pnpm dev` always resolves the file from env / defaults (no `--models` flag). |
+| `PORT` | `pnpm dev`, `start` | Listen port if unset (default **8787**); `start` also accepts `--port`. |
+| `HOST` | `pnpm dev`, `start` | Bind address if unset — **`pnpm dev`** defaults to **0.0.0.0**, **`start`** defaults to **127.0.0.1**; `start` also accepts `--host`. |
+| `LOG_LEVEL` | Server | Optional: `debug`, `info`, `warn`, or `error`; omit to log at all levels. |
+
+---
+
+## Configuration
+
+The proxy uses a `models.json` file to map your custom model IDs to specific backends.
+
+**Resolution order** (see [`src/config/load.ts`](src/config/load.ts)):
+
+1. **`llm-proxy start --models <path>`** — explicit file path.
+2. **`MODELS_PATH`** — if set, that path is used (a starter file is created if missing).
+3. **Otherwise:** **`./models.json`** relative to the **current working directory** — if it exists, it is used.
+4. **Otherwise:** **`~/.config/llm-proxy/models.json`** — canonical user config; if it exists, it is used (good default for a **global** install when you are not in a project directory).
+5. **Otherwise:** **`~/.config/llm-open-gateway/models.json`** — legacy path for backward compatibility with older installs; used only if the canonical path above does not exist.
+6. **Otherwise:** a starter `models.json` is created at **`./models.json`** in the current working directory.
+
+This means a global install does not require a checkout: use a file in the cwd, set `MODELS_PATH`, pass `--models`, or keep your config under `~/.config/llm-proxy/models.json`.
+
+### Example `models.json`
+
+```json
+{
+  "models": [
+    {
+      "id": "ollama-llama3",
+      "adapter": "ollama",
+      "baseUrl": "http://localhost:11434",
+      "model": "llama3"
+    },
+    {
+      "id": "vllm-mixtral",
+      "adapter": "openai_compatible",
+      "baseUrl": "http://localhost:8000",
+      "model": "mixtral-8x7b",
+      "apiKey": "your-secret-api-key"
+    }
+  ]
+}
+```
+
+### Supported adapters
+
+| Adapter | Description |
+| :--- | :--- |
+| `ollama` | Optimized for the [Ollama](https://ollama.com/) HTTP API. |
+| `openai_compatible` | Any server implementing the OpenAI `/v1/chat/completions` contract. |
+| `deepseek` | DeepSeek-compatible HTTP API (see [`src/adapters/deepseek.ts`](src/adapters/deepseek.ts)). |
+
+### Optional fields (per model)
+
+| Field | Description |
+| :--- | :--- |
+| `apiKey` | Secret sent to the upstream (see **Secrets** below). |
+| `apiKeyHeader` | Header name for the API key (requires `apiKey`). |
+| `headers` | Extra static headers as string key/value pairs. |
+| `timeoutMs` | Upstream request timeout in milliseconds. |
+
+### Secrets: `apiKey` and `${ENV_VAR}`
+
+To avoid committing raw keys, you can set `apiKey` to a **single** environment placeholder: the string must be exactly `${` + the variable name + `}`, where the name uses only letters, digits, and underscores. On load, the server replaces that string with the value from `process.env`. If the variable is unset or empty, `apiKey` is dropped so the rest of the entry can still pass validation.
+
+After changing environment variables (for example via **`PUT /admin/env`**), call **`POST /admin/reload`** so `${...}` placeholders are resolved again from the updated process environment.
+
+---
+
+## Web UI (dashboard)
+
+The UI is a static SPA served from **`ui/dist`**. The server registers it **only if that directory exists**; otherwise there is no **`GET /`** redirect to the app (see [`src/server.ts`](src/server.ts)).
+
+**Prerequisite:** run **`pnpm build:ui`** or a full **`pnpm build`** before starting the server if you want the dashboard.
+
+**URLs**
+
+* **`GET /`** → **302** to **`/ui/`** when the UI bundle is present.
+* Static assets are served under **`/ui/`**.
+
+**In-app routes** (hash-based):
+
+* **`#/configuration`** — Edit `models.json` and environment variables through the Admin API.
+* **`#/monitoring`** — Metrics overview with time ranges **15m**, **1h**, and **24h** (aligned with `/admin/metrics/overview`).
+* **`#/models`** — Model list, request logs, and debug message capture.
+
+**Security:** When the process binds to a non-loopback address, **`/admin/*`** and **`/ui/*`** are restricted to localhost clients. To use the dashboard remotely, use local access, SSH port forwarding, or a tunnel you trust.
+
+**UI development:** There is no separate `package.json` under `ui/`; for a local Vite dev server you can run e.g. `pnpm exec vite --config ui/vite.config.ts` from the repo root. Production assets are built with **`pnpm build:ui`**.
+
+---
+
+## Admin API, health, and metrics
+
+*Note: Admin routes are restricted to `localhost` when the server is bound to a non-loopback interface.*
+
+### Public health (no admin guard)
+
+| Method | Path | Description |
+| :--- | :--- | :--- |
+| `GET` | `/healthz` | Liveness: returns `{ ok: true }` while the process is running. Used by **`llm-proxy status`**. |
+| `GET` | `/readyz` | Readiness. With **`?deep=1`**, probes each model's upstream; may return **503** if any probe fails. |
+
+### Admin API (localhost when exposed on a public bind)
+
+**Config and environment**
+
+| Method | Path | Description |
+| :--- | :--- | :--- |
+| `GET` | `/admin/health` | Process/version hint and active config path (not the same as public `/healthz`). |
+| `GET` | `/admin/config` | Current configuration, write target, and metadata. |
+| `PUT` | `/admin/config` | Replace configuration (validated); writes atomically. |
+| `POST` | `/admin/reload` | Reload `models.json` from disk. |
+| `GET` | `/admin/env` | List relevant env keys and `.env` file metadata. |
+| `PUT` | `/admin/env` | Apply env updates to `.env` and `process.env`. |
+
+**Connectivity and metrics**
+
+| Method | Path | Description |
+| :--- | :--- | :--- |
+| `POST` | `/admin/test-connection` | Probe a configured **`modelId`** or an inline adapter/baseUrl/model payload. |
+| `GET` | `/admin/metrics/summary` | JSON snapshot from in-process metrics. |
+| `GET` | `/admin/metrics/overview` | Overview for **`range=15m`**, **`1h`**, or **`24h`** (query param). |
+
+**Logs and requests**
+
+| Method | Path | Description |
+| :--- | :--- | :--- |
+| `GET` | `/admin/requests` | Recent proxy request history (`limit` query param, capped). |
+| `GET` | `/admin/requests/:requestId` | Single request record by id. |
+| `GET` | `/admin/logs/models` | Model-scoped logs with **`range`**, optional **`modelId`**, **`status`**, **`limit`**. |
+| `GET` | `/admin/models/:modelId/debug/messages` | Recent captured **system** / **user** messages for debugging (`limit`, **`roles`**). |
+
+### Prometheus
+
+| Method | Path | Description |
+| :--- | :--- | :--- |
+| `GET` | `/metrics` | Prometheus text exposition format. |
+
+---
+
+## Security and exposure
+
+If you intend to expose `llm-proxy` to the internet (e.g., via **ngrok** or a reverse proxy), please follow these best practices:
+
+1.  **Use HTTPS**: Always use an SSL/TLS tunnel or terminating proxy.
+2.  **Admin and UI**: The server restricts `/admin/*` and `/ui/*` to localhost when it detects a non-loopback bind.
+3.  **Upstream credentials**: Use `apiKey` / `${ENV_VAR}` in `models.json` for provider authentication.
+
+**Example with ngrok:**
+
+```bash
+ngrok http 8787
+```
+
+---
+
+## Contributing
+
+Pull requests are welcome; please keep changes focused and consistent with existing patterns.
+
+---
+
+## License
+
+Distributed under the **MIT License**. See [`LICENSE`](LICENSE) for more information.

package/dist/adapters/base.js

@@ -0,0 +1,2 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });

package/dist/adapters/deepseek.js

@@ -0,0 +1,78 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.createDeepseekAdapter = createDeepseekAdapter;
+const mergeHeaders_1 = require("../config/mergeHeaders");
+const http_1 = require("../http");
+function headersToRecord(headers) {
+    const out = {};
+    headers.forEach((value, key) => {
+        out[key] = value;
+    });
+    return out;
+}
+function contentToText(content) {
+    if (content === null || content === undefined)
+        return null;
+    if (typeof content === "string")
+        return content;
+    if (Array.isArray(content)) {
+        const parts = [];
+        for (const block of content) {
+            if (!block || typeof block !== "object")
+                continue;
+            const anyBlock = block;
+            if (anyBlock.type !== "text")
+                continue;
+            if (typeof anyBlock.text === "string")
+                parts.push(anyBlock.text);
+        }
+        return parts.join("\n");
+    }
+    return String(content);
+}
+function normalizeMessages(messages) {
+    return messages.map((m) => {
+        const normalized = contentToText(m.content);
+        return { ...m, content: normalized };
+    });
+}
+function createDeepseekAdapter(cfg) {
+    const outboundHeaders = (0, mergeHeaders_1.mergeModelOutboundHeaders)(cfg);
+    return {
+        async chatCompletions(req) {
+            const url = (0, http_1.joinUrl)(cfg.baseUrl, "v1/chat/completions");
+            const forwarded = {
+                ...req,
+                model: cfg.model,
+                messages: normalizeMessages(req.messages),
+            };
+            const res = await (0, http_1.postJson)(url, forwarded, {
+                headers: outboundHeaders,
+                timeoutMs: cfg.timeoutMs,
+            });
+            return {
+                status: res.status,
+                headers: headersToRecord(res.headers),
+                body: res.json,
+            };
+        },
+        async chatCompletionsStream(req) {
+            const url = (0, http_1.joinUrl)(cfg.baseUrl, "v1/chat/completions");
+            const forwarded = {
+                ...req,
+                model: cfg.model,
+                stream: true,
+                messages: normalizeMessages(req.messages),
+            };
+            const res = await (0, http_1.postJsonStream)(url, forwarded, {
+                headers: outboundHeaders,
+                timeoutMs: cfg.timeoutMs,
+            });
+            return {
+                status: res.status,
+                headers: headersToRecord(res.headers),
+                body: res.body,
+            };
+        },
+    };
+}

package/dist/adapters/index.js

@@ -0,0 +1,20 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.createAdapter = createAdapter;
+const deepseek_1 = require("./deepseek");
+const ollama_1 = require("./ollama");
+const openaiCompatible_1 = require("./openaiCompatible");
+function createAdapter(cfg) {
+    switch (cfg.adapter) {
+        case "deepseek":
+            return (0, deepseek_1.createDeepseekAdapter)(cfg);
+        case "ollama":
+            return (0, ollama_1.createOllamaAdapter)(cfg);
+        case "openai_compatible":
+            return (0, openaiCompatible_1.createOpenAICompatibleAdapter)(cfg);
+        default: {
+            const neverCfg = cfg.adapter;
+            throw new Error(`Unsupported adapter: ${neverCfg}`);
+        }
+    }
+}

package/dist/adapters/ollama.js

@@ -0,0 +1,182 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.createOllamaAdapter = createOllamaAdapter;
+const mergeHeaders_1 = require("../config/mergeHeaders");
+const http_1 = require("../http");
+function headersToRecord(headers) {
+    const out = {};
+    headers.forEach((value, key) => {
+        out[key] = value;
+    });
+    return out;
+}
+function messagesToPrompt(req) {
+    // Simple, predictable mapping: keep roles inline for local models.
+    return req.messages
+        .map((m) => {
+        const content = Array.isArray(m.content) ? "" : (m.content ?? "");
+        return `${m.role.toUpperCase()}: ${content}`;
+    })
+        .join("\n");
+}
+function createOllamaAdapter(cfg) {
+    const outboundHeaders = (0, mergeHeaders_1.mergeModelOutboundHeaders)(cfg);
+    return {
+        async chatCompletions(req) {
+            const url = (0, http_1.joinUrl)(cfg.baseUrl, "api/generate");
+            const body = {
+                model: cfg.model,
+                prompt: messagesToPrompt(req),
+                stream: false,
+                options: {
+                    temperature: req.temperature,
+                    num_predict: req.max_tokens,
+                },
+            };
+            const res = await (0, http_1.postJson)(url, body, {
+                headers: outboundHeaders,
+                timeoutMs: cfg.timeoutMs,
+            });
+            // Convert Ollama's response into an OpenAI-ish shape so clients
+            // can consistently consume `/v1/chat/completions`.
+            const content = res.json && typeof res.json === "object" && "response" in res.json
+                ? String(res.json.response ?? "")
+                : "";
+            const openaiLike = {
+                id: `chatcmpl_ollama_${Date.now()}`,
+                object: "chat.completion",
+                created: Math.floor(Date.now() / 1000),
+                model: cfg.model,
+                choices: [
+                    {
+                        index: 0,
+                        message: { role: "assistant", content },
+                        finish_reason: "stop",
+                    },
+                ],
+            };
+            return {
+                status: res.status,
+                headers: headersToRecord(res.headers),
+                body: openaiLike,
+            };
+        },
+        async chatCompletionsStream(req) {
+            const url = (0, http_1.joinUrl)(cfg.baseUrl, "api/generate");
+            const body = {
+                model: cfg.model,
+                prompt: messagesToPrompt(req),
+                stream: true,
+                options: {
+                    temperature: req.temperature,
+                    num_predict: req.max_tokens,
+                },
+            };
+            const upstream = await (0, http_1.postJsonStream)(url, body, {
+                headers: outboundHeaders,
+                timeoutMs: cfg.timeoutMs,
+            });
+            // Ollama returns NDJSON objects per line:
+            // { response: "...", done: boolean, ... }
+            // Convert to OpenAI ChatCompletions SSE (chat.completion.chunk).
+            const encoder = new TextEncoder();
+            const decoder = new TextDecoder();
+            const stream = new ReadableStream({
+                async start(controller) {
+                    const id = `chatcmpl_ollama_${Date.now()}`;
+                    const created = Math.floor(Date.now() / 1000);
+                    let sentRole = false;
+                    let buffer = "";
+                    const writeSse = (data) => {
+                        controller.enqueue(encoder.encode(`data: ${JSON.stringify(data)}\n\n`));
+                    };
+                    try {
+                        if (!upstream.body) {
+                            controller.enqueue(encoder.encode(`data: ${JSON.stringify({
+                                error: { message: "Upstream returned empty body" },
+                            })}\n\n`));
+                            controller.enqueue(encoder.encode("data: [DONE]\n\n"));
+                            controller.close();
+                            return;
+                        }
+                        const reader = upstream.body.getReader();
+                        while (true) {
+                            const { value, done } = await reader.read();
+                            if (done)
+                                break;
+                            if (!value)
+                                continue;
+                            buffer += decoder.decode(value, { stream: true });
+                            let idx;
+                            while ((idx = buffer.indexOf("\n")) >= 0) {
+                                const line = buffer.slice(0, idx).trim();
+                                buffer = buffer.slice(idx + 1);
+                                if (!line)
+                                    continue;
+                                let obj;
+                                try {
+                                    obj = JSON.parse(line);
+                                }
+                                catch {
+                                    continue;
+                                }
+                                if (!sentRole) {
+                                    sentRole = true;
+                                    writeSse({
+                                        id,
+                                        object: "chat.completion.chunk",
+                                        created,
+                                        model: cfg.model,
+                                        choices: [{ index: 0, delta: { role: "assistant" }, finish_reason: null }],
+                                    });
+                                }
+                                const deltaText = obj && typeof obj === "object" && "response" in obj
+                                    ? String(obj.response ?? "")
+                                    : "";
+                                if (deltaText) {
+                                    writeSse({
+                                        id,
+                                        object: "chat.completion.chunk",
+                                        created,
+                                        model: cfg.model,
+                                        choices: [
+                                            { index: 0, delta: { content: deltaText }, finish_reason: null },
+                                        ],
+                                    });
+                                }
+                                if (obj?.done) {
+                                    writeSse({
+                                        id,
+                                        object: "chat.completion.chunk",
+                                        created,
+                                        model: cfg.model,
+                                        choices: [{ index: 0, delta: {}, finish_reason: "stop" }],
+                                    });
+                                    controller.enqueue(encoder.encode("data: [DONE]\n\n"));
+                                    controller.close();
+                                    return;
+                                }
+                            }
+                        }
+                        controller.enqueue(encoder.encode("data: [DONE]\n\n"));
+                        controller.close();
+                    }
+                    catch (e) {
+                        controller.enqueue(encoder.encode(`data: ${JSON.stringify({ error: { message: e?.message ?? "Stream error" } })}\n\n`));
+                        controller.enqueue(encoder.encode("data: [DONE]\n\n"));
+                        controller.close();
+                    }
+                },
+            });
+            return {
+                status: upstream.status,
+                headers: {
+                    "content-type": "text/event-stream; charset=utf-8",
+                    "cache-control": "no-cache",
+                    connection: "keep-alive",
+                },
+                body: stream,
+            };
+        },
+    };
+}

package/dist/adapters/openaiCompatible.js

@@ -0,0 +1,50 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.createOpenAICompatibleAdapter = createOpenAICompatibleAdapter;
+const mergeHeaders_1 = require("../config/mergeHeaders");
+const http_1 = require("../http");
+function headersToRecord(headers) {
+    const out = {};
+    headers.forEach((value, key) => {
+        out[key] = value;
+    });
+    return out;
+}
+function createOpenAICompatibleAdapter(cfg) {
+    const outboundHeaders = (0, mergeHeaders_1.mergeModelOutboundHeaders)(cfg);
+    return {
+        async chatCompletions(req) {
+            const url = (0, http_1.joinUrl)(cfg.baseUrl, "v1/chat/completions");
+            const forwarded = {
+                ...req,
+                model: cfg.model,
+            };
+            const res = await (0, http_1.postJson)(url, forwarded, {
+                headers: outboundHeaders,
+                timeoutMs: cfg.timeoutMs,
+            });
+            return {
+                status: res.status,
+                headers: headersToRecord(res.headers),
+                body: res.json,
+            };
+        },
+        async chatCompletionsStream(req) {
+            const url = (0, http_1.joinUrl)(cfg.baseUrl, "v1/chat/completions");
+            const forwarded = {
+                ...req,
+                model: cfg.model,
+                stream: true,
+            };
+            const res = await (0, http_1.postJsonStream)(url, forwarded, {
+                headers: outboundHeaders,
+                timeoutMs: cfg.timeoutMs,
+            });
+            return {
+                status: res.status,
+                headers: headersToRecord(res.headers),
+                body: res.body,
+            };
+        },
+    };
+}

package/dist/admin/auth.js

@@ -0,0 +1,37 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.shouldEnforceLocalhostGuard = shouldEnforceLocalhostGuard;
+exports.isLocalhostRequest = isLocalhostRequest;
+exports.isLocalhostAddress = isLocalhostAddress;
+exports.sendForbiddenNonLocal = sendForbiddenNonLocal;
+/** True when the server accepts remote connections (not loopback-only). */
+function shouldEnforceLocalhostGuard(bindHost) {
+    const h = bindHost.trim().toLowerCase();
+    if (h === "127.0.0.1" || h === "::1" || h === "localhost")
+        return false;
+    return true;
+}
+function isLocalhostRequest(req) {
+    const raw = req.socket?.remoteAddress ??
+        req.ip ??
+        "";
+    return isLocalhostAddress(String(raw));
+}
+function isLocalhostAddress(addr) {
+    if (!addr)
+        return false;
+    if (addr === "127.0.0.1" || addr === "::1")
+        return true;
+    if (addr.startsWith("::ffff:")) {
+        const v4 = addr.slice("::ffff:".length);
+        return v4 === "127.0.0.1";
+    }
+    return false;
+}
+function sendForbiddenNonLocal(reply) {
+    return reply.code(403).send({
+        error: {
+            message: "Admin and UI are only available from localhost when the server accepts remote connections.",
+        },
+    });
+}