nuxt-local-model 0.1.1

package/README.md

@@ -0,0 +1,332 @@
+# Nuxt Local Model
+
+[![npm version][npm-version-src]][npm-version-href]
+[![npm downloads][npm-downloads-src]][npm-downloads-href]
+[![License][license-src]][license-href]
+
+## Scalable local inference for Nuxt
+
+<img width="1280" height="640" alt="Nuxt Local Model banner" src="./assets/module-banner.svg" />
+
+Note: This package is under active development. Please open issues if you run into anything unclear.
+
+- [✨ &nbsp;Release Notes](/CHANGELOG.md)
+- [📖 &nbsp;Documentation](https://github.com/Aft1n/nuxt-local-model)
+
+A Nuxt module for easily integrating local Hugging Face transformer models into your Nuxt 4 application.
+
+## Features
+
+- Easily use local models in your Nuxt app
+- Supports any Hugging Face task and model you want to configure
+- Auto-imported composable, `useLocalModel()` by default for frontend Vue code
+- Server-safe helper, `getLocalModel()` for `server/api` and utilities
+- Fully configurable via `nuxt.config.ts`
+- Supports changing model names, tasks, and settings per usage
+- Optional worker-backed execution on the server or in the browser
+- Server runtime support for Node, Bun, and Deno
+- Works across macOS, Linux, Windows, and Docker
+- Supports persistent model cache directories so models are not re-downloaded on every deploy
+
+## Quick Setup
+
+Install the module into your Nuxt application with one command:
+
+```bash
+npx nuxi module add nuxt-local-model
+```
+
+## Manual Installation
+
+If you prefer to install manually, run:
+
+```bash
+# Using npm
+npm install nuxt-local-model
+
+# Using yarn
+yarn add nuxt-local-model
+
+# Using pnpm
+pnpm add nuxt-local-model
+
+# Using bun
+bun add nuxt-local-model
+```
+
+Then, add it to your Nuxt config:
+
+```ts
+export default defineNuxtConfig({
+  modules: ["nuxt-local-model"],
+})
+```
+
+## Usage
+
+Once installed, you can use `useLocalModel()` in your Vue app code.
+
+For server routes and utilities, use `getLocalModel()`.
+
+### Basic Example
+
+```vue
+<script setup lang="ts">
+const embedder = await useLocalModel("embedding")
+const output = await embedder("Nuxt local model example")
+</script>
+```
+
+### Server Example
+
+```ts
+// server/api/demo/search.get.ts
+import { getLocalModel } from "nuxt-local-model/server"
+
+export default defineEventHandler(async () => {
+  const embedder = await getLocalModel("embedding")
+  return await embedder("hello world")
+})
+```
+
+### Defining Models in `nuxt.config.ts`
+
+```ts
+import { defineLocalModelConfig } from "nuxt-local-model"
+
+export default defineNuxtConfig({
+  modules: ["nuxt-local-model"],
+  localModel: defineLocalModelConfig({
+    runtime: "auto", // auto-detect Node, Bun, or Deno on the server
+    cacheDir: "./.ai-models", // one cache folder for downloads and reuse
+    allowRemoteModels: true, // allow fetching missing models from Hugging Face
+    allowLocalModels: true, // allow reusing cached / mounted model files
+    defaultTask: "feature-extraction", // default pipeline type when a model entry does not override it
+    serverWorker: false, // run inference in a server worker thread on Node, Bun, or Deno
+    browserWorker: false, // run inference in a browser Web Worker; avoid this for very large models
+    models: {
+      embedding: {
+        task: "feature-extraction", // the pipeline type for this alias
+        model: "Xenova/all-MiniLM-L6-v2", // the Hugging Face model id
+        options: {
+          dtype: "q8", // model loading option passed through to Transformers.js
+        },
+      },
+    },
+  }),
+})
+```
+
+Tip: `defineLocalModelConfig()` keeps your alias keys as literal types, so you can reuse them
+with `LocalModelAliases<typeof localModel>` if you want exact autocomplete elsewhere in your app.
+If you are writing server routes, import `getLocalModel()` from `nuxt-local-model/server`.
+In Vue app code, `useLocalModel()` is auto-imported once the module is installed.
+
+### Overriding Settings at the Call Site
+
+You can still provide the options for the model call where it is used:
+
+```vue
+<script setup lang="ts">
+const model = await useLocalModel("embedding", {
+  pooling: "mean",
+  normalize: true,
+})
+</script>
+```
+
+## Configuration Options
+
+You can configure the module in your `nuxt.config.ts`:
+
+```ts
+import { defineLocalModelConfig } from "nuxt-local-model"
+
+export default defineNuxtConfig({
+  modules: ["nuxt-local-model"],
+  localModel: defineLocalModelConfig({
+    runtime: "auto", // or "node", "bun", or "deno"
+    cacheDir: "./.ai-models", // persistent cache folder for downloaded model assets
+    allowRemoteModels: true, // download from Hugging Face if not yet cached
+    allowLocalModels: true, // reuse local cache or mounted volume contents
+    defaultTask: "feature-extraction", // default for aliases that do not override task
+    serverWorker: true, // use a server worker thread so inference does not block the main server thread
+    browserWorker: false, // enable only if you intentionally want browser-side inference
+    models: {
+      embedding: {
+        task: "feature-extraction", // embeddings usually use feature-extraction
+        model: "Xenova/all-MiniLM-L6-v2", // any Hugging Face model id you choose
+        options: {
+          dtype: "q8", // loading/config option forwarded to Transformers.js
+        },
+      },
+    },
+  }),
+})
+```
+
+If `onnxruntime-node` is not available in your server runtime, the module now falls back to the default Transformers.js backend instead of crashing during startup.
+
+### Cache Directory
+
+The cache directory controls where downloaded model files are stored and reused.
+
+Recommended defaults:
+
+- local development: `./.ai-models`
+- Docker: mount a persistent volume to the same path
+
+Important:
+
+- the cache path in `nuxt.config.ts` must match the path inside the Docker container
+- the folder name on your laptop does not have to match the Docker folder name
+- what matters in production is the path the app reads inside the container
+
+Example Docker runtime setup:
+
+```bash
+docker run \
+  -e NUXT_LOCAL_MODEL_CACHE_DIR=/data/local-models \
+  -v local-models:/data/local-models \
+  your-image:latest
+```
+
+This ensures the model files stay available across redeploys and container restarts.
+
+What this does:
+
+- `NUXT_LOCAL_MODEL_CACHE_DIR=/data/local-models` tells the app which folder to use for model caching
+- `-v local-models:/data/local-models` mounts a persistent Docker volume at that same folder
+  - the first container start downloads missing models into the mounted cache folder
+  - later starts reuse the models already stored there
+
+You can rename the host-facing volume however you want. What matters is that the path inside
+the container matches the cache path used by the module.
+
+In Docker, the environment variable and volume path point the app to the mounted folder:
+
+```dockerfile
+ENV NUXT_LOCAL_MODEL_CACHE_DIR=/models-cache
+VOLUME ["/models-cache"]
+```
+
+That means the Nuxt app will use `/models-cache` inside the container, and Docker will
+attach a persistent volume there when you run the container with `-v`.
+
+### Docker Volume Cache Example
+
+If you want Docker to download model files on first launch and reuse them on later redeploys,
+mount a persistent volume at the same cache path the app uses.
+
+The build does not need to copy model files manually. The first container start writes them
+into the mounted volume, and subsequent starts reuse whatever is already there.
+
+```dockerfile
+FROM node:22-alpine AS deps
+WORKDIR /app
+
+COPY package.json pnpm-lock.yaml ./
+RUN corepack enable && pnpm install --frozen-lockfile
+
+FROM deps AS build
+WORKDIR /app
+
+COPY . .
+
+ENV NUXT_LOCAL_MODEL_CACHE_DIR=/models-cache
+RUN pnpm run build
+
+FROM node:22-alpine
+WORKDIR /app
+
+ENV NUXT_LOCAL_MODEL_CACHE_DIR=/models-cache
+VOLUME ["/models-cache"]
+
+COPY --from=build /app/.output ./.output
+COPY --from=deps /app/node_modules ./node_modules
+
+CMD ["node", ".output/server/index.mjs"]
+```
+
+Use this as a template in your Nuxt Docker build if you want a persistent cache path.
+At runtime, the mounted volume should be attached to `/models-cache`, and the app will
+download missing models into that volume the first time it runs.
+
+In other words:
+
+- your local dev cache can be `./.ai-models`
+- your Docker cache can be `/models-cache`
+- both are fine as long as the app config matches the environment it runs in
+
+### Naming Rule
+
+- `useLocalModel()` is for frontend Vue components, pages, and composables
+- `getLocalModel()` is for `server/api` routes and Nitro utilities
+
+Both use the same underlying model-loading logic, so the runtime behavior stays consistent.
+
+### Worker Mode
+
+You can choose where the model runs:
+
+- `serverWorker: true` runs model inference in a Node worker thread on your Nuxt server
+- `browserWorker: true` runs model inference in a browser Web Worker
+
+This is useful if you want to keep heavy inference off the main request or UI thread.
+
+Be careful with `browserWorker` and large models:
+
+- the model must be downloaded into the user’s browser
+- 100s of MB models can be slow or impractical for client delivery
+- server worker mode is usually the better default for large models
+
+### Server Worker vs Browser Worker
+
+| Mode            | Where it runs                    | Best for                                                    | Tradeoff                                  |
+| --------------- | -------------------------------- | ----------------------------------------------------------- | ----------------------------------------- |
+| `serverWorker`  | Nuxt server / Node worker thread | Large models, shared cache, server-rendered apps            | Uses server CPU and memory                |
+| `browserWorker` | User’s browser Web Worker        | Small client-side models, privacy-sensitive local inference | Model must be downloaded into the browser |
+
+## Transformers.js Docs
+
+For model/task behavior and runtime options, see the official Transformers.js docs:
+
+- [Transformers.js docs](https://huggingface.co/docs/transformers.js/main)
+- [Environment settings](https://huggingface.co/docs/transformers.js/main/api/env)
+- [Pipeline behavior](https://huggingface.co/docs/transformers/en/main_classes/pipelines)
+
+## Playground
+
+This package includes a minimal playground app with an embedding example inside `playground/`.
+
+The playground keeps the note list in the page and uses server routes for embeddings and search, so it demonstrates the server-backed flow end to end without a database.
+
+Run it with:
+
+```bash
+npm run dev
+```
+
+## Publishing
+
+If you want to publish this module to GitHub and npm:
+
+1. `cd nuxt-local-model`
+2. `git init`
+3. commit the files
+4. create a GitHub repository
+5. connect the remote and push
+6. run `npm login`
+7. publish with `npm publish --access public`
+
+## Notes
+
+- This module is intentionally generic and does not ship opinionated preset models.
+- The example playground shows how to wire an embedding model, but you can register any task/model combination supported by `@huggingface/transformers`.
+
+[npm-version-src]: https://img.shields.io/npm/v/nuxt-local-model?style=flat-square
+[npm-version-href]: https://www.npmjs.com/package/nuxt-local-model
+[npm-downloads-src]: https://img.shields.io/npm/dm/nuxt-local-model?style=flat-square
+[npm-downloads-href]: https://www.npmjs.com/package/nuxt-local-model
+[license-src]: https://img.shields.io/npm/l/nuxt-local-model?style=flat-square
+[license-href]: https://opensource.org/licenses/MIT

package/dist/module.d.mts

@@ -0,0 +1,18 @@
+import * as nuxt_schema from 'nuxt/schema';
+import { LocalModelRuntimeConfig } from '../dist/runtime/types.js';
+
+interface NuxtLlmModuleOptions extends LocalModelRuntimeConfig {
+}
+declare const _default: nuxt_schema.NuxtModule<NuxtLlmModuleOptions, {
+    runtime: "auto";
+    cacheDir: string;
+    allowRemoteModels: true;
+    allowLocalModels: true;
+    defaultTask: "feature-extraction";
+    serverWorker: false;
+    browserWorker: false;
+    models: {};
+}, true>;
+
+export { _default as default };
+export type { NuxtLlmModuleOptions };

package/dist/module.json

@@ -0,0 +1,9 @@
+{
+  "name": "nuxt-local-model",
+  "configKey": "localModel",
+  "version": "0.1.1",
+  "builder": {
+    "@nuxt/module-builder": "1.0.2",
+    "unbuild": "unknown"
+  }
+}

package/dist/module.mjs

@@ -0,0 +1,68 @@
+import { defineNuxtModule, createResolver, addImports, addPlugin } from '@nuxt/kit';
+import { existsSync } from 'node:fs';
+import { setLocalModelRuntimeConfig } from '../dist/runtime/shared/local-model.js';
+
+const module$1 = defineNuxtModule().with({
+  meta: {
+    name: "nuxt-local-model",
+    configKey: "localModel"
+  },
+  defaults: {
+    runtime: "auto",
+    cacheDir: "./.ai-models",
+    allowRemoteModels: true,
+    allowLocalModels: true,
+    defaultTask: "feature-extraction",
+    serverWorker: false,
+    browserWorker: false,
+    models: {}
+  },
+  setup(options, nuxt) {
+    const { resolve } = createResolver(import.meta.url);
+    const serverWorkerJs = resolve("./runtime/server/worker.js");
+    const serverWorkerTs = resolve("./runtime/server/worker.ts");
+    const serverWorkerEntry = existsSync(serverWorkerJs) ? serverWorkerJs : serverWorkerTs;
+    setLocalModelRuntimeConfig({
+      ...options,
+      serverWorkerEntry
+    });
+    nuxt.options.runtimeConfig.public ||= {};
+    nuxt.options.runtimeConfig.public.localModel = {
+      cacheDir: options.cacheDir,
+      allowRemoteModels: options.allowRemoteModels,
+      allowLocalModels: options.allowLocalModels,
+      runtime: options.runtime,
+      defaultTask: options.defaultTask,
+      serverWorker: options.serverWorker,
+      serverWorkerEntry,
+      browserWorker: options.browserWorker,
+      models: options.models
+    };
+    addImports({
+      name: "useLocalModel",
+      from: resolve("./runtime/composables/useLocalModel")
+    });
+    addPlugin({
+      src: resolve("./runtime/plugins/hf-transformers.server")
+    });
+    addPlugin({
+      src: resolve("./runtime/plugins/hf-transformers.client"),
+      mode: "client"
+    });
+    nuxt.hook("ready", async () => {
+      const modelNames = Object.keys(options.models || {});
+      if (modelNames.length === 0) return;
+      const { loadLocalModel } = await import('../dist/runtime/shared/local-model.js');
+      const results = await Promise.allSettled(modelNames.map((name) => loadLocalModel(name, options)));
+      results.forEach((result, index) => {
+        if (result.status === "rejected") {
+          const name = modelNames[index];
+          const reason = result.reason instanceof Error ? result.reason.message : String(result.reason);
+          console.warn(`[nuxt-local-model] failed to warm model "${name}" during startup: ${reason}`);
+        }
+      });
+    });
+  }
+});
+
+export { module$1 as default };

package/dist/runtime/composables/useLocalModel.d.ts

@@ -0,0 +1,2 @@
+import type { LocalModelRunner, LocalModelPipelineOptions } from "../types.js";
+export declare function useLocalModel(name: string, callOptions?: LocalModelPipelineOptions): Promise<import("@huggingface/transformers").Pipeline | LocalModelRunner>;

package/dist/runtime/composables/useLocalModel.js

@@ -0,0 +1,105 @@
+import { useRuntimeConfig } from "nuxt/app";
+import { getLocalModel, loadLocalModel } from "../shared/local-model.js";
+import { resolveModelDefinition, resolveRuntimeConfig } from "../utils.js";
+const workerCache = /* @__PURE__ */ new Map();
+export async function useLocalModel(name, callOptions = {}) {
+  const runtimeConfig = resolveRuntimeConfig(useRuntimeConfig().public.localModel);
+  const cacheDir = runtimeConfig.cacheDir;
+  const useBrowserWorker = runtimeConfig.browserWorker ?? false;
+  const definition = resolveModelDefinition(name, runtimeConfig);
+  const key = [name, definition.task, definition.model, cacheDir].join("::");
+  const pipelineOptions = definition.options || {};
+  if (process.client && useBrowserWorker) {
+    if (!workerCache.has(key)) {
+      workerCache.set(
+        key,
+        createBrowserWorkerRunner(
+          key,
+          definition.task,
+          definition.model,
+          pipelineOptions,
+          cacheDir,
+          callOptions,
+          runtimeConfig
+        ).catch((error) => {
+          workerCache.delete(key);
+          throw error;
+        })
+      );
+    }
+    return workerCache.get(key);
+  }
+  if (process.server) {
+    return getLocalModel(name, callOptions);
+  }
+  return loadLocalModel(name, runtimeConfig, callOptions);
+}
+function createBrowserWorkerRunner(id, task, model, options, cacheDir, callOptions, runtimeConfig) {
+  return new Promise((resolve, reject) => {
+    const worker = new Worker(new URL("../worker/model.worker.ts", import.meta.url), { type: "module" });
+    const pendingRuns = /* @__PURE__ */ new Map();
+    let settled = false;
+    const failPendingRuns = (reason) => {
+      for (const [, pending] of pendingRuns) {
+        pending.reject(new Error(reason));
+      }
+      pendingRuns.clear();
+    };
+    const runner = Object.assign(
+      async (...args) => new Promise((runResolve, runReject) => {
+        const requestId = `${id}:${Date.now()}:${Math.random().toString(16).slice(2)}`;
+        pendingRuns.set(requestId, { resolve: runResolve, reject: runReject });
+        worker.postMessage({ type: "run", id, requestId, args: [args[0], callOptions] });
+      }),
+      {
+        dispose: () => {
+          failPendingRuns("Browser worker disposed");
+          workerCache.delete(id);
+          worker.removeEventListener("message", handleRun);
+          worker.postMessage({ type: "dispose", id });
+          worker.terminate();
+        }
+      }
+    );
+    const handleInit = (event) => {
+      if (event.data.id !== id || event.data.type !== "init") return;
+      worker.removeEventListener("message", handleInit);
+      if (!event.data.ok) {
+        settled = true;
+        workerCache.delete(id);
+        reject(new Error(event.data.error || "Worker model initialization failed"));
+        return;
+      }
+      settled = true;
+      resolve(runner);
+    };
+    worker.addEventListener("message", handleInit);
+    worker.onerror = (error) => {
+      failPendingRuns("Browser worker crashed");
+      workerCache.delete(id);
+      if (!settled) {
+        settled = true;
+        reject(error);
+      }
+    };
+    const handleRun = (event) => {
+      if (event.data.id !== id || event.data.type !== "run" || !event.data.requestId) return;
+      const pending = pendingRuns.get(event.data.requestId);
+      if (!pending) return;
+      pendingRuns.delete(event.data.requestId);
+      if (event.data.ok) pending.resolve(event.data.result);
+      else pending.reject(new Error(event.data.error || "Worker model execution failed"));
+    };
+    worker.addEventListener("message", handleRun);
+    worker.postMessage({
+      type: "init",
+      id,
+      task,
+      model,
+      options,
+      cacheDir,
+      allowRemoteModels: runtimeConfig.allowRemoteModels,
+      allowLocalModels: runtimeConfig.allowLocalModels
+    });
+  });
+}

package/dist/runtime/config.d.ts

@@ -0,0 +1,3 @@
+import type { LocalModelConfig } from "./types.js";
+export declare function defineLocalModelConfig<const T extends LocalModelConfig>(config: T): T;
+export type { LocalModelAliases } from "./types.js";

package/dist/runtime/config.js

@@ -0,0 +1,3 @@
+export function defineLocalModelConfig(config) {
+  return config;
+}

package/dist/runtime/nuxt.d.ts

@@ -0,0 +1,19 @@
+import type { LocalModelRuntimeConfig } from "./types"
+
+declare module "@nuxt/schema" {
+  interface NuxtConfig {
+    localModel?: LocalModelRuntimeConfig
+  }
+
+  interface NuxtOptions {
+    localModel?: LocalModelRuntimeConfig
+  }
+
+  interface RuntimeConfig {
+    public: {
+      localModel?: LocalModelRuntimeConfig
+    }
+  }
+}
+
+export {}

package/dist/runtime/plugins/hf-transformers.client.d.ts

@@ -0,0 +1,2 @@
+declare const _default: import("nuxt/app").Plugin<Record<string, unknown>> & import("nuxt/app").ObjectPlugin<Record<string, unknown>>;
+export default _default;

package/dist/runtime/plugins/hf-transformers.client.js

@@ -0,0 +1,11 @@
+import { defineNuxtPlugin, useRuntimeConfig } from "nuxt/app";
+import { applyLocalModelEnvironment, resolveRuntimeConfig } from "../utils.js";
+export default defineNuxtPlugin(() => {
+  const runtimeConfig = useRuntimeConfig();
+  const localModel = resolveRuntimeConfig(runtimeConfig.public.localModel);
+  applyLocalModelEnvironment({
+    cacheDir: localModel.cacheDir,
+    allowRemoteModels: localModel.allowRemoteModels,
+    allowLocalModels: false
+  });
+});

package/dist/runtime/plugins/hf-transformers.server.d.ts

@@ -0,0 +1,2 @@
+declare const _default: import("nuxt/app").Plugin<Record<string, unknown>> & import("nuxt/app").ObjectPlugin<Record<string, unknown>>;
+export default _default;

package/dist/runtime/plugins/hf-transformers.server.js

@@ -0,0 +1,27 @@
+import { defineNuxtPlugin, useRuntimeConfig } from "nuxt/app";
+import { readdir, stat } from "node:fs/promises";
+import { setLocalModelRuntimeConfig } from "../shared/local-model.js";
+import { applyLocalModelEnvironment, resolveRuntimeConfig } from "../utils.js";
+async function countCachedEntries(cacheDir) {
+  try {
+    const entries = await readdir(cacheDir, { withFileTypes: true });
+    return entries.filter((entry) => entry.isDirectory() || entry.isFile()).length;
+  } catch {
+    return 0;
+  }
+}
+export default defineNuxtPlugin(async () => {
+  const runtimeConfig = useRuntimeConfig();
+  const localModel = resolveRuntimeConfig(runtimeConfig.public.localModel);
+  applyLocalModelEnvironment(localModel);
+  setLocalModelRuntimeConfig(localModel);
+  const modelNames = Object.keys(localModel.models || {});
+  const cacheStatBefore = await stat(localModel.cacheDir).catch(() => null);
+  const cachedEntriesBefore = cacheStatBefore?.isDirectory() ? await countCachedEntries(localModel.cacheDir) : 0;
+  const cacheStatus = cachedEntriesBefore > 0 ? `\u2705 found ${cachedEntriesBefore} cached model entr${cachedEntriesBefore === 1 ? "y" : "ies"} at ${localModel.cacheDir}` : `\u2B07\uFE0F no cached models found yet at ${localModel.cacheDir}; missing models may download now`;
+  const cacheStat = await stat(localModel.cacheDir).catch(() => null);
+  const cachedEntries = cacheStat?.isDirectory() ? await countCachedEntries(localModel.cacheDir) : 0;
+  console.info(
+    `\u{1F916} [nuxt-local-model] ${cacheStatus} \u2022 \u{1F680} ${modelNames.length} configured model(s) will load when Nuxt finishes starting \u2022 \u{1F4E6} cache now has ${cachedEntries} entr${cachedEntries === 1 ? "y" : "ies"}`
+  );
+});

package/dist/runtime/server/index.d.ts

@@ -0,0 +1 @@
+export { getLocalModel } from "../shared/local-model.js";

package/dist/runtime/server/index.js

@@ -0,0 +1 @@
+export { getLocalModel } from "../shared/local-model.js";

package/dist/runtime/server/worker.d.ts

@@ -0,0 +1 @@
+export {};

package/dist/runtime/server/worker.js

@@ -0,0 +1,34 @@
+import { parentPort, workerData } from "node:worker_threads";
+import { isMainThread, threadId } from "node:worker_threads";
+import { pipeline } from "@huggingface/transformers";
+import { serializeWorkerResult } from "../shared/serialize.js";
+import { applyLocalModelEnvironment } from "../utils.js";
+const { id, task, model, options, cacheDir, allowRemoteModels, allowLocalModels } = workerData;
+applyLocalModelEnvironment({
+  cacheDir: cacheDir || "./.ai-models",
+  allowRemoteModels: allowRemoteModels ?? true,
+  allowLocalModels: allowLocalModels ?? true
+});
+console.info(
+  `\u{1F9F5} [nuxt-local-model] server worker ready threadId=${threadId} mainThread=${isMainThread} cacheDir=${cacheDir || "./.ai-models"}`
+);
+const modelPromise = pipeline(task, model, options || {});
+parentPort?.on("message", async (message) => {
+  try {
+    if (message.type === "dispose") {
+      parentPort?.postMessage({ id, ok: true, type: "dispose" });
+      return;
+    }
+    const runner = await modelPromise;
+    const result = await runner(...message.args);
+    parentPort?.postMessage({ id, requestId: message.requestId, ok: true, type: "run", result: serializeWorkerResult(result) });
+  } catch (error) {
+    parentPort?.postMessage({
+      id,
+      requestId: message.type === "run" ? message.requestId : void 0,
+      ok: false,
+      type: message.type,
+      error: error instanceof Error ? error.message : String(error)
+    });
+  }
+});

package/dist/runtime/shared/local-model.d.ts

@@ -0,0 +1,6 @@
+import type { LocalModelPipelineOptions, LocalModelRuntimeConfig } from "../types.js";
+import { type InternalLocalModelRuntimeConfig } from "../utils.js";
+export declare function isLocalModelRuntimeConfig(value: unknown): value is LocalModelRuntimeConfig;
+export declare function setLocalModelRuntimeConfig(config: InternalLocalModelRuntimeConfig | null | undefined): void;
+export declare function loadLocalModel(name: string, runtimeConfig: LocalModelRuntimeConfig, callOptions?: LocalModelPipelineOptions): Promise<import("@huggingface/transformers").Pipeline>;
+export declare function getLocalModel(name: string, callOptions?: LocalModelPipelineOptions): Promise<import("@huggingface/transformers").Pipeline>;

package/dist/runtime/shared/local-model.js

@@ -0,0 +1,212 @@
+import { pipeline } from "@huggingface/transformers";
+import {
+  applyLocalModelEnvironment,
+  canUseServerWorkerForRuntime,
+  resolveModelDefinition,
+  resolveRuntimeConfig
+} from "../utils.js";
+const modelCache = /* @__PURE__ */ new Map();
+const serverWorkerCache = /* @__PURE__ */ new Map();
+const warnedMessages = /* @__PURE__ */ new Set();
+let onnxBackendPromise = null;
+const runtimeConfigSymbol = Symbol.for("nuxt-local-model:runtime-config");
+const pipelineFactory = pipeline;
+function getRuntimeConfigStore() {
+  return globalThis[runtimeConfigSymbol] || null;
+}
+function setRuntimeConfigStore(config) {
+  ;
+  globalThis[runtimeConfigSymbol] = config;
+}
+function warnOnce(message) {
+  if (warnedMessages.has(message)) return;
+  warnedMessages.add(message);
+  console.warn(message);
+}
+function cacheKey(name, task, model, cacheDir) {
+  return [name, task, model, cacheDir].join("::");
+}
+function createPipelineRunner(task, model, options) {
+  return pipelineFactory(task, model, options);
+}
+async function resolveServerWorkerEntry() {
+  const [{ existsSync }, pathMod, urlMod] = await Promise.all([
+    import("node:fs"),
+    import("node:path"),
+    import("node:url")
+  ]);
+  const baseDir = pathMod.dirname(urlMod.fileURLToPath(import.meta.url));
+  const jsEntry = pathMod.resolve(baseDir, "../server/worker.js");
+  if (existsSync(jsEntry)) return jsEntry;
+  return pathMod.resolve(baseDir, "../server/worker.ts");
+}
+async function canUseServerWorkerRuntime(config) {
+  if (typeof window !== "undefined") return false;
+  if (!canUseServerWorkerForRuntime(config.runtime)) return false;
+  try {
+    const mod = await import("node:worker_threads");
+    return typeof mod.Worker === "function";
+  } catch {
+    return false;
+  }
+}
+async function ensurePreferredOnnxBackend(config) {
+  if (typeof window !== "undefined") return;
+  if (onnxBackendPromise) return onnxBackendPromise;
+  onnxBackendPromise = (async () => {
+    try {
+      const ort = await import("onnxruntime-node");
+      const backend = ort.default ?? ort;
+      const symbol = Symbol.for("onnxruntime");
+      if (!(symbol in globalThis)) {
+        ;
+        globalThis[symbol] = backend;
+      }
+    } catch {
+      warnOnce(
+        `[nuxt-local-model] onnxruntime-node is not available in ${config.runtime}; falling back to the default Transformers.js backend.`
+      );
+    }
+  })();
+  return onnxBackendPromise;
+}
+export function isLocalModelRuntimeConfig(value) {
+  return !!value && typeof value === "object" && ("models" in value || "cacheDir" in value || "runtime" in value);
+}
+export function setLocalModelRuntimeConfig(config) {
+  if (config) {
+    setRuntimeConfigStore(resolveRuntimeConfig(config));
+  }
+}
+function createServerWorkerRunner(name, task, model, options, cacheDir, runtimeConfig) {
+  const key = cacheKey(name, task, model, cacheDir);
+  if (!serverWorkerCache.has(key)) {
+    const workerPromise = (async () => {
+      const workerEntry = runtimeConfig.serverWorkerEntry || await resolveServerWorkerEntry();
+      console.info(`\u{1F9F5} [nuxt-local-model] using server worker for "${name}" (${task} -> ${model}) at ${workerEntry}`);
+      const { Worker } = await import("node:worker_threads");
+      const worker = new Worker(workerEntry, {
+        workerData: {
+          id: key,
+          task,
+          model,
+          options,
+          cacheDir,
+          allowRemoteModels: runtimeConfig.allowRemoteModels,
+          allowLocalModels: runtimeConfig.allowLocalModels
+        }
+      });
+      const pendingRuns = /* @__PURE__ */ new Map();
+      const failPendingRuns = (reason) => {
+        for (const [, pending] of pendingRuns) {
+          clearTimeout(pending.timeout);
+          pending.reject(new Error(reason));
+        }
+        pendingRuns.clear();
+      };
+      const handleMessage = (message) => {
+        if (message.id !== key || message.type !== "run") return;
+        const pending = pendingRuns.get(message.requestId);
+        if (!pending) return;
+        clearTimeout(pending.timeout);
+        pendingRuns.delete(message.requestId);
+        if (message.ok) {
+          pending.resolve(message.result);
+          return;
+        }
+        pending.reject(new Error(message.error || "Server worker model execution failed"));
+      };
+      worker.on("message", handleMessage);
+      worker.once("error", (error) => {
+        worker.off("message", handleMessage);
+        serverWorkerCache.delete(key);
+        failPendingRuns(error instanceof Error ? error.message : "Server worker crashed");
+      });
+      worker.once("exit", (code) => {
+        worker.off("message", handleMessage);
+        serverWorkerCache.delete(key);
+        if (code !== 0) {
+          failPendingRuns(`Server worker exited with code ${code}`);
+        }
+      });
+      return Object.assign(
+        async (...args) => new Promise((runResolve, runReject) => {
+          const requestId = `${key}:${Date.now()}:${Math.random().toString(16).slice(2)}`;
+          const timeout = setTimeout(() => {
+            pendingRuns.delete(requestId);
+            runReject(new Error(`Server worker timed out for "${name}"`));
+          }, 12e4);
+          pendingRuns.set(requestId, { resolve: runResolve, reject: runReject, timeout });
+          worker.postMessage({ type: "run", requestId, args });
+        }),
+        {
+          dispose: async () => {
+            failPendingRuns("Server worker disposed");
+            worker.off("message", handleMessage);
+            serverWorkerCache.delete(key);
+            await worker.terminate();
+          }
+        }
+      );
+    })().catch((error) => {
+      serverWorkerCache.delete(key);
+      throw error;
+    });
+    serverWorkerCache.set(key, workerPromise);
+  }
+  return serverWorkerCache.get(key);
+}
+export async function loadLocalModel(name, runtimeConfig, callOptions = {}) {
+  const resolvedConfig = resolveRuntimeConfig(runtimeConfig);
+  await ensurePreferredOnnxBackend(resolvedConfig);
+  const definition = resolveModelDefinition(name, resolvedConfig);
+  const cacheDir = resolvedConfig.cacheDir;
+  if (typeof window === "undefined" && resolvedConfig.serverWorker) {
+    if (!await canUseServerWorkerRuntime(resolvedConfig)) {
+      warnOnce(
+        `[nuxt-local-model] serverWorker is enabled for ${resolvedConfig.runtime}, but worker threads are not available. Falling back to the main server thread.`
+      );
+    } else {
+      return createServerWorkerRunner(
+        name,
+        definition.task,
+        definition.model,
+        definition.options || {},
+        cacheDir,
+        resolvedConfig
+      );
+    }
+  }
+  applyLocalModelEnvironment({
+    cacheDir,
+    allowRemoteModels: resolvedConfig.allowRemoteModels,
+    allowLocalModels: resolvedConfig.allowLocalModels
+  });
+  const key = cacheKey(name, definition.task, definition.model, cacheDir);
+  if (!modelCache.has(key)) {
+    const modelPromise = (async () => {
+      const loaded = await createPipelineRunner(definition.task, definition.model, definition.options || {});
+      return Object.assign(
+        async (...args) => {
+          const input = args[0];
+          return loaded(input, callOptions);
+        },
+        {
+          dispose: async () => {
+            await loaded.dispose?.();
+            modelCache.delete(key);
+          }
+        }
+      );
+    })().catch((error) => {
+      modelCache.delete(key);
+      throw error;
+    });
+    modelCache.set(key, modelPromise);
+  }
+  return modelCache.get(key);
+}
+export async function getLocalModel(name, callOptions = {}) {
+  const runtimeConfig = getRuntimeConfigStore() || resolveRuntimeConfig(void 0);
+  return loadLocalModel(name, runtimeConfig, callOptions);
+}

package/dist/runtime/shared/serialize.d.ts

@@ -0,0 +1 @@
+export declare function serializeWorkerResult(value: unknown): unknown;

package/dist/runtime/shared/serialize.js

@@ -0,0 +1,42 @@
+export function serializeWorkerResult(value) {
+  if (value === null || value === void 0) return value;
+  if (Array.isArray(value)) {
+    return value.map(serializeWorkerResult);
+  }
+  if (ArrayBuffer.isView(value)) {
+    return Array.from(value).map(Number);
+  }
+  if (value instanceof Date) {
+    return value.toISOString();
+  }
+  if (value instanceof Map) {
+    return Object.fromEntries(Array.from(value.entries()).map(([key, entry]) => [String(key), serializeWorkerResult(entry)]));
+  }
+  if (value instanceof Set) {
+    return Array.from(value.values()).map(serializeWorkerResult);
+  }
+  if (typeof value === "object") {
+    const candidate = value;
+    if (typeof candidate.tolist === "function") {
+      return serializeWorkerResult(candidate.tolist());
+    }
+    if (typeof candidate.toJSON === "function") {
+      const json = candidate.toJSON();
+      if (json !== value) return serializeWorkerResult(json);
+    }
+    if (candidate.data !== void 0) {
+      return {
+        data: serializeWorkerResult(candidate.data),
+        shape: serializeWorkerResult(candidate.shape),
+        dims: serializeWorkerResult(candidate.dims),
+        type: candidate.type
+      };
+    }
+    const plain = {};
+    for (const [key, entry] of Object.entries(candidate)) {
+      plain[key] = serializeWorkerResult(entry);
+    }
+    return plain;
+  }
+  return value;
+}

package/dist/runtime/types.d.ts

@@ -0,0 +1,29 @@
+import { pipeline } from "@huggingface/transformers";
+import type { Pipeline } from "@huggingface/transformers";
+export type LocalModelSupportedRuntime = "node" | "bun" | "deno";
+export type LocalModelRuntime = "auto" | LocalModelSupportedRuntime;
+export type LocalModelTask = "feature-extraction" | "text-classification" | "text-generation" | "fill-mask" | "automatic-speech-recognition" | (string & {});
+export type LocalModelPipeline = Pipeline;
+export type LocalModelPipelineOptions = NonNullable<Parameters<Pipeline>[1]>;
+export type LocalModelPipelineLoadOptions = NonNullable<Parameters<typeof pipeline>[2]>;
+export interface LocalModelDefinition {
+    task: LocalModelTask;
+    model: string;
+    options?: LocalModelPipelineLoadOptions;
+}
+export type LocalModelModelRegistry = Record<string, LocalModelDefinition>;
+export type LocalModelAliases<T extends Pick<LocalModelRuntimeConfig, "models">> = keyof NonNullable<T["models"]> & string;
+export interface LocalModelRuntimeConfig<TModels extends LocalModelModelRegistry = LocalModelModelRegistry> {
+    runtime?: LocalModelRuntime;
+    cacheDir?: string;
+    allowRemoteModels?: boolean;
+    allowLocalModels?: boolean;
+    defaultTask?: LocalModelTask;
+    serverWorker?: boolean;
+    browserWorker?: boolean;
+    models?: TModels;
+}
+export type LocalModelConfig<TModels extends LocalModelModelRegistry = LocalModelModelRegistry> = LocalModelRuntimeConfig<TModels>;
+export type LocalModelRunner = ((...args: any[]) => Promise<unknown>) & {
+    dispose?: () => Promise<void> | void;
+};

package/dist/runtime/utils.d.ts

@@ -0,0 +1,17 @@
+import type { LocalModelDefinition, LocalModelModelRegistry, LocalModelPipelineLoadOptions, LocalModelRuntime, LocalModelRuntimeConfig, LocalModelSupportedRuntime } from "./types.js";
+export interface InternalLocalModelRuntimeConfig extends LocalModelRuntimeConfig {
+    serverWorkerEntry?: string;
+}
+export interface ResolvedLocalModelRuntimeConfig extends Omit<InternalLocalModelRuntimeConfig, "runtime" | "models"> {
+    runtime: LocalModelSupportedRuntime;
+    models: LocalModelModelRegistry;
+    cacheDir: string;
+    allowRemoteModels: boolean;
+    allowLocalModels: boolean;
+}
+export declare function detectLocalModelRuntime(runtime?: LocalModelRuntime): LocalModelSupportedRuntime;
+export declare function resolveCacheDir(cacheDir?: string): string;
+export declare function resolveRuntimeConfig(config?: InternalLocalModelRuntimeConfig): ResolvedLocalModelRuntimeConfig;
+export declare function applyLocalModelEnvironment(config: Pick<ResolvedLocalModelRuntimeConfig, "cacheDir" | "allowRemoteModels" | "allowLocalModels">): void;
+export declare function canUseServerWorkerForRuntime(runtime: LocalModelSupportedRuntime): runtime is "node" | "bun" | "deno";
+export declare function resolveModelDefinition(name: string, runtimeConfig: InternalLocalModelRuntimeConfig | undefined, loadOptions?: LocalModelPipelineLoadOptions): LocalModelDefinition;

package/dist/runtime/utils.js

@@ -0,0 +1,71 @@
+import { env } from "@huggingface/transformers";
+function readEnvValue(name) {
+  const processValue = globalThis.process?.env?.[name];
+  if (typeof processValue === "string" && processValue.trim()) {
+    return processValue.trim();
+  }
+  const deno = globalThis;
+  try {
+    const denoValue = deno.Deno?.env?.get?.(name);
+    if (typeof denoValue === "string" && denoValue.trim()) {
+      return denoValue.trim();
+    }
+  } catch {
+  }
+  return void 0;
+}
+export function detectLocalModelRuntime(runtime = "auto") {
+  if (runtime !== "auto") {
+    return runtime;
+  }
+  const bun = globalThis;
+  if (bun.Bun || bun.process?.versions?.bun) {
+    return "bun";
+  }
+  const deno = globalThis;
+  if (deno.Deno?.version?.deno) {
+    return "deno";
+  }
+  return "node";
+}
+export function resolveCacheDir(cacheDir) {
+  if (cacheDir?.trim()) return cacheDir.trim();
+  return readEnvValue("NUXT_LOCAL_MODEL_CACHE_DIR") || "./.ai-models";
+}
+export function resolveRuntimeConfig(config) {
+  return {
+    runtime: detectLocalModelRuntime(config?.runtime ?? "auto"),
+    allowRemoteModels: config?.allowRemoteModels ?? true,
+    allowLocalModels: config?.allowLocalModels ?? true,
+    cacheDir: resolveCacheDir(config?.cacheDir),
+    defaultTask: config?.defaultTask ?? "feature-extraction",
+    serverWorker: config?.serverWorker ?? false,
+    serverWorkerEntry: config?.serverWorkerEntry,
+    browserWorker: config?.browserWorker ?? false,
+    models: config?.models ?? {}
+  };
+}
+export function applyLocalModelEnvironment(config) {
+  env.cacheDir = resolveCacheDir(config.cacheDir);
+  env.localModelPath = env.cacheDir;
+  env.allowRemoteModels = config.allowRemoteModels;
+  env.allowLocalModels = config.allowLocalModels;
+}
+export function canUseServerWorkerForRuntime(runtime) {
+  return runtime === "node" || runtime === "bun" || runtime === "deno";
+}
+export function resolveModelDefinition(name, runtimeConfig, loadOptions) {
+  const resolved = resolveRuntimeConfig(runtimeConfig);
+  const registryEntry = resolved.models?.[name];
+  if (!registryEntry) {
+    throw new Error(`Local model "${name}" is not defined in nuxt.config.`);
+  }
+  return {
+    task: registryEntry?.task || resolved.defaultTask || "feature-extraction",
+    model: registryEntry?.model || name,
+    options: {
+      ...registryEntry?.options || {},
+      ...loadOptions
+    }
+  };
+}

package/dist/runtime/worker/model.worker.d.ts

@@ -0,0 +1 @@
+export {};

package/dist/runtime/worker/model.worker.js

@@ -0,0 +1,39 @@
+import { pipeline } from "@huggingface/transformers";
+import { serializeWorkerResult } from "../shared/serialize.js";
+import { applyLocalModelEnvironment } from "../utils.js";
+const pipelines = /* @__PURE__ */ new Map();
+self.onmessage = async (event) => {
+  const message = event.data;
+  try {
+    if (message.type === "init") {
+      applyLocalModelEnvironment({
+        cacheDir: message.cacheDir || "./.ai-models",
+        allowRemoteModels: message.allowRemoteModels ?? true,
+        allowLocalModels: message.allowLocalModels ?? false
+      });
+      if (!pipelines.has(message.id)) {
+        pipelines.set(message.id, pipeline(message.task, message.model, message.options || {}));
+      }
+      self.postMessage({ id: message.id, ok: true, type: "init" });
+      return;
+    }
+    if (message.type === "dispose") {
+      pipelines.delete(message.id);
+      self.postMessage({ id: message.id, ok: true, type: "dispose" });
+      return;
+    }
+    const model = pipelines.get(message.id);
+    if (!model) throw new Error("Model pipeline is not initialized.");
+    const resolved = await model;
+    const result = await resolved(...message.args);
+    self.postMessage({ id: message.id, requestId: message.requestId, ok: true, type: "run", result: serializeWorkerResult(result) });
+  } catch (error) {
+    self.postMessage({
+      id: message.id,
+      requestId: message.type === "run" ? message.requestId : void 0,
+      ok: false,
+      type: message.type,
+      error: error instanceof Error ? error.message : String(error)
+    });
+  }
+};

package/dist/types.d.mts

@@ -0,0 +1,9 @@
+import type { NuxtModule } from '@nuxt/schema'
+
+import type { default as Module } from './module.mjs'
+
+export type ModuleOptions = typeof Module extends NuxtModule<infer O> ? Partial<O> : Record<string, any>
+
+export { default } from './module.mjs'
+
+export { type NuxtLlmModuleOptions } from './module.mjs'

package/package.json

@@ -0,0 +1,60 @@
+{
+  "name": "nuxt-local-model",
+  "version": "0.1.1",
+  "description": "A Nuxt module for local in-app Hugging Face transformer models.",
+  "license": "MIT",
+  "repository": "https://github.com/Aft1n/nuxt-local-model",
+  "files": [
+    "dist",
+    "README.md"
+  ],
+  "type": "module",
+  "main": "./dist/module.mjs",
+  "module": "./dist/module.mjs",
+  "types": "./dist/module.d.mts",
+  "exports": {
+    ".": {
+      "types": "./dist/module.d.mts",
+      "import": "./dist/module.mjs"
+    },
+    "./client": {
+      "types": "./dist/runtime/composables/useLocalModel.d.ts",
+      "import": "./dist/runtime/composables/useLocalModel.js"
+    },
+    "./server": {
+      "types": "./dist/runtime/server/index.d.ts",
+      "import": "./dist/runtime/server/index.js"
+    }
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "scripts": {
+    "dev": "nuxi dev playground",
+    "build": "nuxt build-module",
+    "prepare": "nuxt build-module --prepare",
+    "test": "vitest run",
+    "test:watch": "vitest"
+  },
+  "dependencies": {
+    "@huggingface/transformers": "^3.8.1",
+    "onnxruntime-node": "^1.21.0"
+  },
+  "devDependencies": {
+    "@nuxt/kit": "^4.3.0",
+    "@nuxt/module-builder": "^1.0.2",
+    "@nuxt/test-utils": "^4.0.0",
+    "@types/node": "^24.0.0",
+    "nuxt": "^4.3.0",
+    "typescript": "^5.9.3",
+    "vitest": "^4.0.18"
+  },
+  "peerDependencies": {
+    "nuxt": "^4.3.0"
+  },
+  "trustedDependencies": [
+    "@parcel/watcher",
+    "onnxruntime-node",
+    "protobufjs"
+  ]
+}