inferis-ml 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 inferis contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,664 @@
+# inferis-ml
+
+Worker pool for running AI models in the browser — WebGPU/WASM auto-detection, model lifecycle management, token streaming, and cross-tab deduplication.
+
+[![npm version](https://img.shields.io/npm/v/inferis-ml)](https://npmjs.com/package/inferis-ml)
+[![bundle size](https://img.shields.io/bundlephobia/minzip/inferis-ml)](https://bundlephobia.com/package/inferis-ml)
+[![coverage](https://img.shields.io/badge/coverage-93%25-green)](https://github.com/pashunechka/inferis-ml)
+
+> **[Live Examples](https://pashunechka.github.io/inferis-ml/)** — run AI models directly in your browser, no server needed.
+
+## What is this
+
+You want to add smart search, speech recognition, or a chatbot to your website. Normally this requires a server — you send a request to the cloud, wait for a response, pay per call.
+
+**inferis-ml** lets you run AI models directly in the user's browser. The model downloads once, then runs on the user's GPU/CPU. No server, no per-request cost, no data leaving the device.
+
+The catch: running a neural network in the browser is technically painful. Run it on the main thread and the page freezes. Move it to a Web Worker and you're writing `postMessage` boilerplate. inferis-ml takes that pain away.
+
+### Three problems it solves
+
+**1. Page freezes during inference**
+
+Without inferis-ml, running a model on the main thread locks the UI. With inferis-ml, work runs in a background worker — the page stays responsive.
+
+**2. 5 open tabs = 5 model copies in RAM**
+
+Without inferis-ml: 5 tabs × 2 GB LLM = 10 GB RAM. Browser crashes.
+With `crossTab: true`: all tabs share one worker, one model copy in memory.
+
+**3. WebGPU not available everywhere**
+
+Without inferis-ml: you manually detect WebGPU and swap backends.
+With `defaultDevice: 'auto'`: inferis-ml tries WebGPU, silently falls back to WASM if unavailable.
+
+---
+
+## Why
+
+Existing browser AI runtimes (transformers.js, web-llm, onnxruntime-web) give you inference but leave worker management entirely to you:
+
+- Create the Web Worker manually
+- Wire up `postMessage` and response correlation
+- Implement model lifecycle (load → infer → dispose)
+- Avoid loading the same model twice across browser tabs
+- Handle WebGPU → WASM fallback
+- Evict models when memory budget is exceeded
+- Forward streaming tokens to the UI
+
+**inferis-ml** handles all of this. You get a clean async API and focus on building the product.
+
+## Features
+
+- **Runtime-agnostic** — adapters for `@huggingface/transformers`, `@mlc-ai/web-llm`, `onnxruntime-web`, or your own
+- **Zero framework dependencies** — works with React, Vue, Svelte, or vanilla JS
+- **WebGPU → WASM fallback** — auto-detected, or configured explicitly
+- **Streaming** — `ReadableStream` + `for await` for token-by-token LLM output
+- **Memory budget** — LRU eviction when models exceed the configured cap
+- **Cross-tab dedup** — SharedWorker (tier 1), leader election (tier 2), or per-tab fallback (tier 3)
+- **AbortController** — cancel any in-flight inference
+- **TypeScript** — full type safety, generic output types
+
+## Install
+
+```bash
+npm install inferis-ml
+
+# Install the adapter you need (optional peer deps):
+npm install @huggingface/transformers   # for transformersAdapter
+npm install @mlc-ai/web-llm             # for webLlmAdapter
+npm install onnxruntime-web             # for onnxAdapter
+```
+
+## Use Cases
+
+### Semantic search over articles
+
+User types a query — find articles by meaning, not just keywords.
+
+```typescript
+import { createPool } from 'inferis-ml';
+import { transformersAdapter } from 'inferis-ml/adapters/transformers';
+
+const pool = await createPool({ adapter: transformersAdapter() });
+
+const embedder = await pool.load<number[][]>('feature-extraction', {
+  model: 'mixedbread-ai/mxbai-embed-xsmall-v1',
+  onProgress: ({ phase, loaded, total }) => {
+    const pct = total > 0 ? Math.round(loaded / total * 100) : 0;
+    updateProgressBar(pct, phase);
+  },
+});
+
+const articles = ['How to choose a laptop', 'Borscht recipe', 'History of Rome'];
+const embeddings = await embedder.run(articles);
+// embeddings: number[][] — one vector per article
+
+const query = await embedder.run(['buy a computer']);
+// compare query[0] against embeddings with cosine similarity
+```
+
+### Chatbot with streaming output
+
+Answer appears word by word, like ChatGPT.
+
+```typescript
+import { createPool } from 'inferis-ml';
+import { webLlmAdapter } from 'inferis-ml/adapters/web-llm';
+
+const pool = await createPool({
+  adapter: webLlmAdapter(),
+  maxWorkers: 1,           // LLMs use one GPU context
+  defaultDevice: 'webgpu',
+});
+
+const llm = await pool.load<string>('text-generation', {
+  model: 'Llama-3.2-3B-Instruct-q4f32_1-MLC',
+  onProgress: ({ phase }) => setStatus(phase),
+});
+
+const outputDiv = document.getElementById('answer');
+const stream = llm.stream({
+  messages: [
+    { role: 'system', content: 'You are a helpful assistant.' },
+    { role: 'user', content: userQuestion },
+  ],
+});
+
+for await (const token of stream) {
+  outputDiv.textContent += token;
+}
+```
+
+### Speech transcription
+
+```typescript
+const transcriber = await pool.load<{ text: string }>('automatic-speech-recognition', {
+  model: 'openai/whisper-base',
+  estimatedMemoryMB: 80,
+});
+
+const audioData = await getMicrophoneAudio(); // Float32Array
+const result = await transcriber.run(audioData);
+console.log(result.text); // "Hello, how are you?"
+```
+
+### Cancel a running request
+
+```typescript
+const controller = new AbortController();
+stopButton.onclick = () => controller.abort();
+
+try {
+  const stream = llm.stream(input, { signal: controller.signal });
+  for await (const token of stream) {
+    outputDiv.textContent += token;
+  }
+} catch (e) {
+  if (e.name === 'AbortError') outputDiv.textContent += ' [stopped]';
+}
+```
+
+### Model state changes
+
+```typescript
+model.onStateChange((state) => {
+  if (state === 'loading')  showSpinner();
+  if (state === 'ready')    hideSpinner();
+  if (state === 'error')    showError('Failed to load model');
+  if (state === 'disposed') disableUI();
+});
+```
+
+---
+
+## Quick Start
+
+### Embeddings
+
+```typescript
+import { createPool } from 'inferis-ml';
+import { transformersAdapter } from 'inferis-ml/adapters/transformers';
+
+const pool = await createPool({
+  adapter: transformersAdapter(),
+});
+
+const model = await pool.load<number[][]>('feature-extraction', {
+  model: 'mixedbread-ai/mxbai-embed-xsmall-v1',
+  onProgress: (p) => console.log(`${p.phase}: ${(p.loaded / p.total * 100) | 0}%`),
+});
+
+const embeddings = await model.run(['Hello world', 'Another sentence']);
+// embeddings: number[][]
+
+await model.dispose();
+await pool.terminate();
+```
+
+### LLM Streaming
+
+```typescript
+import { createPool } from 'inferis-ml';
+import { webLlmAdapter } from 'inferis-ml/adapters/web-llm';
+
+const pool = await createPool({
+  adapter: webLlmAdapter(),
+  defaultDevice: 'webgpu',
+  maxWorkers: 1,
+});
+
+const llm = await pool.load<string>('text-generation', {
+  model: 'Llama-3.1-8B-Instruct-q4f32_1-MLC',
+  onProgress: ({ phase }) => console.log(phase),
+});
+
+const stream = llm.stream({
+  messages: [
+    { role: 'system', content: 'You are a helpful assistant.' },
+    { role: 'user', content: 'Explain WebGPU in 3 sentences.' },
+  ],
+});
+
+const output = document.getElementById('output');
+for await (const token of stream) {
+  output.textContent += token;
+}
+```
+
+### Abort
+
+```typescript
+const ctrl = new AbortController();
+
+const stream = llm.stream(input, { signal: ctrl.signal });
+
+// Cancel after 5 seconds
+setTimeout(() => ctrl.abort(), 5000);
+
+try {
+  for await (const token of stream) {
+    updateUI(token);
+  }
+} catch (e) {
+  if (e.name === 'AbortError') console.log('Cancelled');
+}
+```
+
+### Cross-Tab Deduplication
+
+```typescript
+// Enable cross-tab model sharing.
+// If you open 5 tabs, the model is loaded only once.
+const pool = await createPool({
+  adapter: transformersAdapter(),
+  crossTab: true,   // auto-selects SharedWorker > leader election > per-tab
+});
+```
+
+### Capability Detection
+
+```typescript
+import { detectCapabilities } from 'inferis-ml';
+
+const caps = await detectCapabilities();
+
+if (caps.webgpu.supported) {
+  console.log('GPU vendor:', caps.webgpu.adapter?.vendor);
+  console.log('Max buffer:', caps.webgpu.limits?.maxBufferSize);
+} else {
+  console.log('Falling back to WASM');
+  console.log('SIMD support:', caps.wasm.simd);
+}
+```
+
+## Custom Adapter
+
+```typescript
+import type { ModelAdapter, ModelAdapterFactory } from 'inferis-ml';
+
+export function myCustomAdapter(): ModelAdapterFactory {
+  return {
+    name: 'my-adapter',
+
+    async create(): Promise<ModelAdapter> {
+      // This runs INSIDE the worker — safe to import heavy libs here
+      const { MyRuntime } = await import('my-runtime');
+
+      return {
+        name: 'my-adapter',
+
+        estimateMemoryMB(_task, config) {
+          return (config.estimatedMemoryMB as number) ?? 50;
+        },
+
+        async load(task, config, device, onProgress) {
+          onProgress({ phase: 'loading', loaded: 0, total: 1 });
+          const instance = await MyRuntime.load(config.model as string, { device });
+          onProgress({ phase: 'done', loaded: 1, total: 1 });
+          return { instance, memoryMB: 50 };
+        },
+
+        async run(model, input) {
+          return (model.instance as MyRuntime).infer(input);
+        },
+
+        async stream(model, input, onChunk) {
+          for await (const chunk of (model.instance as MyRuntime).stream(input)) {
+            onChunk(chunk);
+          }
+        },
+
+        async unload(model) {
+          await (model.instance as MyRuntime).dispose();
+        },
+      };
+    },
+  };
+}
+```
+
+## API Reference
+
+### `createPool(config)`
+
+```typescript
+const pool = await createPool({
+  adapter: transformersAdapter(),   // required
+
+  workerUrl: new URL('inferis-ml/worker', import.meta.url),  // worker bundle URL
+  maxWorkers: navigator.hardwareConcurrency - 1,          // default: cores - 1
+  maxMemoryMB: 2048,                                       // default: 2048
+  defaultDevice: 'auto',                                   // 'webgpu' | 'wasm' | 'auto'
+  crossTab: false,                                         // cross-tab dedup
+  taskTimeout: 120_000,                                    // per-task timeout in ms
+});
+```
+
+### `pool.load<TOutput>(task, config)`
+
+Loads a model and returns a `ModelHandle`. If the model is already loaded, returns the existing handle.
+
+```typescript
+const model = await pool.load<number[][]>('feature-extraction', {
+  model: 'mixedbread-ai/mxbai-embed-xsmall-v1',
+  estimatedMemoryMB: 30,          // hint for memory budget (optional)
+  onProgress: (p) => { ... },     // download/load progress
+});
+```
+
+### `ModelHandle<TOutput>`
+
+| Method | Description |
+|--------|-------------|
+| `run(input, options?)` | Non-streaming inference. Returns `Promise<TOutput>`. |
+| `stream(input, options?)` | Streaming inference. Returns `ReadableStream<TOutput>`. |
+| `dispose()` | Unload model and free memory. |
+| `onStateChange(cb)` | Subscribe to state changes. Returns unsubscribe function. |
+| `id` | Unique model ID (`task:model`). |
+| `state` | Current state: `idle \| loading \| ready \| inferring \| unloading \| error \| disposed`. |
+| `memoryMB` | Approximate memory usage. |
+| `device` | Resolved device: `webgpu` or `wasm`. |
+
+### `InferenceOptions`
+
+```typescript
+interface InferenceOptions {
+  signal?: AbortSignal;              // cancel via AbortController
+  priority?: 'high' | 'normal' | 'low';  // scheduling priority
+}
+```
+
+## Bundler Setup
+
+### Vite
+
+```typescript
+// vite.config.ts
+export default {
+  worker: { format: 'es' },
+};
+```
+
+```typescript
+// Usage
+const pool = await createPool({
+  adapter: transformersAdapter(),
+  workerUrl: new URL('inferis-ml/worker', import.meta.url),
+});
+```
+
+### webpack 5
+
+```typescript
+// webpack.config.js
+module.exports = {
+  experiments: { asyncWebAssembly: true },
+};
+```
+
+### Inline Worker (no bundler config needed)
+
+```typescript
+import { createPool } from 'inferis-ml';
+import { inlineWorkerUrl } from 'inferis-ml/worker-inline';
+
+const pool = await createPool({
+  adapter: transformersAdapter(),
+  workerUrl: inlineWorkerUrl(),  // creates a Blob URL
+});
+```
+
+## Browser Support
+
+| Feature | Chrome | Firefox | Safari | Edge | iOS Safari | Android Chrome |
+|---------|--------|---------|--------|------|------------|----------------|
+| Core (Worker + WASM) | 57+ | 52+ | 11+ | 16+ | 11+ | 57+ |
+| WebGPU | 113+ | 141+ | 26+ | 113+ | 26+ | 121+ |
+| WASM SIMD | 91+ | 89+ | 16.4+ | 91+ | 16.4+ | 91+ |
+| SharedWorker (cross-tab tier 1) | 4+ | 29+ | 16+ | 79+ | — | — |
+| Leader Election (cross-tab tier 2) | 69+ | 96+ | 15.4+ | 79+ | 15.4+ | 69+ |
+| AbortController | 66+ | 57+ | 12.1+ | 16+ | 12.2+ | 66+ |
+
+**Minimum requirement:** Web Workers + WebAssembly (97%+ of browsers worldwide).
+All advanced features (WebGPU, SharedWorker, leader election) are progressive enhancements.
+
+## Performance Tips
+
+- **Set `maxWorkers: 1`** for GPU-bound workloads (LLMs) — GPU has one execution context.
+- **Set `defaultDevice: 'webgpu'`** explicitly if you know your users have modern hardware.
+- **Use `estimatedMemoryMB`** to help the memory budget make accurate eviction decisions.
+- **Reuse `ModelHandle`** — loading a model already in state `ready` is a no-op.
+- **Enable `crossTab: true`** for apps users open in multiple tabs (chat, document editors).
+
+## Popular Models
+
+Models are downloaded automatically from [Hugging Face Hub](https://huggingface.co/models) on first use and cached in the browser's Cache API. Subsequent page loads use the cache — no re-download, works offline.
+
+### Embeddings / Semantic Search
+
+| Model ID | Size | Notes |
+|----------|------|-------|
+| `mixedbread-ai/mxbai-embed-xsmall-v1` | 23 MB | Best quality/size ratio for English |
+| `Xenova/all-MiniLM-L6-v2` | 23 MB | Popular multilingual embedding model |
+| `Xenova/all-mpnet-base-v2` | 86 MB | Higher quality, larger |
+| `Xenova/multilingual-e5-small` | 118 MB | 100+ languages |
+
+```typescript
+const model = await pool.load<number[][]>('feature-extraction', {
+  model: 'mixedbread-ai/mxbai-embed-xsmall-v1',
+});
+const vectors = await model.run(['Hello world', 'Another sentence']);
+```
+
+### Text Generation (LLM)
+
+> Requires `@mlc-ai/web-llm` and `defaultDevice: 'webgpu'`. Models are large — download once, cached permanently.
+
+| Model ID | Size | Notes |
+|----------|------|-------|
+| `Llama-3.2-1B-Instruct-q4f32_1-MLC` | 0.8 GB | Fastest, decent quality |
+| `Llama-3.2-3B-Instruct-q4f32_1-MLC` | 2 GB | Good balance |
+| `Phi-3.5-mini-instruct-q4f16_1-MLC` | 2.2 GB | Microsoft, strong reasoning |
+| `Llama-3.1-8B-Instruct-q4f32_1-MLC` | 5 GB | Best quality, needs 8+ GB RAM |
+| `gemma-2-2b-it-q4f16_1-MLC` | 1.5 GB | Google, fast on mobile GPU |
+
+```typescript
+const llm = await pool.load<string>('text-generation', {
+  model: 'Llama-3.2-3B-Instruct-q4f32_1-MLC',
+});
+const stream = llm.stream({ messages: [{ role: 'user', content: 'Hello!' }] });
+for await (const token of stream) { outputDiv.textContent += token; }
+```
+
+### Speech Recognition
+
+| Model ID | Size | Notes |
+|----------|------|-------|
+| `openai/whisper-tiny` | 39 MB | Fastest, lower accuracy |
+| `openai/whisper-base` | 74 MB | Good balance |
+| `openai/whisper-small` | 244 MB | Better accuracy |
+| `openai/whisper-medium` | 769 MB | Near server-level accuracy |
+
+```typescript
+const model = await pool.load<{ text: string }>('automatic-speech-recognition', {
+  model: 'openai/whisper-base',
+});
+const result = await model.run(float32AudioArray);
+console.log(result.text);
+```
+
+### Text Classification / Sentiment
+
+| Model ID | Size | Notes |
+|----------|------|-------|
+| `Xenova/distilbert-base-uncased-finetuned-sst-2-english` | 67 MB | Positive/negative sentiment |
+| `Xenova/bert-base-multilingual-uncased-sentiment` | 168 MB | Multilingual, 1–5 stars |
+| `Xenova/toxic-bert` | 438 MB | Toxicity detection |
+
+```typescript
+const model = await pool.load<{ label: string; score: number }[]>('text-classification', {
+  model: 'Xenova/distilbert-base-uncased-finetuned-sst-2-english',
+});
+const result = await model.run('I love this product!');
+// [{ label: 'POSITIVE', score: 0.999 }]
+```
+
+### Translation
+
+| Model ID | Size | Notes |
+|----------|------|-------|
+| `Xenova/opus-mt-en-ru` | 74 MB | English → Russian |
+| `Xenova/opus-mt-ru-en` | 74 MB | Russian → English |
+| `Xenova/m2m100_418M` | 418 MB | 100 languages ↔ 100 languages |
+| `Xenova/nllb-200-distilled-600M` | 600 MB | Meta, 200 languages |
+
+```typescript
+const model = await pool.load<{ translation_text: string }[]>('translation', {
+  model: 'Xenova/opus-mt-en-ru',
+});
+const result = await model.run('Hello, world!');
+// [{ translation_text: 'Привет, мир!' }]
+```
+
+### Image Classification
+
+| Model ID | Size | Notes |
+|----------|------|-------|
+| `Xenova/vit-base-patch16-224` | 343 MB | General image classification |
+| `Xenova/mobilevit-small` | 22 MB | Lightweight, mobile-friendly |
+| `Xenova/efficientnet-lite4` | 13 MB | Fastest, 1000 ImageNet classes |
+
+```typescript
+const model = await pool.load<{ label: string; score: number }[]>('image-classification', {
+  model: 'Xenova/efficientnet-lite4',
+});
+const result = await model.run('https://example.com/cat.jpg');
+// [{ label: 'tabby cat', score: 0.92 }]
+```
+
+### How downloads work
+
+```
+First visit:   download from source → save to Cache API → run
+               (5–60s depending on model size and connection)
+
+Next visits:   load from Cache API → run
+               (1–3s initialization only, no network needed)
+
+Offline:       load from Cache API → run
+               (works without internet after first load)
+```
+
+### Where models come from
+
+Models are **not** locked to Hugging Face. Each adapter has its own sources:
+
+**transformers.js** — HF Hub ID or any direct URL:
+
+```typescript
+// From Hugging Face Hub (default)
+await pool.load('feature-extraction', {
+  model: 'mixedbread-ai/mxbai-embed-xsmall-v1',
+});
+
+// From your own CDN or server
+await pool.load('feature-extraction', {
+  model: 'https://your-cdn.com/models/mxbai-embed-xsmall-v1',
+});
+```
+
+The model folder must contain the same file structure as HF Hub: `onnx/model.onnx`, `tokenizer.json`, `config.json`. You can download a model from HF Hub once and re-host it anywhere.
+
+**web-llm** — from the MLC model registry by default. To use your own hosted model, add it to the registry before creating the pool:
+
+```typescript
+import { CreateMLCEngine, prebuiltAppConfig } from '@mlc-ai/web-llm';
+
+// Register a custom model
+const customConfig = {
+  ...prebuiltAppConfig,
+  model_list: [
+    ...prebuiltAppConfig.model_list,
+    {
+      model: 'https://your-cdn.com/my-llm/',  // folder with model shards
+      model_id: 'my-custom-llm',
+      model_lib: 'https://your-cdn.com/my-llm/model.wasm',
+    },
+  ],
+};
+
+// Pass config through the adapter
+const pool = await createPool({ adapter: webLlmAdapter({ appConfig: customConfig }) });
+await pool.load('text-generation', { model: 'my-custom-llm' });
+```
+
+**onnxruntime-web** — direct URL to a `.onnx` file, no registry:
+
+```typescript
+await pool.load('custom', {
+  model: 'https://your-cdn.com/model.onnx',
+});
+```
+
+**Custom adapter** — full control, load from anywhere (fetch, IndexedDB, bundled asset):
+
+```typescript
+async load(task, config, device, onProgress) {
+  // fetch from any source
+  const response = await fetch(config.model as string);
+  const total = Number(response.headers.get('content-length') ?? 0);
+  let loaded = 0;
+
+  const reader = response.body!.getReader();
+  const chunks: Uint8Array[] = [];
+  while (true) {
+    const { done, value } = await reader.read();
+    if (done) break;
+    chunks.push(value);
+    loaded += value.byteLength;
+    onProgress({ loaded, phase: 'downloading', total });
+  }
+
+  // build model from raw bytes
+  const buffer = mergeChunks(chunks);
+  const instance = await MyRuntime.loadFromBuffer(buffer);
+  return { instance, memoryMB: 50 };
+}
+```
+
+---
+
+## When to use
+
+| Scenario | Suitable? |
+|----------|-----------|
+| Semantic search over content | ✓ |
+| Chatbot / text generation | ✓ |
+| Speech transcription | ✓ |
+| Image classification | ✓ |
+| Sentiment analysis | ✓ |
+| Translation | ✓ |
+| Private data processing (data never leaves the device) | ✓ |
+| Offline mode (works after first load, no internet) | ✓ |
+| High-volume batch processing on a server | ✗ use server-side inference |
+| Real-time video/audio streaming analysis | ✗ latency too high for WASM |
+
+### inferis-ml is a good fit when
+
+- You want to avoid per-request API costs
+- Your users' data is sensitive and must not leave the device
+- You need the app to work offline after first load
+- Your users have modern hardware (GPU acceleration is a bonus, not a requirement)
+- You are building a single-page app where the model stays loaded across user interactions
+
+### inferis-ml is not a good fit when
+
+- You need to process large datasets server-side
+- Your model is too large to download in a browser (>4 GB)
+- You need to support very old browsers (IE, Safari < 11)
+- Inference latency must be under 100ms on low-end mobile devices
+
+## License
+
+MIT

package/dist/adapters/onnx.cjs

@@ -0,0 +1,2 @@
+'use strict';function c(){return {name:"onnx",async create(){let d=await import('onnxruntime-web');return {name:"onnx",estimateMemoryMB(n,e){return e.estimatedMemoryMB??50},async load(n,e,a,s){let o=e.model;s({loaded:0,phase:"downloading",total:0});let t=a==="webgpu"?["webgpu","wasm"]:["wasm"],r=await d.InferenceSession.create(o,{executionProviders:t,graphOptimizationLevel:"all"});return s({loaded:1,phase:"done",total:1}),{instance:r,memoryMB:e.estimatedMemoryMB??50}},async run(n,e,a){let s=n.instance,{feeds:o,outputNames:t}=e;return await s.run(o,t)},async stream(n,e,a,s){let o=n.instance,{feeds:t,outputNames:r}=e,i=await o.run(t,r,s);a(i);},async unload(n){await n.instance.release?.();}}}}}exports.onnxAdapter=c;//# sourceMappingURL=onnx.cjs.map
+//# sourceMappingURL=onnx.cjs.map

package/dist/adapters/onnx.cjs.map

@@ -0,0 +1 @@
+{"version":3,"sources":["../../src/adapters/onnx.ts"],"names":["onnxAdapter","ort","_task","config","device","onProgress","modelUrl","executionProviders","session","model","input","_options","feeds","outputNames","onChunk","options","result"],"mappings":"aAyCO,SAASA,GAAmC,CACjD,OAAO,CACL,IAAA,CAAM,OAEN,MAAM,MAAA,EAAgC,CAEpC,IAAMC,EAAM,MAAM,OAAO,iBAAiB,CAAA,CAE1C,OAAO,CACL,IAAA,CAAM,MAAA,CAEN,gBAAA,CAAiBC,EAAeC,CAAAA,CAAyC,CACvE,OAAQA,CAAAA,CAAO,mBAA4C,EAC7D,CAAA,CAEA,MAAM,IAAA,CACJD,EACAC,CAAAA,CACAC,CAAAA,CACAC,EACsB,CACtB,IAAMC,EAAWH,CAAAA,CAAO,KAAA,CAExBE,CAAAA,CAAW,CAAE,OAAQ,CAAA,CAAG,KAAA,CAAO,aAAA,CAAe,KAAA,CAAO,CAAE,CAAC,CAAA,CAExD,IAAME,CAAAA,CAAqBH,IAAW,QAAA,CAClC,CAAC,QAAA,CAAU,MAAM,EACjB,CAAC,MAAM,CAAA,CAELI,CAAAA,CAA4B,MAAMP,CAAAA,CAAI,gBAAA,CAAiB,MAAA,CAAOK,CAAAA,CAAU,CAC5E,kBAAA,CAAAC,CAAAA,CACA,sBAAA,CAAwB,KAC1B,CAAC,CAAA,CAED,OAAAF,EAAW,CAAE,MAAA,CAAQ,EAAG,KAAA,CAAO,MAAA,CAAQ,KAAA,CAAO,CAAE,CAAC,CAAA,CAE1C,CACL,QAAA,CAAUG,CAAAA,CACV,SAAWL,CAAAA,CAAO,iBAAA,EAA4C,EAChE,CACF,EAEA,MAAM,GAAA,CAAIM,CAAAA,CAAoBC,CAAAA,CAAgBC,EAAsC,CAClF,IAAMH,CAAAA,CAAUC,CAAAA,CAAM,SAChB,CAAE,KAAA,CAAAG,CAAAA,CAAO,WAAA,CAAAC,CAAY,CAAA,CAAIH,CAAAA,CAG/B,OADgB,MAAMF,EAAQ,GAAA,CAAII,CAAAA,CAAOC,CAAW,CAEtD,CAAA,CAEA,MAAM,MAAA,CACJJ,CAAAA,CACAC,CAAAA,CACAI,CAAAA,CACAC,EACe,CAGf,IAAMP,CAAAA,CAAUC,CAAAA,CAAM,SAChB,CAAE,KAAA,CAAAG,CAAAA,CAAO,WAAA,CAAAC,CAAY,CAAA,CAAIH,CAAAA,CACzBM,CAAAA,CAAS,MAAMR,EAAQ,GAAA,CAAII,CAAAA,CAAOC,CAAAA,CAAaE,CAAO,EAC5DD,CAAAA,CAAQE,CAAM,EAChB,CAAA,CAEA,MAAM,MAAA,CAAOP,CAAAA,CAAmC,CAE9C,MADgBA,EAAM,QAAA,CACR,OAAA,KAChB,CACF,CACF,CACF,CACF","file":"onnx.cjs","sourcesContent":["import type {\n  Device,\n  LoadedModel,\n  LoadProgressEvent,\n  ModelAdapter,\n  ModelAdapterFactory,\n} from '../core/types.js';\n\n// eslint-disable-next-line ts/no-explicit-any\ntype InferenceSession = any;\n// eslint-disable-next-line ts/no-explicit-any\ntype Tensor = any;\n\ninterface OnnxInput {\n  feeds: Record<string, Tensor>;\n  outputNames?: string[];\n}\n\n/**\n * Adapter for onnxruntime-web.\n *\n * @remarks\n * Provides low-level access to ONNX model inference.\n * Use for custom models not supported by transformers.js or web-llm.\n * Input must be pre-processed `OrtTensor` instances.\n *\n * @example\n * ```ts\n * import { createPool } from 'inferis-ml';\n * import { onnxAdapter } from 'inferis-ml/adapters/onnx';\n * import * as ort from 'onnxruntime-web';\n *\n * const pool = await createPool({ adapter: onnxAdapter() });\n * const model = await pool.load('custom', {\n *   model: 'https://example.com/model.onnx',\n * });\n *\n * const input = new ort.Tensor('float32', data, [1, 3, 224, 224]);\n * const output = await model.run({ feeds: { input } });\n * ```\n */\nexport function onnxAdapter(): ModelAdapterFactory {\n  return {\n    name: 'onnx',\n\n    async create(): Promise<ModelAdapter> {\n      // @ts-expect-error - optional peer dependency, resolved at runtime inside worker\n      const ort = await import('onnxruntime-web');\n\n      return {\n        name: 'onnx',\n\n        estimateMemoryMB(_task: string, config: Record<string, unknown>): number {\n          return (config.estimatedMemoryMB as number | undefined) ?? 50;\n        },\n\n        async load(\n          _task: string,\n          config: Record<string, unknown>,\n          device: Device,\n          onProgress: (event: LoadProgressEvent) => void,\n        ): Promise<LoadedModel> {\n          const modelUrl = config.model as string;\n\n          onProgress({ loaded: 0, phase: 'downloading', total: 0 });\n\n          const executionProviders = device === 'webgpu'\n            ? ['webgpu', 'wasm'] as const\n            : ['wasm'] as const;\n\n          const session: InferenceSession = await ort.InferenceSession.create(modelUrl, {\n            executionProviders,\n            graphOptimizationLevel: 'all',\n          });\n\n          onProgress({ loaded: 1, phase: 'done', total: 1 });\n\n          return {\n            instance: session,\n            memoryMB: (config.estimatedMemoryMB as number | undefined) ?? 50,\n          };\n        },\n\n        async run(model: LoadedModel, input: unknown, _options?: unknown): Promise<unknown> {\n          const session = model.instance as InferenceSession;\n          const { feeds, outputNames } = input as OnnxInput;\n\n          const results = await session.run(feeds, outputNames);\n          return results;\n        },\n\n        async stream(\n          model: LoadedModel,\n          input: unknown,\n          onChunk: (chunk: unknown) => void,\n          options?: unknown,\n        ): Promise<void> {\n          // ONNX Runtime Web doesn't natively support streaming inference.\n          // Run full inference and emit the result as a single chunk.\n          const session = model.instance as InferenceSession;\n          const { feeds, outputNames } = input as OnnxInput;\n          const result = await session.run(feeds, outputNames, options);\n          onChunk(result);\n        },\n\n        async unload(model: LoadedModel): Promise<void> {\n          const session = model.instance as InferenceSession;\n          await session.release?.();\n        },\n      };\n    },\n  };\n}\n"]}