foxmind 0.0.0-stage → 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Pooria Arab
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,309 @@
-# Temporary Holding Version
+# foxmind
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+One API for on-device AI models in the browser and for your own model key.
+
+foxmind gives you one `chat`, `embed`, `extract` and `classify` call over three
+tiers of models: models that run in the browser, a model server on your own
+machine, and a cloud model that you pay for with your own key. Every result
+says which tier answered. When a tier cannot run, foxmind says why. It does not
+switch to another tier in secret.
+
+## Install
+
+```bash
+npm i foxmind
+```
+
+For the in-browser models, also install the optional peer dependency:
+`npm i @huggingface/transformers`.
+
+## Example
+
+This runs in Node 24 or later. Start a local server first, for example
+`ollama pull qwen3:0.6b`.
+
+```js
+import { createMind, ollama, saluki } from "foxmind";
+
+const mind = createMind({
+  providers: [saluki(), ollama({ model: "qwen3:0.6b" })],
+  only: ["browser", "local"], // private mode: no cloud provider is ever called
+});
+
+const result = await mind.chat([{ role: "user", content: "Say hello in five words." }], { maxTokens: 1024 });
+console.log(result.message.content);
+console.log(`answered by ${result.provider} (${result.tier})`);
+console.log(result.skipped.map((skip) => `${skip.provider}: ${skip.code}`));
+```
+
+Output on our test machine. Saluki was not running, so the router skipped it
+and said why:
+
+```text
+Hi there!
+answered by ollama (local)
+[ 'saluki: unreachable' ]
+```
+
+Messages, tools and tool calls use the OpenAI chat completions shape. Code that
+you wrote for that API works here.
+
+## Use cases
+
+| Who | What they build | How foxmind helps |
+|---|---|---|
+| A Firefox extension developer | Local AI features in an extension: search, summaries, form help | `foxmind/browser` runs embeddings and small chat models in the extension's background page, on WebGPU or on single-thread WASM. |
+| A web app team | A "bring your own key" AI feature | `anthropic()` and `openaiCompatible()` take the user's key at call time and map both APIs to one shape. foxmind does not store the key. |
+| An agent builder who wants privacy | A private-mode agent planner (for example in foxloop or foxmate) | `saluki()` runs Underdog Saluki 27B on llama-server on the same machine. `only: ["browser", "local"]` drops the cloud tier, so page text cannot leave the machine through foxmind. |
+| A form-filling tool | Pull names, dates and places out of a request | `gliner2()` runs GLiNER2 entity extraction and label scoring in the browser, as foxpilot does. |
+| A notes or bookmarks app | Local semantic search | `mind.embed()` with `transformers()` or Firefox's own `trialML()` gives vectors without a server. |
+| A developer or a CI job | A check of which model servers run on a machine | `foxmind doctor` probes llama-server, Ollama and LM Studio, and prints the command that starts each one that is down. |
+
+## How it works
+
+```mermaid
+flowchart TB
+  caller["Your code: mind.chat / embed / extract / classify"] --> router["createMind router"]
+  router -->|"1. drop tiers outside only, order by prefer"| order["providers in order"]
+  order -->|"2. probe (cached 30 s)"| probe{"can it run now?"}
+  probe -->|no| skip["add to result.skipped with code and reason"]
+  skip --> order
+  probe -->|yes| call["call the provider"]
+  call -->|ok| result["result + provider + tier + skipped"]
+  call -->|error| error["throw FoxmindError with code, provider, tier"]
+  subgraph browser["Tier 1: in the browser"]
+    T["transformers() embed or chat"]
+    G["gliner2() extract and classify"]
+    M["trialML() / wllama() on browser.trial.ml"]
+  end
+  subgraph local["Tier 2: a server on this machine"]
+    L["llamaServer() / saluki() :8080"]
+    O["ollama() :11434"]
+    S["lmStudio() :1234"]
+  end
+  subgraph cloud["Tier 3: your own key"]
+    A["anthropic()"]
+    C["openaiCompatible({ baseURL, apiKey })"]
+  end
+  call --- browser
+  call --- local
+  call --- cloud
+```
+
+The router drops every provider whose tier is not in `only`, then tries the
+rest in the `prefer` order. `prefer` only changes the order; use `only` when
+text must not reach a tier. A provider that cannot run
+now (no WebGPU, server down, model missing, permission not granted) is skipped,
+and the skip goes into `result.skipped`. When a provider fails during a call,
+the router throws that error. It tries the next provider only when you set
+`fallbackOnError: true`, and never after text has streamed or the caller has
+aborted.
+
+```mermaid
+sequenceDiagram
+  participant Panel as Demo panel (popup or sidebar)
+  participant BG as Background page
+  participant Mind as createMind
+  participant Server as llama-server
+  Panel->>BG: runtime.sendMessage({ op: "local-chat" })
+  BG->>Mind: chat(messages)
+  Mind->>Server: GET /v1/models (probe)
+  Mind->>Server: POST /v1/chat/completions
+  Server-->>Mind: reply
+  Mind-->>BG: reply, provider "llama-server", tier "local"
+  BG-->>Panel: answer and "answered by llama-server (local)"
+```
+
+In an extension, the models live in the background page (an event page with a
+DOM in Firefox). Each view asks it with `runtime.sendMessage`, so the views
+share one loaded model.
+
+## API
+
+### `foxmind` (Node and the browser)
+
+| Export | What it does |
+|---|---|
+| `createMind({ providers, only?, prefer?, fallbackOnError?, probeTtlMs? })` | Makes the router. `only` is the list of tiers it may use (private mode: `["browser", "local"]`); it never probes or calls the others. `prefer` takes provider names or tiers and only changes the order. |
+| `mind.chat(messages, { tools?, json?, onDelta?, temperature?, maxTokens?, timeoutMs?, signal? })` | Chat in the OpenAI shape. `onDelta` streams text. `json: true` fails with `bad_json` when the reply is not JSON. Tool call arguments are checked: they must be a JSON object. `timeoutMs` is the longest wait for the headers and then for each next piece of the body, so a slow stream that keeps sending is not cut. |
+| `mind.embed(texts)` | One vector per text: `{ vectors, provider, tier, model, ms, skipped }`. |
+| `mind.extract(text, labels, { threshold? })` | GLiNER2 entities per label: `{ entities: { label: [{ text, confidence, start, end }] } }`. |
+| `mind.classify(texts, prompt, labels)` | GLiNER2 label scores per text: `{ scores: [{ label: probability }] }`. |
+| `mind.status()` | Each provider's state (`idle`, `loading`, `ready`, `unavailable`, `error`), where it runs, download progress, and the last provider that answered. |
+| `mind.probe()` / `mind.load(name)` | Probe every provider now. Download and start one model now. |
+| `openaiCompatible({ baseURL, model, apiKey?, embedModel?, body?, checkModel?, timeoutMs? })` | Any OpenAI-compatible server: llama-server, Ollama, LM Studio, OpenAI, OpenRouter. The tier is `local` for localhost, else `cloud`. |
+| `ollama({ model })`, `llamaServer()`, `lmStudio({ model })` | Presets with the usual local address and the start command in each failed probe. |
+| `saluki({ thinking? })`, `SALUKI` | Underdog Saluki 27B on llama-server, the default planner for private mode. Thinking is off and temperature is 0 by default, the model card's settings for tool calls. |
+| `anthropic({ apiKey, model?, maxTokens? })` | The Anthropic Messages API, mapped to the OpenAI shape. The default model is `claude-opus-5-5`. |
+| `doctor(options)`, `formatDoctor(report)` | The `foxmind doctor` check as a function. |
+| `FoxmindError` | Every failure. `code` is one of `no_provider`, `unreachable`, `timeout`, `aborted`, `rate_limited` (with `retryAfterMs`), `auth`, `model_not_found`, `http`, `stream_interrupted` (with `partial`), `bad_tool_call` (with `raw`), `bad_json`, `bad_response`, `unsupported`, `download_failed`, `cache_corrupt`, `out_of_memory`, `webgpu_missing`, `permission`. |
+
+To run Saluki, follow its [model card](https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0):
+
+```bash
+huggingface-cli download ConwayResearch/Underdog-Saluki-27B-1.0 Underdog-Saluki-27B-1.0-IQ2-mix.gguf --local-dir .
+llama-server -m Underdog-Saluki-27B-1.0-IQ2-mix.gguf --jinja -ngl 99 -fa on -c 32768
+```
+
+### `foxmind/browser` (extension pages and web pages)
+
+| Export | What it does |
+|---|---|
+| `transformers({ task: "embed" \| "chat", model?, device?, dtype? })` | transformers.js. Defaults: `Xenova/all-MiniLM-L6-v2` for embed, `onnx-community/Qwen3-0.6B-ONNX` for chat. `device: "auto"` uses WebGPU when the browser has an adapter, else WASM, and `status().where` says which. |
+| `gliner2({ model?, device?, dtype? })` | GLiNER2 extract and classify. Default model: `pooria/gliner2-multi-v1-agent-batch-ONNX` (614 MB at fp16). |
+| `trialML({ task, model?, device? })`, `requestTrialML()` | Firefox's own engine, `browser.trial.ml`. Call `requestTrialML()` from a click to get the optional `trialML` permission. |
+| `wllama({ model, modelFile, maxBytes? })` | Experimental. A GGUF model on Firefox's llama.cpp backend through trial ML. It refuses files over `maxBytes` (default 4 GB) before the download. |
+| `configureRuntime({ wasmPaths?, remoteHost? })` | Where ONNX Runtime's WASM files are (default: `ort/` in the extension) and which model hub to use. |
+| `hasWebGPU()`, `purgeModel(model)` | Detect a WebGPU adapter. Delete one model's files from Cache Storage. |
+
+Each cached model file that does not load is deleted and downloaded once more,
+and `status().reason` says so. A download that stops is not kept, so the next
+call downloads again.
+
+### CLI: `foxmind doctor`
+
+```bash
+npx foxmind doctor
+```
+
+```text
+foxmind doctor · Node v25.9.0 · darwin arm64
+
+local    ollama        yes  http://127.0.0.1:11434/v1    models: qwen3:0.6b
+local    llama-server  yes  http://127.0.0.1:8080/v1     models: qwen3-0.6b.gguf
+local    saluki        no   http://127.0.0.1:8080/v1     llama-server serves qwen3-0.6b.gguf, not Saluki. Download it with "huggingface-cli download ConwayResearch/Underdog-Saluki-27B-1.0 Underdog-Saluki-27B-1.0-IQ2-mix.gguf --local-dir .", then start it with "llama-server -m Underdog-Saluki-27B-1.0-IQ2-mix.gguf --jinja -ngl 99 -fa on -c 32768".
+local    lm-studio     no   http://127.0.0.1:1234/v1     Cannot reach http://127.0.0.1:1234/v1/models: connect ECONNREFUSED 127.0.0.1:1234. Start it with "lms server start".
+browser  WebGPU, trial.ml and in-browser models: doctor runs in Node and cannot check them. Load the demo extension in Firefox.
+cloud    anthropic and cloud openaiCompatible: these need your own key. doctor does not read keys.
+```
+
+Flags: `--json`, `--ollama URL`, `--llama-server URL`, `--lm-studio URL`,
+`--timeout MS`. The exit code is 0 when at least one server works, 1 when none
+does, and 2 for bad input. There is no MCP server.
+
+### Demo extension
+
+`extension/` is a small Firefox extension. Its panel opens from the toolbar and
+in the sidebar. It shows which tiers work, sends a test prompt to a local
+server, and compares two sentences with an in-browser embedding model. Build it
+with `pnpm build:ext` and load `dist-ext/manifest.json` from `about:debugging`.
+`pnpm build:ext` makes the build AMO signs. `node scripts/build-ext.mjs --e2e`
+makes the test build that `pnpm e2e` loads; only it has the test-only ops and
+the content script. To let it reach Ollama, start Ollama with
+`OLLAMA_ORIGINS="moz-extension://*"`.
+Ollama refuses extension origins without it.
+
+## Tests
+
+`pnpm ci:local` runs lint, typecheck, 71 tests and the build. The tests start
+fake servers and call them over real HTTP. `docs/failure-modes.md` lists each
+failure mode (F1 to F72) and its test. The tests went in before the code.
+
+`pnpm e2e` runs the real thing and writes `artifacts/e2e-<date>.json`. First it
+sends chat, tool calls, streams and JSON requests to each local server that
+runs. Then it starts Firefox with the demo extension. `FOXMIND_E2E_HEAVY=1`
+adds Qwen3-0.6B chat and GLiNER2 in the browser. Results from our run on
+2026-10-09, in `artifacts/e2e-2026-10-09.json` (Firefox 157.0.1, Apple M3 Pro,
+headless, so WASM; all 32 checks passed):
+
+| Check | Result |
+|---|---|
+| llama-server, Qwen3-0.6B GGUF: chat / tool call / stream | 82 ms / 632 ms / 210 ms |
+| Ollama, qwen3:0.6b: chat / tool call / stream (it thinks first) | 1.7 s / 0.9 s / 1.3 s |
+| MiniLM embedding in Firefox: first load with download / cached load / one call | 1,124 ms / 180 ms / 9 ms |
+| Firefox trial ML, MiniLM embedding: first call / next call | 2.0 s / 17 ms |
+| Qwen3-0.6B chat in Firefox (WASM, q4): first answer with download / tool call | 34.2 s / 28.3 s |
+| GLiNER2 in Firefox (WASM, fp16): load with 614 MB download / extract / classify | 19.6 s / 1.6 s / 1.5 s |
+| GLiNER2 against the Python gliner2 library | 14 of 14 reference cases match |
+| Saluki 27B in the browser | refused before download: 7.90 GB is over the 4 GB limit |
+
+After the review fixes (redirects, aborts, `only`, idle timeouts, the release
+build), we ran `pnpm e2e` again without the heavy models:
+`artifacts/e2e-review-fixes-2026-10-09.json`, 27 of 27 checks passed. The heavy
+run was not repeated after the fixes: the test machine was under heavy load from
+other jobs, and the in-browser Qwen3 call passed the test's 180 s limit.
+
+## Firefox APIs used
+
+| API | MDN | Why |
+|---|---|---|
+| `browser.trial.ml` (`createEngine`, `runEngine`, `onProgress`) | No MDN page: [Firefox source docs](https://firefox-source-docs.mozilla.org/toolkit/components/ml/extensions.html) | `trialML()` and `wllama()` run Firefox's own inference engine. |
+| `permissions.request` / `permissions.contains` | [MDN](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/permissions/request) | Ask for the optional `trialML` permission, and check it before each call. |
+| `optional_permissions` | [MDN](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/optional_permissions) | `trialML` is an optional-only permission. |
+| `host_permissions` | [MDN](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/host_permissions) | The demo calls local servers on `127.0.0.1` and `localhost`. |
+| `runtime.sendMessage` / `runtime.onMessage` | [MDN](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/runtime/sendMessage) | The panel asks the background page, which hosts the models. |
+| `runtime.getURL` | [MDN](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/runtime/getURL) | ONNX Runtime loads its WASM files from inside the extension, because MV3 allows no remote code. |
+| Background scripts (event page) | [MDN](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Background_scripts) | The page that hosts the models. It has a DOM, Cache Storage and WebGPU. |
+| `action` popup and `sidebar_action` | [MDN](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/sidebar_action) | The demo panel opens in both places. |
+| `content_security_policy` with `'wasm-unsafe-eval'` | [MDN](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/content_security_policy) | Lets the extension compile WebAssembly. |
+| `unlimitedStorage` | [MDN](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/permissions#unlimitedstorage) | Model files can be hundreds of MB. |
+| `browser_specific_settings.gecko.data_collection_permissions` | [MDN](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/browser_specific_settings) | The demo collects no data. If your extension sends page text to a cloud key, declare `websiteContent` under `optional`. |
+| WebGPU (`navigator.gpu.requestAdapter`) | [MDN](https://developer.mozilla.org/en-US/docs/Web/API/GPU/requestAdapter) | `device: "auto"` picks WebGPU only when there is an adapter. |
+| WebAssembly | [MDN](https://developer.mozilla.org/en-US/docs/WebAssembly) | ONNX Runtime runs models on the CPU when there is no WebGPU. |
+| `crossOriginIsolated` / `SharedArrayBuffer` | [MDN](https://developer.mozilla.org/en-US/docs/Web/API/Window/crossOriginIsolated) | Extension pages have no `SharedArrayBuffer`, so the runtime uses one WASM thread. |
+| Cache Storage (`caches`) | [MDN](https://developer.mozilla.org/en-US/docs/Web/API/CacheStorage) | transformers.js caches model files there. foxmind deletes a model's files when they do not load. |
+| `fetch`, `AbortSignal.timeout`, `AbortSignal.any` | [MDN](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal/any_static) | Every server call has a timeout and honors the caller's abort. |
+
+## Limits
+
+- We did not run Saluki 27B. We tested the `saluki()` preset against a fake
+  server and checked that it refuses to run in the browser. The Saluki
+  benchmark numbers are from its vendor (Underdog Bench, 120 tasks), not from
+  us.
+- A 27B model in the browser is not proven. `wllama()` refuses GGUF files over
+  4 GB, and trial ML takes models only from the Mozilla and Xenova orgs.
+- `wllama()` is experimental. In our Firefox 157 runs, the llama.cpp engine
+  started but never answered, so the call ends with code `timeout`.
+- We tested `anthropic()` and cloud `openaiCompatible()` against fake servers
+  only. No real API key was used.
+- Small in-browser chat models are slow on WASM (about 28 s per answer for
+  Qwen3-0.6B), and they call a tool only when the prompt asks for it.
+- We ran GLiNER2 on WASM only. Its WebGPU read-back code comes from foxpilot
+  and has no foxmind test yet.
+- For a model as small as MiniLM, WebGPU was not faster than WASM in our
+  headed run.
+- `json: true` on Anthropic is a system-prompt rule plus a JSON check, not
+  constrained decoding.
+- foxmind does not retry. On `rate_limited`, use `retryAfterMs` to decide.
+- Firefox unloads an idle background page, and the loaded model goes with it.
+  An open popup or sidebar keeps the page loaded.
+- Firefox allows one trial ML engine per extension.
+- The in-browser providers do not run in Node. The server and cloud tiers do.
+- Ollama can run a model on ollama.com. foxmind treats a model named
+  `*-cloud` or `*:cloud` as tier `cloud` (for `ollama()` and for
+  `openaiCompatible()` on localhost), so `only: ["browser", "local"]` drops it.
+  For other names, `ollama().probe()` reads `/api/tags` and refuses a model
+  with a `remote_host` unless you pass `tier: "cloud"`. Other local servers
+  that forward to a remote model are not detected.
+- ONNX Runtime adds about 27 MB of WASM to an extension.
+- foxmind does not check model files against a hash after the download, and
+  transformers.js does not either. Hugging Face lists a SHA-256 for each LFS
+  file in its API, so a check is possible later.
+
+## Part of the fox primitives
+
+```mermaid
+flowchart LR
+  foxkit[foxkit] -- template --> foxmind[foxmind]
+  foxmind --> foxpaw[foxpaw]
+  foxmind --> foxloop[foxloop]
+  foxmind -. optional .-> foxshield[foxshield]
+  foxmind --> foxmemory[foxmemory]
+  foxmind --> foxlens[foxlens]
+  foxvault[foxvault] -. will hold the keys .-> foxmind
+  click foxkit "https://github.com/pooriaarab/foxkit"
+  click foxmind "https://github.com/pooriaarab/foxmind"
+  click foxpaw "https://github.com/pooriaarab/foxpaw"
+  click foxloop "https://github.com/pooriaarab/foxloop"
+  click foxshield "https://github.com/pooriaarab/foxshield"
+  click foxmemory "https://github.com/pooriaarab/foxmemory"
+  click foxlens "https://github.com/pooriaarab/foxlens"
+  click foxvault "https://github.com/pooriaarab/foxvault"
+```
+
+foxmind depends on no other fox primitive. The GLiNER2 code comes from
+[foxpilot](https://github.com/pooriaarab/foxpilot) (MIT, same author).
+
+## License
+
+[MIT](LICENSE)

package/dist/bin.d.ts

@@ -0,0 +1,2 @@
+#!/usr/bin/env node
+export {};

package/dist/bin.js

@@ -0,0 +1,6 @@
+#!/usr/bin/env node
+import { main } from "./cli.js";
+process.exitCode = await main(process.argv.slice(2), {
+    out: (text) => process.stdout.write(text),
+    err: (text) => process.stderr.write(text),
+});

package/dist/browser/gliner2-model.d.ts

@@ -0,0 +1,57 @@
+import type { Entity, Labels } from "../types.js";
+export type Encoded = {
+    inputIds: number[];
+    wordPositions: number[];
+    schemaPositions: number[];
+    /** Character offsets of each text word in the (period-terminated) text. */
+    starts: number[];
+    ends: number[];
+    text: string;
+};
+export type EncodeTask = {
+    name: string;
+    marker: "[E]" | "[L]";
+    labels: Labels;
+};
+export declare class Gliner2 {
+    private readonly js;
+    private readonly model;
+    private readonly tokenizer;
+    private readonly ids;
+    private constructor();
+    static load(modelId: string, options?: {
+        device?: "webgpu" | "wasm";
+        dtype?: "fp32" | "fp16";
+        progress_callback?: (info: {
+            status?: string;
+            progress?: number;
+        }) => void;
+        session_options?: Record<string, unknown>;
+    }): Promise<Gliner2>;
+    private pieces;
+    /**
+     * `( [P] name [DESCRIPTION] label: desc … ( [E] label … ) ) [SEP_TEXT] words`.
+     * The prompt and labels keep their case and are tokenized whole; the text
+     * gets a terminal "." if it has none, is split with WORD and lowercased,
+     * and each word is tokenized on its own (its first piece is its position).
+     */
+    encode(input: string, task: EncodeTask): Encoded;
+    /**
+     * Runs the graph once on every row and reads back only `names`. Rows are
+     * padded to the longest with 0 (attention mask 0); a row's outputs past its
+     * own label and word counts are padding. Every output on the GPU is released.
+     */
+    private run;
+    /** Softmax over the labels, like ClassificationSchema().single(..., activation="softmax"). */
+    classify(text: string, name: string, labels: Labels): Promise<Record<string, number>>;
+    /**
+     * classify() for several texts against one prompt and label set, in one run
+     * of the graph and one read back. Each call costs a fixed 300-500 ms in
+     * Firefox (foxpilot #70); a row adds much less (foxpilot #90).
+     */
+    classifyMany(texts: string[], name: string, labels: Labels): Promise<Record<string, number>[]>;
+    /** extract_entities(text, types) with include_confidence and include_spans. */
+    extractEntities(text: string, types: Labels, threshold?: number): Promise<Record<string, Entity[]>>;
+}
+/** gliner2's default span decoder: confidence-first greedy, no character overlaps. */
+export declare function finalizeSpans(raw: Entity[]): Entity[];

package/dist/browser/gliner2-model.js

@@ -0,0 +1,233 @@
+import { transformersJs } from "./runtime.js";
+/** Same pattern as gliner2's WhitespaceTokenSplitter. */
+const WORD = /(?:https?:\/\/[^\s]+|www\.[^\s]+)|[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,}|@[a-z0-9_]+|\w+(?:[-_]\w+)*|\S/giu;
+const SPECIAL = ["[P]", "[E]", "[L]", "[SEP_TEXT]", "[DESCRIPTION]"];
+export class Gliner2 {
+    js;
+    model;
+    tokenizer;
+    ids;
+    constructor(js, model, tokenizer, ids) {
+        this.js = js;
+        this.model = model;
+        this.tokenizer = tokenizer;
+        this.ids = ids;
+    }
+    static async load(modelId, options = {}) {
+        const js = await transformersJs();
+        const device = options.device ?? "webgpu";
+        const tokenizer = await js.AutoTokenizer.from_pretrained(modelId);
+        const model = await js.AutoModel.from_pretrained(modelId, {
+            device,
+            dtype: options.dtype ?? "fp16",
+            progress_callback: options.progress_callback,
+            // On WebGPU, outputs stay on the GPU. Reading every output back costs
+            // about 200 ms per call in Firefox (foxpilot #70), so run() reads only what it needs.
+            session_options: {
+                ...(device === "webgpu" ? { preferredOutputLocation: "gpu-buffer" } : {}),
+                ...options.session_options,
+            },
+        });
+        // Added tokens encode to exactly one id.
+        const ids = Object.fromEntries(SPECIAL.map((token) => {
+            const { input_ids } = tokenizer(token, { add_special_tokens: false });
+            const encoded = Array.from(input_ids.data, Number);
+            if (encoded.length !== 1)
+                throw new Error(`tokenizer has no single id for ${token}`);
+            return [token, encoded[0]];
+        }));
+        return new Gliner2(js, model, tokenizer, ids);
+    }
+    pieces(text) {
+        const { input_ids } = this.tokenizer(text, { add_special_tokens: false });
+        return Array.from(input_ids.data, Number);
+    }
+    /**
+     * `( [P] name [DESCRIPTION] label: desc … ( [E] label … ) ) [SEP_TEXT] words`.
+     * The prompt and labels keep their case and are tokenized whole; the text
+     * gets a terminal "." if it has none, is split with WORD and lowercased,
+     * and each word is tokenized on its own (its first piece is its position).
+     */
+    encode(input, task) {
+        const text = !input ? "." : /[.!?]$/.test(input) ? input : `${input}.`;
+        const labelNames = Object.keys(task.labels);
+        const prompt = task.name +
+            labelNames
+                .map((label) => (task.labels[label] ? ` [DESCRIPTION] ${label}: ${task.labels[label]}` : ""))
+                .join("");
+        const inputIds = [];
+        const schemaPositions = [];
+        const push = (ids) => inputIds.push(...ids);
+        const special = (token) => {
+            if (token === "[P]" || token === task.marker)
+                schemaPositions.push(inputIds.length);
+            inputIds.push(this.ids[token]);
+        };
+        push(this.pieces("("));
+        special("[P]");
+        // Whole string, as the processor does; the tokenizer maps [DESCRIPTION] itself.
+        push(this.pieces(prompt));
+        push(this.pieces("("));
+        for (const label of labelNames) {
+            special(task.marker);
+            push(this.pieces(label));
+        }
+        push(this.pieces(")"));
+        push(this.pieces(")"));
+        special("[SEP_TEXT]");
+        const wordPositions = [];
+        const starts = [];
+        const ends = [];
+        for (const match of text.matchAll(WORD)) {
+            wordPositions.push(inputIds.length);
+            starts.push(match.index);
+            ends.push(match.index + match[0].length);
+            push(this.pieces(match[0].toLowerCase()));
+        }
+        return { inputIds, wordPositions, schemaPositions, starts, ends, text };
+    }
+    /**
+     * Runs the graph once on every row and reads back only `names`. Rows are
+     * padded to the longest with 0 (attention mask 0); a row's outputs past its
+     * own label and word counts are padding. Every output on the GPU is released.
+     */
+    async run(rows, names) {
+        const long = (values) => {
+            const n = Math.max(...values.map((row) => row.length));
+            const data = BigInt64Array.from(values.flatMap((row) => [...row, ...Array.from({ length: n - row.length }, () => 0)]), BigInt);
+            return new this.js.Tensor("int64", data, [values.length, n]);
+        };
+        const outputs = (await this.model({
+            input_ids: long(rows.map((row) => row.inputIds)),
+            attention_mask: long(rows.map((row) => Array.from({ length: row.inputIds.length }, () => 1))),
+            word_positions: long(rows.map((row) => row.wordPositions)),
+            schema_positions: long(rows.map((row) => row.schemaPositions)),
+        }));
+        try {
+            const read = await readBack(this.js, names.map((name) => outputs[name]));
+            return Object.fromEntries(names.map((name, i) => [name, { data: Array.from(read[i].to("float32").data), dims: read[i].dims }]));
+        }
+        finally {
+            for (const tensor of Object.values(outputs))
+                if (tensor.location === "gpu-buffer")
+                    tensor.dispose();
+        }
+    }
+    /** Softmax over the labels, like ClassificationSchema().single(..., activation="softmax"). */
+    async classify(text, name, labels) {
+        return (await this.classifyMany([text], name, labels))[0];
+    }
+    /**
+     * classify() for several texts against one prompt and label set, in one run
+     * of the graph and one read back. Each call costs a fixed 300-500 ms in
+     * Firefox (foxpilot #70); a row adds much less (foxpilot #90).
+     */
+    async classifyMany(texts, name, labels) {
+        if (!texts.length)
+            return [];
+        const rows = texts.map((text) => this.encode(text, { name, marker: "[L]", labels }));
+        const read = (await this.run(rows, ["cls_logits"])).cls_logits;
+        const width = read.dims[1];
+        const names = Object.keys(labels);
+        const results = texts.map((_, row) => {
+            const cls = read.data.slice(row * width, row * width + names.length);
+            const max = Math.max(...cls);
+            const exp = cls.map((x) => Math.exp(x - max));
+            const sum = exp.reduce((a, b) => a + b, 0);
+            return Object.fromEntries(names.map((label, i) => [label, exp[i] / sum]));
+        });
+        return results;
+    }
+    /** extract_entities(text, types) with include_confidence and include_spans. */
+    async extractEntities(text, types, threshold = 0.5) {
+        const encoded = this.encode(text, { name: "entities", marker: "[E]", labels: types });
+        const read = await this.run([encoded], ["count_logits", "span_logits"]);
+        const count = read.count_logits.data;
+        const span = read.span_logits.data;
+        const names = Object.keys(types);
+        const result = Object.fromEntries(names.map((name) => [name, []]));
+        if (argmax(count) <= 0)
+            return result;
+        const [, , words, width] = read.span_logits.dims;
+        names.forEach((name, li) => {
+            const raw = [];
+            for (let start = 0; start < words; start++) {
+                for (let w = 0; w < width; w++) {
+                    const end = start + w + 1;
+                    if (end > words)
+                        continue;
+                    const confidence = sigmoid(span[(li * words + start) * width + w]);
+                    if (confidence < threshold)
+                        continue;
+                    const charStart = encoded.starts[start];
+                    const charEnd = encoded.ends[end - 1];
+                    const surface = encoded.text.slice(charStart, charEnd).trim();
+                    if (surface)
+                        raw.push({ text: surface, confidence, start: charStart, end: charEnd });
+                }
+            }
+            result[name] = finalizeSpans(raw);
+        });
+        return result;
+    }
+}
+/**
+ * Copies every GPU tensor in `tensors` to the CPU with one mapAsync: each
+ * GPU to CPU round trip costs about 90 ms in Firefox (foxpilot #70). CPU tensors pass
+ * through unchanged.
+ */
+async function readBack(js, tensors) {
+    const onGpu = tensors.filter((tensor) => tensor.location === "gpu-buffer");
+    if (!onGpu.length)
+        return tensors;
+    const device = js.env.backends.onnx.webgpu?.device;
+    if (!device)
+        throw new Error("ONNX Runtime has no WebGPU device");
+    // ONNX Runtime pads GPU buffers to 16 bytes, so each padded copy stays inside its buffer.
+    const bytes = onGpu.map((tensor) => {
+        if (tensor.type !== "float32" && tensor.type !== "float16")
+            throw new Error(`cannot read back ${tensor.type}`);
+        return tensor.size * (tensor.type === "float16" ? 2 : 4);
+    });
+    const padded = bytes.map((size) => Math.ceil(size / 16) * 16);
+    const offsets = padded.map((_, i) => padded.slice(0, i).reduce((a, b) => a + b, 0));
+    // MAP_READ | COPY_DST
+    const staging = device.createBuffer({ size: padded.reduce((a, b) => a + b, 0), usage: 0x01 | 0x08 });
+    try {
+        const encoder = device.createCommandEncoder();
+        onGpu.forEach((tensor, i) => encoder.copyBufferToBuffer(tensor.ort_tensor.gpuBuffer, 0, staging, offsets[i], padded[i]));
+        device.queue.submit([encoder.finish()]);
+        await staging.mapAsync(0x01); // GPUMapMode.READ
+        const data = staging.getMappedRange().slice(0);
+        return tensors.map((tensor) => {
+            const i = onGpu.indexOf(tensor);
+            if (i < 0)
+                return tensor;
+            const values = tensor.type === "float16" ? new Uint16Array(data, offsets[i], tensor.size) : new Float32Array(data, offsets[i], tensor.size);
+            return new js.Tensor(tensor.type, values, tensor.dims);
+        });
+    }
+    finally {
+        staging.destroy();
+    }
+}
+/** gliner2's default span decoder: confidence-first greedy, no character overlaps. */
+export function finalizeSpans(raw) {
+    const kept = [];
+    for (const candidate of raw.toSorted((a, b) => b.confidence - a.confidence)) {
+        if (kept.some((existing) => candidate.start < existing.end && existing.start < candidate.end))
+            continue;
+        kept.push(candidate);
+    }
+    return kept;
+}
+function sigmoid(x) {
+    return x >= 0 ? 1 / (1 + Math.exp(-x)) : Math.exp(x) / (1 + Math.exp(x));
+}
+function argmax(values) {
+    let best = 0;
+    for (let i = 1; i < values.length; i++)
+        if (values[i] > values[best])
+            best = i;
+    return best;
+}

package/dist/browser/gliner2.d.ts

@@ -0,0 +1,13 @@
+import type { Provider } from "../types.js";
+import { type Device } from "./runtime.js";
+export interface Gliner2Options {
+    /** Default "pooria/gliner2-multi-v1-agent-batch-ONNX". */
+    model?: string;
+    /** Default "auto": WebGPU when the browser has it, else WASM. */
+    device?: Device;
+    /** Default "fp16". */
+    dtype?: "fp32" | "fp16";
+    /** Default "gliner2". */
+    name?: string;
+}
+export declare function gliner2(options?: Gliner2Options): Provider;