@tanstack-ai-browser/webgllm 0.0.0-stage → 0.2.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 daveycodez
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,122 @@
-# Temporary Holding Version
+# @tanstack-ai-browser/webgllm
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+A [TanStack AI](https://tanstack.com/ai) adapter for [webgllm](https://webgllm.dev): LFM3 1.7B, LFM2.5, Liquid's vision models (LFM3-VL and LFM2.5-VL) and Qwen3.5 2B running in the browser on WebGL 2, in a Web Worker on an OffscreenCanvas. It runs with or without WebGPU; where there's no WebGPU (Firefox on Android, older iPhones) it still reads prompts on the GPU.
+
+```sh
+npm install @tanstack-ai-browser/webgllm @tanstack/ai
+```
+
+```ts
+import { chat } from "@tanstack/ai"
+import { webgllmText } from "@tanstack-ai-browser/webgllm"
+
+const stream = chat({
+  adapter: webgllmText("LiquidAI/LFM3-1.7B-GGUF"),
+  messages,
+  threadId
+})
+```
+
+It needs WebGL 2 with float render targets (`EXT_color_buffer_float`) and OffscreenCanvas in workers, which current Chrome, Firefox and Safari have; an adapter's `availability()` says `"unavailable"` where they're missing.
+
+The package is shaped like `@tanstack/ai-openai`'s text adapter. The root has `webgllmText(model, config?)`, `WebGllmTextAdapter` (with the runtime methods `availability`, `create` and `destroy`) and the types `WebGllmConfig`, `WebGllmModel`, `WebGllmTextModelOptions`, `Availability` and `DownloadProgressCallback`, and the catalog (below).
+
+The engine is the [`webgllm`](https://www.npmjs.com/package/webgllm) package, behind OpenAI's Chat Completions API in the page. The adapter is TanStack's Chat Completions adapter (`@tanstack/openai-base`) with the shared `WebGLLM` instance as its client, so messages, tools, structured output and the stream are TanStack's; it adds loading the model, webgllm's reasoning levels and thinking budget, and its options. Tool-call parsing and repair, thinking, guided tool choice and schema-constrained answers are webgllm's own. See [webgllm.dev](https://webgllm.dev) and its README for how the engine works, its memory and speed.
+
+## The adapter
+
+`webgllmText(model, config?)` takes:
+
+- `contextLength`: most tokens a chat may hold, prompt and reply together; the model's whole context by default (LFM3: 262,144), never more. Smaller takes less memory. Changing it loads the model again.
+- `worker`: a Web Worker of your own to run the model in (see A worker of your own).
+
+There are no callbacks of its own: thinking, text, tool calls and structured output all come back as TanStack's standard stream events, so an app can swap this adapter for TanStack's OpenAI or OpenRouter adapter without other changes.
+
+What maps to what:
+
+| TanStack AI                                       | webgllm request                                                          |
+| ------------------------------------------------- | ------------------------------------------------------------------------ |
+| `systemPrompts`, `messages` (tool calls, results, thinking) | `messages` (a failed tool's error as its result)                |
+| `tools` (their `inputSchema`)                     | `tools` (function `parameters`)                                          |
+| `toolChoice` (`auto`, `none`, `required`, a tool) | `tool_choice`; with `none` webgllm offers the model no tools            |
+| `outputSchema`                                    | `response_format: { type: "json_schema" }`                              |
+| `reasoning` level / `budgetTokens`                | `reasoning_effort` (`none` or `high`) / `reasoning.max_tokens`          |
+| `promptCache: "none"`                             | `cache_prompt: false`                                                    |
+| `modelOptions`                                    | see below                                                                |
+
+`modelOptions` are webgllm's own request fields, sent as given, as TanStack's adapters do with their provider's fields: OpenAI's names (`max_completion_tokens`, thinking included; `temperature`, `top_p`, `presence_penalty`, `stop`, `seed`, `parallel_tool_calls`, false keeping only the first tool call) and webgllm's own (`top_k`, `min_p`, `repetition_penalty`, and `repair_tool_calls`, false skipping its tool-call repair). `frequency_penalty` isn't supported: anything but 0 throws webgllm's `unsupported-parameter` error. Their type is webgllm's `ChatCompletionCreateParamsBase`, without the fields the adapter writes from `chat()`'s own options (messages, tools, the output schema, streaming, reasoning).
+
+```ts
+chat({ adapter: webgllmText("LiquidAI/LFM3-1.7B-GGUF"), messages, modelOptions: { max_completion_tokens: 1024, temperature: 0.2 } })
+```
+
+Unset sampling fields take the model card's values (webgllm's), and there's no output cap unless you set one. Events come back as TanStack's Chat Completions adapter writes them: reasoning, text and tool call events, and `RUN_FINISHED` with the finish reason and token usage. Timings aren't in it: measure them from the stream, as for any adapter. A model that fails to load ends the run with a `RUN_ERROR` whose `code` is webgllm's (`download-failed`, `storage-full`, `webgl2-unavailable`); other failures are TanStack's `RUN_ERROR`. Aborting the request stops the generation and ends the stream without one.
+
+## A worker of your own
+
+webgllm starts a worker for each model it loads. To run the model in one you start (bundled your way, or shared with your own code), write the worker with webgllm's handler and give it to the adapter:
+
+```ts
+// worker.ts
+import { WebGLLMWorkerHandler } from "@tanstack-ai-browser/webgllm"
+
+const handler = new WebGLLMWorkerHandler()
+self.onmessage = (event) => handler.onmessage(event)
+```
+
+```ts
+const worker = new Worker(new URL("./worker.ts", import.meta.url), { type: "module" })
+const adapter = webgllmText("LiquidAI/LFM3-1.7B-GGUF", { worker })
+```
+
+`WebGLLMWorkerHandler` is webgllm's own, exported here too. The worker holds one model: adapters given the same worker share it, and loading another model replaces the one there once its running reply ends. `destroy()` frees the model's GPU memory and leaves the worker running. If the worker fails (its script doesn't load, or it crashes), the call waiting on it rejects with webgllm's `worker-failed` error (webgllm 0.3.1 and later). Without a worker, webgllm keeps several models loaded at once, a worker each, up to its memory budget.
+
+## Thinking and structured output
+
+`chat({ outputSchema })` sends the schema as given in `response_format: json_schema`, as an OpenAI call would; webgllm constrains the answer to the schema while it's written. With `stream: true` it streams as TanStack's standard events, the raw JSON as `TEXT_MESSAGE_CONTENT` deltas and a `CUSTOM` `structured-output.complete` event with the object; `stream: false` resolves to the validated object.
+
+```ts
+const stream = chat({
+  adapter: webgllmText("LiquidAI/LFM3-1.7B-GGUF"),
+  messages,
+  outputSchema: MoveSchema, // a Standard Schema (Zod, …) or JSON Schema
+  stream: true
+})
+
+for await (const event of stream) {
+  if (event.type === "REASONING_MESSAGE_CONTENT") showThinking(event.delta)
+  if (event.type === "TEXT_MESSAGE_CONTENT") showPartialJson(event.delta)
+  if (event.type === "CUSTOM" && event.name === "structured-output.complete") play(event.value.object)
+}
+```
+
+A failure is a `RUN_ERROR`, as in a chat. Aborting stops the generation and ends the stream without one.
+
+## Images
+
+Models with a vision encoder read images: Qwen3.5 2B (its ONNX build since webgllm 0.2.3, its GGUF build with its projector since 0.3.3), and Liquid's vision models, LFM3-VL 2B and LFM2.5-VL 3B, 1.6B and 450M, each in Liquid's GGUF and ONNX builds (webgllm 0.3.3 and later). The adapter's `inputModalities` is `["text", "image"]` for them, from the catalog's `supports.input`; `undefined` for a model the catalog doesn't list. Send image parts in a user message, as data (base64) or URLs: they go to webgllm as OpenAI's `image_url`, with `detail` (`auto` by default) setting how many pixels it reads. webgllm loads the vision encoder only for a request with an image and frees it after, so text chats cost nothing more. An image to a model without one throws webgllm's `unsupported-parameter` error.
+
+## Models and loading
+
+The root has the catalog too, webgllm's own re-exported under the adapter's names: `WEBGLLM_CHAT_MODELS` (webgllm's `WEBGL_TEXT_MODELS`) lists the models webgllm runs and `WEBGLLM_MODEL_META` (its `WEBGL_MODEL_META`) has each one's files, capabilities and sampling. `WEBGLLM_MODEL_REASONING` has each model's reasoning levels in TanStack's shape, for `supportedReasoningLevels()`. Their types are `WebGllmChatModel`, `WebGllmModelMeta` and `WebGllmModelReasoningByName`. LFM3 thinks by default and Qwen3.5 doesn't; `chat({ reasoning })` switches it.
+
+`webgllmText` also takes any Hugging Face repo webgllm can resolve (a supported architecture, in ONNX or GGUF), or the URL of a model's `.gguf` file or `.onnx` graph with its tokenizer files in the same folder: webgllm reads its files and sampling from there. `chat({ reasoning })` then sends `reasoning_effort` `"none"` or `"high"`, which webgllm applies where the model's chat template can switch thinking.
+
+```ts
+import { webgllmText } from "@tanstack-ai-browser/webgllm"
+
+const adapter = webgllmText("LiquidAI/LFM3-1.7B-GGUF", { contextLength: 8192 })
+await adapter.availability() // "unavailable", "downloadable", "downloading" or "available" (loaded)
+await adapter.create({ onDownloadProgress: (progress) => console.log(progress) }) // ahead of time; else the first chat loads it
+await adapter.destroy() // frees its GPU memory
+```
+
+The methods have the Prompt API's values and meanings. `availability()` is `"unavailable"` where the engine can't run (checked once per page: webgllm's check makes a WebGL context), `"downloading"` while this page downloads the model (however the load started: `create` or an adapter's first chat), `"available"` when the model is loaded or every file it loads is in webgllm's storage (`webgllm.models.downloaded()`, which reads the Origin Private File System only), else `"downloadable"`. It makes no request (no `webgllm.models.retrieve()`, which asks Hugging Face for file sizes) and starts no worker. `create({ onDownloadProgress?, signal }?)` reports the download, 0 to 1, then 1 once the model is ready (at once if it already is); it resolves to the adapter, and rejects on failure (fine to leave unawaited). Aborting `signal` rejects that call at once with an `AbortError` and stops its progress reports; when no other caller (another call, or a chat waiting for the model) still waits on the load, webgllm cancels it and the model is left unloaded (`availability()` is back to `"downloadable"`). The config's `onDownloadProgress` hears a download the first chat starts too. Adapters for the same model share it, so `destroy()` on one unloads it for all of them.
+
+## Memory
+
+The page shares one `WebGLLM` instance, which keeps models loaded while they fit its memory budget and unloads the least recently used past it. See webgllm's README for its budget.
+
+## License
+
+MIT

package/dist/adapter.d.ts

@@ -0,0 +1,150 @@
+/**
+ * A TanStack AI text adapter for webgllm: LFM3, LFM2.5 and Qwen3.5 on
+ * WebGL 2 in a Web Worker, for browsers without WebGPU, reading images
+ * where the model has a vision encoder. webgllm answers
+ * OpenAI's Chat Completions API in the page, so this is TanStack's Chat
+ * Completions adapter with the page's `WebGLLM` instance as its client (or
+ * the one for the adapter's own `worker`):
+ * messages, tools, structured output and the stream (text, thinking, tool
+ * calls) are TanStack's. This file adds what's webgllm's own: loading the
+ * model before a call, and its reasoning levels and thinking budget.
+ */
+import type { AdapterYieldChunk, ModelReasoning, TextOptions, TokenUsage } from "@tanstack/ai";
+import type { StructuredOutputOptions, StructuredOutputResult } from "@tanstack/ai/adapters";
+import { OpenAIBaseChatCompletionsTextAdapter } from "@tanstack/openai-base";
+import type { OpenAIClientLike } from "@tanstack/openai-base";
+import { WebGLLM } from "webgllm";
+import type { ChatCompletionCreateParamsBase } from "webgllm";
+import type { WebGllmChatModel, WebGllmModelReasoningByName } from "./catalog.js";
+/**
+ * Where a model stands in this browser, with the Prompt API's values
+ * (`LanguageModel.availability()`).
+ */
+export type Availability = "unavailable" | "downloadable" | "downloading" | "available";
+/** A download's progress, from 0 to 1. */
+export type DownloadProgressCallback = (progress: number) => void;
+/** The page's `WebGLLM`. Exported for tests. */
+export declare const webgllm: WebGLLM;
+/**
+ * Per-call `modelOptions`: webgllm's own request fields (OpenAI's names, and
+ * its `top_k`, `min_p`, `repetition_penalty` and `repair_tool_calls`), sent
+ * as given. The fields the adapter writes from `chat()`'s own options
+ * (messages, tools, the output schema, streaming, reasoning) are left out.
+ */
+export type WebGllmTextModelOptions = Omit<ChatCompletionCreateParamsBase, "model" | "messages" | "stream" | "stream_options" | "tools" | "tool_choice" | "response_format" | "reasoning_effort" | "reasoning">;
+/** Options for `webgllmText`. */
+export type WebGllmConfig = {
+    /**
+     * Most tokens a chat may hold, prompt and reply together: the model's
+     * whole context length by default, never more. A smaller limit takes less
+     * memory and refuses longer chats sooner. Changing it loads the model again.
+     */
+    contextLength?: number;
+    /**
+     * A Web Worker of your own to run the model in, in place of the ones
+     * webgllm starts (webgllm's `worker` option): a module whose `onmessage`
+     * goes to webgllm's `WebGLLMWorkerHandler` (exported here too). It holds
+     * one model, so adapters given the same worker share it and loading
+     * another model replaces the one there; `destroy()` frees the model and
+     * leaves the worker running.
+     */
+    worker?: Worker;
+    /**
+     * The model's download progress, 0 to 1, when a call (`chat()`, say)
+     * starts its download; then 1 once it's ready. To load it ahead of time
+     * with progress, see `create`.
+     */
+    onDownloadProgress?: DownloadProgressCallback;
+};
+/** A model's reasoning levels: a listed model's, none for a listed one that doesn't reason, else the switch. */
+type ResolveReasoning<TModel extends string> = TModel extends keyof WebGllmModelReasoningByName ? WebGllmModelReasoningByName[TModel] : TModel extends WebGllmChatModel ? never : {
+    levels: "off" | "high";
+    budget: false;
+};
+/** A model the catalog lists, any Hugging Face repo webgllm can resolve, or the URL of a `.gguf` file or `.onnx` graph. */
+export type WebGllmModel = WebGllmChatModel | (string & {});
+type MetadataByModality = {
+    text: unknown;
+    image: unknown;
+    audio: unknown;
+    video: unknown;
+    document: unknown;
+};
+/** A Chat Completions request, as the base adapter writes it. */
+type CreateParams = Parameters<OpenAIClientLike["chat"]["completions"]["create"]>[0];
+export declare class WebGllmTextAdapter<TModel extends WebGllmModel> extends OpenAIBaseChatCompletionsTextAdapter<TModel, WebGllmTextModelOptions, ReadonlyArray<"text" | "image">, MetadataByModality, ReadonlyArray<string>, ResolveReasoning<TModel>> {
+    readonly name = "webgllm";
+    readonly api = "webgllm";
+    /**
+     * What the model reads, from webgllm's catalog: images too for a model
+     * with a vision encoder (Qwen3.5's ONNX build). Not known for a model the
+     * catalog doesn't list (webgllm's `models.retrieve()` says, once loaded).
+     */
+    readonly inputModalities: ReadonlyArray<"text" | "image"> | undefined;
+    private readonly contextLength;
+    private readonly onDownloadProgress;
+    /** The `WebGLLM` the model loads in: the page's, or the adapter's own worker's. */
+    private readonly engine;
+    constructor(config: WebGllmConfig, model: TModel);
+    /**
+     * Where the model stands in this browser, with the Prompt API's values:
+     * "unavailable" where the engine can't run (WebGL 2 with float render
+     * targets, in a worker), "downloading" while this page downloads it,
+     * "available" when it's loaded or its files are in webgllm's storage,
+     * else "downloadable". It reads storage only.
+     */
+    availability(): Promise<Availability>;
+    /**
+     * Loads the model for this adapter's `contextLength` (downloading it the
+     * first time), with the download's progress, 0 to 1, then 1 once it's
+     * ready. Resolves to this adapter; rejects if it can't load (fine to leave
+     * unawaited). `chat()` doesn't need it: its first call loads the model.
+     */
+    create(options?: {
+        onDownloadProgress?: DownloadProgressCallback;
+        signal?: AbortSignal;
+    }): Promise<this>;
+    /** Unloads the model, freeing its GPU memory. Its files stay. Never rejects. */
+    destroy(): Promise<void>;
+    /**
+     * Loads the model for this adapter's context length; webgllm returns at
+     * once when it's loaded for it, and shares a load between its callers.
+     */
+    private load;
+    chatStream(options: TextOptions<WebGllmTextModelOptions>): AsyncIterable<AdapterYieldChunk>;
+    structuredOutputStream(options: StructuredOutputOptions<WebGllmTextModelOptions>): AsyncIterable<AdapterYieldChunk>;
+    structuredOutput(options: StructuredOutputOptions<WebGllmTextModelOptions>): Promise<StructuredOutputResult<unknown>>;
+    /** The request, with `modelOptions` as given, the thinking budget and the prompt cache. */
+    /**
+     * The engine's usage, as TanStack converts it, but with a cached count of 0
+     * kept, which TanStack's conversion leaves out: 0 means the whole prompt was
+     * read, where no count means the engine didn't say.
+     */
+    protected chatCompletionsTokenUsage(usage: {
+        prompt_tokens_details?: {
+            cached_tokens?: number;
+        } | null;
+    } | null | undefined, source?: unknown): TokenUsage | undefined;
+    protected mapOptionsToRequest(options: TextOptions): CreateParams & {
+        stream: true;
+    };
+    protected modelReasoning(model: string): ModelReasoning | undefined;
+    /** webgllm streams thinking as `reasoning_content`. */
+    protected extractReasoning(chunk: unknown): {
+        text: string;
+    } | undefined;
+    /** Schemas as given: webgllm constrains to any JSON Schema, without OpenAI's strict-mode rewrite. */
+    protected makeStructuredOutputCompatible(schema: Record<string, any>): Record<string, any>;
+}
+/**
+ * A TanStack AI text adapter for a webgllm model on WebGL 2, for browsers
+ * without WebGPU (Firefox on Android, for one).
+ *
+ * @example
+ * ```ts
+ * chat({ adapter: webgllmText("LiquidAI/LFM3-1.7B-GGUF"), messages, threadId })
+ * ```
+ */
+export declare function webgllmText<TModel extends WebGllmModel>(model: TModel, config?: WebGllmConfig): WebGllmTextAdapter<TModel>;
+export {};
+//# sourceMappingURL=adapter.d.ts.map

package/dist/adapter.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"adapter.d.ts","sourceRoot":"","sources":["../src/adapter.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG;AACH,OAAO,KAAK,EAAE,iBAAiB,EAAE,cAAc,EAAE,WAAW,EAAE,UAAU,EAAE,MAAM,cAAc,CAAA;AAE9F,OAAO,KAAK,EAAE,uBAAuB,EAAE,sBAAsB,EAAE,MAAM,uBAAuB,CAAA;AAC5F,OAAO,EAAE,oCAAoC,EAAE,MAAM,uBAAuB,CAAA;AAC5E,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,uBAAuB,CAAA;AAC7D,OAAO,EAAE,OAAO,EAAE,MAAM,SAAS,CAAA;AACjC,OAAO,KAAK,EAAE,8BAA8B,EAAgB,MAAM,SAAS,CAAA;AAG3E,OAAO,KAAK,EAAE,gBAAgB,EAAE,2BAA2B,EAAE,MAAM,cAAc,CAAA;AAEjF;;;GAGG;AACH,MAAM,MAAM,YAAY,GAAG,aAAa,GAAG,cAAc,GAAG,aAAa,GAAG,WAAW,CAAA;AAEvF,0CAA0C;AAC1C,MAAM,MAAM,wBAAwB,GAAG,CAAC,QAAQ,EAAE,MAAM,KAAK,IAAI,CAAA;AAoBjE,gDAAgD;AAChD,eAAO,MAAM,OAAO,SAAqB,CAAA;AAqBzC;;;;;GAKG;AACH,MAAM,MAAM,uBAAuB,GAAG,IAAI,CACxC,8BAA8B,EAC5B,OAAO,GACP,UAAU,GACV,QAAQ,GACR,gBAAgB,GAChB,OAAO,GACP,aAAa,GACb,iBAAiB,GACjB,kBAAkB,GAClB,WAAW,CACd,CAAA;AAED,iCAAiC;AACjC,MAAM,MAAM,aAAa,GAAG;IAC1B;;;;OAIG;IACH,aAAa,CAAC,EAAE,MAAM,CAAA;IACtB;;;;;;;OAOG;IACH,MAAM,CAAC,EAAE,MAAM,CAAA;IACf;;;;OAIG;IACH,kBAAkB,CAAC,EAAE,wBAAwB,CAAA;CAC9C,CAAA;AAED,gHAAgH;AAChH,KAAK,gBAAgB,CAAC,MAAM,SAAS,MAAM,IAAI,MAAM,SAAS,MAAM,2BAA2B,GAC3F,2BAA2B,CAAC,MAAM,CAAC,GACnC,MAAM,SAAS,gBAAgB,GAC7B,KAAK,GACL;IAAE,MAAM,EAAE,KAAK,GAAG,MAAM,CAAC;IAAC,MAAM,EAAE,KAAK,CAAA;CAAE,CAAA;AAE/C,2HAA2H;AAC3H,MAAM,MAAM,YAAY,GAAG,gBAAgB,GAAG,CAAC,MAAM,GAAG,EAAE,CAAC,CAAA;AAO3D,KAAK,kBAAkB,GAAG;IAAE,IAAI,EAAE,OAAO,CAAC;IAAC,KAAK,EAAE,OAAO,CAAC;IAAC,KAAK,EAAE,OAAO,CAAC;IAAC,KAAK,EAAE,OAAO,CAAC;IAAC,QAAQ,EAAE,OAAO,CAAA;CAAE,CAAA;AAE9G,iEAAiE;AACjE,KAAK,YAAY,GAAG,UAAU,CAAC,gBAAgB,CAAC,MAAM,CAAC,CAAC,aAAa,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,CAAA;AAKpF,qBAAa,kBAAkB,CAAC,MAAM,SAAS,YAAY,CAAE,SAAQ,oCAAoC,CACvG,MAAM,EACN,uBAAuB,EACvB,aAAa,CAAC,MAAM,GAAG,OAAO,CAAC,EAC/B,kBAAkB,EAClB,aAAa,CAAC,MAAM,CAAC,EACrB,gBAAgB,CAAC,MAAM,CAAC,CACzB;IACC,SAAkB,IAAI,aAAY;IAClC,SAAkB,GAAG,aAAY;IACjC;;;;OAIG;IACH,SAAkB,eAAe,EAAE,aAAa,CAAC,MAAM,GAAG,OAAO,CAAC,GAAG,SAAS,CAAA;IAE9E,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAoB;IAClD,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAAsC;IACzE,mFAAmF;IACnF,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAQ;IAE/B,YAAY,MAAM,EAAE,aAAa,EAAE,KAAK,EAAE,MAAM,EAQ/C;IAED;;;;;;OAMG;IACG,YAAY,IAAI,OAAO,CAAC,YAAY,CAAC,CAM1C;IAED;;;;;OAKG;IACH,MAAM,CAAC,OAAO,GAAE;QAAE,kBAAkB,CAAC,EAAE,wBAAwB,CAAC;QAAC,MAAM,CAAC,EAAE,WAAW,CAAA;KAAO,GAAG,OAAO,CAAC,IAAI,CAAC,CAI3G;IAED,gFAAgF;IAChF,OAAO,IAAI,OAAO,CAAC,IAAI,CAAC,CAEvB;IAED;;;OAGG;YACW,IAAI;IAeF,UAAU,CAAC,OAAO,EAAE,WAAW,CAAC,uBAAuB,CAAC,GAAG,aAAa,CAAC,iBAAiB,CAAC,CAG1G;IAEe,sBAAsB,CAAC,OAAO,EAAE,uBAAuB,CAAC,uBAAuB,CAAC,GAAG,aAAa,CAAC,iBAAiB,CAAC,CAGlI;IAEc,gBAAgB,CAAC,OAAO,EAAE,uBAAuB,CAAC,uBAAuB,CAAC,GAAG,OAAO,CAAC,sBAAsB,CAAC,OAAO,CAAC,CAAC,CAGnI;IAED,2FAA2F;IAC3F;;;;OAIG;IACH,UAAmB,yBAAyB,CAC1C,KAAK,EAAE;QAAE,qBAAqB,CAAC,EAAE;YAAE,aAAa,CAAC,EAAE,MAAM,CAAA;SAAE,GAAG,IAAI,CAAA;KAAE,GAAG,IAAI,GAAG,SAAS,EACvF,MAAM,CAAC,EAAE,OAAO,GACf,UAAU,GAAG,SAAS,CAKxB;IAED,UAAmB,mBAAmB,CAAC,OAAO,EAAE,WAAW,GAAG,YAAY,GAAG;QAAE,MAAM,EAAE,IAAI,CAAA;KAAE,CAQ5F;IAED,UAAmB,cAAc,CAAC,KAAK,EAAE,MAAM,GAC+B,cAAc,GAAG,SAAS,CACvG;IAED,uDAAuD;IACvD,UAAmB,gBAAgB,CAAC,KAAK,EAAE,OAAO;;kBAIjD;IAED,qGAAqG;IACrG,UAAmB,8BAA8B,CAAC,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,uBAE5E;CACF;AAED;;;;;;;;GAQG;AACH,wBAAgB,WAAW,CAAC,MAAM,SAAS,YAAY,EAAE,KAAK,EAAE,MAAM,EAAE,MAAM,CAAC,EAAE,aAAa,GAAG,kBAAkB,CAAC,MAAM,CAAC,CAE1H"}

package/dist/adapter.js

@@ -0,0 +1,173 @@
+import { resolveReasoning } from "@tanstack/ai/adapter-internals";
+import { OpenAIBaseChatCompletionsTextAdapter } from "@tanstack/openai-base";
+import { WebGLLM } from "webgllm";
+import { modelMeta, WEBGLLM_MODEL_META } from "./catalog.js";
+/** A `WebGLLM` (running in `worker`, if given), and the models downloading in it, from its events. */
+function createEngine(worker) {
+    const webgllm = new WebGLLM(worker && { worker });
+    const downloading = new Set();
+    webgllm.addEventListener("progress", ({ model, phase, cached }) => {
+        if (phase === "download" && !cached)
+            downloading.add(model);
+    });
+    for (const type of ["load", "error", "unload"]) {
+        webgllm.addEventListener(type, ({ model }) => downloading.delete(model));
+    }
+    return { webgllm, downloading };
+}
+/** The page's engine, shared by every adapter without a worker of its own. */
+const pageEngine = createEngine();
+/** The page's `WebGLLM`. Exported for tests. */
+export const webgllm = pageEngine.webgllm;
+/** The engines in the callers' own workers: one a worker, as each worker's handler holds one model. */
+const workerEngines = new WeakMap();
+/** An adapter's engine: its own worker's (shared by every adapter given that worker), else the page's. */
+function engineFor(worker) {
+    if (!worker)
+        return pageEngine;
+    let engine = workerEngines.get(worker);
+    if (!engine)
+        workerEngines.set(worker, (engine = createEngine(worker)));
+    return engine;
+}
+let supported;
+/** Whether the engine runs here, checked once: webgllm's check makes a WebGL context. */
+const engineRuns = () => (supported ??= WebGLLM.isSupported());
+/** The download's share of webgllm's load progress (downloading is its first 90%). */
+const downloadShare = ({ progress, loadedBytes, totalBytes }) => totalBytes ? Math.min((loadedBytes ?? 0) / totalBytes, 1) : Math.min(progress / 0.9, 1);
+/** The switch for a model the catalog doesn't list: webgllm's `reasoning_effort`, a hint it ignores where it can't apply. */
+const UNLISTED_REASONING = { levels: { off: "none", high: "high" }, budget: false };
+const listed = (model) => Object.prototype.hasOwnProperty.call(WEBGLLM_MODEL_META, model);
+const signalOf = (options) => options.abortController?.signal ?? options.request?.signal ?? undefined;
+export class WebGllmTextAdapter extends OpenAIBaseChatCompletionsTextAdapter {
+    name = "webgllm";
+    api = "webgllm";
+    /**
+     * What the model reads, from webgllm's catalog: images too for a model
+     * with a vision encoder (Qwen3.5's ONNX build). Not known for a model the
+     * catalog doesn't list (webgllm's `models.retrieve()` says, once loaded).
+     */
+    inputModalities;
+    contextLength;
+    onDownloadProgress;
+    /** The `WebGLLM` the model loads in: the page's, or the adapter's own worker's. */
+    engine;
+    constructor(config, model) {
+        const engine = engineFor(config.worker);
+        // Chat Completions (`chat.completions.create`) is all the adapter calls.
+        super(model, "webgllm", engine.webgllm, { strictFallbackWarning: false });
+        this.engine = engine;
+        this.contextLength = config.contextLength;
+        this.onDownloadProgress = config.onDownloadProgress;
+        this.inputModalities = listed(model) ? modelMeta(model).supports.input : undefined;
+    }
+    /**
+     * Where the model stands in this browser, with the Prompt API's values:
+     * "unavailable" where the engine can't run (WebGL 2 with float render
+     * targets, in a worker), "downloading" while this page downloads it,
+     * "available" when it's loaded or its files are in webgllm's storage,
+     * else "downloadable". It reads storage only.
+     */
+    async availability() {
+        const { webgllm, downloading } = this.engine;
+        if (!engineRuns())
+            return "unavailable";
+        if (downloading.has(this.model))
+            return "downloading";
+        const stored = await webgllm.models.downloaded(this.model).catch(() => false);
+        return stored ? "available" : "downloadable";
+    }
+    /**
+     * Loads the model for this adapter's `contextLength` (downloading it the
+     * first time), with the download's progress, 0 to 1, then 1 once it's
+     * ready. Resolves to this adapter; rejects if it can't load (fine to leave
+     * unawaited). `chat()` doesn't need it: its first call loads the model.
+     */
+    create(options = {}) {
+        const ready = this.load(options.signal, options.onDownloadProgress).then(() => this);
+        ready.catch(() => undefined);
+        return ready;
+    }
+    /** Unloads the model, freeing its GPU memory. Its files stay. Never rejects. */
+    destroy() {
+        return this.engine.webgllm.unloadModel(this.model).catch(() => undefined);
+    }
+    /**
+     * Loads the model for this adapter's context length; webgllm returns at
+     * once when it's loaded for it, and shares a load between its callers.
+     */
+    async load(signal, onDownloadProgress) {
+        const report = (progress) => {
+            this.onDownloadProgress?.(progress);
+            onDownloadProgress?.(progress);
+        };
+        await this.engine.webgllm.loadModel(this.model, {
+            ...(this.contextLength !== undefined && { contextLength: this.contextLength }),
+            ...(signal && { signal }),
+            onProgress: (progress) => {
+                if (progress.phase === "download" && !progress.cached)
+                    report(downloadShare(progress));
+            }
+        });
+        report(1);
+    }
+    async *chatStream(options) {
+        await this.load(signalOf(options));
+        yield* super.chatStream(options);
+    }
+    async *structuredOutputStream(options) {
+        await this.load(signalOf(options.chatOptions));
+        yield* super.structuredOutputStream(options);
+    }
+    async structuredOutput(options) {
+        await this.load(signalOf(options.chatOptions));
+        return super.structuredOutput(options);
+    }
+    /** The request, with `modelOptions` as given, the thinking budget and the prompt cache. */
+    /**
+     * The engine's usage, as TanStack converts it, but with a cached count of 0
+     * kept, which TanStack's conversion leaves out: 0 means the whole prompt was
+     * read, where no count means the engine didn't say.
+     */
+    chatCompletionsTokenUsage(usage, source) {
+        const result = super.chatCompletionsTokenUsage(usage, source);
+        const cachedTokens = usage?.prompt_tokens_details?.cached_tokens;
+        if (!result || cachedTokens !== 0)
+            return result;
+        return { ...result, promptTokensDetails: { ...result.promptTokensDetails, cachedTokens } };
+    }
+    mapOptionsToRequest(options) {
+        const request = super.mapOptionsToRequest(options);
+        const resolved = resolveReasoning(options.reasoning, this.modelReasoning(this.model));
+        const budget = resolved?.value === "high" ? resolved.budgetTokens : undefined;
+        return Object.assign(request, {
+            ...(budget !== undefined && { reasoning: { max_tokens: budget } }),
+            ...(options.promptCache?.retention === "none" && { cache_prompt: false })
+        });
+    }
+    modelReasoning(model) {
+        return (listed(model) ? modelMeta(model).reasoning : UNLISTED_REASONING);
+    }
+    /** webgllm streams thinking as `reasoning_content`. */
+    extractReasoning(chunk) {
+        const text = chunk.choices?.[0]?.delta
+            ?.reasoning_content;
+        return text ? { text } : undefined;
+    }
+    /** Schemas as given: webgllm constrains to any JSON Schema, without OpenAI's strict-mode rewrite. */
+    makeStructuredOutputCompatible(schema) {
+        return schema;
+    }
+}
+/**
+ * A TanStack AI text adapter for a webgllm model on WebGL 2, for browsers
+ * without WebGPU (Firefox on Android, for one).
+ *
+ * @example
+ * ```ts
+ * chat({ adapter: webgllmText("LiquidAI/LFM3-1.7B-GGUF"), messages, threadId })
+ * ```
+ */
+export function webgllmText(model, config) {
+    return new WebGllmTextAdapter(config ?? {}, model);
+}

package/dist/catalog.d.ts

@@ -0,0 +1,25 @@
+/**
+ * The models webgllm runs: webgllm's own catalog, re-exported under this
+ * adapter's names, with their reasoning levels in TanStack AI's shape (for
+ * `chat({ reasoning })` and `supportedReasoningLevels()`).
+ */
+import type { ModelReasoning } from "@tanstack/ai";
+import type { ModelReasoningCapability } from "@tanstack/ai/adapter-internals";
+import type { WebGlModelMeta, WebGlModelReasoning, WebGlTextModel } from "webgllm";
+export { WEBGL_TEXT_MODELS as WEBGLLM_CHAT_MODELS, WEBGL_MODEL_META as WEBGLLM_MODEL_META } from "webgllm";
+export type { WebGlTextModel as WebGllmChatModel, WebGlModelMeta as WebGllmModelMeta } from "webgllm";
+type WebGllmChatModel = WebGlTextModel;
+/**
+ * The reasoning levels of each model that thinks, exactly as its catalog
+ * entry gives them: LFM3 and Qwen3.5 switch between `off` and `high`, the
+ * LFM2.5 Thinking builds always think (`high` only). Models that don't think
+ * aren't in it.
+ */
+export type WebGllmModelReasoningByName = {
+    [K in WebGllmChatModel as WebGlModelReasoning<K> extends undefined ? never : K]: ModelReasoningCapability<NonNullable<WebGlModelReasoning<K>>>;
+};
+/** Reasoning data of each model (`false`: it doesn't think), for `supportedReasoningLevels()`. */
+export declare const WEBGLLM_MODEL_REASONING: Readonly<Record<WebGllmChatModel, ModelReasoning>>;
+/** A model's catalog entry. */
+export declare const modelMeta: (model: WebGllmChatModel) => WebGlModelMeta;
+//# sourceMappingURL=catalog.d.ts.map

package/dist/catalog.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"catalog.d.ts","sourceRoot":"","sources":["../src/catalog.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AACH,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,cAAc,CAAA;AAClD,OAAO,KAAK,EAAE,wBAAwB,EAAE,MAAM,gCAAgC,CAAA;AAE9E,OAAO,KAAK,EAAE,cAAc,EAAE,mBAAmB,EAAE,cAAc,EAAE,MAAM,SAAS,CAAA;AAGlF,OAAO,EAAE,iBAAiB,IAAI,mBAAmB,EAAE,gBAAgB,IAAI,kBAAkB,EAAE,MAAM,SAAS,CAAA;AAC1G,YAAY,EAAE,cAAc,IAAI,gBAAgB,EAAE,cAAc,IAAI,gBAAgB,EAAE,MAAM,SAAS,CAAA;AAErG,KAAK,gBAAgB,GAAG,cAAc,CAAA;AAEtC;;;;;GAKG;AACH,MAAM,MAAM,2BAA2B,GAAG;KACvC,CAAC,IAAI,gBAAgB,IAAI,mBAAmB,CAAC,CAAC,CAAC,SAAS,SAAS,GAAG,KAAK,GAAG,CAAC,GAAG,wBAAwB,CACvG,WAAW,CAAC,mBAAmB,CAAC,CAAC,CAAC,CAAC,CACpC;CACF,CAAA;AAED,kGAAkG;AAClG,eAAO,MAAM,uBAAuB,EAE/B,QAAQ,CAAC,MAAM,CAAC,gBAAgB,EAAE,cAAc,CAAC,CAAC,CAAA;AAEvD,+BAA+B;AAC/B,eAAO,MAAM,SAAS,UAAW,gBAAgB,KAAG,cAAyC,CAAA"}

package/dist/catalog.js

@@ -0,0 +1,7 @@
+import { WEBGL_MODEL_META, WEBGL_TEXT_MODELS } from "webgllm";
+// webgllm's catalog as it is: the models it runs, and each one's entry by name.
+export { WEBGL_TEXT_MODELS as WEBGLLM_CHAT_MODELS, WEBGL_MODEL_META as WEBGLLM_MODEL_META } from "webgllm";
+/** Reasoning data of each model (`false`: it doesn't think), for `supportedReasoningLevels()`. */
+export const WEBGLLM_MODEL_REASONING = Object.fromEntries(WEBGL_TEXT_MODELS.map((model) => [model, WEBGL_MODEL_META[model].reasoning ?? false]));
+/** A model's catalog entry. */
+export const modelMeta = (model) => WEBGL_MODEL_META[model];

package/dist/index.d.ts

@@ -0,0 +1,11 @@
+/**
+ * TanStack AI adapter for webgllm: LFM3, LFM2.5 and Qwen3.5 in the browser
+ * on WebGL 2, for browsers without WebGPU (Firefox on Android, for one).
+ * The model catalog (model-meta.ts) is exported here too.
+ */
+export { webgllmText, WebGllmTextAdapter } from "./adapter.js";
+export type { Availability, DownloadProgressCallback, WebGllmConfig, WebGllmModel, WebGllmTextModelOptions } from "./adapter.js";
+export { WebGLLMWorkerHandler } from "webgllm";
+export { WEBGLLM_CHAT_MODELS, WEBGLLM_MODEL_META, WEBGLLM_MODEL_REASONING } from "./catalog.js";
+export type { WebGllmChatModel, WebGllmModelMeta, WebGllmModelReasoningByName } from "./catalog.js";
+//# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AACH,OAAO,EAAE,WAAW,EAAE,kBAAkB,EAAE,MAAM,cAAc,CAAA;AAC9D,YAAY,EAAE,YAAY,EAAE,wBAAwB,EAAE,aAAa,EAAE,YAAY,EAAE,uBAAuB,EAAE,MAAM,cAAc,CAAA;AAEhI,OAAO,EAAE,oBAAoB,EAAE,MAAM,SAAS,CAAA;AAC9C,OAAO,EAAE,mBAAmB,EAAE,kBAAkB,EAAE,uBAAuB,EAAE,MAAM,cAAc,CAAA;AAC/F,YAAY,EAAE,gBAAgB,EAAE,gBAAgB,EAAE,2BAA2B,EAAE,MAAM,cAAc,CAAA"}

package/dist/index.js

@@ -0,0 +1,9 @@
+/**
+ * TanStack AI adapter for webgllm: LFM3, LFM2.5 and Qwen3.5 in the browser
+ * on WebGL 2, for browsers without WebGPU (Firefox on Android, for one).
+ * The model catalog (model-meta.ts) is exported here too.
+ */
+export { webgllmText, WebGllmTextAdapter } from "./adapter.js";
+// webgllm's own worker handler, for a worker of your own (`worker` in the adapter's config).
+export { WebGLLMWorkerHandler } from "webgllm";
+export { WEBGLLM_CHAT_MODELS, WEBGLLM_MODEL_META, WEBGLLM_MODEL_REASONING } from "./catalog.js";

package/dist/model-memory.d.ts

@@ -0,0 +1,46 @@
+/**
+ * The memory the page's models take, shared by this project's runtimes
+ * (Transformers.js, WebGL, wllama), each of which loads models in its own
+ * worker. Before a runtime loads a model it claims the memory: other
+ * runtimes' least recently used models are unloaded until everything fits
+ * the budget. A phone can't hold two 1 GB models (an iPhone Safari tab has
+ * 2 GB), and small models still stay loaded side by side.
+ *
+ * The packages each carry this file and share one registry through
+ * globalThis, so they find each other even when bundled apart. Keep the
+ * copies identical (packages/{transformers-js,webgl,wllama}/src).
+ */
+/**
+ * The default budget: 1.5 GB on iOS (a Safari tab gets 2 GB, the page and
+ * Safari take the rest), half the device's memory where the browser says
+ * (Chrome: up to 8 GB), else 4 GB.
+ */
+export declare function defaultMemoryBudget(): number;
+/**
+ * Sets how many bytes the page's models may take together, across this
+ * project's runtimes, from the next load on; undefined for the default.
+ */
+export declare function setMemoryBudget(bytes: number | undefined): void;
+/**
+ * Makes room for `owner`'s model of about `bytes`, unloading other runtimes'
+ * models, least recently used first, until it fits the budget (models kept
+ * loaded stay), then records it in place of `owner`'s previous one.
+ * Resolves to false, recording nothing, if `signal` aborted meanwhile.
+ */
+export declare function claimMemory(model: {
+    owner: string;
+    model: string;
+    bytes: number;
+    keepLoaded: boolean;
+    unload: () => unknown;
+    /**
+     * The load the claim is for: once it's aborted (canceled or replaced),
+     * nothing more is unloaded for it and nothing is recorded.
+     */
+    signal?: AbortSignal;
+}): Promise<boolean>;
+/** Marks `owner`'s model as just used (for least recently used order). */
+export declare function touchMemory(owner: string): void;
+/** Forgets `owner`'s model, once it's unloaded or failed to load. */
+export declare function releaseMemory(owner: string): void;
+//# sourceMappingURL=model-memory.d.ts.map

package/dist/model-memory.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"model-memory.d.ts","sourceRoot":"","sources":["../src/model-memory.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AA6BH;;;;GAIG;AACH,wBAAgB,mBAAmB,IAAI,MAAM,CAS5C;AAED;;;GAGG;AACH,wBAAgB,eAAe,CAAC,KAAK,EAAE,MAAM,GAAG,SAAS,QAExD;AAED;;;;;GAKG;AACH,wBAAsB,WAAW,CAAC,KAAK,EAAE;IACvC,KAAK,EAAE,MAAM,CAAA;IACb,KAAK,EAAE,MAAM,CAAA;IACb,KAAK,EAAE,MAAM,CAAA;IACb,UAAU,EAAE,OAAO,CAAA;IACnB,MAAM,EAAE,MAAM,OAAO,CAAA;IACrB;;;OAGG;IACH,MAAM,CAAC,EAAE,WAAW,CAAA;CACrB,GAAG,OAAO,CAAC,OAAO,CAAC,CAkBnB;AAED,0EAA0E;AAC1E,wBAAgB,WAAW,CAAC,KAAK,EAAE,MAAM,QAGxC;AAED,qEAAqE;AACrE,wBAAgB,aAAa,CAAC,KAAK,EAAE,MAAM,QAE1C"}