@inbrowser/model 0.1.1 → 0.4.0

package/AGENTS.md

@@ -2,19 +2,36 @@
 
 ## Purpose
 
-On-device LLM inference. Wraps `@huggingface/transformers` behind a
-narrow `Engine` surface so a local Gemma 4 model is a drop-in
-replacement for a cloud provider when consumed through the adapter
-subpaths.
+The model layer. Two halves:
+
+1. **Contract + cloud providers.** `src/contract.ts` defines the one
+   `ModelClient` contract the whole stack shares (relay + agent both
+   consume it). `src/providers/*` are the cloud providers (Gemini,
+   OpenRouter, Requesty, Anthropic, Ollama, Claude-CLI, Claude-Code), each a
+   factory returning a `ModelClient`. `src/with-retry.ts` decorates one.
+2. **On-device engine.** Wraps `@huggingface/transformers` behind a
+   narrow `Engine` surface (`src/engine.ts`) that streams `EngineEvent`.
+
+The engine is also a `ModelClient`, via `createEngineModelClient`
+(`src/engine-client.ts`; exported from the root + the
+`@inbrowser/model/engine-client` subpath). It wraps an `Engine`,
+widening the engine's `EngineEvent` stream to the contract's
+`ModelEvent`. The old engine→relay/agent adapter subpaths were removed;
+this single wrapper replaces them. (The site's in-browser docs-chat
+toggle that drives a local engine through the agent is a separate,
+still-forthcoming piece — the adapter is the building block it needs.)
 
 ## Layering invariants
 
-- `src/types.ts` is the canonical type surface. Every other file in
-  the package imports types from here.
-- `src/engine.ts` is the only module that holds runtime state.
-- `src/adapters/relay.ts` is the only place that imports from
-  `@inbrowser/relay`. `src/adapters/agent.ts` is the only place that
-  imports from `@inbrowser/agent`. The root barrel must not.
+- `src/contract.ts` is type-only (zero runtime imports) so importing the
+  contract never pulls in the engine or `@huggingface/transformers`.
+- `src/types.ts` is the canonical engine type surface. Engine-side files
+  import engine types from here.
+- `src/engine.ts` is the only module that holds runtime model state.
+- Each `src/providers/<name>.ts` is self-contained: it imports the
+  contract types and emits `ModelEvent`s. Providers do not import the
+  relay or the agent — the dependency points inward (relay/agent depend
+  on this package's contract, never the reverse).
 - `src/worker.ts` returns the same `Engine` shape `createEngine`
   returns. Consumers must not need to know which side of `postMessage`
   the engine lives on.
@@ -35,16 +52,25 @@ Use the precise terms — they show up in types, comments, and PRs:
 
 - Don't add `createGemmaEngine` / `createPhi3Engine` / sugar
   factories. New models are new `ModelPreset` entries.
-- Don't put tool-calling polyfill logic here. It belongs in
-  `@inbrowser/agent` — it's a property of the agent runtime, not the
-  model.
+- Don't put the agent's tool-calling polyfill logic here. The native
+  envelope recognition (`parseToolCalls`) is mechanical and stays; the
+  prompt-engineered polyfill is a strategy and belongs in
+  `@inbrowser/agent`.
 - Don't widen `EngineEvent` with cloud-only concepts (cost,
-  thoughtSignature). Translate at the adapter boundary.
+  thoughtSignature). Translate at the `createEngineModelClient`
+  boundary (`src/engine-client.ts`), not in the engine.
+- Don't re-introduce provider exports into `@inbrowser/relay` — the
+  providers live here now and the relay consumes them as
+  `ModelClientFactory`s.
 - Don't make `@huggingface/transformers` a regular dependency. It's
-  a peer dep; consumers control the version.
+  a peer dep; consumers control the version. (The Claude Code Agent SDK,
+  used only by `claudeCodeModelClient`, is an optional peer dep.)
 
 ## Status
 
-POC. Types + adapter surface + worker frames are stable. The
-`@huggingface/transformers` wiring inside `createEngine` is the
-next slice.
+Contract + cloud providers are the live path: relay and agent both
+consume a `ModelClient` from here. The engine loads and `generate()`
+streams real tokens, and the engine is now a `ModelClient` via
+`createEngineModelClient` (the engine→ModelClient adapter). The next
+slice is the site wiring that drives a local engine through the agent
+end to end (the in-browser docs-chat toggle).

package/README.md

@@ -1,19 +1,96 @@
 # @inbrowser/model
 
-On-device LLM engine. Loads ONNX models in the browser via
-`@huggingface/transformers` + ONNX Runtime Web (WebGPU / WASM), and
-exposes them behind a narrow `Engine` surface.
+The model layer for the stack. It owns the one model-call contract —
+`ModelClient` — plus the cloud providers that implement it and the
+on-device LLM engine. `@inbrowser/relay` (transport) and
+`@inbrowser/agent` (runtime) both consume a `ModelClient`, so this is the
+single shared definition of "an LLM" for everything downstream.
 
-> **Status: POC stub.** Types, presets, adapter surface, and worker
-> RPC frames are in place. The `@huggingface/transformers` wiring
-> inside `createEngine` is not yet implemented — `generate()` yields
-> an `error` event today. See `src/engine.ts`.
+Two halves, one package:
 
-## One-liner
+- **The contract + cloud providers.** `@inbrowser/model`
+  defines `ModelClient` / `ModelRequest` / `ModelEvent`. The cloud
+  providers (`geminiModelClient`, `openrouterModelClient`,
+  `requestyModelClient`, `anthropicModelClient`, `openaiCompatModelClient`,
+  `ollamaModelClient`, `llamaServerModelClient`, `claudeCliModelClient`,
+  `claudeCodeModelClient`) are factories that each return a `ModelClient`.
+  `withRetry` decorates one.
+- **The on-device engine.** `createEngine` loads ONNX models in the
+  browser via `@huggingface/transformers` + ONNX Runtime Web (WebGPU /
+  WASM) and exposes them behind a narrow `Engine` surface that streams
+  `EngineEvent`s.
+
+> **Status.** Contract + cloud providers are the live integration path:
+> relay and agent both consume a `ModelClient`. `createEngine` loads a
+> model through `@huggingface/transformers` and `generate()` streams real
+> tokens (the end-to-end load path runs in `examples/local-llm-poc`,
+> headless-verified). The engine is now a `ModelClient` too, via
+> `createEngineModelClient` (root),
+> which widens the engine's `EngineEvent` stream to the contract's
+> `ModelEvent`. The old `@inbrowser/model/relay` and
+> `@inbrowser/model/agent` adapter subpaths have been removed.
+> Known gaps: `GenerateOpts.stop` sequences are accepted but not yet
+> enforced, and the site's in-browser docs-chat path that drives a local
+> engine through the agent is still forthcoming (the adapter exists; the
+> site toggle does not).
+
+## A cloud model as a `ModelClient`
+
+```ts
+import { geminiModelClient } from '@inbrowser/model';
+
+const client = geminiModelClient({ apiKey: process.env.GEMINI_KEY, model: 'gemini-3.5-flash' });
+
+for await (const evt of client.chat(
+  {
+    messages: [{ role: 'user', text: 'Explain WebGPU in one paragraph.' }],
+    tools: [],
+    toolUseEnabled: false,
+  },
+  new AbortController().signal,
+)) {
+  if (evt.kind === 'text') process.stdout.write(evt.text);
+  else if (evt.kind === 'usage') console.error(evt.usage);
+}
+```
+
+The turn ends when the iterable returns; a `usage` event (or a terminal
+`error` event) is the last thing emitted. There is no `turn_complete`
+event.
+
+## A local OpenAI-compatible server
+
+Ollama, llama.cpp's `llama-server`, vLLM, LM Studio, LocalAI, and friends all
+expose the same OpenAI `POST /v1/chat/completions` wire shape. One generic
+factory talks to any of them; two named presets carry the right defaults for
+the common local servers:
+
+```ts
+import {
+  openaiCompatModelClient, // any OAI server — set baseUrl (or endpoint)
+  ollamaModelClient,       // preset: defaults to http://localhost:11434, no auth
+  llamaServerModelClient,  // preset: defaults to http://localhost:8080
+} from '@inbrowser/model';
+
+// Generic: point at any OAI-compatible server. `apiKey` becomes a Bearer token.
+const vllm = openaiCompatModelClient({ baseUrl: 'http://gpu.local:8000', model: 'qwen2.5' });
+
+// llama.cpp llama-server. `--api-key` is optional; pass it as `apiKey`.
+const llama = llamaServerModelClient({ model: 'qwen2.5-coder', apiKey: process.env.LLAMA_KEY });
+```
+
+> **Tool calling on `llama-server` needs `--jinja`.** The server only honors the
+> OpenAI `tools` array when launched with `--jinja` (so it applies a tool-aware
+> chat template); without it, tool calls never stream back. Auth is off unless
+> you start it with `--api-key KEY`.
+
+The presets delegate to `openaiCompatModelClient`; reach for the generic factory
+directly for any server without a named preset.
+
+## An on-device model via the engine
 
 ```ts
-import { createEngine } from '@inbrowser/model';
-import { gemma4_E2B } from '@inbrowser/model/presets';
+import { createEngine, gemma4_E2B } from '@inbrowser/model';
 
 const engine = createEngine(gemma4_E2B);
 await engine.ensureReady();
@@ -25,17 +102,48 @@ for await (const evt of engine.generate([
 }
 ```
 
+The engine speaks `EngineEvent` (`token` / `thinking` / `tool_call` /
+`usage` / `error`), not `ModelEvent`. To use it as a `ModelClient` —
+e.g. to hand it to the agent — wrap it with `createEngineModelClient`:
+
+```ts
+import { createEngine, createEngineModelClient, smollm2_360m } from '@inbrowser/model';
+
+const engine = createEngine(smollm2_360m);
+const client = createEngineModelClient(engine); // a ModelClient
+
+for await (const evt of client.chat(
+  { messages: [{ role: 'user', text: 'Hello' }], tools: [], toolUseEnabled: false },
+  new AbortController().signal,
+)) {
+  if (evt.kind === 'text') process.stdout.write(evt.text);
+}
+```
+
+The adapter maps `token` → `text`, folds the engine's terminal `usage`
+into a `ModelEvent` `usage`, passes `tool_call`s through (no signature),
+and drops the engine-only extras (`decodeMs`, `recoverable`). Wiring a
+local model into the docs-chat site through the agent is forthcoming;
+the `createEngineModelClient` building block it needs now exists.
+
 ## Surface
 
+Everything is imported from the package root `@inbrowser/model`.
+
 | Export | What it gives you |
 |---|---|
-| `createEngine(preset)` | Runtime `Engine` — owns load state + decode loop |
+| `ModelClient`, `ModelRequest`, `ModelEvent`, `ModelMessage`, `ModelUsage`, `ToolSpec`, `ReasoningEffort` | The shared contract (type-only) |
+| `geminiModelClient`, `openrouterModelClient`, `requestyModelClient`, `anthropicModelClient`, `openaiCompatModelClient`, `ollamaModelClient`, `llamaServerModelClient`, `claudeCliModelClient`, `claudeCodeModelClient` | Cloud + local provider factories; each returns a `ModelClient` |
+| `OpenAiCompatConfig`, `OllamaConfig`, `LlamaServerConfig` | Config shapes for the OpenAI-compatible factory and its local presets |
+| `withRetry(client, opts?)` | Decorator that retries transient upstream errors while nothing has streamed |
+| `CloudProviderConfig`, `ModelClientFactory` | Shared provider config + the factory type the relay routes on |
+| `createEngine(preset)` | Runtime `Engine` — owns load state + decode loop, streams `EngineEvent` |
+| `createEngineModelClient(engine, id?)` | Wraps an `Engine` as a `ModelClient` (maps `EngineEvent` → `ModelEvent`) |
 | `definePreset(p)` | Type-safe identity helper for community presets |
-| `ModelPreset`, `Engine`, `EngineEvent`, … | Public types |
-| `@inbrowser/model/presets` | `gemma4_E2B`, `gemma4_E4B` |
-| `@inbrowser/model/relay` | `createLocalInferenceProvider(engine)` → relay `InferenceProvider` |
-| `@inbrowser/model/agent` | `createLocalLlmClient(engine, id)` → agent `LlmClient` |
-| `@inbrowser/model/worker` | `hostEngineInWorker(self)` + `connectWorkerEngine(opts)` |
+| `parseToolCalls`, `splitThinking` | Stream transformers over an `EngineEvent` stream |
+| `ModelPreset`, `Engine`, `EngineEvent`, … | Public engine types |
+| `gemma4_E2B`, `gemma4_E4B`, `qwen2_5_coder_1_5b`, `qwen3_1_7b`, `deepseek_r1_qwen_1_5b`, `smollm2_360m` | The six bundled presets |
+| `hostEngineInWorker(self)`, `connectWorkerEngine(opts)` | Worker host/connect helpers |
 
 ## Vocabulary anchor
 
@@ -54,10 +162,11 @@ for await (const evt of engine.generate([
 - One factory (`createEngine`), many presets. No `createGemmaEngine`.
 - `capabilities` is on the preset, not the engine — interrogable
   pre-load (`gemma4_E2B.capabilities.contextWindow`).
-- `EngineEvent` is narrower than `InferenceEvent`/`ChatEvent`.
-  Adapters widen.
-- Worker subpath returns the same `Engine` shape; the agent runtime
-  cannot tell whether it holds a direct or remote engine.
+- `EngineEvent` is narrower than the contract's `ModelEvent` (no
+  cost, no `thoughtSignature`). `createEngineModelClient` is the place
+  that widens it — translate at that boundary, not in the engine.
+- Worker subpath returns the same `Engine` shape; a consumer cannot
+  tell whether it holds a direct or remote engine.
 - Tool calling is not native to Gemma 4. The polyfill (prompt-engineered
   tool calling + structured-output parsing) lives in `@inbrowser/agent`,
   not here.

package/dist/contract.d.ts

@@ -0,0 +1,104 @@
+/**
+ * `@inbrowser/model/contract` — the one model-call contract for the stack.
+ *
+ * A `ModelClient` is anything that, given a `ModelRequest`, streams `ModelEvent`s:
+ * the cloud providers, the on-device engine, and any adapter all implement it.
+ * Both `@inbrowser/relay` (transport) and `@inbrowser/agent` (runtime) consume a
+ * `ModelClient`, so this is the single shared LLM contract.
+ *
+ * This module is TYPE-ONLY (zero runtime imports), so importing the contract
+ * never pulls in the on-device engine or `@huggingface/transformers`.
+ */
+/** A turn of the conversation handed to a model. */
+export interface ModelMessage {
+    role: 'system' | 'user' | 'assistant' | 'tool';
+    text?: string;
+    /** Tool calls the assistant made (assistant turns). */
+    toolCalls?: {
+        id: string;
+        name: string;
+        args: unknown;
+        signature?: string;
+    }[];
+    /** The call this message answers (tool-result turns). */
+    toolCallId?: string;
+    /** Tool name (tool-result turns). */
+    name?: string;
+    /** Serialized tool result (tool-result turns). */
+    resultJson?: string;
+}
+/**
+ * Tool declaration in the OAI function-calling shape that modern chat templates
+ * accept directly (Qwen, DeepSeek, Llama 3.2+, etc.). Cloud providers that speak
+ * a different wire shape translate internally.
+ */
+export interface ToolSpec {
+    type: 'function';
+    function: {
+        name: string;
+        description: string;
+        parameters: unknown;
+    };
+}
+export type ReasoningEffort = 'off' | 'low' | 'medium' | 'high';
+/** A single model call. */
+export interface ModelRequest {
+    messages: ModelMessage[];
+    tools: ToolSpec[];
+    /** Whether tool use is enabled this turn (cheaper than checking tools.length). */
+    toolUseEnabled: boolean;
+    temperature?: number;
+    topP?: number;
+    topK?: number;
+    reasoningEffort?: ReasoningEffort;
+}
+/** Token + cost accounting for one turn. */
+export interface ModelUsage {
+    promptTokens: number;
+    outputTokens: number;
+    cachedTokens?: number;
+    /** Reasoning tokens, when a provider reports them. */
+    reasoningTokens?: number;
+    /** Real dollar cost, when a provider reports it (e.g. OpenRouter). */
+    costUsd?: number;
+}
+/**
+ * One streamed item from a model call.
+ *
+ * The turn ends when the async iterable returns. On a normal end a `usage` event
+ * MUST be emitted before the return (it carries the final accounting); there is
+ * no separate terminal event. The exception is `error`, which is itself terminal:
+ * after an `error` event the iterable returns with no `usage` event. Consumers
+ * can therefore rely on exactly one of {a `usage` event, an `error` event} per
+ * turn.
+ */
+export type ModelEvent = {
+    kind: 'text';
+    text: string;
+} | {
+    kind: 'thinking';
+    text: string;
+} | {
+    kind: 'tool_call';
+    id: string;
+    name: string;
+    args: unknown;
+    signature?: string;
+} | {
+    kind: 'usage';
+    usage: ModelUsage;
+} | {
+    kind: 'error';
+    message: string;
+};
+/**
+ * The one model-call contract. Implemented by the cloud providers and the
+ * on-device engine; consumed by the relay (transport) and the agent (runtime).
+ */
+export interface ModelClient {
+    /** Stable id for metrics + provenance, e.g. `gemini:gemini-3.5-flash`. */
+    readonly id: string;
+    readonly supportsTools: boolean;
+    chat(req: ModelRequest, signal: AbortSignal): AsyncIterable<ModelEvent>;
+}
+//# sourceMappingURL=contract.d.ts.map

package/dist/contract.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"contract.d.ts","sourceRoot":"","sources":["../src/contract.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG;AAEH,oDAAoD;AACpD,MAAM,WAAW,YAAY;IAC3B,IAAI,EAAE,QAAQ,GAAG,MAAM,GAAG,WAAW,GAAG,MAAM,CAAC;IAC/C,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,uDAAuD;IACvD,SAAS,CAAC,EAAE;QAAE,EAAE,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,OAAO,CAAC;QAAC,SAAS,CAAC,EAAE,MAAM,CAAA;KAAE,EAAE,CAAC;IAC9E,yDAAyD;IACzD,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,qCAAqC;IACrC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,kDAAkD;IAClD,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB;AAED;;;;GAIG;AACH,MAAM,WAAW,QAAQ;IACvB,IAAI,EAAE,UAAU,CAAC;IACjB,QAAQ,EAAE;QACR,IAAI,EAAE,MAAM,CAAC;QACb,WAAW,EAAE,MAAM,CAAC;QACpB,UAAU,EAAE,OAAO,CAAC;KACrB,CAAC;CACH;AAED,MAAM,MAAM,eAAe,GAAG,KAAK,GAAG,KAAK,GAAG,QAAQ,GAAG,MAAM,CAAC;AAEhE,2BAA2B;AAC3B,MAAM,WAAW,YAAY;IAC3B,QAAQ,EAAE,YAAY,EAAE,CAAC;IACzB,KAAK,EAAE,QAAQ,EAAE,CAAC;IAClB,kFAAkF;IAClF,cAAc,EAAE,OAAO,CAAC;IACxB,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,eAAe,CAAC,EAAE,eAAe,CAAC;CACnC;AAED,4CAA4C;AAC5C,MAAM,WAAW,UAAU;IACzB,YAAY,EAAE,MAAM,CAAC;IACrB,YAAY,EAAE,MAAM,CAAC;IACrB,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,sDAAsD;IACtD,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,sEAAsE;IACtE,OAAO,CAAC,EAAE,MAAM,CAAC;CAClB;AAED;;;;;;;;;GASG;AACH,MAAM,MAAM,UAAU,GAClB;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GAC9B;IAAE,IAAI,EAAE,UAAU,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GAClC;IAAE,IAAI,EAAE,WAAW,CAAC;IAAC,EAAE,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,OAAO,CAAC;IAAC,SAAS,CAAC,EAAE,MAAM,CAAA;CAAE,GAClF;IAAE,IAAI,EAAE,OAAO,CAAC;IAAC,KAAK,EAAE,UAAU,CAAA;CAAE,GACpC;IAAE,IAAI,EAAE,OAAO,CAAC;IAAC,OAAO,EAAE,MAAM,CAAA;CAAE,CAAC;AAEvC;;;GAGG;AACH,MAAM,WAAW,WAAW;IAC1B,0EAA0E;IAC1E,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,aAAa,EAAE,OAAO,CAAC;IAChC,IAAI,CAAC,GAAG,EAAE,YAAY,EAAE,MAAM,EAAE,WAAW,GAAG,aAAa,CAAC,UAAU,CAAC,CAAC;CACzE"}

package/dist/contract.js

@@ -0,0 +1,13 @@
+/**
+ * `@inbrowser/model/contract` — the one model-call contract for the stack.
+ *
+ * A `ModelClient` is anything that, given a `ModelRequest`, streams `ModelEvent`s:
+ * the cloud providers, the on-device engine, and any adapter all implement it.
+ * Both `@inbrowser/relay` (transport) and `@inbrowser/agent` (runtime) consume a
+ * `ModelClient`, so this is the single shared LLM contract.
+ *
+ * This module is TYPE-ONLY (zero runtime imports), so importing the contract
+ * never pulls in the on-device engine or `@huggingface/transformers`.
+ */
+export {};
+//# sourceMappingURL=contract.js.map

package/dist/contract.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"contract.js","sourceRoot":"","sources":["../src/contract.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG"}

package/dist/engine-client.d.ts

@@ -0,0 +1,44 @@
+/**
+ * `createEngineModelClient` — wraps an on-device `Engine` as a `ModelClient`.
+ *
+ * This is the adapter that lets the on-device engine plug into the same
+ * `ModelClient` contract the cloud providers (and the relay + agent) speak.
+ * Without it the engine can only be driven directly via its `EngineEvent`
+ * stream; with it the engine is just another `ModelClient` the agent/relay
+ * can route to.
+ *
+ * The mapping is deliberately lossless in the directions that matter and
+ * drops the cloud-irrelevant engine extras:
+ *
+ *   - `EngineEvent.token`     → `{ kind: 'text', text }`
+ *   - `EngineEvent.thinking`  → `{ kind: 'thinking', text }`
+ *   - `EngineEvent.tool_call` → `{ kind: 'tool_call', id, name, args }`
+ *                               (the engine emits no signature — omitted)
+ *   - `EngineEvent.usage`     → `{ kind: 'usage', usage: { promptTokens,
+ *                               outputTokens } }` (`decodeMs` is dropped)
+ *   - `EngineEvent.error`     → `{ kind: 'error', message }`
+ *                               (`recoverable` is dropped)
+ *
+ * The engine already emits exactly one terminal `usage` (success) or `error`
+ * (failure) before its stream returns, so the contract's "exactly one of
+ * {usage, error} per turn" invariant carries straight through — this adapter
+ * synthesizes nothing.
+ *
+ * This module has runtime imports (it constructs a `ModelClient` at runtime
+ * and imports engine types), so it lives on the engine surface, NOT in the
+ * type-only `./contract` module.
+ */
+import type { ModelClient } from './contract.js';
+import type { Engine } from './types.js';
+/**
+ * Wrap an `Engine` as a `ModelClient`.
+ *
+ * @param engine The on-device engine to drive.
+ * @param id Stable id for metrics + provenance. Defaults to
+ *   `local:${engine.model.modelId}` when the engine exposes a model id,
+ *   else `'local'`. The engine has no preset id of its own — `engine.model`
+ *   is a bare `ModelRef` (HF Hub `modelId`), which is the most stable handle
+ *   available.
+ */
+export declare function createEngineModelClient(engine: Engine, id?: string): ModelClient;
+//# sourceMappingURL=engine-client.d.ts.map

package/dist/engine-client.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"engine-client.d.ts","sourceRoot":"","sources":["../src/engine-client.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA6BG;AAEH,OAAO,KAAK,EAAE,WAAW,EAA0C,MAAM,eAAe,CAAC;AACzF,OAAO,KAAK,EAAE,MAAM,EAAiB,MAAM,YAAY,CAAC;AAExD;;;;;;;;;GASG;AACH,wBAAgB,uBAAuB,CAAC,MAAM,EAAE,MAAM,EAAE,EAAE,CAAC,EAAE,MAAM,GAAG,WAAW,CAkBhF"}

package/dist/engine-client.js

@@ -0,0 +1,136 @@
+/**
+ * `createEngineModelClient` — wraps an on-device `Engine` as a `ModelClient`.
+ *
+ * This is the adapter that lets the on-device engine plug into the same
+ * `ModelClient` contract the cloud providers (and the relay + agent) speak.
+ * Without it the engine can only be driven directly via its `EngineEvent`
+ * stream; with it the engine is just another `ModelClient` the agent/relay
+ * can route to.
+ *
+ * The mapping is deliberately lossless in the directions that matter and
+ * drops the cloud-irrelevant engine extras:
+ *
+ *   - `EngineEvent.token`     → `{ kind: 'text', text }`
+ *   - `EngineEvent.thinking`  → `{ kind: 'thinking', text }`
+ *   - `EngineEvent.tool_call` → `{ kind: 'tool_call', id, name, args }`
+ *                               (the engine emits no signature — omitted)
+ *   - `EngineEvent.usage`     → `{ kind: 'usage', usage: { promptTokens,
+ *                               outputTokens } }` (`decodeMs` is dropped)
+ *   - `EngineEvent.error`     → `{ kind: 'error', message }`
+ *                               (`recoverable` is dropped)
+ *
+ * The engine already emits exactly one terminal `usage` (success) or `error`
+ * (failure) before its stream returns, so the contract's "exactly one of
+ * {usage, error} per turn" invariant carries straight through — this adapter
+ * synthesizes nothing.
+ *
+ * This module has runtime imports (it constructs a `ModelClient` at runtime
+ * and imports engine types), so it lives on the engine surface, NOT in the
+ * type-only `./contract` module.
+ */
+/**
+ * Wrap an `Engine` as a `ModelClient`.
+ *
+ * @param engine The on-device engine to drive.
+ * @param id Stable id for metrics + provenance. Defaults to
+ *   `local:${engine.model.modelId}` when the engine exposes a model id,
+ *   else `'local'`. The engine has no preset id of its own — `engine.model`
+ *   is a bare `ModelRef` (HF Hub `modelId`), which is the most stable handle
+ *   available.
+ */
+export function createEngineModelClient(engine, id) {
+    const resolvedId = id ?? (engine.model?.modelId ? `local:${engine.model.modelId}` : 'local');
+    return {
+        id: resolvedId,
+        supportsTools: engine.capabilities.supportsTools,
+        chat(req, signal) {
+            const engineMessages = toEngineMessages(req.messages);
+            const stream = engine.generate(engineMessages, {
+                tools: req.toolUseEnabled ? req.tools : undefined,
+                temperature: req.temperature,
+                topP: req.topP,
+                topK: req.topK,
+                signal,
+            });
+            return mapEvents(stream);
+        },
+    };
+}
+/**
+ * Flatten the contract's `ModelMessage[]` into the engine's toolless
+ * `EngineMessage[]`. `EngineMessage` has no tool round-trip fields (role is
+ * `system | user | assistant`, plus `text`), so two shapes the engine can't
+ * represent are flattened into plain text it can still read for grounding:
+ *
+ *   - A `role: 'tool'` result becomes a `user` line:
+ *     `Tool ${name} result: ${resultJson}`.
+ *   - An `assistant` turn carrying `toolCalls` keeps its text (if any) and
+ *     appends a `Tool call: ${name}(${args})` line per call, so the call the
+ *     assistant made survives into the prompt rather than being silently lost.
+ *
+ * For the retrieval strategy these are just system/user messages and pass
+ * straight through; the flattening exists so the general case is lossless.
+ */
+function toEngineMessages(messages) {
+    const out = [];
+    for (const m of messages) {
+        if (m.role === 'tool') {
+            const name = m.name ?? 'tool';
+            const body = m.resultJson ?? m.text ?? '';
+            out.push({ role: 'user', text: `Tool ${name} result: ${body}` });
+            continue;
+        }
+        if (m.role === 'assistant' && m.toolCalls && m.toolCalls.length > 0) {
+            const lines = [];
+            if (m.text)
+                lines.push(m.text);
+            for (const call of m.toolCalls) {
+                lines.push(`Tool call: ${call.name}(${stringifyArgs(call.args)})`);
+            }
+            out.push({ role: 'assistant', text: lines.join('\n') });
+            continue;
+        }
+        // system / user / plain assistant → role + text straight through.
+        out.push({ role: m.role, text: m.text ?? '' });
+    }
+    return out;
+}
+function stringifyArgs(args) {
+    if (typeof args === 'string')
+        return args;
+    try {
+        return JSON.stringify(args ?? {});
+    }
+    catch {
+        return String(args);
+    }
+}
+/** Translate the engine's `EngineEvent` stream into `ModelEvent`s. */
+async function* mapEvents(source) {
+    for await (const ev of source) {
+        switch (ev.kind) {
+            case 'token':
+                yield { kind: 'text', text: ev.text };
+                break;
+            case 'thinking':
+                yield { kind: 'thinking', text: ev.text };
+                break;
+            case 'tool_call':
+                // The engine emits no signature; omit it.
+                yield { kind: 'tool_call', id: ev.id, name: ev.name, args: ev.args };
+                break;
+            case 'usage':
+                // Drop `decodeMs` — not part of the cloud `ModelUsage` shape.
+                yield {
+                    kind: 'usage',
+                    usage: { promptTokens: ev.promptTokens, outputTokens: ev.outputTokens },
+                };
+                break;
+            case 'error':
+                // Drop `recoverable` — the contract's error event is terminal + flat.
+                yield { kind: 'error', message: ev.message };
+                break;
+        }
+    }
+}
+//# sourceMappingURL=engine-client.js.map

package/dist/engine-client.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"engine-client.js","sourceRoot":"","sources":["../src/engine-client.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA6BG;AAKH;;;;;;;;;GASG;AACH,MAAM,UAAU,uBAAuB,CAAC,MAAc,EAAE,EAAW;IACjE,MAAM,UAAU,GAAG,EAAE,IAAI,CAAC,MAAM,CAAC,KAAK,EAAE,OAAO,CAAC,CAAC,CAAC,SAAS,MAAM,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC;IAE7F,OAAO;QACL,EAAE,EAAE,UAAU;QACd,aAAa,EAAE,MAAM,CAAC,YAAY,CAAC,aAAa;QAChD,IAAI,CAAC,GAAiB,EAAE,MAAmB;YACzC,MAAM,cAAc,GAAG,gBAAgB,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;YACtD,MAAM,MAAM,GAAG,MAAM,CAAC,QAAQ,CAAC,cAAc,EAAE;gBAC7C,KAAK,EAAE,GAAG,CAAC,cAAc,CAAC,CAAC,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS;gBACjD,WAAW,EAAE,GAAG,CAAC,WAAW;gBAC5B,IAAI,EAAE,GAAG,CAAC,IAAI;gBACd,IAAI,EAAE,GAAG,CAAC,IAAI;gBACd,MAAM;aACP,CAAC,CAAC;YACH,OAAO,SAAS,CAAC,MAAM,CAAC,CAAC;QAC3B,CAAC;KACF,CAAC;AACJ,CAAC;AAED;;;;;;;;;;;;;;GAcG;AACH,SAAS,gBAAgB,CAAC,QAAqC;IAC7D,MAAM,GAAG,GAAoB,EAAE,CAAC;IAChC,KAAK,MAAM,CAAC,IAAI,QAAQ,EAAE,CAAC;QACzB,IAAI,CAAC,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;YACtB,MAAM,IAAI,GAAG,CAAC,CAAC,IAAI,IAAI,MAAM,CAAC;YAC9B,MAAM,IAAI,GAAG,CAAC,CAAC,UAAU,IAAI,CAAC,CAAC,IAAI,IAAI,EAAE,CAAC;YAC1C,GAAG,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,QAAQ,IAAI,YAAY,IAAI,EAAE,EAAE,CAAC,CAAC;YACjE,SAAS;QACX,CAAC;QAED,IAAI,CAAC,CAAC,IAAI,KAAK,WAAW,IAAI,CAAC,CAAC,SAAS,IAAI,CAAC,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YACpE,MAAM,KAAK,GAAa,EAAE,CAAC;YAC3B,IAAI,CAAC,CAAC,IAAI;gBAAE,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;YAC/B,KAAK,MAAM,IAAI,IAAI,CAAC,CAAC,SAAS,EAAE,CAAC;gBAC/B,KAAK,CAAC,IAAI,CAAC,cAAc,IAAI,CAAC,IAAI,IAAI,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;YACrE,CAAC;YACD,GAAG,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,WAAW,EAAE,IAAI,EAAE,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;YACxD,SAAS;QACX,CAAC;QAED,kEAAkE;QAClE,GAAG,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC,IAAI,IAAI,EAAE,EAAE,CAAC,CAAC;IACjD,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,SAAS,aAAa,CAAC,IAAa;IAClC,IAAI,OAAO,IAAI,KAAK,QAAQ;QAAE,OAAO,IAAI,CAAC;IAC1C,IAAI,CAAC;QACH,OAAO,IAAI,CAAC,SAAS,CAAC,IAAI,IAAI,EAAE,CAAC,CAAC;IACpC,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,MAAM,CAAC,IAAI,CAAC,CAAC;IACtB,CAAC;AACH,CAAC;AAED,sEAAsE;AACtE,KAAK,SAAS,CAAC,CAAC,SAAS,CACvB,MAAuD;IAEvD,IAAI,KAAK,EAAE,MAAM,EAAE,IAAI,MAAM,EAAE,CAAC;QAC9B,QAAQ,EAAE,CAAC,IAAI,EAAE,CAAC;YAChB,KAAK,OAAO;gBACV,MAAM,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,EAAE,CAAC,IAAI,EAAE,CAAC;gBACtC,MAAM;YACR,KAAK,UAAU;gBACb,MAAM,EAAE,IAAI,EAAE,UAAU,EAAE,IAAI,EAAE,EAAE,CAAC,IAAI,EAAE,CAAC;gBAC1C,MAAM;YACR,KAAK,WAAW;gBACd,0CAA0C;gBAC1C,MAAM,EAAE,IAAI,EAAE,WAAW,EAAE,EAAE,EAAE,EAAE,CAAC,EAAE,EAAE,IAAI,EAAE,EAAE,CAAC,IAAI,EAAE,IAAI,EAAE,EAAE,CAAC,IAAI,EAAE,CAAC;gBACrE,MAAM;YACR,KAAK,OAAO;gBACV,8DAA8D;gBAC9D,MAAM;oBACJ,IAAI,EAAE,OAAO;oBACb,KAAK,EAAE,EAAE,YAAY,EAAE,EAAE,CAAC,YAAY,EAAE,YAAY,EAAE,EAAE,CAAC,YAAY,EAAE;iBACxE,CAAC;gBACF,MAAM;YACR,KAAK,OAAO;gBACV,sEAAsE;gBACtE,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,OAAO,EAAE,EAAE,CAAC,OAAO,EAAE,CAAC;gBAC7C,MAAM;QACV,CAAC;IACH,CAAC;AACH,CAAC"}

package/dist/engine.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"engine.d.ts","sourceRoot":"","sources":["../src/engine.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;GAyBG;AAaH,OAAO,KAAK,EAEV,gBAAgB,EAChB,MAAM,EAUP,MAAM,YAAY,CAAC;AAEpB,wBAAgB,YAAY,CAAC,IAAI,EAAE,gBAAgB,GAAG,MAAM,CAqS3D;AAED;;;;GAIG;AACH,wBAAgB,YAAY,CAAC,CAAC,SAAS,OAAO,YAAY,EAAE,WAAW,EAAE,CAAC,EAAE,CAAC,GAAG,CAAC,CAEhF"}
+{"version":3,"file":"engine.d.ts","sourceRoot":"","sources":["../src/engine.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;GAyBG;AAeH,OAAO,KAAK,EAEV,gBAAgB,EAChB,MAAM,EAUP,MAAM,YAAY,CAAC;AAEpB,wBAAgB,YAAY,CAAC,IAAI,EAAE,gBAAgB,GAAG,MAAM,CA4S3D;AAED;;;;GAIG;AACH,wBAAgB,YAAY,CAAC,CAAC,SAAS,OAAO,YAAY,EAAE,WAAW,EAAE,CAAC,EAAE,CAAC,GAAG,CAAC,CAEhF"}

package/dist/engine.js

@@ -24,8 +24,15 @@
  * honored — needs a `StoppingCriteria` adapter. Tracked as a
  * follow-up.
  */
-import { AutoModelForCausalLM, AutoTokenizer, TextStreamer, env as transformersEnv, } from '@huggingface/transformers';
 import { parseToolCalls } from './parse-tool-calls.js';
+// Lazy-load the heavy transformers runtime so importing @inbrowser/model (e.g.
+// for a cloud provider) never statically bundles ONNX/WASM. The chunk is
+// fetched only when an engine actually loads or generates.
+let transformersModule = null;
+function loadTransformers() {
+    transformersModule ??= import('@huggingface/transformers');
+    return transformersModule;
+}
 export function createEngine(opts) {
     const model = opts.model;
     const capabilities = opts.capabilities;
@@ -79,16 +86,17 @@ export function createEngine(opts) {
         if (loadPromise)
             return loadPromise;
         setState('loading');
-        // `weightsBaseUrl` overrides the HF Hub origin for self-hosted
-        // mirrors. Transformers.js exposes this as the global
-        // `env.remoteHost`; we set it process-wide before load. Documented
-        // limitation: with multiple engines spanning different remotes,
-        // the last one to load wins. Realistic use case (one app, one
-        // mirror) is unaffected.
-        if (opts.weightsBaseUrl) {
-            transformersEnv.remoteHost = opts.weightsBaseUrl;
-        }
         loadPromise = (async () => {
+            const { AutoTokenizer, AutoModelForCausalLM, env: transformersEnv, } = await loadTransformers();
+            // `weightsBaseUrl` overrides the HF Hub origin for self-hosted
+            // mirrors. Transformers.js exposes this as the global
+            // `env.remoteHost`; we set it process-wide before load. Documented
+            // limitation: with multiple engines spanning different remotes,
+            // the last one to load wins. Realistic use case (one app, one
+            // mirror) is unaffected.
+            if (opts.weightsBaseUrl) {
+                transformersEnv.remoteHost = opts.weightsBaseUrl;
+            }
             // AutoTokenizer (not AutoProcessor): text-only models like
             // SmolLM2 ship no preprocessor_config.json and AutoProcessor
             // 404s on them. Multimodal models (e.g., Gemma 4 audio) still
@@ -200,6 +208,8 @@ export function createEngine(opts) {
         // channels but inconsistent emission (Gemma 4 family — see
         // presets.ts) deliberately omit `thinkingTags` to take this path.
         const preserveSpecialTokens = useThinking && capabilities.thinkingTags !== undefined;
+        // Cached after `ensureReady` above already loaded it — resolves instantly.
+        const { TextStreamer } = await loadTransformers();
         const streamer = new TextStreamer(tokenizer, {
             skip_prompt: true,
             ...(preserveSpecialTokens ? { skip_special_tokens: false } : {}),