@tryhamster/gerbil 1.1.2 → 1.2.0

package/docs/mobile.md

@@ -1,99 +0,0 @@
-# Mobile (iOS / Android / Safari)
-
-Gerbil runs entirely on-device via WebGPU, so it works on modern phones and
-tablets — but mobile browsers impose two hard ceilings that desktop doesn't.
-Understanding them (and one setup step for your users) is the difference between
-a great mobile experience and a frustrating one.
-
-## The two ceilings
-
-On-device models are large — the default Qwen3.5-0.8B 4-bit is ~404 MB; vision and
-larger models are GBs. Mobile browsers, **iOS Safari especially**, wall a web
-origin off from the device with two independent limits:
-
-| Ceiling | What it limits | iOS Safari (uninstalled) | The fix |
-|---|---|---|---|
-| **Storage quota** | Disk for the model cache | ~1 GB, **evictable**, regardless of free disk | Install to Home Screen → persistent storage |
-| **Tab memory** | RAM during load + inference | ~hundreds of MB | Smaller model / fewer dispatches |
-
-A device with 40 GB free disk still only grants an *uninstalled* Safari tab ~1 GB,
-and that 1 GB can be evicted between visits — so without the step below, a large
-model **re-downloads every visit**.
-
-## Recommend "Add to Home Screen" (PWA) for the best mobile UX
-
-The storage ceiling lifts when your site is **installed to the Home Screen** as a
-PWA. Installed, iOS grants **persistent storage** tied to real disk (tens of GB,
-never evicted), so models download **once** and stay cached forever. This is the
-single biggest mobile UX win you can offer your users.
-
-iOS Safari has **no programmatic install prompt** — installation is manual (Share →
-Add to Home Screen) — so the right pattern is to *detect* the situation and nudge
-the user before a large download.
-
-## The storage helpers
-
-`@tryhamster/gerbil/browser` exposes helpers so you can surface this to users:
-
-```ts
-import {
-  getStorageStatus,
-  canCacheModel,
-  requestPersistentStorage,
-  getInstallGuidance,
-  isStandalone,
-} from "@tryhamster/gerbil/browser";
-
-// Before downloading a large model on mobile, check whether it will cache.
-const fit = await canCacheModel(404); // model size in MB
-if (fit.recommendInstall) {
-  // Not installed (or won't fit the current quota) — nudge the user.
-  const { manual, steps } = getInstallGuidance();
-  // manual === true on iOS (show the Share → Add to Home Screen steps);
-  // on Android/Chrome you can capture `beforeinstallprompt` for a one-tap button.
-  showBanner(steps);
-}
-
-// Ask for eviction-exempt storage (best-effort; on iOS effectively granted only
-// once installed to the Home Screen).
-await requestPersistentStorage();
-
-// Full snapshot for diagnostics / UI.
-const s = await getStorageStatus();
-// { quotaMB, usageMB, availableMB, persisted, installed, ios }
-```
-
-| Helper | Returns |
-|---|---|
-| `getStorageStatus()` | `{ quotaMB, usageMB, availableMB, persisted, installed, ios }` |
-| `canCacheModel(sizeMB)` | `{ fits, availableMB, recommendInstall }` |
-| `requestPersistentStorage()` | `boolean` — is storage persistent now |
-| `getInstallGuidance()` | `{ installed, manual, steps }` — platform-aware install steps |
-| `isStandalone()` / `isIOS()` | `boolean` |
-
-## What Gerbil handles automatically
-
-- **Per-device speed tuning.** On WebKit, each GPU dispatch is submitted and
-  drained separately, so decode is bound by CPU↔GPU round-trips. Gerbil batches
-  dispatches per command buffer and **auto-calibrates the safe batch size per
-  device** (a crash-surviving probe persisted in `localStorage`): tablets/desktop
-  Safari jump to a fast group automatically, phones stay at the safe floor. On a
-  modern iPad this is ~3–4× faster than the unbatched floor. You can override with
-  the `?group=N` URL parameter.
-- **Durable, self-healing model cache.** Weights are cached per-tensor (OPFS where
-  available, else CacheStorage), and superseded cache namespaces are auto-evicted
-  on load so orphaned data can't fill the quota and block new caching.
-- **`dtype: "auto"`** picks INT4 on mobile to keep both download size and memory
-  in budget.
-
-## Practical recommendations
-
-1. Default to a small 4-bit model on mobile (the ~404 MB Qwen3.5-0.8B is a good
-   baseline; avoid GB-scale models unless the user has installed the PWA).
-2. Make your site **installable** (web app manifest + service worker for the app
-   shell — *not* the model; Gerbil caches the model itself) and nudge mobile users
-   to install before a large download using the helpers above.
-3. Call `requestPersistentStorage()` early.
-4. Expect first load to download once; subsequent loads read from cache (instant)
-   — provided the model fits the quota (≈ install on iOS for anything approaching
-   1 GB).

package/docs/observability.md

@@ -1,230 +0,0 @@
-# Production Observability
-
-Gerbil includes built-in support for production observability through telemetry hooks and request queuing.
-
-## Telemetry Hooks
-
-Configure telemetry hooks to integrate with Sentry, DataDog, or any monitoring system:
-
-```typescript
-import { Gerbil } from "@tryhamster/gerbil";
-import * as Sentry from "@sentry/node";
-
-const g = new Gerbil({
-  telemetry: {
-    // Called on any error (model load, generation, etc.)
-    onError: (error, context) => {
-      Sentry.captureException(error, {
-        extra: context,
-        tags: { operation: context.operation },
-      });
-    },
-
-    // Called after successful generation
-    onGenerate: (event) => {
-      console.log(`Generated ${event.result.tokensGenerated} tokens`);
-      // Track in your metrics system
-      metrics.histogram("gerbil.tokens_generated", event.result.tokensGenerated);
-      metrics.histogram("gerbil.tokens_per_second", event.result.tokensPerSecond);
-    },
-
-    // Called after model loading (success or failure)
-    onModelLoad: (event) => {
-      if (event.success) {
-        console.log(`Loaded ${event.modelId} in ${event.loadTimeMs}ms on ${event.device}`);
-      } else {
-        console.error(`Failed to load ${event.modelId}: ${event.error}`);
-      }
-    },
-
-    // Called when requests wait in queue (>100ms)
-    onQueueWait: (waitTimeMs) => {
-      metrics.histogram("gerbil.queue_wait_ms", waitTimeMs);
-    },
-  },
-});
-```
-
-### Telemetry Events
-
-#### `onError(error, context)`
-
-Called whenever an error occurs during Gerbil operations.
-
-```typescript
-type ErrorContext = {
-  operation: "generate" | "load" | "embed" | "speak" | "transcribe" | "json";
-  modelId?: string;
-  extra?: Record<string, unknown>;
-};
-```
-
-#### `onGenerate(event)`
-
-Called after successful text generation.
-
-```typescript
-type GenerateEvent = {
-  modelId: string;
-  result: GenerateResult;
-  cached: boolean;
-  queueTimeMs?: number; // Only if waited >100ms
-};
-```
-
-#### `onModelLoad(event)`
-
-Called after model loading completes (success or failure).
-
-```typescript
-type ModelLoadEvent = {
-  modelId: string;
-  loadTimeMs: number;
-  fromCache: boolean;
-  device: "webgpu" | "cpu" | "wasm";
-  success: boolean;
-  error?: string;
-};
-```
-
-#### `onQueueWait(waitTimeMs)`
-
-Called when a request waits in the queue for more than 100ms. Useful for detecting congestion.
-
-## Request Queue
-
-Gerbil uses a request queue to prevent GPU OOM errors under concurrent load. LLM inference can only run one request at a time on the GPU.
-
-### Default Behavior
-
-- **Concurrency**: 1 (single request at a time)
-- **Timeout**: 5 minutes (300,000ms)
-- Requests are processed in FIFO order
-- Timeout errors are thrown if exceeded
-
-### Custom Configuration
-
-```typescript
-const g = new Gerbil({
-  concurrency: {
-    maxConcurrent: 1,       // Max parallel requests (default: 1)
-    timeout: 300_000,       // Request timeout in ms (default: 5 min)
-  },
-});
-```
-
-### Why Queue?
-
-LLM inference on GPU is:
-1. **Memory-bound**: Models consume most of GPU VRAM
-2. **Non-concurrent**: Running multiple inferences simultaneously causes OOM
-3. **Variable duration**: Generation time depends on output length
-
-The queue ensures:
-- Predictable memory usage
-- No OOM crashes under load
-- Fair request ordering
-
-## Rate Limiting
-
-Gerbil does **not** include rate limiting. This is intentional—rate limiting is best handled at the application layer using middleware specific to your framework:
-
-```typescript
-// Express
-import rateLimit from "express-rate-limit";
-import { gerbil } from "@tryhamster/gerbil/express";
-
-app.use("/ai", rateLimit({ windowMs: 60000, max: 10 }));
-app.use("/ai", gerbil());
-
-// Next.js
-import { Ratelimit } from "@upstash/ratelimit";
-import { Redis } from "@upstash/redis";
-
-const ratelimit = new Ratelimit({
-  redis: Redis.fromEnv(),
-  limiter: Ratelimit.slidingWindow(10, "60s"),
-});
-
-export async function POST(req: Request) {
-  const ip = req.headers.get("x-forwarded-for") ?? "anonymous";
-  const { success } = await ratelimit.limit(ip);
-  if (!success) return Response.json({ error: "Rate limited" }, { status: 429 });
-  
-  // Continue with Gerbil...
-}
-```
-
-## Example: Full Production Setup
-
-```typescript
-import { Gerbil } from "@tryhamster/gerbil";
-import * as Sentry from "@sentry/node";
-
-Sentry.init({ dsn: process.env.SENTRY_DSN });
-
-const g = new Gerbil({
-  model: "qwen3-0.6b",
-  
-  telemetry: {
-    onError: (error, context) => {
-      Sentry.captureException(error, { extra: context });
-    },
-    
-    onGenerate: ({ result, queueTimeMs }) => {
-      // Log slow generations
-      if (result.totalTime > 10000) {
-        console.warn(`Slow generation: ${result.totalTime}ms`);
-      }
-      
-      // Track queue congestion
-      if (queueTimeMs && queueTimeMs > 5000) {
-        Sentry.captureMessage("High queue wait time", {
-          level: "warning",
-          extra: { queueTimeMs },
-        });
-      }
-    },
-    
-    onModelLoad: (event) => {
-      if (!event.success) {
-        Sentry.captureMessage(`Model load failed: ${event.error}`, {
-          level: "error",
-          extra: event,
-        });
-      }
-    },
-  },
-  
-  concurrency: {
-    maxConcurrent: 1,
-    timeout: 120_000, // 2 minute timeout for your use case
-  },
-});
-
-// Preload model on startup
-await g.loadModel();
-console.log("Gerbil ready for production");
-```
-
-## Health Checks
-
-For production deployments, implement a health check endpoint:
-
-```typescript
-// Express
-app.get("/health", async (req, res) => {
-  try {
-    const info = g.getInfo();
-    res.json({
-      status: "ok",
-      model: info.model?.id,
-      device: info.device.backend,
-      ready: info.device.status === "ready",
-    });
-  } catch (error) {
-    res.status(503).json({ status: "error", message: String(error) });
-  }
-});
-```
-

package/docs/onnx-removal-plan.md

@@ -1,339 +0,0 @@
-# ONNX / transformers.js Removal Plan — Going Pure Native
-
-> Status: **PLANNING DOC**. No code is deleted by this document. It is the
-> authoritative inventory + phased plan to hand to a removal workflow.
->
-> Branch: `feat/webgpu-engine-mobile`
-> Goal: remove the legacy ONNX Runtime / transformers.js / puppeteer-Chrome
-> inference stack so Gerbil runs **pure native** on the WebGPU engine
-> (`src/gpu/`), across text (Qwen3.5 / LFM2.5 / Gemma4), vision (Qwen3.5 ViT,
-> Gemma4 vision), embeddings (EmbeddingGemma / Qwen3-Embedding), STT (Moonshine)
-> and TTS (Kani).
-
----
-
-## 0. TL;DR — Headline Findings
-
-- **~10,800 lines of inference-stack code become deletable** once the native
-  paths are wired in and the browser/REPL consumers are migrated. The biggest
-  single chunks: `chrome-backend.ts` (1,836), `worker.ts` (1,056),
-  `tts.ts` (1,031), `worker-entry.ts` (599), `stt.ts` (605),
-  `worker-code.generated.ts` (1.35 MB generated bundle).
-- **3 npm deps drop out:** `puppeteer-core`, `@huggingface/transformers`,
-  `onnxruntime-web`. Likely also `kokoro-js` (ONNX-backed TTS) and possibly
-  `@huggingface/hub` (only if nothing else needs HF tree/download APIs).
-- **The "all modalities are native" premise is only true inside `src/gpu/`.**
-  `WebGPUEngine` (`src/gpu/index.ts`) exposes native `generate`, `encodeImage`,
-  `describeImage`, `embed`, `speak` and `transcribe` (Moonshine). **But the
-  public surfaces — the core Node `Gerbil` class and the browser hooks — do NOT
-  yet route STT / TTS / embeddings to the native engine.** They still call ONNX.
-  **This wiring is the prerequisite for removal**, not an afterthought.
-
-### The 2-3 real decisions the user must make
-
-1. **CPU/WASM fallback (transformers.js wasm) — keep or kill?** Pure-native
-   means *WebGPU required*. Dropping the wasm path removes the only way to run
-   on machines/browsers without WebGPU (older Safari, locked-down enterprise,
-   CI without a GPU). Recommendation: **kill it** for pure-native, but this is a
-   reach tradeoff the user owns. (Detail in §3a.)
-2. **chrome-backend.ts (puppeteer headless Chrome) — remove now?** It was the
-   Node-side WebGPU path before node-dawn/native. It is already legacy. **Yes,
-   safe to remove** — but it has surprisingly wide tentacles in the CLI/REPL
-   (cache listing, process management commands). (Detail in §3b.)
-3. **Browser ONNX worker + hooks — remove once site/REPL stop using them?**
-   The chat worker (`worker.ts` + `worker-entry.ts` + the 1.35 MB
-   `worker-code.generated.ts`) and `use-embedding`/`use-speech` ONNX paths can
-   go once `use-native-engine` covers chat and native embed/speech hooks exist.
-   (Detail in §3c.)
-
----
-
-## 1. Full Inventory — File by File
-
-Legend: **[DELETE]** = remove entirely · **[GUT]** = strip ONNX path, keep file ·
-**[KEEP]** = native, no change · **[REWIRE]** = repoint to native then clean.
-
-### 1a. Core (`src/core/`)
-
-| File | Lines | ONNX/legacy usage | Disposition |
-|---|---:|---|---|
-| `chrome-backend.ts` | 1,836 | `puppeteer-core` headless-Chrome WebGPU backend. `ChromeGPUBackend`, `getChromeCachedModels`, global browser/process management. | **[DELETE]** (after CLI/REPL detached) |
-| `stt.ts` | 605 | `WhisperSTT`: `pipeline("automatic-speech-recognition", "onnx-community/whisper-*")` via `@huggingface/transformers`, ORT wasm CDN. | **[DELETE]** → replace with native Moonshine wrapper |
-| `tts.ts` | 1,031 | `KokoroTTS` (`kokoro-js`, ONNX) + `SupertonicTTS` (`pipeline("text-to-speech")`, ONNX). | **[DELETE]** → replace with native Kani wrapper |
-| `model-compat.ts` | 79 | `patchHFCacheConfig` — patches HF cache so transformers.js loads new model_types (qwen3_5→qwen3). Pure ONNX-loader workaround. | **[DELETE]** (native loader reads safetensors directly, no patch needed) |
-| `gerbil.ts` | 2,686 | The hub. `import { AutoModelForCausalLM, AutoModelForImageTextToText, pipeline, ... } from "@huggingface/transformers"`. `effectiveBackend` switch with `onnx` / `webgpu-native` / chrome branches. `this.model` (ONNX), `this.visionModel` (ONNX), `this.chromeBackend`, `this.embedder` (ONNX pipeline). Native path = `this.webgpuEngine`. `embed()`/`embedBatch()` still ONNX (`pipeline("feature-extraction", "Xenova/all-MiniLM-L6-v2")`). STT/TTS delegate to ONNX `WhisperSTT`/`createTTS`. | **[GUT]** — collapse to native-only; remove all ONNX branches, chrome branch, `pipeline` import, model-compat call |
-
-Key ONNX surfaces inside `gerbil.ts`:
-- Imports (lines ~6-21): `AutoModelForCausalLM`, `AutoModelForImageTextToText`,
-  `pipeline as rawPipeline`, etc.
-- `effectiveBackend` switch (~386-393): three-way `onnx` / `webgpu-native` /
-  auto. After removal this becomes always-native.
-- Branches: `isBrowser && webgpu && onnx` (~440), `!isBrowser && webgpu && onnx`
-  → chrome backend (~467-486), CPU/WASM pipeline fallback (~488+), error
-  fallback to pipeline (~525-530).
-- `loadVisionModel` chrome branch (~600-620).
-- `generate`/`generateStream` dispatch: `if (this.webgpuEngine)` (native) vs
-  `else if (this.chromeBackend)` vs `else this.model.generate` (ONNX) — lines
-  ~1235-1300, ~1424-1510.
-- `embed()` (~1824) / `embedBatch()` (~1849): pure ONNX, **no native branch yet**.
-- STT/TTS methods (~2058-2310): delegate to ONNX `tts`/`stt` objects.
-- Chrome status/memory passthroughs (~1055-1101), getModelRepo ONNX maps
-  (~1026-1041).
-- Cleanup (~2388-2461): disposes webgpuEngine, chromeBackend, tts, stt.
-
-### 1b. Browser (`src/browser/`)
-
-| File | Lines | ONNX/legacy usage | Disposition |
-|---|---:|---|---|
-| `worker.ts` | 1,056 | Chat web-worker host. Builds inline worker that imports `@huggingface/transformers` from CDN, ORT wasm paths, `AutoTokenizer`/`AutoModelForCausalLM`/`AutoModelForImageTextToText`. iOS main-thread inference path. | **[DELETE]** (after chat → `use-native-engine`) |
-| `worker-entry.ts` | 599 | The real worker source compiled to the IIFE. Imports transformers v4, ORT wasm CDN, vision+text ONNX. | **[DELETE]** |
-| `worker-code.generated.ts` | ~1.35 MB | Generated IIFE bundle of `worker-entry.ts` (transformers.js fully inlined). Produced by `scripts/build-worker.mjs`. | **[DELETE]** + delete generator + build step |
-| `use-embedding.ts` | 485 | Worker that imports transformers.js from CDN, `pipeline("feature-extraction", "Xenova/all-MiniLM-L6-v2")`, ORT wasm, iOS single-thread. | **[GUT/REWIRE]** to native embed (EmbeddingGemma via `WebGPUEngine.embed`) |
-| `use-speech.ts` | 689 | TTS hook: Kokoro (`kokoro-js`) + Supertonic (`pipeline("text-to-speech")`). All ONNX. | **[REWIRE]** to native Kani |
-| `use-voice-input.ts` | 1,227 | STT+chat+TTS combo hook. Uses `createGerbilWorker` (ONNX chat worker) + ONNX STT/TTS models (`whisper-tiny.en`, `kokoro-82m`). | **[REWIRE]** to native engine + native STT/TTS |
-| `backend-selector.ts` | 245 | `selectBackend` returning `webgpu`/`wasm`/`cpu`, fallback chains, iOS wasm fallbacks. The wasm/cpu half is ONNX-fallback machinery. | **[GUT]** — drop wasm/cpu fallback (decision §3a) |
-| `use-native-engine.ts` | 217 | **[KEEP]** Already native — "no ONNX Runtime, no transformers.js." This is the template the others migrate to. | **[KEEP]** |
-| `index.ts` | — | Re-exports worker, use-embedding, use-speech, use-voice-input, backend-selector. Notes core Gerbil excluded due to chrome-backend/puppeteer. | **[REWIRE]** exports after deletions |
-
-Other browser files referencing legacy (mostly comments/guards): `preload.ts`,
-`device-guards.ts`, `download.ts`, `webgpu.ts` — touch only as needed (mostly
-ONNX wasm path hints / iOS guards). Low priority.
-
-### 1c. CLI / REPL (`src/cli/`) — see §4 for full audit
-
-| File | Lines | usage | Disposition |
-|---|---:|---|---|
-| `index.ts` | 1,289 | `--backend native|onnx|auto` flag (maps to webgpu-native/onnx). `transcribe`, `tts`, `voice` commands (ONNX STT/TTS). `chrome` admin command (`ChromeGPUBackend.getGlobalBrowserStatus`, `killAllBackends`, kill page, ps/pkill of `gerbil/chrome-cache`). transformers cache dir scanning. | **[REWIRE]** drop onnx/auto backend options → native-only; delete `chrome` command; repoint STT/TTS |
-| `repl/utils.ts` | 738 | `getChromeCachedModels`, `refreshCachedModelSizes`, transformers cache dir, `.onnx` file detection, HF `/tree/main/onnx` fetch picking q4f16/q4 ONNX files, `onnx-community/*-ONNX` model registry. | **[GUT]** remove chrome + ONNX file logic; native models load safetensors |
-| `repl/App.tsx` | 1,340 | `backend` prop ("auto"/"webgpu-native"/"onnx"), device switch webgpu/cpu, ttsModel kokoro / sttModel whisper defaults, transformers.js disposal comment. | **[REWIRE]** native-only |
-| `repl/index.tsx` | — | Chrome cleanup delay comment, backend prop. | **[GUT]** |
-| `repl/views/ModelView.tsx`, `ChatView.tsx`, `FrameworksView.tsx` | — | reference chrome-backend / onnx model ids. | **[GUT]** |
-
-### 1d. Skills / Integrations
-
-| File | usage | Disposition |
-|---|---|---|
-| `skills/builtin/transcribe.ts` (77) | Enum of whisper ONNX model ids; calls `g.transcribe`. | **[REWIRE]** to native STT model ids (Moonshine) |
-| `integrations/langchain.ts` | `GerbilEmbeddings` → `g.embed`/`g.embedBatch`. No direct ONNX, rides on core `embed()`. | **[KEEP]** (auto-native once core `embed()` is native) |
-| `integrations/ai-sdk.ts` | Embedding settings → `g.embed`. | **[KEEP]** |
-| `src/index.ts` | Exports `ChromeGPUBackend`, `getChromeCachedModels`, `WhisperSTT`, `WHISPER_MODELS`, `KokoroTTS`, `TTS_MODELS`, STT/TTS types. | **[REWIRE]** remove chrome + ONNX-class exports; export native equivalents |
-
-### 1e. Native engine (`src/gpu/`) — the target, all KEEP
-
-`index.ts` (`WebGPUEngine`: `generate`, `encodeImage`, `describeImage`, `embed`,
-`speak`, `transcribe`), `moonshine-stt.ts` / `moonshine-executor.ts` (native
-STT), `kani-tts.ts` (native TTS), `vision-executor.ts` / `vision-preprocess.ts`,
-`model-loader.ts` (safetensors), `architectures/*` (qwen2, qwen3_5, qwen3_5_vision,
-lfm2, gemma4, gemma4_vision, gemma3_encoder, moonshine, kani_tts, omnivoice).
-**No changes — this is what everything migrates onto.**
-
-### 1f. Build / deps
-
-- `scripts/build-worker.mjs` — bundles `worker-entry.ts` with transformers
-  inlined → `worker-code.generated.ts`. **[DELETE]** with the worker.
-- `package.json` scripts `build` and `pretypecheck` call `build-worker.mjs`.
-  **[EDIT]** remove that step.
-- `tsdown.config.ts` — `@huggingface/transformers` in `nodeExternals`; browser
-  build "BUNDLES transformers.js for clean DX". **[EDIT]** remove transformers
-  external + the bundling comment/path.
-- `package.json` deps to drop: `puppeteer-core`, `@huggingface/transformers`,
-  `onnxruntime-web`, `kokoro-js`; re-evaluate `@huggingface/hub`.
-
----
-
-## 2. What Each ONNX Usage Did → Native Replacement
-
-| Modality | Old (ONNX) | New (native) | Wiring status |
-|---|---|---|---|
-| **Text gen** | transformers.js `AutoModelForCausalLM` (browser CDN + Node), chrome-backend (Node GPU) | `WebGPUEngine.generate` (qwen2/qwen3_5/lfm2/gemma4) | ✅ Wired in core `gerbil.ts` (`webgpu-native` branch) and browser `use-native-engine.ts`. ONNX text path now **DEAD** for GPU; still the CPU/WASM + chrome fallback. |
-| **Vision (VLM)** | `AutoModelForImageTextToText` (browser + chrome) | `WebGPUEngine.encodeImage` / `describeImage` (qwen3_5_vision, gemma4_vision) | ✅ Native exists. Core vision branch still has chrome fallback (~600-620) and browser worker has ONNX vision. **Partially dead.** |
-| **Embeddings** | `pipeline("feature-extraction", "Xenova/all-MiniLM-L6-v2")` (core + `use-embedding` worker) | `WebGPUEngine.embed` (EmbeddingGemma / Qwen3-Embedding) | ❌ **NOT wired** into core `embed()` or `use-embedding.ts`. Still 100% ONNX. **This is live, must be migrated, not just deleted.** |
-| **STT** | `WhisperSTT` (`pipeline("automatic-speech-recognition")`, onnx-community/whisper-*) | `MoonshineSTT` / `WebGPUEngine.transcribe` | ❌ **NOT wired** into core `transcribe()` (delegates to ONNX `WhisperSTT`). Native Moonshine sits unused in `src/gpu/`. **Live, must migrate.** |
-| **TTS** | Kokoro (`kokoro-js`) + Supertonic (`pipeline("text-to-speech")`) | `KaniTTS` / `WebGPUEngine.speak` | ❌ **NOT wired** into core `speak()` (delegates to ONNX `createTTS`). Native Kani unused outside `src/gpu/`. **Live, must migrate.** |
-
-**Dead vs deliberate-keep summary:**
-- **DEAD now (GPU mode):** ONNX text & vision generation paths in core + browser
-  worker. Safe to delete once consumers stop importing the worker.
-- **LIVE (must migrate first):** core `embed/embedBatch`, core STT (`WhisperSTT`),
-  core TTS (`createTTS`/Kokoro), and their browser hooks. These are the real
-  work — the "pure native" claim is aspirational for these three modalities at
-  the public-API layer.
-- **Deliberate keep:** all of `src/gpu/`; `use-native-engine.ts`; langchain/
-  ai-sdk integrations (ride on native `embed`).
-
----
-
-## 3. Decision Points
-
-### 3a. CPU/WASM fallback (transformers.js wasm) — keep or remove?
-
-- **What it is:** `backend-selector.ts` wasm/cpu chains, the `pipeline()` CPU/WASM
-  branches in `gerbil.ts` (~488, ~525 fallback), iOS single-thread ORT wasm in
-  `use-embedding.ts` / worker. The *only* execution path when WebGPU is absent.
-- **Pure-native cost:** removing it = **WebGPU becomes a hard requirement.**
-  No fallback for: older Safari without WebGPU, headless CI without a GPU
-  adapter, locked-down enterprise browsers, very old Android.
-- **Recommendation:** the user wants pure native → **remove**, but ship a clear
-  capability error ("WebGPU required") rather than a silent failure. Flagged as
-  a **reach tradeoff** — quantify lost reach before committing. If any reach
-  matters, the cheap compromise is to keep `onnxruntime-web` for embeddings only
-  (smallest, most reuse-heavy) and drop everything else; but that keeps one ONNX
-  dep alive, so it half-defeats the goal.
-- **Decision owner: user.**
-
-### 3b. chrome-backend.ts (puppeteer headless Chrome) — safe to remove?
-
-- **What it was:** Node-side WebGPU via headless Chrome (puppeteer-core) before
-  node-dawn/native landed. Already deprecated; the native engine replaced it for
-  Node GPU inference.
-- **Hidden tentacles:** beyond inference it powers CLI/REPL features —
-  `getChromeCachedModels` (cache listing in `repl/utils.ts`, `index.ts`), the
-  `gerbil chrome` admin command (status/kill/ps/pkill of `gerbil/chrome-cache`),
-  and exports in `src/index.ts`. These must be detached first or builds break.
-- **Recommendation:** **YES, remove.** Lowest-risk *inference* removal, but do
-  it as a coordinated phase (detach CLI/REPL/index consumers → delete file →
-  drop `puppeteer-core`). 1,836 lines + a heavy dep gone.
-- **Decision: effectively yes; confirm no external consumer imports
-  `ChromeGPUBackend` from the package.**
-
-### 3c. Browser ONNX worker + hooks — remove once site/REPL stop using them?
-
-- **What it is:** `worker.ts` + `worker-entry.ts` + 1.35 MB generated bundle +
-  ONNX paths in `use-embedding`/`use-speech`/`use-voice-input`.
-- **Blocker:** chat must move to `use-native-engine`; embeddings/STT/TTS hooks
-  need native rewrites. The site migration doc (`docs/gerbil-site-native-
-  migration.md`, `docs/site-update-plan.md`) tracks consumer migration.
-- **Recommendation:** remove **after** native hooks exist and the site/REPL are
-  repointed. Biggest bundle-size win (drops the multi-MB inlined transformers).
-- **Decision: yes, sequenced last among browser work.**
-
----
-
-## 4. REPL / CLI Audit
-
-The REPL/CLI is **partly native already** (it accepts `--backend native` →
-`webgpu-native` and the `webgpuEngine` path handles chat), but is riddled with
-ONNX/chrome assumptions:
-
-- **`cli/index.ts`**: every command exposes `--backend native|onnx|auto`
-  (default `auto`). `auto` + ONNX must go for native-only. The `transcribe`,
-  `tts`, and `voice` commands use ONNX STT/TTS (`whisper-tiny.en`, `kokoro-82m`)
-  — repoint to native Moonshine/Kani. The entire `chrome` admin subcommand
-  (status/kill/ps) is chrome-backend-only — **delete**. Cache listing scans the
-  transformers cache dir — switch to native cache dir.
-- **`repl/utils.ts`**: model registry is ONNX repos (`onnx-community/*-ONNX`,
-  `Phi-3-...-onnx`), cache scanning keys on `.onnx` files and the HF
-  `/tree/main/onnx` API, plus `getChromeCachedModels`. Rewrite registry to
-  native safetensors repos (`Qwen/Qwen3.5-0.8B`, etc. — the `nativeRepoMap` in
-  `gerbil.ts` already lists the mappings), scan for `.safetensors`, drop chrome.
-- **`repl/App.tsx` / `index.tsx` / views**: `backend` prop, cpu/webgpu device
-  switch, kokoro/whisper defaults, chrome cleanup delay. Collapse to native;
-  device switch becomes meaningful only if wasm fallback is kept (§3a).
-
-**Plan:** native-only REPL — remove the `onnx`/`auto` backend options (or make
-`auto` == native), delete the `chrome` command, repoint STT/TTS/embeddings model
-ids to native, rewrite cache/registry helpers to safetensors. If §3a keeps wasm,
-retain the cpu/webgpu switch; otherwise drop it.
-
----
-
-## 5. Phased Removal Plan
-
-Ordering principle: **wire native replacements first, detach consumers, then
-delete; deps drop only after their last importer is gone.** Each phase ends with
-the no-regression suite (§6) green.
-
-### Phase 0 — Safety net (no deletion)
-- Snapshot current behavior: run `pnpm test` (vitest) + the native e2e scripts
-  (`scripts/engine/test-gerbil-e2e.mjs`, `test-vision-e2e.mjs`,
-  `test-embedding-gemma.mjs`, `test-moonshine-transcribe.mjs`,
-  `test-kani-speak.mjs`). Record baseline outputs/tokens.
-- **Risk: none.** Verifies: native paths actually work before we lean on them.
-
-### Phase 1 — Wire native embeddings / STT / TTS into the public API (ADD, don't delete)
-- Core `gerbil.ts`: make `embed`/`embedBatch` route to `WebGPUEngine.embed`;
-  make `transcribe`/`speak` route to native Moonshine/Kani (new thin wrappers
-  replacing `WhisperSTT`/`createTTS`).
-- Add native browser hooks (native embed, native speech) modeled on
-  `use-native-engine.ts`.
-- **Risk: MEDIUM** (behavioral parity — output dims, voices, languages).
-  Verify: embedding cosine-sim sanity, transcribe WER on a fixture, speak audio
-  length/sample-rate. **This phase makes the "pure native" claim true.**
-
-### Phase 2 — Remove chrome-backend (Node puppeteer GPU)
-- Detach: `cli/index.ts` (`chrome` command, cache listing), `repl/utils.ts`
-  (`getChromeCachedModels`/`refreshCachedModelSizes`), `repl` views, `src/index.ts`
-  exports, `gerbil.ts` chrome branches + passthroughs.
-- Delete `src/core/chrome-backend.ts`. Drop `puppeteer-core` from package.json.
-- **Risk: LOW–MEDIUM** (wide but mechanical; CLI admin command removed is a
-  user-visible change). Verify: build, CLI smoke (`gerbil "hi"` native), REPL
-  loads.
-
-### Phase 3 — Migrate browser chat to native, delete ONNX worker
-- Repoint site/REPL chat consumers to `use-native-engine`.
-- Delete `worker.ts`, `worker-entry.ts`, `worker-code.generated.ts`,
-  `scripts/build-worker.mjs`; remove the build/pretypecheck worker step;
-  fix `browser/index.ts` exports.
-- **Risk: MEDIUM** (iOS/Safari chat path was the worker's reason to exist;
-  confirm `use-native-engine` covers WKWebView per `docs/mobile-failure-
-  diagnosis.md`). Verify: browser e2e chat, iPad smoke.
-
-### Phase 4 — Remove ONNX embeddings / STT / TTS source
-- Delete `src/core/stt.ts`, `src/core/tts.ts`, `src/core/model-compat.ts`.
-- Gut/rewire `use-embedding.ts`, `use-speech.ts`, `use-voice-input.ts` to native.
-- Remove `gerbil.ts` ONNX imports (`AutoModel*`, `pipeline`), the `embedder`
-  ONNX field, model-compat call, getModelRepo ONNX maps.
-- Update `src/index.ts` exports (drop `WhisperSTT`/`WHISPER_MODELS`/`KokoroTTS`/
-  `TTS_MODELS` or replace with native).
-- Drop `kokoro-js` from package.json.
-- **Risk: MEDIUM.** Verify: STT/TTS/embedding suites; integration tests
-  (langchain/ai-sdk embeddings).
-
-### Phase 5 — Decide & remove CPU/WASM fallback + final dep purge (gated on §3a)
-- If removing fallback: delete wasm/cpu branches in `gerbil.ts` +
-  `backend-selector.ts` wasm chains; replace with "WebGPU required" error.
-- Remove `@huggingface/transformers` + `onnxruntime-web` from package.json and
-  `tsdown.config.ts` externals/bundling. Re-evaluate `@huggingface/hub`.
-- Native-only REPL cleanup (backend flags, device switch).
-- **Risk: HIGH** (reach loss is permanent; this is the one that needs the user's
-  explicit sign-off). Verify: full suite on a WebGPU-capable machine; confirm
-  graceful error on a no-WebGPU environment.
-
-### Net result
-~10,800 LOC removed; deps removed: `puppeteer-core`, `@huggingface/transformers`,
-`onnxruntime-web`, `kokoro-js` (+ maybe `@huggingface/hub`); the 1.35 MB worker
-bundle and its build step gone; one inference path (native) to maintain.
-
----
-
-## 6. Subagent Execution Flow (for the removal workflow)
-
-Run as a **sequence of agents, one per phase** (phases are dependency-ordered;
-do NOT parallelize across phases — later phases assume earlier deletions). Within
-a phase, independent detach edits can be parallel sub-agents.
-
-Each agent's contract:
-1. **Read** this doc's relevant phase + the target files.
-2. **Make the phase's edits** (wire or delete per the disposition table).
-3. **Verify no regression:** `pnpm build` (TypeScript must pass — catches dangling
-   imports), `pnpm test` (vitest), and the phase-relevant native e2e scripts:
-   - Text: `scripts/engine/test-gerbil-e2e.mjs`
-   - Vision: `scripts/engine/test-vision-e2e.mjs`
-   - Embeddings: `scripts/engine/test-embedding-gemma.mjs`
-   - STT: `scripts/engine/test-moonshine-transcribe.mjs`
-   - TTS: `scripts/engine/test-kani-speak.mjs`
-4. **Grep-gate:** after a delete phase, `grep -rn "<removed symbol>" src/` must be
-   empty (e.g. `ChromeGPUBackend`, `@huggingface/transformers`, `kokoro-js`).
-5. **Report** diff summary + verification results to the orchestrator; STOP on any
-   red so a human can adjudicate (especially Phase 5 reach loss).
-
-Suggested agent lineup:
-- **Agent A (Phase 0):** baseline capture, no edits.
-- **Agent B (Phase 1):** native wiring for embed/STT/TTS — the gating change.
-- **Agent C (Phase 2):** chrome-backend removal + `puppeteer-core` drop.
-- **Agent D (Phase 3):** browser native chat migration + worker deletion.
-- **Agent E (Phase 4):** ONNX STT/TTS/embed source removal + `kokoro-js` drop.
-- **Agent F (Phase 5):** GATED — fallback decision, final dep purge, REPL cleanup.
-
-Orchestrator gates: B must pass before C–E delete the ONNX sources they replace;
-F runs only after explicit user sign-off on §3a.