@tryhamster/gerbil 1.1.2 → 1.1.3

package/docs/ai-sdk.md

@@ -1,205 +0,0 @@
-# Gerbil + AI SDK
-
-Gerbil works as a [Vercel AI SDK v5](https://sdk.vercel.ai/) provider, supporting text generation, embeddings, speech synthesis (TTS), and transcription (STT).
-
-> **Pre-1.0 note.** The AI SDK provider routes through the `Gerbil` class, which now runs on
-> the native WebGPU engine (no ONNX / transformers.js). TTS uses Kani-TTS-2, STT uses
-> Moonshine, and embeddings use EmbeddingGemma regardless of the model-id string you pass —
-> legacy ids like `kokoro-82m` / `whisper-tiny.en` are vestigial labels and the device must
-> have WebGPU (there is no CPU/WASM fallback). The first-class surface for the engine is
-> `WebGPUEngine` / `useEngine` (see the [README](../README.md), [TTS](./tts.md),
-> [STT](./stt.md), [Embeddings](./embeddings.md) docs).
-
-## Setup
-
-```typescript
-import { generateText, streamText } from "ai";
-import { gerbil } from "@tryhamster/gerbil/ai";
-```
-
-## Text Generation
-
-### Generate Text
-
-```typescript
-const { text } = await generateText({
-  model: gerbil("qwen3-0.6b"),
-  prompt: "Write a commit message for adding dark mode",
-});
-```
-
-### Stream Text
-
-```typescript
-const stream = streamText({
-  model: gerbil("qwen3-0.6b"),
-  prompt: "Explain React hooks",
-});
-
-for await (const chunk of stream.textStream) {
-  process.stdout.write(chunk);
-}
-```
-
-### With System Prompt
-
-```typescript
-const { text } = await generateText({
-  model: gerbil("qwen3-0.6b"),
-  system: "You are a helpful coding assistant.",
-  prompt: "How do I handle errors in async/await?",
-});
-```
-
-### Thinking Mode
-
-```typescript
-import { createGerbil } from "@tryhamster/gerbil/ai";
-
-const local = createGerbil({ device: "gpu" });
-
-const { text } = await generateText({
-  model: local("qwen3-0.6b", { thinking: true }),
-  prompt: "What is 127 × 43?",
-});
-```
-
-## Embeddings
-
-> **Native.** `gerbil.embedding()` runs native EmbeddingGemma-300M on the WebGPU engine
-> (768-dim; the `all-MiniLM-L6-v2` default id is a vestigial label — the old ONNX
-> MiniLM/BGE/GTE lane has been removed). For direct control use `engine.embed()` — see
-> [Embeddings docs](./embeddings.md). Requires WebGPU.
-
-Generate text embeddings for semantic search, similarity, and RAG:
-
-```typescript
-import { embed, embedMany } from "ai";
-import { gerbil } from "@tryhamster/gerbil/ai";
-
-// Single embedding (768-dim EmbeddingGemma vector)
-const { embedding } = await embed({
-  model: gerbil.embedding(),
-  value: "Hello world",
-});
-
-// Multiple embeddings
-const { embeddings } = await embedMany({
-  model: gerbil.embedding(),
-  values: ["Hello", "World", "How are you?"],
-});
-```
-
-## Speech Generation (TTS)
-
-> **Native.** `gerbil.speech()` runs Kani-TTS-2 on the native engine (the `kokoro-82m`
-> default id is a vestigial label). For direct control use `engine.speak()` — see
-> [TTS docs](./tts.md). Requires WebGPU.
-
-Generate speech from text:
-
-```typescript
-import { experimental_generateSpeech as generateSpeech } from "ai";
-import { gerbil } from "@tryhamster/gerbil/ai";
-
-const result = await generateSpeech({
-  model: gerbil.speech(),        // native Kani-TTS-2
-  text: "Hello, welcome to Gerbil!",
-});
-
-// result.audio is the synthesized PCM clip
-await writeFile("output.wav", result.audio);
-```
-
-> Voice/speed selection from the old Kokoro lane no longer applies — Kani-TTS-2 uses its own
-> default voice. For full control over the native speech path, use `engine.speak()` directly
-> (see [TTS docs](./tts.md)).
-
-## Transcription (STT)
-
-> **Native.** `gerbil.transcription()` runs Moonshine on the native engine (the
-> `whisper-tiny.en` default id is a vestigial label; English only). For direct control use
-> `MoonshineSTT` — see [STT docs](./stt.md). Requires WebGPU.
-
-Transcribe audio to text:
-
-```typescript
-import { experimental_transcribe as transcribe } from "ai";
-import { gerbil } from "@tryhamster/gerbil/ai";
-import { readFile } from "fs/promises";
-
-const result = await transcribe({
-  model: gerbil.transcription(),   // native Moonshine (English)
-  audio: await readFile("audio.wav"),
-});
-
-console.log(result.text);              // "Hello world"
-console.log(result.language);          // "en"
-```
-
-> The Whisper model family (multilingual variants, timestamped segments) has been removed.
-> The native path is Moonshine, English-only, and does not produce timestamps. For full
-> control use `MoonshineSTT` directly (see [STT docs](./stt.md)).
-
-## Custom Provider
-
-```typescript
-import { createGerbil } from "@tryhamster/gerbil/ai";
-
-const local = createGerbil({
-  device: "gpu",
-  dtype: "q4",
-});
-
-// Text generation
-const { text } = await generateText({
-  model: local("qwen3-0.6b"),
-  prompt: "Hello",
-});
-
-// Speech
-const speech = await generateSpeech({
-  model: local.speech(),
-  text: "Hello",
-});
-
-// Transcription
-const transcript = await transcribe({
-  model: local.transcription(),
-  audio: audioData,
-});
-```
-
-## Model Preloading
-
-Download models ahead of time via the provider:
-
-```typescript
-import { gerbil } from "@tryhamster/gerbil/ai";
-
-// Check if cached
-if (!(await gerbil.isCached("qwen3-0.6b"))) {
-  // Preload during app init
-  await gerbil.preload("qwen3-0.6b", {
-    onProgress: (p) => console.log(p.status, p.progress),
-  });
-}
-
-// Later: generateText loads instantly from cache
-```
-
-## Specification
-
-Gerbil implements the following AI SDK v5 interfaces:
-
-| Interface | Purpose | Method |
-|-----------|---------|--------|
-| `LanguageModelV2` | Text generation | `gerbil(modelId)` |
-| `SpeechModelV2` | Text-to-Speech | `gerbil.speech()` |
-| `TranscriptionModelV2` | Speech-to-Text | `gerbil.transcription()` |
-
-All models support:
-- `specificationVersion: "v2"`
-- Proper warning reporting
-- Request/response metadata
-

package/docs/architecture/README.md

@@ -1,84 +0,0 @@
-# Gerbil Architecture
-
-Technical deep-dive into how Gerbil works under the hood.
-
-## Contents
-
-| Document | Description |
-|----------|-------------|
-| [Overview](./overview.md) | High-level architecture and design decisions |
-| [Inference Pipeline](./inference.md) | ONNX Runtime, transformers.js, quantization |
-| [WebGPU](./webgpu.md) | GPU acceleration in browser and Node.js |
-| [Streaming](./streaming.md) | Web Worker architecture and token streaming |
-| [Caching](./caching.md) | Model caching strategies |
-
-## Quick Overview
-
-```
-┌───────────────────────────────────────────────────────────────────┐
-│                         Your Application                          │
-├───────────────────────────────────────────────────────────────────┤
-│                                                                   │
-│  ┌─────────────┐    ┌─────────────┐    ┌─────────────────────┐    │
-│  │   Browser   │    │   Node.js   │    │      CLI/REPL       │    │
-│  │  (WebGPU)   │    │ (CPU/WebGPU)│    │    (CPU/WebGPU)     │    │
-│  └──────┬──────┘    └──────┬──────┘    └──────────┬──────────┘    │
-│         │                  │                       │              │
-│         ▼                  ▼                       ▼              │
-│  ┌─────────────────────────────────────────────────────────────┐  │
-│  │                     Gerbil Core                             │  │
-│  │  ┌─────────┐  ┌──────────┐  ┌────────┐  ┌───────────────┐   │  │
-│  │  │ Models  │  │ Generate │  │ Stream │  │ JSON/Embed    │   │  │
-│  │  └────┬────┘  └────┬─────┘  └───┬────┘  └───────┬───────┘   │  │
-│  └───────┼────────────┼────────────┼───────────────┼───────────┘  │
-│          │            │            │               │              │
-│          ▼            ▼            ▼               ▼              │
-│  ┌─────────────────────────────────────────────────────────────┐  │
-│  │                   transformers.js                           │  │
-│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │  │
-│  │  │  Tokenizer  │  │    Model    │  │   TextStreamer      │  │  │
-│  │  └─────────────┘  └──────┬──────┘  └─────────────────────┘  │  │
-│  └──────────────────────────┼──────────────────────────────────┘  │
-│                             │                                     │
-│                             ▼                                     │
-│  ┌─────────────────────────────────────────────────────────────┐  │
-│  │                    ONNX Runtime                             │  │
-│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │  │
-│  │  │   WebGPU    │  │     CPU     │  │       WASM          │  │  │
-│  │  │  (Browser)  │  │   (Node.js) │  │   (Fallback)        │  │  │
-│  │  └─────────────┘  └─────────────┘  └─────────────────────┘  │  │
-│  └─────────────────────────────────────────────────────────────┘  │
-│                                                                   │
-└───────────────────────────────────────────────────────────────────┘
-```
-
-## Key Design Decisions
-
-### 1. transformers.js as the Foundation
-
-We use [Hugging Face transformers.js](https://huggingface.co/docs/transformers.js) which provides:
-- Pre-converted ONNX models from Hugging Face Hub
-- Tokenizers that match the original models exactly
-- Unified API across CPU/GPU/WASM backends
-
-### 2. WebGPU First
-
-WebGPU provides 5-10x speedup over CPU for inference:
-- **Browser**: Native WebGPU via `navigator.gpu`
-- **Node.js**: Headless Chrome as a WebGPU accelerator (ChromeGPUBackend)
-
-### 3. Quantization for Speed
-
-All models use quantized weights:
-- **q4f16**: 4-bit weights, fp16 compute (WebGPU)
-- **q4**: 4-bit weights, fp32 compute (CPU/WASM)
-
-This reduces model size by ~4x and improves inference speed.
-
-### 4. Streaming via Web Workers
-
-Browser inference runs in a Web Worker to:
-- Keep the UI responsive during model loading
-- Stream tokens in real-time without blocking
-- Isolate GPU memory from the main thread
-

package/docs/architecture/caching.md

@@ -1,227 +0,0 @@
-# Model Caching
-
-How Gerbil caches models for fast subsequent loads.
-
-## Overview
-
-Model files are large (100MB - 500MB). Gerbil caches them locally to avoid re-downloading:
-
-| Environment | Cache Location | Mechanism |
-|-------------|----------------|-----------|
-| Browser | IndexedDB | transformers.js built-in |
-| Node.js (CPU) | `~/.cache/huggingface/hub` | transformers.js built-in |
-| Node.js (WebGPU) | Chrome's IndexedDB | Via ChromeGPUBackend |
-
-## Browser Caching
-
-### IndexedDB
-
-transformers.js automatically caches model files in IndexedDB:
-
-```
-IndexedDB
-└── transformers-cache
-    └── onnx-community/Qwen3-0.6B-ONNX
-        ├── tokenizer.json
-        ├── config.json
-        ├── model_q4f16.onnx
-        └── ...
-```
-
-### Cache Behavior
-
-1. **First load**: Downloads from Hugging Face Hub (~15-30s)
-2. **Subsequent loads**: Reads from IndexedDB (~1-2s)
-
-### Checking Cache
-
-```typescript
-import { getWebGPUInfo } from "@tryhamster/gerbil/browser";
-
-// Models are cached per-origin
-// Same origin = same cache
-```
-
-### Clearing Cache
-
-```javascript
-// In browser DevTools:
-indexedDB.deleteDatabase("transformers-cache");
-```
-
-## Node.js CPU Caching
-
-### Hugging Face Hub Cache
-
-transformers.js uses the standard HF cache directory:
-
-```
-~/.cache/huggingface/hub/
-└── models--onnx-community--Qwen3-0.6B-ONNX/
-    ├── blobs/
-    │   └── [sha256 hashes]
-    ├── refs/
-    │   └── main
-    └── snapshots/
-        └── [commit hash]/
-            ├── tokenizer.json
-            ├── config.json
-            └── model_q4.onnx
-```
-
-### Environment Variables
-
-```bash
-# Custom cache directory
-export HF_HOME=/path/to/cache
-export TRANSFORMERS_CACHE=/path/to/cache
-
-# Offline mode (use cache only)
-export TRANSFORMERS_OFFLINE=1
-```
-
-### CLI Cache Management
-
-```bash
-# View cache
-npx @tryhamster/gerbil cache
-
-# Clear cache
-npx @tryhamster/gerbil cache --clean
-
-# Clear old models
-npx @tryhamster/gerbil cache --older-than 30
-```
-
-## Node.js WebGPU Caching
-
-### ChromeGPUBackend Cache
-
-When using WebGPU in Node.js, models are cached in Chrome's IndexedDB:
-
-```
-~/.gerbil/chrome-cache/
-└── Default/
-    └── IndexedDB/
-        └── http_127.0.0.1_43724.indexeddb.leveldb/
-            └── [model cache]
-```
-
-### Why a Fixed Port?
-
-The ChromeGPUBackend uses port 43724 ("GERBI") for a critical reason:
-
-IndexedDB caches are **origin-specific**. The origin includes:
-- Protocol: `http://`
-- Host: `127.0.0.1`
-- Port: `43724`
-
-A fixed port ensures the same origin every time → same cache.
-
-```typescript
-const GERBIL_LOCAL_PORT = 43724; // "GERBI" on phone keypad
-
-// Always same origin:
-// http://127.0.0.1:43724
-```
-
-### Cache Persistence
-
-The Chrome user data directory persists between runs:
-
-```typescript
-this.userDataDir = join(homedir(), ".gerbil", "chrome-cache");
-```
-
-This means:
-- Model downloads are cached
-- Shader compilations are cached
-- ~1.5s startup when cached vs ~20s first run
-
-## Cache Sizes
-
-| Model | Download Size | Cache Size |
-|-------|--------------|------------|
-| qwen3-0.6b | ~400MB | ~400MB |
-| smollm2-360m | ~250MB | ~250MB |
-| smollm2-135m | ~100MB | ~100MB |
-
-## Preloading Models
-
-### Browser
-
-```typescript
-// Preload during idle time
-const gerbil = await createGerbilWorker({
-  modelId: "qwen3-0.6b",
-  onProgress: (p) => {
-    if (p.status === "ready") {
-      console.log("Model cached and ready");
-    }
-  },
-});
-
-// Model is now in IndexedDB for instant loads
-```
-
-### Node.js
-
-```typescript
-// Preload in background
-const g = new Gerbil();
-await g.loadModel("qwen3-0.6b");
-// Model is now in HF cache
-```
-
-### CLI
-
-```bash
-# Download without running
-npx @tryhamster/gerbil info -m qwen3-0.6b
-# Model is now cached
-```
-
-## Offline Usage
-
-Once cached, models work offline:
-
-```typescript
-// Browser: Works if model is in IndexedDB
-const gerbil = await createGerbilWorker({ modelId: "qwen3-0.6b" });
-
-// Node.js: Set offline mode
-process.env.TRANSFORMERS_OFFLINE = "1";
-const g = new Gerbil();
-await g.loadModel("qwen3-0.6b"); // Uses cache only
-```
-
-## Troubleshooting
-
-### "Model not found" after cache clear
-
-Re-download by loading the model:
-
-```typescript
-await g.loadModel("qwen3-0.6b"); // Will re-download
-```
-
-### Cache taking too much space
-
-```bash
-# View cache size
-npx @tryhamster/gerbil cache
-
-# Clear old models
-npx @tryhamster/gerbil cache --older-than 7
-
-# Clear everything
-npx @tryhamster/gerbil cache --clean
-```
-
-### Browser cache not persisting
-
-Check browser settings:
-- Cookies/site data must be allowed
-- IndexedDB must not be blocked
-- Storage quota must not be exceeded
-

package/docs/architecture/inference.md

@@ -1,176 +0,0 @@
-# Inference Pipeline
-
-How Gerbil runs LLM inference using ONNX Runtime and transformers.js.
-
-## The Stack
-
-```
-┌─────────────────────────────────────┐
-│           transformers.js           │  ← Tokenization, model loading, generation
-├─────────────────────────────────────┤
-│            ONNX Runtime             │  ← Neural network execution
-├─────────────────────────────────────┤
-│     WebGPU  │   CPU   │   WASM      │  ← Execution backends
-└─────────────────────────────────────┘
-```
-
-## transformers.js
-
-[transformers.js](https://huggingface.co/docs/transformers.js) is a JavaScript port of Hugging Face Transformers that runs ONNX models in the browser and Node.js.
-
-### Model Loading
-
-```typescript
-import { AutoModelForCausalLM, AutoTokenizer } from "@huggingface/transformers";
-
-const tokenizer = await AutoTokenizer.from_pretrained(modelId);
-const model = await AutoModelForCausalLM.from_pretrained(modelId, {
-  dtype: "q4f16",   // Quantization
-  device: "webgpu", // Backend
-});
-```
-
-### Generation
-
-```typescript
-const inputs = tokenizer.apply_chat_template(messages, {
-  add_generation_prompt: true,
-  return_dict: true,
-});
-
-const output = await model.generate({
-  ...inputs,
-  max_new_tokens: 256,
-  temperature: 0.7,
-  do_sample: true,
-});
-
-const text = tokenizer.decode(output[0], { skip_special_tokens: true });
-```
-
-### Streaming with TextStreamer
-
-```typescript
-const streamer = new TextStreamer(tokenizer, {
-  skip_prompt: true,
-  skip_special_tokens: true,
-  callback_function: (text) => {
-    console.log(text); // Called for each token
-  },
-});
-
-await model.generate({ ...inputs, streamer });
-```
-
-## ONNX Runtime
-
-ONNX Runtime is the execution engine that runs the neural network operations.
-
-### Backends
-
-| Backend | Environment | Speed | Notes |
-|---------|-------------|-------|-------|
-| **WebGPU** | Browser, Chrome | ~70-100 tok/s | Fastest, requires GPU |
-| **CPU** | Node.js | ~10-30 tok/s | Uses SIMD, good on Apple Silicon |
-| **WASM** | Browser fallback | ~5-10 tok/s | Works everywhere |
-
-### Execution Providers
-
-ONNX Runtime selects execution providers based on availability:
-
-```
-WebGPU EP → WASM EP → CPU EP
-```
-
-Gerbil explicitly requests the desired backend:
-
-```typescript
-// For WebGPU
-await AutoModelForCausalLM.from_pretrained(modelId, { device: "webgpu" });
-
-// For CPU
-await pipeline("text-generation", modelId, { device: "cpu" });
-```
-
-## Quantization
-
-Quantization reduces model size and improves inference speed by using lower-precision numbers.
-
-### Quantization Types
-
-| Type | Weights | Compute | Size Reduction | Use Case |
-|------|---------|---------|----------------|----------|
-| **fp32** | 32-bit float | 32-bit | 1x (baseline) | Training |
-| **fp16** | 16-bit float | 16-bit | 2x | GPU inference |
-| **q4f16** | 4-bit int | 16-bit | ~4x | WebGPU inference |
-| **q4** | 4-bit int | 32-bit | ~4x | CPU inference |
-
-### Why q4f16 for WebGPU?
-
-WebGPU shaders work best with fp16 compute. The `q4f16` format:
-- Stores weights as 4-bit integers (small download)
-- Dequantizes to fp16 during inference (fast on GPU)
-- Maintains good quality for small models
-
-### Model Sizes
-
-| Model | Original | q4f16 | Download |
-|-------|----------|-------|----------|
-| Qwen3-0.6B | ~2.4GB | ~400MB | ~400MB |
-| SmolLM2-360M | ~1.4GB | ~250MB | ~250MB |
-| SmolLM2-135M | ~540MB | ~100MB | ~100MB |
-
-## Tokenization
-
-### Chat Templates
-
-Gerbil uses model-specific chat templates to format conversations:
-
-```typescript
-const messages = [
-  { role: "system", content: "You are helpful." },
-  { role: "user", content: "Hello!" },
-];
-
-const inputs = tokenizer.apply_chat_template(messages, {
-  add_generation_prompt: true,  // Add assistant turn start
-  return_dict: true,            // Return input_ids + attention_mask
-  enable_thinking: true,        // Qwen3 thinking mode
-});
-```
-
-### Thinking Mode (Qwen3)
-
-Qwen3 models support a "thinking" mode where the model shows reasoning:
-
-```
-<think>
-Let me work through this step by step...
-127 × 43 = 127 × 40 + 127 × 3 = 5080 + 381 = 5461
-</think>
-The answer is 5461.
-```
-
-Enabled via `enable_thinking: true` in the chat template.
-
-## KV Cache
-
-The Key-Value cache stores intermediate attention states to speed up autoregressive generation:
-
-```typescript
-const { past_key_values, sequences } = await model.generate({
-  ...inputs,
-  past_key_values: previousCache,  // Reuse from last turn
-  return_dict_in_generate: true,
-});
-
-// Save for next turn
-cache = past_key_values;
-```
-
-Benefits:
-- Faster multi-turn conversations
-- Reduced compute for long contexts
-
-Gerbil manages the KV cache automatically for multi-turn chat.
-