@tryhamster/gerbil 1.1.2 → 1.1.3

package/docs/browser.md

@@ -1,762 +0,0 @@
-# Browser Usage
-
-Run models directly in the browser with WebGPU. No server required.
-
-Browser inference runs on the **native engine** — the React hooks at
-`@tryhamster/gerbil/gpu/hooks` (`useEngine` / `useChat` / `useText` / `useVision` /
-`useEmbedding` / `useTTS` / `useSTT` / `useVoiceChat` / `useMemory`),
-backed by the from-scratch WGSL `WebGPUEngine`. Pure compute shaders, no ONNX, no
-transformers.js. This is the supported path for text, vision, embeddings, and speech, and the
-lane the Gerbil site itself runs on.
-
-> The old inline transformers.js/ONNX worker hooks (`useChat`, `useSpeech`, `useVoiceInput`,
-> `useEmbedding`, `createGerbilWorker`, `preload*`) have been **removed** from
-> `@tryhamster/gerbil/browser`. `@tryhamster/gerbil/browser` now exports only device/WebGPU
-> utilities (`isModelSafeForDevice`, `detectMemoryCrash`, `downloadModelChunked`,
-> `checkWebGPUCapabilities`, `getBrowserDiagnostics`, …). The "Legacy Worker Lane" sections
-> below are retained for historical reference and no longer reflect the shipped API.
-
-> **Pre-1.0.** APIs may still shift before 1.0.
-
-## Native Engine (recommended)
-
-```tsx
-import { useEngine } from "@tryhamster/gerbil/gpu/hooks";
-
-function Chat() {
-  const { complete, completion, isLoading, isGenerating, tps } = useEngine({
-    model: "mlx-community/Qwen3.5-0.8B-4bit",
-    autoLoad: true, // dtype defaults to "auto": int4 on mobile, native on desktop
-  });
-
-  if (isLoading) return <div>Loading model…</div>;
-
-  return (
-    <div>
-      <button onClick={() => complete("Write a haiku about coding")} disabled={isGenerating}>
-        Generate
-      </button>
-      <p>{completion}</p>
-      {isGenerating && <span>{tps?.toFixed(1)} tok/s</span>}
-    </div>
-  );
-}
-```
-
-`useEngine` owns the engine lifecycle — load, unload, hot-swap on config change, and
-**reference-counted sharing** so multiple components requesting the same
-`model|dtype|vision|embedding|maxSeqLen` share ONE engine (weights uploaded to the GPU once).
-
-```typescript
-const {
-  complete,        // (prompt, opts?) => Promise<string> — streams into `completion`
-  describeImage,   // (image, prompt?, opts?) => Promise<string> — needs enableVision
-  embed,           // (text, { taskType }?) => Promise<Float32Array> — needs embedding
-  similarity,      // (a, b) => Promise<number> — needs embedding
-  completion,      // string — current text, streams token by token
-  isLoading, loadingProgress, isGenerating, isReady, tps, error, errorKind,
-  load, stop, dispose,
-} = useEngine({
-  model: "mlx-community/Qwen3.5-0.8B-4bit", // HF repo id
-  dtype: "auto",        // "auto" (default) | "f32" | "q4"
-  maxSeqLen,            // default: 2048 mobile / 4096 desktop
-  autoLoad: false,      // load on mount
-  enableVision: false,  // build the ViT so describeImage() works
-  embedding: false,     // load as an embedding model
-  onReady, onError,     // onError(err, kind) — kind: "no-webgpu" | "oom" | …
-});
-```
-
-Use `enableVision: true` for image→text (see [Vision](./vision.md)) and `embedding: true`
-for embeddings (see [Embeddings](./embeddings.md)).
-
-### Browser support (native engine)
-
-- **Chrome / Edge 113+**
-- **Safari 26+ (iOS / iPadOS 26+)**
-- **Firefox 141+**
-
-On devices without WebGPU the hook reports an `errorKind` of `"no-webgpu"` rather than
-silently degrading.
-
----
-
-## Legacy Worker Lane (removed — historical reference only)
-
-> **Removed.** Everything below this point documents the old inline transformers.js/ONNX
-> worker hooks, which have been **deleted** from the package (no `useChat`/`useSpeech`/
-> `useVoiceInput`/`useEmbedding` worker exports, no `createGerbilWorker`, no `preload*`
-> functions, and no ONNX/transformers.js dependency). It is kept here only for historical
-> reference. Use the native hooks from `@tryhamster/gerbil/gpu/hooks` instead.
-
-## Quick Start (React)
-
-```tsx
-import { useChat } from "@tryhamster/gerbil/browser";
-
-function Chat() {
-  const { messages, input, setInput, handleSubmit, isLoading, isGenerating } = useChat();
-
-  if (isLoading) return <div>Loading model...</div>;
-
-  return (
-    <div>
-      {messages.map(m => (
-        <div key={m.id}>
-          <strong>{m.role}:</strong> {m.content}
-        </div>
-      ))}
-      <form onSubmit={handleSubmit}>
-        <input 
-          value={input} 
-          onChange={e => setInput(e.target.value)}
-          placeholder="Say something..."
-        />
-        <button disabled={isGenerating}>Send</button>
-      </form>
-    </div>
-  );
-}
-```
-
-That's it! The hook handles model loading, streaming, and state management.
-
-## Model Preloading
-
-Download models during app initialization so they're ready when users need them:
-
-```typescript
-import { 
-  preloadChatModel,
-  preloadEmbeddingModel,
-  preloadTTSModel,
-  preloadSTTModel 
-} from "@tryhamster/gerbil/browser";
-
-// During app initialization
-async function initApp() {
-  // Preload LLM
-  await preloadChatModel("qwen3-0.6b", {
-    onProgress: (p) => {
-      if (p.status === "downloading") {
-        console.log(`Downloading ${p.file}: ${p.progress}%`);
-      }
-    },
-  });
-
-  // Preload other models as needed
-  await preloadEmbeddingModel("Xenova/all-MiniLM-L6-v2");
-  await preloadTTSModel("kokoro-82m");
-  await preloadSTTModel("whisper-tiny.en");
-}
-
-initApp();
-```
-
-After preloading, hooks like `useChat` will load instantly from IndexedDB cache.
-
-## React Hooks
-
-### `useChat`
-
-Full-featured chat hook with message history, streaming, and thinking mode.
-
-```tsx
-import { useChat } from "@tryhamster/gerbil/browser";
-
-function Chat() {
-  const {
-    messages,        // Message[] - chat history
-    input,           // string - current input
-    setInput,        // (value: string) => void
-    handleSubmit,    // (e?) => void - submit handler
-    isLoading,       // boolean - model loading
-    loadingProgress, // { status, file?, progress?, downloadCount? }
-    isGenerating,    // boolean - generating response
-    thinking,        // string - current thinking (streaming)
-    stop,            // () => void - stop generation
-    clear,           // () => void - clear messages
-    tps,             // number - tokens per second
-    isReady,         // boolean - model ready
-    error,           // string | null
-    load,            // () => void - manually load model
-  } = useChat({
-    model: "qwen3-0.6b",     // Model ID
-    system: "You are helpful.", // System prompt
-    thinking: true,          // Enable thinking mode
-    maxTokens: 512,          // Max tokens per response
-    temperature: 0.7,        // Sampling temperature
-    autoLoad: false,         // Auto-load on mount (default: false)
-  });
-}
-```
-
-#### Message Type
-
-```typescript
-interface Message {
-  id: string;
-  role: "user" | "assistant";
-  content: string;
-  thinking?: string;  // Thinking content (if enabled)
-}
-```
-
-#### Auto-Loading Behavior
-
-By default, hooks **don't** auto-load on mount (to avoid unexpected downloads). The model loads when:
-1. You call `handleSubmit()` / `complete()` - triggers load automatically
-2. You call `load()` manually
-3. You set `autoLoad: true` - loads on mount
-
-```tsx
-// Default: loads when user submits first message
-function Chat() {
-  const { handleSubmit, isLoading } = useChat();
-  // Model loads on first handleSubmit()
-}
-
-// Explicit preload on mount
-function ChatPreloaded() {
-  const { handleSubmit, isLoading } = useChat({ autoLoad: true });
-  // Model starts loading immediately
-}
-
-// Manual load control
-function ChatManual() {
-  const { load, isLoading, isReady } = useChat();
-  return <button onClick={load} disabled={isLoading || isReady}>Load Model</button>;
-}
-```
-
-#### With Thinking Mode
-
-```tsx
-function ChatWithThinking() {
-  const { messages, input, setInput, handleSubmit, isLoading } = useChat({
-    thinking: true,
-  });
-
-  if (isLoading) return <div>Loading...</div>;
-
-  return (
-    <div>
-      {messages.map(m => (
-        <div key={m.id}>
-          {m.thinking && (
-            <details>
-              <summary>Thinking...</summary>
-              <pre>{m.thinking}</pre>
-            </details>
-          )}
-          <p>{m.content}</p>
-        </div>
-      ))}
-      <form onSubmit={handleSubmit}>
-        <input value={input} onChange={e => setInput(e.target.value)} />
-      </form>
-    </div>
-  );
-}
-```
-
-#### With Loading Progress
-
-The `loadingProgress` object tells you what's happening:
-
-- `status: "loading"` - Initial load / compiling shaders
-- `status: "downloading"` - Downloading model files from network
-- `status: "ready"` - Model ready
-
-When loading from **cache**, you'll only see `"loading"` → `"ready"` (no `"downloading"`).
-When **downloading**, you'll see `"downloading"` with progress info.
-
-```tsx
-function ChatWithProgress() {
-  const { messages, handleSubmit, isLoading, loadingProgress } = useChat();
-
-  if (isLoading) {
-    const p = loadingProgress;
-    
-    if (p?.status === "downloading") {
-      // Downloading from network - show progress
-      return <div>Downloading {p.file}: {p.progress}%</div>;
-    }
-    
-    // Loading from cache or compiling shaders
-    return <div>{p?.message || "Loading model..."}</div>;
-  }
-
-  // ... rest of chat UI
-}
-```
-
-### `useCompletion`
-
-Simple completion hook for one-off generations.
-
-```tsx
-import { useCompletion } from "@tryhamster/gerbil/browser";
-
-function App() {
-  const { complete, completion, isLoading, isGenerating, tps } = useCompletion();
-
-  if (isLoading) return <div>Loading model...</div>;
-
-  return (
-    <div>
-      <button 
-        onClick={() => complete("Write a haiku about coding")}
-        disabled={isGenerating}
-      >
-        Generate
-      </button>
-      <p>{completion}</p>
-      {isGenerating && <span>{tps.toFixed(0)} tok/s</span>}
-    </div>
-  );
-}
-```
-
-#### API
-
-```typescript
-const {
-  completion,      // string - generated text
-  thinking,        // string - thinking content
-  complete,        // (prompt: string, options?) => Promise<string>
-  isLoading,       // boolean - model loading
-  loadingProgress, // { status, file?, progress? }
-  isGenerating,    // boolean - generating
-  stop,            // () => void - stop generation
-  tps,             // number - tokens per second
-  isReady,         // boolean - model ready
-  error,           // string | null
-} = useCompletion({
-  model: "qwen3-0.6b",
-  thinking: false,
-  maxTokens: 512,
-});
-```
-
-#### Vision (Image Analysis)
-
-> For native image→text, use `useEngine({ enableVision: true }).describeImage(...)` —
-> see [Vision docs](./vision.md). The legacy-lane example below uses the retired ONNX worker.
-
-Use `useCompletion` with a vision model to analyze images:
-
-```tsx
-import { useCompletion } from "@tryhamster/gerbil/browser";
-
-function ImageAnalyzer() {
-  const { complete, completion, isLoading, isGenerating } = useCompletion({
-    model: "ministral-3b",  // Vision model (legacy lane)
-    maxTokens: 2048,
-  });
-  const [imageUrl, setImageUrl] = useState<string | null>(null);
-
-  const handleFile = (e: React.ChangeEvent<HTMLInputElement>) => {
-    const file = e.target.files?.[0];
-    if (file) {
-      const reader = new FileReader();
-      reader.onload = () => setImageUrl(reader.result as string);
-      reader.readAsDataURL(file);
-    }
-  };
-
-  const analyze = () => {
-    if (imageUrl) {
-      // Pass images array in the second argument
-      complete("Describe this image in detail", { images: [imageUrl] });
-    }
-  };
-
-  if (isLoading) return <div>Loading vision model...</div>;
-
-  return (
-    <div>
-      <input type="file" accept="image/*" onChange={handleFile} />
-      {imageUrl && <img src={imageUrl} style={{ maxWidth: 300 }} />}
-      <button onClick={analyze} disabled={!imageUrl || isGenerating}>
-        Analyze Image
-      </button>
-      <p>{completion}</p>
-    </div>
-  );
-}
-```
-
-Images can be:
-- **Data URIs** (`data:image/png;base64,...`) — from FileReader or canvas
-- **HTTP URLs** — external image links (must be CORS-accessible)
-
-Both formats work:
-```tsx
-// Plain strings
-complete("Describe", { images: ["https://example.com/photo.jpg"] });
-
-// ImageInput objects (same as core Gerbil API)
-complete("Describe", { images: [{ source: "https://example.com/photo.jpg" }] });
-```
-
-## Voice Hooks
-
-### `useSpeech` (TTS)
-
-Generate speech from text in the browser:
-
-```tsx
-import { useSpeech } from "@tryhamster/gerbil/browser";
-
-function SpeechDemo() {
-  const { speak, stop, isSpeaking, isLoading, listVoices } = useSpeech();
-
-  if (isLoading) return <div>Loading TTS model...</div>;
-
-  return (
-    <div>
-      <button onClick={() => speak("Hello from the browser!")}>
-        {isSpeaking ? "Speaking..." : "Speak"}
-      </button>
-      {isSpeaking && <button onClick={stop}>Stop</button>}
-    </div>
-  );
-}
-```
-
-#### API
-
-```typescript
-const {
-  speak,           // (text: string, opts?) => Promise<void>
-  stop,            // () => void
-  isSpeaking,      // boolean
-  isLoading,       // boolean
-  isReady,         // boolean
-  listVoices,      // () => VoiceInfo[]
-  currentVoice,    // string
-  setVoice,        // (id: string) => void
-  currentSpeed,    // number
-  setSpeed,        // (speed: number) => void
-  loadingProgress, // { status, file?, progress? }
-  error,           // string | null
-} = useSpeech({
-  voice: "af_heart",  // Default voice
-  speed: 1.0,         // Speed multiplier
-  autoLoad: false,    // Loads on first speak()
-});
-```
-
-📖 See [TTS docs](./tts.md) for voice list and options.
-
-### `useVoiceInput` (STT)
-
-Record and transcribe audio:
-
-```tsx
-import { useVoiceInput } from "@tryhamster/gerbil/browser";
-
-function VoiceInput() {
-  const { startRecording, stopRecording, isRecording, transcript } = useVoiceInput({
-    onTranscript: (text) => console.log("User said:", text),
-  });
-
-  return (
-    <button onClick={isRecording ? stopRecording : startRecording}>
-      {isRecording ? "🔴 Stop" : "🎤 Record"}
-    </button>
-  );
-}
-```
-
-#### Streaming Transcription (Real-time)
-
-Transcribe audio in chunks as the user speaks - perfect for live captioning or call transcription:
-
-```tsx
-function LiveTranscription() {
-  const { 
-    startRecording, 
-    stopRecording, 
-    isRecording, 
-    transcript,      // Full accumulated transcript
-    streamingChunk,  // Current chunk being transcribed
-    chunkCount,      // Number of chunks processed
-  } = useVoiceInput({
-    streaming: true,        // Enable streaming mode
-    chunkDuration: 3000,    // Transcribe every 3 seconds
-    onChunk: (text, idx) => console.log(`Chunk ${idx}: ${text}`),
-  });
-
-  return (
-    <div>
-      <button onClick={isRecording ? stopRecording : startRecording}>
-        {isRecording ? "Stop" : "Start Live Transcription"}
-      </button>
-      {streamingChunk && <p style={{ color: 'gray' }}>Current: {streamingChunk}</p>}
-      <p>Transcript: {transcript}</p>
-    </div>
-  );
-}
-```
-
-#### API
-
-```typescript
-const {
-  startRecording,   // () => Promise<void>
-  stopRecording,    // () => Promise<string>
-  cancelRecording,  // () => void
-  transcribe,       // (audio: Float32Array) => Promise<string>
-  isRecording,      // boolean
-  isTranscribing,   // boolean
-  isLoading,        // boolean
-  isReady,          // boolean
-  transcript,       // string - full transcript
-  streamingChunk,   // string - current chunk (streaming mode)
-  chunkCount,       // number - chunks processed (streaming mode)
-  loadingProgress,  // { status, file?, progress? }
-  error,            // string | null
-} = useVoiceInput({
-  model: "whisper-tiny.en",
-  autoLoad: false,
-  onTranscript: (text) => {},
-  // Streaming options:
-  streaming: false,      // Enable streaming mode
-  chunkDuration: 1500,   // ms between transcriptions (default)
-  onChunk: (text, idx) => {},  // Called for each chunk
-});
-```
-
-📖 See [STT docs](./stt.md) for model options.
-
-### `useVoiceChat` (Full Voice Conversation)
-
-Complete voice-to-voice: record → transcribe → LLM → speak:
-
-```tsx
-import { useVoiceChat } from "@tryhamster/gerbil/browser";
-
-function VoiceAssistant() {
-  const {
-    messages,
-    startListening,
-    stopListening,
-    isListening,
-    isSpeaking,
-    stage,  // "idle" | "listening" | "transcribing" | "thinking" | "speaking"
-  } = useVoiceChat({
-    llmModel: "qwen3-0.6b",
-    sttModel: "whisper-tiny.en",
-    voice: "af_bella",
-    system: "You are a helpful assistant.",
-  });
-
-  return (
-    <button
-      onMouseDown={startListening}
-      onMouseUp={stopListening}
-    >
-      {stage === "idle" ? "🎤 Hold to Speak" : stage}
-    </button>
-  );
-}
-```
-
-### Audio Playback Utilities
-
-```typescript
-import { playAudio, createAudioPlayer } from "@tryhamster/gerbil/browser";
-
-// One-shot playback
-const controller = await playAudio(audioFloat32Array, 24000);
-await controller.onEnded;
-
-// Streaming playback
-const player = createAudioPlayer(24000);
-for await (const chunk of gerbil.speakStream("Long text...")) {
-  player.queue(chunk.samples);
-}
-```
-
-## Embeddings Hook
-
-### `useEmbedding`
-
-Generate embeddings for semantic search and similarity:
-
-```tsx
-import { useEmbedding } from "@tryhamster/gerbil/browser";
-
-function SemanticSearch() {
-  const { embed, similarity, search, isLoading, isReady, load } = useEmbedding({
-    model: "Xenova/all-MiniLM-L6-v2",  // Default
-    autoLoad: false,
-  });
-
-  if (isLoading) return <div>Loading embedding model...</div>;
-
-  const handleSearch = async () => {
-    const results = await search("capital of France", [
-      "Paris is beautiful",
-      "London is in England",
-      "Dogs are pets",
-    ], 2);  // topK = 2
-    
-    console.log(results);
-    // [{ text: "Paris is beautiful", score: 0.89, index: 0 }, ...]
-  };
-
-  const handleSimilarity = async () => {
-    const score = await similarity("Hello world", "Hi there");
-    console.log(score); // 0.85
-  };
-
-  return (
-    <div>
-      <button onClick={handleSearch}>Search</button>
-      <button onClick={handleSimilarity}>Compare</button>
-    </div>
-  );
-}
-```
-
-### Options
-
-```typescript
-const {
-  // Actions
-  embed,            // (text: string) => Promise<number[]>
-  embedBatch,       // (texts: string[]) => Promise<{ vector, text }[]>
-  similarity,       // (a: string, b: string) => Promise<number>
-  search,           // (query: string, corpus: string[], topK?) => Promise<SearchResult[]>
-  findNearest,      // (embedding: number[], candidates: string[], topK?) => Promise<SearchResult[]>
-  cosineSimilarity, // (a: number[], b: number[]) => number (sync)
-  load,             // () => void - manually load model
-  
-  // State
-  isLoading,        // boolean - model loading
-  isReady,          // boolean - model ready
-  loadingProgress,  // { status, message?, progress? }
-  error,            // string | null
-} = useEmbedding({
-  model: "Xenova/all-MiniLM-L6-v2",  // Embedding model
-  normalize: true,                    // Normalize vectors (default: true)
-  autoLoad: false,                    // Load on mount (default: false)
-  onReady: () => {},
-  onError: (err) => {},
-});
-```
-
-## Low-Level API
-
-For full control, use `createGerbilWorker` directly:
-
-```typescript
-import { createGerbilWorker, isWebGPUSupported } from "@tryhamster/gerbil/browser";
-
-if (!isWebGPUSupported()) {
-  console.log("WebGPU not supported");
-  return;
-}
-
-const gerbil = await createGerbilWorker({
-  modelId: "qwen3-0.6b",
-  onProgress: (p) => console.log(p.status),
-  onToken: (token) => console.log(token.text),
-  onComplete: (result) => console.log(`${result.tps} tok/s`),
-});
-
-await gerbil.generate("Hello!", { thinking: true });
-gerbil.terminate();
-```
-
-### `createGerbilWorker(options)`
-
-```typescript
-interface GerbilWorkerOptions {
-  modelId?: string;                           // Default: "qwen3-0.6b"
-  onProgress?: (progress: WorkerProgress) => void;
-  onToken?: (token: WorkerToken) => void;
-  onComplete?: (result: WorkerComplete) => void;
-  onError?: (error: string) => void;
-}
-
-interface GerbilWorker {
-  generate: (prompt: string, options?: GenerateOptions) => Promise<string>;
-  interrupt: () => void;
-  reset: () => void;
-  terminate: () => void;
-  isReady: () => boolean;
-}
-```
-
-### Generation Options
-
-```typescript
-interface GenerateOptions {
-  maxTokens?: number;    // Default: 256
-  temperature?: number;  // Default: 0.7
-  topP?: number;         // Default: 0.9
-  topK?: number;         // Default: 20
-  thinking?: boolean;    // Enable thinking mode
-  system?: string;       // System prompt
-}
-```
-
-## Utilities
-
-### `isWebGPUSupported()`
-
-```typescript
-import { isWebGPUSupported } from "@tryhamster/gerbil/browser";
-
-if (!isWebGPUSupported()) {
-  // Show fallback or error
-}
-```
-
-### `getWebGPUInfo()`
-
-```typescript
-import { getWebGPUInfo } from "@tryhamster/gerbil/browser";
-
-const info = await getWebGPUInfo();
-// { supported: true, adapter: "Apple", device: "Apple M4 Max" }
-```
-
-## Models (legacy worker lane)
-
-| Model | Size | Best For |
-|-------|------|----------|
-| `qwen3-0.6b` | ~400MB | General use, thinking mode |
-| `smollm2-360m` | ~250MB | Faster, smaller |
-| `smollm2-135m` | ~100MB | Fastest, basic tasks |
-
-> For vision, embeddings, and speech use the native engine (`useEngine`) — see the
-> [Native Engine](#native-engine-recommended) section above.
-
-Legacy-lane models are cached in IndexedDB after first download.
-
-## Browser Support (legacy worker lane)
-
-- **Chrome/Edge 113+** — Full WebGPU support
-- **Safari 26+ (iOS/iPadOS 26+)** — WebGPU support
-- **Firefox 141+** — WebGPU support
-
-## Troubleshooting
-
-### "WebGPU not supported"
-
-- Update to Chrome/Edge 113+
-- Check `chrome://gpu` for WebGPU status
-- Try `chrome://flags/#enable-unsafe-webgpu`
-
-### Slow first load
-
-First load downloads the model (~400MB for qwen3-0.6b) and compiles WebGPU shaders. Subsequent loads use IndexedDB cache (~2-5s).
-
-### Out of memory
-
-Use smaller models like `smollm2-135m`. Close other GPU-intensive tabs.