@tryhamster/gerbil 1.0.0-rc.0 → 1.0.0-rc.10

package/docs/ai-sdk.md

@@ -1,6 +1,6 @@
 # Gerbil + AI SDK
 
-Gerbil works as a [Vercel AI SDK v5](https://sdk.vercel.ai/) provider.
+Gerbil works as a [Vercel AI SDK v5](https://sdk.vercel.ai/) provider, supporting text generation, speech synthesis (TTS), and transcription (STT).
 
 ## Setup
 
@@ -9,7 +9,9 @@ import { generateText, streamText } from "ai";
 import { gerbil } from "@tryhamster/gerbil/ai";
 ```
 
-## Generate Text
+## Text Generation
+
+### Generate Text
 
 ```typescript
 const { text } = await generateText({
@@ -18,7 +20,7 @@ const { text } = await generateText({
 });
 ```
 
-## Stream Text
+### Stream Text
 
 ```typescript
 const stream = streamText({
@@ -31,7 +33,7 @@ for await (const chunk of stream.textStream) {
 }
 ```
 
-## With System Prompt
+### With System Prompt
 
 ```typescript
 const { text } = await generateText({
@@ -41,40 +43,154 @@ const { text } = await generateText({
 });
 ```
 
-## Model Settings
+### Thinking Mode
 
 ```typescript
 import { createGerbil } from "@tryhamster/gerbil/ai";
 
-// Create provider with defaults
-const local = createGerbil({
-  device: "gpu",
-  dtype: "q4",
-});
+const local = createGerbil({ device: "gpu" });
 
-// Use with settings
 const { text } = await generateText({
   model: local("qwen3-0.6b", { thinking: true }),
   prompt: "What is 127 × 43?",
 });
 ```
 
-## Available Options
+## Speech Generation (TTS)
+
+Generate speech from text using Kokoro TTS:
+
+```typescript
+import { experimental_generateSpeech as generateSpeech } from "ai";
+import { gerbil } from "@tryhamster/gerbil/ai";
+
+const result = await generateSpeech({
+  model: gerbil.speech(),        // kokoro-82m by default
+  text: "Hello, welcome to Gerbil!",
+  voice: "af_heart",             // Female American voice
+});
+
+// result.audio is a Uint8Array in WAV format
+await writeFile("output.wav", result.audio);
+```
+
+### Available Voices
+
+```typescript
+const voices = gerbil.listVoices();
+// Returns: [{ id, name, gender, language }, ...]
+
+// Example voices:
+// - af_heart (Female, American)
+// - bf_emma (Female, British)
+// - am_fenrir (Male, American)
+// - bm_daniel (Male, British)
+```
+
+### Speech Options
+
+```typescript
+const result = await generateSpeech({
+  model: gerbil.speech("kokoro-82m", { 
+    voice: "bf_emma",   // Default voice
+    speed: 1.2,         // Speed multiplier
+  }),
+  text: "Speak faster!",
+});
+```
+
+## Transcription (STT)
+
+Transcribe audio to text using Whisper:
+
+```typescript
+import { experimental_transcribe as transcribe } from "ai";
+import { gerbil } from "@tryhamster/gerbil/ai";
+import { readFile } from "fs/promises";
+
+const result = await transcribe({
+  model: gerbil.transcription(),   // whisper-tiny.en by default
+  audio: await readFile("audio.wav"),
+});
+
+console.log(result.text);              // "Hello world"
+console.log(result.language);          // "en"
+console.log(result.durationInSeconds); // 2.5
+console.log(result.segments);          // Timestamped segments
+```
+
+### Available Models
 
 ```typescript
-gerbil(modelId, {
-  thinking?: boolean,    // Enable reasoning mode (Qwen3)
-  device?: "auto" | "gpu" | "cpu",
-  dtype?: "q4" | "q8" | "fp16" | "fp32",
-})
+const models = gerbil.listTranscriptionModels();
+
+// Models (smallest to largest):
+// - whisper-tiny.en (39M, English only, fastest)
+// - whisper-tiny (39M, multilingual)
+// - whisper-base.en (74M, English only)
+// - whisper-base (74M, multilingual)
+// - whisper-small.en (244M, English only)
+// - whisper-small (244M, multilingual)
+// - whisper-large-v3-turbo (809M, 80+ languages, best quality)
+```
+
+### Larger Models
+
+```typescript
+// Use a larger model for better accuracy
+const result = await transcribe({
+  model: gerbil.transcription("whisper-base"),
+  audio: audioBuffer,
+});
+
+// Use multilingual model with language hint
+const result = await transcribe({
+  model: gerbil.transcription("whisper-small", { language: "es" }),
+  audio: spanishAudio,
+});
+```
+
+## Custom Provider
+
+```typescript
+import { createGerbil } from "@tryhamster/gerbil/ai";
+
+const local = createGerbil({
+  device: "gpu",
+  dtype: "q4",
+});
+
+// Text generation
+const { text } = await generateText({
+  model: local("qwen3-0.6b"),
+  prompt: "Hello",
+});
+
+// Speech
+const speech = await generateSpeech({
+  model: local.speech(),
+  text: "Hello",
+});
+
+// Transcription
+const transcript = await transcribe({
+  model: local.transcription(),
+  audio: audioData,
+});
 ```
 
 ## Specification
 
-Gerbil implements `LanguageModelV2` from `@ai-sdk/provider`:
+Gerbil implements the following AI SDK v5 interfaces:
 
-- `specificationVersion: "v2"`
-- Streaming with `text-start`, `text-delta`, `text-end`, `finish` events
-- Reasoning content type for thinking mode
+| Interface | Purpose | Method |
+|-----------|---------|--------|
+| `LanguageModelV2` | Text generation | `gerbil(modelId)` |
+| `SpeechModelV2` | Text-to-Speech | `gerbil.speech()` |
+| `TranscriptionModelV2` | Speech-to-Text | `gerbil.transcription()` |
 
+All models support:
+- `specificationVersion: "v2"`
+- Proper warning reporting
+- Request/response metadata
 

package/docs/browser.md

@@ -1,6 +1,6 @@
 # Browser Usage
 
-Run LLMs directly in the browser with WebGPU acceleration. No server required.
+Run LLMs, TTS, and STT directly in the browser with WebGPU acceleration. No server required.
 
 ## Quick Start (React)
 
@@ -203,7 +203,7 @@ function App() {
 const {
   completion,      // string - generated text
   thinking,        // string - thinking content
-  complete,        // (prompt: string) => Promise<string>
+  complete,        // (prompt: string, options?) => Promise<string>
   isLoading,       // boolean - model loading
   loadingProgress, // { status, file?, progress? }
   isGenerating,    // boolean - generating
@@ -218,6 +218,244 @@ const {
 });
 ```
 
+#### Vision (Image Analysis)
+
+Use `useCompletion` with a vision model to analyze images:
+
+```tsx
+import { useCompletion } from "@tryhamster/gerbil/browser";
+
+function ImageAnalyzer() {
+  const { complete, completion, isLoading, isGenerating } = useCompletion({
+    model: "ministral-3b",  // Vision model
+    maxTokens: 2048,
+  });
+  const [imageUrl, setImageUrl] = useState<string | null>(null);
+
+  const handleFile = (e: React.ChangeEvent<HTMLInputElement>) => {
+    const file = e.target.files?.[0];
+    if (file) {
+      const reader = new FileReader();
+      reader.onload = () => setImageUrl(reader.result as string);
+      reader.readAsDataURL(file);
+    }
+  };
+
+  const analyze = () => {
+    if (imageUrl) {
+      // Pass images array in the second argument
+      complete("Describe this image in detail", { images: [imageUrl] });
+    }
+  };
+
+  if (isLoading) return <div>Loading vision model...</div>;
+
+  return (
+    <div>
+      <input type="file" accept="image/*" onChange={handleFile} />
+      {imageUrl && <img src={imageUrl} style={{ maxWidth: 300 }} />}
+      <button onClick={analyze} disabled={!imageUrl || isGenerating}>
+        Analyze Image
+      </button>
+      <p>{completion}</p>
+    </div>
+  );
+}
+```
+
+Images can be:
+- **Data URIs** (`data:image/png;base64,...`) — from FileReader or canvas
+- **HTTP URLs** — external image links (must be CORS-accessible)
+
+Both formats work:
+```tsx
+// Plain strings
+complete("Describe", { images: ["https://example.com/photo.jpg"] });
+
+// ImageInput objects (same as core Gerbil API)
+complete("Describe", { images: [{ source: "https://example.com/photo.jpg" }] });
+```
+
+## Voice Hooks
+
+### `useSpeech` (TTS)
+
+Generate speech from text in the browser:
+
+```tsx
+import { useSpeech } from "@tryhamster/gerbil/browser";
+
+function SpeechDemo() {
+  const { speak, stop, isSpeaking, isLoading, listVoices } = useSpeech();
+
+  if (isLoading) return <div>Loading TTS model...</div>;
+
+  return (
+    <div>
+      <button onClick={() => speak("Hello from the browser!")}>
+        {isSpeaking ? "Speaking..." : "Speak"}
+      </button>
+      {isSpeaking && <button onClick={stop}>Stop</button>}
+    </div>
+  );
+}
+```
+
+#### API
+
+```typescript
+const {
+  speak,           // (text: string, opts?) => Promise<void>
+  stop,            // () => void
+  isSpeaking,      // boolean
+  isLoading,       // boolean
+  isReady,         // boolean
+  listVoices,      // () => VoiceInfo[]
+  currentVoice,    // string
+  setVoice,        // (id: string) => void
+  currentSpeed,    // number
+  setSpeed,        // (speed: number) => void
+  loadingProgress, // { status, file?, progress? }
+  error,           // string | null
+} = useSpeech({
+  voice: "af_heart",  // Default voice
+  speed: 1.0,         // Speed multiplier
+  autoLoad: false,    // Loads on first speak()
+});
+```
+
+📖 See [TTS docs](./tts.md) for voice list and options.
+
+### `useVoiceInput` (STT)
+
+Record and transcribe audio:
+
+```tsx
+import { useVoiceInput } from "@tryhamster/gerbil/browser";
+
+function VoiceInput() {
+  const { startRecording, stopRecording, isRecording, transcript } = useVoiceInput({
+    onTranscript: (text) => console.log("User said:", text),
+  });
+
+  return (
+    <button onClick={isRecording ? stopRecording : startRecording}>
+      {isRecording ? "🔴 Stop" : "🎤 Record"}
+    </button>
+  );
+}
+```
+
+#### Streaming Transcription (Real-time)
+
+Transcribe audio in chunks as the user speaks - perfect for live captioning or call transcription:
+
+```tsx
+function LiveTranscription() {
+  const { 
+    startRecording, 
+    stopRecording, 
+    isRecording, 
+    transcript,      // Full accumulated transcript
+    streamingChunk,  // Current chunk being transcribed
+    chunkCount,      // Number of chunks processed
+  } = useVoiceInput({
+    streaming: true,        // Enable streaming mode
+    chunkDuration: 3000,    // Transcribe every 3 seconds
+    onChunk: (text, idx) => console.log(`Chunk ${idx}: ${text}`),
+  });
+
+  return (
+    <div>
+      <button onClick={isRecording ? stopRecording : startRecording}>
+        {isRecording ? "Stop" : "Start Live Transcription"}
+      </button>
+      {streamingChunk && <p style={{ color: 'gray' }}>Current: {streamingChunk}</p>}
+      <p>Transcript: {transcript}</p>
+    </div>
+  );
+}
+```
+
+#### API
+
+```typescript
+const {
+  startRecording,   // () => Promise<void>
+  stopRecording,    // () => Promise<string>
+  cancelRecording,  // () => void
+  transcribe,       // (audio: Float32Array) => Promise<string>
+  isRecording,      // boolean
+  isTranscribing,   // boolean
+  isLoading,        // boolean
+  isReady,          // boolean
+  transcript,       // string - full transcript
+  streamingChunk,   // string - current chunk (streaming mode)
+  chunkCount,       // number - chunks processed (streaming mode)
+  loadingProgress,  // { status, file?, progress? }
+  error,            // string | null
+} = useVoiceInput({
+  model: "whisper-tiny.en",
+  autoLoad: false,
+  onTranscript: (text) => {},
+  // Streaming options:
+  streaming: false,      // Enable streaming mode
+  chunkDuration: 1500,   // ms between transcriptions (default)
+  onChunk: (text, idx) => {},  // Called for each chunk
+});
+```
+
+📖 See [STT docs](./stt.md) for model options.
+
+### `useVoiceChat` (Full Voice Conversation)
+
+Complete voice-to-voice: record → transcribe → LLM → speak:
+
+```tsx
+import { useVoiceChat } from "@tryhamster/gerbil/browser";
+
+function VoiceAssistant() {
+  const {
+    messages,
+    startListening,
+    stopListening,
+    isListening,
+    isSpeaking,
+    stage,  // "idle" | "listening" | "transcribing" | "thinking" | "speaking"
+  } = useVoiceChat({
+    llmModel: "qwen3-0.6b",
+    sttModel: "whisper-tiny.en",
+    voice: "af_bella",
+    system: "You are a helpful assistant.",
+  });
+
+  return (
+    <button
+      onMouseDown={startListening}
+      onMouseUp={stopListening}
+    >
+      {stage === "idle" ? "🎤 Hold to Speak" : stage}
+    </button>
+  );
+}
+```
+
+### Audio Playback Utilities
+
+```typescript
+import { playAudio, createAudioPlayer } from "@tryhamster/gerbil/browser";
+
+// One-shot playback
+const controller = await playAudio(audioFloat32Array, 24000);
+await controller.onEnded;
+
+// Streaming playback
+const player = createAudioPlayer(24000);
+for await (const chunk of gerbil.speakStream("Long text...")) {
+  player.queue(chunk.samples);
+}
+```
+
 ## Low-Level API
 
 For full control, use `createGerbilWorker` directly:
@@ -302,6 +540,7 @@ const info = await getWebGPUInfo();
 | `qwen3-0.6b` | ~400MB | General use, thinking mode |
 | `smollm2-360m` | ~250MB | Faster, smaller |
 | `smollm2-135m` | ~100MB | Fastest, basic tasks |
+| `ministral-3b` | ~2.5GB | **Vision** — image analysis |
 
 Models are cached in IndexedDB after first download.
 

package/docs/memory.md

@@ -212,6 +212,72 @@ await gerbil.dispose();
 // Closes Chrome page, releases memory
 ```
 
+## Response Caching
+
+Gerbil also supports caching inference **responses** (different from KV cache). This is useful for repeated prompts:
+
+### Enable Response Caching
+
+```typescript
+// First call: ~150ms (runs inference)
+const result = await g.generate("What is 2+2?", { cache: true });
+
+// Second call: ~0ms (returns from cache!)
+const cached = await g.generate("What is 2+2?", { cache: true });
+console.log(cached.cached); // true
+```
+
+### Custom TTL
+
+```typescript
+// Cache for 10 minutes (default: 5 min)
+await g.generate("prompt", { 
+  cache: true, 
+  cacheTtl: 10 * 60 * 1000 
+});
+```
+
+### Cache Statistics
+
+```typescript
+const stats = g.getResponseCacheStats();
+console.log(stats);
+// { hits: 1, misses: 1, size: 1, hitRate: 50 }
+```
+
+### Clear Response Cache
+
+```typescript
+// Clear all cached responses
+g.clearResponseCache();
+```
+
+### Cache Key
+
+The cache key is a hash of:
+- Prompt text
+- Model ID
+- maxTokens, temperature, topP, topK
+- System prompt
+- Thinking mode
+
+Different parameters = different cache entries.
+
+### Limitations
+
+Response caching is **not supported** for:
+- Streaming calls (`onToken` callback)
+- Vision/image calls
+
+### KV Cache vs Response Cache
+
+| Feature | KV Cache | Response Cache |
+|---------|----------|----------------|
+| What's cached | Attention states | Full responses |
+| Purpose | Conversation context | Repeated prompts |
+| Clear method | `clearCache()` | `clearResponseCache()` |
+| Default | Always on | Off (`cache: true` to enable) |
+
 ## Testing
 
 Run the memory management test suite:
@@ -227,3 +293,9 @@ This verifies:
 - Threshold-based cleanup works
 - Proper cleanup on dispose
 
+Run response caching test:
+
+```bash
+npx tsx examples/test-cache.ts
+```
+