@rudderjs/ai 1.5.0 → 1.6.1

package/README.md

@@ -16,7 +16,9 @@ pnpm add openai                         # OpenAI (GPT) — also used for OpenRou
 pnpm add @google/genai                  # Google (Gemini)
 pnpm add cohere-ai                      # Cohere (reranking + embeddings)
 pnpm add @aws-sdk/client-bedrock-runtime # AWS Bedrock
-# Jina — no extra package needed
+# ElevenLabs (premium TTS + STT)        — no extra package needed (direct HTTP)
+# VoyageAI (embeddings + reranking)     — no extra package needed (direct HTTP)
+# Jina                                   — no extra package needed (direct HTTP)
 ```
 
 ## Runtime Compatibility
@@ -28,6 +30,12 @@ pnpm add @aws-sdk/client-bedrock-runtime # AWS Bedrock
 | `@rudderjs/ai` | Node, browser, Electron main+renderer, React Native | Agents, tools, streaming, providers — any `fetch`-capable JS runtime |
 | `@rudderjs/ai/node` | Node only | `documentFromPath()`, `imageFromPath()`, `transcribeFromPath()` (filesystem helpers) |
 | `@rudderjs/ai/server` | Node only | `AiProvider` (the RudderJS `ServiceProvider` — auto-discovered, you rarely import it) |
+| `@rudderjs/ai/mcp` | Node only (in practice) | `mcpClientTools()` + `mcpServerFromAgent()` — requires `@modelcontextprotocol/sdk` |
+| `@rudderjs/ai/memory-orm` | Node only | `OrmUserMemory` + `UserMemoryRecord` — ORM-backed `UserMemory` |
+| `@rudderjs/ai/memory-embedding` | Node only | `EmbeddingUserMemory` — semantic recall via the registered embedding model |
+| `@rudderjs/ai/budget-orm` | Node only | `OrmBudgetStorage` + `BudgetUsageRecord` — ORM-backed `BudgetStorage` |
+| `@rudderjs/ai/eval` | Any `fetch`-capable runtime | `evalSuite()` + `runSuite()` + metrics for testing agents against real models |
+| `@rudderjs/ai/computer-use` | Node only (in practice) | `executeComputerAction(page, action, state)` — lower-level Playwright dispatcher |
 
 The main entry has zero `node:*` static imports, so you can call agents and tools directly from a React Native screen, an Electron renderer, or a browser. `@rudderjs/core` is an optional peer — only `/server` consumers pull it in.
 
@@ -46,6 +54,8 @@ export default {
     ollama:    { driver: 'ollama',    baseUrl: 'http://localhost:11434' },
     cohere:    { driver: 'cohere',    apiKey: process.env.COHERE_API_KEY! },
     jina:      { driver: 'jina',      apiKey: process.env.JINA_API_KEY! },
+    voyage:    { driver: 'voyage',    apiKey: process.env.VOYAGE_API_KEY! },        // embeddings + reranking
+    elevenlabs:{ driver: 'elevenlabs', apiKey: process.env.ELEVENLABS_API_KEY! },   // premium TTS + STT
     openrouter: {
       driver:   'openrouter',
       apiKey:   process.env.OPENROUTER_API_KEY!,
@@ -60,11 +70,10 @@ export default {
   },
 }
 
-// bootstrap/providers.ts
-import { AiProvider } from '@rudderjs/ai/server'
-export default [AiProvider]
 ```
 
+`AiProvider` is picked up by [auto-discovery](https://github.com/rudderjs/rudder/blob/main/docs/guide/service-providers.md#auto-discovery) — `pnpm rudder providers:discover` is all that's needed. The class lives at `@rudderjs/ai/server` (the main entry is runtime-agnostic); auto-discovery reads `rudderjs.providerSubpath` and loads it for you.
+
 ## Usage
 
 ### Agent Class
@@ -573,6 +582,68 @@ const agent = AI.agent({
 })
 ```
 
+### Computer Use (Anthropic)
+
+`computerUseTool({ page })` exposes a Playwright `Page` to an Anthropic Claude model via the native `computer_20250124` tool block — the model takes screenshots, moves the cursor, clicks, types, scrolls, presses keys. Anthropic-only (`anthropic/*` and `bedrock/anthropic.*`); OpenRouter-routed Anthropic models throw `ComputerUseProviderError`.
+
+```ts
+import { chromium } from 'playwright'
+import { Agent, computerUseTool } from '@rudderjs/ai'
+
+const browser = await chromium.launch()
+const page    = await browser.newPage({ viewport: { width: 1280, height: 800 } })
+
+class BrowserAgent extends Agent {
+  model() { return 'anthropic/claude-sonnet-4-5' }
+  tools() {
+    return [
+      computerUseTool({
+        page,
+        viewport:      { width: 1280, height: 800 },
+        needsApproval: true,    // default — pauses the loop before every action
+        maxActions:    50,       // per-instance safety cap; throws ComputerUseLimitError when exceeded
+      }),
+    ]
+  }
+}
+
+await new BrowserAgent().prompt('Find the cheapest flight from SFO to JFK next Tuesday.')
+```
+
+Playwright is **not** a dep of `@rudderjs/ai` — install it in your app. The tool accepts a structural `PageLike` subset so types check without the 300MB Playwright bundle. Action failures surface as `is_error: true` tool-results so the model can retry. `needsApproval: true` (the default) routes every action through the standard approval gate — review what the model wants to click before it clicks it.
+
+### Hosted vector stores + `fileSearch`
+
+`VectorStores` is a CRUD façade over the provider's hosted vector store; `fileSearch({ stores })` is the agent tool that queries them. The provider runs ingestion, chunking, embedding, and retrieval server-side — no embedding pipeline, no pgvector setup, no `execute` to write. Supported on **OpenAI** (`vectorStores.*`) and **Gemini** (`fileSearchStores`). Same façade, same agent surface.
+
+```ts
+import { Agent, VectorStores, fileSearch } from '@rudderjs/ai'
+
+// 1. Manage the store
+const kb = await VectorStores.create('Knowledge Base')                      // OpenAI default
+await kb.add({ filePath: './report.pdf', attributes: { author: 'Alice', year: 2026 } })
+
+// 2. Use it as an agent tool
+class SupportAgent extends Agent {
+  model() { return 'openai/gpt-4o' }                                        // or 'google/gemini-2.5-flash'
+  tools() {
+    return [
+      fileSearch({
+        stores:     [kb.id],
+        where:      { author: 'Alice', year: 2026 },                        // server-side metadata filter
+        maxResults: 10,
+      }),
+    ]
+  }
+}
+```
+
+**Provider override:** pass `{ provider: 'google' }` to `VectorStores.create(...)` for Gemini. Store ids are full resource paths (`fileSearchStores/foo-bar`) on Gemini, opaque (`vs_abc123`) on OpenAI — apps pass them back verbatim through the same `VectorStores` API.
+
+**Self-hosted RAG fallback.** `fileSearch({ ..., fallback: { model, column, embedWith } })` lifts a `similaritySearch` `execute` onto the tool. Providers that recognize the file-search hint (OpenAI, Gemini) still emit their native block; other providers serialize the tool as a function-call and run the fallback against a local pgvector model. Same agent prompt across hosted and self-hosted RAG.
+
+Full surface (provider-differences table, `where`/filter shapes, testing with `AiFake`): the framework's [Vector Stores guide](https://github.com/rudderjs/rudder/blob/main/docs/guide/vector-stores.md).
+
 ### Reranking
 
 Reorder documents by relevance to a query — useful for RAG pipelines:
@@ -664,6 +735,54 @@ for await (const chunk of stream) {
 const final = await response // full AgentResponse when stream completes
 ```
 
+### Queued prompts (`agent.queue()`)
+
+Push the agent run onto the queue for background execution. Returns a builder so you can configure the queue, attach success/failure callbacks, and (optionally) stream progress to a broadcast channel as it runs.
+
+Requires `@rudderjs/queue` (and `@rudderjs/broadcast` if you call `.broadcast()`).
+
+```ts
+// Fire-and-forget background run
+await new SupportAgent()
+  .queue('Help with refund request')
+  .onQueue('ai')
+  .send()
+
+// With success/failure callbacks
+await new ResearchAgent()
+  .queue('Research GPT-5 architecture')
+  .then(response => console.log('Done:', response.text))
+  .catch(error  => console.error('Failed:', error))
+  .send()
+```
+
+#### Stream progress to a broadcast channel — `.broadcast(channel)`
+
+Background AI work + live UI without polling. Each stream chunk is broadcast to the channel as the job runs; the final response is broadcast as a `done` event:
+
+```ts
+await new SupportAgent()
+  .queue('Help with refund request')
+  .broadcast(`user.${userId}.support`)
+  .send()
+
+// Subscribers on `user.${userId}.support` receive:
+//   { event: 'chunk', data: <StreamChunk> }   // one per stream chunk (text-delta, tool-call, ...)
+//   { event: 'done',  data: <AgentResponse> } // final result, after the loop ends
+//   { event: 'error', data: { message } }     // on failure
+```
+
+The wire shape matches the framework's normal `StreamChunk` types — the same `text-delta` / `tool-call` / `tool-result` shapes you'd iterate from `agent.stream()`. Frontends can subscribe to the channel and reuse their existing chunk-handling code.
+
+Pass `eventPrefix` to namespace events when the channel carries other unrelated messages:
+
+```ts
+.broadcast('shared-channel', { eventPrefix: 'agent.' })
+// emits 'agent.chunk', 'agent.done', 'agent.error'
+```
+
+**Process model:** `@rudderjs/broadcast`'s `broadcast()` writes to the WS server in the same process. In the typical RudderJS dev setup (single process running both web + `queue:work`) this works out of the box. Production deployments that run the queue worker as a separate process from the broadcast WS server will need a pub/sub bridge (Redis, Reverb, etc.) — outside the scope of v1.
+
 ### Conversation History
 
 Pass message history to maintain context across turns:
@@ -694,6 +813,152 @@ await new ChatAgent().prompt('Continue?')  // resumes same thread (per user + cl
 
 Returning `false` (the default) keeps the agent stateless. Async returns are awaited; an optional `historyLimit` caps loaded messages. Per-call escape hatches: `prompt(input, { conversation: false })` or `agent.forUser(id).prompt()` / `agent.continue(id).prompt()` — explicit always wins. See `docs/guide/ai.md` for the full precedence chain.
 
+### User memory beyond conversation history (Mem0-style)
+
+Conversation history persists messages; user memory persists **facts** that should travel across conversations. Useful when the agent needs to remember "Alice's project is named Foo" in a brand-new thread without replaying the entire prior session.
+
+```ts
+import type { UserMemory } from '@rudderjs/ai'
+import { MemoryUserMemory } from '@rudderjs/ai'
+
+// config/ai.ts — wire a backend
+export default {
+  default: 'anthropic/claude-sonnet-4-5',
+  providers: { /* ... */ },
+  memory: new MemoryUserMemory(),    // in-process; swap for an ORM- or embedding-backed store in production
+} satisfies AiConfig
+
+// Use it directly
+const memory = app().make<UserMemory>('ai.memory')
+await memory.remember('user_123', 'Project name is Foo', { tags: ['project'] })
+const facts = await memory.recall('user_123', 'project')
+//=> [{ fact: 'Project name is Foo', tags: ['project'], ... }]
+```
+
+Or declare on an agent class to opt into auto-inject — relevant facts get prepended to the system prompt before each turn, with no plumbing on the caller's side:
+
+```ts
+class SupportAgent extends Agent {
+  remembers() {
+    return {
+      user:               ctx.user.id,
+      inject:            'auto',          // recall + prepend matching facts before each model call
+      tags:              ['support'],     // recall scope
+      injectLimit:       5,               // cap facts per turn
+      injectTokenBudget: 400,             // hard token cap; lowest-score facts drop first
+    }
+  }
+}
+
+await new SupportAgent().prompt('Where is my project deployed?')
+// system prompt sent to the model:
+//   "You are a support agent.\n\n
+//    <user-memory>\n
+//    - Project Foo deploys to fly.io us-east\n
+//    - …\n
+//    </user-memory>"
+```
+
+The auto-cascade runs in `Agent.prompt` / `Agent.stream`, before conversation persistence. `withMemoryInject(spec)` is also exported so you can drop it into `agent.middleware()` manually if you want full control.
+
+**Continuation note:** when you pass `options.messages` (e.g. resuming after a client-tool round-trip), both auto-inject and auto-extract are skipped — the system prompt was already augmented on the original turn, and re-extracting would write the same facts twice.
+
+#### Auto-extract — distill facts from each turn
+
+Set `extract: 'auto'` (and an `extractWith` model) and a small model is asked to pull durable facts from each successful turn:
+
+```ts
+class SupportAgent extends Agent {
+  remembers() {
+    return {
+      user:        ctx.user.id,
+      inject:      'auto',
+      extract:     'auto',
+      extractWith: 'anthropic/claude-haiku-4-5',     // small model for fact distillation
+      tags:        ['support'],
+    }
+  }
+}
+
+await new SupportAgent().prompt('hey, my project is named Foo and lives at /var/www/foo')
+// On success, the small model is asked to distill durable facts. Survivors above
+// the confidence threshold (default 0.7) get written via `mem.remember()`:
+//   - "Project name is Foo"  (score ~0.95, tags: ['support', 'project'])
+```
+
+Failures (network, JSON parse, schema mismatch, store write) route through `MemoryExtractOptions.onError` and never break the parent run. Failed parent runs do NOT trigger extract.
+
+**Poisoning mitigation** — auto-extraction trusts the user's own conversation as input. The default 0.7 confidence threshold is the v1 defense against adversarial "facts." Pair with `MemoryExtractOptions.onExtracted` for an audit log when shipping to production, and tighten the threshold for high-risk domains.
+
+#### Production backend — `OrmUserMemory`
+
+For production, swap `MemoryUserMemory` for `OrmUserMemory` (subpath `@rudderjs/ai/memory-orm`) — persists rows via your registered `@rudderjs/orm` adapter (Prisma today; Drizzle once you wire the tables):
+
+```ts
+// config/ai.ts
+import type { AiConfig } from '@rudderjs/ai'
+import { OrmUserMemory } from '@rudderjs/ai/memory-orm'
+
+export default {
+  default: 'anthropic/claude-sonnet-4-5',
+  providers: { /* ... */ },
+  memory: new OrmUserMemory(),
+} satisfies AiConfig
+```
+
+Add the schema to your Prisma file (or import the reference string `userMemoryPrismaSchema` from `@rudderjs/ai/memory-orm`):
+
+```prisma
+model UserMemory {
+  id        String   @id @default(cuid())
+  userId    String
+  fact      String
+  /// JSON-encoded `string[]` of tags, or null
+  tags      String?
+  /// Confidence score in [0, 1] — extract sets this from the model's self-rating
+  score     Float?
+  /// Phase 5 — vector embedding for cosine recall (nullable so Phase 4 ignores it)
+  embedding Bytes?
+  createdAt DateTime @default(now())
+  updatedAt DateTime @updatedAt
+
+  @@index([userId])
+}
+```
+
+Then run `pnpm exec prisma db push` (dev) or `pnpm exec prisma migrate dev` (prod). The `embedding Bytes?` column is intentionally nullable — Phase 5's `EmbeddingUserMemory` populates it without forcing a follow-up migration.
+
+`OrmUserMemory.recall()` uses **OR-of-LIKE token overlap** on the `fact` column — same semantic as `MemoryUserMemory`. Tag-array filtering happens JS-side after fetch (pushing tags into the WHERE is adapter-specific; that lands in a follow-up).
+
+#### Embedding backend — `EmbeddingUserMemory` (Phase 5)
+
+For semantic recall ("Where do I deploy?" matching "Project Foo lives at fly.io"), wrap `OrmUserMemory` with `EmbeddingUserMemory` from `@rudderjs/ai/memory-embedding`:
+
+```ts
+import { OrmUserMemory } from '@rudderjs/ai/memory-orm'
+import { EmbeddingUserMemory } from '@rudderjs/ai/memory-embedding'
+
+export default {
+  default: 'anthropic/claude-sonnet-4-5',
+  providers: { /* ... */ },
+  memory: new EmbeddingUserMemory({
+    inner: new OrmUserMemory(),
+    model: 'openai/text-embedding-3-small',
+    threshold: 0.5,                    // cosine floor; matches below get dropped
+  }),
+} satisfies AiConfig
+```
+
+`remember()` embeds the fact via `AI.embed()` and writes the Float32-packed vector into the row's `embedding` column. `recall()` embeds the query and ranks all of the user's facts by **pure-JS cosine similarity** (acceptable up to a few thousand facts/user; for larger workloads, B7 lands a pgvector-backed variant).
+
+**GDPR right-to-be-forgotten cascades automatically** — the embedding lives in the same row as the fact, so `forget()` / `forgetAll()` delete both. No second store to keep in sync.
+
+**Backward compat with Phase 4:** rows persisted before `EmbeddingUserMemory` was wired in have `embedding === null`. The default `nullEmbeddingFallback: 'token-overlap'` falls back to the same token-overlap matching `MemoryUserMemory` uses, so upgrading from `OrmUserMemory` doesn't lose recall on existing rows. New `remember()` calls populate the embedding column going forward. Set `nullEmbeddingFallback: 'skip'` to drop pre-embedding rows entirely.
+
+`embed()` failures (network down, missing peer SDK) are swallowed: `remember()` still persists the entry with `embedding === null`, and `recall()` falls back to token-overlap. The parent prompt never breaks because of memory work.
+
+**A4 status (all phases shipped):** interface, in-process backend, per-call/class declaration, auto-inject, auto-extract, ORM-backed `OrmUserMemory`, and embedding-backed `EmbeddingUserMemory` all ship today. The roadmap item is complete.
+
 ### Model Selection
 
 Configure available models for user selection (used by `@rudderjs/panels` chat UI):
@@ -729,6 +994,44 @@ const loggingMiddleware: AiMiddleware = {
 }
 ```
 
+### Per-user budgets — `withBudget(...)`
+
+Cap daily or monthly spend per user. The middleware pre-debits the estimated input cost on every iteration (refusing with `BudgetExceededError` when the cap would be exceeded) and trues up the actual delta after each step's usage report:
+
+```ts
+import { withBudget, memoryBudgetStorage, BudgetExceededError } from '@rudderjs/ai'
+
+class ChatAgent extends Agent {
+  model() { return 'anthropic/claude-sonnet-4-5' }
+  middleware() {
+    return [
+      withBudget({
+        user:    () => req.user?.id ?? null,         // null bypasses enforcement (unauth)
+        budget:  { period: 'monthly', cap: 5.00 },   // USD; also 'daily'
+        storage: memoryBudgetStorage(),               // swap for ormBudgetStorage in production
+        // timezone:    'America/Los_Angeles',         // optional — period rollover boundary
+        // onExceeded:  ({ spent, cap }) => log.warn({ spent, cap }, 'budget hit'),
+        // pricing:     { ...ModelPricing, 'custom/model': { ... } },   // overrides
+      }),
+    ]
+  }
+}
+```
+
+**Production storage — `OrmBudgetStorage`** persists spend rows via your registered ORM adapter:
+
+```ts
+import { OrmBudgetStorage, BudgetUsageRecord } from '@rudderjs/ai/budget-orm'
+
+withBudget({
+  user:    () => req.user.id,
+  budget:  { period: 'monthly', cap: 25 },
+  storage: new OrmBudgetStorage(),
+})
+```
+
+Schema reference is exported as `budgetUsagePrismaSchema` from `@rudderjs/ai/budget-orm` (also lives at `playground/prisma/schema/ai.prisma`). The `@@unique([userId, period, periodKey])` index is load-bearing — it provides first-write race protection. Caveats: refunds on errors are **not** issued; cache token deltas (Anthropic ephemeral, OpenAI prefix) aren't yet exposed on `TokenUsage`, so cached requests bill at full input rate; default token estimator is `text.length / 4` (override via `estimateTokens` for `tiktoken`). Under high single-user concurrency, total spend can briefly exceed `cap` by up to `costUsd × concurrency` (R-M-W race in the cap-check). The `BudgetExceededError` bubbles up — catch it at the route boundary to return a friendly 402.
+
 ### Testing
 
 ```ts
@@ -778,6 +1081,179 @@ await new ChatAgent().prompt('again')   // throws "Stray prompt: no scripted res
 
 Under strict mode, only `respondWithSequence` entries count as valid responses; ambient `respondWith` is ignored. Force a single-step script via `respondWithSequence([{ text: '...' }])` if you want exact-one-prompt tests with content.
 
+### Evals — `@rudderjs/ai/eval`
+
+`AiFake` proves the agent's wiring works; **evals** prove the agent does the right thing on real models. Define a suite of input cases + assertions, run them against any `Agent`, get a console report with pass/fail + cost + tokens:
+
+```ts
+// evals/support-agent.eval.ts
+import { evalSuite, llmJudge, exactMatch, regex } from '@rudderjs/ai/eval'
+import { SupportAgent } from '../app/Agents/SupportAgent.js'
+
+export default evalSuite('SupportAgent', {
+  agent: () => new SupportAgent(),
+  cases: [
+    { name: 'password reset',
+      input: 'How do I reset my password?',
+      assert: llmJudge('mentions a password reset link') },
+    { name: 'price',
+      input: 'How much does this cost?',
+      assert: exactMatch('$99/month') },
+    { name: 'support email',
+      input: 'How do I contact support?',
+      assert: regex(/support@example\.com/) },
+  ],
+})
+```
+
+Run via the CLI (Phase 2):
+
+```bash
+pnpm rudder ai:eval                    # all suites under evals/**/*.eval.ts
+pnpm rudder ai:eval support            # only suites whose name includes "support"
+pnpm rudder ai:eval --bail             # stop on first failing suite
+pnpm rudder ai:eval --json             # machine-readable envelope to stdout
+```
+
+```text
+SupportAgent (3 cases, 2.3s, $0.014)
+  ✓ password reset             1.2s   $0.003   tokens: 487
+  ✓ price                      0.8s   $0.002   tokens: 312
+  ✗ support email              1.1s   $0.002   tokens: 425
+      pattern /support@example\.com/ did not match "Reach us at hello@…"
+
+  2 passed, 1 failed
+  total: $0.007  •  cumulative tokens: 1,224
+```
+
+Exits 0 when every case passes, 1 on any failure. `--json` emits `{ suites: [{ suite, passed, failed, cases: [{ name, status, pass, score?, reason?, tokens, cost, duration }] }] }` to stdout — pipe directly into `jq` for CI gates.
+
+Override the discovery pattern via `config('ai').eval.pattern` (`'evals/**/*.eval.ts'` by default; supports `<dir>/**/*<suffix>` and `*<suffix>` shapes).
+
+Or run programmatically:
+
+```ts
+import { runSuite, reportConsole, reportJson } from '@rudderjs/ai/eval'
+import suite from './evals/support-agent.eval.ts'
+
+reportConsole(await runSuite(suite))
+// reportJson(await runSuite(suite))   // structured envelope for CI scripts
+```
+
+**Built-in metrics:**
+
+| Metric | Behavior |
+|---|---|
+| `exactMatch(string)` | `response.text === expected` |
+| `regex(RegExp)` | `pattern.test(response.text)` |
+| `llmJudge(criterion, opts?)` | Asks a small model whether the response satisfies a natural-language criterion. Returns the judge's reasoning in `reason` so failures are debuggable. |
+| `jsonShape(zodSchema)` | Strips ```` ``` ```` fences, parses, runs zod `safeParse`. Surfaces the zod issue path on failure. Pairs with `Output.object({ schema })` on the agent. |
+| `semanticMatch(reference, opts?)` | Embeds reference + response via `AI.embed()`, cosine similarity vs `opts.threshold` (default `0.85`). Embed cost rolls into the case's cost rollup. Requires a provider with `createEmbedding()` (openai/google/mistral/cohere/jina). |
+| `tokenCost(threshold)` | Passes when `response.usage.totalTokens <= threshold`. Detects prompt-size regressions before they show up as a billing surprise. |
+
+`compose(...metrics)` runs them in order, short-circuits on the first failure, surfaces its reason. Useful for "must be valid JSON AND under budget" assertions:
+
+```ts
+{ input: '…',
+  assert: compose(jsonShape(SummarySchema), tokenCost(800)) }
+```
+
+User-defined metrics implement `(response, ctx) => MetricResult` — no inheritance, no decorators. The catalog is just a starting set.
+
+**Failure semantics:** the runner never throws upward. Agent errors AND assertion throws become `failed` rows with the message in `reason`. Per-case `timeout` (ms) caps long runs. Per-case `agent` factory overrides the suite default — useful for stress-testing one case against a different model.
+
+**Record + replay:**
+
+```bash
+pnpm rudder ai:eval --record support     # run live, save fixtures
+pnpm rudder ai:eval --replay support     # zero API calls, deterministic
+```
+
+`--record` runs each matching case against the real provider and writes assistant turns (text + tool calls) to `evals/__fixtures__/<suite>/<case>.json` (commit these alongside the suite for diffable regression history). `--replay` swaps the runtime with `AiFake` and feeds each case its recorded fixture — same agent code path, scripted responses. Cases without a fixture fall through to a normal run with a stderr warning. The two modes are mutually exclusive.
+
+**Telescope hook:** `aiObservers` emits an `agent.eval.completed` event after every case (passing, failing, skipped). Telescope's AI collector aggregates pass-rate per `(suite, case)` over time.
+
+**HTML report:**
+
+```bash
+pnpm rudder ai:eval --html report.html      # write a self-contained HTML report
+```
+
+Self-contained HTML (inline CSS + vanilla JS, no external assets), pasteable into PR comments / Slack threads, openable offline. Coexists with `--json` (JSON to stdout, HTML to disk). Click any case row to expand the prompt + response.
+
+Annotate suites with optional metadata:
+
+```ts
+export default evalSuite('SupportAgent', {
+  agent: () => new SupportAgent(),
+  cases: [/* … */],
+  metadata: {
+    owner:        '@billing-team',
+    lastReviewed: '2026-05-01',
+    ticket:       'AI-42',
+  },
+})
+```
+
+### MCP integration
+
+`@rudderjs/ai/mcp` bridges agents and Model Context Protocol servers in both directions. Optional peer: `@modelcontextprotocol/sdk`.
+
+```ts
+import { mcpClientTools, mcpServerFromAgent } from '@rudderjs/ai/mcp'
+```
+
+#### Consume MCP tools in an Agent — `mcpClientTools(transport, opts?)`
+
+Connect to a remote MCP server and surface its tools to an agent.
+
+```ts
+// HTTP transport
+const tools = await mcpClientTools('https://api.example.com/mcp')
+
+// Local subprocess (stdio)
+const tools = await mcpClientTools({ command: 'npx', args: ['some-mcp-server'] })
+
+// Already-connected SDK Client (caller owns lifecycle)
+const tools = await mcpClientTools(myClient)
+
+class ResearchAgent extends Agent {
+  instructions() { return 'You have access to remote tools via MCP.' }
+  tools() { return tools }
+}
+```
+
+The remote server's JSON Schema flows directly to providers via the `jsonSchema` passthrough field on `ToolDefinitionOptions` — no zod round-trip. When this connector owns the underlying client (URL or stdio transport), the returned array exposes a non-enumerable `close()` for shutdown:
+
+```ts
+const tools = await mcpClientTools('https://api.example.com/mcp')
+// ... use tools in agent ...
+await tools.close?.()
+```
+
+Options: `filter` (drop tools by name), `namePrefix` (avoid collisions across multiple servers), `streaming` (forward MCP `notifications/progress` as `tool-update` chunks; default `true`).
+
+#### Expose an Agent as an MCP server — `mcpServerFromAgent(AgentClass, opts?)`
+
+Wrap an `Agent` so external MCP clients (Claude Desktop, Cursor, etc.) can call it. Returns a `McpServer` from `@modelcontextprotocol/sdk` — connect with any SDK transport.
+
+```ts
+import { mcpServerFromAgent } from '@rudderjs/ai/mcp'
+import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'
+
+const server = await mcpServerFromAgent(ResearchAgent)
+await server.connect(new StdioServerTransport())
+```
+
+Three exposure modes via `opts.expose`:
+- `'tools'` *(default)* — one MCP tool per `agent.tools()` entry; the wrapping agent isn't called, individual tools execute directly
+- `'agent'` — one MCP tool that runs the whole agent (`prompt(text) → response.text`); the differentiator move — ship an agent, callable from any MCP-aware client
+- `'both'` — individual tools and the agent prompt-tool side by side
+
+Other options: `name`, `version`, `instructions` (defaults to `agent.instructions()`), `agentToolName` (renames the prompt-tool when `expose: 'agent' | 'both'`).
+
+Approval gates (`needsApproval: true`) are dropped on the MCP side — there's no MCP-protocol way to forward "this tool needs human approval" to a remote client. The gate fires only inside the wrapping agent, not for external MCP callers.
+
 ## Providers
 
 | Provider | SDK | Model String | Text | Embeddings | Images | TTS/STT | Reranking | Files |
@@ -787,6 +1263,8 @@ Under strict mode, only `respondWithSequence` entries count as valid responses;
 | Google | `@google/genai` | `google/gemini-2.5-pro` | ✓ | ✓ | ✓ | | | ✓ |
 | Cohere | `cohere-ai` | `cohere/rerank-v3.5` | | ✓ | | | ✓ | |
 | Jina | *(none)* | `jina/jina-reranker-v2-base-multilingual` | | ✓ | | | ✓ | |
+| VoyageAI | *(none)* | `voyage/voyage-3-large` | | ✓ | | | ✓ | |
+| ElevenLabs | *(none)* | `elevenlabs/eleven_multilingual_v2` | | | | ✓ | | |
 | Ollama | *(none)* | `ollama/llama3` | ✓ | | | | | |
 | Groq | *(none)* | `groq/llama-3.3-70b` | ✓ | | | | | |
 | DeepSeek | *(none)* | `deepseek/deepseek-chat` | ✓ | | | | | |

package/boost/guidelines.md

@@ -162,6 +162,65 @@ for await (const chunk of stream) {
 const final = await response // full AgentResponse after stream ends
 ```
 
+### MCP integration (`@rudderjs/ai/mcp`)
+
+Bridge agents and Model Context Protocol servers in both directions. Optional peer: `@modelcontextprotocol/sdk`.
+
+**Consume MCP tools in an agent:**
+
+```ts
+import { mcpClientTools } from '@rudderjs/ai/mcp'
+
+const tools = await mcpClientTools('https://api.example.com/mcp')
+// or: await mcpClientTools({ command: 'npx', args: ['some-mcp-server'] })
+
+class ResearchAgent extends Agent {
+  instructions() { return 'You can call remote MCP tools.' }
+  tools() { return tools }
+}
+```
+
+**Expose an agent as an MCP server** (callable from Claude Desktop, Cursor, etc.):
+
+```ts
+import { mcpServerFromAgent } from '@rudderjs/ai/mcp'
+import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'
+
+// Default: each agent.tools() entry becomes an MCP tool
+const server = await mcpServerFromAgent(MyAgent)
+
+// Or expose the whole agent as one prompt-tool
+const promptServer = await mcpServerFromAgent(MyAgent, { expose: 'agent' })
+
+await server.connect(new StdioServerTransport())
+```
+
+When mcpClientTools owns the underlying client (URL or stdio transport), the returned array exposes `close()` for shutdown — call it when the agent is done. With a caller-provided client, lifecycle stays with the caller.
+
+### Queued prompts + live broadcast (`agent.queue().broadcast()`)
+
+`agent.queue(input)` ships the run to the background queue (`@rudderjs/queue`). Add `.broadcast(channel)` to stream chunks to a `@rudderjs/broadcast` channel as the job runs — background AI work + live UI without polling.
+
+```ts
+import { agent } from '@rudderjs/ai'
+
+// Plain queued — no live updates
+await agent('You help with refunds.')
+  .queue('Process refund for order #1234')
+  .onQueue('ai')
+  .send()
+
+// Stream chunks to the user's channel as they arrive
+await new SupportAgent()
+  .queue('Help with refund request')
+  .broadcast(`user.${userId}.support`)
+  .send()
+```
+
+Subscribers on the channel receive `chunk` events (one per `StreamChunk`), then a `done` event with the final `AgentResponse`, or an `error` event on failure. Optional `eventPrefix` namespaces events: `.broadcast('chan', { eventPrefix: 'agent.' })` emits `agent.chunk` / `agent.done` / `agent.error`.
+
+`@rudderjs/broadcast`'s in-process WS state is process-local — same-process web + `queue:work` works out of the box; a separate worker process needs a future pub/sub bridge.
+
 ### Structured Output
 
 Use `Output` to constrain responses to typed schemas:
@@ -195,6 +254,7 @@ import { Image, Document } from '@rudderjs/ai'             // attachments
 import { MemoryConversationStore, setConversationStore } from '@rudderjs/ai'
 import { Output } from '@rudderjs/ai'                      // structured output
 import { AiRegistry } from '@rudderjs/ai'                  // provider registry
+import { mcpClientTools, mcpServerFromAgent } from '@rudderjs/ai/mcp'  // MCP bridge (Node)
 import { stepCountIs, hasToolCall } from '@rudderjs/ai'    // stop conditions
 import type { AgentResponse, AiConfig, AiMiddleware, AnyTool, HasTools, HasMiddleware } from '@rudderjs/ai'
 ```

package/boost/skills/ai-agents/SKILL.md

@@ -1,6 +1,13 @@
 ---
 name: ai-agents
 description: Building AI agents with tools, streaming, conversation memory, approval flows, and middleware in RudderJS
+license: MIT
+appliesTo:
+  - '@rudderjs/ai'
+trigger: building an `Agent` class, calling `Agent.prompt()`/`.stream()`, defining `tools()` or `middleware()`, or wiring conversations / failover
+skip: writing a tool definition by itself — load `ai-tools` instead
+metadata:
+  author: rudderjs
 ---
 
 # AI Agents

package/boost/skills/ai-tools/SKILL.md

@@ -1,6 +1,13 @@
 ---
 name: ai-tools
 description: Defining server and client tools with Zod schemas, approval gates, streaming yields, and modelOutput for RudderJS AI agents
+license: MIT
+appliesTo:
+  - '@rudderjs/ai'
+trigger: writing a `toolDefinition()`, defining server or client tools, adding streaming yields, or wiring approval gates
+skip: configuring an `Agent` class itself — load `ai-agents` instead
+metadata:
+  author: rudderjs
 ---
 
 # AI Tools