llm-agent-runtime 0.1.6 → 0.1.16

package/USAGE.md

@@ -4,28 +4,405 @@ Reusable LangGraph agent engine. Your app supplies **intents, prompts, and tools
 
 `resolveFlow → runAgent → extract tool outputs → finalize`
 
-Same shape as Xel-agent, but flexible for any product.
+RAG stays in **your app**: index docs, retrieve chunks, inject via `context` / `buildSystemPrompt`.
+
+**Where to add RAG docs:** put paths/URLs in `.env.local` as `RAG_SOURCES` (see [Example B](#example-b--rag-chatbot-matches-plain-agent-demo)). Prefer `.md` / `.docx` (some PDFs fail with `bad XRef entry`).
+
+```bash
+RAG_SOURCES=doc/faq.md,doc/handbook.docx,https://cdn.example.com/policy.docx
+```
+
+Index **once** with `getRagStore()`; each message only runs `retrieve()` (query embed + top-K). Example B matches `plain-agent-demo`.
 
 ---
 
 ## Install
 
 ```bash
-npm install llm-agent-runtime
-# or
-pnpm add llm-agent-runtime
+pnpm add llm-agent-runtime @langchain/core @langchain/langgraph @langchain/langgraph-checkpoint zod
+```
+
+**Optional (recommended for production without Ollama):**
+
+```bash
+pnpm add @xenova/transformers   # free in-process embeddings
+pnpm add langsmith              # tracing
 ```
 
-**Peer dependencies** (usually already in a LangChain app):
+If `@xenova/transformers` fails on `sharp` (pnpm ignored build scripts):
 
-- `@langchain/core`
-- `@langchain/langgraph`
-- `@langchain/langgraph-checkpoint`
-- `zod`
+```bash
+cd node_modules/sharp && npm run install
+```
 
 ---
 
-## Quick start
+## Configure a project (`.env`)
+
+Minimal **chatbot** on OpenRouter (free embeddings via Xenova):
+
+```bash
+# Chat
+LLM_PROVIDER=openrouter
+OPENROUTER_API_KEY=sk-or-v1-...
+OPENROUTER_MODEL=meta-llama/llama-3.1-70b-instruct
+OPENROUTER_INTENT_ROUTER_MODEL=meta-llama/llama-3.1-70b-instruct
+
+# Embeddings — omit EMBEDDINGS_PROVIDER → xenova when chat is openrouter
+EMBEDDINGS_PROVIDER=xenova
+# XENOVA_EMBEDDINGS_MODEL=Xenova/all-MiniLM-L6-v2
+
+# Optional Redis (flow state / cache). Plain Redis works for cache;
+# Redis Stack (RedisJSON) needed for durable LangGraph checkpoints.
+REDIS_URL=redis://localhost:6379
+
+# Optional LangSmith
+LANGSMITH_TRACING=true
+LANGSMITH_API_KEY=lsv2_pt_...
+LANGSMITH_PROJECT=my-agent
+```
+
+**Embeddings options**
+
+| `EMBEDDINGS_PROVIDER` | Cost | When to use |
+| --- | --- | --- |
+| `xenova` | Free | Default for OpenRouter/OpenAI chat when unset |
+| `ollama` | Free (self-host) | Default when `LLM_PROVIDER=ollama` |
+| `openai` | Paid | Strongest cloud embeddings |
+
+---
+
+## Suggested app layout
+
+```
+your-app/
+  .env / .env.local
+  agent/
+    runtime.ts       # createAgentRuntime({ ... })
+    intents.ts       # intents + menuMap + aliases
+    prompts/         # system / intent prompts
+    tools/           # LangChain tools (APIs, DB, …)
+  rag/               # optional — or skip and use getRagStore() from the package
+  controllers/       # HTTP / WhatsApp → retrieve then invoke
+```
+
+Put document paths/URLs in `.env` as `RAG_SOURCES=...` (see Example B). No custom `rag/store.ts` required — import `getRagStore` from `llm-agent-runtime`.
+
+---
+
+## Example A — Normal chatbot (no RAG)
+
+Intent routing + tools + sticky flows. No vector store.
+
+```ts
+// agent/runtime.ts
+import {
+  createAgentRuntime,
+  llmConfigFromEnv,
+} from "llm-agent-runtime";
+import { tool } from "@langchain/core/tools";
+import { z } from "zod";
+
+const greet = tool(async () => "Welcome! Reply 1 to recharge or 2 to login.", {
+  name: "show_menu",
+  description: "Show the main menu",
+  schema: z.object({}),
+});
+
+const recharge = tool(
+  async ({ amount, phone }) => ({ ok: true, amount, phone }),
+  {
+    name: "buy_airtime",
+    description: "Buy airtime for a phone number",
+    schema: z.object({
+      amount: z.number(),
+      phone: z.string(),
+    }),
+  },
+);
+
+export async function getRuntime() {
+  const llm = llmConfigFromEnv(); // reads LLM_PROVIDER, EMBEDDINGS_PROVIDER, …
+
+  const runtime = await createAgentRuntime({
+    redisUrl: process.env.REDIS_URL,
+    llm,
+    tracing: {
+      enabled: Boolean(process.env.LANGSMITH_API_KEY),
+      projectName: process.env.LANGSMITH_PROJECT ?? "my-agent",
+    },
+
+    intents: [
+      {
+        name: "greetings",
+        examples: ["hi", "hello", "menu", "help"],
+        description: "greetings or main menu",
+      },
+      {
+        name: "recharge",
+        examples: ["buy airtime", "recharge my phone", "buy data"],
+        sticky: true,
+        description: "buy airtime or data",
+        regex: /(buy|recharge).*(airtime|data)/i,
+      },
+    ],
+
+    menuMap: { "1": "recharge" },
+
+    intentRegistry: {
+      greetings: {
+        prompt: () =>
+          "You are a helpful recharge assistant. Greet the user and show menu options.",
+        tools: () => [greet],
+      },
+      recharge: {
+        prompt: () =>
+          "Collect amount and phone, then call buy_airtime when ready.",
+        tools: () => [recharge],
+      },
+    },
+
+    toolOutputMap: { buy_airtime: "transaction" },
+    completeOn: ["buy_airtime"],
+  });
+
+  await runtime.connect();
+  return runtime;
+}
+```
+
+```ts
+// controllers/chat.ts
+const runtime = await getRuntime();
+
+const result = await runtime.invoke({
+  message: req.body.message,
+  threadId: req.body.phoneNumber, // conversation id
+  phoneKey: req.body.phoneNumber,
+  context: { userInfo: req.body.user ?? {} },
+});
+
+res.json({
+  message: result.answer,
+  intent: result.intent,
+  flow: result.flow,
+  toolOutputs: result.toolOutputs,
+});
+```
+
+---
+
+## Example B — RAG chatbot (matches `plain-agent-demo`)
+
+Point the app at **local files and/or http(s) URLs**, **index once**, then **retrieve per message**.
+Document chunking + embeddings are **not** rebuilt on every chat — only the user query is embedded each time.
+
+### Layout (what each file does)
+
+```
+your-app/
+  .env.local              # RAG_SOURCES + LLM + EMBEDDINGS_PROVIDER
+  index.ts                # load .env.local FIRST, mount POST /chat-bot/rag-chatbot
+  src/
+    routers/index.ts      # route → ragChatBotController
+    controllers/index.ts  # retrieve snippets → invoke RAG runtime
+    rag/
+      store.ts            # optional warm-up (getRagStore lives in the package)
+      runtime.ts          # createAgentRuntime + buildSystemPrompt (RAG Q&A)
+```
+
+| Piece | What it does |
+|-------|----------------|
+| `RAG_SOURCES` | Comma-separated paths/URLs to index |
+| `getRagStore()` | Singleton: load → chunk → embed docs **once** per process |
+| `store.retrieve(msg, 4)` | Embed the **query only**, return top-K text snippets |
+| `buildSystemPrompt` | Puts snippets into the system prompt as `Knowledge:` |
+| `getRagRuntime()` | Separate agent (no tools); answers from Knowledge only |
+| `explicitIntent: "qa"` | Skip intent-router LLM (faster, deterministic) |
+
+### 1) Env — documents + embeddings
+
+```bash
+# .env.local — `import "dotenv/config"` alone only loads `.env`, not `.env.local`
+EMBEDDINGS_PROVIDER=xenova
+
+# Prefer .md / .docx (some PDFs fail pdf.js with "bad XRef entry")
+RAG_SOURCES=doc/faq.md,doc/handbook.docx,https://cdn.example.com/policy.docx
+
+# Multiple files = comma-separated
+# RAG_SOURCES=/abs/a.docx,/abs/b.md
+```
+
+```bash
+pnpm add pdf-parse mammoth @xenova/transformers
+```
+
+```ts
+// index.ts — load env BEFORE getRagStore / createAgentRuntime
+import dotenv from "dotenv";
+dotenv.config({ path: ".env.local" });
+dotenv.config(); // optional `.env` fallback (won't override keys already set)
+```
+
+### 2) Optional warm-up (`src/rag/store.ts`)
+
+```ts
+import { getRagStore, getRagDocumentCount } from "llm-agent-runtime";
+
+/** Optional: index at startup so the first HTTP request is faster. */
+export async function warmRagStore() {
+  const store = await getRagStore(); // reads RAG_SOURCES; chunks + embeds once
+  console.log(`[rag] ready — ${getRagDocumentCount()} document(s) indexed`);
+  return store;
+}
+
+export { getRagStore, getRagDocumentCount };
+```
+
+- **`getRagStore()`** — lazy singleton. First await indexes; later awaits reuse memory.
+- **`resetRagStore()`** — clear singleton (tests / re-index after changing sources).
+- Overrides: `getRagStore({ embeddings, env, chunkSize, overlap, log: false })`.
+- Unreadable sources are **skipped with a warning**; if none load, it throws (next call can retry).
+
+### 3) RAG agent runtime (`src/rag/runtime.ts`)
+
+```ts
+import {
+  createAgentRuntime,
+  llmConfigFromEnv,
+  withGeneralIntents, // or defineIntents([...]) without the general catch-all
+  type AgentRuntime,
+} from "llm-agent-runtime";
+
+export type RagContext = {
+  ragContext?: string;
+  userInfo?: Record<string, unknown>;
+};
+
+export const buildSystemPrompt = (ctx: { context?: RagContext }) => {
+  const rag = String(ctx.context?.ragContext ?? "").trim();
+  const base =
+    "Answer using ONLY the knowledge snippets below. If missing, say you don't know.";
+  return rag ? `${base}\n\nKnowledge:\n${rag}` : base;
+};
+
+let ragPromise: Promise<AgentRuntime<RagContext>> | null = null;
+
+export async function getRagRuntime(): Promise<AgentRuntime<RagContext>> {
+  if (!ragPromise) {
+    ragPromise = (async () => {
+      // Unified intent: name/examples/description + prompt/tools in ONE object
+      const { intents, intentRegistry } = withGeneralIntents([
+        {
+          name: "qa",
+          examples: ["what is", "how do I", "explain", "tell me about", "question"],
+          description: "answer questions from indexed documents",
+          prompt: () =>
+            "Answer the user question from the knowledge provided. Be concise.",
+          tools: () => [],
+        },
+      ]);
+
+      const runtime = await createAgentRuntime<RagContext>({
+        // omit redisUrl for RAG Q&A
+        llm: llmConfigFromEnv(),
+        intents,
+        intentRegistry,
+        defaultIntent: "qa",
+        buildSystemPrompt,
+        recursionLimit: 4,
+      });
+
+      await runtime.connect();
+      return runtime;
+    })();
+  }
+  return ragPromise;
+}
+```
+
+`withGeneralIntents` / `defineIntents` split that one object into the two lists the runtime still uses internally (`intents` for routing, `intentRegistry` for prompt/tools).
+
+### 4) Controller — retrieve then invoke
+
+```ts
+import { getRagStore } from "llm-agent-runtime";
+import { getRagRuntime } from "../rag/runtime.js";
+
+export const ragChatBotController = async (req, res) => {
+  const { message, phoneNumber } = req.body;
+  if (!message || !phoneNumber) {
+    return res.status(400).json({ error: "message and phoneNumber are required" });
+  }
+
+  try {
+    const threadId = String(req.body.sessionId ?? phoneNumber);
+
+    // Singleton store (indexes once). retrieve() embeds ONLY the user message.
+    const store = await getRagStore();
+    const ragContext = await store.retrieve(String(message), 4);
+
+    const runtime = await getRagRuntime();
+    const result = await runtime.invoke({
+      message: String(message),
+      threadId,
+      phoneKey: threadId,
+      explicitIntent: "qa", // skip intent-router LLM
+      context: {
+        ragContext, // consumed by buildSystemPrompt
+        userInfo: req.body.user ?? {},
+      },
+    });
+
+    res.json({
+      message: result.answer,
+      intent: result.intent,
+      ragChars: ragContext.length,
+    });
+  } catch (err) {
+    console.error("[rag] chat failed:", err);
+    res.status(500).json({
+      error: err instanceof Error ? err.message : "RAG chat failed",
+    });
+  }
+};
+```
+
+```ts
+chatBotRouter.post("/rag-chatbot", ragChatBotController);
+// POST /chat-bot/rag-chatbot  { "message": "...", "phoneNumber": "..." }
+```
+
+### Flow (once vs per message)
+
+```
+Startup / first getRagStore():
+  RAG_SOURCES=doc/a.md,doc/b.docx
+      → read/download files
+      → extract text (.md utf8, .docx mammoth, .pdf pdf-parse)
+      → chunk (default 800 / overlap 120) + embed each chunk
+      → keep in memory (singleton)
+
+Each POST /chat-bot/rag-chatbot:
+  store.retrieve(userMessage, 4)   // embed query only + cosine top-K
+      → invoke({ explicitIntent: "qa", context: { ragContext } })
+      → buildSystemPrompt injects Knowledge
+      → LLM answers from snippets only
+```
+
+### Tips / pitfalls
+
+| Issue | Fix |
+|-------|-----|
+| `No RAG sources configured` | Set `RAG_SOURCES` in `.env.local` and load it |
+| Request hangs forever | Omit `redisUrl` on the RAG runtime |
+| `bad XRef entry` on PDF | Prefer `.docx` / `.md`; bad PDFs are skipped with a warning |
+| First chat slow | Call `warmRagStore()` at startup |
+| Re-index after changing files | `resetRagStore()` then `getRagStore()` (or restart) |
+
+---
+
+## Quick start (shortest)
 
 ```ts
 import { createAgentRuntime } from "llm-agent-runtime";
@@ -39,37 +416,20 @@ const greet = tool(async () => "ok", {
 });
 
 const runtime = await createAgentRuntime({
-  redisUrl: process.env.REDIS_URL, // omit → in-memory checkpointer
-  // llm omitted → reads process.env (see Env vars below)
-
+  redisUrl: process.env.REDIS_URL,
   intents: [
     {
       name: "greetings",
-      examples: ["hi", "hello", "good morning"],
-      description: "hello or small talk",
-    },
-    {
-      name: "create_campaign",
-      examples: ["create a campaign", "start fundraising"],
-      sticky: true,
-      description: "new campaign wizard",
+      examples: ["hi", "hello"],
+      description: "hello",
     },
   ],
-
   intentRegistry: {
     greetings: {
       prompt: (ctx) =>
-        `Greet the user. Name: ${ctx.context?.contactName ?? "friend"}.`,
+        `Greet ${ctx.context?.contactName ?? "friend"}.`,
       tools: () => [greet],
     },
-    create_campaign: {
-      prompt: () => "Help create a campaign step by step. Call create_campaign when ready.",
-      tools: (phoneKey) => [/* your tools scoped by phoneKey */],
-    },
-  },
-
-  toolOutputMap: {
-    create_campaign: "campaign",
   },
 });
 
@@ -81,7 +441,7 @@ const result = await runtime.invoke({
   context: { contactName: "Ada" },
 });
 
-console.log(result.answer, result.intent, result.flow, result.toolOutputs);
+console.log(result.answer, result.intent);
 ```
 
 ---
@@ -91,178 +451,121 @@ console.log(result.answer, result.intent, result.flow, result.toolOutputs);
 | You configure | Library handles |
 | --- | --- |
 | `intents` (examples + `sticky` + `regex`) | Hybrid regex + LLM + embedding router |
-| `intentRegistry` (prompt + tools) | LangGraph agent + Redis/memory checkpointer |
-| `toolOutputMap` | Parse tool calls into structured outputs |
-| `completeOn` | Sticky-flow completion + Redis flow state |
-| `menuMap` (optional) | Digit → intent after a numbered menu |
+| `intentRegistry` (prompt + tools) | LangGraph agent + checkpointer |
+| `toolOutputMap` / `completeOn` | Tool output parse + sticky-flow end |
+| `menuMap` | Digit → intent |
+| RAG retrieve + inject via `context` | App calls `getRagStore()` / `store.retrieve` |
 | Prompts, API tools, webhooks | — stay in your app |
 
 ---
 
 ## Config reference
 
-### `createAgentRuntime(config)`
-
-#### Required
+### Required
 
 ```ts
 intents: IntentDefinition[]
 intentRegistry: Record<string, { prompt: (ctx) => string; tools: (phoneKey) => Tool[] }>
 ```
 
-#### Common optional
+### Common optional
 
 | Option | Default | Purpose |
 | --- | --- | --- |
-| `redisUrl` | none (MemorySaver) | Checkpointer + flow state + optional answer cache |
-| `llm` | `llmConfigFromEnv()` | Provider / models |
+| `redisUrl` | MemorySaver | Checkpointer + flow state + optional answer cache |
+| `llm` | `llmConfigFromEnv()` | Provider / models / embeddings |
 | `cache.enabled` | `false` | Exact-match answer cache (needs Redis) |
 | `toolOutputMap` | raw tool args | Map tool name → output key |
-| `completeOn` | keys of `toolOutputMap` | Tool names that finish a sticky flow |
-| `menuMap` | `{}` | `{ "1": "create_campaign" }` for digit menus |
-| `intentAliases` | `{}` | Legacy name → catalog name |
-| `defaultIntent` | first intent | Fallback intent |
-| `buildSystemPrompt` | registry `prompt` | One builder for all intents |
-| `resolvePhoneKey` | `phoneKey ?? threadId` | Key passed to `tools(phoneKey)` |
-| `tracing` | auto-detect | LangSmith tracing config |
+| `completeOn` | keys of `toolOutputMap` | Tools that finish a sticky flow |
+| `menuMap` | `{}` | `{ "1": "recharge" }` |
+| `buildSystemPrompt` | registry `prompt` (+ `sharedPrompt`) | One builder (use this for RAG) |
+| `sharedPrompt` | none | Prepended to **every** intent prompt (emojis, tone, …) |
+| `tracing` | auto if `LANGSMITH_API_KEY` | LangSmith |
 
 ---
 
 ## Intents
 
-```ts
-intents: [
-  {
-    name: "greetings",
-    examples: ["hi", "hello"],
-    description: "Shown to the LLM router",
-  },
-  {
-    name: "create_campaign",
-    examples: ["create a campaign"],
-    sticky: true,    // keep this flow across turns until completeOn fires
-    regex: /create.*(campaign|fundrais)/i,  // fast-path: skip LLM if matched
-  },
-]
-```
-
-- **examples** — embedding routing
-- **sticky** — multi-step wizards
-- **description** — LLM intent router
-- **regex** — fast-path: if message matches, skip LLM/embeddings entirely (zero tokens)
-
-### Intent detection flow
-
-```
-1. explicitIntent passed?     → Use it directly (score 1.0)
-2. Wizard input + sticky?     → Reuse cached intent (skip LLM)
-3. Menu digit match?          → Use mapped intent (score 1.0)
-4. Regex matches?             → Use matched intent (score 1.0, 0 tokens)
-5. Embeddings + LLM (hybrid)  → Existing flow
-```
-
-The regex fast-path is a token-saving optimization. When a user message matches an intent's regex pattern, the library returns that intent immediately without calling the LLM or embeddings. Define regexes for high-confidence, pattern-based intents (e.g. "generate an image of...") and leave regex undefined for ambiguous intents that need LLM classification.
-
----
-
-## Prompts & tools (`intentRegistry`)
-
-```ts
-intentRegistry: {
-  greetings: {
-    prompt: (ctx) => string,
-    tools: (phoneKey) => DynamicStructuredTool[],
-  },
-}
-```
-
-`prompt` receives:
-
 ```ts
 {
-  hasConversationHistory: boolean
-  intent: string
-  intentScore?: number
-  message?: string
-  context?: Record<string, unknown>  // whatever you passed to invoke()
+  name: "recharge",
+  examples: ["buy airtime"],     // embedding routing
+  sticky: true,                  // multi-step wizard
+  description: "buy airtime",    // LLM router
+  regex: /buy.*airtime/i,        // fast-path, 0 tokens
 }
 ```
 
-Or use one builder for everything:
+Detection order: `explicitIntent` → sticky wizard skip → `menuMap` → `regex` → embeddings + LLM.
 
-```ts
-buildSystemPrompt: (ctx) => combineGeneralAndIntent(ctx)
-```
+**Fallback:** `defaultIntent` (optional) — defaults to the first intent in your list, else `"greetings"`.
 
----
+### Optional general intent (Xel-style catch-all)
 
-## Tool outputs (`toolOutputMap`)
-
-Replaces hand-written extractors:
+Not required. If you want a first catch-all (hello / help / unclear → general chat):
 
 ```ts
-toolOutputMap: {
-  create_campaign: "campaign",
-  buy_airtime: "buyAirtime",
-  // Nested field on args + seed:
-  propose_tickets: { as: "tickets", from: "tickets", initial: [] },
-}
-```
-
-Result lands on `result.toolOutputs` and completed keys on `result.extras`.
-
----
-
-## Flow completion (`completeOn`)
+import {
+  createAgentRuntime,
+  withGeneralIntent,
+} from "llm-agent-runtime";
 
-Pass tool **names**. When any of them produces output, the sticky flow ends:
+const { intents, intentRegistry } = withGeneralIntent(
+  [{ name: "recharge", examples: ["buy airtime"], sticky: true }],
+  {
+    recharge: {
+      prompt: () => "Collect amount and phone…",
+      tools: () => [buyAirtime],
+    },
+  },
+  // Optional: match Xel naming
+  // { intent: { name: "greetings" } },
+);
 
-```ts
-completeOn: ["create_campaign", "buy_airtime", "add_airtime"]
+const runtime = await createAgentRuntime({
+  intents,
+  intentRegistry,
+  defaultIntent: intents[0]!.name, // "general" (or "greetings")
+});
 ```
 
-If you omit `completeOn`, the library defaults `completeOn` to the keys of `toolOutputMap`.
-
-This is tuned to Xel-style wizards: when a sticky flow is already active and the user message looks like wizard input (digits / yes-no), the router LLM call is skipped for that turn to reduce tokens.
-
----
+Helpers: `createGeneralIntent`, `createGeneralIntentBinding`, `GENERAL_INTENT_NAME`.
+If you already define `greetings` / `general` yourself, skip these — `withGeneralIntent` is a no-op when that name exists.
 
-## Menu digits (`menuMap`)
+### Shared prompt (rules for every intent)
 
-Optional. Needed only if users pick a numbered menu (`1`, `2`, …).
+`withGeneralIntent` is only a **catch-all routing intent**. It does **not** inject style rules into recharge / other flows.
 
-The **prompt** shows the menu text. **`menuMap`** tells the **router** that `"1"` means `create_campaign`. Without it, a bare `"1"` is not reliably routed.
+For instructions that should apply to **all** intents (emojis, tone, language), use optional `sharedPrompt` — the package prepends it to whichever intent prompt is active:
 
 ```ts
-menuMap: {
-  "1": "create_campaign",
-  "2": "support_campaign",
-  "3": "buy_airtime_and_data",
-}
+const runtime = await createAgentRuntime({
+  intents,
+  intentRegistry,
+  sharedPrompt:
+    "Always use friendly emojis. Keep replies concise and warm.",
+  // or: sharedPrompt: (ctx) => `User: ${ctx.context?.userInfo?.name ?? "friend"}`,
+});
 ```
 
+You do **not** need to repeat those rules in every `intentRegistry.*.prompt`.
+
 ---
 
-## Invoke
+## Invoke & result
 
 ```ts
-const result = await runtime.invoke({
+await runtime.invoke({
   message: "hi",
-  threadId: "whatsapp:234...",   // conversation / checkpoint id
-  phoneKey: "234...",            // optional; passed to tools()
-  explicitIntent: "greetings",   // optional; skips hybrid router
-  context: {
-    contactName: "Ada",
-    userInfo: { email: "ada@example.com" },
-  },
+  threadId: "whatsapp:234...",
+  phoneKey: "234...",
+  explicitIntent: "greetings", // optional
+  context: { contactName: "Ada", ragContext: "..." },
 });
 ```
 
-**Result**
-
 ```ts
 {
-  messages: unknown[]
   answer?: string
   intent: { intent: string, score: number }
   flow: { active: boolean, intent: string | null }
@@ -270,167 +573,85 @@ const result = await runtime.invoke({
   flowCompleted: boolean
   extras?: Record<string, unknown>
   router?: { action: "continue" | "switch" | "cancel", reason?: string }
+  usage?: {
+    turn:  { inputTokens, outputTokens, totalTokens }  // this invoke
+    flow:  { …, intent }                               // sticky flow bucket
+    total: { inputTokens, outputTokens, totalTokens }  // whole thread
+  }
 }
 ```
 
----
+Token totals are stored in **Redis** when `redisUrl` is set (`usage:thread:{id}`, `usage:flow:{id}:{intent}`), otherwise in memory.
 
-## Lifecycle
+### Clear session (frontend)
 
 ```ts
-await runtime.connect()       // Redis (if redisUrl set)
-await runtime.invoke(...)
-await runtime.clearFlow(id)   // drop sticky flow state
-await runtime.disconnect()
+// Clears checkpoint history + sticky flow + token usage for this threadId
+await runtime.clearSession(threadId);
+// aliases: clearThread(threadId)  |  clearFlow(threadId) only sticky state
+const total = await runtime.getUsage(threadId);
 ```
 
----
-
-## LLM Providers
+Example HTTP for the UI “New chat / Clear session” button:
 
-### Supported providers
+```ts
+// DELETE /session  body: { sessionId }
+await runtime.clearSession(req.body.sessionId);
+res.json({ ok: true });
+```
 
-| Provider | `LLM_PROVIDER` | API Key Env Var | Default Model |
-| --- | --- | --- | --- |
-| Ollama | `ollama` (default) | — | `qwen2.5:7b` |
-| OpenAI | `openai` | `OPENAI_API_KEY` | `gpt-4o-mini` |
-| OpenRouter | `openrouter` | `OPENROUTER_API_KEY` | `meta-llama/llama-4-scout` |
+```ts
+await runtime.connect()
+await runtime.clearFlow(threadId)
+await runtime.clearSession(threadId)
+await runtime.disconnect()
+```
 
-### Env vars
+---
 
-| Variable | Notes |
-| --- | --- |
-| `LLM_PROVIDER` | `ollama` (default), `openai`, or `openrouter` |
-| `EMBEDDINGS_PROVIDER` | defaults to `LLM_PROVIDER` |
-| `OLLAMA_MODEL` | default `qwen2.5:7b` |
-| `OLLAMA_BASE_URL` | e.g. `http://localhost:11434` |
-| `OLLAMA_EMBEDDINGS_MODEL` | default `mxbai-embed-large:latest` |
-| `INTENT_ROUTER_MODEL` | optional smaller router model (ollama) |
-| `OPENAI_API_KEY` | required for openai |
-| `OPENAI_MODEL` | default `gpt-4o-mini` |
-| `OPENAI_EMBEDDINGS_MODEL` | default `text-embedding-3-small` |
-| `OPENROUTER_API_KEY` | required for openrouter |
-| `OPENROUTER_MODEL` | default `meta-llama/llama-4-scout` |
-| `OPENROUTER_BASE_URL` | default `https://openrouter.ai/api/v1` |
-| `OPENROUTER_INTENT_ROUTER_MODEL` | optional separate router model |
-| `REDIS_URL` | your app usually passes this as `redisUrl` |
-| `ANSWER_CACHE` | app convention: `"1"` to enable cache |
-
-### OpenRouter example
+## LLM & embeddings
 
-```bash
-LLM_PROVIDER=openrouter OPENROUTER_API_KEY=sk-or-... pnpm dev
-```
+| Chat `LLM_PROVIDER` | Key | Default model |
+| --- | --- | --- |
+| `ollama` | — | `qwen2.5:7b` |
+| `openai` | `OPENAI_API_KEY` | `gpt-4o-mini` |
+| `openrouter` | `OPENROUTER_API_KEY` | `meta-llama/llama-4-scout` |
 
-Or pass explicitly:
+| `EMBEDDINGS_PROVIDER` | Default model |
+| --- | --- |
+| `xenova` (default for openrouter/openai when unset) | `Xenova/all-MiniLM-L6-v2` |
+| `ollama` (default for ollama chat when unset) | `mxbai-embed-large:latest` |
+| `openai` | `text-embedding-3-small` |
 
 ```ts
 createAgentRuntime({
   llm: {
     provider: "openrouter",
-    model: "meta-llama/llama-4-scout",
-    apiKey: "sk-or-...",
+    model: "meta-llama/llama-3.1-70b-instruct",
+    apiKey: process.env.OPENROUTER_API_KEY,
+    embeddingsProvider: "xenova",
+    embeddingsModel: "Xenova/all-MiniLM-L6-v2",
   },
   // ...
 });
 ```
 
-### LangSmith tracing
-
-Tracing auto-enables when `LANGSMITH_API_KEY` is set in the environment. No code changes needed.
+### LangSmith
 
 ```bash
-LANGSMITH_API_KEY=ls_... LLM_PROVIDER=openrouter OPENROUTER_API_KEY=sk-or-... pnpm dev
-```
-
-Or enable explicitly:
-
-```ts
-createAgentRuntime({
-  tracing: { enabled: true, projectName: "my-project" },
-  // ...
-});
+LANGSMITH_TRACING=true
+LANGSMITH_API_KEY=lsv2_pt_...
+LANGSMITH_PROJECT=my-agent
 ```
 
-Tracing sets `LANGCHAIN_TRACING_V2`, `LANGCHAIN_API_KEY`, and `LANGCHAIN_PROJECT` env vars before model creation. The `langsmith` package must be installed (`pnpm add langsmith`).
+Or `tracing: { enabled: true, projectName: "my-agent" }`.
 
 ---
 
-## Minimal app shape (recommended)
-
-```
-your-app/
-  agent/
-    runtime.ts      # createAgentRuntime({ ... })
-    intents.ts      # intents + menuMap + aliases
-    prompts/        # prompt strings
-    tools/          # LangChain tools / API clients
-  controllers/      # HTTP / WhatsApp — call runtime.invoke()
-```
-
-Keep channel adapters and product APIs **out** of this package.
-
----
-
-## Full example (Xel-like)
-
-```ts
-import { createAgentRuntime } from "llm-agent-runtime";
-
-const runtime = await createAgentRuntime({
-  redisUrl: process.env.REDIS_URL ?? "redis://localhost:6379",
-  cache: { enabled: process.env.ANSWER_CACHE === "1" },
-
-  intents: [
-    { name: "greetings", examples: ["hi", "hello"] },
-    {
-      name: "create_campaign",
-      examples: ["create a campaign"],
-      sticky: true,
-      regex: /create.*(campaign|fundrais)/i,
-    },
-    {
-      name: "buy_airtime_and_data",
-      examples: ["buy airtime", "vend airtime"],
-      sticky: true,
-      regex: /(buy|vend|purchase).*(airtime|data|credit)/i,
-    },
-  ],
-
-  menuMap: {
-    "1": "create_campaign",
-    "4": "buy_airtime_and_data",
-  },
-
-  intentAliases: {
-    buy_airtime: "buy_airtime_and_data",
-  },
-
-  intentRegistry: {
-    greetings: {
-      prompt: (ctx) => `Greet ${ctx.context?.contactName ?? "there"}. Show menu 1–7.`,
-      tools: () => [],
-    },
-    create_campaign: {
-      prompt: () => "Walk through campaign creation.",
-      tools: (phoneKey) => createCampaignTools(phoneKey),
-    },
-    buy_airtime_and_data: {
-      prompt: () => "Help buy airtime/data.",
-      tools: (phoneKey) => airtimeTools(phoneKey),
-    },
-  },
+## Redis note
 
-  toolOutputMap: {
-    create_campaign: "campaign",
-    buy_airtime: "buyAirtime",
-    add_airtime: "addAirtime",
-  },
-  completeOn: ["create_campaign", "buy_airtime", "add_airtime"],
-});
-
-await runtime.connect();
-```
+- **Flow cache / answer cache / token usage:** plain Redis is fine.
+- **Durable LangGraph checkpoints:** need **Redis Stack** (RedisJSON). Without it the package falls back to an in-memory checkpointer and logs a warning.
 
 ---
 
@@ -440,9 +661,27 @@ await runtime.connect();
 import {
   createAgentRuntime,
   llmConfigFromEnv,
+  resolveEmbeddingsProvider,
+  XenovaEmbeddings,
+  // Optional catch-all intent
+  withGeneralIntent,
+  withGeneralIntents,
+  defineIntents,
+  createGeneralIntent,
+  createGeneralIntentBinding,
+  // RAG (path + URL)
+  getRagStore,
+  getRagDocumentCount,
+  resetRagStore,
+  ragSourcesFromEnv,
+  createSimpleRagFromEnv,
+  createSimpleRagFromSources,
   getLastMessageContent,
-  // advanced (usually not needed)
-  createToolOutputExtractor,
-  toolOutputMapToExtractors,
 } from "llm-agent-runtime";
 ```
+
+---
+
+## Version
+
+Current local package: **0.1.16** (`defineIntents` / `withGeneralIntents` unified intent objects, `getRagStore` singleton, Example B aligned with `plain-agent-demo`, Xenova embeddings, token usage + `clearSession`).