@theokit/sdk 1.7.0 → 1.8.0

package/claude-template/dot-claude/skills/theokit-gateways/SKILL.md

@@ -0,0 +1,209 @@
+---
+user-invocable: false
+description: Gateway architecture and 10 platform adapters (Telegram, Slack, Discord, WhatsApp, Teams, Email, SMS, Mattermost, LINE, Matrix).
+paths:
+  - "**/*gateway*"
+  - "**/*Gateway*"
+  - "**/*telegram*"
+  - "**/*slack*"
+  - "**/*discord*"
+  - "**/*whatsapp*"
+---
+
+# TheoKit SDK -- Gateways
+
+Quick reference for the gateway architecture and all 10 platform adapters.
+
+## Architecture
+
+The gateway system uses a base adapter pattern. Each platform adapter extends `BasePlatformAdapter` from `@theokit/gateway`. The core package defines:
+
+- `BasePlatformAdapter` -- abstract class with `connect`, `disconnect`, `sendMessage`, `onInbound`.
+- `GatewayMessageEvent` -- discriminated union of all platform inbound events (keyed by `platform`).
+- `OutboundMessage` -- what `sendMessage` accepts.
+- `SendResult` -- `{ ok, messageId?, error? }`.
+- Session router for multi-platform agent dispatch.
+
+## BasePlatformAdapter
+
+```typescript
+abstract class BasePlatformAdapter {
+  abstract readonly platform: PlatformName;
+  abstract connect(): Promise<boolean>;
+  abstract disconnect(): Promise<void>;
+  abstract sendMessage(out: OutboundMessage): Promise<SendResult>;
+  abstract onInbound(handler: (event: GatewayMessageEvent) => Promise<void>): () => void;
+  async startTyping(channelId: string): Promise<void> { /* noop */ }
+  async stopTyping(channelId: string): Promise<void> { /* noop */ }
+}
+```
+
+## GatewayMessageEvent (common fields)
+
+```typescript
+interface BaseMessageEvent {
+  id: string;
+  platform: PlatformName;
+  sender: { id: string; username?: string; displayName?: string };
+  channel: { id: string; type: "dm" | "group" | "thread"; topicId?: string };
+  text: string;
+  receivedAt: number;
+  replyTo?: string;
+}
+```
+
+## PlatformName
+
+```typescript
+type PlatformName =
+  | "telegram" | "discord" | "slack" | "whatsapp" | "teams"
+  | "email" | "sms" | "mattermost" | "line" | "matrix";
+```
+
+## Platform adapters
+
+### 1. Telegram (`@theokit/gateway-telegram`)
+
+```bash
+pnpm add @theokit/gateway-telegram @theokit/gateway grammy
+```
+
+```typescript
+import { TelegramAdapter } from "@theokit/gateway-telegram";
+
+const adapter = new TelegramAdapter({ botToken: process.env.TELEGRAM_BOT_TOKEN! });
+await adapter.connect();
+```
+
+Platform-specific: `event.telegram.chatId`, `event.telegram.messageId`, `event.telegram.threadId?`.
+
+### 2. Discord (`@theokit/gateway-discord`)
+
+```bash
+pnpm add @theokit/gateway-discord @theokit/gateway discord.js
+```
+
+```typescript
+import { DiscordAdapter } from "@theokit/gateway-discord";
+
+const adapter = new DiscordAdapter({ botToken: process.env.DISCORD_BOT_TOKEN! });
+await adapter.connect();
+```
+
+Platform-specific: `event.discord.guildId`, `event.discord.channelId`, `event.discord.messageId`.
+
+### 3. Slack (`@theokit/gateway-slack`)
+
+```bash
+pnpm add @theokit/gateway-slack @theokit/gateway @slack/bolt @slack/web-api
+```
+
+```typescript
+import { SlackAdapter } from "@theokit/gateway-slack";
+
+const adapter = new SlackAdapter({
+  botToken: process.env.SLACK_BOT_TOKEN!,
+  appToken: process.env.SLACK_APP_TOKEN!,
+  requireMention: true,  // default; public channels need @bot mention
+});
+await adapter.connect();
+```
+
+Platform-specific: `event.slack.teamId`, `event.slack.channelId`, `event.slack.ts`, `event.slack.threadTs?`.
+
+### 4. WhatsApp (`@theokit/gateway-whatsapp`)
+
+```bash
+pnpm add @theokit/gateway-whatsapp @theokit/gateway
+```
+
+Two backends: `"cloud"` (Meta Cloud API) and `"web"` (whatsapp-web.js bridge).
+
+Platform-specific: `event.whatsapp.wamid`, `event.whatsapp.backend`, `event.whatsapp.phoneNumberId?`.
+
+### 5. Teams (`@theokit/gateway-teams`)
+
+```bash
+pnpm add @theokit/gateway-teams @theokit/gateway botbuilder
+```
+
+Platform-specific: `event.teams.activityId`, `event.teams.conversationId`, `event.teams.conversationType`, `event.teams.tenantId?`.
+
+### 6. Email (`@theokit/gateway-email`)
+
+```bash
+pnpm add @theokit/gateway-email @theokit/gateway
+```
+
+Uses IMAP for inbound and SMTP for outbound.
+
+Platform-specific: `event.email.messageId`, `event.email.subject`, `event.email.fromAddress`, `event.email.recipients`.
+
+### 7. SMS (`@theokit/gateway-sms`)
+
+```bash
+pnpm add @theokit/gateway-sms @theokit/gateway
+```
+
+Backends: `"twilio"`, `"plivo"`, `"vonage"`. Inbound via webhook server.
+
+Platform-specific: `event.sms.backend`, `event.sms.from` (E.164), `event.sms.to` (E.164).
+
+### 8. Mattermost (`@theokit/gateway-mattermost`)
+
+```bash
+pnpm add @theokit/gateway-mattermost @theokit/gateway
+```
+
+Platform-specific: `event.mattermost.postId`, `event.mattermost.channelId`, `event.mattermost.rootId?`.
+
+### 9. LINE (`@theokit/gateway-line`)
+
+```bash
+pnpm add @theokit/gateway-line @theokit/gateway
+```
+
+Inbound via webhook with signature verification. Reply tokens are cached (one-shot, 60s TTL).
+
+Platform-specific: `event.line.sourceType`, `event.line.sourceId`, `event.line.messageId`.
+
+### 10. Matrix (`@theokit/gateway-matrix`)
+
+```bash
+pnpm add @theokit/gateway-matrix @theokit/gateway matrix-js-sdk
+```
+
+Platform-specific: `event.matrix.roomId`, `event.matrix.eventId`, `event.matrix.memberCount`.
+
+## Common usage pattern
+
+```typescript
+import { Agent } from "@theokit/sdk";
+
+const adapter = new TelegramAdapter({ botToken: process.env.TELEGRAM_BOT_TOKEN! });
+await adapter.connect();
+
+adapter.onInbound(async (event) => {
+  const agent = await Agent.getOrCreate(`${event.platform}-${event.sender.id}`, {
+    apiKey: process.env.THEOKIT_API_KEY!,
+    model: { id: "google/gemini-2.0-flash-001" },
+    local: { cwd: process.cwd() },
+    memory: { enabled: true, namespace: "bot", scope: "user", userId: event.sender.id },
+  });
+
+  await adapter.startTyping(event.channel.id);
+  const run = await agent.send(event.text);
+  const result = await run.wait();
+  await adapter.stopTyping(event.channel.id);
+
+  await adapter.sendMessage({
+    channel: event.channel,
+    text: result.result ?? "No response.",
+    replyTo: event.id,
+  });
+});
+```
+
+## Error mapping convention
+
+Each adapter maps platform errors to canonical `SendResult.error.code` values: `rate_limit`, `channel_not_found`, `no_permission`, `auth_error`, `message_too_long`, `platform_error`.

package/claude-template/dot-claude/skills/theokit-memory/SKILL.md

@@ -0,0 +1,176 @@
+---
+user-invocable: false
+description: Memory API, embedding providers, dreaming, active recall, and backends for @theokit/sdk.
+paths:
+  - "**/*memory*"
+  - "**/*Memory*"
+  - "**/*embed*"
+---
+
+# TheoKit SDK -- Memory
+
+Quick reference for durable memory, embedding providers, dreaming, and backends.
+
+## Enabling memory on an agent
+
+```typescript
+import { Agent } from "@theokit/sdk";
+
+const agent = await Agent.create({
+  apiKey: process.env.THEOKIT_API_KEY!,
+  model: { id: "google/gemini-2.0-flash-001" },
+  local: { cwd: process.cwd() },
+  memory: {
+    enabled: true,
+    namespace: "my-app",
+    userId: "user-123",
+    scope: "user",
+  },
+});
+
+await (await agent.send("Remember: my preferred test runner is Vitest.")).wait();
+```
+
+## MemoryOptions
+
+```typescript
+interface MemoryOptions {
+  enabled: boolean;
+  namespace?: string;   // separates application domains
+  userId?: string;      // isolates user memories
+  scope?: "agent" | "user" | "team";
+  storePath?: string;   // relative to workspace; cannot escape
+}
+```
+
+## Scopes
+
+| Scope | Use for |
+|---|---|
+| `"agent"` | Durable state for one agent ID. Default scope. |
+| `"user"` | Stable user preferences across agent instances. Requires `userId`. |
+| `"team"` | Shared team facts safe for every authorized caller. |
+
+## Safety rules
+
+- Memory MUST NOT store API keys, bearer tokens, passwords, or credential material.
+- Local `storePath` is resolved relative to the workspace. Path traversal raises `ConfigurationError`.
+- Memory is durable by `{ namespace, userId, scope }`, not by JavaScript process.
+
+## SDKMemoryManager
+
+```typescript
+interface SDKMemoryManager {
+  // Reserved for explicit inspection and deletion APIs.
+}
+```
+
+The agent uses memory during runs automatically. Public management APIs are narrow until deletion and audit semantics are finalized.
+
+## Memory backends (v1.2+)
+
+```typescript
+import { Memory } from "@theokit/sdk";
+
+const memory = await Memory.create({
+  cwd: process.cwd(),
+  index: {
+    backend: "sqlite-vec",  // default
+    embedding: { provider: "openai", model: "text-embedding-3-small" },
+  },
+});
+
+// Or use LanceDB for >100k facts:
+const memory = await Memory.create({
+  cwd: process.cwd(),
+  index: {
+    backend: "lance",
+    embedding: { provider: "openai", model: "text-embedding-3-small" },
+  },
+});
+```
+
+- `@lancedb/lancedb` is an optional peer dep. If missing with `backend: "lance"`, throws `ConfigurationError(code: "lance_backend_unavailable")`.
+- Embedding dimension validated when opening existing index. Mismatch throws `ConfigurationError(code: "embedding_dimension_mismatch")`.
+
+## Embedding providers (ADR D11)
+
+Locked provider union: `openai`, `mistral`, `openrouter`, `voyage`, `deepinfra`.
+
+```typescript
+index: {
+  embedding: {
+    provider: "openai",
+    model: "text-embedding-3-small",
+  },
+}
+```
+
+Each provider adapter implements:
+
+```typescript
+interface EmbeddingAdapter {
+  id: string;
+  model: string;
+  dimension: number;
+  embed(texts: string[]): Promise<number[][]>;
+}
+```
+
+## Memory migration CLI (v1.2+)
+
+Migrate SQLite memory index to LanceDB without data loss:
+
+```bash
+# Dry-run (preview, no writes)
+pnpm exec theokit-migrate-memory --cwd . --dry-run
+
+# Real migration
+pnpm exec theokit-migrate-memory --cwd .
+```
+
+Options: `--cwd <path>`, `--dry-run`, `--keep-sqlite`, `--batch-size <n>`.
+
+Algorithm: read SQLite, write to staging `lance-new/`, validate count + sample text match, atomic rename, prompt to delete SQLite.
+
+## Dreaming (consolidation sweeps)
+
+```typescript
+await Memory.runDreamingSweep({
+  cwd: process.cwd(),
+  embedding: { provider: "openai", model: "text-embedding-3-small" },
+});
+```
+
+Dreaming consolidates redundant facts into compressed summaries. In v1.x, consolidation is deterministic only (no LLM-mediated narrative). LLM narrative mode is deferred.
+
+## Active recall
+
+Active recall queries memory during `agent.send` to inject relevant facts into the LLM context. Configured via the memory options on the agent. Query modes:
+
+- `"embedding"` -- cosine similarity search against the vector index.
+- `"keyword"` -- keyword-based search.
+- `"hybrid"` -- combines embedding and keyword results.
+
+## Semantic cache (related)
+
+`Cache.semantic` from `@theokit/sdk` reuses embedding adapters for LLM response caching with cosine-similarity matching. See the cache documentation for details.
+
+## Resume behavior
+
+Memory is durable by `{ namespace, userId, scope }`, not by process. Recreating or resuming an agent with the same memory config can recall durable facts. Inline secrets and MCP servers are NOT persisted through memory.
+
+## Conversation storage (pluggable persistence)
+
+```typescript
+import { Agent, InMemoryConversationStorage } from "@theokit/sdk";
+
+const agent = await Agent.create({
+  apiKey, model,
+  conversationStorage: new InMemoryConversationStorage(),
+});
+```
+
+Built-in adapters: `FileSystemConversationStorage`, `InMemoryConversationStorage`. Custom adapters implement `ConversationStorageAdapter`.
+
+When using custom storage, `Agent.resume` requires the adapter to be passed again -- silent FS fallback is rejected with `ConfigurationError(code: "conversation_storage_required")`.

package/claude-template/dot-claude/skills/theokit-rag/SKILL.md

@@ -0,0 +1,226 @@
+---
+user-invocable: false
+description: RAG primitives -- VectorRetriever, CohereReranker, text splitters from @theokit/sdk/rag.
+paths:
+  - "**/*retriev*"
+  - "**/*rerank*"
+  - "**/*splitter*"
+  - "**/*rag*"
+---
+
+# TheoKit SDK -- RAG
+
+Quick reference for the RAG (Retrieval-Augmented Generation) sub-path at `@theokit/sdk/rag`.
+
+## Installation
+
+The RAG module ships with `@theokit/sdk` -- no additional install needed.
+
+```typescript
+import {
+  VectorRetriever,
+  CohereReranker,
+  NoopReranker,
+  splitByCharacter,
+  splitBySentence,
+  splitRecursive,
+} from "@theokit/sdk/rag";
+```
+
+## Types
+
+```typescript
+interface Document {
+  id: string;
+  text: string;
+  metadata?: Record<string, unknown>;
+}
+
+interface Chunk {
+  text: string;
+  index: number;
+  metadata?: Record<string, unknown>;
+}
+
+interface RetrievalResult {
+  text: string;
+  score: number;
+  metadata?: Record<string, unknown>;
+}
+
+interface RankedChunk {
+  text: string;
+  score: number;
+  originalIndex: number;
+  metadata?: Record<string, unknown>;
+}
+
+interface SplitOptions {
+  chunkSize: number;
+  overlap?: number;
+}
+```
+
+## Interfaces
+
+### Retriever
+
+```typescript
+interface Retriever {
+  retrieve(query: string, options?: { topK?: number }): Promise<RetrievalResult[]>;
+}
+```
+
+### Reranker
+
+```typescript
+interface Reranker {
+  rerank(query: string, chunks: RetrievalResult[]): Promise<RankedChunk[]>;
+}
+```
+
+## VectorRetriever
+
+Wraps any index that implements `search(query, topK)`.
+
+```typescript
+interface VectorIndex {
+  search(query: string, topK: number): Promise<RetrievalResult[]>;
+}
+
+interface VectorRetrieverOptions {
+  index: VectorIndex;
+  topK?: number; // default 5
+}
+```
+
+Usage:
+
+```typescript
+import { VectorRetriever } from "@theokit/sdk/rag";
+
+const retriever = new VectorRetriever({
+  index: myVectorIndex,
+  topK: 10,
+});
+
+const results = await retriever.retrieve("How does auth work?");
+// results: RetrievalResult[] sorted by relevance
+```
+
+The `VectorIndex` interface is the DI boundary. Consumers depend on the interface; implementations (e.g., backed by Memory's SQLite-vec or LanceDB index) depend on the index adapter.
+
+## CohereReranker
+
+Calls the Cohere Rerank v2 API to re-score retrieval results by relevance.
+
+```typescript
+interface CohereRerankerOptions {
+  apiKey: string;
+  model?: string; // default "rerank-v3.5"
+}
+```
+
+Usage:
+
+```typescript
+import { CohereReranker } from "@theokit/sdk/rag";
+
+const reranker = new CohereReranker({
+  apiKey: process.env.COHERE_API_KEY!,
+  model: "rerank-v3.5",
+});
+
+const ranked = await reranker.rerank("auth middleware", retrievalResults);
+// ranked: RankedChunk[] re-scored by Cohere
+```
+
+## NoopReranker
+
+Passes through results unchanged. Useful as a baseline or when reranking is not needed.
+
+```typescript
+import { NoopReranker } from "@theokit/sdk/rag";
+
+const reranker = new NoopReranker();
+const ranked = await reranker.rerank(query, results);
+// ranked === results (with originalIndex added)
+```
+
+## Text splitters
+
+Three strategies for splitting documents into chunks. All return `Chunk[]` with `text` and `index`. Empty input returns an empty array.
+
+### splitByCharacter
+
+Fixed-size character windows with optional overlap.
+
+```typescript
+import { splitByCharacter } from "@theokit/sdk/rag";
+
+const chunks = splitByCharacter(longText, { chunkSize: 500, overlap: 50 });
+```
+
+### splitBySentence
+
+Groups sentences into chunks up to `chunkSize` characters.
+
+```typescript
+import { splitBySentence } from "@theokit/sdk/rag";
+
+const chunks = splitBySentence(longText, { chunkSize: 500 });
+```
+
+Splits on sentence boundaries (`.`, `!`, `?` followed by whitespace). Sentences are never broken mid-sentence.
+
+### splitRecursive
+
+Three-level cascading split: paragraph, then sentence, then character.
+
+```typescript
+import { splitRecursive } from "@theokit/sdk/rag";
+
+const chunks = splitRecursive(longText, { chunkSize: 500, overlap: 50 });
+```
+
+Algorithm:
+1. Split by double newlines (paragraphs).
+2. Paragraphs that exceed `chunkSize` are split by sentence.
+3. Sentences that still exceed `chunkSize` are split by character.
+
+This is the recommended default for most RAG use cases.
+
+## Full RAG pipeline example
+
+```typescript
+import { VectorRetriever, CohereReranker, splitRecursive } from "@theokit/sdk/rag";
+
+// 1. Split documents
+const chunks = splitRecursive(documentText, { chunkSize: 500, overlap: 50 });
+
+// 2. Index chunks (your vector store)
+await vectorStore.upsert(chunks.map((c, i) => ({
+  id: `doc-${i}`,
+  text: c.text,
+  embedding: await embed(c.text),
+})));
+
+// 3. Retrieve
+const retriever = new VectorRetriever({ index: vectorStore, topK: 20 });
+const results = await retriever.retrieve(userQuery);
+
+// 4. Rerank
+const reranker = new CohereReranker({ apiKey: process.env.COHERE_API_KEY! });
+const ranked = await reranker.rerank(userQuery, results);
+
+// 5. Use top results as agent context
+const context = ranked.slice(0, 5).map((r) => r.text).join("\n\n");
+const agent = await Agent.create({
+  systemPrompt: `Use this context:\n${context}`,
+  // ...
+});
+```
+
+## DI integration
+
+Use `@Retriever` and `@Reranker` decorators from `@theokit/di-agent` to register RAG components in the DI container. See the theokit-di-agent skill for details.