@soulcraft/sdk 1.0.0

package/docs/USAGE.md

@@ -0,0 +1,646 @@
+# @soulcraft/sdk — Usage Guide
+
+This guide is for engineers, AI assistants, kit developers, and anyone building on
+the Soulcraft platform. It covers every implemented module with working examples.
+
+**Three rules to remember:**
+1. Server code (Hono, SvelteKit routes, Bun backends) imports from `@soulcraft/sdk/server`
+2. Browser code (kit apps, Svelte components) imports from `@soulcraft/sdk/client`
+3. Shared types import from `@soulcraft/sdk`
+
+---
+
+## Table of Contents
+
+1. [Installation](#installation)
+2. [Server Setup — createSDK](#server-setup--createsdk)
+3. [Client Setup — Transports](#client-setup--transports)
+4. [sdk.brainy — Graph Data](#sdkbrainy--graph-data)
+5. [sdk.vfs — Virtual Filesystem](#sdkvfs--virtual-filesystem)
+6. [sdk.auth — Authentication & Tokens](#sdkauth--authentication--tokens)
+7. [sdk.ai — Claude AI Integration](#sdkai--claude-ai-integration)
+8. [sdk.events — Platform Event Bus](#sdkevents--platform-event-bus)
+9. [sdk.skills — Skill System](#sdkskills--skill-system)
+10. [BrainyInstancePool — Instance Lifecycle](#brainyinstancepool--instance-lifecycle)
+11. [Capability Tokens — Cross-Product Access](#capability-tokens--cross-product-access)
+12. [Auth Middleware](#auth-middleware)
+13. [For AI Assistants](#for-ai-assistants)
+
+---
+
+## Installation
+
+```bash
+# Requires npm login to the @soulcraft org:
+npm login
+bun add @soulcraft/sdk
+```
+
+Peer dependencies — must already be in your project:
+```bash
+bun add @soulcraft/brainy @soulcraft/cortex
+```
+
+**GCE VM deploys:** The SDK cannot be installed on the VM without an npm auth token.
+Bundle it from local `node_modules` in your `build.sh`:
+```bash
+cp -r node_modules/@soulcraft/sdk build/node_modules/@soulcraft/sdk
+```
+
+---
+
+## Server Setup — createSDK
+
+Use `createSDK` in any request handler that has already resolved a Brainy instance.
+`createSDK` is a thin wrapper — create one per request, not one per process.
+
+```typescript
+import { BrainyInstancePool, createSDK } from '@soulcraft/sdk/server'
+
+// One pool per process — module singleton
+const pool = new BrainyInstancePool({
+  storage: process.env.STORAGE_TYPE === 'mmap-filesystem' ? 'mmap-filesystem' : 'filesystem',
+  dataPath: process.env.BRAINY_DATA_PATH ?? './brainy-data',
+  strategy: 'per-user',
+  maxInstances: 200,
+  flushOnEvict: true,
+})
+
+// In a request handler:
+app.get('/api/data', requireAuth, async (c) => {
+  const user = c.get('user')!
+  const brain = await pool.forUser(user.emailHash, 'main')
+  const sdk = createSDK({ brain })
+
+  // All namespaces now available:
+  const items = await sdk.brainy.find({ query: 'inventory items' })
+  const readme = await sdk.vfs.readFile('/projects/README.md')
+  const skill = await sdk.skills.load('inventory-health', 'wicks-and-whiskers')
+
+  return c.json({ items })
+})
+```
+
+### Instance Strategies
+
+| Strategy | When to use | Pool method |
+|----------|-------------|-------------|
+| `per-user` | One Brainy per user (Workshop) | `pool.forUser(emailHash, workspaceId)` |
+| `per-tenant` | One Brainy per org/location (Venue) | `pool.forTenant(slug)` |
+| `per-scope` | Custom key — Academy or bespoke | `pool.forScope(key, factory)` |
+
+```typescript
+// per-scope example (Academy — content vs learner brains):
+const contentPool = new BrainyInstancePool({ strategy: 'per-scope', ... })
+const brain = await contentPool.forScope(
+  `content:${workspaceId}`,
+  () => initBrainy(`content/${workspaceId}`, storagePath)
+)
+```
+
+---
+
+## Client Setup — Transports
+
+Used in browser kit apps and remote connections.
+
+```typescript
+import { createBrainyProxy, HttpTransport, WsTransport } from '@soulcraft/sdk/client'
+
+// HTTP transport — stateless, for simple kit apps:
+const transport = new HttpTransport('https://workshop.soulcraft.com', undefined, true)
+const brainy = createBrainyProxy(transport)
+
+const results = await brainy.find({ query: 'inventory' })
+
+// WebSocket transport — real-time, change push events:
+const wsTransport = new WsTransport(
+  'wss://venue.soulcraft.com/api/brainy/ws',
+  capabilityToken,
+  'wicks-and-whiskers'
+)
+await wsTransport.connect()
+const brainy = createBrainyProxy(wsTransport)
+
+brainy.onDataChange((event) => {
+  console.log('Live change:', event)
+})
+```
+
+### Which transport to use?
+
+| Scenario | Transport | Why |
+|----------|-----------|-----|
+| Kit app reading/writing data | `http` | Stateless, simple, no connection overhead |
+| Real-time sync + change events | `ws` | Bidirectional, MessagePack, change push |
+| Live update notifications only | `sse` | Lightweight server-push, auto-reconnects |
+| Server handler (same process) | `local` (via `createSDK`) | Zero overhead |
+
+---
+
+## sdk.brainy — Graph Data
+
+The full Brainy graph API. Works identically in server mode (via `createSDK`) and client mode (via transport proxy).
+
+```typescript
+import { NounType, VerbType } from '@soulcraft/brainy'
+
+// Add entities
+await sdk.brainy.add({
+  id: 'product-lavender',
+  type: 'Product',
+  metadata: { name: 'Lavender Soy Candle', price: 28, stock: 45 },
+})
+
+// Find with semantic search
+const results = await sdk.brainy.find({
+  query: 'soy candle lavender scent',
+  type: 'Product',
+  limit: 10,
+})
+
+// Find with filter
+const inStock = await sdk.brainy.find({
+  query: 'candles',
+  filter: { stock: { $gt: 0 } },
+})
+
+// Get by ID
+const product = await sdk.brainy.get('product-lavender')
+
+// Update
+await sdk.brainy.update({
+  id: 'product-lavender',
+  metadata: { stock: 40 },
+})
+
+// Delete
+await sdk.brainy.delete('product-lavender')
+
+// Relationships
+await sdk.brainy.relate({
+  from: 'product-lavender',
+  to: 'category-candles',
+  type: 'BelongsTo',
+})
+
+const relations = await sdk.brainy.getRelations({ from: 'product-lavender' })
+await sdk.brainy.unrelate({ from: 'product-lavender', to: 'category-candles', type: 'BelongsTo' })
+
+// Real-time change events (local transport: use sdk.events; WS transport: use onDataChange)
+sdk.brainy.onDataChange((event) => {
+  // event: { type: 'change', event: 'add'|'update'|'delete'|'relate'|'unrelate', entity?, relation? }
+  console.log('Brainy change:', event.event, event.entity?.id)
+})
+```
+
+---
+
+## sdk.vfs — Virtual Filesystem
+
+VFS stores documents, code, slides, and any file content alongside Brainy graph data.
+Accessed via `sdk.vfs.*` — the same API as `brain.vfs.*` but through the SDK proxy.
+
+```typescript
+// Write files
+await sdk.vfs.writeFile('/projects/my-project/README.md', '# My Project\n...')
+await sdk.vfs.writeFile('/notes/todo.txt', 'Buy more candles')
+
+// Read files — returns Buffer; convert to string for text:
+const buffer = await sdk.vfs.readFile('/projects/my-project/README.md')
+const content = buffer.toString('utf-8')
+
+// List directory
+const entries = await sdk.vfs.readdir('/projects/my-project')
+// → ['README.md', 'src', 'package.json']
+
+// File info
+const stat = await sdk.vfs.stat('/projects/my-project/README.md')
+// → { size, mtime, isDirectory, ... }
+
+// Rename / move
+await sdk.vfs.rename('/projects/old-name', '/projects/new-name')
+
+// Delete
+await sdk.vfs.unlink('/notes/todo.txt')
+await sdk.vfs.rmdir('/projects/old-project', { recursive: true })
+
+// Make directories
+await sdk.vfs.mkdir('/projects/new-project', { recursive: true })
+```
+
+---
+
+## sdk.auth — Authentication & Tokens
+
+The auth namespace handles capability tokens for cross-product server-to-server calls.
+Session auth (OIDC, middleware) is configured at the product level — see [Auth Middleware](#auth-middleware).
+
+```typescript
+import { createCapabilityToken, verifyCapabilityToken } from '@soulcraft/sdk/server'
+
+// Or via sdk.auth (same functions):
+const token = await sdk.auth.createToken({
+  email: user.email,
+  scope: 'wicks-and-whiskers',
+  secret: process.env.VENUE_RPC_SECRET!,
+  // ttlMs: 3_600_000   // default: 1 hour
+})
+
+// On the receiving server:
+const claims = await sdk.auth.verifyToken(token, process.env.VENUE_RPC_SECRET!)
+if (!claims) return c.json({ error: 'Unauthorized' }, 401)
+console.log(claims.email, claims.scope, claims.exp)
+```
+
+Token format: `<base64url(JSON payload)>.<base64url(HMAC-SHA256 signature)>`
+
+---
+
+## sdk.ai — Claude AI Integration
+
+Wraps the Anthropic Claude API with Soulcraft defaults. Requires `ANTHROPIC_API_KEY`.
+
+```typescript
+import { AI_MODELS } from '@soulcraft/sdk'
+
+// Basic completion
+const response = await sdk.ai.complete({
+  messages: [{ role: 'user', content: 'What candles are low in stock?' }],
+  systemPrompt: kit.shared.aiPersona,   // from kit.json → shared.aiPersona
+  model: AI_MODELS.haiku,              // fast + cheap for simple lookups
+  tools: [
+    {
+      name: 'search_inventory',
+      description: 'Search Brainy for inventory data',
+      inputSchema: {
+        type: 'object',
+        properties: { query: { type: 'string' } },
+        required: ['query'],
+      },
+    }
+  ],
+})
+
+// response.text — Claude's text reply (null if only tool calls)
+// response.toolCalls — tool invocations Claude wants to make (null if none)
+// response.stopReason — 'end_turn' | 'tool_use' | 'max_tokens'
+// response.usage — { inputTokens, outputTokens }
+
+// Handle tool calls:
+if (response.toolCalls) {
+  for (const call of response.toolCalls) {
+    if (call.name === 'search_inventory') {
+      const results = await sdk.brainy.find({ query: call.input.query as string })
+      // Continue the conversation by appending a tool_result message
+    }
+  }
+}
+
+// Streaming — yields text, tool_use, and done events:
+for await (const event of sdk.ai.stream({
+  messages: [{ role: 'user', content: 'Write a product description for lavender candles' }],
+  systemPrompt: 'You are a Wicks & Whiskers product writer.',
+  model: AI_MODELS.sonnet,
+})) {
+  if (event.type === 'text') process.stdout.write(event.text)
+  if (event.type === 'tool_use') console.log('Tool call:', event.toolCall.name)
+  if (event.type === 'done') console.log('Total tokens:', event.result.usage)
+}
+```
+
+### Model tiers
+
+| Constant | Model ID | When to use |
+|----------|----------|-------------|
+| `AI_MODELS.haiku` | `claude-haiku-4-5-20251001` | Fast lookups, data extraction, simple Q&A |
+| `AI_MODELS.sonnet` | `claude-sonnet-4-6` | Most kit operations, content generation, analysis |
+| `AI_MODELS.opus` | `claude-opus-4-6` | Complex reasoning, long-form synthesis |
+
+---
+
+## sdk.events — Platform Event Bus
+
+A typed EventEmitter for coordinating platform events across modules and kit code.
+Events are local to the SDK instance — they do not cross process boundaries.
+
+```typescript
+// Subscribe to built-in platform events:
+sdk.events.on('brainy:change', (event) => {
+  if (event.event === 'add' && event.entity?.nounType === 'Product') {
+    updateInventoryCache(event.entity)
+  }
+})
+
+sdk.events.on('vfs:write', ({ path, content }) => {
+  if (path.endsWith('.md')) {
+    broadcastDocumentUpdate(path, content)
+  }
+})
+
+sdk.events.on('vfs:delete', ({ path }) => {
+  invalidateCache(path)
+})
+
+sdk.events.on('vfs:rename', ({ from, to }) => {
+  updateFileTree(from, to)
+})
+
+// Emit custom application events:
+sdk.events.emit('kit:session-completed', {
+  userId: user.id,
+  sessionId,
+  kitId: 'wicks-and-whiskers',
+})
+
+// Remove a listener:
+const handler = (event) => { ... }
+sdk.events.on('brainy:change', handler)
+sdk.events.off('brainy:change', handler)
+```
+
+### Adding custom event types
+
+Extend `SoulcraftEventMap` with declaration merging for type-safe custom events:
+
+```typescript
+// In your project (e.g. types.d.ts):
+declare module '@soulcraft/sdk' {
+  interface SoulcraftEventMap {
+    'kit:session-completed': { userId: string; sessionId: string; kitId: string }
+    'venue:booking-created': { bookingId: string; tenantId: string }
+    'academy:enrollment': { userId: string; courseId: string; workspaceId: string }
+  }
+}
+```
+
+After this declaration, `sdk.events.on('kit:session-completed', ...)` is fully typed.
+
+---
+
+## sdk.skills — Skill System
+
+Skills are SKILL.md files that define an AI persona, capabilities, and workflow for
+a kit domain. The skills module loads from VFS first, then falls back to the bundled
+`@soulcraft/kits` registry.
+
+```typescript
+// Load a specific skill (VFS first, then bundled):
+const skill = await sdk.skills.load('inventory-health', 'wicks-and-whiskers')
+if (skill) {
+  console.log(skill.id)       // 'inventory-health'
+  console.log(skill.kitId)    // 'wicks-and-whiskers'
+  console.log(skill.source)   // 'vfs' or 'bundled'
+  console.log(skill.content)  // full SKILL.md content string
+}
+
+// List all skills for a kit:
+const skills = await sdk.skills.list({ kitId: 'wicks-and-whiskers' })
+const systemPrompt = buildSystemPrompt(kit, skills.map(s => s.content))
+
+// List all skills across all kits:
+const allSkills = await sdk.skills.list()
+
+// List only VFS skills (user-installed), no bundled:
+const userSkills = await sdk.skills.list({
+  kitId: 'wicks-and-whiskers',
+  includeBundled: false,
+})
+
+// Install a skill into the VFS (makes it available to future load() calls):
+await sdk.skills.install({
+  id: 'custom-flow',
+  kitId: 'wicks-and-whiskers',
+  content: `---
+id: custom-flow
+title: Custom Checkout Flow
+---
+You are a candle shop checkout specialist...`,
+})
+```
+
+### VFS path convention for skills
+
+```
+/skills/{kitId}/{skillId}.md
+```
+
+Example: `/skills/wicks-and-whiskers/inventory-health.md`
+
+---
+
+## BrainyInstancePool — Instance Lifecycle
+
+The pool manages Brainy instances: creation, caching, LRU eviction, and graceful shutdown.
+
+```typescript
+import { BrainyInstancePool } from '@soulcraft/sdk/server'
+
+const pool = new BrainyInstancePool({
+  storage: 'mmap-filesystem',  // or 'filesystem' for dev
+  dataPath: '/mnt/brainy-data',
+  strategy: 'per-user',
+  maxInstances: 200,
+  flushOnEvict: true,
+  onInit: async (brain, storagePath) => {
+    // Optional: post-init hook for migrations, integrity checks, etc.
+    await runMigrations(brain)
+  },
+})
+
+// per-user (Workshop):
+const brain = await pool.forUser(user.emailHash, workspaceId)
+
+// per-tenant (Venue):
+const brain = await pool.forTenant('wicks-and-whiskers')
+
+// per-scope (custom key — Academy, multi-branch, etc.):
+const brain = await pool.forScope(
+  `content:${workspaceId}`,
+  () => initBrainy('content', storagePath)
+)
+
+// Pool stats:
+const stats = pool.getStats()
+// { size: 12, maxSize: 200, keys: ['abc123:ws-1', ...] }
+
+// Graceful shutdown (flush all, clear cache):
+await pool.shutdown()
+
+// Flush and evict one instance:
+await pool.flush('abc123:workspace-1')
+```
+
+### Concurrent init deduplication
+
+If two requests arrive simultaneously for the same scope key while Brainy is still
+initializing, the pool deduplicates — only one Brainy instance is ever created per key,
+and both requests await the same initialization promise.
+
+---
+
+## Capability Tokens — Cross-Product Access
+
+Used when a product backend (e.g. Workshop) needs to call another product's Brainy RPC
+endpoint. Short-lived HMAC-SHA256 tokens replace session cookies for server-to-server calls.
+
+```typescript
+import { createCapabilityToken, verifyCapabilityToken } from '@soulcraft/sdk/server'
+
+// Issuing server — create a scoped token:
+const token = await createCapabilityToken({
+  email: user.email,
+  scope: 'wicks-and-whiskers',     // optional — restricts to one tenant
+  secret: process.env.VENUE_RPC_SECRET!,
+  ttlMs: 3_600_000,                // 1 hour (default)
+})
+
+// Send token in Authorization header:
+// Authorization: Bearer <token>
+
+// Receiving server — verify in the RPC handler:
+const claims = await verifyCapabilityToken(token, process.env.VENUE_RPC_SECRET!)
+if (!claims) return new Response('Unauthorized', { status: 401 })
+
+console.log(claims.email)   // original user's email
+console.log(claims.scope)   // 'wicks-and-whiskers'
+console.log(claims.exp)     // expiry timestamp (ms)
+```
+
+---
+
+## Auth Middleware
+
+The SDK provides Hono middleware that resolves better-auth sessions and injects the
+typed user into request context.
+
+```typescript
+import { betterAuth } from 'better-auth'
+import {
+  SOULCRAFT_USER_FIELDS,
+  SOULCRAFT_SESSION_CONFIG,
+  computeEmailHash,
+  createAuthMiddleware,
+} from '@soulcraft/sdk/server'
+import type { AuthContext } from '@soulcraft/sdk/server'
+
+// 1. Configure better-auth with Soulcraft fields:
+const auth = betterAuth({
+  database: new Database('./auth.db'),
+  secret: process.env.BETTER_AUTH_SECRET!,
+  session: SOULCRAFT_SESSION_CONFIG,     // 30-day sessions, 24h refresh
+  user: { additionalFields: SOULCRAFT_USER_FIELDS },
+  databaseHooks: {
+    user: {
+      create: {
+        before: async (user) => ({
+          data: { ...user, emailHash: computeEmailHash(user.email), platformRole: 'creator' }
+        })
+      }
+    }
+  },
+})
+
+// 2. Create middleware:
+const { requireAuth, optionalAuth } = createAuthMiddleware(auth)
+
+// 3. Use in routes:
+app.all('/api/auth/*', (c) => auth.handler(c.req.raw))
+
+app.get('/api/data', requireAuth, async (c) => {
+  const user = c.get('user')!   // SoulcraftSessionUser — always non-null after requireAuth
+  // user.id, user.email, user.emailHash, user.platformRole
+})
+
+app.get('/api/public', optionalAuth, async (c) => {
+  const user = c.get('user')    // SoulcraftSessionUser | null
+})
+```
+
+### Auth modes
+
+| Mode | When | How |
+|------|------|-----|
+| Standalone | `SOULCRAFT_IDP_URL` not set | better-auth runs locally with SQLite — local dev default |
+| OIDC client | `SOULCRAFT_IDP_URL=https://auth.soulcraft.com` | Delegates all auth to the central IdP — all production deployments |
+
+Production always uses OIDC. Standalone is the dev default — no internet connection,
+no credentials, no setup required.
+
+### Backchannel logout (OIDC)
+
+For products running in OIDC client mode:
+
+```typescript
+import { createBackchannelLogoutHandler } from '@soulcraft/sdk/server'
+
+const backchannelHandler = createBackchannelLogoutHandler({
+  auth,
+  clientSecret: process.env.SOULCRAFT_OIDC_CLIENT_SECRET!,
+  idpUrl: process.env.SOULCRAFT_IDP_URL!,
+  clientId: 'workshop',
+})
+
+app.post('/api/auth/backchannel-logout', backchannelHandler)
+```
+
+---
+
+## For AI Assistants
+
+When helping implement Soulcraft server code, follow this checklist:
+
+**1. Import correctly:**
+```typescript
+// Server code (Hono, SvelteKit server routes, Bun scripts):
+import { BrainyInstancePool, createSDK, createAuthMiddleware, computeEmailHash } from '@soulcraft/sdk/server'
+import type { SoulcraftSessionUser, AuthContext } from '@soulcraft/sdk'
+
+// Shared types only:
+import { AI_MODELS } from '@soulcraft/sdk'
+```
+
+**2. Resolve Brainy from the pool, then create SDK:**
+```typescript
+const brain = await pool.forUser(user.emailHash, workspaceId)  // or forTenant, forScope
+const sdk = createSDK({ brain })
+```
+
+**3. Call modules on the sdk object:**
+```typescript
+await sdk.brainy.find({ query: '...' })
+await sdk.vfs.readFile('/path/to/file')
+await sdk.ai.complete({ messages, systemPrompt, model: AI_MODELS.haiku })
+await sdk.skills.load('skill-id', 'kit-id')
+sdk.events.on('brainy:change', handler)
+sdk.events.emit('custom:event', payload)
+```
+
+**4. Module implementation status:**
+
+| Module | Status | Notes |
+|--------|--------|-------|
+| `sdk.brainy.*` | ✅ Implemented | Full Brainy API via proxy |
+| `sdk.vfs.*` | ✅ Implemented | Brainy VFS sub-API via proxy |
+| `sdk.auth.*` | ✅ Implemented | Capability tokens only; middleware is product-level |
+| `sdk.ai.*` | ✅ Implemented | Claude complete() + stream() |
+| `sdk.events.*` | ✅ Implemented | EventEmitter, typed, local to SDK instance |
+| `sdk.skills.*` | ✅ Implemented | VFS + bundled registry fallback |
+| `sdk.versions.*` | ✅ Types only | Runtime proxy works; Brainy versions API |
+| `sdk.license.*` | ❌ Not yet | Will throw if accessed |
+| `sdk.kits.*` | ❌ Not yet | Will throw if accessed |
+| `sdk.formats.*` | ❌ Not yet | Will throw if accessed |
+| `sdk.billing.*` | ❌ Not yet | Will throw if accessed |
+| `sdk.notifications.*` | ❌ Not yet | Will throw if accessed |
+
+**5. Environment variables required:**
+
+| Variable | Module | Required in prod |
+|----------|--------|-----------------|
+| `ANTHROPIC_API_KEY` | `sdk.ai` | Yes |
+| `SOULCRAFT_IDP_URL` | Auth (OIDC mode) | Production only |
+| `SOULCRAFT_OIDC_CLIENT_ID` | Auth (OIDC mode) | If SOULCRAFT_IDP_URL is set |
+| `SOULCRAFT_OIDC_CLIENT_SECRET` | Auth (OIDC mode) | If SOULCRAFT_IDP_URL is set |
+| `BETTER_AUTH_SECRET` | better-auth | Yes |
+| `BRAINY_DATA_PATH` | BrainyInstancePool | Yes (defaults to `./brainy-data`) |
+| `STORAGE_TYPE` | BrainyInstancePool | Yes (`filesystem` or `mmap-filesystem`) |