@soulcraft/sdk 1.0.0

package/docs/ADR-001-sdk-design.md

@@ -0,0 +1,282 @@
+# ADR-001: @soulcraft/sdk — Architecture and Design Decisions
+
+**Status:** Accepted
+**Date:** 2026-02-27
+**Decision session:** Workshop session, informed by cross-project research across
+Workshop, Venue, Academy, and the existing `@soulcraft/brainy-client` and
+`@soulcraft/auth` packages.
+
+---
+
+## Context
+
+The Soulcraft platform consists of four products (Workshop, Venue, Academy, Portal) and
+a growing ecosystem of kit apps deployed on those products. As of 2026-02-27:
+
+- `@soulcraft/brainy-client` provides transport-level Brainy access (HTTP/WS/Local)
+- `@soulcraft/auth` provides shared session types and auth utilities
+- Each product duplicates significant auth, licensing, billing, and AI-integration logic
+- Kit apps injected into Venue or Workshop lack a standard API surface
+
+The proliferation of per-product implementations of the same concerns (auth middleware,
+Cortex activation, usage metering, AI model selection) creates drift and bugs. A unified
+SDK is the correct architectural response.
+
+---
+
+## Decision 1: Scope — Absorb, Not Wrap
+
+**Decision:** `@soulcraft/sdk` absorbs all code from `@soulcraft/brainy-client` and
+`@soulcraft/auth`. Both packages are deprecated immediately. No backward-compat re-export
+shims. All imports in all products are updated in the same session that ships the SDK.
+
+**Rejected alternative:** SDK wraps brainy-client + auth internally, keeping them as
+independent packages. Rejected because it creates three packages to maintain and the
+abstraction boundary adds no value — the SDK is the new contract.
+
+---
+
+## Decision 2: Uniform API Surface — Server and Client Modes
+
+**Decision:** Both server and client modes expose identical `sdk.*` method namespaces.
+Server mode runs Brainy in-process. Client mode serializes calls over a transport.
+App code written against the SDK works in both contexts without modification.
+
+```typescript
+// Same code works in server mode (in-process) and client mode (over HTTP):
+async function listConcepts(sdk: SoulcraftSDK) {
+  return sdk.brainy.find({ where: { type: NounType.Concept } })
+}
+```
+
+**Rejected alternative:** Separate entry points with different API shapes. Rejected
+because it forces app authors to branch on context — the entire value of the SDK is
+that they don't have to.
+
+---
+
+## Decision 3: Entry Points — Conditional Exports
+
+**Decision:** Three conditional exports for proper tree-shaking and bundle safety:
+
+| Export | Who uses it |
+|--------|-------------|
+| `@soulcraft/sdk` | Shared types + `createSDK` factory |
+| `@soulcraft/sdk/server` | Product backends — Brainy lifecycle, Cortex, middleware |
+| `@soulcraft/sdk/client` | Browser kit apps — transports, SSE, cookie auth |
+
+**Rationale:** Server-only code (Brainy core, Cortex, better-auth) must never land in
+browser bundles. Conditional exports enforce this at the package level. Bundlers tree-shake
+anything not reachable from the imported entry point.
+
+---
+
+## Decision 4: Server Mode — Full Instance Lifecycle Management
+
+**Decision:** SDK server mode creates and manages Brainy instances, replacing
+per-product patterns (Workshop's `brainy-singleton.ts`, Venue's `tenantCache`, etc.).
+Three built-in instance strategies:
+
+| Strategy | Key function | Use case |
+|----------|-------------|----------|
+| `per-user` | `userId` | Workshop — one Brainy per authenticated user |
+| `per-tenant` | `tenantSlug` | Venue — LRU cache max 50, flush-on-evict |
+| `per-scope` | Custom function | Academy or bespoke scenarios |
+
+All strategies include: LRU cache, configurable max instances, flush-on-evict,
+graceful shutdown with `sdk.shutdown()`.
+
+---
+
+## Decision 5: Transports — Four Protocols, Fixed Serialization
+
+**Decision:** Four transports are supported:
+
+| Transport | Serialization | Direction | Use case |
+|-----------|--------------|-----------|---------|
+| `local` | None (in-process) | — | Server mode, zero overhead |
+| `http` | JSON | Request/response | Stateless RPC — kit apps, simple clients |
+| `ws` | MessagePack binary | Bidirectional | Real-time — RPC + change push events |
+| `sse` | text/event-stream | Server → Client | Live updates — VFS/entity events |
+
+Serialization is fixed per transport and never configurable. HTTP=JSON because it is
+universally debuggable. WebSocket=MessagePack because it is ~40% smaller for
+high-frequency bidirectional traffic and the existing protocol is already MessagePack.
+
+**Rejected alternatives:**
+- MessagePack over HTTP: non-standard, harder to debug, no significant benefit at typical HTTP RPC volumes.
+- Configurable serializer: unnecessary complexity.
+- gRPC/Protobuf: over-engineered for the current scale.
+
+---
+
+## Decision 6: Full Brainy API — No Curation
+
+**Decision:** `sdk.brainy.*` exposes every public method on the Brainy class. No method
+blocklist at the SDK level. Permission scoping (blocking `clear`, `migrate` for kit apps)
+is a server-side concern configured at instance creation, not baked into the SDK types.
+
+**Rationale:** Curating a subset means every new Brainy feature requires an SDK update
+before it can be used. The SDK is the platform — it should never be the bottleneck.
+
+---
+
+## Decision 7: VFS as First-Class Namespace
+
+**Decision:** Virtual filesystem operations are exposed as `sdk.vfs.*`, not nested under
+`sdk.brainy.vfs.*`. VFS is a first-class concern — it stores documents, slides,
+visualizations, and all Soulcraft content formats. Hiding it under brainy implies it is
+an implementation detail, which is wrong.
+
+```typescript
+sdk.brainy.find(...)     // Entity/graph operations
+sdk.vfs.readFile(...)    // File operations — same SDK, different namespace
+```
+
+---
+
+## Decision 8: Auth — OIDC-Centralized in Production, Standalone for Dev
+
+**Decision:** `auth.soulcraft.com` is the single auth source for all production deployments
+of Workshop, Venue, Academy, and Portal. Products are pre-registered OIDC clients.
+Cross-subdomain cookies (`domain=.soulcraft.com`) provide seamless SSO.
+
+In local development (when `SOULCRAFT_IDP_URL` is unset), better-auth runs standalone
+with a local SQLite database. This is the intended dev default.
+
+**Rationale:** `auth.soulcraft.com` is live and running in production. All three products
+switch to OIDC when `SOULCRAFT_IDP_URL=https://auth.soulcraft.com` is set in the
+environment. Standalone mode is kept for two reasons:
+1. Local dev requires no internet connection, no OAuth credentials, and no prior setup.
+2. `auth.soulcraft.com` itself needs to be testable without self-dependency.
+
+The `SOULCRAFT_IDP_URL` env var is the clean, zero-ambiguity boundary. It is always
+set in all production and staging environments. It is never set in local dev.
+
+**SDK provides:**
+- `getAuthMode()` — returns `'standalone'` or `'oidc-client'` based on env
+- `getOIDCClientConfig()` — reads OIDC config from env, validates required vars
+- `createAuthMiddleware(auth)` — Hono `requireAuth` / `optionalAuth` middleware
+- `createBackchannelLogoutHandler(config)` — OIDC backchannel logout endpoint factory
+- `SOULCRAFT_USER_FIELDS`, `SOULCRAFT_SESSION_CONFIG` — shared better-auth config
+- `computeEmailHash(email)` — SHA-256 digest for Brainy path isolation
+- Capability tokens for cross-product server-to-server RPC auth
+
+---
+
+## Decision 9: Module Scope — Core Platform + Billing + Notifications
+
+**Decision:** v1.0 includes:
+
+**In SDK:**
+- `sdk.brainy.*` — data graph
+- `sdk.vfs.*` — filesystem
+- `sdk.versions.*` — version history
+- `sdk.auth.*` — auth + sessions + capability tokens
+- `sdk.license.*` — Cortex activation + plans + credits + BYOK
+- `sdk.ai.*` — Claude model tiers, delegator, tool definitions, Briggy companion API
+- `sdk.events.*` — platform event bus (30+ event types)
+- `sdk.skills.*` — VFS-backed skill system
+- `sdk.kits.*` — kit loader and initialization
+- `sdk.formats.*` — Soulcraft format types (WVIZ, WDOC, WSLIDE, WQUIZ)
+- `sdk.billing.*` — usage metering, quotas, top-ups, Stripe subscriptions
+- `sdk.notifications.*` — Postmark email + Twilio SMS + dev console
+
+**Stays in product (not in SDK):**
+- Academy pedagogical engine (enrollment, certification, socratic tutor, gamification)
+- Venue POS/booking/calendar/gift card flows
+- Workshop publish/deploy flows
+- Product-specific AI tool definitions (beyond shared types)
+
+**Rationale:** Billing and notifications have patterns shared across products (usage
+metering is structurally identical across Workshop, Venue, and Academy; the
+NotificationSender interface is already provider-agnostic). Academy's pedagogical engine
+is deeply domain-specific and has no cross-product value.
+
+---
+
+## Decision 10: Licensing — Full Pipeline in SDK
+
+**Decision:** `sdk.license.*` handles the complete Cortex + plan lifecycle:
+1. Read activation code from `.soulcraft.json`
+2. Exchange with Portal's license server for JWT
+3. Activate Cortex native SIMD/mmap providers
+4. Cache JWT locally in `.soulcraft/` directory
+5. Background heartbeat every 4 hours
+6. Expose plan info: `sdk.license.plan`, `sdk.license.isActive`, `sdk.license.features`
+7. AI usage: `sdk.license.checkAICredits(userId)`, `sdk.license.consumeAICredit(userId)`
+8. BYOK: `sdk.license.validateBYOK(key)`, `sdk.license.getAIProvider(userId)`
+
+**Rationale:** Today every product duplicates the `.soulcraft.json` → Cortex activation
+flow. Centralizing it eliminates drift and ensures the heartbeat + JWT cache pattern is
+consistent everywhere.
+
+---
+
+## Implementation Plan
+
+### Phase 1 — Foundation (new repo + package scaffold)
+- [x] Create `soulcraftlabs/sdk` GitHub repo
+- [x] Initialize TypeScript + Bun package with conditional exports
+- [x] Write this ADR and CLAUDE.md
+- [ ] Scaffold `src/` directory structure (empty modules with JSDoc stubs)
+- [ ] Write `tsconfig.json`, `vitest.config.ts`, `.env.example`
+
+### Phase 2 — Core Data Layer
+- [ ] `src/transports/` — migrate Local, HTTP, WS from brainy-client; add SSE
+- [ ] `src/modules/brainy/` — full Brainy API proxy (all methods, all sub-APIs)
+- [ ] `src/modules/vfs/` — VFS namespace (delegates to `brain.vfs.*`)
+- [ ] `src/modules/versions/` — versions namespace (delegates to `brain.versions.*`)
+- [ ] `src/types.ts` — core shared types (SoulcraftSDK, SDKOptions, etc.)
+- [ ] Server mode instance pool (`per-user`, `per-tenant`, `per-scope`)
+- [ ] Tests: all transport modes, all brainy methods via proxy
+
+### Phase 3 — Auth + License
+- [ ] `src/modules/auth/` — OIDC client config, session cache, capability tokens,
+      Hono middleware, SvelteKit handle, backchannel logout
+- [ ] `src/modules/license/` — Cortex activation, plan verification, credit metering,
+      BYOK validation, heartbeat background task
+- [ ] Absorb `@soulcraft/auth` package entirely
+- [ ] Tests: auth middleware, capability token create/verify, license exchange
+
+### Phase 4 — AI + Events + Skills + Kits + Formats
+- [ ] `src/modules/ai/` — model tiers, delegator, Briggy API, tool definition types,
+      KitAIContext, UI action registry
+- [ ] `src/modules/events/` — platform event bus (DataChangeEmitter, all 30+ event types)
+- [ ] `src/modules/skills/` — skill loading, bundled skills, user-invocable registry
+- [ ] `src/modules/kits/` — kit loader, initialization, template injection
+- [ ] `src/modules/formats/` — WVIZ, WDOC, WSLIDE, WQUIZ types (migrate from @soulcraft/views + richDocument.ts)
+
+### Phase 5 — Billing + Notifications
+- [ ] `src/modules/billing/` — usage tracking, quota checks, top-up credits,
+      Stripe subscription management, billing provider (Firestore vs local)
+- [ ] `src/modules/notifications/` — NotificationSender interface, Postmark, Twilio,
+      dev-mode console sender, notification templates
+
+### Phase 6 — Publish + First npm Release
+- [ ] Publish `@soulcraft/sdk@1.0.0` to npmjs.com (private/restricted)
+- [ ] Update Workshop: replace all `@soulcraft/brainy-client` + `@soulcraft/auth` imports
+- [ ] Update Venue: same
+- [ ] Update Academy: same
+- [ ] Deprecate (do not unpublish) `@soulcraft/brainy-client` and `@soulcraft/auth`
+- [ ] Add `@soulcraft/sdk` to bundle loop in all three `build.sh` files
+
+---
+
+## Packages Deprecated by This SDK
+
+| Package | Status after SDK ships |
+|---------|----------------------|
+| `@soulcraft/brainy-client` | Deprecated — all code moved into SDK |
+| `@soulcraft/auth` | Deprecated — all code moved into SDK |
+
+Both packages remain published on npmjs.com (no unpublish) but get a deprecation
+notice in their README and `package.json` `"deprecated"` field pointing to `@soulcraft/sdk`.
+
+---
+
+## Related Documents
+
+- `docs/ADR-002-transport-protocol.md` — Wire protocol specification (to be written in Phase 2)
+- `docs/ADR-003-instance-strategies.md` — Brainy instance pooling spec (to be written in Phase 2)
+- `/home/dpsifr/.strategy/PLATFORM-HANDOFF.md` — Cross-project coordination thread