@nightowlsdev/storage-supabase 0.3.0 → 2.0.0

package/README.md

@@ -1,90 +1,53 @@
 # @nightowlsdev/storage-supabase
 
 A Supabase/Postgres-backed `StorageAdapter` for `@nightowlsdev/core` — implements the
-`agents` / `runs` / `events` / `messages` seams, a redacted Realtime broadcast event
-bus (`events.subscribe`), and a Postgres-backed Mastra store for cross-process durable
-suspend/resume.
+`agents` / `runs` / `events` / `messages` / `scratchpad` seams, a Postgres LISTEN/NOTIFY
+cache-invalidation bus, a cross-instance `ContainerFloor` for serverless mutual exclusion,
+and a Mastra-backed durable suspend/resume store.
 
 The server adapter authenticates with the **secret** key (`BYPASSRLS`) and is the real
 authorization boundary — it scopes every query by `tenantId`/`org_id` in code. The SQL
 RLS in the packaged `0001_core` migration is defense-in-depth for the browser /
 Realtime read path only.
 
-The tables live in a dedicated **`owl`** schema with **bare names**
-(`nightowls.orgs`, `nightowls.runs`, `nightowls.events`, `nightowls.agents`, …). The adapter pool
-sets `search_path=nightowls,public`, so the adapter's own queries stay unqualified. The
+The tables live in a dedicated **`nightowls`** schema. The adapter pool sets
+`search_path=nightowls,public`, so the adapter's own queries stay unqualified. The
 schema + tables are created by this package's **inlined migrations** (`MIGRATIONS`),
 which `@nightowlsdev/cli` **installs** into your host's `supabase/migrations/` — you apply them
 with your own tooling (`supabase db push`). Night Owls never runs DDL itself.
 
-## Connection rules
+## Install
 
-### Use the Session/Direct pooler port (5432) — NEVER the Transaction pooler (6543)
-
-`pg` and `@mastra/pg` rely on prepared statements, which **break on the Transaction
-pooler (port 6543)**. Always pass the **Session/Direct** Postgres URL (port **5432**).
-Both `createSupabaseStorage` and `createMastraPgStore` throw if they detect `:6543` in
-the `dbUrl`.
-
-```
-✅  postgresql://…@db.<ref>.supabase.co:5432/postgres   (Session/Direct)
-❌  postgresql://…@…pooler.supabase.com:6543/postgres    (Transaction — breaks pg)
+```bash
+pnpm add @nightowlsdev/storage-supabase
 ```
 
-### Two keys, two roles
-
-| Key | Where it goes | Why |
-|---|---|---|
-| **secret** (`sb_secret_…` / `service_role`) | the server **adapter** (`secretKey`) | `BYPASSRLS`; the real authz boundary. Never ships to the browser. |
-| **publishable** (`sb_publishable_…` / `anon`) | the **browser / Realtime** client | RLS-gated; used by the host app to subscribe to `run:<id>` broadcasts as the end user. |
-
-The adapter is server-only. The publishable key is for the host app's browser Realtime
-subscription, which the defense-in-depth RLS policy (`members_receive_run_events`)
-gates by org membership.
-
-## Applying the schema: install via `@nightowlsdev/cli`
-
-This package ships its migrations **inlined** (`MIGRATIONS`: `{ version, name, sql }[]`,
-fully-qualified `nightowls.*`) and exports the `nightOwlsPlugin` manifest that `@nightowlsdev/cli`
-discovers. You don't apply them from this package — instead the CLI **installs** them into
-your host's classic `supabase/migrations/`, and you apply them with your own tooling:
+Peer dependencies (must be installed by the host):
 
 ```bash
-owl install storage-supabase   # installs the migrations into supabase/migrations/
-supabase db push                  # apply them (your tooling — Night Owls never runs DDL)
+pnpm add @nightowlsdev/core @mastra/core @mastra/pg
 ```
 
-The install writes one `<timestamp>_corale_<version>.sql` per migration and is idempotent
-(already-installed versions are skipped). `owl db install` re-installs after an upgrade.
-
-## Mastra durable resume: `disableInit:true` + the `0002_mastra` migration
-
-On an RLS-enabled Supabase project you do **not** want `PostgresStore` to run its own
-DDL/init at runtime. `createMastraPgStore` defaults to `disableInit: true` and passes
-`schemaName: 'nightowls'`, so the `nightowls.mastra_*` tables **must pre-exist**. They are
-created by the packaged `0002_mastra` migration (the real `@mastra/pg` DDL via
-`exportSchemas('nightowls')`), including the resume-critical
-`nightowls.mastra_workflow_snapshot`. The installed migrations (above) create them.
-
 ## Usage
 
 ```ts
 import {
   createSupabaseStorage,
   createMastraPgStore,
+  createMastraVectorStore,
+  createPostgresFloor,
   publishAgentVersion,
 } from "@nightowlsdev/storage-supabase";
 import { SwarmEngine, allowListModelProvider } from "@nightowlsdev/core";
 
 const storage = createSupabaseStorage({
-  url: process.env.SUPABASE_URL!,        // API_URL — for supabase-js Realtime
-  secretKey: process.env.SUPABASE_SECRET_KEY!, // sb_secret_… / service_role (BYPASSRLS)
-  dbUrl: process.env.SUPABASE_DB_URL!,   // Session/Direct Postgres URL (port 5432)
-  // ssl defaults to `true` (verify cert) for remote dbUrl; localhost is unencrypted.
+  url: process.env.SUPABASE_URL!,           // API URL — for supabase-js Realtime
+  secretKey: process.env.SUPABASE_SECRET_KEY!, // service_role / sb_secret_… (BYPASSRLS)
+  dbUrl: process.env.DATABASE_URL!,         // Session/Direct Postgres URL (port 5432)
+  // ssl defaults to `true` (verify cert) for remote dbUrl; localhost stays unencrypted.
 });
 
 // Seed/publish an agent version (management helper, not part of the core interface).
-// Pass the adapter's typed internal handle `storage.ctx` (no `as any` coupling).
 await publishAgentVersion(storage.ctx, {
   tenantId: orgId,
   slug: "support",
@@ -93,23 +56,29 @@ await publishAgentVersion(storage.ctx, {
   capabilities: [],
   skillNames: [],
   delegateSlugs: [],
-  modelId: "openai/gpt-5.5",
+  modelId: "openai/gpt-4o",
 });
 
 // Cross-process durable suspend/resume: inject the Postgres-backed Mastra store.
+// Cross-instance mutual exclusion (serverless): inject the Postgres floor.
 const engine = new SwarmEngine({
   storage,
-  model: allowListModelProvider({ allow: ["openai/gpt-5.5"] }),
+  model: allowListModelProvider({ allow: ["openai/gpt-4o"] }),
   modelFactory: (modelId) => /* your LanguageModelV3 for modelId */ undefined as never,
   cost: { maxSteps: 10, maxCostUsd: 1 },
-  mastraStore: createMastraPgStore({ dbUrl: process.env.SUPABASE_DB_URL! }), // disableInit:true
+  mastraStore: createMastraPgStore({ dbUrl: process.env.DATABASE_URL! }),
+  floor: createPostgresFloor(storage.ctx.pool),
 });
 
+// Wire cross-process agent-cache invalidation (LISTEN/NOTIFY).
+// Call once, before any run; storage.close() tears it down.
+storage.subscribeInvalidations((key) => { /* key = "tenantId:slug" */ });
+
 for await (const e of engine.run({ message: "hi" }, ctx)) {
   // SwarmEvents are persisted to nightowls.events and broadcast on run:<id>
 }
 
-await storage.close(); // tears down Realtime channels, then ends the pg pool
+await storage.close(); // tears down the LISTEN connection, Realtime channels, then ends the pg pool
 ```
 
 Subscribe to the live event stream (server-side, BYPASSRLS):
@@ -120,6 +89,186 @@ for await (const event of storage.events.subscribe(runId)) {
 }
 ```
 
+## API
+
+### `createSupabaseStorage(opts: SupabaseStorageOpts): SupabaseStorage`
+
+Creates the Supabase `StorageAdapter`. Returns a `SupabaseStorage` which extends
+`StorageAdapter` with a typed internal handle (`ctx`) and a `close()` teardown.
+
+**`SupabaseStorageOpts`**
+
+| Field | Type | Purpose |
+|---|---|---|
+| `url` | `string` | Supabase project API URL — for supabase-js Realtime |
+| `secretKey` | `string` | `service_role` / `sb_secret_…` key — BYPASSRLS; never ships to the browser |
+| `dbUrl` | `string` | Session/Direct Postgres URL, **port 5432** (not 6543 — throws if detected) |
+| `max?` | `number` | pg pool size (default 10) |
+| `ssl?` | `boolean \| { rejectUnauthorized: boolean }` | TLS override; defaults to `true` (verify cert) for remote URLs |
+
+**`SupabaseStorage`** (returned value)
+
+Implements `StorageAdapter` plus:
+
+- `ctx: Ctx` — typed internal handle (`{ pool: Pool; sb: SupabaseClient }`); pass to
+  management helpers (`publishAgentVersion`, `createPostgresFloor`, etc.) without `as any`.
+- `subscribeInvalidations(onInvalidate: (key: string) => void): () => void` — registers a
+  Postgres LISTEN subscription (one per storage, enforced). `onInvalidate` is called with
+  `"tenantId:slug"` whenever `publishAgentVersion` NOTIFYs after a commit. Returns an
+  unsubscribe function; `close()` also tears it down.
+- `close(): Promise<void>` — releases the LISTEN connection, removes Realtime channels,
+  then ends the pg pool.
+
+---
+
+### `createPostgresFloor(pool: Pool, opts?: PostgresFloorOpts): ContainerFloor`
+
+A cross-instance `ContainerFloor` backed by Postgres (FIFO, heartbeat-liveness, stale-lock
+reclaim). Wire it into `SwarmEngine` for serverless mutual exclusion:
+
+```ts
+floor: createPostgresFloor(storage.ctx.pool)
+```
+
+**`PostgresFloorOpts`**
+
+| Field | Default | Purpose |
+|---|---|---|
+| `maxHoldMs?` | `180_000` | Stale-lock reclaim window (a held lock older than this is stealable) |
+| `pollMs?` | `250` | Poll interval while queued for the floor |
+
+---
+
+### `publishAgentVersion(ctx: Ctx, def: PublishAgentDef): Promise<{ version: number }>`
+
+Inserts a new agent version, flips the head pointer, audits, and NOTIFYs the R12
+cache-invalidation channel. Enforces the non-bypassable agent-bar (SP6): an `agent`
+principal can never publish. Returns the derived `(max+1)` version number.
+
+**`PublishAgentDef`** — `AgentVersionContent & { tenantId: string; actor?: SwarmActor; version?: number }`
+
+---
+
+### `rollbackAgentVersion(ctx: Ctx, args): Promise<{ version: number; restoredFrom: number }>`
+
+Append-only rollback: republishes a prior version's content as a NEW head (like `git revert`,
+not `reset`). Audited as `action='rollback'`. NOTIFYs R12 cache invalidation. Throws if
+`toVersion` doesn't exist for this tenant/slug.
+
+**`args`:** `{ tenantId: string; slug: string; toVersion: number; actor?: SwarmActor }`
+
+---
+
+### `listAgentVersions(ctx: Ctx, tenantId: string, slug: string): Promise<AgentVersionInfo[]>`
+
+Returns all versions for an agent (oldest → newest), flagging the current head. Use to
+pick a `toVersion` for `rollbackAgentVersion`. Returns `[]` for an unknown agent.
+
+---
+
+### `makeVersionedRepo(ctx: Ctx): VersionedRepo`
+
+The writable agent definition repo (read + publish + rollback + listVersions) backed by
+Postgres. `createSupabaseStorage` wires this as both `storage.agents` and
+`storage.agentsWritable`.
+
+---
+
+### `createMastraPgStore(opts: { dbUrl: string; disableInit?: boolean }): PostgresStore`
+
+A `@mastra/pg` `PostgresStore` scoped to the `nightowls` schema with `disableInit: true`
+(the `nightowls.mastra_*` tables are created by the packaged `0002_mastra` migration — not
+at runtime). Backs durable suspend/resume (`nightowls.mastra_workflow_snapshot`). Throws
+if `dbUrl` is port 6543.
+
+---
+
+### `createMastraVectorStore(opts: { dbUrl: string }): PgVector`
+
+A `@mastra/pg` `PgVector` scoped to the `nightowls` schema — backs semantic recall. The
+`pgvector` extension is enabled by the packaged `0004_memory_vector` migration; `PgVector`
+auto-creates its index table on first use. Throws if `dbUrl` is port 6543.
+
+---
+
+### `MIGRATIONS: Migration[]`
+
+The packaged SQL migration set (`{ version, name, sql }[]`, fully-qualified
+`nightowls.*`), consumed by `nightOwlsPlugin`. Covers 15 migrations: core schema,
+Mastra tables, followups, pgvector memory, thread text IDs, scratchpad, floor locks,
+Realtime presence/broadcast, thread-scoped resource IDs, schema rename, agent version
+immutability, and dropping the SaaS-only `tenant_policies`/`eval_runs` tables (host-owned,
+never used by the engine).
+
+---
+
+### `nightOwlsPlugin`
+
+The plugin manifest that `@nightowlsdev/cli` discovers. Declares:
+
+- `migrations` — ejected into the host's `supabase/migrations/` at install
+- `env` — `SUPABASE_URL`, `SUPABASE_SECRET_KEY`, `DATABASE_URL`, `SUPABASE_ANON_KEY`
+  merged into `.env.example`
+- `config` — import + wiring snippet inserted at the `// nightowls:storage` marker
+- `commands` — `owl storage-supabase info` (pure, no DB/network)
+
+---
+
+### Key exported types
+
+- `SupabaseStorageOpts` — constructor options
+- `SupabaseStorage` — the returned adapter (`StorageAdapter & { ctx; close; subscribeInvalidations }`)
+- `Ctx` — internal pool + supabase-js client handle (`{ pool: Pool; sb: SupabaseClient }`)
+- `PublishAgentDef` — argument to `publishAgentVersion`
+- `PostgresFloorOpts` — options for `createPostgresFloor`
+- `Migration` — one migration entry (`{ version, name, sql }`)
+- `AgentVersionInfo` — re-exported from `@nightowlsdev/core`
+
+## Configuration / Environment
+
+The env vars this package reads at runtime (via `SupabaseStorageOpts` / plugin manifest):
+
+| Variable | Purpose |
+|---|---|
+| `SUPABASE_URL` | Supabase project API URL (for supabase-js Realtime) |
+| `SUPABASE_SECRET_KEY` | `service_role` / `sb_secret_…` key — BYPASSRLS; server-only |
+| `DATABASE_URL` | Session/Direct Postgres URL — **port 5432** (not 6543) |
+| `SUPABASE_ANON_KEY` | publishable/anon key — browser auth + Realtime only (not read by the adapter itself) |
+
+### Use the Session/Direct pooler port (5432) — NEVER the Transaction pooler (6543)
+
+`pg` and `@mastra/pg` rely on prepared statements, which **break on the Transaction
+pooler (port 6543)**. Both `createSupabaseStorage` and `createMastraPgStore` and
+`createMastraVectorStore` throw if they detect `:6543` in the URL.
+
+```
+✅  postgresql://…@db.<ref>.supabase.co:5432/postgres   (Session/Direct)
+❌  postgresql://…@…pooler.supabase.com:6543/postgres    (Transaction — breaks pg)
+```
+
+## Applying the schema: install via `@nightowlsdev/cli`
+
+This package ships its migrations **inlined** and exports the `nightOwlsPlugin` manifest
+that `@nightowlsdev/cli` discovers. Install and apply:
+
+```bash
+owl install storage-supabase   # installs migrations into supabase/migrations/
+supabase db push                  # apply them (your tooling — Night Owls never runs DDL)
+```
+
+The install writes one `<timestamp>_nightowls_<version>.sql` per migration and is
+idempotent (already-installed versions are skipped). `owl db install` re-installs after an
+upgrade.
+
+## Plugin manifest
+
+`nightOwlsPlugin` is the declarative manifest consumed by `@nightowlsdev/cli` (`owl install`
+and `owl storage-supabase info`). It declares the migration set, the env vars to merge into
+`.env.example`, the import + wiring snippet for the `// nightowls:storage` marker, and a
+pure `info` subcommand that prints the installed migrations and required env vars. Night
+Owls never applies DDL itself — all schema changes go through `supabase db push` (or your
+equivalent tooling).
+
 ## Known limitations
 
 - **Multi-tenant read-scoping (C1) deferred.** `events.list`, `runs.loadSnapshot`, and
@@ -128,11 +277,6 @@ for await (const event of storage.events.subscribe(runId)) {
   pass it. For single-tenant v1 the caller (engine/runner) authorizes the `runId` /
   `threadId` before reading. Multi-tenant hardening = widening those core signatures to
   take `tenantId`, tracked for the runner/auth plan (Plan 3).
-- **Cross-instance row-cache bust deferred.** The adapter's per-instance `RowCache`
-  (LRU+TTL) for run→org lookups, and the engine's `AgentVersion` cache, are not busted
-  across instances. A `postgres_changes` subscription on `nightowls.agents.current_version_id`
-  to invalidate caches cluster-wide is deferred to the runner/multi-instance plan; the
-  Plan-1 TTL is sufficient until then.
 - **Other v1 simplifications:** `RunStore.setStatus` ignores its optional `patch`
   (status-only persistence); `runner` is hardcoded `'nextjs'` (`NewRun` has no runner
   field); `events.subscribe` shares the adapter's single secret-role `SupabaseClient`