@north-light/sdk 0.3.544 → 0.3.545

package/README.md

@@ -176,6 +176,10 @@ CREATE INDEX connection_updated_at_idx ON connection (updated_at, user_id);
 
 `save` upserts all fields and sets `updated_at` to the database clock. `listStale(before)` selects connections with `updated_at < before`, oldest first; it is a **last save** timestamp, not necessarily last successful refresh if an app also calls `save` for another reason. For idle-token maintenance, schedule `oauth.refreshIdleConnections({olderThanMs: 30 * 24 * 60 * 60 * 1000})` (for example, daily). It selects stale connections, refreshes under the store lock, reloads there to avoid rotating an already-used token, saves a rotated connection, and reports `{refreshed, refused, failures}`. A connection the directory refuses goes to the client's `onRefusal(refusal)` and is then removed (`remove` deletes the row only while it still holds the refused token); `refusal.kind` is `'removed'` (`grant_removed`: the runtime deleted the app's runs; delete what you store for the person), `'paused'` (`grant_suspended`: nothing was deleted; keep the data until they sign in again), or `'reconnect'` (`refresh_token_expired`, `refresh_token_reused`, `grant_revoked`: nothing was deleted; ask them to sign in again). `refusalOf(error)` returns the same `Refusal` from a refused call's error. Load the signing key `crouter-sdk keygen` wrote with `privateKey: await privateKeyFromFile('local/app-key.json')` (Node), or check one from an environment variable with `parsePrivateJwk(text)`; both refuse anything but a private JWK with a `kid`. The SDK does not start a timer. `listStale` returns connections, not timestamps, so a worker needing a strict cutoff recheck must maintain that policy separately. The table contains credentials: restrict database access, encrypt backups, and delete a row when its user disconnects. PostgreSQL `timestamptz` must be returned as a JavaScript `Date` (the `pg` default parser).
 
+## PostgreSQL object kit (Node)
+
+`PostgresObjectStore` from `@north-light/sdk/stores/postgres` keeps per-person custom objects, the nodes watching them, and a durable publish outbox. `objectKitSchemaSql(tables?)` returns the DDL for the three tables (`watch_object`, `watch_watcher`, `watch_publish` by default) for your migration. `new PostgresObjectStore(pool, {providerId, runtime: (sub) => oauth.appClient(sub), tables?, log?})` has `instance(sub, kind, target, {name, description})`, `watch(objectId, node, machine?)`, `enqueue(client, objectId, {event, summary, state?})` (pass your open transaction's client), `tick(batch?)` and `startPublisher({tick: 5000, batch: 100})`. Delivery is at least once under an advisory lock, backs off to 15 minutes, and drops a row after 24 hours or a permanent refusal (HTTP 400, 404 or 422). Full guide: [Watchable objects](https://sdk.docs.northlight.ai/docs/guides/watchable-objects).
+
 ## Example application
 
 See the complete [localhost SDK demo](https://github.com/Northlight-Systems/crouter/tree/main/examples/localhost-demo) for streaming, structured output, and an HTTP plugin in one local page.

package/dist/skill/northlight/SKILL.md

@@ -2,7 +2,7 @@
 name: northlight
 description: "Use when building, debugging or shipping an app that signs people in with Northlight, starts runs on their Northlight runtime, or calls Northlight providers (sms, search, integrations, context, device) with @north-light/sdk."
 metadata:
-  sdk-version: "0.3.544"
+  sdk-version: "0.3.545"
 ---
 
 # Building a Northlight app
@@ -15,7 +15,7 @@ The default Northlight app is one people text. It gets its own iMessage number t
 
 [comms.northlight.ai/llms.txt](https://comms.northlight.ai/llms.txt) is the whole Comms guide: read it and follow it rather than this summary. [Reach people by text](https://sdk.docs.northlight.ai/docs/reach-people-by-text.md) shows the webhook handler.
 
-1. **Build in fake mode.** `npx @north-light/sdk@latest init <dir>`, then `npm i @north-light/comms --legacy-peer-deps` (its 0.3.0 release pins an older `@north-light/sdk` as an optional peer). Run `npx @north-light/comms fake --webhook http://localhost:3000/hook`, and play a person with `npx @north-light/comms fake send --from +15550001 --text "hi"`. No account is needed yet.
+1. **Build in fake mode.** `npx @north-light/sdk@latest init <dir>`, then `npm i @north-light/comms`. Run `npx @north-light/comms fake --webhook http://localhost:3000/hook`, and play a person with `npx @north-light/comms fake send --from +15550001 --text "hi"`. No account is needed yet.
 2. **Sign up, log in, register.** Steps 1 to 4 of the journey below. Comms signs in with the same app key.
 3. **Connect the app to Comms.** Ask the person to sign in at `https://comms.northlight.ai` and choose **Connect an app**; an admin of the app approves it there. This is done in a browser only.
 4. **Turn Texting on**, on the app's page there or with `crtr comms turn-on`: display name, square avatar, HELP text, test webhook URL. Copy the test signing secret it shows into the server's environment.
@@ -27,7 +27,7 @@ Never text a person first: Comms refuses a send to anyone who has not texted you
 
 ## First: check this skill matches the SDK
 
-This skill is for SDK version `0.3.544`. If the project has `node_modules/@north-light/sdk/package.json`, compare its `version` with that number. If they differ, run `npx @north-light/sdk skill` in the project, then read the rewritten `.agents/skills/northlight/SKILL.md` (or `.claude/skills/northlight/SKILL.md`) instead of this copy. It needs no account. A project that has no SDK yet starts with `npx @north-light/sdk@latest init <dir>`.
+This skill is for SDK version `0.3.545`. If the project has `node_modules/@north-light/sdk/package.json`, compare its `version` with that number. If they differ, run `npx @north-light/sdk skill` in the project, then read the rewritten `.agents/skills/northlight/SKILL.md` (or `.claude/skills/northlight/SKILL.md`) instead of this copy. It needs no account. A project that has no SDK yet starts with `npx @north-light/sdk@latest init <dir>`.
 
 ## Where things run
 

package/dist/stores/postgres.d.ts

@@ -1,3 +1,4 @@
+import type { AppClient } from '../client.js';
 import type { Connection, ConnectionStore } from '../oauth/index.js';
 interface QueryResult {
     rows: Record<string, unknown>[];
@@ -29,4 +30,89 @@ export declare class PostgresConnectionStore implements ConnectionStore {
     /** Connections not saved since `before`, oldest first. Save activity is not necessarily refresh activity. */
     listStale(before: Date): Promise<Connection[]>;
 }
+/** Anything with `query`: a `pg.Pool`, a `pg.PoolClient`, or a client inside the caller's open transaction. */
+export interface PostgresQueryable {
+    query(sql: string, values?: unknown[]): Promise<QueryResult>;
+}
+/** The part of a person's runtime client the kit uses. `ConnectedCrouter` (`oauth.appClient(sub)`) satisfies it. */
+export type ObjectRuntime = Pick<AppClient, 'customObjects' | 'canvas'>;
+export interface ObjectKitTables {
+    /** One row per (person, kind, target): the registered custom object. Default `watch_object`. */
+    object?: string;
+    /** Nodes linked to an object. Default `watch_watcher`. */
+    watcher?: string;
+    /** The publish outbox. Default `watch_publish`. */
+    publish?: string;
+}
+/** The DDL the app migrates for the object kit. Table names follow `ObjectKitTables` (lowercase identifiers, optionally schema-qualified). */
+export declare function objectKitSchemaSql(tables?: ObjectKitTables): string;
+export interface PostgresObjectStoreOptions {
+    /** The app's provider id (`CROUTER_CLIENT_ID`). A kind `problem` registers as the custom-object kind `<providerId>.problem`, which the plugin's `objectKinds` declares. */
+    providerId: string;
+    /** The app's client on a person's runtime, e.g. `(sub) => oauth.appClient(sub)`. */
+    runtime(sub: string): Promise<ObjectRuntime>;
+    tables?: ObjectKitTables;
+    /** Receives retry and drop notices from the publisher. Default `console`. */
+    log?: {
+        warn(message: string): void;
+        error(message: string, error?: unknown): void;
+    };
+}
+export interface ObjectInstance {
+    objectId: string;
+    sub: string;
+    kind: string;
+    target: string;
+    name: string;
+}
+export interface PublishRequest {
+    event: string;
+    /** Truncated to 500 characters. */
+    summary: string;
+    state?: Record<string, unknown> | null;
+}
+export interface PublisherOptions {
+    /** Milliseconds between ticks. Default 5000. */
+    tick?: number;
+    /** Rows delivered per tick. Default 100. */
+    batch?: number;
+}
+export interface Publisher {
+    /** Stops the timer and waits for a tick in progress. */
+    stop(): Promise<void>;
+}
+/** A publish not delivered this long after it was enqueued is dropped. */
+export declare const PUBLISH_WINDOW_MS: number;
+/**
+ * Per-person custom objects on each person's hosted runtime, the nodes that watch them, and an at-least-once publish
+ * outbox. The app migrates the tables (`objectKitSchemaSql`), registers the kinds in its plugin, calls `instance` and
+ * `watch` from its commands, calls `enqueue` inside the transaction of the write an event records, and runs
+ * `startPublisher` in each process.
+ */
+export declare class PostgresObjectStore {
+    private readonly pool;
+    private readonly options;
+    private readonly object;
+    private readonly watcher;
+    private readonly publish;
+    private readonly lockKey;
+    private readonly log;
+    constructor(pool: PostgresPool, options: PostgresObjectStoreOptions);
+    /** The person's object for (kind, target), registered on their runtime the first time. A concurrent first call registers a second object; the row that committed first wins and the loser is unregistered. `target` is `''` for a kind with one object per person. */
+    instance(sub: string, kind: string, target: string, meta: {
+        name: string;
+        description: string;
+    }): Promise<ObjectInstance>;
+    /** Links `node` (on `machine`, `undefined` or `''` for the person's hosted runtime) to the object, then records the link. A node the runtime does not know rejects with `NotFoundError`. */
+    watch(objectId: string, node: string, machine?: string): Promise<void>;
+    /** Adds a publish row. Pass the client of the caller's open transaction so the row exists exactly when the write it records does. */
+    enqueue(client: PostgresQueryable, objectId: string, event: PublishRequest): Promise<void>;
+    /** Delivers due rows once under the advisory lock; false when another process holds it. */
+    tick(batch?: number): Promise<boolean>;
+    /** Runs `tick` every `tick` ms (default 5000, 100 rows). A failed tick is logged and the next one runs. */
+    startPublisher(options?: PublisherOptions): Publisher;
+    private find;
+    private deliver;
+    private end;
+}
 export {};

package/dist/stores/postgres.js

@@ -1,4 +1,7 @@
 import { AsyncLocalStorage } from 'node:async_hooks';
+import { randomUUID } from 'node:crypto';
+import { customObjectPublishFailure } from '@north-light/plugin';
+import { APIError } from '../errors.js';
 function tableName(name) {
     const parts = name.split('.');
     if (parts.length < 1 || parts.length > 2 || parts.some((part) => !/^[a-z_][a-z_0-9]*$/.test(part))) {
@@ -90,3 +93,182 @@ export class PostgresConnectionStore {
         return result.rows.map(toConnection);
     }
 }
+const DEFAULT_TABLES = { object: 'watch_object', watcher: 'watch_watcher', publish: 'watch_publish' };
+/** The DDL the app migrates for the object kit. Table names follow `ObjectKitTables` (lowercase identifiers, optionally schema-qualified). */
+export function objectKitSchemaSql(tables = {}) {
+    const t = { ...DEFAULT_TABLES, ...tables };
+    const [object, watcher, publish] = [t.object, t.watcher, t.publish].map(tableName);
+    const base = t.publish.split('.').pop();
+    return `CREATE TABLE ${object} (
+  object_id text PRIMARY KEY,
+  sub text NOT NULL,
+  kind text NOT NULL,
+  target text NOT NULL,
+  name text NOT NULL,
+  created_at timestamptz NOT NULL DEFAULT now(),
+  UNIQUE (sub, kind, target)
+);
+CREATE TABLE ${watcher} (
+  object_id text NOT NULL REFERENCES ${object} (object_id) ON DELETE CASCADE,
+  node text NOT NULL,
+  machine text NOT NULL DEFAULT '',
+  PRIMARY KEY (object_id, node, machine)
+);
+CREATE TABLE ${publish} (
+  id text PRIMARY KEY,
+  object_id text NOT NULL REFERENCES ${object} (object_id) ON DELETE CASCADE,
+  event text NOT NULL,
+  summary text NOT NULL,
+  state jsonb,
+  status text NOT NULL DEFAULT 'pending',
+  attempts integer NOT NULL DEFAULT 0,
+  last_error text,
+  next_attempt_at timestamptz NOT NULL DEFAULT now(),
+  created_at timestamptz NOT NULL DEFAULT now()
+);
+CREATE INDEX "${base}_due_idx" ON ${publish} (status, next_attempt_at);
+CREATE INDEX "${base}_object_idx" ON ${publish} (object_id);
+`;
+}
+/** A publish not delivered this long after it was enqueued is dropped. */
+export const PUBLISH_WINDOW_MS = 24 * 60 * 60 * 1000;
+const MAX_BACKOFF_MS = 15 * 60 * 1000;
+/**
+ * Per-person custom objects on each person's hosted runtime, the nodes that watch them, and an at-least-once publish
+ * outbox. The app migrates the tables (`objectKitSchemaSql`), registers the kinds in its plugin, calls `instance` and
+ * `watch` from its commands, calls `enqueue` inside the transaction of the write an event records, and runs
+ * `startPublisher` in each process.
+ */
+export class PostgresObjectStore {
+    pool;
+    options;
+    object;
+    watcher;
+    publish;
+    lockKey;
+    log;
+    constructor(pool, options) {
+        this.pool = pool;
+        this.options = options;
+        const t = { ...DEFAULT_TABLES, ...options.tables };
+        this.object = tableName(t.object);
+        this.watcher = tableName(t.watcher);
+        this.publish = tableName(t.publish);
+        this.lockKey = `north-light-sdk-object-publish:${t.publish}`;
+        this.log = options.log ?? { warn: (m) => console.warn(m), error: (m, e) => console.error(m, e) };
+    }
+    /** The person's object for (kind, target), registered on their runtime the first time. A concurrent first call registers a second object; the row that committed first wins and the loser is unregistered. `target` is `''` for a kind with one object per person. */
+    async instance(sub, kind, target, meta) {
+        const existing = await this.find(sub, kind, target);
+        if (existing)
+            return existing;
+        const runtime = await this.options.runtime(sub);
+        const registered = await runtime.customObjects.register({
+            kind: `${this.options.providerId}.${kind}`,
+            name: meta.name,
+            description: meta.description,
+            privacy: 'private',
+        });
+        const inserted = await this.pool.query(`INSERT INTO ${this.object} (object_id, sub, kind, target, name) VALUES ($1, $2, $3, $4, $5)
+       ON CONFLICT (sub, kind, target) DO NOTHING RETURNING object_id`, [registered.id, sub, kind, target, meta.name]);
+        if (inserted.rows.length > 0)
+            return { objectId: registered.id, sub, kind, target, name: meta.name };
+        const winner = await this.find(sub, kind, target);
+        if (!winner)
+            throw new Error(`object ${sub}/${kind}/${target} vanished during registration`);
+        if (winner.objectId !== registered.id)
+            await runtime.customObjects.unregister(registered.id);
+        return winner;
+    }
+    /** Links `node` (on `machine`, `undefined` or `''` for the person's hosted runtime) to the object, then records the link. A node the runtime does not know rejects with `NotFoundError`. */
+    async watch(objectId, node, machine) {
+        const found = await this.pool.query(`SELECT sub FROM ${this.object} WHERE object_id = $1`, [objectId]);
+        const sub = found.rows[0]?.sub;
+        if (sub === undefined)
+            throw new Error(`no object ${objectId}`);
+        const runtime = await this.options.runtime(sub);
+        await runtime.canvas.watch(objectId, { watcher: node, ...(machine ? { watcher_machine: machine } : {}) });
+        await this.pool.query(`INSERT INTO ${this.watcher} (object_id, node, machine) VALUES ($1, $2, $3) ON CONFLICT DO NOTHING`, [objectId, node, machine ?? '']);
+    }
+    /** Adds a publish row. Pass the client of the caller's open transaction so the row exists exactly when the write it records does. */
+    async enqueue(client, objectId, event) {
+        await client.query(`INSERT INTO ${this.publish} (id, object_id, event, summary, state) VALUES ($1, $2, $3, $4, $5::jsonb)`, [randomUUID(), objectId, event.event, event.summary.slice(0, 500), event.state == null ? null : JSON.stringify(event.state)]);
+    }
+    /** Delivers due rows once under the advisory lock; false when another process holds it. */
+    async tick(batch = 100) {
+        const lock = await this.pool.connect();
+        try {
+            const got = await lock.query('SELECT pg_try_advisory_lock(hashtextextended($1, 0)) AS ok', [this.lockKey]);
+            if (!got.rows[0]?.ok)
+                return false;
+            try {
+                const due = await this.pool.query(`SELECT p.id, p.object_id, o.sub, p.event, p.summary, p.state, p.attempts, p.created_at
+           FROM ${this.publish} p JOIN ${this.object} o ON o.object_id = p.object_id
+           WHERE p.status = 'pending' AND p.next_attempt_at <= now()
+           ORDER BY p.created_at ASC, p.id ASC LIMIT $1`, [batch]);
+                for (const row of due.rows)
+                    await this.deliver(row);
+            }
+            finally {
+                await lock.query('SELECT pg_advisory_unlock(hashtextextended($1, 0))', [this.lockKey]);
+            }
+            return true;
+        }
+        finally {
+            lock.release();
+        }
+    }
+    /** Runs `tick` every `tick` ms (default 5000, 100 rows). A failed tick is logged and the next one runs. */
+    startPublisher(options = {}) {
+        let running;
+        const timer = setInterval(() => {
+            if (running)
+                return; // a pass is still delivering: skip this firing
+            running = this.tick(options.batch)
+                .catch((error) => this.log.error('object publish tick failed', error))
+                .finally(() => { running = undefined; });
+        }, options.tick ?? 5_000);
+        timer.unref();
+        return {
+            stop: async () => {
+                clearInterval(timer);
+                await running;
+            },
+        };
+    }
+    async find(sub, kind, target) {
+        const result = await this.pool.query(`SELECT object_id, name FROM ${this.object} WHERE sub = $1 AND kind = $2 AND target = $3`, [sub, kind, target]);
+        const row = result.rows[0];
+        return row ? { objectId: row.object_id, sub, kind, target, name: row.name } : null;
+    }
+    async deliver(row) {
+        const now = Date.now();
+        if (now - row.created_at.getTime() > PUBLISH_WINDOW_MS)
+            return this.end(row, 'dropped', '24 h window elapsed');
+        try {
+            const runtime = await this.options.runtime(row.sub);
+            await runtime.customObjects.publish(row.object_id, {
+                event: row.event,
+                summary: row.summary,
+                ...(row.state == null ? {} : { state: row.state }),
+            });
+        }
+        catch (error) {
+            const message = error instanceof Error ? error.message : String(error);
+            const refusal = error instanceof APIError ? { status: error.status, code: error.code } : undefined;
+            if (refusal && customObjectPublishFailure(refusal) === 'drop') {
+                return this.end(row, 'dropped', `${refusal.code}: ${message}`);
+            }
+            const attempts = row.attempts + 1;
+            await this.pool.query(`UPDATE ${this.publish} SET attempts = $2, last_error = $3, next_attempt_at = now() + $4 * interval '1 millisecond' WHERE id = $1`, [row.id, attempts, message.slice(0, 500), Math.min(2 ** attempts * 10_000, MAX_BACKOFF_MS)]);
+            this.log.warn(`publish ${row.id} attempt ${attempts} failed: ${message}`);
+            return;
+        }
+        await this.end(row, 'done', null);
+    }
+    async end(row, status, lastError) {
+        if (status === 'dropped')
+            this.log.warn(`dropped publish ${row.id}: ${lastError}`);
+        await this.pool.query(`UPDATE ${this.publish} SET status = $2, last_error = $3, attempts = attempts + 1 WHERE id = $1`, [row.id, status, lastError]);
+    }
+}

package/dist/version.d.ts

@@ -1,5 +1,5 @@
 /** This SDK's release version. Informational only: compatibility is the contract level, not this number. */
-export declare const SDK_VERSION = "0.3.544";
+export declare const SDK_VERSION = "0.3.545";
 /** The release sent in `Crouter-Minimum-Runtime-Version` on every request. It is frozen, so a runtime from before the
  *  contract level existed (which only knows this header) refuses this SDK, while every later runtime accepts it. */
 export declare const LEGACY_FLOOR = "0.3.423";

package/dist/version.js

@@ -1,6 +1,6 @@
 // SDK_VERSION is written by scripts/release-version.mjs on every release; do not edit by hand.
 /** This SDK's release version. Informational only: compatibility is the contract level, not this number. */
-export const SDK_VERSION = '0.3.544';
+export const SDK_VERSION = '0.3.545';
 // LEGACY_FLOOR is written once by `scripts/contract.mjs release`, at the first release that carries the contract level
 // (the `0.0.0` placeholder is replaced there and never again); do not edit by hand.
 /** The release sent in `Crouter-Minimum-Runtime-Version` on every request. It is frozen, so a runtime from before the

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@north-light/sdk",
-  "version": "0.3.544",
+  "version": "0.3.545",
   "crouterContract": "1.29",
   "description": "Typed Node and browser client for running crouter agents through the crtrd /v1 API.",
   "type": "module",
@@ -44,8 +44,9 @@
     "prepublishOnly": "node -e \"if (process.env.npm_config_tag !== 'next') { console.error('@north-light/sdk publishes only with --tag next; scripts/promote-sdk.mjs is the only mover of latest.'); process.exit(1); }\""
   },
   "dependencies": {
-    "@north-light/api": "0.3.544",
-    "@north-light/identity": "0.3.544",
+    "@north-light/api": "0.3.545",
+    "@north-light/identity": "0.3.545",
+    "@north-light/plugin": "0.3.545",
     "jose": "^6.2.1"
   },
   "peerDependencies": {