@syncular/server 0.2.1 → 0.3.0

package/README.md

@@ -49,18 +49,16 @@ landing in a plain isolate wakes the partition's DO (the in-platform
 LISTEN/NOTIFY analogue). Full shape, wiring, hibernation semantics, and the
 manual real-workerd smoke recipe in `@syncular/server-workers/README.md`.
 
-**Relay does not return (decision).** v1 shipped a *relay* — a bridge that let
-a self-hosted server forward realtime to a managed realtime service, because
-v1's realtime was a separate socketed subsystem the self-hosted core couldn't
-serve on its own. v2 has no such gap: realtime is the **second binding of the
-same handler** (§8.7, Direction decision 1 — the WS-native loop), so any host
-that runs the core serves realtime directly; multi-instance fanout is covered
-by **LISTEN/NOTIFY** on Postgres (below), and the Workers case is covered by
-the **DO design** (writes and sockets co-located per partition). Every job the
-relay did is now done by a binding of the core or by in-database fanout —
-reintroducing a relay would add a hop, a second protocol surface, and a
-managed dependency for zero capability the core lacks. So it is retired, not
-ported.
+**No relay (decision).** There is deliberately no *relay* — no bridge that
+forwards realtime from a self-hosted server to a managed realtime service.
+Realtime is the **second binding of the same handler** (§8.7, Direction
+decision 1 — the WS-native loop), so any host that runs the core serves
+realtime directly; multi-instance fanout is covered by **LISTEN/NOTIFY** on
+Postgres (below), and the Workers case is covered by the **DO design**
+(writes and sockets co-located per partition). Every job a relay would do is
+done by a binding of the core or by in-database fanout — a relay would add a
+hop, a second protocol surface, and a managed dependency for zero capability
+the core lacks.
 
 ## Structured events (the ops seam)
 
@@ -127,8 +125,8 @@ exists) `partition` / `actorId`.
 The operator-facing read surface over the server core. It is a module in
 this package — **not** a separate UI package — and adds **zero** wire
 protocol: SPEC.md says nothing about it, because authorization for these
-reads is entirely the host's. It is the v2 answer to v1's full React
-console app: the same 80% operator value (who's connected, what's flowing,
+reads is entirely the host's. It delivers the 80% operator value (who's
+connected, what's flowing,
 horizon health, the event tail) as a handful of read-only, partition-scoped,
 JSON-able queries.
 
@@ -215,8 +213,8 @@ framework, no build step, no React. It fetches the sibling JSON endpoints
 (relative to its own mount path, so it works under any prefix and the same
 guard covers its XHRs), renders tables for horizon, store stats, clients,
 recent commits, and the event tail, with an auto-refresh toggle (2 s poll).
-This is the ~300-line answer to v1's console app: 5% of the code, the 80%
-operator value.
+This is a deliberate ~300-line console: 5% of the code a full console app
+would cost, the 80% operator value.
 
 **No SSE (yet).** `GET /events` is a polled ring query; the page's
 auto-refresh polls it. Server-Sent-Events streaming was deliberately
@@ -571,8 +569,7 @@ whose cursor record was touched within `activeWindowMs` (default 14
 days). Two escape hatches keep laggards from pinning the log forever:
 commits older than `ageForceMs` (default 30 days) may be pruned
 regardless, and at least the newest `minRetainedCommits` (default 1000)
-commits are always kept. Defaults are the v1 production values; raise
-them freely, lower them with care.
+commits are always kept. Raise the defaults freely, lower them with care.
 
 **What `sync.cursor_expired` means operationally.** A client whose
 cursor fell behind the horizon gets `SUB_START.status = reset` and
@@ -624,7 +621,7 @@ same table+scope during a storm, your TTL is shorter than the storm.
 production, `PostgresServerStorage` implements the same `ServerStorage`
 contract against Postgres, with the inverted scope index carried through
 as **covering indexes** so scope fanout is an index range scan, never a
-scan-before-LIMIT (REVISE B2 — this was v1's production wound). The
+scan-before-LIMIT. The
 schema (`POSTGRES_DDL`) and its index design live in
 `src/postgres-storage.ts`; `storage.migrate()` applies it idempotently
 (every DDL is `CREATE … IF NOT EXISTS`, run statement-by-statement, so

package/dist/admin.d.ts

@@ -2,9 +2,9 @@
  * `SyncularAdmin` — the operator-facing read surface over the server core
  * (TODO §2.5). A read-only, partition-scoped, JSON-able query layer over
  * `ServerStorage`, the optional segment/blob store stats, and an in-memory
- * event ring. This is v2's answer to v1's full React console app: the same
- * 80% operator value (who's connected, what's flowing, horizon health, the
- * event tail) as a handful of queries in the server package — no separate
+ * event ring. It delivers the 80% operator value (who's connected, what's
+ * flowing, horizon health, the event tail) as a handful of queries in the
+ * server package — no separate
  * UI package, no framework, no wire-protocol surface (SPEC.md is untouched;
  * this is host surface, mirrored in the server README).
  *

package/dist/index.d.ts

@@ -32,6 +32,7 @@ export * from './s3-blob-store.js';
 export * from './s3-segment-store.js';
 export * from './schema.js';
 export * from './scopes.js';
+export * from './seed.js';
 export * from './segment-download.js';
 export * from './segment-store.js';
 export * from './signed-url.js';

package/dist/index.js

@@ -36,6 +36,7 @@ export * from './s3-blob-store.js';
 export * from './s3-segment-store.js';
 export * from './schema.js';
 export * from './scopes.js';
+export * from './seed.js';
 export * from './segment-download.js';
 export * from './segment-store.js';
 export * from './signed-url.js';

package/dist/seed.d.ts

@@ -0,0 +1,34 @@
+import type { SyncServerConfig } from './context.js';
+/** One app-shaped seed mutation — the same vocabulary as client mutations. */
+export type SeedMutation = {
+    readonly table: string;
+    readonly op: 'upsert';
+    /**
+     * Full-row values keyed by column name (§6.1). Keys are accepted in
+     * the SQL-truth snake_case or the generated row types' camelCase;
+     * missing nullable columns become NULL.
+     */
+    readonly values: Readonly<Record<string, unknown>>;
+} | {
+    readonly table: string;
+    readonly op: 'delete';
+    readonly rowId: string;
+};
+export interface SeedTarget {
+    readonly partition: string;
+    readonly actorId: string;
+    /**
+     * The synthetic client identity of the seed (default `'seed'`). Together
+     * with `commitId` it forms the §2.3 idempotency key — keep both stable
+     * for a re-runnable seed, vary `commitId` to seed additional batches.
+     */
+    readonly clientId?: string;
+    /** The client commit id (default `'seed-commit-1'`). */
+    readonly commitId?: string;
+}
+/**
+ * Seed `mutations` into a partition through the real push path. Throws a
+ * `SyncError` when the push is rejected or any operation fails, so a broken
+ * seed fails loud at boot instead of silently serving an empty database.
+ */
+export declare function seedMutations(config: SyncServerConfig, target: SeedTarget, mutations: readonly SeedMutation[]): Promise<void>;

package/dist/seed.js

@@ -0,0 +1,92 @@
+/**
+ * The public seeding helper (RFC 0002 §2.5): push app-shaped mutations
+ * through the REAL push path — one `handleSyncRequest` round built from the
+ * server schema — so demos, dev servers, and ops scripts seed data with one
+ * supported call. This is the same §6 pipeline every client write takes
+ * (authorization, validation, idempotency, realtime fanout), so seeded rows
+ * behave exactly like synced rows.
+ *
+ * Idempotent by construction (§2.3): the commit id is stable, so re-running
+ * the seed replays the cached push result and writes nothing twice.
+ */
+import { decodeMessage, encodeMessage, encodeRow, PROTOCOL_WIRE_VERSION, } from '@syncular/core';
+import { SyncError } from './errors.js';
+import { handleSyncRequest } from './handler.js';
+function camelToSnake(name) {
+    return name.replace(/[A-Z]/g, (ch) => `_${ch.toLowerCase()}`);
+}
+/**
+ * Seed `mutations` into a partition through the real push path. Throws a
+ * `SyncError` when the push is rejected or any operation fails, so a broken
+ * seed fails loud at boot instead of silently serving an empty database.
+ */
+export async function seedMutations(config, target, mutations) {
+    const clientId = target.clientId ?? 'seed';
+    const clientCommitId = target.commitId ?? 'seed-commit-1';
+    const operations = mutations.map((mutation) => {
+        const table = config.schema.tables.find((t) => t.name === mutation.table);
+        if (table === undefined) {
+            throw new SyncError('sync.invalid_request', `seedMutations: unknown table ${JSON.stringify(mutation.table)}`);
+        }
+        if (mutation.op === 'delete') {
+            return { table: mutation.table, rowId: mutation.rowId, op: 'delete' };
+        }
+        // Normalize record keys to column positions (snake_case or camelCase,
+        // the two casings the client tier accepts).
+        const byColumn = new Map();
+        for (const [key, value] of Object.entries(mutation.values)) {
+            const name = table.columns.some((c) => c.name === key)
+                ? key
+                : camelToSnake(key);
+            if (!table.columns.some((c) => c.name === name)) {
+                throw new SyncError('sync.invalid_request', `seedMutations: table ${table.name}: unknown column ${JSON.stringify(key)}`);
+            }
+            if (byColumn.has(name)) {
+                throw new SyncError('sync.invalid_request', `seedMutations: table ${table.name}: column ${JSON.stringify(name)} appears twice (snake_case and camelCase) — pass it once`);
+            }
+            byColumn.set(name, value);
+        }
+        const values = table.columns.map((column) => (byColumn.get(column.name) ?? null));
+        const pkIndex = table.columns.findIndex((c) => c.name === table.primaryKey);
+        const rowId = values[pkIndex];
+        if (typeof rowId !== 'string' || rowId.length === 0) {
+            throw new SyncError('sync.invalid_request', `seedMutations: table ${table.name}: upsert requires a non-empty string primary key`);
+        }
+        return {
+            table: mutation.table,
+            rowId,
+            op: 'upsert',
+            payload: encodeRow(table.columns, values),
+        };
+    });
+    const frames = [
+        { type: 'REQ_HEADER', clientId, schemaVersion: config.schema.version },
+        { type: 'PUSH_COMMIT', clientCommitId, operations },
+        // A pull that asks for nothing: the round exists for its push half.
+        {
+            type: 'PULL_HEADER',
+            limitCommits: 0,
+            limitSnapshotRows: 0,
+            maxSnapshotPages: 0,
+            accept: 0b0011,
+        },
+    ];
+    const response = await handleSyncRequest(encodeMessage({
+        wireVersion: PROTOCOL_WIRE_VERSION,
+        msgKind: 'request',
+        frames,
+    }), { ...config, partition: target.partition, actorId: target.actorId });
+    // Fail loud: surface the first rejected/failed operation.
+    const message = decodeMessage(response);
+    const result = message.frames.find((frame) => frame.type === 'PUSH_RESULT' && frame.clientCommitId === clientCommitId);
+    if (result === undefined) {
+        throw new SyncError('sync.invalid_request', 'seedMutations: the sync response carried no push result');
+    }
+    if (result.status === 'rejected') {
+        const failed = result.results.find((r) => r.status !== 'applied');
+        const detail = failed !== undefined && 'code' in failed
+            ? ` (op ${failed.opIndex}: ${failed.code} — ${failed.message})`
+            : '';
+        throw new SyncError('sync.invalid_request', `seedMutations: the seed commit was rejected${detail}`);
+    }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@syncular/server",
-  "version": "0.2.1",
+  "version": "0.3.0",
   "description": "Syncular server: handleSyncRequest + storage/auth interfaces for the sync protocol",
   "license": "Apache-2.0",
   "author": "Benjamin Kniffler",
@@ -28,7 +28,7 @@
   "exports": {
     ".": {
       "bun": "./src/index.ts",
-      "browser": "./src/index.ts",
+      "browser": "./dist/index.js",
       "import": {
         "types": "./dist/index.d.ts",
         "default": "./dist/index.js"
@@ -36,7 +36,7 @@
     },
     "./pglite": {
       "bun": "./src/pg-executor-pglite.ts",
-      "browser": "./src/pg-executor-pglite.ts",
+      "browser": "./dist/pg-executor-pglite.js",
       "import": {
         "types": "./dist/pg-executor-pglite.d.ts",
         "default": "./dist/pg-executor-pglite.js"

package/src/admin.ts

@@ -2,9 +2,9 @@
  * `SyncularAdmin` — the operator-facing read surface over the server core
  * (TODO §2.5). A read-only, partition-scoped, JSON-able query layer over
  * `ServerStorage`, the optional segment/blob store stats, and an in-memory
- * event ring. This is v2's answer to v1's full React console app: the same
- * 80% operator value (who's connected, what's flowing, horizon health, the
- * event tail) as a handful of queries in the server package — no separate
+ * event ring. It delivers the 80% operator value (who's connected, what's
+ * flowing, horizon health, the event tail) as a handful of queries in the
+ * server package — no separate
  * UI package, no framework, no wire-protocol surface (SPEC.md is untouched;
  * this is host surface, mirrored in the server README).
  *

package/src/index.ts

@@ -36,6 +36,7 @@ export * from './s3-blob-store';
 export * from './s3-segment-store';
 export * from './schema';
 export * from './scopes';
+export * from './seed';
 export * from './segment-download';
 export * from './segment-store';
 export * from './signed-url';

package/src/postgres-storage.ts

@@ -3,8 +3,8 @@
  *
  * Semantics mirror `SqliteServerStorage` exactly (both run the shared
  * storage contract in `test/storage-contract.ts`); the difference is that
- * scope fanout MUST survive contact with Postgres. v1's production wound was
- * scan-before-LIMIT here, so (REVISE B2, performance-by-construction):
+ * scope fanout MUST survive contact with Postgres — no scan-before-LIMIT,
+ * ever (performance-by-construction):
  *
  *   - both the commit log and the current-row table carry a (table, var,
  *     value) inverted scope index;

package/src/seed.ts

@@ -0,0 +1,168 @@
+/**
+ * The public seeding helper (RFC 0002 §2.5): push app-shaped mutations
+ * through the REAL push path — one `handleSyncRequest` round built from the
+ * server schema — so demos, dev servers, and ops scripts seed data with one
+ * supported call. This is the same §6 pipeline every client write takes
+ * (authorization, validation, idempotency, realtime fanout), so seeded rows
+ * behave exactly like synced rows.
+ *
+ * Idempotent by construction (§2.3): the commit id is stable, so re-running
+ * the seed replays the cached push result and writes nothing twice.
+ */
+import {
+  decodeMessage,
+  encodeMessage,
+  encodeRow,
+  PROTOCOL_WIRE_VERSION,
+  type PushOperation,
+  type PushResultFrame,
+  type RequestFrame,
+  type RowValue,
+} from '@syncular/core';
+import type { SyncServerConfig } from './context';
+import { SyncError } from './errors';
+import { handleSyncRequest } from './handler';
+
+/** One app-shaped seed mutation — the same vocabulary as client mutations. */
+export type SeedMutation =
+  | {
+      readonly table: string;
+      readonly op: 'upsert';
+      /**
+       * Full-row values keyed by column name (§6.1). Keys are accepted in
+       * the SQL-truth snake_case or the generated row types' camelCase;
+       * missing nullable columns become NULL.
+       */
+      readonly values: Readonly<Record<string, unknown>>;
+    }
+  | {
+      readonly table: string;
+      readonly op: 'delete';
+      readonly rowId: string;
+    };
+
+export interface SeedTarget {
+  readonly partition: string;
+  readonly actorId: string;
+  /**
+   * The synthetic client identity of the seed (default `'seed'`). Together
+   * with `commitId` it forms the §2.3 idempotency key — keep both stable
+   * for a re-runnable seed, vary `commitId` to seed additional batches.
+   */
+  readonly clientId?: string;
+  /** The client commit id (default `'seed-commit-1'`). */
+  readonly commitId?: string;
+}
+
+function camelToSnake(name: string): string {
+  return name.replace(/[A-Z]/g, (ch) => `_${ch.toLowerCase()}`);
+}
+
+/**
+ * Seed `mutations` into a partition through the real push path. Throws a
+ * `SyncError` when the push is rejected or any operation fails, so a broken
+ * seed fails loud at boot instead of silently serving an empty database.
+ */
+export async function seedMutations(
+  config: SyncServerConfig,
+  target: SeedTarget,
+  mutations: readonly SeedMutation[],
+): Promise<void> {
+  const clientId = target.clientId ?? 'seed';
+  const clientCommitId = target.commitId ?? 'seed-commit-1';
+  const operations: PushOperation[] = mutations.map((mutation) => {
+    const table = config.schema.tables.find((t) => t.name === mutation.table);
+    if (table === undefined) {
+      throw new SyncError(
+        'sync.invalid_request',
+        `seedMutations: unknown table ${JSON.stringify(mutation.table)}`,
+      );
+    }
+    if (mutation.op === 'delete') {
+      return { table: mutation.table, rowId: mutation.rowId, op: 'delete' };
+    }
+    // Normalize record keys to column positions (snake_case or camelCase,
+    // the two casings the client tier accepts).
+    const byColumn = new Map<string, unknown>();
+    for (const [key, value] of Object.entries(mutation.values)) {
+      const name = table.columns.some((c) => c.name === key)
+        ? key
+        : camelToSnake(key);
+      if (!table.columns.some((c) => c.name === name)) {
+        throw new SyncError(
+          'sync.invalid_request',
+          `seedMutations: table ${table.name}: unknown column ${JSON.stringify(key)}`,
+        );
+      }
+      if (byColumn.has(name)) {
+        throw new SyncError(
+          'sync.invalid_request',
+          `seedMutations: table ${table.name}: column ${JSON.stringify(name)} appears twice (snake_case and camelCase) — pass it once`,
+        );
+      }
+      byColumn.set(name, value);
+    }
+    const values = table.columns.map(
+      (column) => (byColumn.get(column.name) ?? null) as RowValue,
+    );
+    const pkIndex = table.columns.findIndex((c) => c.name === table.primaryKey);
+    const rowId = values[pkIndex];
+    if (typeof rowId !== 'string' || rowId.length === 0) {
+      throw new SyncError(
+        'sync.invalid_request',
+        `seedMutations: table ${table.name}: upsert requires a non-empty string primary key`,
+      );
+    }
+    return {
+      table: mutation.table,
+      rowId,
+      op: 'upsert',
+      payload: encodeRow(table.columns, values),
+    };
+  });
+
+  const frames: RequestFrame[] = [
+    { type: 'REQ_HEADER', clientId, schemaVersion: config.schema.version },
+    { type: 'PUSH_COMMIT', clientCommitId, operations },
+    // A pull that asks for nothing: the round exists for its push half.
+    {
+      type: 'PULL_HEADER',
+      limitCommits: 0,
+      limitSnapshotRows: 0,
+      maxSnapshotPages: 0,
+      accept: 0b0011,
+    },
+  ];
+  const response = await handleSyncRequest(
+    encodeMessage({
+      wireVersion: PROTOCOL_WIRE_VERSION,
+      msgKind: 'request',
+      frames,
+    }),
+    { ...config, partition: target.partition, actorId: target.actorId },
+  );
+
+  // Fail loud: surface the first rejected/failed operation.
+  const message = decodeMessage(response);
+  const result = message.frames.find(
+    (frame): frame is PushResultFrame =>
+      frame.type === 'PUSH_RESULT' && frame.clientCommitId === clientCommitId,
+  );
+  if (result === undefined) {
+    throw new SyncError(
+      'sync.invalid_request',
+      'seedMutations: the sync response carried no push result',
+    );
+  }
+  if (result.status === 'rejected') {
+    const failed = result.results.find((r) => r.status !== 'applied');
+    const detail =
+      failed !== undefined && 'code' in failed
+        ? ` (op ${failed.opIndex}: ${failed.code} — ${failed.message})`
+        : '';
+    throw new SyncError(
+      'sync.invalid_request',
+      `seedMutations: the seed commit was rejected${detail}`,
+    );
+  }
+}