@lyku/para-sync 0.0.1-pre.0

package/README.md

@@ -0,0 +1,109 @@
+# @lyku/para-sync
+
+`synced<T>` distributed object sync for the para:\* suite — server-authoritative
+records with live, version-reconciled client replicas over the existing
+single-WS-per-browser objectfeed.
+
+> **Pre-release (0.0.1-pre).** API will change before 0.1.0. This package
+> currently ships the **transport layer** (`InProcessTransport` +
+> `NatsTransport`) and the **client-side reconciler**. The full `synced<T>`
+> primitive (server resolver + SSR plumbing) and the server-write `::` gate land
+> on top — those touch Postgres/Valkey and the schema-version layer, so they
+> live closer to the app.
+
+## What's here today: the transport layer
+
+`synced<T>` must not couple to any one message bus. It depends only on the
+`SyncTransport` interface; the concrete transport is chosen by deployment config.
+
+```js
+import { InProcessTransport } from "@lyku/para-sync";
+
+const transport = new InProcessTransport();
+
+// listen handler (e.g. a WS-stream handler) subscribes for a key
+const off = transport.subscribe("user:123", (envelope) => {
+  // envelope = { value, schema_version, sequence }
+  // parse-gate + version-check + apply happen in the consumer, not here
+});
+
+// write handler publishes the change envelope after persisting
+transport.publish("user:123", { value: updatedUser, schema_version: "3.1", sequence: 42 });
+
+off(); // idempotent unsubscribe
+```
+
+### `SyncTransport` contract
+
+| member | behavior |
+| --- | --- |
+| `publish(key, envelope)` | delivers `envelope` to every current subscriber of `key`, in subscription order; no-op if none |
+| `subscribe(key, handler)` | registers `handler`; returns an idempotent `Unsub` |
+
+The transport is a **dumb pipe**: it does not retain the latest value, does not
+validate the envelope (`parse` gating is the consumer's job at the apply
+boundary), and does not dedupe by sequence. A subscriber receives only publishes
+that happen **after** it subscribes — initial state arrives via the SSR seed.
+
+### Implementations
+
+- **`InProcessTransport`** (here) — monolith / edge / IoT / all-in-one, where the
+  write handler and listen handlers share a process and there is no inter-service
+  bus. A keyed `Map<key, Set<handler>>` emitter; delivery is a synchronous call.
+  Empty key entries are GC'd when their last subscriber leaves (`keyCount()` is a
+  leak-check diagnostic).
+- **`NatsTransport`** — multi-service deployments; the change crosses services
+  over NATS, matching Lyku's existing full-object-over-NATS convention. Inject a
+  `connection` (callback-adapted: `publish(subject, bytes)` /
+  `subscribe(subject, onMessage) → unsub`), a wire `codec` (BON/msgpackr in
+  production — bigint IDs rule out JSON), and an optional `subjectOf(key)`.
+  N local subscribers to one key share a single bus subscription (local fanout),
+  torn down when the last leaves.
+
+The **client side is identical** across transports: WS receive → `parse` →
+version-check → apply. Only the server-internal delivery path differs.
+
+## Client reconciler
+
+`createClientReplica` is the client half of `synced<T>`: it takes the SSR seed
+and the stream of envelopes, gates every inbound value through the schema
+`parse`, reconciles by `(schema_version, sequence)`, and applies the
+authoritative value into a reactive cell so the DOM reacts.
+
+```js
+import { createClientReplica, InProcessTransport } from "@lyku/para-sync";
+
+const replica = createClientReplica({
+  key: "user:123",
+  schema: User,                 // anything with parse(v) => {tag:'Ok',value} | {tag:'Err',error}
+  transport,                    // a SyncTransport
+  seed: ssrEnvelope,            // {value, schema_version, sequence} embedded in the HTML
+  refetch: () => fetchSnapshot("user:123") // Err/skew/gap fallback → current snapshot
+});
+
+replica.get();   // current value, tracked (read inside an effect/template → reacts)
+replica.meta();  // { schemaVersion, sequence, status }, tracked
+replica.dispose();
+```
+
+Reconcile rules (Tier 1):
+
+- **parse gate** on every inbound value (SSR seed, receipt, refetch). `Err` →
+  status `skew`, the cell is **not** poisoned, and a `refetch` recovers a
+  known-good snapshot. Gates branch on `.tag` — they never throw (that is why
+  `::`, which throws on `Err`, is reserved for the server-write gate only).
+- **baseline (re)seed** — hydration, a recovery refetch, or the first value ever
+  seen is accepted unconditionally as the authoritative base.
+- **steady-state receipt** — apply iff `sequence === current + 1`; `<= current`
+  is ignored (stale/duplicate/out-of-order); `> current + 1` is a gap → refetch
+  + resync.
+
+`replica.stats` exposes counters (`applied`, `ignoredStale`, `gaps`,
+`parseErrors`, `refetches`); `replica.whenIdle()` resolves when no recovery
+refetch is in flight (a test/await aid).
+
+## Test
+
+```
+bun test        # from packages/para-sync
+```

package/package.json

@@ -0,0 +1,33 @@
+{
+  "name": "@lyku/para-sync",
+  "version": "0.0.1-pre.0",
+  "description": "synced<T> distributed object sync for the para:* suite — server-authoritative records with live, version-reconciled client replicas. Pre-release: API may change before 0.1.0. Currently provides the pluggable SyncTransport interface and the InProcessTransport implementation (monolith/edge/no-bus deployments); NatsTransport and the synced<T> primitive land on top.",
+  "publishConfig": {
+    "access": "public"
+  },
+  "type": "module",
+  "main": "./src/index.js",
+  "module": "./src/index.js",
+  "exports": {
+    ".": "./src/index.js"
+  },
+  "dependencies": {
+    "@lyku/para-signals": "^0.0.1-pre.0"
+  },
+  "files": [
+    "src/",
+    "README.md"
+  ],
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/airgap/parabun.git",
+    "directory": "packages/para-sync"
+  },
+  "keywords": [
+    "sync",
+    "replication",
+    "reactive",
+    "para"
+  ]
+}

package/src/client.js

@@ -0,0 +1,175 @@
+// @lyku/para-sync — client-side replica reconciler (Tier 1 core).
+//
+// The heart of the client half of synced<T>: take the SSR seed and the stream
+// of change envelopes, gate every inbound value through the schema `parse`,
+// reconcile by (schema_version, sequence), and apply the authoritative value
+// into a reactive cell so the DOM reacts.
+//
+// What this is NOT: it does not open the WS, does not run the connect-time
+// handshake (Tier 1 step 4), and does not write (Tier 2). It is the pure
+// receive → parse → version-check → apply engine, transport-agnostic.
+
+import { signal } from "@lyku/para-signals";
+
+/** @typedef {import('./transport.js').SyncEnvelope} SyncEnvelope */
+/** @typedef {import('./transport.js').SyncTransport} SyncTransport */
+
+/**
+ * A schema's parse result — matches para-schema's Result<T, string>.
+ * @typedef {{ tag: 'Ok', value: any } | { tag: 'Err', error: string }} Result
+ */
+
+/**
+ * Anything with a `parse` returning {@link Result}. In production this is a
+ * para-schema `SchemaValue`; in tests it can be a hand-rolled gate. The replica
+ * depends only on this shape, never on para-schema directly — which is also why
+ * the client gates branch on `.tag` instead of using the throw-on-Err `::`
+ * convention (a malformed delta must trigger recovery, not crash the apply).
+ * @typedef {{ parse(v: unknown): Result }} SyncSchema
+ */
+
+/**
+ * A minimal reactive value cell: get / peek / set. A para-signals `signal()`
+ * satisfies it exactly and is the default. Injectable for testing and for the
+ * fork-backed cell on the para-svelte side.
+ * @typedef {{ get(): any, peek(): any, set(v: any): void }} Cell
+ */
+
+/**
+ * @typedef {'ok' | 'stale' | 'skew' | 'refetching'} ReplicaStatus
+ *  - ok          last apply succeeded; replica is current
+ *  - stale       uninitialized, or a refetch failed / none available
+ *  - skew        an inbound value failed `parse` (malformed or schema-skew)
+ *  - refetching  a recovery refetch is in flight
+ */
+
+/**
+ * @typedef {object} ReplicaMeta
+ * @property {string | null} schemaVersion  schema version of the applied value
+ * @property {number} sequence              sequence of the applied value (-1 if uninitialized)
+ * @property {ReplicaStatus} status
+ */
+
+/**
+ * Create a client replica for one synced key.
+ *
+ * @param {object} opts
+ * @param {string} opts.key                              synced key, e.g. "user:123"
+ * @param {SyncSchema} opts.schema                       the `parse` gate
+ * @param {SyncTransport} opts.transport                 change-envelope source
+ * @param {SyncEnvelope} [opts.seed]                     SSR-embedded initial envelope
+ * @param {() => Promise<SyncEnvelope>} [opts.refetch]   Err/skew/gap fallback: fetch the
+ *        current authoritative snapshot. Omit → no recovery (status goes 'stale').
+ * @param {Cell} [opts.cell]                             reactive cell (default: a para signal)
+ */
+export function createClientReplica({ key, schema, transport, seed, refetch, cell }) {
+  const value = cell ?? signal(undefined);
+  /** @type {Cell} */
+  const meta = signal(
+    /** @type {ReplicaMeta} */ ({ schemaVersion: null, sequence: -1, status: "stale" })
+  );
+
+  let initialized = false;
+  let disposed = false;
+  let pending = Promise.resolve();
+
+  const stats = { applied: 0, ignoredStale: 0, gaps: 0, parseErrors: 0, refetches: 0 };
+
+  /** @param {Partial<ReplicaMeta>} patch */
+  const setMeta = (patch) => meta.set({ ...meta.peek(), ...patch });
+
+  /** @param {any} parsedValue @param {SyncEnvelope} envelope */
+  function commit(parsedValue, envelope) {
+    value.set(parsedValue);
+    initialized = true;
+    setMeta({ schemaVersion: envelope.schema_version, sequence: envelope.sequence, status: "ok" });
+    stats.applied++;
+  }
+
+  function startRefetch() {
+    if (!refetch) {
+      setMeta({ status: "stale" });
+      return;
+    }
+    stats.refetches++;
+    setMeta({ status: "refetching" });
+    pending = (async () => {
+      try {
+        const snap = await refetch();
+        if (!disposed) ingest(snap, "refetch");
+      } catch {
+        if (!disposed) setMeta({ status: "stale" });
+      }
+    })();
+  }
+
+  /**
+   * @param {SyncEnvelope} envelope
+   * @param {'hydration' | 'receipt' | 'refetch'} source
+   */
+  function ingest(envelope, source) {
+    if (disposed) return;
+
+    // ── parse gate (every inbound value crosses a trust boundary) ──
+    const res = schema.parse(envelope.value);
+    if (res.tag !== "Ok") {
+      stats.parseErrors++;
+      setMeta({ status: "skew" });
+      // Don't poison the cell. Recover via a known-good snapshot — but never
+      // refetch in response to a refetch result (avoids an Err→refetch loop).
+      if (source !== "refetch") startRefetch();
+      return;
+    }
+
+    // ── baseline (re)seed ──
+    // SSR hydration, a recovery refetch, or the very first value we have seen:
+    // accept unconditionally as the new authoritative baseline.
+    if (source === "hydration" || source === "refetch" || !initialized) {
+      commit(res.value, envelope);
+      return;
+    }
+
+    // ── steady-state receipt: reconcile by sequence ──
+    const cur = meta.peek().sequence;
+    if (envelope.sequence <= cur) {
+      stats.ignoredStale++; // stale / duplicate / out-of-order
+      return;
+    }
+    if (envelope.sequence === cur + 1) {
+      commit(res.value, envelope); // in-order
+      return;
+    }
+    // Gap: one or more envelopes were missed. v2 step 5 → refetch the full
+    // snapshot and resync. (Under the full-object delta model the gapped
+    // envelope already carries the complete current value, so committing it
+    // directly would be correct and cheaper — a documented future optimization;
+    // we follow v2's explicit gap→refetch here.)
+    stats.gaps++;
+    startRefetch();
+  }
+
+  // ── wire up ──
+  const unsub = transport.subscribe(key, (envelope) => ingest(envelope, "receipt"));
+  if (seed !== undefined) ingest(seed, "hydration"); // SSR-hydration parse gate
+
+  return {
+    /** current value (tracked read) */
+    get: () => value.get(),
+    /** current value (untracked) */
+    peek: () => value.peek(),
+    /** reconcile metadata (tracked): { schemaVersion, sequence, status } */
+    meta: () => meta.get(),
+    /** reconcile metadata (untracked) */
+    peekMeta: () => meta.peek(),
+    /** observability counters — read directly */
+    stats,
+    /** resolves when no recovery refetch is in flight (test/await aid) */
+    whenIdle: () => pending,
+    /** stop listening; idempotent */
+    dispose: () => {
+      if (disposed) return;
+      disposed = true;
+      unsub();
+    }
+  };
+}

package/src/index.js

@@ -0,0 +1,13 @@
+// @lyku/para-sync — distributed object sync for the para:* suite.
+//
+// Public barrel. The package is split by concern:
+//   - transport.js — the pluggable SyncTransport interface + InProcessTransport
+//     (server-internal: carry a change envelope from writer to listeners).
+//   - client.js    — createClientReplica: the client-side reconciler
+//     (receive → parse → version-check → apply into a reactive cell).
+//
+// `synced<T>(key)` (the full primitive) composes the client reconciler with the
+// server resolver + SSR plumbing; it lands on top of these pieces.
+
+export * from "./transport.js";
+export * from "./client.js";

package/src/transport.js

@@ -0,0 +1,260 @@
+// @lyku/para-sync — distributed object sync for the para:* suite.
+//
+// This module currently provides the TRANSPORT layer: the server-internal
+// mechanism that carries a synced object's change envelope from the write
+// handler to the listen handlers. `synced<T>` (the primitive itself) and the
+// three `parse` gates build on top of this.
+//
+// Transport is a PLUGGABLE interface with (initially) two implementations:
+//   - InProcessTransport — monolith / edge / IoT / all-in-one, where the write
+//     handler and the listen handlers share a process and there is no
+//     inter-service bus to stand up. (This file.)
+//   - NatsTransport — multi-service deployments, where the change crosses
+//     from the writer's service to listeners' services over NATS. (Later.)
+//
+// The CLIENT side is identical across both: WS receive → parse → version-check
+// → apply. Only the server-internal "how the write reaches the listen handler"
+// differs, selected by deployment config.
+
+/**
+ * The change envelope. Delta/full-object model: the new value travels with
+ * the reconcile key, so steady-state replication is deliver → parse → apply,
+ * with no refetch round-trip (refetch is the Err/gap/skew fallback only).
+ *
+ * @typedef {object} SyncEnvelope
+ * @property {unknown} value          The full changed object. NOT validated by
+ *                                    the transport — `parse` gating is the
+ *                                    consumer's job at the apply boundary.
+ * @property {string} schema_version  Reconcile key, part 1: the model's schema
+ *                                    version (e.g. "3.1"). Distinguishes a
+ *                                    compatible-but-behind replica from a
+ *                                    different/breaking shape.
+ * @property {number} sequence        Reconcile key, part 2: the object's
+ *                                    monotonic Postgres-authoritative sequence.
+ *                                    Sole job is ordering + gap detection.
+ */
+
+/**
+ * A listener for a key's change envelopes.
+ * @callback SyncHandler
+ * @param {SyncEnvelope} envelope
+ * @returns {void}
+ */
+
+/**
+ * Returned by subscribe(); calling it removes the subscription. Idempotent.
+ * @callback Unsub
+ * @returns {void}
+ */
+
+/**
+ * The pluggable transport contract. `synced<T>` depends ONLY on this shape, not
+ * on any concrete transport — that decoupling is what lets the same primitive
+ * run on a single box (InProcessTransport) or across services (NatsTransport).
+ *
+ * Contract notes that every implementation must honor:
+ *   - publish(key, envelope) delivers `envelope` to every current subscriber of
+ *     `key`. Publishing to a key with no subscribers is a no-op (never throws).
+ *   - The transport is a dumb pipe: it does NOT retain the latest value, does
+ *     NOT validate the envelope, and does NOT dedupe by sequence. A subscriber
+ *     receives only publishes that happen AFTER it subscribes — initial state
+ *     arrives via the SSR seed, not the transport.
+ *   - subscribe(key, handler) returns an idempotent Unsub.
+ *
+ * @typedef {object} SyncTransport
+ * @property {(key: string, envelope: SyncEnvelope) => void} publish
+ * @property {(key: string, handler: SyncHandler) => Unsub} subscribe
+ */
+
+/**
+ * In-process implementation of {@link SyncTransport}. A keyed pub/sub emitter:
+ * `Map<key, Set<handler>>`. No bus, no network, no serialization — the write
+ * and the listen happen in the same process, so delivery is a synchronous call.
+ *
+ * Why a plain emitter and not a para-signal per key: the transport contract is
+ * notification semantics ("deliver future publishes to this handler"), not
+ * value-cell semantics ("hand me the current value on subscribe, then deltas").
+ * Initial state is the SSR seed's job; the transport only carries changes. A
+ * keyed emitter models that exactly, without the current-value-on-subscribe and
+ * Object.is-dedupe behavior a signal would impose.
+ *
+ * @implements {SyncTransport}
+ */
+export class InProcessTransport {
+  constructor() {
+    /**
+     * key → set of handlers. A key is present iff it has ≥1 live subscriber;
+     * the entry is deleted when its last subscriber unsubscribes (no key leak).
+     * @type {Map<string, Set<SyncHandler>>}
+     */
+    this._subs = new Map();
+  }
+
+  /**
+   * Deliver `envelope` to every current subscriber of `key`, in subscription
+   * order. No-op if `key` has no subscribers.
+   * @param {string} key
+   * @param {SyncEnvelope} envelope
+   */
+  publish(key, envelope) {
+    const handlers = this._subs.get(key);
+    if (handlers === undefined) return;
+    // Snapshot: a handler may subscribe/unsubscribe during delivery. Iterating
+    // a copy gives well-defined semantics — handlers live at publish time each
+    // receive this envelope; a handler added mid-delivery does not.
+    for (const handler of Array.from(handlers)) handler(envelope);
+  }
+
+  /**
+   * Subscribe `handler` to `key`'s change envelopes. Returns an idempotent
+   * Unsub; calling it more than once is safe and will not remove a handler that
+   * was re-subscribed in between.
+   * @param {string} key
+   * @param {SyncHandler} handler
+   * @returns {Unsub}
+   */
+  subscribe(key, handler) {
+    let handlers = this._subs.get(key);
+    if (handlers === undefined) {
+      handlers = new Set();
+      this._subs.set(key, handlers);
+    }
+    handlers.add(handler);
+
+    let active = true;
+    return () => {
+      if (!active) return; // idempotent: a second call does nothing
+      active = false;
+      const set = this._subs.get(key);
+      if (set === undefined) return;
+      set.delete(handler);
+      if (set.size === 0) this._subs.delete(key); // GC the empty key
+    };
+  }
+
+  /**
+   * Diagnostic: number of keys with at least one live subscriber. Useful for
+   * leak checks (a healthy server returns to a steady key count as sessions
+   * come and go).
+   * @returns {number}
+   */
+  keyCount() {
+    return this._subs.size;
+  }
+}
+
+/**
+ * A NATS connection, callback-adapted. NatsTransport expects delivery as a
+ * callback + an unsubscribe, NOT nats.js's raw async-iterable subscription —
+ * the iterable→callback adaptation is a 3-line caller concern that Lyku already
+ * does (`const sub = nc.subscribe(subj); (async () => { for await (const m of
+ * sub) onMessage(m.data); })(); return () => sub.unsubscribe();`). Keeping that
+ * out of the transport makes delivery synchronous and deterministic to test.
+ *
+ * @typedef {object} SyncNatsConnection
+ * @property {(subject: string, payload: Uint8Array) => void} publish
+ * @property {(subject: string, onMessage: (payload: Uint8Array) => void) => (() => void)} subscribe
+ *   subscribe to `subject`; `onMessage` is called per message with the raw
+ *   payload; returns an unsubscribe function.
+ */
+
+/**
+ * Wire codec: envelope ⇄ bytes. NATS payloads are `Uint8Array`. In production
+ * inject a BON or msgpackr codec (envelopes carry bigint IDs, which JSON cannot
+ * represent — BON is the SSR/wire serializer for exactly this reason). Defaults
+ * to identity (object passthrough), which works for in-memory fakes/tests but
+ * NOT a real NATS connection.
+ *
+ * @typedef {object} SyncCodec
+ * @property {(envelope: SyncEnvelope) => any} encode
+ * @property {(payload: any) => SyncEnvelope} decode
+ */
+
+/** @type {SyncCodec} */
+const IDENTITY_CODEC = { encode: (e) => e, decode: (p) => p };
+
+/**
+ * NATS implementation of {@link SyncTransport}, for multi-service deployments
+ * where a write in the writer's service must reach listeners in other services.
+ * Matches Lyku's existing full-object-over-NATS convention.
+ *
+ * Honors the same contract as InProcessTransport (deliver to current
+ * subscribers, no retention, dumb pipe, idempotent unsub). On top it adds:
+ *   - subject mapping (key → NATS subject),
+ *   - the wire codec (encode on publish, decode on receipt),
+ *   - LOCAL FANOUT: N local subscribers to one key share ONE bus subscription,
+ *     and that bus subscription is torn down when the last local subscriber
+ *     leaves (the cross-service analog of subscriber-set GC).
+ *
+ * @implements {SyncTransport}
+ */
+export class NatsTransport {
+  /**
+   * @param {object} opts
+   * @param {SyncNatsConnection} opts.connection
+   * @param {SyncCodec} [opts.codec]                     default: identity (tests only)
+   * @param {(key: string) => string} [opts.subjectOf]   default: `synced.${key}`
+   */
+  constructor({ connection, codec, subjectOf }) {
+    if (!connection) throw new Error("NatsTransport requires a connection");
+    this._nc = connection;
+    this._codec = codec ?? IDENTITY_CODEC;
+    this._subjectOf = subjectOf ?? ((key) => `synced.${key}`);
+    /**
+     * key → { natsUnsub, handlers } — present iff the key has ≥1 local
+     * subscriber (and therefore one live bus subscription).
+     * @type {Map<string, { natsUnsub: () => void, handlers: Set<SyncHandler> }>}
+     */
+    this._keys = new Map();
+  }
+
+  /**
+   * @param {string} key
+   * @param {SyncEnvelope} envelope
+   */
+  publish(key, envelope) {
+    this._nc.publish(this._subjectOf(key), this._codec.encode(envelope));
+  }
+
+  /**
+   * @param {string} key
+   * @param {SyncHandler} handler
+   * @returns {Unsub}
+   */
+  subscribe(key, handler) {
+    let entry = this._keys.get(key);
+    if (entry === undefined) {
+      /** @type {Set<SyncHandler>} */
+      const handlers = new Set();
+      // One bus subscription per key; decode once, fan out to local handlers.
+      const natsUnsub = this._nc.subscribe(this._subjectOf(key), (payload) => {
+        const envelope = this._codec.decode(payload);
+        for (const h of Array.from(handlers)) h(envelope);
+      });
+      entry = { natsUnsub, handlers };
+      this._keys.set(key, entry);
+    }
+    entry.handlers.add(handler);
+
+    let active = true;
+    return () => {
+      if (!active) return; // idempotent
+      active = false;
+      const e = this._keys.get(key);
+      if (e === undefined) return;
+      e.handlers.delete(handler);
+      if (e.handlers.size === 0) {
+        e.natsUnsub(); // last local subscriber gone → tear down the bus sub
+        this._keys.delete(key);
+      }
+    };
+  }
+
+  /**
+   * Diagnostic: number of keys with a live bus subscription.
+   * @returns {number}
+   */
+  keyCount() {
+    return this._keys.size;
+  }
+}