@blamejs/core 0.6.33 → 0.6.35

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -8,6 +8,8 @@ upgrading across more than a few patches at a time.
8
8
 
9
9
  ## v0.6.x
10
10
 
11
+ - **0.6.35** (2026-05-02) — `cluster-provider-db` MySQL dialect. **`b.cluster.create({ provider: clusterProviderDb.create({ dialect: "mysql", ... }) })`** — operators on MySQL no longer have to supply their own provider; the framework's default DB-row leader-election provider now speaks all three of postgres / sqlite / mysql. The MySQL acquireLease shape uses `INSERT INTO _blamejs_leader (...) VALUES (...) ON DUPLICATE KEY UPDATE col = IF(expiresAt < ?, VALUES(col), col), ..., expiresAt = IF(expiresAt < ?, VALUES(expiresAt), expiresAt)` so a still-valid lease is preserved untouched and an expired one is overwritten — atomic at the row level — followed by a `SELECT FROM _blamejs_leader WHERE scope='leader'` to read who holds (MySQL has no `RETURNING`). The expiresAt assignment runs LAST so per-column IF() predicates evaluate against pre-update row state. Renew uses `UPDATE ... SET expiresAt=?, endpoint=? WHERE scope='leader' AND nodeId=? AND leaseId=?` followed by a check-SELECT to surface takeover races as `LEASE_LOST`. Schema generation: `BIGINT` int columns (was `INTEGER` for SQLite), `VARCHAR(64)` for primary-key text columns and `VARCHAR(255)` for body text (MySQL needs explicit lengths on PRIMARY KEY columns), `CHECK (scope = 'leader' / 'state')` constraint dropped on MySQL because some MariaDB / MySQL 5.x versions parse-then-silently-drop CHECK clauses (would surface as version drift); the constant-scope invariant is enforced by application code anyway. Placeholder style auto-flips to `?` for MySQL (Postgres / SQLite continue to use `$1..$N`). **Tests**: 16 layer-1 assertions exercising the MySQL dialect path against a fake mysql-shaped driver (`_makeFakeMysqlDriver` in `test/helpers/drivers.js`) that emulates `INSERT ... ON DUPLICATE KEY UPDATE` with the per-column `IF()` semantics; covers acquire-empty, blocked-while-held, renew-no-fencing-bump, takeover-with-fencing-bump, old-leader-renew-throws-LEASE_LOST, currentLeader, releaseLease — plus a SQL-shape audit (validates VARCHAR primary keys, ON DUPLICATE KEY syntax, ?-placeholders, IF() gating). 14 live integration assertions against the docker MySQL 8.4 container via a docker-exec-based external-db driver shim (no npm mysql client; the framework already requires operator-supplied driver wiring, the shim demonstrates one path) covering ensureSchema, acquireLease, blocked-second-node, currentLeader, renewLease, release, takeover-after-expiry, fencingToken bump, old-leader LEASE_LOST. Smoke 6833 / wiki e2e 178 / per-primitive integration 16 files / wiki integration 32 / shellcheck clean / eslint clean.
12
+ - **0.6.34** (2026-05-02) — distributed pub/sub primitive. **`b.pubsub`** — single API for cross-node fan-out across the framework, with three backends. `local` dispatches in-process to registered handlers (zero coordination overhead for single-node deploys); `cluster` polls a shared `_blamejs_pubsub_messages` table at `pollIntervalMs` (default 100ms) and dispatches new rows past `lastSeenId` from other nodes (publishedBy=self filter prevents loopback); `redis` opens a SUBSCRIBE-mode connection on `lib/redis-client.js` (now with new push-message demultiplexing — server-pushed `["message", channel, payload]` and `["pmessage", pattern, channel, payload]` arrays route through `setOnPushMessage` instead of consuming a pending request slot, while SUBSCRIBE / UNSUBSCRIBE acks still flow through the normal command pipeline). Per-instance nonce stamped on outgoing redis payloads so the SUBSCRIBE socket recognizes its own publishes and skips the loopback (without it every same-instance publish would fire local handlers twice). Operator API: `ps.subscribe(channel, handler) → token`, `ps.subscribePattern(pattern, handler) → token` (glob-style on local + cluster, native PSUBSCRIBE on redis), `ps.unsubscribe(token)`, `await ps.publish(channel, payload) → { local, remote }`, `await ps.close()`. `topicPrefix` opt scopes channel names so independent pubsub instances sharing a backend (cache invalidation + websocket channels + custom) don't collide. Handler errors are caught + logged via the framework's boot logger; they never abort dispatch to other handlers on the same channel. **`lib/websocket-channels.js`** — replaces the inline cluster-poll-and-fan-out logic with `b.pubsub` consumption. The hub now owns one pubsub instance per primitive; cross-node delivery is `pubsub.subscribe` per channel the hub joins, with the hub's `_localDispatch` as the handler. Per-channel pubsub subscription is refcounted by local conn count (subscribe on first conn, unsubscribe on last). The `_blamejs_ws_messages` table is renamed to `_blamejs_pubsub_messages` (column `channel` → `topic`) reflecting the generalization; pre-v1, no compat shim — operators upgrading wipe the previous table (which carried only ephemeral fan-out rows with default 60s retention). **`b.cache.create({ invalidationPubsub })`** — passing a `b.pubsub.create()` instance auto-publishes on every successful `del` / `clear` / `invalidateTag`, and subscribes for the same events so other cache instances on other nodes (or processes sharing the pubsub backend) react locally — primarily useful for the memory backend so stale per-node entries don't survive a global tag wipe. Re-entrancy guard prevents inbound invalidation events from re-publishing (no fan-out loops). Tests: 29 layer-0 assertions covering local + cluster behavior, topicPrefix isolation, pattern subscribe, handler error isolation, post-close error path, end-to-end cache invalidationPubsub fan-out via local pubsub. 10 live integration assertions against the docker redis container covering single-instance round-trip, PSUBSCRIBE pattern, cross-instance fan-out, and cache invalidation through redis PUB/SUB. Smoke 6817 / wiki e2e 178 / per-primitive integration 15 files / wiki integration 32 / shellcheck clean / eslint clean.
11
13
  - **0.6.33** (2026-05-02) — log-stream sinks parity sweep. **`b.logStream` syslog sink (RFC 5424)** — first-class `protocol: "syslog"` for `b.logStream.init({ sinks: { ... } })`; new module `lib/log-stream-syslog.js`. URL-driven transport selection: `udp://host:514` / `tcp://host:514` / `tls://host:6514`. UDP is best-effort one-datagram-per-record; TCP / TLS use RFC 6587 octet-counting framing (`<length> <message>`) so collectors that prefer it over the older non-transparent newline framing parse cleanly. TLS is TLS 1.3 minimum; operators with private CAs pass `ca` (PEM string or array) for trust pinning, `servername` for SNI override (auto-suppressed on IP literals per the v0.6.28 redis-client convention). Outgoing records are formatted with PRI = `(facility << 3) | severity`, default facility `local0` (16), severity mapped from the framework's level field (debug=7 / info=6 / warn=4 / error=3); operators override via `facility`, `appName` (default `blamejs`), `procId` (default `process.pid`), `hostname` (default `os.hostname()`), `structuredData` (default `-`). Meta is JSON-encoded into the MSG body so the structured payload survives the wire as a single token. TCP / TLS reconnect with exponential backoff (default 250ms→30s); records buffer during the down window with `bufferLimit`-bounded oldest-drop semantics, replay on reconnect. `close()` waits up to 3s for an in-flight (re)connect to drain the buffer before tearing down — without this the slower TLS handshake raced fire-and-forget shutdown emits and silently dropped records. Operator-supplied `onDrop({reason, batch, error})` surfaces every drop class (`overflow` / `udp-send-error` / `write-error` / `sink-closed`). Removed `syslog` from `DEFERRED_PROTOCOLS`. **`b.logStream` cloudwatch sink: `autoCreate`** — pass `{ autoCreate: true }` to have the framework issue `CreateLogGroup` + `CreateLogStream` on first emit so operators provisioning collectors via env vars / runbooks don't need a separate aws-cli step. Idempotent: AWS's `ResourceAlreadyExistsException` is treated as success on both calls. Hard failures (5xx, AccessDenied, etc.) drop the batch with `onDrop` reason `autocreate-failed`. Default remains `autoCreate: false` so the AWS posture (operator pre-creates via aws / CDK / Terraform) stays the recommended path; `autoCreate` exists for ephemeral / dynamic-stream deployments where pre-provisioning is impractical. **Test suite**: 19 new framework-level cloudwatch assertions cover autoCreate fires both Create calls in order before PutLogEvents, ResourceAlreadyExists doesn't abort the post-create PutLogEvents, hard 5xx during CreateLogGroup drops the batch via `onDrop`, autoCreate=false skips Create calls entirely. Live syslog integration covers UDP / TCP wire delivery against the docker syslog-ng container (`/var/log/blamejs-test.log`) plus an in-process `tls.createServer` receiver that asserts the on-the-wire RFC 6587 octet-counting framing + RFC 5424 PRI / timestamp / structured-data slot the framework emits over TLS. **Test infrastructure fix**: `docker/init/generate-certs.sh` was overwriting an existing CA when re-run with `.complete` removed, invalidating every previously-issued leaf cert; now reuses an existing `(ca.crt, ca.key)` pair across `.complete`-only resets and only generates a fresh CA when neither file is present. Smoke 6790 / wiki e2e 178 / per-primitive integration 14 files (log-stream up from 16→18 checks, cloudwatch suite up from 39→58 assertions) / wiki integration 32 / shellcheck clean / eslint clean.
12
14
  - **0.6.32** (2026-05-02) — E1 from the v0.6.x scope plan. **`b.cache` Redis backend** — first-class `backend: "redis"` for `b.cache.create({...})`, no operator-supplied glue needed. New module `lib/cache-redis.js` consumes `lib/redis-client.js` directly; storage layout: `<namespace>:e:<key>` (STRING, JSON-encoded value, PEXPIREAT-bounded), `<namespace>:t:<tag>` (SET, cacheKeys carrying that tag — powers `invalidateTag` fan-out), `<namespace>:k:<key>:tags` (SET, tags this key carries — powers per-key tag cleanup on `del`/`set`-overwrite, expires alongside the entry). TTL is enforced by Redis itself (PEXPIREAT) so the framework's sweeper is a no-op for this backend; sliding TTL bumps the entry's expiry on every read when `slidingTtl: true` AND `ttlMs` is finite. New opts on `cache.create`: `redisUrl` (required when `backend: "redis"`), `redisPassword`, `redisUsername`, `redisTls`, `redisCa` (private-CA trust pinning per v0.6.28), `redisServername`, `redisConnectTimeoutMs`, `redisCommandTimeoutMs`, `redisMaxReconnectAttempts`. Concurrency: invalidateTag filters out ghost entries (key already PEXPIRE'd from the keyspace but lingering in a tag SET) by EXISTS-checking before del. Lazy connect — `cache.create({backend:"redis"})` stays sync-safe; first op opens the socket. Tests: 10 new live integration assertions on top of the existing memory + custom-backend coverage (set+get round-trip, has, del, complex JSON value round-trip, short-TTL Redis-side expiry, multi-tag invalidateTag fan-out + preservation of untagged entries, wrap() single-flight memoization through Redis). Smoke 6780 / wiki e2e 178 / per-primitive integration 14 files (cache up from 15→25 checks) / wiki integration green.
13
15
  - **0.6.31** (2026-05-02) — lazy-deferral cleanup wave from v0.6.27. **A1: queue-redis priority ordering** — `b.queue.enqueue({ priority })` on the Redis backend now matches queue-local's `ORDER BY priority DESC, availableAt ASC, enqueuedAt ASC` semantics. The Redis ZSET stays scored by availableAt only (so ZRANGEBYSCORE 0..nowMs cleanly filters ready vs not-yet-ready jobs), but `LEASE_LUA` now over-fetches `maxRows*5` candidates by score, HMGETs priority + availableAt + enqueuedAt for each, sorts server-side in Lua by the same triple queue-local sorts on, and leases the top `maxRows`. Closes the v0.6.27 deferral that punted priority ordering as "ZSET-incompatible". **A2: queue-redis flow `dependsOn` cascade** — `b.queue.enqueue({ flowId, flowChildName, dependsOn })` on the Redis backend now releases dependent jobs when their parents complete, mirroring queue-local's `_maybeReleaseFlowChildren`. A per-flow `<prefix>:flow:<flowId>` Redis SET tracks every job in the flow; `complete()` walks the set, looks up siblings whose `dependsOn` includes the just-completed job (by id OR `flowChildName`) AND every other dep is also satisfied, and HSET-bumps their `availableAt` to now + ZADDs them into the ready zset. The flow set persists across `complete()` calls so a later sibling's deps check can still find earlier-done parents by name; `purge()` cleans the set on queue teardown. Closes the v0.6.27 deferral. **A4: WebSocket per-message-deflate (RFC 7692)** — `b.websocket.handleUpgrade` negotiates the `permessage-deflate` extension when the client offers it, accepting `client_max_window_bits` / `server_max_window_bits` constraints (8-15 range, default 15) and always asserting `client_no_context_takeover` + `server_no_context_takeover` so every message uses a fresh zlib state. The connection's send path compresses TEXT/BINARY frames via `zlib.deflateRawSync`, strips the 4-byte `0x00 0x00 0xff 0xff` trailer per RFC 7692 §7.2.1, and sets RSV1 on the first frame of each compressed message. The receive path appends the trailer back and inflates via `zlib.inflateRawSync`. RSV1 on a continuation frame, RSV1 without negotiated extension, RSV2/RSV3 set, and decompressed payload exceeding `maxMessageBytes` all close with the appropriate RFC 6455 status (`PROTOCOL_ERROR`, `INVALID_PAYLOAD`, `MESSAGE_TOO_BIG`). Operator opt-out: pass `permessageDeflate: false` to `handleUpgrade` to refuse the extension even when offered. Tests cover the 101-handshake echo of negotiated params, real compression on a redundant payload (4000 bytes → ~30 bytes on the wire), uncompressed-frame round-trip on the same connection (mixed-mode per RFC 7692 §6), and unknown-extension graceful ignore. v0.6.27's third lazy deferral A3 (TLS-redis live test) was already covered by `test/integration/redis-client-tls.test.js` shipped in v0.6.28. Smoke 6780 / wiki e2e 178 / per-primitive integration 14 files green / wiki integration green.
package/README.md CHANGED
@@ -46,7 +46,7 @@ The framework bundles the surface a typical Node app reaches for. Every primitiv
46
46
  - **Crypto** — envelope-versioned PQC at rest (ML-KEM-1024 + P-384 hybrid, XChaCha20-Poly1305, SHAKE256), vault sealing, field-level crypto + cryptographic erasure (`b.cryptoField.eraseRow`), signed webhooks (SLH-DSA-SHAKE-256f), ECIES API encryption (`b.crypto`, `b.vault`, `b.webhook`); pure-JS mTLS CA that issues clientAuth / serverAuth / dual-EKU certs with SAN entries and auto-detects the highest-PQC signature algorithm the vendored x509 library accepts (today: ECDSA-P384-SHA384 bridge; self-upgrades to SLH-DSA / ML-DSA when the X.509 ecosystem catches up), PQC TLS gates inbound + outbound (`b.mtlsCa`, `b.pqcGate`, `b.pqcAgent`).
47
47
  - **HTTP** — router with schema-validated routes + OpenAPI publication; full middleware stack (CSRF, CORS, rate-limit, security headers, CSP nonce, body parser, compression, SSE, request log, request-time DB role binding via `b.middleware.dbRoleFor`, in-process CIDR fence via `b.middleware.networkAllowlist`) wired by `createApp`; HTTP/1.1 + HTTP/2 outbound client with SSRF gate (cloud-metadata IPs hard-denied unconditionally; private / loopback / link-local overridable per call), scheme + userinfo + per-host (wildcard / per-method) destination allowlist, redirects, multipart, interceptors, progress, encrypted cookie jar (`b.httpClient`, `b.ssrfGuard`, `b.safeUrl`); operator-tunable network configurability — env-driven NTP / NTS (RFC 8915 authenticated time), IPv4-or-IPv6 NTP servers, DNS with IPv6 / DoH / DoT (private-CA trust pinning via `opts.ca`) / cache / lookup timeout, outbound HTTP proxy (`HTTP_PROXY` / `HTTPS_PROXY` / `NO_PROXY`), runtime DPI trust-store CA additions, application-level heartbeats, TCP socket defaults (`b.network`).
48
48
  - **Defensive parsers** — `b.safeJson`, `b.safeBuffer`, `b.safeSql`, `b.safeSchema`, `b.parsers` (XML / TOML / YAML / .env), `b.config` (schema-validated env), `b.fileType` magic-byte content classification with deny-on-upload categories (image / document / archive / executable / etc.).
49
- - **Communication** — WebSockets with channel/room fan-out across cluster replicas (`b.websocket`, `b.websocketChannels`); mail with multipart + attachments + DKIM + calendar invites + bounce intake (`b.mail`, `b.mailBounce`); generic notification dispatcher with operator-supplied transports (`b.notify`).
49
+ - **Communication** — WebSockets with channel/room fan-out across cluster replicas (`b.websocket`, `b.websocketChannels`); generic distributed pub/sub with cluster-table / Redis PUB/SUB / custom backends (`b.pubsub`); mail with multipart + attachments + DKIM + calendar invites + bounce intake (`b.mail`, `b.mailBounce`); generic notification dispatcher with operator-supplied transports (`b.notify`).
50
50
  - **Observability** — tamper-evident audit chain with SLH-DSA-signed checkpoints, metrics, tracing (OTel pass-through when wired), PII redaction, log-stream sinks (local file rotation, generic webhook, OTLP/HTTP-JSON to an OTel collector, AWS CloudWatch Logs via SigV4 with optional autoCreate, RFC 5424 syslog over UDP/TCP/TLS), OTLP/HTTP-JSON exporter for traces + metrics (`b.audit`, `b.metrics`, `b.tracing`, `b.redact`, `b.logStream`, `b.otelExport`); operator-callable boot-time security policy assertions (`b.security.assertProduction`) and tamper-evident config-baseline drift detection signed with the audit-signing key (`b.configDrift`).
51
51
  - **i18n** — CLDR plural rules, Accept-Language negotiation, Intl formatters, RTL (`b.i18n`).
52
52
  - **Format helpers** — RFC 4180 CSV with Excel formula-injection prevention (`b.csv`), RFC 9562 UUID v4 + v7 (`b.uuid`), URL-safe slugs (`b.slug`), TZ-aware datetime (`b.time`), ZIP creation (`b.archive`), HMAC-signed cursor pagination (`b.pagination`), HTML form rendering + validation + CSRF (`b.forms`).
package/index.js CHANGED
@@ -113,6 +113,7 @@ var time = require("./lib/time");
113
113
  var uuid = require("./lib/uuid");
114
114
  var mail = require("./lib/mail");
115
115
  var mailBounce = require("./lib/mail-bounce");
116
+ var pubsub = require("./lib/pubsub");
116
117
  var websocketChannels = require("./lib/websocket-channels");
117
118
  var nonceStore = require("./lib/nonce-store");
118
119
  var scheduler = require("./lib/scheduler");
@@ -227,6 +228,7 @@ module.exports = {
227
228
  uuid: uuid,
228
229
  mail: mail,
229
230
  mailBounce: mailBounce,
231
+ pubsub: pubsub,
230
232
  websocketChannels: websocketChannels,
231
233
  nonceStore: nonceStore,
232
234
  scheduler: scheduler,
package/lib/cache.js CHANGED
@@ -739,6 +739,15 @@ function create(opts) {
739
739
  "redisUrl", "redisPassword", "redisUsername", "redisTls", "redisCa",
740
740
  "redisServername", "redisConnectTimeoutMs", "redisCommandTimeoutMs",
741
741
  "redisMaxReconnectAttempts",
742
+ // Cross-node invalidation: when set, every successful
743
+ // del/clear/invalidateTag publishes an event on the supplied
744
+ // pubsub instance. Other cache instances on other nodes (or in
745
+ // other processes sharing the pubsub backend) react locally —
746
+ // mostly useful for the memory backend so stale per-node entries
747
+ // don't survive a global tag wipe. The cluster + redis backends
748
+ // are coherent by virtue of their shared store, but a hot
749
+ // memory-tier on top of either still benefits.
750
+ "invalidationPubsub",
742
751
  ], "cache");
743
752
  _validateCreateOpts(opts);
744
753
 
@@ -756,6 +765,19 @@ function create(opts) {
756
765
  var audit = opts.audit || null;
757
766
  var operatorObs = opts.observability || null;
758
767
  var clock = opts.clock || function () { return Date.now(); };
768
+ var invalidationPubsub = opts.invalidationPubsub || null;
769
+ if (invalidationPubsub && (
770
+ typeof invalidationPubsub.publish !== "function" ||
771
+ typeof invalidationPubsub.subscribe !== "function" ||
772
+ typeof invalidationPubsub.unsubscribe !== "function")) {
773
+ throw _err("BAD_OPT",
774
+ "cache.create: invalidationPubsub must implement { publish, subscribe, unsubscribe } (b.pubsub.create instance)");
775
+ }
776
+ var invalidationChannel = "cache:" + namespace + ":invalidate";
777
+ var invalidationToken = null;
778
+ // Re-entrancy guard — when we receive an invalidation event from
779
+ // another node we MUST NOT re-publish it (infinite fan-out loop).
780
+ var inboundInvalidation = false;
759
781
 
760
782
  function emitObs(name, labels) {
761
783
  try {
@@ -908,6 +930,7 @@ function create(opts) {
908
930
  }
909
931
  if (existed) emitObs("cache.del", { namespace: namespace });
910
932
  softExpiry.delete(key);
933
+ _publishInvalidation({ kind: "del", key: key });
911
934
  return existed;
912
935
  }
913
936
 
@@ -946,6 +969,7 @@ function create(opts) {
946
969
  inflight.clear();
947
970
  swrInflight.clear();
948
971
  softExpiry.clear();
972
+ _publishInvalidation({ kind: "clear" });
949
973
  return purged;
950
974
  }
951
975
 
@@ -1003,6 +1027,7 @@ function create(opts) {
1003
1027
  // drop matches clear()'s safer-than-stale posture.
1004
1028
  inflight.clear();
1005
1029
  swrInflight.clear();
1030
+ _publishInvalidation({ kind: "tag", tag: tag });
1006
1031
  return purged;
1007
1032
  }
1008
1033
 
@@ -1134,9 +1159,45 @@ function create(opts) {
1134
1159
  return promise;
1135
1160
  }
1136
1161
 
1162
+ function _publishInvalidation(ev) {
1163
+ if (!invalidationPubsub || inboundInvalidation) return;
1164
+ try { invalidationPubsub.publish(invalidationChannel, ev); }
1165
+ catch (_e) { /* publish best-effort — local invalidation already happened */ }
1166
+ }
1167
+
1168
+ async function _onInboundInvalidation(ev /*, meta */) {
1169
+ if (!ev || closed) return;
1170
+ inboundInvalidation = true;
1171
+ try {
1172
+ if (ev.kind === "tag" && typeof ev.tag === "string" &&
1173
+ typeof backend.invalidateTag === "function") {
1174
+ try { await backend.invalidateTag(ev.tag); } catch (_e) {}
1175
+ } else if (ev.kind === "del" && typeof ev.key === "string") {
1176
+ try { await backend.del(ev.key); } catch (_e) {}
1177
+ } else if (ev.kind === "clear") {
1178
+ try { await backend.clear(); } catch (_e) {}
1179
+ }
1180
+ // Wipe local in-flight memoization so a freshly-invalidated key
1181
+ // can't resolve from a still-pending fetch on this node.
1182
+ inflight.clear();
1183
+ swrInflight.clear();
1184
+ softExpiry.clear();
1185
+ } finally {
1186
+ inboundInvalidation = false;
1187
+ }
1188
+ }
1189
+
1190
+ if (invalidationPubsub) {
1191
+ invalidationToken = invalidationPubsub.subscribe(invalidationChannel, _onInboundInvalidation);
1192
+ }
1193
+
1137
1194
  async function close() {
1138
1195
  if (closed) return;
1139
1196
  closed = true;
1197
+ if (invalidationPubsub && invalidationToken) {
1198
+ try { invalidationPubsub.unsubscribe(invalidationToken); } catch (_e) {}
1199
+ invalidationToken = null;
1200
+ }
1140
1201
  inflight.clear();
1141
1202
  swrInflight.clear();
1142
1203
  softExpiry.clear();
@@ -14,11 +14,14 @@
14
14
  * which fences out a partitioned old leader even if its application-
15
15
  * layer `_requireLeader()` gate somehow allowed the call through.
16
16
  *
17
- * Dialects: Postgres and SQLite use identical surface SQL. MySQL's
18
- * `ON DUPLICATE KEY UPDATE` doesn't support a WHERE clause; a MySQL
19
- * fallback (SELECT ... FOR UPDATE inside an explicit transaction) is
20
- * not yet implemented and operators on MySQL must supply their own
21
- * provider until it lands.
17
+ * Dialects: Postgres / SQLite use `INSERT ... ON CONFLICT ... DO
18
+ * UPDATE WHERE ... RETURNING` (atomic acquire-or-steal in one
19
+ * statement). MySQL takes a different shape because its
20
+ * `ON DUPLICATE KEY UPDATE` doesn't support a WHERE clause: each
21
+ * column update is gated by `IF(expiresAt < <nowMs>, VALUES(col), col)`
22
+ * so a non-expired lease is preserved untouched, and the followup
23
+ * SELECT reveals who currently holds the row. Both shapes are
24
+ * row-level atomic at the database — no client-side locking required.
22
25
  *
23
26
  * Public API:
24
27
  * create({ externalDbBackend, dialect? }) → provider instance
@@ -55,57 +58,67 @@ function create(config) {
55
58
  }
56
59
  var backendName = config.externalDbBackend;
57
60
  var dialect = (config.dialect || "postgres").toLowerCase();
58
- if (dialect !== "postgres" && dialect !== "sqlite") {
61
+ if (dialect !== "postgres" && dialect !== "sqlite" && dialect !== "mysql") {
59
62
  throw _err("UNSUPPORTED_DIALECT",
60
- "cluster-provider-db dialect must be 'postgres' or 'sqlite' (got: " + dialect + ")",
63
+ "cluster-provider-db dialect must be 'postgres', 'sqlite', or 'mysql' (got: " + dialect + ")",
61
64
  true);
62
65
  }
63
66
 
64
- // Postgres uses $1/$2 placeholders; SQLite accepts the same so we use
65
- // them uniformly. If a future dialect needs ?-placeholders, swap here.
66
- function _placeholder(n) { return "$" + n; }
67
+ // Postgres + SQLite use $1/$2 placeholders; MySQL uses ?. Keeping a
68
+ // single helper means the SQL builder doesn't have to care.
69
+ function _placeholder(n) {
70
+ return dialect === "mysql" ? "?" : "$" + n;
71
+ }
67
72
 
68
73
  function _q(sql, params) {
69
74
  return externalDb.query(sql, params || [], { backend: backendName });
70
75
  }
71
76
 
72
77
  async function ensureSchema() {
73
- // Postgres: BIGINT for ms-precision timestamps. SQLite: INTEGER works
74
- // for either; pick INTEGER so the same DDL parses on both.
75
- var intType = dialect === "postgres" ? "BIGINT" : "INTEGER";
78
+ // Postgres + MySQL: BIGINT for ms-precision timestamps. SQLite:
79
+ // INTEGER (which is wide enough to hold a 64-bit value).
80
+ var intType = dialect === "sqlite" ? "INTEGER" : "BIGINT";
81
+ // MySQL needs explicit lengths on TEXT columns when used as a
82
+ // PRIMARY KEY; VARCHAR(64) covers our scope/nodeId/leaseId values
83
+ // with room to spare. Postgres and SQLite happily PRIMARY-KEY a
84
+ // plain TEXT column.
85
+ var pkText = dialect === "mysql" ? "VARCHAR(64)" : "TEXT";
86
+ var bodyText = dialect === "mysql" ? "VARCHAR(255)" : "TEXT";
87
+ // MySQL doesn't enforce CHECK constraints in some MariaDB / older
88
+ // MySQL versions; the constant-scope invariant is a documentation
89
+ // belt-and-braces — application code only ever writes 'leader' /
90
+ // 'state' so the check is informational. Skip on MySQL to avoid
91
+ // CREATE TABLE failures on installations where CHECK is parsed
92
+ // but then dropped silently (which would cause version drift).
93
+ var leaderCheck = dialect === "mysql" ? "" : ", CHECK (scope = 'leader')";
94
+ var stateCheck = dialect === "mysql" ? "" : ", CHECK (scope = 'state')";
76
95
 
77
96
  await _q(
78
97
  "CREATE TABLE IF NOT EXISTS _blamejs_leader (" +
79
- " scope TEXT PRIMARY KEY," +
80
- " nodeId TEXT NOT NULL," +
81
- " leaseId TEXT NOT NULL," +
98
+ " scope " + pkText + " PRIMARY KEY," +
99
+ " nodeId " + bodyText + " NOT NULL," +
100
+ " leaseId " + bodyText + " NOT NULL," +
82
101
  " acquiredAt " + intType + " NOT NULL," +
83
102
  " expiresAt " + intType + " NOT NULL," +
84
103
  " fencingToken " + intType + " NOT NULL," +
85
- " endpoint TEXT," +
86
- " CHECK (scope = 'leader')" +
104
+ " endpoint " + bodyText + leaderCheck +
87
105
  ")"
88
106
  );
89
107
  // Migration for installs that pre-date the endpoint column. Both
90
108
  // Postgres (≥9.6) and SQLite (≥3.35, March 2021) support ADD COLUMN
91
- // IF NOT EXISTS, but we go through try/catch to keep the path
92
- // dialect-version-agnostic — the only "expected" failure here is
93
- // "column already exists," which we swallow.
109
+ // IF NOT EXISTS; MySQL 8.0.29+ does as well. We go through try/catch
110
+ // to keep the path version-agnostic — the only "expected" failure
111
+ // here is "column already exists," which we swallow.
94
112
  try {
95
- await _q("ALTER TABLE _blamejs_leader ADD COLUMN endpoint TEXT");
113
+ await _q("ALTER TABLE _blamejs_leader ADD COLUMN endpoint " + bodyText);
96
114
  } catch (_e) { /* column already exists — fine */ }
97
115
 
98
- // _blamejs_cluster_state — single-row consistency check used to
99
- // detect vault-key drift across cluster nodes. Same single-row
100
- // invariant pattern as _blamejs_leader (PRIMARY KEY on a
101
- // constant-valued scope column + CHECK).
102
116
  await _q(
103
117
  "CREATE TABLE IF NOT EXISTS _blamejs_cluster_state (" +
104
- " scope TEXT PRIMARY KEY," +
105
- " vaultKeyFp TEXT NOT NULL," +
118
+ " scope " + pkText + " PRIMARY KEY," +
119
+ " vaultKeyFp " + bodyText + " NOT NULL," +
106
120
  " recordedAt " + intType + " NOT NULL," +
107
- " recordedByNode TEXT NOT NULL," +
108
- " CHECK (scope = 'state')" +
121
+ " recordedByNode " + bodyText + " NOT NULL" + stateCheck +
109
122
  ")"
110
123
  );
111
124
  }
@@ -120,35 +133,65 @@ function create(config) {
120
133
  var nowMs = Date.now();
121
134
  var expiresAt = nowMs + leaseTtlMs;
122
135
 
123
- // Atomic acquire: insert if no row, OR steal if existing row's
124
- // expiresAt has passed. Bump fencingToken on every successful
125
- // mutation. RETURNING gives us the post-write state which we
126
- // compare to our nodeId to know whether we won.
127
- var sql =
128
- "INSERT INTO _blamejs_leader " +
129
- " (scope, nodeId, leaseId, acquiredAt, expiresAt, fencingToken, endpoint) " +
130
- "VALUES " +
131
- " ('leader', " + _placeholder(1) + ", " + _placeholder(2) + ", " +
132
- " " + _placeholder(3) + ", " + _placeholder(4) + ", 1, " + _placeholder(5) + ") " +
133
- "ON CONFLICT (scope) DO UPDATE SET " +
134
- " nodeId = EXCLUDED.nodeId," +
135
- " leaseId = EXCLUDED.leaseId," +
136
- " acquiredAt = EXCLUDED.acquiredAt," +
137
- " expiresAt = EXCLUDED.expiresAt," +
138
- " fencingToken = _blamejs_leader.fencingToken + 1," +
139
- " endpoint = EXCLUDED.endpoint " +
140
- "WHERE _blamejs_leader.expiresAt < " + _placeholder(6) + " " +
141
- "RETURNING nodeId, leaseId, acquiredAt, expiresAt, fencingToken, endpoint";
142
-
143
- var result = await _q(sql, [nodeId, leaseId, nowMs, expiresAt, endpoint, nowMs]);
144
-
145
- if (!result.rows || result.rows.length === 0) {
146
- // The WHERE clause excluded the row someone else still holds.
147
- return null;
136
+ var row;
137
+ if (dialect === "mysql") {
138
+ // MySQL has no `ON CONFLICT ... DO UPDATE WHERE` and no
139
+ // `RETURNING`. Atomicity comes from `INSERT ... ON DUPLICATE
140
+ // KEY UPDATE` evaluated as one statement; the WHERE-clause
141
+ // semantics are implemented per-column with `IF(expiresAt <
142
+ // nowMs, VALUES(col), col)` so a still-valid lease is preserved
143
+ // and an expired one is overwritten. The follow-up SELECT
144
+ // reveals who currently holds the row same as Postgres'
145
+ // RETURNING but as a separate statement.
146
+ var insertSql =
147
+ "INSERT INTO _blamejs_leader " +
148
+ " (scope, nodeId, leaseId, acquiredAt, expiresAt, fencingToken, endpoint) " +
149
+ "VALUES " +
150
+ " ('leader', ?, ?, ?, ?, 1, ?) " +
151
+ "ON DUPLICATE KEY UPDATE " +
152
+ " nodeId = IF(expiresAt < ?, VALUES(nodeId), nodeId)," +
153
+ " leaseId = IF(expiresAt < ?, VALUES(leaseId), leaseId)," +
154
+ " acquiredAt = IF(expiresAt < ?, VALUES(acquiredAt), acquiredAt)," +
155
+ " fencingToken = IF(expiresAt < ?, fencingToken + 1, fencingToken)," +
156
+ " endpoint = IF(expiresAt < ?, VALUES(endpoint), endpoint)," +
157
+ // expiresAt MUST be the last assignment — IF() evaluates each
158
+ // column against the row state BEFORE that column's update is
159
+ // applied, so checking expiresAt for the other columns first
160
+ // and overwriting it last keeps the predicate consistent.
161
+ " expiresAt = IF(expiresAt < ?, VALUES(expiresAt), expiresAt)";
162
+ await _q(insertSql, [
163
+ nodeId, leaseId, nowMs, expiresAt, endpoint,
164
+ nowMs, nowMs, nowMs, nowMs, nowMs, nowMs,
165
+ ]);
166
+ var sel = await _q(
167
+ "SELECT nodeId, leaseId, acquiredAt, expiresAt, fencingToken, endpoint " +
168
+ "FROM _blamejs_leader WHERE scope = 'leader'"
169
+ );
170
+ if (!sel.rows || sel.rows.length === 0) return null;
171
+ row = sel.rows[0];
172
+ } else {
173
+ // Postgres / SQLite — single-statement RETURNING.
174
+ var sql =
175
+ "INSERT INTO _blamejs_leader " +
176
+ " (scope, nodeId, leaseId, acquiredAt, expiresAt, fencingToken, endpoint) " +
177
+ "VALUES " +
178
+ " ('leader', " + _placeholder(1) + ", " + _placeholder(2) + ", " +
179
+ " " + _placeholder(3) + ", " + _placeholder(4) + ", 1, " + _placeholder(5) + ") " +
180
+ "ON CONFLICT (scope) DO UPDATE SET " +
181
+ " nodeId = EXCLUDED.nodeId," +
182
+ " leaseId = EXCLUDED.leaseId," +
183
+ " acquiredAt = EXCLUDED.acquiredAt," +
184
+ " expiresAt = EXCLUDED.expiresAt," +
185
+ " fencingToken = _blamejs_leader.fencingToken + 1," +
186
+ " endpoint = EXCLUDED.endpoint " +
187
+ "WHERE _blamejs_leader.expiresAt < " + _placeholder(6) + " " +
188
+ "RETURNING nodeId, leaseId, acquiredAt, expiresAt, fencingToken, endpoint";
189
+ var result = await _q(sql, [nodeId, leaseId, nowMs, expiresAt, endpoint, nowMs]);
190
+ if (!result.rows || result.rows.length === 0) return null;
191
+ row = result.rows[0];
148
192
  }
149
- var row = result.rows[0];
150
193
  if (row.nodeId !== nodeId || row.leaseId !== leaseId) {
151
- // Another node won the race (RETURNING showed their values, not ours).
194
+ // Another node won the race (the row reflects their values, not ours).
152
195
  return null;
153
196
  }
154
197
  return {
@@ -171,24 +214,52 @@ function create(config) {
171
214
  var endpoint = (opts && opts.endpoint !== undefined) ? opts.endpoint : lease.endpoint || null;
172
215
 
173
216
  // Match on (nodeId, leaseId) so a takeover is detectable: if our
174
- // leaseId is no longer in the row, 0 rows update and we throw
175
- // NotLeader. Don't bump fencingToken on renewal only on a fresh
176
- // acquire.
177
- var sql =
178
- "UPDATE _blamejs_leader SET " +
179
- " expiresAt = " + _placeholder(1) + "," +
180
- " endpoint = " + _placeholder(2) + " " +
181
- "WHERE scope = 'leader' AND nodeId = " + _placeholder(3) +
182
- " AND leaseId = " + _placeholder(4) + " " +
183
- "RETURNING nodeId, leaseId, acquiredAt, expiresAt, fencingToken, endpoint";
184
-
185
- var result = await _q(sql, [newExpiresAt, endpoint, lease.nodeId, lease.leaseId]);
186
- if (!result.rows || result.rows.length === 0) {
187
- throw _err("LEASE_LOST",
188
- "lease for node '" + lease.nodeId + "' was taken over (renewal rejected)",
189
- false);
217
+ // leaseId is no longer in the row, the SELECT-after-UPDATE
218
+ // returns either no row OR a row with a different leaseId, and
219
+ // we throw LEASE_LOST. Don't bump fencingToken on renewal — only
220
+ // on a fresh acquire.
221
+ var row;
222
+ if (dialect === "mysql") {
223
+ var rv = await _q(
224
+ "UPDATE _blamejs_leader SET " +
225
+ " expiresAt = ?, endpoint = ? " +
226
+ "WHERE scope = 'leader' AND nodeId = ? AND leaseId = ?",
227
+ [newExpiresAt, endpoint, lease.nodeId, lease.leaseId]
228
+ );
229
+ var affected = rv && (rv.affectedRows || rv.rowCount || 0);
230
+ if (!affected) {
231
+ throw _err("LEASE_LOST",
232
+ "lease for node '" + lease.nodeId + "' was taken over (renewal rejected)",
233
+ false);
234
+ }
235
+ var sel = await _q(
236
+ "SELECT nodeId, leaseId, acquiredAt, expiresAt, fencingToken, endpoint " +
237
+ "FROM _blamejs_leader WHERE scope = 'leader'"
238
+ );
239
+ if (!sel.rows || sel.rows.length === 0 ||
240
+ sel.rows[0].nodeId !== lease.nodeId ||
241
+ sel.rows[0].leaseId !== lease.leaseId) {
242
+ throw _err("LEASE_LOST",
243
+ "lease for node '" + lease.nodeId + "' was taken over after renewal",
244
+ false);
245
+ }
246
+ row = sel.rows[0];
247
+ } else {
248
+ var sql =
249
+ "UPDATE _blamejs_leader SET " +
250
+ " expiresAt = " + _placeholder(1) + "," +
251
+ " endpoint = " + _placeholder(2) + " " +
252
+ "WHERE scope = 'leader' AND nodeId = " + _placeholder(3) +
253
+ " AND leaseId = " + _placeholder(4) + " " +
254
+ "RETURNING nodeId, leaseId, acquiredAt, expiresAt, fencingToken, endpoint";
255
+ var result = await _q(sql, [newExpiresAt, endpoint, lease.nodeId, lease.leaseId]);
256
+ if (!result.rows || result.rows.length === 0) {
257
+ throw _err("LEASE_LOST",
258
+ "lease for node '" + lease.nodeId + "' was taken over (renewal rejected)",
259
+ false);
260
+ }
261
+ row = result.rows[0];
190
262
  }
191
- var row = result.rows[0];
192
263
  return {
193
264
  nodeId: row.nodeId,
194
265
  leaseId: row.leaseId,
package/lib/db.js CHANGED
@@ -110,7 +110,7 @@ var RESERVED_TABLE_NAMES = new Set([
110
110
  "_blamejs_audit_purge_anchor",
111
111
  "_blamejs_scheduler_ticks",
112
112
  "_blamejs_rate_limit_counters",
113
- "_blamejs_ws_messages",
113
+ "_blamejs_pubsub_messages",
114
114
  "_blamejs_api_encrypt_nonces",
115
115
  "_blamejs_api_keys",
116
116
  "_blamejs_cache",
@@ -260,15 +260,16 @@ var FRAMEWORK_SCHEMA = [
260
260
  sealedFields: [],
261
261
  },
262
262
  {
263
- // _blamejs_ws_messages — cluster fan-out for the WebSocket
264
- // channel hub. publish() on any node writes a row here; the
265
- // other nodes poll for new ids and dispatch to their local
266
- // subscribers. Rows older than the configured retention window
267
- // are pruned by the hub on a rate-limited basis.
268
- name: "_blamejs_ws_messages",
263
+ // _blamejs_pubsub_messages — cluster fan-out for `b.pubsub` (the
264
+ // generalization of the previous WebSocket-specific table). Any
265
+ // pubsub instance using the `cluster` backend writes a row on
266
+ // publish; other nodes poll for new ids and dispatch to their
267
+ // local subscribers. Rows older than the configured retention
268
+ // window are pruned by the backend on a rate-limited basis.
269
+ name: "_blamejs_pubsub_messages",
269
270
  columns: {
270
271
  id: "INTEGER PRIMARY KEY AUTOINCREMENT",
271
- channel: "TEXT NOT NULL",
272
+ topic: "TEXT NOT NULL",
272
273
  payload: "TEXT NOT NULL",
273
274
  publishedAt: "INTEGER NOT NULL",
274
275
  publishedBy: "TEXT NOT NULL",
@@ -94,7 +94,7 @@ var LOCAL_TO_EXTERNAL = Object.freeze({
94
94
  // WebSocket channel-hub cluster fan-out — publish() writes a row,
95
95
  // other nodes poll for new ids and dispatch to their local
96
96
  // subscribers. Same dual-storage shape as sessions / jobs / etc.
97
- _blamejs_ws_messages: "_blamejs_ws_messages",
97
+ _blamejs_pubsub_messages: "_blamejs_pubsub_messages",
98
98
  _blamejs_api_encrypt_nonces: "_blamejs_api_encrypt_nonces",
99
99
  // _blamejs_api_keys — operator-facing API-key registry table for the
100
100
  // b.apiKey primitive. PRIMARY KEY is namespace-scoped id (so multiple
@@ -368,14 +368,15 @@ function _rateLimitCountersDDL(dialect) {
368
368
  };
369
369
  }
370
370
 
371
- // _blamejs_ws_messages — cluster fan-out for the WebSocket channel
372
- // hub. publish() on any node writes a row; the other nodes poll for
373
- // new ids past their last seen and dispatch to local subscribers.
374
- // Auto-incrementing id is essential postgres needs BIGSERIAL,
375
- // sqlite gets INTEGER PRIMARY KEY (which auto-increments implicitly).
376
- function _wsMessagesDDL(dialect) {
371
+ // _blamejs_pubsub_messages — cluster fan-out for `b.pubsub` (the
372
+ // generalization of the previous WebSocket-specific table). publish()
373
+ // on any node writes a row; other nodes poll for new ids past their
374
+ // last seen and dispatch to local subscribers. Auto-incrementing id
375
+ // is essential postgres needs BIGSERIAL, sqlite gets INTEGER
376
+ // PRIMARY KEY (which auto-increments implicitly).
377
+ function _pubsubMessagesDDL(dialect) {
377
378
  var t = _types(dialect);
378
- var name = LOCAL_TO_EXTERNAL._blamejs_ws_messages;
379
+ var name = LOCAL_TO_EXTERNAL._blamejs_pubsub_messages;
379
380
  var idCol = dialect === "postgres"
380
381
  ? "id BIGSERIAL PRIMARY KEY"
381
382
  : "id INTEGER PRIMARY KEY AUTOINCREMENT";
@@ -383,7 +384,7 @@ function _wsMessagesDDL(dialect) {
383
384
  create:
384
385
  "CREATE TABLE IF NOT EXISTS " + name + " (" +
385
386
  " " + idCol + "," +
386
- " channel TEXT NOT NULL," +
387
+ " topic TEXT NOT NULL," +
387
388
  " payload TEXT NOT NULL," +
388
389
  " publishedAt " + t.INT + " NOT NULL," +
389
390
  " publishedBy TEXT NOT NULL" +
@@ -683,7 +684,7 @@ async function ensureSchema(opts) {
683
684
  _auditPurgeAnchorDDL(dialect),
684
685
  _schedulerTicksDDL(dialect),
685
686
  _rateLimitCountersDDL(dialect),
686
- _wsMessagesDDL(dialect),
687
+ _pubsubMessagesDDL(dialect),
687
688
  _apiEncryptNoncesDDL(dialect),
688
689
  _apiKeysDDL(dialect),
689
690
  _sessionsDDL(dialect),