@blamejs/core 0.6.29 → 0.6.32

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -8,6 +8,9 @@ upgrading across more than a few patches at a time.
8
8
 
9
9
  ## v0.6.x
10
10
 
11
+ - **0.6.32** (2026-05-02) — E1 from the v0.6.x scope plan. **`b.cache` Redis backend** — first-class `backend: "redis"` for `b.cache.create({...})`, no operator-supplied glue needed. New module `lib/cache-redis.js` consumes `lib/redis-client.js` directly; storage layout: `<namespace>:e:<key>` (STRING, JSON-encoded value, PEXPIREAT-bounded), `<namespace>:t:<tag>` (SET, cacheKeys carrying that tag — powers `invalidateTag` fan-out), `<namespace>:k:<key>:tags` (SET, tags this key carries — powers per-key tag cleanup on `del`/`set`-overwrite, expires alongside the entry). TTL is enforced by Redis itself (PEXPIREAT) so the framework's sweeper is a no-op for this backend; sliding TTL bumps the entry's expiry on every read when `slidingTtl: true` AND `ttlMs` is finite. New opts on `cache.create`: `redisUrl` (required when `backend: "redis"`), `redisPassword`, `redisUsername`, `redisTls`, `redisCa` (private-CA trust pinning per v0.6.28), `redisServername`, `redisConnectTimeoutMs`, `redisCommandTimeoutMs`, `redisMaxReconnectAttempts`. Concurrency: invalidateTag filters out ghost entries (key already PEXPIRE'd from the keyspace but lingering in a tag SET) by EXISTS-checking before del. Lazy connect — `cache.create({backend:"redis"})` stays sync-safe; first op opens the socket. Tests: 10 new live integration assertions on top of the existing memory + custom-backend coverage (set+get round-trip, has, del, complex JSON value round-trip, short-TTL Redis-side expiry, multi-tag invalidateTag fan-out + preservation of untagged entries, wrap() single-flight memoization through Redis). Smoke 6780 / wiki e2e 178 / per-primitive integration 14 files (cache up from 15→25 checks) / wiki integration green.
12
+ - **0.6.31** (2026-05-02) — lazy-deferral cleanup wave from v0.6.27. **A1: queue-redis priority ordering** — `b.queue.enqueue({ priority })` on the Redis backend now matches queue-local's `ORDER BY priority DESC, availableAt ASC, enqueuedAt ASC` semantics. The Redis ZSET stays scored by availableAt only (so ZRANGEBYSCORE 0..nowMs cleanly filters ready vs not-yet-ready jobs), but `LEASE_LUA` now over-fetches `maxRows*5` candidates by score, HMGETs priority + availableAt + enqueuedAt for each, sorts server-side in Lua by the same triple queue-local sorts on, and leases the top `maxRows`. Closes the v0.6.27 deferral that punted priority ordering as "ZSET-incompatible". **A2: queue-redis flow `dependsOn` cascade** — `b.queue.enqueue({ flowId, flowChildName, dependsOn })` on the Redis backend now releases dependent jobs when their parents complete, mirroring queue-local's `_maybeReleaseFlowChildren`. A per-flow `<prefix>:flow:<flowId>` Redis SET tracks every job in the flow; `complete()` walks the set, looks up siblings whose `dependsOn` includes the just-completed job (by id OR `flowChildName`) AND every other dep is also satisfied, and HSET-bumps their `availableAt` to now + ZADDs them into the ready zset. The flow set persists across `complete()` calls so a later sibling's deps check can still find earlier-done parents by name; `purge()` cleans the set on queue teardown. Closes the v0.6.27 deferral. **A4: WebSocket per-message-deflate (RFC 7692)** — `b.websocket.handleUpgrade` negotiates the `permessage-deflate` extension when the client offers it, accepting `client_max_window_bits` / `server_max_window_bits` constraints (8-15 range, default 15) and always asserting `client_no_context_takeover` + `server_no_context_takeover` so every message uses a fresh zlib state. The connection's send path compresses TEXT/BINARY frames via `zlib.deflateRawSync`, strips the 4-byte `0x00 0x00 0xff 0xff` trailer per RFC 7692 §7.2.1, and sets RSV1 on the first frame of each compressed message. The receive path appends the trailer back and inflates via `zlib.inflateRawSync`. RSV1 on a continuation frame, RSV1 without negotiated extension, RSV2/RSV3 set, and decompressed payload exceeding `maxMessageBytes` all close with the appropriate RFC 6455 status (`PROTOCOL_ERROR`, `INVALID_PAYLOAD`, `MESSAGE_TOO_BIG`). Operator opt-out: pass `permessageDeflate: false` to `handleUpgrade` to refuse the extension even when offered. Tests cover the 101-handshake echo of negotiated params, real compression on a redundant payload (4000 bytes → ~30 bytes on the wire), uncompressed-frame round-trip on the same connection (mixed-mode per RFC 7692 §6), and unknown-extension graceful ignore. v0.6.27's third lazy deferral A3 (TLS-redis live test) was already covered by `test/integration/redis-client-tls.test.js` shipped in v0.6.28. Smoke 6780 / wiki e2e 178 / per-primitive integration 14 files green / wiki integration green.
13
+ - **0.6.30** (2026-05-02) — wiki-app integration gate + framework follow-ups surfaced by it. **`scripts/test-wiki-integration.js`** boots `examples/wiki` against the docker-compose fixture stack (real Redis / MinIO / Mailpit / CoreDNS / NTP / mtls-ca) and drives every backend through the wiki's HTTP surface AND the underlying framework primitives — validates that each primitive routes through the configured backend (cache → custom backend, queue → Redis Streams, mail → Mailpit SMTP, object-store → MinIO sigv4, log-stream → webhook receiver) AND that every fix shipped in v0.6.28 holds in a real-app context, not just in unit-test isolation. **Wiki gains test-only routes** under `/test/*` (gated by `WIKI_INTEGRATION_TEST=1`, mounted before CSRF/staticServe so test POSTs don't have to round-trip a token cookie) covering cache get/set/del, queue enqueue/size, mail send, object-store put/get, http-client fetch, log-stream emit, mtls-ca issue, ntp query, dns lookup, ssrf classify+check, plus `/test/diagnostic` that surfaces the active backend posture (network snapshot, queue backends, log sinks, mtls algorithm). **External-integration two-gate rule** — when a release diff touches a primitive that talks to an external service (`lib/redis-client.js`, `lib/queue-redis.js`, `lib/mail.js`, `lib/network-dns.js`, `lib/object-store/*`, `lib/log-stream*.js`, `lib/external-db.js`, `lib/cluster-*.js`, `lib/mtls-ca.js`, `lib/ssrf-guard.js`, `lib/http-client.js`, `lib/ntp-check.js`, `lib/cache.js`, `lib/webhook.js`), operators MUST run BOTH `scripts/test-integration.js` (per-primitive against real backends) AND `scripts/test-wiki-integration.js` (wiki app exercising the same backends end-to-end) before pushing. Documented in CONTRIBUTING.md's pre-push gate list and CLAUDE.md's release workflow as step 4a. The smoke + wiki-e2e gates above stay PURE (no docker dependency, runs in CI / on a developer laptop / inside prepack-guard); the two-gate is operator-run, opt-in, and surfaces bugs that mocks miss — fire-and-forget races, shutdown drains, TLS pinning, real DNS resolution, real protocol handshakes. **Framework fixes surfaced by the new gate**: `b.mtlsCa.create` now auto-creates `opts.dataDir` with `mode: 0o700` if it doesn't exist (matches `b.logStream`'s local sink, `b.backup`, `b.restoreBundle` — without it the first `initCA()` call hit `ENOENT` writing `ca.key.tmp`); `b.logStream` webhook-sink `close()` now waits for the in-flight `_flush()` before draining the buffer (the v0.6.28 drain fix only tracked emit-time wrapper promises, not the actual HTTP POST in flight; a record arriving mid-flush would buffer + the emit-time `_flush` early-returned on `if (inFlight) return`, and shutdown would then strand it). `b.queue.bootFromEnv()` is now wired in the wiki app's `build-app.js` unconditionally so the wiki picks up `BLAMEJS_QUEUE_PROTOCOL=redis` from env without code changes (was previously a documented-but-commented call site). **Test architecture rule (extends the v0.6.28 rule)** — live tests for the framework's primitives go under `test/integration/`; live tests for the wiki app exercising the framework end-to-end go under `examples/wiki/test/integration.js`. Smoke (`test/smoke.js`) and the existing wiki e2e (`examples/wiki/test/e2e.js`) stay pure. Smoke / wiki-e2e green; per-primitive integration suite green; wiki integration green. Eslint pin alignment + shellcheck added to the documented gate list (carry-over from v0.6.29).
11
14
  - **0.6.29** (2026-05-02) — CI cleanup. **shellcheck gate green**: `docker/init/generate-certs.sh`'s footer line counted output via `ls "$CERT_DIR" | wc -l` (SC2012 — fragile on filenames with newlines or quoting metacharacters); replaced with `find "$CERT_DIR" -maxdepth 1 -mindepth 1 | wc -l`. Same semantics, robust to any cert filename pki-init might end up writing. **shellcheck added to the documented pre-push gate list** in CONTRIBUTING.md alongside smoke / wiki e2e / eslint / api-snapshot / primitive-section validators — CI's `Lint summary` job has always run it, but the local-dev recipe didn't, so contributors hit it post-push instead of catching it locally. **eslint pin alignment**: `.github/PULL_REQUEST_TEMPLATE.md` and CONTRIBUTING.md still pinned `eslint@10` even though v0.6.16 swapped CI runners to `eslint@latest`; both bumped to `@latest` to match the forward-track posture of every other CI tool. No framework code changes; the script + doc updates never ship in the npm package (excluded by `files` array).
12
15
  - **0.6.28** (2026-05-02) — bug-finding pass: live integration test suite + every framework bug it caught. **`b.ssrfGuard.checkUrl` cloud-metadata hard-deny** — `allowInternal: true` no longer permits 169.254.169.254 (AWS / GCP / Azure metadata). Loopback / private / link-local / reserved stay allowInternal-overridable per existing semantics; cloud-metadata is unconditional because a blanket override would let any compromised request exfiltrate instance credentials. **`b.mtlsCa` issues server certs and dual-EKU certs** — `generateClientCert({ usage: "client" | "server" | "both", sans: [...] })`. `usage` controls EKU (`client` = clientAuth, `server` = serverAuth, `both` = both); `sans` accepts `DNS:` / `IP:` / bare-DNS entries; serverAuth without an explicit SAN auto-adds the CN. Operators wiring inbound mTLS reverse-proxy fronts no longer hit "unsuitable certificate purpose" on the handshake. **`b.mtlsCa` algorithm auto-detect** — the engine probes webcrypto + the vendored x509 library at first cert issuance and picks the highest-PQC option that round-trips: SLH-DSA-SHAKE-256f → SLH-DSA-SHAKE-128f → ML-DSA-87 → ML-DSA-65 → ECDSA-P384-SHA384 (current bridge — what the X.509 ecosystem accepts today). When the bridge condition lifts and the vendor refresh ships, the same `b.mtlsCa.create(...)` call self-upgrades; `b.mtlsCa.status().cert.label` and `.posture` surface the chosen algorithm. **`b.redisClient.create({ ca, servername })`** — managed-Redis / on-prem-cluster operators connecting over `rediss://` against a private CA can pin trust roots; `servername` auto-suppresses for IP literals so `rediss://127.0.0.1:6380` no longer trips node:tls's IP-as-SNI rule. **`b.network.dns.useDnsOverTls` / `useDnsOverHttps` accept `ca`** — same trust-pinning surface for self-signed / private-PKI DoT and DoH endpoints. **DoT TLS handshake errors** now route as DnsError on the lookup promise rather than leaking as `unhandledRejection` — the secureConnect / error event-listener race had let cert-verification failures escape the per-query Promise wrapping. **DoT socket lifecycle** — was unconditionally `sock.unref()`'d after construct, causing node to exit during in-flight lookups when no other I/O kept the loop alive; now ref'd while a query is in flight and unref'd when idle. **`b.ntpCheck.querySingle` honours IPv6 servers** — was hardcoded to `dgram.createSocket("udp4")`, so `::1` / `fd00::…` queries failed with EINVAL; now auto-detects family from the server string. **`b.logStream.shutdown` drains in-flight emit microtasks** before closing sink fds — fire-and-forget emits queued just before shutdown were silently dropped because `close()` ran ahead of the microtask. **`b.logStream` webhook-sink `close()` flushes BEFORE setting `closed = true`** — `_flush`'s while loop bailed on `!closed`, leaving any records buffered just before shutdown stranded. **Integration test suite** (`test/integration/`, runner at `scripts/test-integration.js`) — 13 live test files across `b.redisClient` plain + TLS, `b.queue` Redis backend lifecycle, `b.mail` SMTP + STARTTLS + multi-rcpt + dot-stuffing transparency, `b.mail.dkim` rsa-sha256 + ed25519-sha256 + bad-algorithm reject, `b.ntpCheck` v4 + v6 + bootCheck + bad-host + bad-port, `b.network.dns` plain + DoT + DoH with strict CA pinning + bad-servername + cache, `b.network.heartbeat` http + tcp + state-change callbacks, `b.objectStore` sigv4 PUT / GET / list / delete on plain HTTP + TLS variants, `b.cache` memory + Redis-backed cluster, `b.httpClient` direct + Squid forward proxy + TLS pin, `b.ssrfGuard` classify + checkUrl + cloud-metadata block, `b.logStream` local + webhook + deferred-syslog error path, `b.mtlsCa` CA bootstrap + clientAuth + serverAuth + dual-EKU live mTLS handshakes. Companion `docker-compose.test.yml` stands up redis (plain + TLS) / postgres / mysql / mongo / minio (HTTP + HTTPS variants) / rabbitmq (plain + TLS) / nats / syslog / ntp / mailpit / coredns (plain + DoT + DoH) / haproxy / caddy / mitmproxy / squid / pki-init (auto-generates Ed25519 CA + per-service leaf certs into a docker volume); host port bindings dual-stack on `127.0.0.1` AND `[::1]`. `scripts/check-services.js` does host-side TCP / TLS handshake / SNTPv4 / DNS-A / Redis-PING / SMTP-banner / HTTP probes across every endpoint; `scripts/test-integration.js` exports the test CA via `docker cp` and sets `NODE_EXTRA_CA_CERTS` per-test child process so the framework's TLS verification stays strict (no `rejectUnauthorized: false` bypass anywhere in the test surface). **Test architecture rule** — live tests go in `test/integration/`, NEVER in `test/layer-N-*/`: smoke must remain pure (no docker dependency, runs in CI / on a developer laptop / inside prepack-guard) and a "skip silently when service is down" branch in a layer-N test makes the gate's pass count misleading and masks bugs that only surface against a live backend. The `0.6.27` `queue-redis.test.js` was moved to `test/integration/queue-redis.test.js` to enforce this. Smoke and wiki e2e green; integration suite ships green against the docker-compose stack.
13
16
  - **0.6.27** (2026-05-02) — Redis backend for `b.queue` so multi-replica apps can share a single queue without each needing to be cluster leader. **Bespoke RESP2 client** (`lib/redis-client.js`) — zero npm runtime deps. TCP via `node:net` + TLS via `node:tls` (`rediss://` auto-detected), legacy single-arg AUTH + ACL `AUTH user pass`, `SELECT db`, pipelining, exponential-backoff reconnect, EVAL helper. **`b.queue` protocol "redis"** (`lib/queue-redis.js`) — full enqueue/lease/extendLease/complete/fail/sweepExpired/size/purge/dlqList/dlqRetry/dlqSize parity with the local backend. Atomicity comes from server-side Lua scripts so concurrent consumers can't double-lease and a sweep can't race a complete. Storage layout: per-job HASH (sealed payload + lastError via `cryptoField.sealRow("_blamejs_jobs", row)` — same crypto config as the local backend), per-queue ready ZSET scored by availableAt, per-queue inflight ZSET scored by leaseExpiresAt, per-queue dlq ZSET scored by finishedAt, plus a queues SET so sweepExpired walks every known queue without a global secondary index. Cron-repeat handled in `complete()` JS — re-enqueues the next firing as a fresh jobId with availableAt=next-cron-fire. **`b.queue.bootFromEnv({ env })`** — env-driven init mirroring `b.network.bootFromEnv` and `b.logStream.bootFromEnv`. Reads `BLAMEJS_QUEUE_PROTOCOL` (`local`|`redis`), `BLAMEJS_QUEUE_REDIS_URL`, `BLAMEJS_QUEUE_REDIS_PASSWORD`, `BLAMEJS_QUEUE_REDIS_USERNAME`, `BLAMEJS_QUEUE_REDIS_TLS`, `BLAMEJS_QUEUE_REDIS_KEY_PREFIX`. Operators flip from local to Redis without a code change; both wiki docker-compose configs declare the new env knobs. Removed `redis` from `DEFERRED_PROTOCOLS`. **Out of scope for v1** (deferred to follow-up patches with explicit re-open conditions): Redis Cluster (slot-routing), Sentinel (managed primary failover), priority ordering on the Redis backend (queue-local supports `priority` opt; Redis backend orders strictly by availableAt for v1), flow children with `dependsOn` cascade. **Wiki**: queue-cache page documents the Redis backend opts schema, bootFromEnv, and the layout. **Tests**: 24 RESP2 protocol parser unit tests (`test/layer-0-primitives/redis-client.test.js`) cover URL parsing, command encoding (binary-safe), every reply type (simple string / error / integer / bulk / nil bulk / nested array / pipelined / incomplete-mid-frame). Live Redis round-trip tests (`test/layer-0-primitives/queue-redis.test.js`) cover enqueue+lease, availableAt scheduling, visibility-timeout sweep, fail+retry path, DLQ list/retry/size, extendLease, purge, and concurrent-leaser no-double-lease — skip cleanly when `BLAMEJS_TEST_REDIS_URL` is not set so the smoke suite passes on dev boxes without a Redis container. Smoke 6780 OK.
@@ -0,0 +1,261 @@
1
+ "use strict";
2
+ /**
3
+ * Redis-backed cache backend for b.cache.
4
+ *
5
+ * Storage layout (operator-overridable namespace; default = "blamejs:cache"):
6
+ * <namespace>:e:<key> STRING — JSON-encoded value
7
+ * PEXPIREAT honours the cache's TTL
8
+ * <namespace>:t:<tag> SET — cacheKeys carrying that tag
9
+ * (powers invalidateTag fan-out)
10
+ * <namespace>:k:<key>:tags SET — tags this key carries
11
+ * (powers per-key tag cleanup on
12
+ * del / set-overwrite, expires
13
+ * alongside the entry)
14
+ *
15
+ * Same backend contract as the framework's other cache backends (memory,
16
+ * cluster, custom): get / set / del / has / clear / size / close, plus
17
+ * the optional invalidateTag + getTags + bytes + _startSweep hooks the
18
+ * `_customBackend` wrapper looks for.
19
+ *
20
+ * TTL is enforced by Redis itself (PEXPIREAT) — the framework does NOT
21
+ * run a sweep timer for this backend; expired entries vanish from the
22
+ * key space without app-side intervention.
23
+ *
24
+ * Sliding TTL: bumps the entry's expiry to `now + defaultTtlMs` on every
25
+ * read when the cache was created with `slidingTtl: true` and the cache
26
+ * has a finite defaultTtlMs. Same shape as the cluster backend.
27
+ *
28
+ * Concurrency: Redis is single-threaded for command execution; our get
29
+ * + set + tag updates are sequential, not pipelined inside a MULTI, so
30
+ * a worst-case interleaving might leave a tag set with a key that no
31
+ * longer exists. The reverse `keyTags` set + `del()` cleanup means the
32
+ * stale entry is reaped on the next `invalidateTag` (the SET membership
33
+ * is filtered against actual key existence — see invalidateTag below).
34
+ */
35
+
36
+ var redisClient = require("./redis-client");
37
+ var { CacheError } = require("./framework-error");
38
+
39
+ var _err = CacheError.factory;
40
+
41
+ function _toStr(v) {
42
+ if (v === null || v === undefined) return null;
43
+ return Buffer.isBuffer(v) ? v.toString("utf8") : String(v);
44
+ }
45
+
46
+ function create(cfg) {
47
+ cfg = cfg || {};
48
+ if (typeof cfg.url !== "string" || cfg.url.length === 0) {
49
+ throw _err("BAD_OPT", "cache-redis: opts.url is required (e.g. redis://localhost:6379/0)");
50
+ }
51
+ var namespace = cfg.namespace;
52
+ var clock = cfg.clock || function () { return Date.now(); };
53
+ var emitObs = cfg.emitObs || function () {};
54
+ var slidingTtl = cfg.slidingTtl;
55
+ var defaultTtlMs = cfg.defaultTtlMs;
56
+
57
+ var client = redisClient.create({
58
+ url: cfg.url,
59
+ password: cfg.password,
60
+ username: cfg.username,
61
+ tls: cfg.tls,
62
+ ca: cfg.ca,
63
+ servername: cfg.servername,
64
+ connectTimeoutMs: cfg.connectTimeoutMs,
65
+ commandTimeoutMs: cfg.commandTimeoutMs,
66
+ maxReconnectAttempts: cfg.maxReconnectAttempts,
67
+ });
68
+
69
+ // Lazy connect — defer until first op so cache.create stays sync-safe.
70
+ var connectPromise = null;
71
+ function _ensureConnected() {
72
+ if (client.isOpen()) return Promise.resolve();
73
+ if (!connectPromise) connectPromise = client.connect();
74
+ return connectPromise;
75
+ }
76
+
77
+ function _key(k) { return namespace + ":e:" + k; }
78
+ function _tagKey(t) { return namespace + ":t:" + t; }
79
+ function _keyTagsKey(k) { return namespace + ":k:" + k + ":tags"; }
80
+
81
+ async function get(key) {
82
+ await _ensureConnected();
83
+ var v = await client.command("GET", _key(key));
84
+ var s = _toStr(v);
85
+ if (s === null) return undefined;
86
+ var parsed;
87
+ try { parsed = JSON.parse(s); }
88
+ catch (_e) { return undefined; }
89
+ // Sliding TTL: extend the entry's life on every read. Best-effort
90
+ // (PEXPIREAT may race with delete; if it fails the operator just
91
+ // sees the original TTL play out).
92
+ if (slidingTtl && typeof defaultTtlMs === "number" && isFinite(defaultTtlMs) && defaultTtlMs > 0) {
93
+ var newExp = clock() + defaultTtlMs;
94
+ client.command("PEXPIREAT", _key(key), String(Math.floor(newExp)))
95
+ .catch(function () { /* best-effort */ });
96
+ client.command("PEXPIREAT", _keyTagsKey(key), String(Math.floor(newExp)))
97
+ .catch(function () { /* best-effort */ });
98
+ }
99
+ return parsed;
100
+ }
101
+
102
+ async function set(key, value, expiresAt, meta) {
103
+ await _ensureConnected();
104
+ var json = JSON.stringify(value);
105
+
106
+ // Drop any prior tag membership for this key (tags may have changed
107
+ // across sets). The reverse-tag set tells us which tag SETs need
108
+ // pruning without scanning every tag in the namespace.
109
+ var oldTagsRv = await client.command("SMEMBERS", _keyTagsKey(key));
110
+ var oldTags = (oldTagsRv || []).map(_toStr).filter(Boolean);
111
+ for (var ot = 0; ot < oldTags.length; ot++) {
112
+ try { await client.command("SREM", _tagKey(oldTags[ot]), key); }
113
+ catch (_e) { /* best-effort */ }
114
+ }
115
+ if (oldTags.length > 0) {
116
+ try { await client.command("DEL", _keyTagsKey(key)); }
117
+ catch (_e) { /* best-effort */ }
118
+ }
119
+
120
+ // Write the value with PEXPIREAT for finite TTLs; un-expiring SET
121
+ // for Infinity (operator opted for "cache forever until evicted").
122
+ if (typeof expiresAt === "number" && isFinite(expiresAt)) {
123
+ await client.command("SET", _key(key), json, "PXAT", String(Math.floor(expiresAt)));
124
+ } else {
125
+ await client.command("SET", _key(key), json);
126
+ }
127
+
128
+ // Wire the new tags. SADD is idempotent on duplicate tag names.
129
+ var tags = meta && Array.isArray(meta.tags) ? meta.tags : null;
130
+ if (tags && tags.length > 0) {
131
+ for (var t = 0; t < tags.length; t++) {
132
+ await client.command("SADD", _tagKey(tags[t]), key);
133
+ await client.command("SADD", _keyTagsKey(key), tags[t]);
134
+ }
135
+ // Match the entry's lifetime on the keyTags reverse-set so
136
+ // membership doesn't outlive the value.
137
+ if (typeof expiresAt === "number" && isFinite(expiresAt)) {
138
+ try { await client.command("PEXPIREAT", _keyTagsKey(key), String(Math.floor(expiresAt))); }
139
+ catch (_e) { /* best-effort */ }
140
+ }
141
+ }
142
+ emitObs("cache.redis.set", { namespace: namespace });
143
+ }
144
+
145
+ async function del(key) {
146
+ await _ensureConnected();
147
+ var oldTagsRv = await client.command("SMEMBERS", _keyTagsKey(key));
148
+ var oldTags = (oldTagsRv || []).map(_toStr).filter(Boolean);
149
+ for (var i = 0; i < oldTags.length; i++) {
150
+ try { await client.command("SREM", _tagKey(oldTags[i]), key); }
151
+ catch (_e) { /* best-effort */ }
152
+ }
153
+ var dels = await Promise.all([
154
+ client.command("DEL", _key(key)),
155
+ client.command("DEL", _keyTagsKey(key)),
156
+ ]);
157
+ return Number(dels[0]) === 1;
158
+ }
159
+
160
+ async function has(key) {
161
+ await _ensureConnected();
162
+ var rv = await client.command("EXISTS", _key(key));
163
+ return Number(rv) === 1;
164
+ }
165
+
166
+ async function clear() {
167
+ await _ensureConnected();
168
+ // SCAN-and-DEL every key under this namespace. Using SCAN avoids
169
+ // KEYS' O(N) blocking pass and lets a busy Redis serve other
170
+ // commands between cursor batches.
171
+ var cursor = "0";
172
+ do {
173
+ var rv = await client.command("SCAN", cursor, "MATCH", namespace + ":*", "COUNT", "200");
174
+ cursor = _toStr(rv[0]) || "0";
175
+ var keys = (rv[1] || []).map(_toStr).filter(Boolean);
176
+ if (keys.length > 0) {
177
+ await client.command.apply(client, ["DEL"].concat(keys));
178
+ }
179
+ } while (cursor !== "0");
180
+ }
181
+
182
+ async function size() {
183
+ await _ensureConnected();
184
+ var cursor = "0";
185
+ var n = 0;
186
+ do {
187
+ var rv = await client.command("SCAN", cursor, "MATCH", namespace + ":e:*", "COUNT", "200");
188
+ cursor = _toStr(rv[0]) || "0";
189
+ n += (rv[1] || []).length;
190
+ } while (cursor !== "0");
191
+ return n;
192
+ }
193
+
194
+ function bytes() {
195
+ // Per-entry byte accounting on Redis would mean a MEMORY USAGE call
196
+ // per key, which is operator-expensive and Redis-version-dependent.
197
+ // Operators wanting this use the memory backend or run their own
198
+ // monitoring against MEMORY STATS at the cluster level.
199
+ return Promise.resolve(0);
200
+ }
201
+
202
+ async function invalidateTag(tag) {
203
+ await _ensureConnected();
204
+ var rv = await client.command("SMEMBERS", _tagKey(tag));
205
+ var keys = (rv || []).map(_toStr).filter(Boolean);
206
+ var dropped = 0;
207
+ for (var i = 0; i < keys.length; i++) {
208
+ // Filter out stale tag-membership entries (key already expired
209
+ // from PEXPIREAT but lingered in the tag SET) by checking
210
+ // EXISTS before del. Saves an unnecessary DEL round-trip on
211
+ // ghost keys.
212
+ var existsRv = await client.command("EXISTS", _key(keys[i]));
213
+ if (Number(existsRv) === 1) {
214
+ await del(keys[i]);
215
+ dropped += 1;
216
+ } else {
217
+ // Ghost — just SREM from this tag set + drop the reverse.
218
+ try { await client.command("SREM", _tagKey(tag), keys[i]); }
219
+ catch (_e) {}
220
+ }
221
+ }
222
+ // Drop the tag set itself if it's now empty (or empty after the
223
+ // ghost cleanup above).
224
+ var remaining = await client.command("SCARD", _tagKey(tag));
225
+ if (Number(remaining) === 0) {
226
+ try { await client.command("DEL", _tagKey(tag)); }
227
+ catch (_e) {}
228
+ }
229
+ emitObs("cache.redis.invalidateTag", { namespace: namespace, tag: tag, dropped: dropped });
230
+ return dropped;
231
+ }
232
+
233
+ async function getTags(key) {
234
+ await _ensureConnected();
235
+ var rv = await client.command("SMEMBERS", _keyTagsKey(key));
236
+ return (rv || []).map(_toStr).filter(Boolean);
237
+ }
238
+
239
+ async function close() {
240
+ try { await client.close(); }
241
+ catch (_e) { /* best-effort */ }
242
+ }
243
+
244
+ return {
245
+ name: "redis",
246
+ get: get,
247
+ set: set,
248
+ del: del,
249
+ has: has,
250
+ clear: clear,
251
+ size: size,
252
+ bytes: bytes,
253
+ invalidateTag: invalidateTag,
254
+ getTags: getTags,
255
+ close: close,
256
+ // Redis enforces TTL itself; the framework's sweeper is a no-op here.
257
+ _startSweep: function () {},
258
+ };
259
+ }
260
+
261
+ module.exports = { create: create };
package/lib/cache.js CHANGED
@@ -184,10 +184,15 @@ function _validateCreateOpts(opts) {
184
184
  }
185
185
  if (opts.backend !== undefined) {
186
186
  if (typeof opts.backend === "string") {
187
- if (opts.backend !== "memory" && opts.backend !== "cluster") {
188
- throw _err("BAD_OPT", "cache.create: backend string must be 'memory' or 'cluster', got " +
187
+ if (opts.backend !== "memory" && opts.backend !== "cluster" && opts.backend !== "redis") {
188
+ throw _err("BAD_OPT", "cache.create: backend string must be 'memory' | 'cluster' | 'redis', got " +
189
189
  JSON.stringify(opts.backend));
190
190
  }
191
+ if (opts.backend === "redis") {
192
+ if (typeof opts.redisUrl !== "string" || opts.redisUrl.length === 0) {
193
+ throw _err("BAD_OPT", "cache.create: backend='redis' requires opts.redisUrl (e.g. redis://localhost:6379/0)");
194
+ }
195
+ }
191
196
  } else {
192
197
  _validateBackendObject(opts.backend);
193
198
  }
@@ -729,6 +734,11 @@ function create(opts) {
729
734
  "sweepIntervalMs", "staleWhileRevalidate", "slidingTtl",
730
735
  "auditFailures", "auditClear",
731
736
  "audit", "observability", "clock",
737
+ // backend === "redis" connection options. Ignored for memory /
738
+ // cluster / custom-backend modes.
739
+ "redisUrl", "redisPassword", "redisUsername", "redisTls", "redisCa",
740
+ "redisServername", "redisConnectTimeoutMs", "redisCommandTimeoutMs",
741
+ "redisMaxReconnectAttempts",
732
742
  ], "cache");
733
743
  _validateCreateOpts(opts);
734
744
 
@@ -792,6 +802,24 @@ function create(opts) {
792
802
  backend = _memoryBackend(cfg);
793
803
  } else if (backendKind === "cluster") {
794
804
  backend = _clusterBackend(cfg);
805
+ } else if (backendKind === "redis") {
806
+ var cacheRedis = require("./cache-redis");
807
+ backend = _customBackend(cacheRedis.create({
808
+ namespace: namespace,
809
+ url: opts.redisUrl,
810
+ password: opts.redisPassword,
811
+ username: opts.redisUsername,
812
+ tls: opts.redisTls,
813
+ ca: opts.redisCa,
814
+ servername: opts.redisServername,
815
+ connectTimeoutMs: opts.redisConnectTimeoutMs,
816
+ commandTimeoutMs: opts.redisCommandTimeoutMs,
817
+ maxReconnectAttempts: opts.redisMaxReconnectAttempts,
818
+ slidingTtl: slidingTtl,
819
+ defaultTtlMs: defaultTtlMs,
820
+ clock: clock,
821
+ emitObs: emitObs,
822
+ }), cfg);
795
823
  } else {
796
824
  backend = _customBackend(opts.backend, cfg);
797
825
  }
@@ -122,32 +122,42 @@ function create(config) {
122
122
  flushTimer.unref();
123
123
  }
124
124
 
125
+ // Track the in-flight flush as a promise so close() can await it.
126
+ // Without this, a record arriving mid-flush gets buffered, the
127
+ // emit-time _flush() early-returns on `if (inFlight) return`, and
128
+ // the buffered record is stranded if shutdown fires before
129
+ // _scheduleFlush() drains it.
130
+ var inFlightPromise = null;
125
131
  async function _flush() {
126
- if (inFlight) return;
132
+ if (inFlight) return inFlightPromise;
127
133
  if (buffer.length === 0) return;
128
134
  inFlight = true;
129
- try {
130
- while (buffer.length > 0 && !closed) {
131
- var batch = buffer.splice(0, cfg.batchSize);
132
- var body = _serializeBatch(batch, cfg.bodyShape);
133
- try {
134
- await retryHelper.withRetry(function () {
135
- return _post(cfg.url, body, headers, cfg.timeoutMs, cfg.allowedProtocols, cfg.allowInternal);
136
- }, cfg.retry);
137
- } catch (e) {
138
- // Batch permanently rejected — surface via dropCount AND the
139
- // operator-supplied onDrop callback. The dispatcher path
140
- // wraps its own audit hook around emit(); operators using
141
- // this sink directly rely on dropCount + onDrop.
142
- dropCount += batch.length;
143
- _emitDrop("retry-exhausted", batch, e);
144
- break;
135
+ inFlightPromise = (async function () {
136
+ try {
137
+ while (buffer.length > 0 && !closed) {
138
+ var batch = buffer.splice(0, cfg.batchSize);
139
+ var body = _serializeBatch(batch, cfg.bodyShape);
140
+ try {
141
+ await retryHelper.withRetry(function () {
142
+ return _post(cfg.url, body, headers, cfg.timeoutMs, cfg.allowedProtocols, cfg.allowInternal);
143
+ }, cfg.retry);
144
+ } catch (e) {
145
+ // Batch permanently rejected surface via dropCount AND the
146
+ // operator-supplied onDrop callback. The dispatcher path
147
+ // wraps its own audit hook around emit(); operators using
148
+ // this sink directly rely on dropCount + onDrop.
149
+ dropCount += batch.length;
150
+ _emitDrop("retry-exhausted", batch, e);
151
+ break;
152
+ }
145
153
  }
154
+ } finally {
155
+ inFlight = false;
156
+ inFlightPromise = null;
157
+ if (buffer.length > 0) _scheduleFlush();
146
158
  }
147
- } finally {
148
- inFlight = false;
149
- if (buffer.length > 0) _scheduleFlush();
150
- }
159
+ })();
160
+ return inFlightPromise;
151
161
  }
152
162
 
153
163
  function emit(record) {
@@ -171,9 +181,13 @@ function create(config) {
171
181
  // Drain BEFORE flipping closed=true. _flush()'s while loop bails on
172
182
  // !closed, so flipping the flag first leaves any buffered records
173
183
  // stranded — the very records the operator queued just before
174
- // calling shutdown(). Order: stop the timer, drain, then refuse
175
- // any new enqueues by setting closed.
184
+ // calling shutdown(). Order: stop the timer, await any in-flight
185
+ // flush so its records POST before we touch the buffer, drain
186
+ // anything still queued, THEN refuse new enqueues.
176
187
  if (flushTimer) { clearTimeout(flushTimer); flushTimer = null; }
188
+ if (inFlightPromise) {
189
+ try { await inFlightPromise; } catch (_e) { /* surfaced via onDrop */ }
190
+ }
177
191
  await _flush();
178
192
  closed = true;
179
193
  }
package/lib/mtls-ca.js CHANGED
@@ -129,6 +129,15 @@ function create(opts) {
129
129
  throw new MtlsCaError("mtls-ca/no-datadir",
130
130
  "mtlsCa.create requires opts.dataDir");
131
131
  }
132
+ // Auto-create the dataDir with restrictive perms (CA keys live here).
133
+ // Matches the behaviour of other framework primitives that own a
134
+ // dataDir — log-stream-local, backup, restore-bundle. Without this
135
+ // the first initCA() / generateClientCert() call fails with ENOENT
136
+ // on `ca.key.tmp` because the atomic-file write expects the parent
137
+ // dir to exist.
138
+ if (!fs.existsSync(opts.dataDir)) {
139
+ fs.mkdirSync(opts.dataDir, { recursive: true, mode: 0o700 });
140
+ }
132
141
  var paths = _resolvePaths(opts.dataDir, opts.paths);
133
142
  var vault = opts.vault || null;
134
143
  var caKeySealedMode = (opts.caKeySealedMode || "auto").toLowerCase();
@@ -29,12 +29,6 @@
29
29
  * Out of scope (defer to follow-up patches):
30
30
  * - Redis Cluster (slot-routing across multi-node Redis)
31
31
  * - Sentinel (managed primary failover)
32
- * - Job priority (queue-local supports `priority` opt; Redis backend
33
- * orders strictly by availableAt for v1 — re-introduce when a real
34
- * operator demand surfaces with a clean Lua-side ordering scheme)
35
- * - Flow children with dependsOn (queue-local's _maybeReleaseFlowChildren
36
- * coordination — orthogonal to backend choice; ships when flow primitive
37
- * itself becomes backend-agnostic)
38
32
  */
39
33
  var C = require("./constants");
40
34
  var cryptoField = require("./crypto-field");
@@ -57,9 +51,23 @@ var DEFAULT_PREFIX = "blamejs:queue";
57
51
  // ---- Lua scripts ----
58
52
  //
59
53
  // LEASE_LUA — atomically pull up to maxRows jobs from the ready zset
60
- // whose score (availableAt) is <= nowMs, move them to the inflight
61
- // zset with score = leaseExpiresAt, increment attempts, flip status,
62
- // and return the jobIds. The JS side then HGETALLs each id.
54
+ // whose availableAt is <= nowMs, move them to the inflight zset with
55
+ // score = leaseExpiresAt, increment attempts, flip status, and return
56
+ // the jobIds in priority order. The JS side then HGETALLs each id.
57
+ //
58
+ // Priority semantics — matches queue-local's `ORDER BY priority DESC,
59
+ // availableAt ASC, enqueuedAt ASC` over the same `WHERE availableAt
60
+ // <= nowMs` filter. The Redis ZSET orders by availableAt only (the
61
+ // score), which is enough to filter "ready vs not-yet-ready" via
62
+ // ZRANGEBYSCORE 0..nowMs but NOT enough to surface a high-priority
63
+ // job ahead of an earlier-availableAt low-priority job. So we
64
+ // over-fetch maxRows*5 candidates by score, HMGET the priority +
65
+ // availableAt for each, sort priority-DESC / availableAt-ASC server-
66
+ // side in Lua, and lease the top maxRows. The over-fetch factor is
67
+ // chosen so a queue with a typical priority distribution surfaces
68
+ // the right jobs without scanning the entire ready set; queues with
69
+ // >5x as many priority-0 jobs as priority-N jobs would still pull
70
+ // the top-priority ones first.
63
71
  //
64
72
  // KEYS[1] = ready zset
65
73
  // KEYS[2] = inflight zset
@@ -74,10 +82,29 @@ var LEASE_LUA = [
74
82
  'local leaseExpiresAt = tonumber(ARGV[2])',
75
83
  'local maxRows = tonumber(ARGV[3])',
76
84
  'local jobKeyPrefix = ARGV[4]',
77
- 'local jobIds = redis.call("ZRANGEBYSCORE", readyKey, 0, nowMs, "LIMIT", 0, maxRows)',
85
+ 'local oversample = maxRows * 5',
86
+ 'local jobIds = redis.call("ZRANGEBYSCORE", readyKey, 0, nowMs, "LIMIT", 0, oversample)',
78
87
  'if #jobIds == 0 then return {} end',
88
+ // Pull priority + availableAt + enqueuedAt for each candidate so the
89
+ // sort uses the same triple queue-local sorts on. enqueuedAt is the
90
+ // tiebreaker among same-priority same-availableAt jobs (FIFO within
91
+ // a priority lane).
92
+ 'local rows = {}',
79
93
  'for i = 1, #jobIds do',
80
94
  ' local jobId = jobIds[i]',
95
+ ' local h = redis.call("HMGET", jobKeyPrefix..jobId, "priority", "availableAt", "enqueuedAt")',
96
+ ' rows[i] = { jobId, tonumber(h[1] or "0") or 0, tonumber(h[2] or "0") or 0, tonumber(h[3] or "0") or 0 }',
97
+ 'end',
98
+ // priority DESC, availableAt ASC, enqueuedAt ASC.
99
+ 'table.sort(rows, function(a, b)',
100
+ ' if a[2] ~= b[2] then return a[2] > b[2] end',
101
+ ' if a[3] ~= b[3] then return a[3] < b[3] end',
102
+ ' return a[4] < b[4]',
103
+ 'end)',
104
+ 'local picked = {}',
105
+ 'local n = math.min(maxRows, #rows)',
106
+ 'for i = 1, n do',
107
+ ' local jobId = rows[i][1]',
81
108
  ' redis.call("ZREM", readyKey, jobId)',
82
109
  ' redis.call("ZADD", inflightKey, leaseExpiresAt, jobId)',
83
110
  ' redis.call("HINCRBY", jobKeyPrefix..jobId, "attempts", 1)',
@@ -85,8 +112,9 @@ var LEASE_LUA = [
85
112
  ' "status", "inflight",',
86
113
  ' "leasedAt", nowMs,',
87
114
  ' "leaseExpiresAt", leaseExpiresAt)',
115
+ ' picked[i] = jobId',
88
116
  'end',
89
- 'return jobIds',
117
+ 'return picked',
90
118
  ].join("\n");
91
119
 
92
120
  // SWEEP_LUA — find jobs in inflight whose lease expired, push back to
@@ -251,6 +279,7 @@ function create(opts) {
251
279
  function _dlqKey(queueName) { return prefix + ":q:" + queueName + ":dlq"; }
252
280
  function _queuesKey() { return prefix + ":queues"; }
253
281
  function _jobKeyPrefix() { return prefix + ":job:"; }
282
+ function _flowKey(flowId) { return prefix + ":flow:" + flowId; }
254
283
 
255
284
  // ---- Row encoding ----
256
285
  //
@@ -348,13 +377,22 @@ function create(opts) {
348
377
  };
349
378
  var sealed = cryptoField.sealRow("_blamejs_jobs", row);
350
379
 
351
- // Pipeline: HSET job + ZADD ready + SADD queues. Pipelined writes
352
- // hit Redis without round-trips between them.
380
+ // Pipeline: HSET job + ZADD ready + SADD queues + (if flowId)
381
+ // SADD flow set. Pipelined writes hit Redis without round-trips
382
+ // between them. The flow set is the per-flow registry that
383
+ // complete() walks to release dependents — it lets us avoid a
384
+ // SCAN over every job hash when releasing children, matching the
385
+ // queue-local pattern of "SELECT siblings WHERE flowId = ?".
353
386
  var hsetArgs = _hsetArgs(jobId, sealed);
354
- var p1 = client.command.apply(null, hsetArgs);
355
- var p2 = client.command("ZADD", _readyKey(queueName), String(availableAt), jobId);
356
- var p3 = client.command("SADD", _queuesKey(), queueName);
357
- await Promise.all([p1, p2, p3]);
387
+ var pipeline = [
388
+ client.command.apply(null, hsetArgs),
389
+ client.command("ZADD", _readyKey(queueName), String(availableAt), jobId),
390
+ client.command("SADD", _queuesKey(), queueName),
391
+ ];
392
+ if (row.flowId) {
393
+ pipeline.push(client.command("SADD", _flowKey(row.flowId), jobId));
394
+ }
395
+ await Promise.all(pipeline);
358
396
 
359
397
  return {
360
398
  jobId: jobId,
@@ -450,9 +488,104 @@ function create(opts) {
450
488
  });
451
489
  } catch (_e) { /* best-effort — cron resumes next tick if op fixes the issue */ }
452
490
  }
491
+
492
+ // Flow propagation: walk siblings whose dependsOn includes this
493
+ // jobId (or this job's flowChildName) and bump availableAt to now
494
+ // if ALL their deps are now complete. Mirrors queue-local's
495
+ // _maybeReleaseFlowChildren, but uses a per-flow Redis SET as
496
+ // the sibling registry instead of a SQL SELECT.
497
+ if (raw.flowId) {
498
+ try {
499
+ await _maybeReleaseFlowChildren(raw.flowId, jobId, raw.flowChildName || null, nowMs);
500
+ } catch (_e) { /* best-effort — sweepExpired retries if a deps check fails */ }
501
+ // The completed job stays in the flow set so a LATER sibling
502
+ // whose dependsOn includes this job's flowChildName can still
503
+ // find a "done" sibling when its own complete() walks the flow.
504
+ // The set is purge()'d when the queue is purged or — for the
505
+ // typical case where the operator wants to reclaim flow memory
506
+ // after the whole flow finishes — by an explicit operator call
507
+ // to purge() OR by letting `_maybeReleaseFlowChildren` reach a
508
+ // state where every sibling has status='done' (which the
509
+ // operator can detect via b.queue.dlqList / app-level queries).
510
+ }
453
511
  return true;
454
512
  }
455
513
 
514
+ async function _maybeReleaseFlowChildren(flowId, completedJobId, completedChildName, nowMs) {
515
+ var flowKey = _flowKey(flowId);
516
+ var members = await client.command("SMEMBERS", flowKey);
517
+ if (!members || members.length === 0) return;
518
+
519
+ // Pull dependsOn + status + flowChildName for every sibling in one
520
+ // pipelined batch — far cheaper than per-sibling HMGET round trips.
521
+ var siblingIds = members.map(function (m) {
522
+ return Buffer.isBuffer(m) ? m.toString("utf8") : String(m);
523
+ }).filter(function (id) { return id !== completedJobId; });
524
+ if (siblingIds.length === 0) return;
525
+
526
+ var hmgetCalls = siblingIds.map(function (sibId) {
527
+ return client.command("HMGET", _jobKey(sibId),
528
+ "dependsOn", "status", "flowChildName");
529
+ });
530
+ var results = await Promise.all(hmgetCalls);
531
+
532
+ // For each pending sibling with a dependsOn array, check if all
533
+ // deps are satisfied (the just-completed job AND any prior
534
+ // already-done sibling in the flow).
535
+ for (var i = 0; i < siblingIds.length; i++) {
536
+ var sibId = siblingIds[i];
537
+ var rv = results[i];
538
+ if (!rv || rv.length < 3) continue;
539
+ var rawDeps = rv[0] && (Buffer.isBuffer(rv[0]) ? rv[0].toString("utf8") : String(rv[0]));
540
+ var status = rv[1] && (Buffer.isBuffer(rv[1]) ? rv[1].toString("utf8") : String(rv[1]));
541
+ if (!rawDeps || status !== "pending") continue;
542
+ var deps;
543
+ try { deps = JSON.parse(rawDeps); }
544
+ catch (_e) { continue; }
545
+ if (!Array.isArray(deps) || deps.length === 0) continue;
546
+
547
+ // Resolve each dep against the just-completed job (id or child
548
+ // name) OR a prior-completed sibling in the flow.
549
+ var allDone = true;
550
+ for (var d = 0; d < deps.length; d++) {
551
+ var dep = deps[d];
552
+ if (dep === completedJobId) continue;
553
+ if (completedChildName && dep === completedChildName) continue;
554
+ // Look up by id first; if no hit, fall back to scanning the
555
+ // flow set for a sibling whose flowChildName matches.
556
+ var depHash = await client.command("HMGET", _jobKey(dep), "status", "flowId");
557
+ if (depHash && depHash[0]) {
558
+ var depStatus = Buffer.isBuffer(depHash[0]) ? depHash[0].toString("utf8") : String(depHash[0]);
559
+ var depFlow = depHash[1] ? (Buffer.isBuffer(depHash[1]) ? depHash[1].toString("utf8") : String(depHash[1])) : "";
560
+ if (depStatus === "done" && depFlow === flowId) continue;
561
+ }
562
+ // Scan the flow's set for a child-name match — cheap because
563
+ // a flow typically has 5-50 children, not thousands.
564
+ var matched = false;
565
+ for (var s = 0; s < siblingIds.length && !matched; s++) {
566
+ if (siblingIds[s] === sibId) continue;
567
+ var sRv = results[s];
568
+ if (!sRv || !sRv[2]) continue;
569
+ var sName = Buffer.isBuffer(sRv[2]) ? sRv[2].toString("utf8") : String(sRv[2]);
570
+ var sStatus = sRv[1] ? (Buffer.isBuffer(sRv[1]) ? sRv[1].toString("utf8") : String(sRv[1])) : "";
571
+ if (sName === dep && sStatus === "done") matched = true;
572
+ }
573
+ if (!matched) { allDone = false; break; }
574
+ }
575
+
576
+ if (allDone) {
577
+ // Read queueName so the ZADD targets the right ready zset.
578
+ var qBuf = await client.command("HGET", _jobKey(sibId), "queueName");
579
+ if (!qBuf) continue;
580
+ var queueName = Buffer.isBuffer(qBuf) ? qBuf.toString("utf8") : String(qBuf);
581
+ await Promise.all([
582
+ client.command("HSET", _jobKey(sibId), "availableAt", String(nowMs)),
583
+ client.command("ZADD", _readyKey(queueName), String(nowMs), sibId),
584
+ ]);
585
+ }
586
+ }
587
+ }
588
+
456
589
  async function fail(jobId, errorMessage, retryDelayMs) {
457
590
  await _ensureConnected();
458
591
  var nowMs = Date.now();
@@ -503,8 +636,11 @@ function create(opts) {
503
636
 
504
637
  async function purge(queueName) {
505
638
  await _ensureConnected();
506
- // Walk the ready + inflight + dlq zsets, delete the per-job
507
- // hashes, then drop the zsets and the queues-set membership.
639
+ // Walk the ready + inflight + dlq zsets, collect every job id,
640
+ // also clear each job from its flow set (if any), then DEL the
641
+ // job hashes + zsets + queues-membership. Flow sets are cleared
642
+ // job-by-job because a single flow can span multiple queues, and
643
+ // we only want to evict THIS queue's contributors.
508
644
  var [readyMembers, inflightMembers, dlqMembers] = await Promise.all([
509
645
  client.command("ZRANGE", _readyKey(queueName), "0", "-1"),
510
646
  client.command("ZRANGE", _inflightKey(queueName), "0", "-1"),
@@ -512,6 +648,18 @@ function create(opts) {
512
648
  ]);
513
649
  var allIds = [].concat(readyMembers || [], inflightMembers || [], dlqMembers || [])
514
650
  .map(function (b) { return Buffer.isBuffer(b) ? b.toString("utf8") : String(b); });
651
+ // Pull the flowId for each job (if set) so we can SREM from the
652
+ // matching flow set BEFORE we DEL the job hash.
653
+ var flowIdLookups = await Promise.all(allIds.map(function (id) {
654
+ return client.command("HGET", _jobKey(id), "flowId");
655
+ }));
656
+ var flowSrems = [];
657
+ for (var fi = 0; fi < allIds.length; fi++) {
658
+ var fIdBuf = flowIdLookups[fi];
659
+ if (!fIdBuf) continue;
660
+ var fId = Buffer.isBuffer(fIdBuf) ? fIdBuf.toString("utf8") : String(fIdBuf);
661
+ if (fId) flowSrems.push(client.command("SREM", _flowKey(fId), allIds[fi]));
662
+ }
515
663
  var dels = allIds.map(function (id) { return client.command("DEL", _jobKey(id)); });
516
664
  var zdrops = [
517
665
  client.command("DEL", _readyKey(queueName)),
@@ -519,7 +667,7 @@ function create(opts) {
519
667
  client.command("DEL", _dlqKey(queueName)),
520
668
  client.command("SREM", _queuesKey(), queueName),
521
669
  ];
522
- await Promise.all(dels.concat(zdrops));
670
+ await Promise.all(flowSrems.concat(dels, zdrops));
523
671
  return allIds.length;
524
672
  }
525
673
 
package/lib/websocket.js CHANGED
@@ -82,6 +82,7 @@
82
82
  */
83
83
 
84
84
  var nodeCrypto = require("crypto");
85
+ var zlib = require("zlib");
85
86
  var { EventEmitter } = require("events");
86
87
  var C = require("./constants");
87
88
  var requestHelpers = require("./request-helpers");
@@ -220,7 +221,7 @@ function isOriginAllowed(req, origins) {
220
221
  return false;
221
222
  }
222
223
 
223
- function buildUpgradeResponse(secWebSocketKey, subprotocol) {
224
+ function buildUpgradeResponse(secWebSocketKey, subprotocol, extensionHeader) {
224
225
  var lines = [
225
226
  "HTTP/1.1 101 Switching Protocols",
226
227
  "Upgrade: websocket",
@@ -228,9 +229,107 @@ function buildUpgradeResponse(secWebSocketKey, subprotocol) {
228
229
  "Sec-WebSocket-Accept: " + computeAcceptKey(secWebSocketKey),
229
230
  ];
230
231
  if (subprotocol) lines.push("Sec-WebSocket-Protocol: " + subprotocol);
232
+ if (extensionHeader) lines.push("Sec-WebSocket-Extensions: " + extensionHeader);
231
233
  return lines.join("\r\n") + "\r\n\r\n";
232
234
  }
233
235
 
236
+ // ---- permessage-deflate (RFC 7692) ----
237
+ //
238
+ // Negotiate compression at handshake, compress per-message on send,
239
+ // decompress per-message on receive. The framework runs in
240
+ // "no_context_takeover" mode in both directions — every message uses a
241
+ // fresh zlib state, no LZ77 history carried across messages. This
242
+ // trade-off makes message processing stateless (no per-connection
243
+ // zlib stream lifetime to manage) at a small compression-ratio cost.
244
+ // Operators with throughput-sensitive workloads can extend this later
245
+ // to keep state across messages.
246
+ //
247
+ // Per RFC 7692 §7.2.1 the deflate output is the standard zlib raw
248
+ // deflate WITH the trailing 4 bytes 0x00 0x00 0xff 0xff stripped. The
249
+ // matching inflate path appends them back before inflating.
250
+ var DEFLATE_TRAILING = Buffer.from([0x00, 0x00, 0xff, 0xff]);
251
+
252
+ function _parseExtensionHeader(header) {
253
+ // Sec-WebSocket-Extensions: foo; param=val; param2, bar; ...
254
+ // Returns [{ name, params: { paramName: value | true } }]
255
+ if (!header) return [];
256
+ var entries = String(header).split(",");
257
+ var out = [];
258
+ for (var i = 0; i < entries.length; i++) {
259
+ var parts = entries[i].split(";").map(function (s) { return s.trim(); });
260
+ if (!parts[0]) continue;
261
+ var ext = { name: parts[0].toLowerCase(), params: {} };
262
+ for (var j = 1; j < parts.length; j++) {
263
+ var kv = parts[j].split("=");
264
+ var k = kv[0].trim().toLowerCase();
265
+ if (!k) continue;
266
+ var v = kv.length > 1 ? kv.slice(1).join("=").trim() : true;
267
+ // Strip surrounding quotes per the token-or-quoted-string grammar.
268
+ if (typeof v === "string" && v.length >= 2 &&
269
+ v.charAt(0) === '"' && v.charAt(v.length - 1) === '"') {
270
+ v = v.slice(1, -1);
271
+ }
272
+ ext.params[k] = v;
273
+ }
274
+ out.push(ext);
275
+ }
276
+ return out;
277
+ }
278
+
279
+ function _negotiatePermessageDeflate(reqHeader) {
280
+ var entries = _parseExtensionHeader(reqHeader);
281
+ for (var i = 0; i < entries.length; i++) {
282
+ if (entries[i].name !== "permessage-deflate") continue;
283
+ var p = entries[i].params;
284
+ // Reject unknown params (RFC 7692 §7 lists exactly four).
285
+ var KNOWN = {
286
+ "server_no_context_takeover": true, "client_no_context_takeover": true,
287
+ "server_max_window_bits": true, "client_max_window_bits": true,
288
+ };
289
+ var ok = true;
290
+ for (var k in p) { if (Object.prototype.hasOwnProperty.call(p, k) && !KNOWN[k]) { ok = false; break; } }
291
+ if (!ok) continue;
292
+ // Always negotiate WITH no_context_takeover in BOTH directions, so
293
+ // every message uses a fresh zlib state. Echo any client window-
294
+ // bits constraints back unchanged (we honour them on the server's
295
+ // outgoing compression).
296
+ var responseParams = ["client_no_context_takeover", "server_no_context_takeover"];
297
+ if (p.client_max_window_bits && p.client_max_window_bits !== true) {
298
+ responseParams.push("client_max_window_bits=" + p.client_max_window_bits);
299
+ }
300
+ if (p.server_max_window_bits && p.server_max_window_bits !== true) {
301
+ responseParams.push("server_max_window_bits=" + p.server_max_window_bits);
302
+ }
303
+ return {
304
+ negotiated: true,
305
+ responseHeader: "permessage-deflate; " + responseParams.join("; "),
306
+ // window-bits constraints we honour; default 15 (max) when unset.
307
+ serverMaxWindowBits: p.server_max_window_bits && p.server_max_window_bits !== true
308
+ ? Math.max(8, Math.min(15, parseInt(p.server_max_window_bits, 10) || 15)) : 15,
309
+ clientMaxWindowBits: p.client_max_window_bits && p.client_max_window_bits !== true
310
+ ? Math.max(8, Math.min(15, parseInt(p.client_max_window_bits, 10) || 15)) : 15,
311
+ };
312
+ }
313
+ return { negotiated: false };
314
+ }
315
+
316
+ function _deflateMessage(payload, windowBits) {
317
+ // Per RFC 7692 §7.2.1, strip the 4-byte 0x00 0x00 0xff 0xff trailer.
318
+ var raw = zlib.deflateRawSync(payload, { windowBits: windowBits, level: zlib.constants.Z_DEFAULT_COMPRESSION });
319
+ if (raw.length >= 4 &&
320
+ raw[raw.length - 4] === 0x00 && raw[raw.length - 3] === 0x00 &&
321
+ raw[raw.length - 2] === 0xff && raw[raw.length - 1] === 0xff) {
322
+ return raw.slice(0, raw.length - 4);
323
+ }
324
+ return raw;
325
+ }
326
+
327
+ function _inflateMessage(payload, windowBits) {
328
+ // Per RFC 7692 §7.2.2, append the 4-byte trailer before inflating.
329
+ var withTrailer = Buffer.concat([payload, DEFLATE_TRAILING]);
330
+ return zlib.inflateRawSync(withTrailer, { windowBits: windowBits });
331
+ }
332
+
234
333
  // ---- Frame parser ----
235
334
  //
236
335
  // Incremental — push(chunk) accepts arbitrary buffer slices from the
@@ -339,6 +438,11 @@ function serializeFrame(opcode, payload, opts) {
339
438
  opts = opts || {};
340
439
  var fin = opts.fin !== false;
341
440
  var mask = opts.mask === true; // server-side defaults false
441
+ // RSV1 — set on the first frame of a permessage-deflate-compressed
442
+ // message (RFC 7692). Caller passes opts.rsv1 = true; we wire it
443
+ // into the header byte. RSV2 / RSV3 stay zero (no other extensions
444
+ // negotiated).
445
+ var rsv1 = opts.rsv1 === true;
342
446
  payload = payload || Buffer.alloc(0);
343
447
  if (typeof payload === "string") payload = Buffer.from(payload, "utf8");
344
448
  if (!Buffer.isBuffer(payload)) {
@@ -355,7 +459,7 @@ function serializeFrame(opcode, payload, opts) {
355
459
  if (mask) headerLen += 4;
356
460
 
357
461
  var header = Buffer.alloc(headerLen);
358
- header[0] = (fin ? 0x80 : 0) | (opcode & 0x0F);
462
+ header[0] = (fin ? 0x80 : 0) | (rsv1 ? 0x40 : 0) | (opcode & 0x0F);
359
463
  header[1] = (mask ? 0x80 : 0) | lenByte;
360
464
 
361
465
  var off = 2;
@@ -396,6 +500,10 @@ class WebSocketConnection extends EventEmitter {
396
500
  // exists to protect against in h1 (proxy
397
501
  // cache-poisoning via raw text on the wire).
398
502
  this.transport = opts.transport === "h2" ? "h2" : "h1";
503
+ // permessage-deflate state — `null` means extension not negotiated.
504
+ // When negotiated the object carries serverMaxWindowBits +
505
+ // clientMaxWindowBits the inflate/deflate paths use per message.
506
+ this._permessageDeflate = opts.permessageDeflate || null;
399
507
  var pingMs = opts.pingIntervalMs || DEFAULT_PING_INTERVAL_MS;
400
508
  var pongMs = opts.pongTimeoutMs || DEFAULT_PONG_TIMEOUT_MS;
401
509
  // Grace period after we send a close frame before forcing the
@@ -488,11 +596,20 @@ class WebSocketConnection extends EventEmitter {
488
596
  return this._abort(CLOSE_PROTOCOL_ERROR, "frame must not be masked (h2)");
489
597
  }
490
598
  // Reserved bits — must be zero unless a negotiated extension uses them.
491
- // We don't negotiate any extensions today (compression deferred), so
492
- // any RSV bit set is a protocol error.
493
- if (frame.rsv1 || frame.rsv2 || frame.rsv3) {
599
+ // RSV1 is permessage-deflate (RFC 7692). RSV2/RSV3 unused; any RSV2
600
+ // or RSV3 bit set, OR RSV1 set when permessage-deflate wasn't
601
+ // negotiated, is a protocol error.
602
+ if (frame.rsv2 || frame.rsv3) {
494
603
  return this._abort(CLOSE_PROTOCOL_ERROR, "reserved bits set without extension");
495
604
  }
605
+ if (frame.rsv1 && !this._permessageDeflate) {
606
+ return this._abort(CLOSE_PROTOCOL_ERROR, "RSV1 set without permessage-deflate negotiated");
607
+ }
608
+ // RSV1 is only legal on the FIRST frame of a message (TEXT/BINARY).
609
+ // Continuation frames inherit the compression flag from the start.
610
+ if (frame.rsv1 && frame.opcode === OPCODE_CONTINUATION) {
611
+ return this._abort(CLOSE_PROTOCOL_ERROR, "RSV1 on continuation frame (must be on start)");
612
+ }
496
613
 
497
614
  if (frame.opcode === OPCODE_CONTINUATION) {
498
615
  if (this._fragOpcode === null) {
@@ -506,6 +623,7 @@ class WebSocketConnection extends EventEmitter {
506
623
  this._fragOpcode = frame.opcode;
507
624
  this._fragChunks = [frame.payload];
508
625
  this._fragLen = frame.payload.length;
626
+ this._fragCompressed = !!frame.rsv1;
509
627
  if (frame.fin) this._emitMessage();
510
628
  } else if (frame.opcode === OPCODE_CLOSE) {
511
629
  this._handleClose(frame);
@@ -535,9 +653,26 @@ class WebSocketConnection extends EventEmitter {
535
653
  ? this._fragChunks[0]
536
654
  : Buffer.concat(this._fragChunks, this._fragLen);
537
655
  var opcode = this._fragOpcode;
656
+ var wasCompressed = this._fragCompressed;
538
657
  this._fragOpcode = null;
539
658
  this._fragChunks = null;
540
659
  this._fragLen = 0;
660
+ this._fragCompressed = false;
661
+ // Decompress before emitting if the start frame had RSV1 set.
662
+ // RFC 7692: malformed deflate is a protocol error, surfaced as
663
+ // CLOSE_INVALID_PAYLOAD per §5.6 / §6 of RFC 6455.
664
+ if (wasCompressed) {
665
+ try {
666
+ data = _inflateMessage(data, this._permessageDeflate.clientMaxWindowBits);
667
+ } catch (e) {
668
+ return this._abort(CLOSE_INVALID_PAYLOAD,
669
+ "permessage-deflate inflate failed: " + ((e && e.message) || String(e)));
670
+ }
671
+ if (data.length > this.maxMessageBytes) {
672
+ return this._abort(CLOSE_MESSAGE_TOO_BIG,
673
+ "decompressed message exceeds maxMessageBytes");
674
+ }
675
+ }
541
676
  if (opcode === OPCODE_TEXT) {
542
677
  // §5.6: text frames MUST be valid UTF-8. Buffer.toString silently
543
678
  // replaces invalid sequences with U+FFFD; explicit validation
@@ -581,7 +716,7 @@ class WebSocketConnection extends EventEmitter {
581
716
  this._sendFrame(OPCODE_CLOSE, payload);
582
717
  }
583
718
 
584
- _sendFrame(opcode, payload) {
719
+ _sendFrame(opcode, payload, opts) {
585
720
  if (this._state === STATE_CLOSED) return;
586
721
  // Socket may have been destroyed by the peer between our last
587
722
  // 'close' event check and this write — Node's 'close' event is
@@ -593,28 +728,48 @@ class WebSocketConnection extends EventEmitter {
593
728
  return;
594
729
  }
595
730
  try {
596
- this.socket.write(serializeFrame(opcode, payload));
731
+ this.socket.write(serializeFrame(opcode, payload, opts));
597
732
  } catch (err) {
598
733
  this._transitionToClosed(1006, (err && err.message) || "write failed", false, err);
599
734
  }
600
735
  }
601
736
 
737
+ _sendDataFrame(opcode, payload) {
738
+ // Compress entire-message-in-one-frame when permessage-deflate
739
+ // negotiated. RSV1 set on the FIRST frame of the message to mark
740
+ // it compressed; opcode-only continuation frames don't repeat
741
+ // RSV1 (see _onFrame's RSV1+continuation guard).
742
+ if (this._permessageDeflate && opcode !== OPCODE_PING &&
743
+ opcode !== OPCODE_PONG && opcode !== OPCODE_CLOSE) {
744
+ try {
745
+ var compressed = _deflateMessage(payload, this._permessageDeflate.serverMaxWindowBits);
746
+ this._sendFrame(opcode, compressed, { rsv1: true });
747
+ return;
748
+ } catch (_e) {
749
+ // Compression failure on send — fall through to uncompressed
750
+ // (we still have the original payload) so the connection
751
+ // keeps working. The underlying issue surfaces as observability.
752
+ }
753
+ }
754
+ this._sendFrame(opcode, payload);
755
+ }
756
+
602
757
  send(data) {
603
758
  if (this._state !== STATE_OPEN) {
604
759
  throw new WebSocketError("ws/closed",
605
760
  "connection is " + this._state + ", cannot send");
606
761
  }
607
762
  if (typeof data === "string") {
608
- this._sendFrame(OPCODE_TEXT, Buffer.from(data, "utf8"));
763
+ this._sendDataFrame(OPCODE_TEXT, Buffer.from(data, "utf8"));
609
764
  } else if (Buffer.isBuffer(data)) {
610
- this._sendFrame(OPCODE_BINARY, data);
765
+ this._sendDataFrame(OPCODE_BINARY, data);
611
766
  } else {
612
767
  data = safeBuffer.toBuffer(data, {
613
768
  errorClass: WebSocketError,
614
769
  typeCode: "ws/invalid-payload",
615
770
  typeMessage: "send() requires Buffer, Uint8Array, or string",
616
771
  });
617
- this._sendFrame(OPCODE_BINARY, data);
772
+ this._sendDataFrame(OPCODE_BINARY, data);
618
773
  }
619
774
  }
620
775
 
@@ -687,9 +842,20 @@ function handleUpgrade(req, socket, head, opts) {
687
842
  // Subprotocol negotiation.
688
843
  var subprotocol = negotiateSubprotocol(req, opts.subprotocols);
689
844
 
845
+ // permessage-deflate negotiation. Skipped (no echo header, no
846
+ // compression state on the connection) when the operator passes
847
+ // opts.permessageDeflate = false OR when the client didn't offer it.
848
+ var pmd = null;
849
+ if (opts.permessageDeflate !== false) {
850
+ var negotiated = _negotiatePermessageDeflate(req.headers["sec-websocket-extensions"]);
851
+ if (negotiated.negotiated) pmd = negotiated;
852
+ }
853
+
690
854
  // Send 101.
691
855
  try {
692
- socket.write(buildUpgradeResponse(req.headers["sec-websocket-key"], subprotocol));
856
+ socket.write(buildUpgradeResponse(
857
+ req.headers["sec-websocket-key"], subprotocol,
858
+ pmd ? pmd.responseHeader : null));
693
859
  } catch (err) {
694
860
  log.error("failed to write upgrade response: " + err.message);
695
861
  try { socket.destroy(); } catch (_e) {}
@@ -701,10 +867,11 @@ function handleUpgrade(req, socket, head, opts) {
701
867
  // parser via a synthetic data event. Most clients don't send
702
868
  // anything before the 101 response, but the spec allows it.
703
869
  var conn = new WebSocketConnection(socket, {
704
- subprotocol: subprotocol,
705
- maxMessageBytes: opts.maxMessageBytes,
706
- pingIntervalMs: opts.pingIntervalMs,
707
- pongTimeoutMs: opts.pongTimeoutMs,
870
+ subprotocol: subprotocol,
871
+ maxMessageBytes: opts.maxMessageBytes,
872
+ pingIntervalMs: opts.pingIntervalMs,
873
+ pongTimeoutMs: opts.pongTimeoutMs,
874
+ permessageDeflate: pmd,
708
875
  });
709
876
  if (head && head.length > 0) {
710
877
  // Manually invoke the data path with the pre-read bytes.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@blamejs/core",
3
- "version": "0.6.29",
3
+ "version": "0.6.32",
4
4
  "description": "The Node framework that owns its stack.",
5
5
  "license": "Apache-2.0",
6
6
  "author": "blamejs contributors",
@@ -2,10 +2,10 @@
2
2
  "$schema": "http://cyclonedx.org/schema/bom-1.5.schema.json",
3
3
  "bomFormat": "CycloneDX",
4
4
  "specVersion": "1.5",
5
- "serialNumber": "urn:uuid:438bf95a-273c-4fba-b130-bb7dddcdb3de",
5
+ "serialNumber": "urn:uuid:bf737d55-37d5-49a4-95fb-c1149d627468",
6
6
  "version": 1,
7
7
  "metadata": {
8
- "timestamp": "2026-05-02T18:04:26.378Z",
8
+ "timestamp": "2026-05-02T19:10:22.247Z",
9
9
  "lifecycles": [
10
10
  {
11
11
  "phase": "build"
@@ -19,14 +19,14 @@
19
19
  }
20
20
  ],
21
21
  "component": {
22
- "bom-ref": "@blamejs/core@0.6.29",
22
+ "bom-ref": "@blamejs/core@0.6.32",
23
23
  "type": "library",
24
24
  "name": "blamejs",
25
- "version": "0.6.29",
25
+ "version": "0.6.32",
26
26
  "scope": "required",
27
27
  "author": "blamejs contributors",
28
28
  "description": "The Node framework that owns its stack.",
29
- "purl": "pkg:npm/%40blamejs/core@0.6.29",
29
+ "purl": "pkg:npm/%40blamejs/core@0.6.32",
30
30
  "properties": [],
31
31
  "externalReferences": [
32
32
  {
@@ -54,7 +54,7 @@
54
54
  "components": [],
55
55
  "dependencies": [
56
56
  {
57
- "ref": "@blamejs/core@0.6.29",
57
+ "ref": "@blamejs/core@0.6.32",
58
58
  "dependsOn": []
59
59
  }
60
60
  ]