@blamejs/core 0.6.28 → 0.6.31
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +3 -0
- package/lib/log-stream-webhook.js +37 -23
- package/lib/mtls-ca.js +9 -0
- package/lib/queue-redis.js +168 -20
- package/lib/websocket.js +182 -15
- package/package.json +1 -1
- package/sbom.cyclonedx.json +6 -6
package/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,9 @@ upgrading across more than a few patches at a time.
|
|
|
8
8
|
|
|
9
9
|
## v0.6.x
|
|
10
10
|
|
|
11
|
+
- **0.6.31** (2026-05-02) — lazy-deferral cleanup wave from v0.6.27. **A1: queue-redis priority ordering** — `b.queue.enqueue({ priority })` on the Redis backend now matches queue-local's `ORDER BY priority DESC, availableAt ASC, enqueuedAt ASC` semantics. The Redis ZSET stays scored by availableAt only (so ZRANGEBYSCORE 0..nowMs cleanly filters ready vs not-yet-ready jobs), but `LEASE_LUA` now over-fetches `maxRows*5` candidates by score, HMGETs priority + availableAt + enqueuedAt for each, sorts server-side in Lua by the same triple queue-local sorts on, and leases the top `maxRows`. Closes the v0.6.27 deferral that punted priority ordering as "ZSET-incompatible". **A2: queue-redis flow `dependsOn` cascade** — `b.queue.enqueue({ flowId, flowChildName, dependsOn })` on the Redis backend now releases dependent jobs when their parents complete, mirroring queue-local's `_maybeReleaseFlowChildren`. A per-flow `<prefix>:flow:<flowId>` Redis SET tracks every job in the flow; `complete()` walks the set, looks up siblings whose `dependsOn` includes the just-completed job (by id OR `flowChildName`) AND every other dep is also satisfied, and HSET-bumps their `availableAt` to now + ZADDs them into the ready zset. The flow set persists across `complete()` calls so a later sibling's deps check can still find earlier-done parents by name; `purge()` cleans the set on queue teardown. Closes the v0.6.27 deferral. **A4: WebSocket per-message-deflate (RFC 7692)** — `b.websocket.handleUpgrade` negotiates the `permessage-deflate` extension when the client offers it, accepting `client_max_window_bits` / `server_max_window_bits` constraints (8-15 range, default 15) and always asserting `client_no_context_takeover` + `server_no_context_takeover` so every message uses a fresh zlib state. The connection's send path compresses TEXT/BINARY frames via `zlib.deflateRawSync`, strips the 4-byte `0x00 0x00 0xff 0xff` trailer per RFC 7692 §7.2.1, and sets RSV1 on the first frame of each compressed message. The receive path appends the trailer back and inflates via `zlib.inflateRawSync`. RSV1 on a continuation frame, RSV1 without negotiated extension, RSV2/RSV3 set, and decompressed payload exceeding `maxMessageBytes` all close with the appropriate RFC 6455 status (`PROTOCOL_ERROR`, `INVALID_PAYLOAD`, `MESSAGE_TOO_BIG`). Operator opt-out: pass `permessageDeflate: false` to `handleUpgrade` to refuse the extension even when offered. Tests cover the 101-handshake echo of negotiated params, real compression on a redundant payload (4000 bytes → ~30 bytes on the wire), uncompressed-frame round-trip on the same connection (mixed-mode per RFC 7692 §6), and unknown-extension graceful ignore. v0.6.27's third lazy deferral A3 (TLS-redis live test) was already covered by `test/integration/redis-client-tls.test.js` shipped in v0.6.28. Smoke 6780 / wiki e2e 178 / per-primitive integration 14 files green / wiki integration green.
|
|
12
|
+
- **0.6.30** (2026-05-02) — wiki-app integration gate + framework follow-ups surfaced by it. **`scripts/test-wiki-integration.js`** boots `examples/wiki` against the docker-compose fixture stack (real Redis / MinIO / Mailpit / CoreDNS / NTP / mtls-ca) and drives every backend through the wiki's HTTP surface AND the underlying framework primitives — validates that each primitive routes through the configured backend (cache → custom backend, queue → Redis Streams, mail → Mailpit SMTP, object-store → MinIO sigv4, log-stream → webhook receiver) AND that every fix shipped in v0.6.28 holds in a real-app context, not just in unit-test isolation. **Wiki gains test-only routes** under `/test/*` (gated by `WIKI_INTEGRATION_TEST=1`, mounted before CSRF/staticServe so test POSTs don't have to round-trip a token cookie) covering cache get/set/del, queue enqueue/size, mail send, object-store put/get, http-client fetch, log-stream emit, mtls-ca issue, ntp query, dns lookup, ssrf classify+check, plus `/test/diagnostic` that surfaces the active backend posture (network snapshot, queue backends, log sinks, mtls algorithm). **External-integration two-gate rule** — when a release diff touches a primitive that talks to an external service (`lib/redis-client.js`, `lib/queue-redis.js`, `lib/mail.js`, `lib/network-dns.js`, `lib/object-store/*`, `lib/log-stream*.js`, `lib/external-db.js`, `lib/cluster-*.js`, `lib/mtls-ca.js`, `lib/ssrf-guard.js`, `lib/http-client.js`, `lib/ntp-check.js`, `lib/cache.js`, `lib/webhook.js`), operators MUST run BOTH `scripts/test-integration.js` (per-primitive against real backends) AND `scripts/test-wiki-integration.js` (wiki app exercising the same backends end-to-end) before pushing. Documented in CONTRIBUTING.md's pre-push gate list and CLAUDE.md's release workflow as step 4a. The smoke + wiki-e2e gates above stay PURE (no docker dependency, runs in CI / on a developer laptop / inside prepack-guard); the two-gate is operator-run, opt-in, and surfaces bugs that mocks miss — fire-and-forget races, shutdown drains, TLS pinning, real DNS resolution, real protocol handshakes. **Framework fixes surfaced by the new gate**: `b.mtlsCa.create` now auto-creates `opts.dataDir` with `mode: 0o700` if it doesn't exist (matches `b.logStream`'s local sink, `b.backup`, `b.restoreBundle` — without it the first `initCA()` call hit `ENOENT` writing `ca.key.tmp`); `b.logStream` webhook-sink `close()` now waits for the in-flight `_flush()` before draining the buffer (the v0.6.28 drain fix only tracked emit-time wrapper promises, not the actual HTTP POST in flight; a record arriving mid-flush would buffer + the emit-time `_flush` early-returned on `if (inFlight) return`, and shutdown would then strand it). `b.queue.bootFromEnv()` is now wired in the wiki app's `build-app.js` unconditionally so the wiki picks up `BLAMEJS_QUEUE_PROTOCOL=redis` from env without code changes (was previously a documented-but-commented call site). **Test architecture rule (extends the v0.6.28 rule)** — live tests for the framework's primitives go under `test/integration/`; live tests for the wiki app exercising the framework end-to-end go under `examples/wiki/test/integration.js`. Smoke (`test/smoke.js`) and the existing wiki e2e (`examples/wiki/test/e2e.js`) stay pure. Smoke / wiki-e2e green; per-primitive integration suite green; wiki integration green. Eslint pin alignment + shellcheck added to the documented gate list (carry-over from v0.6.29).
|
|
13
|
+
- **0.6.29** (2026-05-02) — CI cleanup. **shellcheck gate green**: `docker/init/generate-certs.sh`'s footer line counted output via `ls "$CERT_DIR" | wc -l` (SC2012 — fragile on filenames with newlines or quoting metacharacters); replaced with `find "$CERT_DIR" -maxdepth 1 -mindepth 1 | wc -l`. Same semantics, robust to any cert filename pki-init might end up writing. **shellcheck added to the documented pre-push gate list** in CONTRIBUTING.md alongside smoke / wiki e2e / eslint / api-snapshot / primitive-section validators — CI's `Lint summary` job has always run it, but the local-dev recipe didn't, so contributors hit it post-push instead of catching it locally. **eslint pin alignment**: `.github/PULL_REQUEST_TEMPLATE.md` and CONTRIBUTING.md still pinned `eslint@10` even though v0.6.16 swapped CI runners to `eslint@latest`; both bumped to `@latest` to match the forward-track posture of every other CI tool. No framework code changes; the script + doc updates never ship in the npm package (excluded by `files` array).
|
|
11
14
|
- **0.6.28** (2026-05-02) — bug-finding pass: live integration test suite + every framework bug it caught. **`b.ssrfGuard.checkUrl` cloud-metadata hard-deny** — `allowInternal: true` no longer permits 169.254.169.254 (AWS / GCP / Azure metadata). Loopback / private / link-local / reserved stay allowInternal-overridable per existing semantics; cloud-metadata is unconditional because a blanket override would let any compromised request exfiltrate instance credentials. **`b.mtlsCa` issues server certs and dual-EKU certs** — `generateClientCert({ usage: "client" | "server" | "both", sans: [...] })`. `usage` controls EKU (`client` = clientAuth, `server` = serverAuth, `both` = both); `sans` accepts `DNS:` / `IP:` / bare-DNS entries; serverAuth without an explicit SAN auto-adds the CN. Operators wiring inbound mTLS reverse-proxy fronts no longer hit "unsuitable certificate purpose" on the handshake. **`b.mtlsCa` algorithm auto-detect** — the engine probes webcrypto + the vendored x509 library at first cert issuance and picks the highest-PQC option that round-trips: SLH-DSA-SHAKE-256f → SLH-DSA-SHAKE-128f → ML-DSA-87 → ML-DSA-65 → ECDSA-P384-SHA384 (current bridge — what the X.509 ecosystem accepts today). When the bridge condition lifts and the vendor refresh ships, the same `b.mtlsCa.create(...)` call self-upgrades; `b.mtlsCa.status().cert.label` and `.posture` surface the chosen algorithm. **`b.redisClient.create({ ca, servername })`** — managed-Redis / on-prem-cluster operators connecting over `rediss://` against a private CA can pin trust roots; `servername` auto-suppresses for IP literals so `rediss://127.0.0.1:6380` no longer trips node:tls's IP-as-SNI rule. **`b.network.dns.useDnsOverTls` / `useDnsOverHttps` accept `ca`** — same trust-pinning surface for self-signed / private-PKI DoT and DoH endpoints. **DoT TLS handshake errors** now route as DnsError on the lookup promise rather than leaking as `unhandledRejection` — the secureConnect / error event-listener race had let cert-verification failures escape the per-query Promise wrapping. **DoT socket lifecycle** — was unconditionally `sock.unref()`'d after construct, causing node to exit during in-flight lookups when no other I/O kept the loop alive; now ref'd while a query is in flight and unref'd when idle. **`b.ntpCheck.querySingle` honours IPv6 servers** — was hardcoded to `dgram.createSocket("udp4")`, so `::1` / `fd00::…` queries failed with EINVAL; now auto-detects family from the server string. **`b.logStream.shutdown` drains in-flight emit microtasks** before closing sink fds — fire-and-forget emits queued just before shutdown were silently dropped because `close()` ran ahead of the microtask. **`b.logStream` webhook-sink `close()` flushes BEFORE setting `closed = true`** — `_flush`'s while loop bailed on `!closed`, leaving any records buffered just before shutdown stranded. **Integration test suite** (`test/integration/`, runner at `scripts/test-integration.js`) — 13 live test files across `b.redisClient` plain + TLS, `b.queue` Redis backend lifecycle, `b.mail` SMTP + STARTTLS + multi-rcpt + dot-stuffing transparency, `b.mail.dkim` rsa-sha256 + ed25519-sha256 + bad-algorithm reject, `b.ntpCheck` v4 + v6 + bootCheck + bad-host + bad-port, `b.network.dns` plain + DoT + DoH with strict CA pinning + bad-servername + cache, `b.network.heartbeat` http + tcp + state-change callbacks, `b.objectStore` sigv4 PUT / GET / list / delete on plain HTTP + TLS variants, `b.cache` memory + Redis-backed cluster, `b.httpClient` direct + Squid forward proxy + TLS pin, `b.ssrfGuard` classify + checkUrl + cloud-metadata block, `b.logStream` local + webhook + deferred-syslog error path, `b.mtlsCa` CA bootstrap + clientAuth + serverAuth + dual-EKU live mTLS handshakes. Companion `docker-compose.test.yml` stands up redis (plain + TLS) / postgres / mysql / mongo / minio (HTTP + HTTPS variants) / rabbitmq (plain + TLS) / nats / syslog / ntp / mailpit / coredns (plain + DoT + DoH) / haproxy / caddy / mitmproxy / squid / pki-init (auto-generates Ed25519 CA + per-service leaf certs into a docker volume); host port bindings dual-stack on `127.0.0.1` AND `[::1]`. `scripts/check-services.js` does host-side TCP / TLS handshake / SNTPv4 / DNS-A / Redis-PING / SMTP-banner / HTTP probes across every endpoint; `scripts/test-integration.js` exports the test CA via `docker cp` and sets `NODE_EXTRA_CA_CERTS` per-test child process so the framework's TLS verification stays strict (no `rejectUnauthorized: false` bypass anywhere in the test surface). **Test architecture rule** — live tests go in `test/integration/`, NEVER in `test/layer-N-*/`: smoke must remain pure (no docker dependency, runs in CI / on a developer laptop / inside prepack-guard) and a "skip silently when service is down" branch in a layer-N test makes the gate's pass count misleading and masks bugs that only surface against a live backend. The `0.6.27` `queue-redis.test.js` was moved to `test/integration/queue-redis.test.js` to enforce this. Smoke and wiki e2e green; integration suite ships green against the docker-compose stack.
|
|
12
15
|
- **0.6.27** (2026-05-02) — Redis backend for `b.queue` so multi-replica apps can share a single queue without each needing to be cluster leader. **Bespoke RESP2 client** (`lib/redis-client.js`) — zero npm runtime deps. TCP via `node:net` + TLS via `node:tls` (`rediss://` auto-detected), legacy single-arg AUTH + ACL `AUTH user pass`, `SELECT db`, pipelining, exponential-backoff reconnect, EVAL helper. **`b.queue` protocol "redis"** (`lib/queue-redis.js`) — full enqueue/lease/extendLease/complete/fail/sweepExpired/size/purge/dlqList/dlqRetry/dlqSize parity with the local backend. Atomicity comes from server-side Lua scripts so concurrent consumers can't double-lease and a sweep can't race a complete. Storage layout: per-job HASH (sealed payload + lastError via `cryptoField.sealRow("_blamejs_jobs", row)` — same crypto config as the local backend), per-queue ready ZSET scored by availableAt, per-queue inflight ZSET scored by leaseExpiresAt, per-queue dlq ZSET scored by finishedAt, plus a queues SET so sweepExpired walks every known queue without a global secondary index. Cron-repeat handled in `complete()` JS — re-enqueues the next firing as a fresh jobId with availableAt=next-cron-fire. **`b.queue.bootFromEnv({ env })`** — env-driven init mirroring `b.network.bootFromEnv` and `b.logStream.bootFromEnv`. Reads `BLAMEJS_QUEUE_PROTOCOL` (`local`|`redis`), `BLAMEJS_QUEUE_REDIS_URL`, `BLAMEJS_QUEUE_REDIS_PASSWORD`, `BLAMEJS_QUEUE_REDIS_USERNAME`, `BLAMEJS_QUEUE_REDIS_TLS`, `BLAMEJS_QUEUE_REDIS_KEY_PREFIX`. Operators flip from local to Redis without a code change; both wiki docker-compose configs declare the new env knobs. Removed `redis` from `DEFERRED_PROTOCOLS`. **Out of scope for v1** (deferred to follow-up patches with explicit re-open conditions): Redis Cluster (slot-routing), Sentinel (managed primary failover), priority ordering on the Redis backend (queue-local supports `priority` opt; Redis backend orders strictly by availableAt for v1), flow children with `dependsOn` cascade. **Wiki**: queue-cache page documents the Redis backend opts schema, bootFromEnv, and the layout. **Tests**: 24 RESP2 protocol parser unit tests (`test/layer-0-primitives/redis-client.test.js`) cover URL parsing, command encoding (binary-safe), every reply type (simple string / error / integer / bulk / nil bulk / nested array / pipelined / incomplete-mid-frame). Live Redis round-trip tests (`test/layer-0-primitives/queue-redis.test.js`) cover enqueue+lease, availableAt scheduling, visibility-timeout sweep, fail+retry path, DLQ list/retry/size, extendLease, purge, and concurrent-leaser no-double-lease — skip cleanly when `BLAMEJS_TEST_REDIS_URL` is not set so the smoke suite passes on dev boxes without a Redis container. Smoke 6780 OK.
|
|
13
16
|
- **0.6.26** (2026-05-02) — `blamejs restore` and `blamejs audit verify-chain` subcommands wrap the existing `b.restore` and `b.audit.verifyChain` primitives so operators can drive them from runbooks without writing app code. **`blamejs restore`**: `list` (enumerate bundles in storage), `inspect` (manifest summary without touching live data), `apply` (live in-place restore with rollback preserved), `rollback` (revert to most-recent OR named restore point), `list-rollbacks` (enumerate preserved rollback points). Two ways to identify a bundle — `--bundle <dir>` matches the shape `blamejs backup extract` produces (parent dir is treated as storage root, basename as bundle id), or `--storage-root <root> --bundle-id <id>` for multi-bundle stores. `apply` honors `--max-pulled-bytes` / `--max-pulled-files` (defaults 4 GiB / 100K), `--rollback-root` (default `<data-dir>.rollbacks`), `--no-audit`, and `BLAMEJS_BACKUP_PASSPHRASE` env. **`blamejs audit verify-chain`**: walks the live audit chain end-to-end, reports tampering with `breakAt` / `breakRowId` / expected-vs-actual prevHash; honours `--max-rows` to bound long walks; default table is `audit_log`. **Wiki CLI snapshot test now validates subcommand pairs**, not just top commands. Walks every wiki + README invocation of the form `blamejs <cmd> <sub>` and verifies that <sub> exists in the perCommand[<cmd>].subcommands list parsed from `lib/cli.js`. Surfaced one real drift on first run: `examples/wiki/seeders/prod/pages/backup-restore.js` documented `blamejs audit verify-signing` after vault rotation, but no such subcommand existed (only `verify-bundle`); fixed by shipping the new `verify-chain` subcommand and updating the wiki to reference it. Two top-level command gaps surfaced and fixed: `blamejs restore` (now real) and `blamejs network status` (was prose-promised in `network-config.js`; reworded to point at `b.network.snapshot()` for /healthz / custom diagnostics routes since wiring a CLI command for it is operator-side work). README CLI table updated with `restore` row and the two new audit subcommands.
|
|
@@ -122,32 +122,42 @@ function create(config) {
|
|
|
122
122
|
flushTimer.unref();
|
|
123
123
|
}
|
|
124
124
|
|
|
125
|
+
// Track the in-flight flush as a promise so close() can await it.
|
|
126
|
+
// Without this, a record arriving mid-flush gets buffered, the
|
|
127
|
+
// emit-time _flush() early-returns on `if (inFlight) return`, and
|
|
128
|
+
// the buffered record is stranded if shutdown fires before
|
|
129
|
+
// _scheduleFlush() drains it.
|
|
130
|
+
var inFlightPromise = null;
|
|
125
131
|
async function _flush() {
|
|
126
|
-
if (inFlight) return;
|
|
132
|
+
if (inFlight) return inFlightPromise;
|
|
127
133
|
if (buffer.length === 0) return;
|
|
128
134
|
inFlight = true;
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
135
|
+
inFlightPromise = (async function () {
|
|
136
|
+
try {
|
|
137
|
+
while (buffer.length > 0 && !closed) {
|
|
138
|
+
var batch = buffer.splice(0, cfg.batchSize);
|
|
139
|
+
var body = _serializeBatch(batch, cfg.bodyShape);
|
|
140
|
+
try {
|
|
141
|
+
await retryHelper.withRetry(function () {
|
|
142
|
+
return _post(cfg.url, body, headers, cfg.timeoutMs, cfg.allowedProtocols, cfg.allowInternal);
|
|
143
|
+
}, cfg.retry);
|
|
144
|
+
} catch (e) {
|
|
145
|
+
// Batch permanently rejected — surface via dropCount AND the
|
|
146
|
+
// operator-supplied onDrop callback. The dispatcher path
|
|
147
|
+
// wraps its own audit hook around emit(); operators using
|
|
148
|
+
// this sink directly rely on dropCount + onDrop.
|
|
149
|
+
dropCount += batch.length;
|
|
150
|
+
_emitDrop("retry-exhausted", batch, e);
|
|
151
|
+
break;
|
|
152
|
+
}
|
|
145
153
|
}
|
|
154
|
+
} finally {
|
|
155
|
+
inFlight = false;
|
|
156
|
+
inFlightPromise = null;
|
|
157
|
+
if (buffer.length > 0) _scheduleFlush();
|
|
146
158
|
}
|
|
147
|
-
}
|
|
148
|
-
|
|
149
|
-
if (buffer.length > 0) _scheduleFlush();
|
|
150
|
-
}
|
|
159
|
+
})();
|
|
160
|
+
return inFlightPromise;
|
|
151
161
|
}
|
|
152
162
|
|
|
153
163
|
function emit(record) {
|
|
@@ -171,9 +181,13 @@ function create(config) {
|
|
|
171
181
|
// Drain BEFORE flipping closed=true. _flush()'s while loop bails on
|
|
172
182
|
// !closed, so flipping the flag first leaves any buffered records
|
|
173
183
|
// stranded — the very records the operator queued just before
|
|
174
|
-
// calling shutdown(). Order: stop the timer,
|
|
175
|
-
//
|
|
184
|
+
// calling shutdown(). Order: stop the timer, await any in-flight
|
|
185
|
+
// flush so its records POST before we touch the buffer, drain
|
|
186
|
+
// anything still queued, THEN refuse new enqueues.
|
|
176
187
|
if (flushTimer) { clearTimeout(flushTimer); flushTimer = null; }
|
|
188
|
+
if (inFlightPromise) {
|
|
189
|
+
try { await inFlightPromise; } catch (_e) { /* surfaced via onDrop */ }
|
|
190
|
+
}
|
|
177
191
|
await _flush();
|
|
178
192
|
closed = true;
|
|
179
193
|
}
|
package/lib/mtls-ca.js
CHANGED
|
@@ -129,6 +129,15 @@ function create(opts) {
|
|
|
129
129
|
throw new MtlsCaError("mtls-ca/no-datadir",
|
|
130
130
|
"mtlsCa.create requires opts.dataDir");
|
|
131
131
|
}
|
|
132
|
+
// Auto-create the dataDir with restrictive perms (CA keys live here).
|
|
133
|
+
// Matches the behaviour of other framework primitives that own a
|
|
134
|
+
// dataDir — log-stream-local, backup, restore-bundle. Without this
|
|
135
|
+
// the first initCA() / generateClientCert() call fails with ENOENT
|
|
136
|
+
// on `ca.key.tmp` because the atomic-file write expects the parent
|
|
137
|
+
// dir to exist.
|
|
138
|
+
if (!fs.existsSync(opts.dataDir)) {
|
|
139
|
+
fs.mkdirSync(opts.dataDir, { recursive: true, mode: 0o700 });
|
|
140
|
+
}
|
|
132
141
|
var paths = _resolvePaths(opts.dataDir, opts.paths);
|
|
133
142
|
var vault = opts.vault || null;
|
|
134
143
|
var caKeySealedMode = (opts.caKeySealedMode || "auto").toLowerCase();
|
package/lib/queue-redis.js
CHANGED
|
@@ -29,12 +29,6 @@
|
|
|
29
29
|
* Out of scope (defer to follow-up patches):
|
|
30
30
|
* - Redis Cluster (slot-routing across multi-node Redis)
|
|
31
31
|
* - Sentinel (managed primary failover)
|
|
32
|
-
* - Job priority (queue-local supports `priority` opt; Redis backend
|
|
33
|
-
* orders strictly by availableAt for v1 — re-introduce when a real
|
|
34
|
-
* operator demand surfaces with a clean Lua-side ordering scheme)
|
|
35
|
-
* - Flow children with dependsOn (queue-local's _maybeReleaseFlowChildren
|
|
36
|
-
* coordination — orthogonal to backend choice; ships when flow primitive
|
|
37
|
-
* itself becomes backend-agnostic)
|
|
38
32
|
*/
|
|
39
33
|
var C = require("./constants");
|
|
40
34
|
var cryptoField = require("./crypto-field");
|
|
@@ -57,9 +51,23 @@ var DEFAULT_PREFIX = "blamejs:queue";
|
|
|
57
51
|
// ---- Lua scripts ----
|
|
58
52
|
//
|
|
59
53
|
// LEASE_LUA — atomically pull up to maxRows jobs from the ready zset
|
|
60
|
-
// whose
|
|
61
|
-
//
|
|
62
|
-
//
|
|
54
|
+
// whose availableAt is <= nowMs, move them to the inflight zset with
|
|
55
|
+
// score = leaseExpiresAt, increment attempts, flip status, and return
|
|
56
|
+
// the jobIds in priority order. The JS side then HGETALLs each id.
|
|
57
|
+
//
|
|
58
|
+
// Priority semantics — matches queue-local's `ORDER BY priority DESC,
|
|
59
|
+
// availableAt ASC, enqueuedAt ASC` over the same `WHERE availableAt
|
|
60
|
+
// <= nowMs` filter. The Redis ZSET orders by availableAt only (the
|
|
61
|
+
// score), which is enough to filter "ready vs not-yet-ready" via
|
|
62
|
+
// ZRANGEBYSCORE 0..nowMs but NOT enough to surface a high-priority
|
|
63
|
+
// job ahead of an earlier-availableAt low-priority job. So we
|
|
64
|
+
// over-fetch maxRows*5 candidates by score, HMGET the priority +
|
|
65
|
+
// availableAt for each, sort priority-DESC / availableAt-ASC server-
|
|
66
|
+
// side in Lua, and lease the top maxRows. The over-fetch factor is
|
|
67
|
+
// chosen so a queue with a typical priority distribution surfaces
|
|
68
|
+
// the right jobs without scanning the entire ready set; queues with
|
|
69
|
+
// >5x as many priority-0 jobs as priority-N jobs would still pull
|
|
70
|
+
// the top-priority ones first.
|
|
63
71
|
//
|
|
64
72
|
// KEYS[1] = ready zset
|
|
65
73
|
// KEYS[2] = inflight zset
|
|
@@ -74,10 +82,29 @@ var LEASE_LUA = [
|
|
|
74
82
|
'local leaseExpiresAt = tonumber(ARGV[2])',
|
|
75
83
|
'local maxRows = tonumber(ARGV[3])',
|
|
76
84
|
'local jobKeyPrefix = ARGV[4]',
|
|
77
|
-
'local
|
|
85
|
+
'local oversample = maxRows * 5',
|
|
86
|
+
'local jobIds = redis.call("ZRANGEBYSCORE", readyKey, 0, nowMs, "LIMIT", 0, oversample)',
|
|
78
87
|
'if #jobIds == 0 then return {} end',
|
|
88
|
+
// Pull priority + availableAt + enqueuedAt for each candidate so the
|
|
89
|
+
// sort uses the same triple queue-local sorts on. enqueuedAt is the
|
|
90
|
+
// tiebreaker among same-priority same-availableAt jobs (FIFO within
|
|
91
|
+
// a priority lane).
|
|
92
|
+
'local rows = {}',
|
|
79
93
|
'for i = 1, #jobIds do',
|
|
80
94
|
' local jobId = jobIds[i]',
|
|
95
|
+
' local h = redis.call("HMGET", jobKeyPrefix..jobId, "priority", "availableAt", "enqueuedAt")',
|
|
96
|
+
' rows[i] = { jobId, tonumber(h[1] or "0") or 0, tonumber(h[2] or "0") or 0, tonumber(h[3] or "0") or 0 }',
|
|
97
|
+
'end',
|
|
98
|
+
// priority DESC, availableAt ASC, enqueuedAt ASC.
|
|
99
|
+
'table.sort(rows, function(a, b)',
|
|
100
|
+
' if a[2] ~= b[2] then return a[2] > b[2] end',
|
|
101
|
+
' if a[3] ~= b[3] then return a[3] < b[3] end',
|
|
102
|
+
' return a[4] < b[4]',
|
|
103
|
+
'end)',
|
|
104
|
+
'local picked = {}',
|
|
105
|
+
'local n = math.min(maxRows, #rows)',
|
|
106
|
+
'for i = 1, n do',
|
|
107
|
+
' local jobId = rows[i][1]',
|
|
81
108
|
' redis.call("ZREM", readyKey, jobId)',
|
|
82
109
|
' redis.call("ZADD", inflightKey, leaseExpiresAt, jobId)',
|
|
83
110
|
' redis.call("HINCRBY", jobKeyPrefix..jobId, "attempts", 1)',
|
|
@@ -85,8 +112,9 @@ var LEASE_LUA = [
|
|
|
85
112
|
' "status", "inflight",',
|
|
86
113
|
' "leasedAt", nowMs,',
|
|
87
114
|
' "leaseExpiresAt", leaseExpiresAt)',
|
|
115
|
+
' picked[i] = jobId',
|
|
88
116
|
'end',
|
|
89
|
-
'return
|
|
117
|
+
'return picked',
|
|
90
118
|
].join("\n");
|
|
91
119
|
|
|
92
120
|
// SWEEP_LUA — find jobs in inflight whose lease expired, push back to
|
|
@@ -251,6 +279,7 @@ function create(opts) {
|
|
|
251
279
|
function _dlqKey(queueName) { return prefix + ":q:" + queueName + ":dlq"; }
|
|
252
280
|
function _queuesKey() { return prefix + ":queues"; }
|
|
253
281
|
function _jobKeyPrefix() { return prefix + ":job:"; }
|
|
282
|
+
function _flowKey(flowId) { return prefix + ":flow:" + flowId; }
|
|
254
283
|
|
|
255
284
|
// ---- Row encoding ----
|
|
256
285
|
//
|
|
@@ -348,13 +377,22 @@ function create(opts) {
|
|
|
348
377
|
};
|
|
349
378
|
var sealed = cryptoField.sealRow("_blamejs_jobs", row);
|
|
350
379
|
|
|
351
|
-
// Pipeline: HSET job + ZADD ready + SADD queues
|
|
352
|
-
// hit Redis without round-trips
|
|
380
|
+
// Pipeline: HSET job + ZADD ready + SADD queues + (if flowId)
|
|
381
|
+
// SADD flow set. Pipelined writes hit Redis without round-trips
|
|
382
|
+
// between them. The flow set is the per-flow registry that
|
|
383
|
+
// complete() walks to release dependents — it lets us avoid a
|
|
384
|
+
// SCAN over every job hash when releasing children, matching the
|
|
385
|
+
// queue-local pattern of "SELECT siblings WHERE flowId = ?".
|
|
353
386
|
var hsetArgs = _hsetArgs(jobId, sealed);
|
|
354
|
-
var
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
387
|
+
var pipeline = [
|
|
388
|
+
client.command.apply(null, hsetArgs),
|
|
389
|
+
client.command("ZADD", _readyKey(queueName), String(availableAt), jobId),
|
|
390
|
+
client.command("SADD", _queuesKey(), queueName),
|
|
391
|
+
];
|
|
392
|
+
if (row.flowId) {
|
|
393
|
+
pipeline.push(client.command("SADD", _flowKey(row.flowId), jobId));
|
|
394
|
+
}
|
|
395
|
+
await Promise.all(pipeline);
|
|
358
396
|
|
|
359
397
|
return {
|
|
360
398
|
jobId: jobId,
|
|
@@ -450,9 +488,104 @@ function create(opts) {
|
|
|
450
488
|
});
|
|
451
489
|
} catch (_e) { /* best-effort — cron resumes next tick if op fixes the issue */ }
|
|
452
490
|
}
|
|
491
|
+
|
|
492
|
+
// Flow propagation: walk siblings whose dependsOn includes this
|
|
493
|
+
// jobId (or this job's flowChildName) and bump availableAt to now
|
|
494
|
+
// if ALL their deps are now complete. Mirrors queue-local's
|
|
495
|
+
// _maybeReleaseFlowChildren, but uses a per-flow Redis SET as
|
|
496
|
+
// the sibling registry instead of a SQL SELECT.
|
|
497
|
+
if (raw.flowId) {
|
|
498
|
+
try {
|
|
499
|
+
await _maybeReleaseFlowChildren(raw.flowId, jobId, raw.flowChildName || null, nowMs);
|
|
500
|
+
} catch (_e) { /* best-effort — sweepExpired retries if a deps check fails */ }
|
|
501
|
+
// The completed job stays in the flow set so a LATER sibling
|
|
502
|
+
// whose dependsOn includes this job's flowChildName can still
|
|
503
|
+
// find a "done" sibling when its own complete() walks the flow.
|
|
504
|
+
// The set is purge()'d when the queue is purged or — for the
|
|
505
|
+
// typical case where the operator wants to reclaim flow memory
|
|
506
|
+
// after the whole flow finishes — by an explicit operator call
|
|
507
|
+
// to purge() OR by letting `_maybeReleaseFlowChildren` reach a
|
|
508
|
+
// state where every sibling has status='done' (which the
|
|
509
|
+
// operator can detect via b.queue.dlqList / app-level queries).
|
|
510
|
+
}
|
|
453
511
|
return true;
|
|
454
512
|
}
|
|
455
513
|
|
|
514
|
+
async function _maybeReleaseFlowChildren(flowId, completedJobId, completedChildName, nowMs) {
|
|
515
|
+
var flowKey = _flowKey(flowId);
|
|
516
|
+
var members = await client.command("SMEMBERS", flowKey);
|
|
517
|
+
if (!members || members.length === 0) return;
|
|
518
|
+
|
|
519
|
+
// Pull dependsOn + status + flowChildName for every sibling in one
|
|
520
|
+
// pipelined batch — far cheaper than per-sibling HMGET round trips.
|
|
521
|
+
var siblingIds = members.map(function (m) {
|
|
522
|
+
return Buffer.isBuffer(m) ? m.toString("utf8") : String(m);
|
|
523
|
+
}).filter(function (id) { return id !== completedJobId; });
|
|
524
|
+
if (siblingIds.length === 0) return;
|
|
525
|
+
|
|
526
|
+
var hmgetCalls = siblingIds.map(function (sibId) {
|
|
527
|
+
return client.command("HMGET", _jobKey(sibId),
|
|
528
|
+
"dependsOn", "status", "flowChildName");
|
|
529
|
+
});
|
|
530
|
+
var results = await Promise.all(hmgetCalls);
|
|
531
|
+
|
|
532
|
+
// For each pending sibling with a dependsOn array, check if all
|
|
533
|
+
// deps are satisfied (the just-completed job AND any prior
|
|
534
|
+
// already-done sibling in the flow).
|
|
535
|
+
for (var i = 0; i < siblingIds.length; i++) {
|
|
536
|
+
var sibId = siblingIds[i];
|
|
537
|
+
var rv = results[i];
|
|
538
|
+
if (!rv || rv.length < 3) continue;
|
|
539
|
+
var rawDeps = rv[0] && (Buffer.isBuffer(rv[0]) ? rv[0].toString("utf8") : String(rv[0]));
|
|
540
|
+
var status = rv[1] && (Buffer.isBuffer(rv[1]) ? rv[1].toString("utf8") : String(rv[1]));
|
|
541
|
+
if (!rawDeps || status !== "pending") continue;
|
|
542
|
+
var deps;
|
|
543
|
+
try { deps = JSON.parse(rawDeps); }
|
|
544
|
+
catch (_e) { continue; }
|
|
545
|
+
if (!Array.isArray(deps) || deps.length === 0) continue;
|
|
546
|
+
|
|
547
|
+
// Resolve each dep against the just-completed job (id or child
|
|
548
|
+
// name) OR a prior-completed sibling in the flow.
|
|
549
|
+
var allDone = true;
|
|
550
|
+
for (var d = 0; d < deps.length; d++) {
|
|
551
|
+
var dep = deps[d];
|
|
552
|
+
if (dep === completedJobId) continue;
|
|
553
|
+
if (completedChildName && dep === completedChildName) continue;
|
|
554
|
+
// Look up by id first; if no hit, fall back to scanning the
|
|
555
|
+
// flow set for a sibling whose flowChildName matches.
|
|
556
|
+
var depHash = await client.command("HMGET", _jobKey(dep), "status", "flowId");
|
|
557
|
+
if (depHash && depHash[0]) {
|
|
558
|
+
var depStatus = Buffer.isBuffer(depHash[0]) ? depHash[0].toString("utf8") : String(depHash[0]);
|
|
559
|
+
var depFlow = depHash[1] ? (Buffer.isBuffer(depHash[1]) ? depHash[1].toString("utf8") : String(depHash[1])) : "";
|
|
560
|
+
if (depStatus === "done" && depFlow === flowId) continue;
|
|
561
|
+
}
|
|
562
|
+
// Scan the flow's set for a child-name match — cheap because
|
|
563
|
+
// a flow typically has 5-50 children, not thousands.
|
|
564
|
+
var matched = false;
|
|
565
|
+
for (var s = 0; s < siblingIds.length && !matched; s++) {
|
|
566
|
+
if (siblingIds[s] === sibId) continue;
|
|
567
|
+
var sRv = results[s];
|
|
568
|
+
if (!sRv || !sRv[2]) continue;
|
|
569
|
+
var sName = Buffer.isBuffer(sRv[2]) ? sRv[2].toString("utf8") : String(sRv[2]);
|
|
570
|
+
var sStatus = sRv[1] ? (Buffer.isBuffer(sRv[1]) ? sRv[1].toString("utf8") : String(sRv[1])) : "";
|
|
571
|
+
if (sName === dep && sStatus === "done") matched = true;
|
|
572
|
+
}
|
|
573
|
+
if (!matched) { allDone = false; break; }
|
|
574
|
+
}
|
|
575
|
+
|
|
576
|
+
if (allDone) {
|
|
577
|
+
// Read queueName so the ZADD targets the right ready zset.
|
|
578
|
+
var qBuf = await client.command("HGET", _jobKey(sibId), "queueName");
|
|
579
|
+
if (!qBuf) continue;
|
|
580
|
+
var queueName = Buffer.isBuffer(qBuf) ? qBuf.toString("utf8") : String(qBuf);
|
|
581
|
+
await Promise.all([
|
|
582
|
+
client.command("HSET", _jobKey(sibId), "availableAt", String(nowMs)),
|
|
583
|
+
client.command("ZADD", _readyKey(queueName), String(nowMs), sibId),
|
|
584
|
+
]);
|
|
585
|
+
}
|
|
586
|
+
}
|
|
587
|
+
}
|
|
588
|
+
|
|
456
589
|
async function fail(jobId, errorMessage, retryDelayMs) {
|
|
457
590
|
await _ensureConnected();
|
|
458
591
|
var nowMs = Date.now();
|
|
@@ -503,8 +636,11 @@ function create(opts) {
|
|
|
503
636
|
|
|
504
637
|
async function purge(queueName) {
|
|
505
638
|
await _ensureConnected();
|
|
506
|
-
// Walk the ready + inflight + dlq zsets,
|
|
507
|
-
//
|
|
639
|
+
// Walk the ready + inflight + dlq zsets, collect every job id,
|
|
640
|
+
// also clear each job from its flow set (if any), then DEL the
|
|
641
|
+
// job hashes + zsets + queues-membership. Flow sets are cleared
|
|
642
|
+
// job-by-job because a single flow can span multiple queues, and
|
|
643
|
+
// we only want to evict THIS queue's contributors.
|
|
508
644
|
var [readyMembers, inflightMembers, dlqMembers] = await Promise.all([
|
|
509
645
|
client.command("ZRANGE", _readyKey(queueName), "0", "-1"),
|
|
510
646
|
client.command("ZRANGE", _inflightKey(queueName), "0", "-1"),
|
|
@@ -512,6 +648,18 @@ function create(opts) {
|
|
|
512
648
|
]);
|
|
513
649
|
var allIds = [].concat(readyMembers || [], inflightMembers || [], dlqMembers || [])
|
|
514
650
|
.map(function (b) { return Buffer.isBuffer(b) ? b.toString("utf8") : String(b); });
|
|
651
|
+
// Pull the flowId for each job (if set) so we can SREM from the
|
|
652
|
+
// matching flow set BEFORE we DEL the job hash.
|
|
653
|
+
var flowIdLookups = await Promise.all(allIds.map(function (id) {
|
|
654
|
+
return client.command("HGET", _jobKey(id), "flowId");
|
|
655
|
+
}));
|
|
656
|
+
var flowSrems = [];
|
|
657
|
+
for (var fi = 0; fi < allIds.length; fi++) {
|
|
658
|
+
var fIdBuf = flowIdLookups[fi];
|
|
659
|
+
if (!fIdBuf) continue;
|
|
660
|
+
var fId = Buffer.isBuffer(fIdBuf) ? fIdBuf.toString("utf8") : String(fIdBuf);
|
|
661
|
+
if (fId) flowSrems.push(client.command("SREM", _flowKey(fId), allIds[fi]));
|
|
662
|
+
}
|
|
515
663
|
var dels = allIds.map(function (id) { return client.command("DEL", _jobKey(id)); });
|
|
516
664
|
var zdrops = [
|
|
517
665
|
client.command("DEL", _readyKey(queueName)),
|
|
@@ -519,7 +667,7 @@ function create(opts) {
|
|
|
519
667
|
client.command("DEL", _dlqKey(queueName)),
|
|
520
668
|
client.command("SREM", _queuesKey(), queueName),
|
|
521
669
|
];
|
|
522
|
-
await Promise.all(
|
|
670
|
+
await Promise.all(flowSrems.concat(dels, zdrops));
|
|
523
671
|
return allIds.length;
|
|
524
672
|
}
|
|
525
673
|
|
package/lib/websocket.js
CHANGED
|
@@ -82,6 +82,7 @@
|
|
|
82
82
|
*/
|
|
83
83
|
|
|
84
84
|
var nodeCrypto = require("crypto");
|
|
85
|
+
var zlib = require("zlib");
|
|
85
86
|
var { EventEmitter } = require("events");
|
|
86
87
|
var C = require("./constants");
|
|
87
88
|
var requestHelpers = require("./request-helpers");
|
|
@@ -220,7 +221,7 @@ function isOriginAllowed(req, origins) {
|
|
|
220
221
|
return false;
|
|
221
222
|
}
|
|
222
223
|
|
|
223
|
-
function buildUpgradeResponse(secWebSocketKey, subprotocol) {
|
|
224
|
+
function buildUpgradeResponse(secWebSocketKey, subprotocol, extensionHeader) {
|
|
224
225
|
var lines = [
|
|
225
226
|
"HTTP/1.1 101 Switching Protocols",
|
|
226
227
|
"Upgrade: websocket",
|
|
@@ -228,9 +229,107 @@ function buildUpgradeResponse(secWebSocketKey, subprotocol) {
|
|
|
228
229
|
"Sec-WebSocket-Accept: " + computeAcceptKey(secWebSocketKey),
|
|
229
230
|
];
|
|
230
231
|
if (subprotocol) lines.push("Sec-WebSocket-Protocol: " + subprotocol);
|
|
232
|
+
if (extensionHeader) lines.push("Sec-WebSocket-Extensions: " + extensionHeader);
|
|
231
233
|
return lines.join("\r\n") + "\r\n\r\n";
|
|
232
234
|
}
|
|
233
235
|
|
|
236
|
+
// ---- permessage-deflate (RFC 7692) ----
|
|
237
|
+
//
|
|
238
|
+
// Negotiate compression at handshake, compress per-message on send,
|
|
239
|
+
// decompress per-message on receive. The framework runs in
|
|
240
|
+
// "no_context_takeover" mode in both directions — every message uses a
|
|
241
|
+
// fresh zlib state, no LZ77 history carried across messages. This
|
|
242
|
+
// trade-off makes message processing stateless (no per-connection
|
|
243
|
+
// zlib stream lifetime to manage) at a small compression-ratio cost.
|
|
244
|
+
// Operators with throughput-sensitive workloads can extend this later
|
|
245
|
+
// to keep state across messages.
|
|
246
|
+
//
|
|
247
|
+
// Per RFC 7692 §7.2.1 the deflate output is the standard zlib raw
|
|
248
|
+
// deflate WITH the trailing 4 bytes 0x00 0x00 0xff 0xff stripped. The
|
|
249
|
+
// matching inflate path appends them back before inflating.
|
|
250
|
+
var DEFLATE_TRAILING = Buffer.from([0x00, 0x00, 0xff, 0xff]);
|
|
251
|
+
|
|
252
|
+
function _parseExtensionHeader(header) {
|
|
253
|
+
// Sec-WebSocket-Extensions: foo; param=val; param2, bar; ...
|
|
254
|
+
// Returns [{ name, params: { paramName: value | true } }]
|
|
255
|
+
if (!header) return [];
|
|
256
|
+
var entries = String(header).split(",");
|
|
257
|
+
var out = [];
|
|
258
|
+
for (var i = 0; i < entries.length; i++) {
|
|
259
|
+
var parts = entries[i].split(";").map(function (s) { return s.trim(); });
|
|
260
|
+
if (!parts[0]) continue;
|
|
261
|
+
var ext = { name: parts[0].toLowerCase(), params: {} };
|
|
262
|
+
for (var j = 1; j < parts.length; j++) {
|
|
263
|
+
var kv = parts[j].split("=");
|
|
264
|
+
var k = kv[0].trim().toLowerCase();
|
|
265
|
+
if (!k) continue;
|
|
266
|
+
var v = kv.length > 1 ? kv.slice(1).join("=").trim() : true;
|
|
267
|
+
// Strip surrounding quotes per the token-or-quoted-string grammar.
|
|
268
|
+
if (typeof v === "string" && v.length >= 2 &&
|
|
269
|
+
v.charAt(0) === '"' && v.charAt(v.length - 1) === '"') {
|
|
270
|
+
v = v.slice(1, -1);
|
|
271
|
+
}
|
|
272
|
+
ext.params[k] = v;
|
|
273
|
+
}
|
|
274
|
+
out.push(ext);
|
|
275
|
+
}
|
|
276
|
+
return out;
|
|
277
|
+
}
|
|
278
|
+
|
|
279
|
+
function _negotiatePermessageDeflate(reqHeader) {
|
|
280
|
+
var entries = _parseExtensionHeader(reqHeader);
|
|
281
|
+
for (var i = 0; i < entries.length; i++) {
|
|
282
|
+
if (entries[i].name !== "permessage-deflate") continue;
|
|
283
|
+
var p = entries[i].params;
|
|
284
|
+
// Reject unknown params (RFC 7692 §7 lists exactly four).
|
|
285
|
+
var KNOWN = {
|
|
286
|
+
"server_no_context_takeover": true, "client_no_context_takeover": true,
|
|
287
|
+
"server_max_window_bits": true, "client_max_window_bits": true,
|
|
288
|
+
};
|
|
289
|
+
var ok = true;
|
|
290
|
+
for (var k in p) { if (Object.prototype.hasOwnProperty.call(p, k) && !KNOWN[k]) { ok = false; break; } }
|
|
291
|
+
if (!ok) continue;
|
|
292
|
+
// Always negotiate WITH no_context_takeover in BOTH directions, so
|
|
293
|
+
// every message uses a fresh zlib state. Echo any client window-
|
|
294
|
+
// bits constraints back unchanged (we honour them on the server's
|
|
295
|
+
// outgoing compression).
|
|
296
|
+
var responseParams = ["client_no_context_takeover", "server_no_context_takeover"];
|
|
297
|
+
if (p.client_max_window_bits && p.client_max_window_bits !== true) {
|
|
298
|
+
responseParams.push("client_max_window_bits=" + p.client_max_window_bits);
|
|
299
|
+
}
|
|
300
|
+
if (p.server_max_window_bits && p.server_max_window_bits !== true) {
|
|
301
|
+
responseParams.push("server_max_window_bits=" + p.server_max_window_bits);
|
|
302
|
+
}
|
|
303
|
+
return {
|
|
304
|
+
negotiated: true,
|
|
305
|
+
responseHeader: "permessage-deflate; " + responseParams.join("; "),
|
|
306
|
+
// window-bits constraints we honour; default 15 (max) when unset.
|
|
307
|
+
serverMaxWindowBits: p.server_max_window_bits && p.server_max_window_bits !== true
|
|
308
|
+
? Math.max(8, Math.min(15, parseInt(p.server_max_window_bits, 10) || 15)) : 15,
|
|
309
|
+
clientMaxWindowBits: p.client_max_window_bits && p.client_max_window_bits !== true
|
|
310
|
+
? Math.max(8, Math.min(15, parseInt(p.client_max_window_bits, 10) || 15)) : 15,
|
|
311
|
+
};
|
|
312
|
+
}
|
|
313
|
+
return { negotiated: false };
|
|
314
|
+
}
|
|
315
|
+
|
|
316
|
+
function _deflateMessage(payload, windowBits) {
|
|
317
|
+
// Per RFC 7692 §7.2.1, strip the 4-byte 0x00 0x00 0xff 0xff trailer.
|
|
318
|
+
var raw = zlib.deflateRawSync(payload, { windowBits: windowBits, level: zlib.constants.Z_DEFAULT_COMPRESSION });
|
|
319
|
+
if (raw.length >= 4 &&
|
|
320
|
+
raw[raw.length - 4] === 0x00 && raw[raw.length - 3] === 0x00 &&
|
|
321
|
+
raw[raw.length - 2] === 0xff && raw[raw.length - 1] === 0xff) {
|
|
322
|
+
return raw.slice(0, raw.length - 4);
|
|
323
|
+
}
|
|
324
|
+
return raw;
|
|
325
|
+
}
|
|
326
|
+
|
|
327
|
+
function _inflateMessage(payload, windowBits) {
|
|
328
|
+
// Per RFC 7692 §7.2.2, append the 4-byte trailer before inflating.
|
|
329
|
+
var withTrailer = Buffer.concat([payload, DEFLATE_TRAILING]);
|
|
330
|
+
return zlib.inflateRawSync(withTrailer, { windowBits: windowBits });
|
|
331
|
+
}
|
|
332
|
+
|
|
234
333
|
// ---- Frame parser ----
|
|
235
334
|
//
|
|
236
335
|
// Incremental — push(chunk) accepts arbitrary buffer slices from the
|
|
@@ -339,6 +438,11 @@ function serializeFrame(opcode, payload, opts) {
|
|
|
339
438
|
opts = opts || {};
|
|
340
439
|
var fin = opts.fin !== false;
|
|
341
440
|
var mask = opts.mask === true; // server-side defaults false
|
|
441
|
+
// RSV1 — set on the first frame of a permessage-deflate-compressed
|
|
442
|
+
// message (RFC 7692). Caller passes opts.rsv1 = true; we wire it
|
|
443
|
+
// into the header byte. RSV2 / RSV3 stay zero (no other extensions
|
|
444
|
+
// negotiated).
|
|
445
|
+
var rsv1 = opts.rsv1 === true;
|
|
342
446
|
payload = payload || Buffer.alloc(0);
|
|
343
447
|
if (typeof payload === "string") payload = Buffer.from(payload, "utf8");
|
|
344
448
|
if (!Buffer.isBuffer(payload)) {
|
|
@@ -355,7 +459,7 @@ function serializeFrame(opcode, payload, opts) {
|
|
|
355
459
|
if (mask) headerLen += 4;
|
|
356
460
|
|
|
357
461
|
var header = Buffer.alloc(headerLen);
|
|
358
|
-
header[0] = (fin ? 0x80 : 0) | (opcode & 0x0F);
|
|
462
|
+
header[0] = (fin ? 0x80 : 0) | (rsv1 ? 0x40 : 0) | (opcode & 0x0F);
|
|
359
463
|
header[1] = (mask ? 0x80 : 0) | lenByte;
|
|
360
464
|
|
|
361
465
|
var off = 2;
|
|
@@ -396,6 +500,10 @@ class WebSocketConnection extends EventEmitter {
|
|
|
396
500
|
// exists to protect against in h1 (proxy
|
|
397
501
|
// cache-poisoning via raw text on the wire).
|
|
398
502
|
this.transport = opts.transport === "h2" ? "h2" : "h1";
|
|
503
|
+
// permessage-deflate state — `null` means extension not negotiated.
|
|
504
|
+
// When negotiated the object carries serverMaxWindowBits +
|
|
505
|
+
// clientMaxWindowBits the inflate/deflate paths use per message.
|
|
506
|
+
this._permessageDeflate = opts.permessageDeflate || null;
|
|
399
507
|
var pingMs = opts.pingIntervalMs || DEFAULT_PING_INTERVAL_MS;
|
|
400
508
|
var pongMs = opts.pongTimeoutMs || DEFAULT_PONG_TIMEOUT_MS;
|
|
401
509
|
// Grace period after we send a close frame before forcing the
|
|
@@ -488,11 +596,20 @@ class WebSocketConnection extends EventEmitter {
|
|
|
488
596
|
return this._abort(CLOSE_PROTOCOL_ERROR, "frame must not be masked (h2)");
|
|
489
597
|
}
|
|
490
598
|
// Reserved bits — must be zero unless a negotiated extension uses them.
|
|
491
|
-
//
|
|
492
|
-
//
|
|
493
|
-
|
|
599
|
+
// RSV1 is permessage-deflate (RFC 7692). RSV2/RSV3 unused; any RSV2
|
|
600
|
+
// or RSV3 bit set, OR RSV1 set when permessage-deflate wasn't
|
|
601
|
+
// negotiated, is a protocol error.
|
|
602
|
+
if (frame.rsv2 || frame.rsv3) {
|
|
494
603
|
return this._abort(CLOSE_PROTOCOL_ERROR, "reserved bits set without extension");
|
|
495
604
|
}
|
|
605
|
+
if (frame.rsv1 && !this._permessageDeflate) {
|
|
606
|
+
return this._abort(CLOSE_PROTOCOL_ERROR, "RSV1 set without permessage-deflate negotiated");
|
|
607
|
+
}
|
|
608
|
+
// RSV1 is only legal on the FIRST frame of a message (TEXT/BINARY).
|
|
609
|
+
// Continuation frames inherit the compression flag from the start.
|
|
610
|
+
if (frame.rsv1 && frame.opcode === OPCODE_CONTINUATION) {
|
|
611
|
+
return this._abort(CLOSE_PROTOCOL_ERROR, "RSV1 on continuation frame (must be on start)");
|
|
612
|
+
}
|
|
496
613
|
|
|
497
614
|
if (frame.opcode === OPCODE_CONTINUATION) {
|
|
498
615
|
if (this._fragOpcode === null) {
|
|
@@ -506,6 +623,7 @@ class WebSocketConnection extends EventEmitter {
|
|
|
506
623
|
this._fragOpcode = frame.opcode;
|
|
507
624
|
this._fragChunks = [frame.payload];
|
|
508
625
|
this._fragLen = frame.payload.length;
|
|
626
|
+
this._fragCompressed = !!frame.rsv1;
|
|
509
627
|
if (frame.fin) this._emitMessage();
|
|
510
628
|
} else if (frame.opcode === OPCODE_CLOSE) {
|
|
511
629
|
this._handleClose(frame);
|
|
@@ -535,9 +653,26 @@ class WebSocketConnection extends EventEmitter {
|
|
|
535
653
|
? this._fragChunks[0]
|
|
536
654
|
: Buffer.concat(this._fragChunks, this._fragLen);
|
|
537
655
|
var opcode = this._fragOpcode;
|
|
656
|
+
var wasCompressed = this._fragCompressed;
|
|
538
657
|
this._fragOpcode = null;
|
|
539
658
|
this._fragChunks = null;
|
|
540
659
|
this._fragLen = 0;
|
|
660
|
+
this._fragCompressed = false;
|
|
661
|
+
// Decompress before emitting if the start frame had RSV1 set.
|
|
662
|
+
// RFC 7692: malformed deflate is a protocol error, surfaced as
|
|
663
|
+
// CLOSE_INVALID_PAYLOAD per §5.6 / §6 of RFC 6455.
|
|
664
|
+
if (wasCompressed) {
|
|
665
|
+
try {
|
|
666
|
+
data = _inflateMessage(data, this._permessageDeflate.clientMaxWindowBits);
|
|
667
|
+
} catch (e) {
|
|
668
|
+
return this._abort(CLOSE_INVALID_PAYLOAD,
|
|
669
|
+
"permessage-deflate inflate failed: " + ((e && e.message) || String(e)));
|
|
670
|
+
}
|
|
671
|
+
if (data.length > this.maxMessageBytes) {
|
|
672
|
+
return this._abort(CLOSE_MESSAGE_TOO_BIG,
|
|
673
|
+
"decompressed message exceeds maxMessageBytes");
|
|
674
|
+
}
|
|
675
|
+
}
|
|
541
676
|
if (opcode === OPCODE_TEXT) {
|
|
542
677
|
// §5.6: text frames MUST be valid UTF-8. Buffer.toString silently
|
|
543
678
|
// replaces invalid sequences with U+FFFD; explicit validation
|
|
@@ -581,7 +716,7 @@ class WebSocketConnection extends EventEmitter {
|
|
|
581
716
|
this._sendFrame(OPCODE_CLOSE, payload);
|
|
582
717
|
}
|
|
583
718
|
|
|
584
|
-
_sendFrame(opcode, payload) {
|
|
719
|
+
_sendFrame(opcode, payload, opts) {
|
|
585
720
|
if (this._state === STATE_CLOSED) return;
|
|
586
721
|
// Socket may have been destroyed by the peer between our last
|
|
587
722
|
// 'close' event check and this write — Node's 'close' event is
|
|
@@ -593,28 +728,48 @@ class WebSocketConnection extends EventEmitter {
|
|
|
593
728
|
return;
|
|
594
729
|
}
|
|
595
730
|
try {
|
|
596
|
-
this.socket.write(serializeFrame(opcode, payload));
|
|
731
|
+
this.socket.write(serializeFrame(opcode, payload, opts));
|
|
597
732
|
} catch (err) {
|
|
598
733
|
this._transitionToClosed(1006, (err && err.message) || "write failed", false, err);
|
|
599
734
|
}
|
|
600
735
|
}
|
|
601
736
|
|
|
737
|
+
_sendDataFrame(opcode, payload) {
|
|
738
|
+
// Compress entire-message-in-one-frame when permessage-deflate
|
|
739
|
+
// negotiated. RSV1 set on the FIRST frame of the message to mark
|
|
740
|
+
// it compressed; opcode-only continuation frames don't repeat
|
|
741
|
+
// RSV1 (see _onFrame's RSV1+continuation guard).
|
|
742
|
+
if (this._permessageDeflate && opcode !== OPCODE_PING &&
|
|
743
|
+
opcode !== OPCODE_PONG && opcode !== OPCODE_CLOSE) {
|
|
744
|
+
try {
|
|
745
|
+
var compressed = _deflateMessage(payload, this._permessageDeflate.serverMaxWindowBits);
|
|
746
|
+
this._sendFrame(opcode, compressed, { rsv1: true });
|
|
747
|
+
return;
|
|
748
|
+
} catch (_e) {
|
|
749
|
+
// Compression failure on send — fall through to uncompressed
|
|
750
|
+
// (we still have the original payload) so the connection
|
|
751
|
+
// keeps working. The underlying issue surfaces as observability.
|
|
752
|
+
}
|
|
753
|
+
}
|
|
754
|
+
this._sendFrame(opcode, payload);
|
|
755
|
+
}
|
|
756
|
+
|
|
602
757
|
send(data) {
|
|
603
758
|
if (this._state !== STATE_OPEN) {
|
|
604
759
|
throw new WebSocketError("ws/closed",
|
|
605
760
|
"connection is " + this._state + ", cannot send");
|
|
606
761
|
}
|
|
607
762
|
if (typeof data === "string") {
|
|
608
|
-
this.
|
|
763
|
+
this._sendDataFrame(OPCODE_TEXT, Buffer.from(data, "utf8"));
|
|
609
764
|
} else if (Buffer.isBuffer(data)) {
|
|
610
|
-
this.
|
|
765
|
+
this._sendDataFrame(OPCODE_BINARY, data);
|
|
611
766
|
} else {
|
|
612
767
|
data = safeBuffer.toBuffer(data, {
|
|
613
768
|
errorClass: WebSocketError,
|
|
614
769
|
typeCode: "ws/invalid-payload",
|
|
615
770
|
typeMessage: "send() requires Buffer, Uint8Array, or string",
|
|
616
771
|
});
|
|
617
|
-
this.
|
|
772
|
+
this._sendDataFrame(OPCODE_BINARY, data);
|
|
618
773
|
}
|
|
619
774
|
}
|
|
620
775
|
|
|
@@ -687,9 +842,20 @@ function handleUpgrade(req, socket, head, opts) {
|
|
|
687
842
|
// Subprotocol negotiation.
|
|
688
843
|
var subprotocol = negotiateSubprotocol(req, opts.subprotocols);
|
|
689
844
|
|
|
845
|
+
// permessage-deflate negotiation. Skipped (no echo header, no
|
|
846
|
+
// compression state on the connection) when the operator passes
|
|
847
|
+
// opts.permessageDeflate = false OR when the client didn't offer it.
|
|
848
|
+
var pmd = null;
|
|
849
|
+
if (opts.permessageDeflate !== false) {
|
|
850
|
+
var negotiated = _negotiatePermessageDeflate(req.headers["sec-websocket-extensions"]);
|
|
851
|
+
if (negotiated.negotiated) pmd = negotiated;
|
|
852
|
+
}
|
|
853
|
+
|
|
690
854
|
// Send 101.
|
|
691
855
|
try {
|
|
692
|
-
socket.write(buildUpgradeResponse(
|
|
856
|
+
socket.write(buildUpgradeResponse(
|
|
857
|
+
req.headers["sec-websocket-key"], subprotocol,
|
|
858
|
+
pmd ? pmd.responseHeader : null));
|
|
693
859
|
} catch (err) {
|
|
694
860
|
log.error("failed to write upgrade response: " + err.message);
|
|
695
861
|
try { socket.destroy(); } catch (_e) {}
|
|
@@ -701,10 +867,11 @@ function handleUpgrade(req, socket, head, opts) {
|
|
|
701
867
|
// parser via a synthetic data event. Most clients don't send
|
|
702
868
|
// anything before the 101 response, but the spec allows it.
|
|
703
869
|
var conn = new WebSocketConnection(socket, {
|
|
704
|
-
subprotocol:
|
|
705
|
-
maxMessageBytes:
|
|
706
|
-
pingIntervalMs:
|
|
707
|
-
pongTimeoutMs:
|
|
870
|
+
subprotocol: subprotocol,
|
|
871
|
+
maxMessageBytes: opts.maxMessageBytes,
|
|
872
|
+
pingIntervalMs: opts.pingIntervalMs,
|
|
873
|
+
pongTimeoutMs: opts.pongTimeoutMs,
|
|
874
|
+
permessageDeflate: pmd,
|
|
708
875
|
});
|
|
709
876
|
if (head && head.length > 0) {
|
|
710
877
|
// Manually invoke the data path with the pre-read bytes.
|
package/package.json
CHANGED
package/sbom.cyclonedx.json
CHANGED
|
@@ -2,10 +2,10 @@
|
|
|
2
2
|
"$schema": "http://cyclonedx.org/schema/bom-1.5.schema.json",
|
|
3
3
|
"bomFormat": "CycloneDX",
|
|
4
4
|
"specVersion": "1.5",
|
|
5
|
-
"serialNumber": "urn:uuid:
|
|
5
|
+
"serialNumber": "urn:uuid:1f3b5321-8232-4a90-a6f9-c43160b7018d",
|
|
6
6
|
"version": 1,
|
|
7
7
|
"metadata": {
|
|
8
|
-
"timestamp": "2026-05-
|
|
8
|
+
"timestamp": "2026-05-02T19:03:09.695Z",
|
|
9
9
|
"lifecycles": [
|
|
10
10
|
{
|
|
11
11
|
"phase": "build"
|
|
@@ -19,14 +19,14 @@
|
|
|
19
19
|
}
|
|
20
20
|
],
|
|
21
21
|
"component": {
|
|
22
|
-
"bom-ref": "@blamejs/core@0.6.
|
|
22
|
+
"bom-ref": "@blamejs/core@0.6.31",
|
|
23
23
|
"type": "library",
|
|
24
24
|
"name": "blamejs",
|
|
25
|
-
"version": "0.6.
|
|
25
|
+
"version": "0.6.31",
|
|
26
26
|
"scope": "required",
|
|
27
27
|
"author": "blamejs contributors",
|
|
28
28
|
"description": "The Node framework that owns its stack.",
|
|
29
|
-
"purl": "pkg:npm/%40blamejs/core@0.6.
|
|
29
|
+
"purl": "pkg:npm/%40blamejs/core@0.6.31",
|
|
30
30
|
"properties": [],
|
|
31
31
|
"externalReferences": [
|
|
32
32
|
{
|
|
@@ -54,7 +54,7 @@
|
|
|
54
54
|
"components": [],
|
|
55
55
|
"dependencies": [
|
|
56
56
|
{
|
|
57
|
-
"ref": "@blamejs/core@0.6.
|
|
57
|
+
"ref": "@blamejs/core@0.6.31",
|
|
58
58
|
"dependsOn": []
|
|
59
59
|
}
|
|
60
60
|
]
|