@blamejs/core 0.15.16 → 0.15.17
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +2 -0
- package/lib/atomic-file.js +32 -9
- package/lib/middleware/api-encrypt.js +105 -40
- package/lib/queue-local.js +123 -46
- package/lib/queue.js +13 -9
- package/lib/self-update-standalone-verifier.js +69 -7
- package/lib/self-update.js +11 -2
- package/lib/vendor/MANIFEST.json +11 -11
- package/lib/vendor/public-suffix-list.dat +6 -2
- package/lib/vendor/public-suffix-list.data.js +689 -688
- package/package.json +1 -1
- package/sbom.cdx.json +6 -6
package/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,8 @@ upgrading across more than a few patches at a time.
|
|
|
8
8
|
|
|
9
9
|
## v0.15.x
|
|
10
10
|
|
|
11
|
+
- v0.15.17 (2026-06-22) — **The job queue runs on a Postgres or MySQL cluster backend, and a set of correctness and confidentiality fixes land for encrypted-API rejections, flow-job ordering, the self-update signature verifier, and bounded file reads.** The local job queue's lease path is now dialect-aware, so the queue works when its store is a Postgres or MySQL cluster backend rather than only single-node SQLite. The previous lease was a single SQLite-only statement: it double-quoted identifiers (which MySQL reads as string literals), updated a table named in its own subquery (MySQL error 1093), and used RETURNING (which MySQL rejects), so the first enqueue or lease against a cluster backend failed outright. The lease now resolves the active backend dialect and composes per-dialect SQL — a transactional SELECT ... FOR UPDATE SKIP LOCKED claim over the head of the queue followed by a guarded UPDATE against a literal id list, returning the claimed rows via RETURNING on Postgres and a re-select on MySQL — with backtick (MySQL) or double-quote (Postgres) identifier quoting, mirroring the framework's outbox claim. enqueue, lease, complete, fail, sweep, and dead-letter all run against the cluster backend, verified end-to-end against live MySQL and Postgres. Alongside the queue work: a request rejected on an established encrypted-API session is now returned inside the session envelope instead of leaking the rejection reason in cleartext; a flow child that declares dependencies can no longer be leased before those dependencies run during the brief window of the two-pass flow enqueue; the standalone self-update verifier classifies an ECDSA signature's encoding by its ASN.1 structure rather than its byte length and fails closed on an indeterminate shape; the self-update poller surfaces each asset's content digest; and the bounded synchronous file reader routes every failure branch through its typed error mapper. **Changed:** *Bundled Public Suffix List refreshed to current upstream* — The vendored Mozilla Public Suffix List — which b.publicSuffix uses to derive organizational domains for DMARC (psd= / np=), BIMI, cookie-scope checks, and same-site policy — is refreshed to the latest upstream revision. Domain classification reflects recent additions and removals to the list. **Fixed:** *The job queue's lease runs dialect-correct SQL on a Postgres or MySQL cluster backend* — When the queue's store is a cluster backend, the lease previously failed at the first enqueue or claim: it was composed as one SQLite-only statement that double-quoted column identifiers (a syntax error on MySQL, which treats double quotes as string literals), updated the jobs table from a subquery that named the same table (MySQL error 1093), and relied on RETURNING (which MySQL does not support). The lease now resolves the active backend dialect and builds per-dialect SQL — a transactional SELECT ... FOR UPDATE SKIP LOCKED over the head of the queue, then a guarded UPDATE ... WHERE status = 'pending' AND id IN (...) with a literal id list (no self-referencing subquery), reading the claimed rows back via RETURNING on Postgres and a re-select by id on MySQL — and quotes identifiers with backticks on MySQL and double quotes on Postgres. enqueue, lease, complete, fail, sweep, and the dead-letter operations now work against a cluster backend, with sealed-at-rest job payloads round-tripping correctly; single-node SQLite continues to use the original single-statement claim. The behavior is verified end-to-end against live MySQL and Postgres. · *Encrypted-API rejections that would disclose session state ride the session envelope* — On a per-session encrypted API channel, a rejection that reveals session state — an expired or rotated session, or a request admitted past the replay gate whose ciphertext then failed authentication — returned its reason as cleartext JSON, disclosing which check failed to a network observer of an otherwise-encrypted channel. Those rejections are now encrypted inside the session envelope under the response counter the server persists for them. The generic 'rejected' refusals that carry no session-lifecycle detail (a stale or duplicate request counter) stay plaintext: they reveal nothing a 400 status does not, and encrypting them would consume a response counter the server does not persist on those paths — desyncing the session's monotonic counter and bricking it for the next genuine request. Pre-session handshake failures (a malformed bootstrap, a stale timestamp, an unknown session) also remain plaintext, since no key exists yet to encrypt under. · *A flow child with dependencies can no longer be leased before its dependencies* — b.queue.enqueueFlow inserts a flow's children in two passes (the second resolves dependency names to the sibling job ids assigned in the first). The first pass previously enqueued every child as immediately leaseable and relied on the second pass to park the dependency-bearing children, leaving a window in which a consumer polling between the two passes could lease a child before the jobs it depends on had run. A dependency-bearing child is now parked at enqueue time on the first pass, so it is never leaseable in that window; the second pass only rewrites its dependency names to the resolved job ids. Children with no dependencies remain immediately leaseable. · *The self-update poller carries each asset's content digest* — b.selfUpdate.poll() now includes the content digest for each asset and its detached signature in the result it returns, so a caller can verify a downloaded asset against the digest advertised by the release feed. · *Bounded file reads always return the typed error shape* — The bounded synchronous file reader could surface an unmapped error on a few failure branches — a refused symlink whose target had since been removed, a short read, an integrity mismatch — rather than the typed error its callers expect. Every failure branch now passes through the reader's error mapper, so a caller always receives the documented error kind. **Security:** *The standalone self-update verifier detects ECDSA signature encoding by structure* — The standalone update-signature verifier inferred whether an ECDSA signature was DER- or IEEE-P1363-encoded from its byte length — a heuristic two distinct encodings can collide on, which could lead a valid-but-misclassified signature to be parsed under the wrong scheme. It now parses the signature's ASN.1 structure to classify the encoding and fails closed on an indeterminate shape rather than guessing.
|
|
12
|
+
|
|
11
13
|
- v0.15.16 (2026-06-22) — **A correctness and hardening release: a broad set of credential and certificate verifiers now fail closed on a present-but-unparseable timestamp instead of silently skipping the check, outbound network clients gain an overall wall-clock timeout and a bounded framing buffer, DMARC refuses a record with no usable policy, append-only log sinks refuse a symlinked path, and the interactive-transaction and replay-store contracts refuse a backend that cannot honor them.** This release closes a family of fail-open and resource-exhaustion gaps without adding opt-ins to the request path. A recurring pattern across the verifiers — a timestamp parsed with a lenient parser, then gated by isFinite() so an unparseable value disables the check — is converted to fail-closed everywhere it appears: DKIM x=/t=/l=, ARC AMS/AS t=/x=, SAML Bearer and holder-of-key NotBefore, and the FIDO MDS3 / BIMI / S/MIME certificate validity windows now refuse a malformed value rather than accepting the credential. DMARC now validates and requires a usable p= policy and never recommends delivery for a failing message with an absent or unrecognized policy. Outbound clients that exposed a single timeout but applied it only as an idle (zero-progress) timer now also enforce it as an overall wall-clock cap, so a peer that trickles bytes within the idle window can no longer hold a request open indefinitely: the encrypted HTTP client, the object-store and SQS request paths, the CloudWatch / OTLP / webhook log sinks, and the password breach check. The DNS resolver and the lower-level DNS client gain a per-query wall-clock deadline that also tears the socket down, the SMTP client and the proxy CONNECT tunnel bound their framing buffers and add an overall deadline, and the NTS key-establishment handshake bounds its accumulator. The append-only local log sink and the daemon log open with O_NOFOLLOW so a symlink planted at the path is refused rather than followed. b.externalDb.transaction and b.outbox now refuse a backend that declares it cannot provide an interactive transaction instead of silently running a non-atomic block, and the GraphQL-federation replay store adopts the framework's atomic check-and-insert contract. **Added:** *b.atomicFile.openAppendNoFollowSync — symlink-refusing append open* — Opens a long-lived append target (an active log file kept open across appends and reopened on rotation) with O_WRONLY | O_APPEND | O_CREAT | O_NOFOLLOW: the file is created or appended normally, but a symlink at the final path component fails the open closed (ELOOP) rather than redirecting writes. The append-sink counterpart to openNoFollowSync (read) and the exclusive-create temp open. · *b.externalDb.supportsTransactions() and stateless-adapter declaration* — A backend adapter may declare supportsTransactions: false (a stateless / autocommit-per-statement adapter) and/or provide a batch(client, statements) hook for an atomic multi-statement path. b.externalDb.supportsTransactions() reports whether the picked backend can provide an interactive transaction, so a consumer built on the dual-write guarantee can refuse a non-atomic backend up front. · *Store-free device fingerprint* — b.sessionDeviceBinding.fingerprint(req, opts) is now a static entry point returning the request-shape digest with no store, and b.sessionDeviceBinding.create() with neither a bindingStore nor storeInSession returns an instance whose stateless fingerprint() works while bind()/verify() throw a clear "no store configured". Soft device binding for self-validating tokens (a sealed cookie or JWT carrying the fingerprint) no longer needs a fabricated no-op store. **Changed:** *Device fingerprint IP masking is canonical* — b.sessionDeviceBinding masked the client IP into its prefix bucket by textual group slicing with no IPv6 normalization, so equivalent forms of one address (a ::-compressed form vs its fully-expanded form) hashed to different fingerprints and a roaming client could be logged out on a false drift. It now masks through the same canonical prefix helper b.session uses, preserving the configured prefix width (the IPv6 default stays /48, and ipPrefixBits is honored). b.requestHelpers.ipPrefix accepts { v4Bits, v6Bits } to override the /24 + /64 default for a function-form fingerprint or other reuse. **Fixed:** *b.session.destroyAllForUser works for pluggable-store consumers* — A consumer configuring sessions through b.session.useStore without calling b.db.init() got a db/not-initialized error from destroyAllForUser — every "log out everywhere" / suspend / delete path rejected with a 500 after the store rows had already been deleted, because the stateless valid-from boundary write was routed only to the framework database. The boundary now prefers the framework database when one is initialized and otherwise falls back to the configured session store (provisioning its table on demand), so store-backed deployments raise the stateless boundary successfully; it still fails closed when neither a framework database nor a store is available. **Security:** *Certificate and credential verifiers fail closed on an unparseable timestamp* — A present-but-unparseable date no longer silently disables a validity check. DKIM x= / t= (signature expiry, future-date, ordering) and l= (body-length cap), ARC AMS/AS t= / x=, SAML Bearer and holder-of-key SubjectConfirmationData NotBefore, and the FIDO MDS3, BIMI, and S/MIME certificate validity windows previously parsed the value, then guarded the comparison with isFinite() — so a value that did not parse (NaN) skipped the check and the credential was accepted. Each now refuses a present-but-unparseable value (a permerror / not-valid result) and only an absent optional field is ignored, matching the existing fail-closed handling of the SAML Conditions block. · *DMARC refuses a record with no usable policy* — The DMARC policy tag p= (and the subdomain sp=) were taken verbatim with no validation and p= was never required, so the disposition mapped a null or typo'd policy to the catch-all "deliver". A failing, unaligned message against a domain whose record omitted p= or carried an unrecognized value was therefore delivered. p= and sp= are now validated against none|quarantine|reject, p= is required for a record to carry a policy at all, an invalid record is a permerror (not a transient temperror), and the disposition is an explicit fail-closed mapping that never delivers a failing message on an absent or unrecognized policy. · *Outbound clients enforce an overall wall-clock timeout, not only an idle timeout* — Several clients exposed a single timeout but forwarded it to the underlying request only as idleTimeoutMs (a zero-progress cap that resets on every received byte), leaving no overall bound — a peer that trickles one byte inside each idle window holds the call open indefinitely. b.httpClient.encrypted (which dropped timeoutMs entirely, and now also forwards the caller's AbortSignal), the object-store request path and Azure/GCS bucket operations, the SQS queue client, the CloudWatch / OTLP / webhook log sinks, and the password HIBP breach check now set the overall wall-clock timeout alongside the idle timeout. · *DNS lookups gain a default wall-clock deadline and tear the socket down* — The DNS resolver's DoH transport had no time bound of any kind and the lower-level DNS client defaulted its lookup timeout to zero (no deadline). A non-responsive or slow-trickle upstream — reachable while resolving DKIM/SPF/BIMI records for inbound mail — held the lookup pending forever. The resolver now takes a per-query timeout (default 10s) wrapping every query and CNAME hop, the DNS client defaults its lookup timeout to 10s (operators opt out with a zero timeout), and the raw DoH/DoT request paths arm a socket timeout that destroys the connection on a stall rather than leaking the descriptor. · *SMTP, proxy-CONNECT, and NTS clients bound their read buffers and add deadlines* — The SMTP client accumulated the server's reply without a byte cap and relied on an idle timer alone; a hostile or broken MX that never sent a line terminator could exhaust memory, and a slow trickle held the transaction open. It now caps the accumulated reply (refusing once it exceeds the response ceiling) and enforces an overall transaction deadline. The proxy CONNECT-tunnel client, which accumulated the proxy's reply unbounded with no socket timeout, now caps the header buffer and times out the connect. The NTS key-establishment handshake reader caps its record accumulator so a server streaming non-terminating records cannot exhaust memory before the handshake timer fires. · *Append-only log sinks refuse a symlinked path* — The local log sink and the daemon log opened the active file in append mode without O_NOFOLLOW, so a symlink planted at the log path (including one re-planted in the race window after a caller's pre-check) was followed, redirecting log writes to an attacker-chosen file. Both now open with O_NOFOLLOW via the new b.atomicFile.openAppendNoFollowSync, refusing a symlinked final path component. · *Transaction and replay-store contracts refuse a backend that cannot honor them* — b.externalDb.transaction and b.outbox.create now refuse a backend that declares it cannot provide an interactive transaction (a stateless / autocommit-per-statement adapter) instead of silently running BEGIN, the body, and COMMIT on different sessions — no isolation, no rollback — while reporting success. b.graphqlFederation.guardSdl now consults its replay nonce store through the framework's atomic checkAndInsert(nonce, expireAt) contract (the same one b.webhook and b.nonceStore use) rather than a non-atomic has()-then-remember() check that raced concurrent redeliveries; a store lacking checkAndInsert is refused at construction. **Migration:** *DNS lookups now time out by default* — The lower-level DNS client's lookup timeout previously defaulted to zero (no deadline); it now defaults to 10 seconds. A deployment that intentionally ran DNS lookups with no wall-clock bound must set the lookup timeout to zero explicitly to restore the old behavior. The b.network.dns.resolver default timeout is also 10 seconds and is configurable via its timeoutMs option. · *b.graphqlFederation.guardSdl nonceStore contract* — guardSdl's optional replay nonceStore now requires the atomic checkAndInsert(nonce, expireAt) contract (b.nonceStore.create is the reference store) and refuses the previous { has, remember } shape at construction. Operators using the old shape should pass b.nonceStore.create(...) or an adapter exposing checkAndInsert. · *Interactive transactions on a stateless backend are refused* — If an externalDb backend adapter declares supportsTransactions: false, b.externalDb.transaction and b.outbox.create now throw rather than running a silently non-atomic block. Existing stateful adapters (Postgres / MySQL / SQLite, and any adapter that supplies beginTx/commit/rollback or omits the flag) are unaffected. A stateless / autocommit-per-statement adapter should supply interactive transaction hooks or a batch adapter.
|
|
12
14
|
|
|
13
15
|
- v0.15.15 (2026-06-21) — **A focused correctness and hardening release: more authentication and signature verifiers fail closed (JWT issuer, SAML recipient binding, OAuth nonce, CIBA token binding, FIDO certification level), the WebSocket client bounds a fragmented message before it completes, DMARC resolves the From domain before the policy lookup, and the CSP builder refuses a directive-injecting source; fixes a middleware pipeline promise that never settled on a write-and-halt, and exposes the X.509 CA-bit issuer test, an IP-prefix helper, and a reverse-proxy-aware session fingerprint option.** This release continues to tighten existing protections without adding opt-ins on the request path. A further set of authentication and signature verifiers now refuse malformed or unbound credentials: JWT verify rejects an array-valued issuer claim (an any-match bypass, CVE-2025-30144 class), SAML requires the mandatory Recipient on a Bearer or holder-of-key confirmation and enforces every AudienceRestriction, OAuth no longer skips the ID-token nonce check on an empty nonce, CIBA binds the returned ID token to its auth_req_id, the FIDO MDS3 certification level reflects the authenticator's current status rather than its historical maximum, and a JOSE header that decodes to a non-object is a typed error rather than an uncaught throw. The WebSocket client now bounds a fragmented incoming message by its running total instead of only at the final frame, DMARC validates the From-header domain before the policy lookup, the data-residency write gate compares the residency column case-insensitively, and the SQL builder refuses an equality against NULL and validates every IN-list element. The CSP builder refuses a source token containing whitespace or a semicolon (directive injection). A middleware pipeline that a request handler halted by writing the response without calling next() left its promise pending forever, retaining the request closure — it now settles. New surface is additive and backward compatible: the basicConstraints-enforcing X.509 issuer test is exposed publicly, the session device fingerprint can resolve the client IP through a trusted-proxy allowlist, and the /24+/64 IP-prefix masking it uses is exposed for reuse. **Added:** *b.x509Chain — the CA-bit-enforcing X.509 issuer test* — isCaCert(cert) and issuerValidlyIssued(issuer, subject) are now public. Node's X509Certificate.checkIssued() does not enforce the basicConstraints cA bit (CVE-2002-0862 class), so a non-CA leaf can be accepted as an issuer; these helpers require cA:TRUE and verify the signature, fail closed on any malformed input, and are the same test the framework's own chain walkers use. A consumer validating a chain outside a TLS handshake (an operator-uploaded CA bundle, a non-handshake PQ-signed certificate) can now use the hardened path instead of raw checkIssued(). · *Reverse-proxy-aware session device fingerprint* — b.session.create / verify / rotate accept { trustedProxies } (CIDRs) or { clientIpResolver } so the built-in clientIp / clientIpPrefix fingerprint fields resolve the real client IP behind a trusted proxy, consistent with b.requestHelpers.trustedClientIp. Without the option the fingerprint still binds to the bare socket peer (unchanged default), so existing fingerprints keep matching; pass the same option to create, verify, and rotate. The /24 (IPv4) + /64 (IPv6) subnet masking is exposed as b.requestHelpers.ipPrefix for an operator using a function-form fingerprint field. **Changed:** *DKIM l= is counted over the canonicalized body* — On verify, the l= body-length tag was applied to the raw body before canonicalization, so a legitimate relaxed/relaxed signature whose l= matched the canonicalized length was rejected with a body-hash mismatch whenever relaxed canonicalization changed the byte count within the first l= octets. The body is now canonicalized first and the canonicalized octet stream truncated to l= (RFC 6376 §3.4.5). · *PIPL cross-border security-assessment threshold documentation corrected* — The b.pipl.sccFilingAssessment contract stated the CAC security assessment becomes mandatory above 100,000 cumulative PI subjects (the superseded 2022 figure). The builder enforces the current CAC Provisions on Promoting and Regulating Cross-Border Data Flows — a security assessment above 1,000,000 non-sensitive PI subjects or 10,000 sensitive, with the 100,000–1,000,000 band in the standard-contract / certification tier. The documentation now matches the enforced thresholds. **Fixed:** *composePipeline settles its promise when a middleware halts* — A regular middleware that wrote the response and returned without calling next() — the intended way to halt the chain from an auth / rate-limit / bot block — left the promise returned by b.middleware.composePipeline pending forever, retaining its request/response closure (an unbounded leak under sustained blocked traffic). The halt path now settles the promise without advancing to the route handler; the same applies to an error handler that consumes the error without calling next(). · *Quality-list header parsing is quote-aware* — parseQualityList mis-read a q= substring that appeared inside a quoted media-range or Accept-* parameter, mis-weighting content negotiation. It now parses the q parameter with quote-aware splitting. **Security:** *More authentication and signature verifiers fail closed* — JWT verify rejects an array-valued iss claim (it had passed through a generic any-match built for the legitimately-array aud claim, so a multi-issuer array satisfied a single-issuer expectation — CVE-2025-30144 class), matching the external JWT and OIDC verifiers. SAML verifyResponse requires the Recipient attribute on a Bearer or holder-of-key SubjectConfirmation and confirms it equals the ACS URL (an absent Recipient was silently skipped), and enforces every AudienceRestriction (AND-combined per SAML core). OAuth no longer fails open and skips the ID-token nonce replay check when the supplied nonce is empty. CIBA binds the polled / notified ID token to its auth_req_id so a token minted for another request can't be accepted. The FIDO MDS3 certification level reflects the authenticator's current status, not the highest level it ever held, so a later decertification or downgrade is honored by a step-up policy. A JOSE header or payload that decodes to JSON null or a non-object now raises a typed authentication error instead of an uncaught TypeError. · *WebSocket fragmented-message reassembly is bounded before completion* — The client enforced maxMessageBytes only when a fragmented message's final frame arrived, so a peer could stream unbounded continuation frames and exhaust memory before the limit was ever checked. The running reassembly total is now enforced per frame, and a frame arriving after close is ignored. · *DMARC resolves the From domain before the policy lookup* — The From-header bare addr-spec domain extraction did not reject RFC 5322 group syntax or a trailing semicolon, so a crafted From header could yield a corrupted domain that defeated the DMARC alignment / policy lookup. The domain is now validated (length-bounded, rejecting the group and punctuation forms) before it is used. · *Data-residency and SQL builder correctness* — The per-row data-residency write gate compares the residency column name case-insensitively, so a raw UPDATE that spells the column in a different letter case can no longer slip the cross-border transfer check. The SQL builder refuses an equality against NULL (col = NULL is always false; use IS NULL) and validates every element of an IN-list on the Postgres = ANY(?) path, matching the sqlite / MySQL path. · *CSP builder refuses a directive-injecting source* — A CSP source is a single non-whitespace token. build() and mergeDirectives() now reject a source containing whitespace or a semicolon — because the emitter space-joins sources and semicolon-joins directives, a value like "https://x; script-src https://evil" injected a live directive.
|
package/lib/atomic-file.js
CHANGED
|
@@ -895,6 +895,19 @@ function fdSafeReadSync(filepath, opts) {
|
|
|
895
895
|
var errorFor = opts.errorFor || function (kind, detail) {
|
|
896
896
|
return new AtomicFileError((detail && detail.message) || ("atomic-file: " + kind), "atomic-file/" + kind);
|
|
897
897
|
};
|
|
898
|
+
// Every failure KIND routes through this one mapper-then-fallback gate: throw
|
|
899
|
+
// the caller's typed error when errorFor returns truthy, else the fallback —
|
|
900
|
+
// the raw OS error for enoent (preserving the rethrow-raw posture network-tls
|
|
901
|
+
// / vault / backup rely on), or a synthetic AtomicFileError for the kinds with
|
|
902
|
+
// no underlying OS error to rethrow. An errorFor that returns undefined for a
|
|
903
|
+
// kind must never make us throw the literal `undefined` (the openSync/enoent
|
|
904
|
+
// branch already guarded this; symlink / too-large / toctou / short-read /
|
|
905
|
+
// integrity now do too — #358).
|
|
906
|
+
function _raise(kind, detail, fallbackErr) {
|
|
907
|
+
var typed = errorFor(kind, detail);
|
|
908
|
+
throw typed || fallbackErr ||
|
|
909
|
+
new AtomicFileError((detail && detail.message) || ("atomic-file: " + kind), "atomic-file/" + kind);
|
|
910
|
+
}
|
|
898
911
|
if (opts.maxBytes !== undefined) _validateMaxBytes(opts.maxBytes);
|
|
899
912
|
var mode = opts.mode === undefined ? 0o600 : opts.mode;
|
|
900
913
|
// refuseSymlink: lstat the path first and refuse a symlink source —
|
|
@@ -902,10 +915,21 @@ function fdSafeReadSync(filepath, opts) {
|
|
|
902
915
|
// fd's inode is re-checked against this lstat's inode below.
|
|
903
916
|
var lstat = null;
|
|
904
917
|
if (opts.refuseSymlink) {
|
|
905
|
-
|
|
906
|
-
|
|
918
|
+
try {
|
|
919
|
+
lstat = nodeFs.lstatSync(filepath);
|
|
920
|
+
} catch (lstatErr) {
|
|
921
|
+
// A missing file makes lstat throw raw ENOENT BEFORE the openSync branch
|
|
922
|
+
// that consults errorFor("enoent"). Route it through the same mapping so
|
|
923
|
+
// the missing-file error class is identical with or without refuseSymlink
|
|
924
|
+
// (#358); any other lstat error rethrows raw.
|
|
925
|
+
if (lstatErr && lstatErr.code === "ENOENT") {
|
|
926
|
+
_raise("enoent", { path: filepath, cause: lstatErr }, lstatErr);
|
|
927
|
+
}
|
|
928
|
+
throw lstatErr;
|
|
929
|
+
}
|
|
930
|
+
if (lstat.isSymbolicLink()) _raise("symlink", { path: filepath });
|
|
907
931
|
if (opts.maxBytes !== undefined && lstat.size > opts.maxBytes) {
|
|
908
|
-
|
|
932
|
+
_raise("too-large", { size: lstat.size, max: opts.maxBytes });
|
|
909
933
|
}
|
|
910
934
|
}
|
|
911
935
|
// The third argument pins an owner-only mode (0o600 default). The flag
|
|
@@ -918,8 +942,7 @@ function fdSafeReadSync(filepath, opts) {
|
|
|
918
942
|
fd = nodeFs.openSync(filepath, "r", mode);
|
|
919
943
|
} catch (openErr) {
|
|
920
944
|
if (openErr && openErr.code === "ENOENT") {
|
|
921
|
-
|
|
922
|
-
if (typed) throw typed;
|
|
945
|
+
_raise("enoent", { path: filepath, cause: openErr }, openErr);
|
|
923
946
|
}
|
|
924
947
|
throw openErr;
|
|
925
948
|
}
|
|
@@ -933,10 +956,10 @@ function fdSafeReadSync(filepath, opts) {
|
|
|
933
956
|
// plain (no-inodeCheck) reader reports an over-cap as too-large.
|
|
934
957
|
if (lstat && opts.inodeCheck) {
|
|
935
958
|
if (fstat.ino !== lstat.ino || (opts.maxBytes !== undefined && fstat.size > opts.maxBytes)) {
|
|
936
|
-
|
|
959
|
+
_raise("toctou", { path: filepath });
|
|
937
960
|
}
|
|
938
961
|
} else if (opts.maxBytes !== undefined && fstat.size > opts.maxBytes) {
|
|
939
|
-
|
|
962
|
+
_raise("too-large", { size: fstat.size, max: opts.maxBytes });
|
|
940
963
|
}
|
|
941
964
|
buf = Buffer.alloc(fstat.size);
|
|
942
965
|
var read = 0;
|
|
@@ -947,7 +970,7 @@ function fdSafeReadSync(filepath, opts) {
|
|
|
947
970
|
}
|
|
948
971
|
if (read !== fstat.size) {
|
|
949
972
|
if (opts.allowShortRead) { buf = buf.slice(0, read); }
|
|
950
|
-
else {
|
|
973
|
+
else { _raise("short-read", { read: read, size: fstat.size }); }
|
|
951
974
|
}
|
|
952
975
|
} finally {
|
|
953
976
|
try { nodeFs.closeSync(fd); } catch (_c) { /* close best-effort */ }
|
|
@@ -955,7 +978,7 @@ function fdSafeReadSync(filepath, opts) {
|
|
|
955
978
|
if (opts.expectedHash) {
|
|
956
979
|
var actual = sha3Hash(buf);
|
|
957
980
|
if (actual !== opts.expectedHash) {
|
|
958
|
-
|
|
981
|
+
_raise("integrity", { expected: opts.expectedHash, actual: actual });
|
|
959
982
|
}
|
|
960
983
|
}
|
|
961
984
|
var content = opts.encoding ? buf.toString(opts.encoding) : buf;
|
|
@@ -223,12 +223,39 @@ function _validSid(sid) {
|
|
|
223
223
|
SID_RE.test(sid);
|
|
224
224
|
}
|
|
225
225
|
|
|
226
|
-
|
|
226
|
+
// _writeRejection — emit a protocol-level rejection body.
|
|
227
|
+
//
|
|
228
|
+
// On an ESTABLISHED per-session encrypted channel the rejection MUST
|
|
229
|
+
// travel inside the session envelope, exactly like a successful
|
|
230
|
+
// response: a client on a keyed channel that sends a stale / replayed /
|
|
231
|
+
// malformed request would otherwise learn, in cleartext, which check
|
|
232
|
+
// failed over an otherwise-encrypted channel. The middleware stamps
|
|
233
|
+
// `req.apiEncryptRejectEncode` the moment a session is resolved with a
|
|
234
|
+
// valid session key; when present, the body is wrapped through the same
|
|
235
|
+
// response-encryption path successful responses use. Absent it (a
|
|
236
|
+
// pre-session handshake error, where no session context exists yet, or
|
|
237
|
+
// per-request mode) the body falls back to plaintext.
|
|
238
|
+
function _writeRejection(req, res, code, body, opts) {
|
|
227
239
|
if (res.headersSent || res.writableEnded) return;
|
|
228
|
-
if (typeof res.writeHead
|
|
229
|
-
|
|
230
|
-
|
|
240
|
+
if (typeof res.writeHead !== "function") return;
|
|
241
|
+
var out = body;
|
|
242
|
+
// opts.plaintext forces a cleartext body even on an established channel.
|
|
243
|
+
// Used for the generic "encrypted-payload-rejected" refusals that DO NOT
|
|
244
|
+
// delete the session (the monotonic-counter replay and the atomic-claim
|
|
245
|
+
// loss): riding those on the session envelope would emit a response _ctr
|
|
246
|
+
// the client tracks as consumed, but those paths return before the
|
|
247
|
+
// server persists responsesEmitted — so the next genuine response reuses
|
|
248
|
+
// that _ctr and the client refuses it as a replay, bricking the session.
|
|
249
|
+
// The body is already generic (no session-lifecycle reason leaks the way
|
|
250
|
+
// session-expired / rotation-required would), so plaintext here costs no
|
|
251
|
+
// meaningful confidentiality.
|
|
252
|
+
var encode = (!opts || !opts.plaintext) && req && req.apiEncryptRejectEncode;
|
|
253
|
+
if (typeof encode === "function") {
|
|
254
|
+
try { out = encode(body); }
|
|
255
|
+
catch (_e) { out = body; } // encryption failed → fall back to plaintext rather than hang
|
|
231
256
|
}
|
|
257
|
+
res.writeHead(code, { "Content-Type": "application/json" });
|
|
258
|
+
res.end(JSON.stringify(out));
|
|
232
259
|
}
|
|
233
260
|
|
|
234
261
|
// ---- Server-side middleware ----
|
|
@@ -458,6 +485,26 @@ function create(opts) {
|
|
|
458
485
|
return encrypted;
|
|
459
486
|
}
|
|
460
487
|
|
|
488
|
+
// _installRejectEncoder — stamp req with a function that wraps a
|
|
489
|
+
// protocol-level rejection body in the SAME session envelope a
|
|
490
|
+
// successful response uses, so an error emitted on an established
|
|
491
|
+
// per-session encrypted channel does not leak (in cleartext) which
|
|
492
|
+
// check the request tripped. Called the moment a session is resolved
|
|
493
|
+
// with a valid session key, BEFORE the expiry / rotation / replay
|
|
494
|
+
// gates that fire on a keyed channel. The rejection rides a fresh
|
|
495
|
+
// response counter (sid bound, strictly above the session's last
|
|
496
|
+
// emitted response) so the client's monotonic _ctr check still holds.
|
|
497
|
+
// The session-deleting rejections (expired / rotation) and the
|
|
498
|
+
// post-claim tag-mismatch ride this encoder; the two generic surviving
|
|
499
|
+
// rejections (monotonic-counter replay, atomic-claim loss) opt out via
|
|
500
|
+
// _writeRejection({ plaintext: true }) because they return before the
|
|
501
|
+
// consumed counter is persisted — see _writeRejection.
|
|
502
|
+
function _installRejectEncoder(req, sessionKey, sid, responseCtr) {
|
|
503
|
+
req.apiEncryptRejectEncode = function (body) {
|
|
504
|
+
return _encodeEnvelope(body, sessionKey, { sid: sid, responseCtr: responseCtr });
|
|
505
|
+
};
|
|
506
|
+
}
|
|
507
|
+
|
|
461
508
|
// _wrapResJson — install res.json that encrypts the response with the
|
|
462
509
|
// session key. In per-request mode the response is `{ _ct }`; in
|
|
463
510
|
// per-session mode it carries `{ _ct, _sid, _ctr }` so the client can
|
|
@@ -507,18 +554,18 @@ function create(opts) {
|
|
|
507
554
|
var body = req.body;
|
|
508
555
|
if (!body || typeof body !== "object") {
|
|
509
556
|
_emitFailure(req, "shape");
|
|
510
|
-
return _writeRejection(res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-required" });
|
|
557
|
+
return _writeRejection(req, res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-required" });
|
|
511
558
|
}
|
|
512
559
|
|
|
513
560
|
var now = Date.now();
|
|
514
561
|
var ct = body._ct, ts = body._ts;
|
|
515
562
|
if (typeof ct !== "string" || typeof ts !== "number") {
|
|
516
563
|
_emitFailure(req, "shape");
|
|
517
|
-
return _writeRejection(res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-required" });
|
|
564
|
+
return _writeRejection(req, res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-required" });
|
|
518
565
|
}
|
|
519
566
|
if (Math.abs(now - ts) > replayWindowMs) {
|
|
520
567
|
_emitFailure(req, "stale");
|
|
521
|
-
return _writeRejection(res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-rejected" });
|
|
568
|
+
return _writeRejection(req, res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-rejected" });
|
|
522
569
|
}
|
|
523
570
|
|
|
524
571
|
// Per-request OR per-session bootstrap path: shape includes _ek + _nonce.
|
|
@@ -540,25 +587,25 @@ function create(opts) {
|
|
|
540
587
|
try { freshNonce = await store.checkAndInsert(nonceHash, expireAt); }
|
|
541
588
|
catch (_e) {
|
|
542
589
|
_emitFailure(req, "nonce-store-error");
|
|
543
|
-
return _writeRejection(res, HTTP_STATUS.INTERNAL_SERVER_ERROR, { error: "nonce-store-unavailable" });
|
|
590
|
+
return _writeRejection(req, res, HTTP_STATUS.INTERNAL_SERVER_ERROR, { error: "nonce-store-unavailable" });
|
|
544
591
|
}
|
|
545
592
|
if (!freshNonce) {
|
|
546
593
|
_emitFailure(req, "replay");
|
|
547
|
-
return _writeRejection(res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-rejected" });
|
|
594
|
+
return _writeRejection(req, res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-rejected" });
|
|
548
595
|
}
|
|
549
596
|
sessionKey = _decryptEkToSessionKey(ek);
|
|
550
597
|
if (!sessionKey) {
|
|
551
598
|
_emitFailure(req, "tag");
|
|
552
|
-
return _writeRejection(res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-rejected" });
|
|
599
|
+
return _writeRejection(req, res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-rejected" });
|
|
553
600
|
}
|
|
554
601
|
if (keying === "per-session") {
|
|
555
602
|
if (!_validSid(sid)) {
|
|
556
603
|
_emitFailure(req, "shape");
|
|
557
|
-
return _writeRejection(res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-required" });
|
|
604
|
+
return _writeRejection(req, res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-required" });
|
|
558
605
|
}
|
|
559
606
|
if (!numericBounds.isNonNegativeFiniteInt(ctr)) {
|
|
560
607
|
_emitFailure(req, "shape");
|
|
561
|
-
return _writeRejection(res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-required" });
|
|
608
|
+
return _writeRejection(req, res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-required" });
|
|
562
609
|
}
|
|
563
610
|
// Bootstrap a new session row keyed by sid. responsesEmitted is
|
|
564
611
|
// set to 1 (this bootstrap emits one response) BEFORE the store
|
|
@@ -579,7 +626,7 @@ function create(opts) {
|
|
|
579
626
|
try { await sessionStore.set(sid, session, { ttlMs: sessionTtlMs }); }
|
|
580
627
|
catch (_e) {
|
|
581
628
|
_emitFailure(req, "session-store-error");
|
|
582
|
-
return _writeRejection(res, HTTP_STATUS.INTERNAL_SERVER_ERROR, { error: "session-store-unavailable" });
|
|
629
|
+
return _writeRejection(req, res, HTTP_STATUS.INTERNAL_SERVER_ERROR, { error: "session-store-unavailable" });
|
|
583
630
|
}
|
|
584
631
|
_emitObs("apiEncrypt.session.created", 1, { mode: "per-session" });
|
|
585
632
|
_emitSessionAudit("apiEncrypt.session.created", {
|
|
@@ -588,28 +635,56 @@ function create(opts) {
|
|
|
588
635
|
requestId: req.requestId || null,
|
|
589
636
|
});
|
|
590
637
|
sessionCtx = { sid: sid, responseCtr: 1 };
|
|
638
|
+
// Session now established — a post-bootstrap rejection (e.g. the
|
|
639
|
+
// final decrypt below) rides the session envelope under the same
|
|
640
|
+
// counter the success response would have used.
|
|
641
|
+
_installRejectEncoder(req, sessionKey, sid, 1);
|
|
591
642
|
}
|
|
592
643
|
} else if (keying === "per-session" &&
|
|
593
644
|
typeof sid === "string" && typeof ctr === "number") {
|
|
594
645
|
// ---- Per-session subsequent-request path ----
|
|
595
646
|
if (!_validSid(sid)) {
|
|
596
647
|
_emitFailure(req, "shape");
|
|
597
|
-
return _writeRejection(res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-required" });
|
|
648
|
+
return _writeRejection(req, res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-required" });
|
|
598
649
|
}
|
|
599
650
|
if (!numericBounds.isNonNegativeFiniteInt(ctr)) {
|
|
600
651
|
_emitFailure(req, "shape");
|
|
601
|
-
return _writeRejection(res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-required" });
|
|
652
|
+
return _writeRejection(req, res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-required" });
|
|
602
653
|
}
|
|
603
654
|
try { session = await sessionStore.get(sid); }
|
|
604
655
|
catch (_e) {
|
|
605
656
|
_emitFailure(req, "session-store-error");
|
|
606
|
-
return _writeRejection(res, HTTP_STATUS.INTERNAL_SERVER_ERROR, { error: "session-store-unavailable" });
|
|
657
|
+
return _writeRejection(req, res, HTTP_STATUS.INTERNAL_SERVER_ERROR, { error: "session-store-unavailable" });
|
|
607
658
|
}
|
|
608
659
|
if (!session) {
|
|
609
660
|
_emitObs("apiEncrypt.session.unknown", 1, {});
|
|
610
661
|
_emitFailure(req, "session-unknown");
|
|
611
|
-
return _writeRejection(res, HTTP_STATUS.UNAUTHORIZED, { error: "session-unknown" });
|
|
662
|
+
return _writeRejection(req, res, HTTP_STATUS.UNAUTHORIZED, { error: "session-unknown" });
|
|
663
|
+
}
|
|
664
|
+
// Recover + coerce the session key BEFORE the expiry / rotation /
|
|
665
|
+
// replay gates so a rejection on this established channel can ride
|
|
666
|
+
// the session envelope (the channel is keyed the moment the sid
|
|
667
|
+
// resolves to a stored session, even if THIS request is then
|
|
668
|
+
// refused). An operator store may have JSON-serialised the buffer.
|
|
669
|
+
sessionKey = session.sessionKey;
|
|
670
|
+
if (Buffer.isBuffer(sessionKey) === false) {
|
|
671
|
+
if (typeof sessionKey === "string") {
|
|
672
|
+
sessionKey = Buffer.from(sessionKey, "base64");
|
|
673
|
+
} else if (sessionKey && sessionKey.type === "Buffer" && Array.isArray(sessionKey.data)) {
|
|
674
|
+
sessionKey = Buffer.from(sessionKey.data);
|
|
675
|
+
} else if (sessionKey instanceof Uint8Array) {
|
|
676
|
+
sessionKey = Buffer.from(sessionKey);
|
|
677
|
+
}
|
|
678
|
+
}
|
|
679
|
+
if (!Buffer.isBuffer(sessionKey) || sessionKey.length !== SESSION_KEY_BYTES) {
|
|
680
|
+
sessionKey = null;
|
|
681
|
+
_emitFailure(req, "session-store-error");
|
|
682
|
+
return _writeRejection(req, res, HTTP_STATUS.INTERNAL_SERVER_ERROR, { error: "session-store-unavailable" });
|
|
612
683
|
}
|
|
684
|
+
// From here the channel is established: error bodies encrypt. The
|
|
685
|
+
// rejection counter sits one above the session's last emitted
|
|
686
|
+
// response so the client's strictly-increasing _ctr check holds.
|
|
687
|
+
_installRejectEncoder(req, sessionKey, sid, session.responsesEmitted + 1);
|
|
613
688
|
if (now > session.expiresAt) {
|
|
614
689
|
try { await sessionStore.delete(sid); } catch (_e) { /* best-effort */ }
|
|
615
690
|
_emitObs("apiEncrypt.session.expired", 1, {});
|
|
@@ -620,7 +695,7 @@ function create(opts) {
|
|
|
620
695
|
requestId: req.requestId || null,
|
|
621
696
|
});
|
|
622
697
|
_emitFailure(req, "session-expired");
|
|
623
|
-
return _writeRejection(res, HTTP_STATUS.UNAUTHORIZED, { error: "session-expired" });
|
|
698
|
+
return _writeRejection(req, res, HTTP_STATUS.UNAUTHORIZED, { error: "session-expired" });
|
|
624
699
|
}
|
|
625
700
|
if (session.responsesEmitted >= sessionMaxResponses) {
|
|
626
701
|
try { await sessionStore.delete(sid); } catch (_e) { /* best-effort */ }
|
|
@@ -632,7 +707,7 @@ function create(opts) {
|
|
|
632
707
|
requestId: req.requestId || null,
|
|
633
708
|
});
|
|
634
709
|
_emitFailure(req, "session-rotation-required");
|
|
635
|
-
return _writeRejection(res, HTTP_STATUS.UNAUTHORIZED, { error: "session-rotation-required" });
|
|
710
|
+
return _writeRejection(req, res, HTTP_STATUS.UNAUTHORIZED, { error: "session-rotation-required" });
|
|
636
711
|
}
|
|
637
712
|
// Replay defense: counter MUST strictly increase.
|
|
638
713
|
if (ctr <= session.lastReqCtr) {
|
|
@@ -644,7 +719,10 @@ function create(opts) {
|
|
|
644
719
|
requestId: req.requestId || null,
|
|
645
720
|
});
|
|
646
721
|
_emitFailure(req, "counter-replay");
|
|
647
|
-
|
|
722
|
+
// Plaintext: this path does not persist a consumed response counter
|
|
723
|
+
// (see _writeRejection). Keeping it cleartext avoids desyncing the
|
|
724
|
+
// session's response-counter sequence.
|
|
725
|
+
return _writeRejection(req, res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-rejected" }, { plaintext: true });
|
|
648
726
|
}
|
|
649
727
|
// Atomic replay gate (CWE-367). The monotonic counter check above is
|
|
650
728
|
// an ordering fast-path only: on a clustered session store, get(sid)
|
|
@@ -674,7 +752,7 @@ function create(opts) {
|
|
|
674
752
|
try { ctrFresh = await store.checkAndInsert(ctrKey, session.expiresAt); }
|
|
675
753
|
catch (_e) {
|
|
676
754
|
_emitFailure(req, "nonce-store-error");
|
|
677
|
-
return _writeRejection(res, HTTP_STATUS.INTERNAL_SERVER_ERROR, { error: "nonce-store-unavailable" });
|
|
755
|
+
return _writeRejection(req, res, HTTP_STATUS.INTERNAL_SERVER_ERROR, { error: "nonce-store-unavailable" });
|
|
678
756
|
}
|
|
679
757
|
if (!ctrFresh) {
|
|
680
758
|
_emitObs("apiEncrypt.session.replay_rejected", 1, { lane: "atomic" });
|
|
@@ -685,23 +763,10 @@ function create(opts) {
|
|
|
685
763
|
requestId: req.requestId || null,
|
|
686
764
|
});
|
|
687
765
|
_emitFailure(req, "counter-replay");
|
|
688
|
-
|
|
689
|
-
|
|
690
|
-
|
|
691
|
-
|
|
692
|
-
// Operator-supplied store may have JSON-serialised the buffer.
|
|
693
|
-
// Accept hex / base64 / Uint8Array and coerce.
|
|
694
|
-
if (typeof sessionKey === "string") {
|
|
695
|
-
sessionKey = Buffer.from(sessionKey, "base64");
|
|
696
|
-
} else if (sessionKey && sessionKey.type === "Buffer" && Array.isArray(sessionKey.data)) {
|
|
697
|
-
sessionKey = Buffer.from(sessionKey.data);
|
|
698
|
-
} else if (sessionKey instanceof Uint8Array) {
|
|
699
|
-
sessionKey = Buffer.from(sessionKey);
|
|
700
|
-
}
|
|
701
|
-
}
|
|
702
|
-
if (!Buffer.isBuffer(sessionKey) || sessionKey.length !== SESSION_KEY_BYTES) {
|
|
703
|
-
_emitFailure(req, "session-store-error");
|
|
704
|
-
return _writeRejection(res, HTTP_STATUS.INTERNAL_SERVER_ERROR, { error: "session-store-unavailable" });
|
|
766
|
+
// Plaintext for the same reason as the monotonic-replay path above:
|
|
767
|
+
// the atomic-claim loser returns before responsesEmitted is persisted,
|
|
768
|
+
// so encrypting it would consume a response _ctr the server never records.
|
|
769
|
+
return _writeRejection(req, res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-rejected" }, { plaintext: true });
|
|
705
770
|
}
|
|
706
771
|
session.lastReqCtr = ctr;
|
|
707
772
|
session.lastUsedAt = now;
|
|
@@ -711,7 +776,7 @@ function create(opts) {
|
|
|
711
776
|
sessionCtx = { sid: sid, responseCtr: session.responsesEmitted };
|
|
712
777
|
} else {
|
|
713
778
|
_emitFailure(req, "shape");
|
|
714
|
-
return _writeRejection(res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-required" });
|
|
779
|
+
return _writeRejection(req, res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-required" });
|
|
715
780
|
}
|
|
716
781
|
|
|
717
782
|
// Decrypt _ct → cleartext payload bytes → JSON object. The request
|
|
@@ -726,7 +791,7 @@ function create(opts) {
|
|
|
726
791
|
clearObj = safeJson.parse(ptBuf.toString("utf8"), { maxBytes: maxDecryptedBytes });
|
|
727
792
|
} catch (_e) {
|
|
728
793
|
_emitFailure(req, "tag");
|
|
729
|
-
return _writeRejection(res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-rejected" });
|
|
794
|
+
return _writeRejection(req, res, HTTP_STATUS.BAD_REQUEST, { error: "encrypted-payload-rejected" });
|
|
730
795
|
}
|
|
731
796
|
|
|
732
797
|
// Replace req.body with cleartext, stash session key for any
|
package/lib/queue-local.js
CHANGED
|
@@ -230,14 +230,28 @@ function create(config) {
|
|
|
230
230
|
// open each verb builder pre-bound to this table so the table reference
|
|
231
231
|
// is resolved in exactly one place.
|
|
232
232
|
var ref = _resolveTableRef(config);
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
//
|
|
238
|
-
//
|
|
239
|
-
//
|
|
240
|
-
|
|
233
|
+
// Resolve the ACTIVE backend dialect every verb builds for — sqlite in
|
|
234
|
+
// single-node, the operator-configured postgres/mysql in cluster mode — so
|
|
235
|
+
// b.sql emits dialect-correct identifier quoting (backticks on MySQL, not
|
|
236
|
+
// double-quotes that MySQL reads as string literals) and its own dialect
|
|
237
|
+
// guards fire. The default store IS clusterStorage (it knows the live
|
|
238
|
+
// dialect); a bring-your-own store declares config.dialect (or exposes its
|
|
239
|
+
// own dialect()), defaulting to sqlite. Resolved per call (lazy) like the
|
|
240
|
+
// sibling clusterStorage data-layer files, since cluster.init may run after
|
|
241
|
+
// queue.create.
|
|
242
|
+
function _dialect() {
|
|
243
|
+
if (store === clusterStorage) return clusterStorage.dialect();
|
|
244
|
+
if (typeof store.dialect === "function") return store.dialect();
|
|
245
|
+
return (typeof config.dialect === "string" && config.dialect) ? config.dialect : "sqlite";
|
|
246
|
+
}
|
|
247
|
+
function _opts() { return Object.assign({}, ref.opts, { dialect: _dialect() }); }
|
|
248
|
+
function _select() { return sql.select(ref.name, _opts()); }
|
|
249
|
+
function _insert() { return sql.insert(ref.name, _opts()); }
|
|
250
|
+
function _update() { return sql.update(ref.name, _opts()); }
|
|
251
|
+
function _delete() { return sql.delete(ref.name, _opts()); }
|
|
252
|
+
// Quoted column expression for a setRaw RHS that references the column's own
|
|
253
|
+
// pre-update value (attempts/availableAt), quoted for the ACTIVE dialect.
|
|
254
|
+
function _qc(col) { return safeSql.quoteIdentifier(col, _dialect(), { allowReserved: true }); }
|
|
241
255
|
|
|
242
256
|
async function enqueue(queueName, payload, opts) {
|
|
243
257
|
cluster.requireLeader();
|
|
@@ -324,22 +338,9 @@ function create(config) {
|
|
|
324
338
|
};
|
|
325
339
|
}
|
|
326
340
|
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
var leaseExpiresAt = nowMs + leaseMs;
|
|
331
|
-
var maxRows = count != null ? count : 1;
|
|
332
|
-
|
|
333
|
-
// Single-statement atomic lease. The IN-subquery picks the head of
|
|
334
|
-
// the queue; the outer UPDATE locks those rows and only updates
|
|
335
|
-
// rows that still match status='pending' after the lock acquires
|
|
336
|
-
// (Postgres EvalPlanQual; SQLite is single-writer so the same row
|
|
337
|
-
// can't be picked twice). RETURNING hands back the leased columns
|
|
338
|
-
// so we don't need a separate SELECT after the UPDATE. maxRows is a
|
|
339
|
-
// framework-computed integer emitted inline via b.sql's .limit() (a
|
|
340
|
-
// bound LIMIT param has no portable form across the subquery path);
|
|
341
|
-
// attempts = attempts + 1 is a setRaw over the column's own value.
|
|
342
|
-
var leaseInner = _select()
|
|
341
|
+
// Build the head-of-queue candidate SELECT (shared by both lease paths).
|
|
342
|
+
function _leaseCandidates(queueName, nowMs, maxRows) {
|
|
343
|
+
return _select()
|
|
343
344
|
.columns(["_id"])
|
|
344
345
|
.where("queueName", queueName)
|
|
345
346
|
.where("status", "pending")
|
|
@@ -348,20 +349,87 @@ function create(config) {
|
|
|
348
349
|
.orderBy("availableAt", "asc")
|
|
349
350
|
.orderBy("enqueuedAt", "asc")
|
|
350
351
|
.limit(maxRows);
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
var
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
352
|
+
}
|
|
353
|
+
|
|
354
|
+
async function lease(queueName, leaseMs, count) {
|
|
355
|
+
cluster.requireLeader();
|
|
356
|
+
var nowMs = Date.now();
|
|
357
|
+
var leaseExpiresAt = nowMs + leaseMs;
|
|
358
|
+
var maxRows = count != null ? count : 1;
|
|
359
|
+
var dialect = _dialect();
|
|
360
|
+
var i;
|
|
361
|
+
|
|
362
|
+
// SQLite (single-writer): one self-contained statement is atomic. The
|
|
363
|
+
// IN-subquery picks the head of the queue, the outer UPDATE flips it, and
|
|
364
|
+
// RETURNING hands the leased rows back. This shape is valid ONLY on
|
|
365
|
+
// sqlite — Postgres freezes the materialized subquery qual (so a
|
|
366
|
+
// concurrent leaser double-claims the same row) and MySQL refuses both
|
|
367
|
+
// RETURNING and updating a table named in its own subquery (error 1093).
|
|
368
|
+
if (dialect === "sqlite") {
|
|
369
|
+
var leaseBuilt = _update()
|
|
370
|
+
.set("status", "inflight")
|
|
371
|
+
.set("leasedAt", nowMs)
|
|
372
|
+
.set("leaseExpiresAt", leaseExpiresAt)
|
|
373
|
+
.setRaw("attempts", _qc("attempts") + " + 1", [])
|
|
374
|
+
.whereIn("_id", _leaseCandidates(queueName, nowMs, maxRows))
|
|
375
|
+
.returning(LEASE_RETURN_COLS)
|
|
376
|
+
.toSql();
|
|
377
|
+
var result = await store.execute(leaseBuilt.sql, leaseBuilt.params);
|
|
378
|
+
var leased = [];
|
|
379
|
+
for (i = 0; i < result.rows.length; i++) leased.push(_shapeLeasedRow(result.rows[i]));
|
|
380
|
+
return leased;
|
|
381
|
+
}
|
|
382
|
+
|
|
383
|
+
// Postgres / MySQL: claim inside a transaction. SELECT ... FOR UPDATE SKIP
|
|
384
|
+
// LOCKED row-locks the head-of-queue ids so concurrent leasers see
|
|
385
|
+
// disjoint sets (no double-lease); the guarded UPDATE ... WHERE
|
|
386
|
+
// status='pending' AND _id IN (locked ids) uses a LITERAL id list (not a
|
|
387
|
+
// self-referencing subquery — avoids MySQL 1093), with the status guard as
|
|
388
|
+
// belt-and-suspenders. Postgres reads the leased rows back via RETURNING;
|
|
389
|
+
// MySQL (no RETURNING) re-selects them by the locked ids. Mirrors
|
|
390
|
+
// outbox._claimBatch — the framework's canonical competing-consumer claim.
|
|
391
|
+
if (typeof store.transaction !== "function") {
|
|
392
|
+
throw _err("CLUSTER_TX_UNSUPPORTED",
|
|
393
|
+
"queue lease on a '" + dialect + "' backend requires an interactive transaction, but the " +
|
|
394
|
+
"configured store exposes no transaction(); use the default cluster store or supply a " +
|
|
395
|
+
"transaction-capable store", true);
|
|
363
396
|
}
|
|
364
|
-
return
|
|
397
|
+
return await store.transaction(async function (tx) {
|
|
398
|
+
var selBuilt = _leaseCandidates(queueName, nowMs, maxRows)
|
|
399
|
+
.forUpdate({ skipLocked: true })
|
|
400
|
+
.toSql();
|
|
401
|
+
var selRes = await tx.execute(selBuilt.sql, selBuilt.params);
|
|
402
|
+
var ids = ((selRes && selRes.rows) || []).map(function (r) { return r._id; });
|
|
403
|
+
if (ids.length === 0) return [];
|
|
404
|
+
|
|
405
|
+
var upd = _update()
|
|
406
|
+
.set("status", "inflight")
|
|
407
|
+
.set("leasedAt", nowMs)
|
|
408
|
+
.set("leaseExpiresAt", leaseExpiresAt)
|
|
409
|
+
.setRaw("attempts", _qc("attempts") + " + 1", [])
|
|
410
|
+
.where("status", "pending")
|
|
411
|
+
.whereInArray("_id", ids);
|
|
412
|
+
|
|
413
|
+
var rows;
|
|
414
|
+
if (dialect === "postgres") {
|
|
415
|
+
var updBuilt = upd.returning(LEASE_RETURN_COLS).toSql();
|
|
416
|
+
var updRes = await tx.execute(updBuilt.sql, updBuilt.params);
|
|
417
|
+
rows = (updRes && updRes.rows) || [];
|
|
418
|
+
} else {
|
|
419
|
+
var u = upd.toSql();
|
|
420
|
+
await tx.execute(u.sql, u.params);
|
|
421
|
+
var rbBuilt = _select()
|
|
422
|
+
.columns(LEASE_RETURN_COLS)
|
|
423
|
+
.where("status", "inflight")
|
|
424
|
+
.whereInArray("_id", ids)
|
|
425
|
+
.toSql();
|
|
426
|
+
var rbRes = await tx.execute(rbBuilt.sql, rbBuilt.params);
|
|
427
|
+
rows = (rbRes && rbRes.rows) || [];
|
|
428
|
+
}
|
|
429
|
+
var out = [];
|
|
430
|
+
for (i = 0; i < rows.length; i++) out.push(_shapeLeasedRow(rows[i]));
|
|
431
|
+
return out;
|
|
432
|
+
});
|
|
365
433
|
}
|
|
366
434
|
|
|
367
435
|
// extendLease — push the lease expiry forward for a long-running job.
|
|
@@ -633,19 +701,28 @@ function create(config) {
|
|
|
633
701
|
return result.rowCount || 0;
|
|
634
702
|
}
|
|
635
703
|
|
|
636
|
-
// patchFlowDeps — the second pass of enqueueFlow.
|
|
637
|
-
// dependsOn
|
|
638
|
-
//
|
|
639
|
-
// so it targets THIS backend's configured store +
|
|
640
|
-
// bring-your-own table receives the flow graph the same way the
|
|
641
|
-
// first-pass enqueue did, instead of the dispatcher writing to the
|
|
642
|
-
//
|
|
643
|
-
//
|
|
704
|
+
// patchFlowDeps — the second pass of enqueueFlow. Rewrites the child's
|
|
705
|
+
// dependsOn from the dependency NAMES the first pass wrote to the resolved
|
|
706
|
+
// sibling jobIds (now that every sibling's jobId is known). Lives on the
|
|
707
|
+
// backend (not in queue.js) so it targets THIS backend's configured store +
|
|
708
|
+
// table — a bring-your-own table receives the flow graph the same way the
|
|
709
|
+
// first-pass enqueue did, instead of the dispatcher writing to the default
|
|
710
|
+
// jobs table behind the backend's back. depIds is serialized to JSON.
|
|
711
|
+
//
|
|
712
|
+
// It must NOT touch availableAt: the first pass already parked the child at
|
|
713
|
+
// FLOW_BLOCKED_AVAILABLE_AT (it enqueues deps-bearing children WITH their
|
|
714
|
+
// dependsOn), and a dependency that completes in the window between the two
|
|
715
|
+
// passes drives complete() → _maybeReleaseFlowChildren, which bumps the
|
|
716
|
+
// child's availableAt to now. Re-parking here would clobber that release,
|
|
717
|
+
// and since the dependency is already done it never completes again — the
|
|
718
|
+
// child would sit pending-but-unleaseable forever. Parking is owned by the
|
|
719
|
+
// first-pass enqueue; releasing is owned by completion. This pass only
|
|
720
|
+
// resolves names → ids (harmless to rewrite even on an already-released
|
|
721
|
+
// child: a leased child's own dependsOn is never re-read).
|
|
644
722
|
async function patchFlowDeps(jobId, depIds) {
|
|
645
723
|
cluster.requireLeader();
|
|
646
724
|
var built = _update()
|
|
647
725
|
.set("dependsOn", JSON.stringify(depIds))
|
|
648
|
-
.set("availableAt", FLOW_BLOCKED_AVAILABLE_AT)
|
|
649
726
|
.where("_id", jobId)
|
|
650
727
|
.toSql();
|
|
651
728
|
var result = await store.execute(built.sql, built.params);
|