@blamejs/core 0.18.42 → 0.18.43
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +20 -0
- package/README.md +2 -2
- package/lib/file-type.js +80 -0
- package/lib/money.js +70 -7
- package/lib/queue.js +346 -98
- package/package.json +1 -1
- package/sbom.cdx.json +6 -6
package/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,26 @@ upgrading across more than a few patches at a time.
|
|
|
8
8
|
|
|
9
9
|
## v0.18.x
|
|
10
10
|
|
|
11
|
+
- v0.18.43 (2026-08-20) — **Three things the framework already knew how to do, reachable from outside it.** Each of these existed as working machinery with no public door: the largest-remainder split was locked behind a currency table that carries 43 of the ~180 ISO 4217 codes, the durable queue could only be driven by a resident consumer, and the MIME/extension table was reachable only through a test hook. In each case the predictable consequence downstream was a partial reimplementation of the part that is hardest to get right. **Added:** *`b.queue.tick(opts)` — drain what is due, then return* — Leases at most `max` available jobs, runs the handler over them, applies the same leasing, deterministic backoff and dead-letter transitions `b.queue.consume` applies, and settles. Nothing is left running when the promise resolves: no resident loop, no timer, no consumer for shutdown to wait on. Returns `{ leased, succeeded, failed, stale, unsettled, movedToDlq, mayHaveMore, queueDepth }`. Loop on `mayHaveMore` — the batch came back full, so more may be due right now. Not on `queueDepth`: that is the queue's total active depth and counts jobs whose `availableAt` is still in the future, so a caller looping on it spins through empty ticks waiting for a delayed job to come due. `queueDepth` is `null` when the backend could not be asked, because unknown is not empty. Two counts are not terminal states and mean different things. `unsettled` is the one to alert on: the backend REFUSED to record the outcome — complete or fail rejecting after its own retries — so the job is still inflight and the tick does not know whether the handler succeeded. A later sweep recovers it, and if the handler had already succeeded its side effects run again. `stale` is benign by comparison and counts jobs the tick ran but did not settle — the handler outlived the lease, the job was swept and re-leased, and the backend's attempt fence refused the completion. A non-zero `stale` means `leaseMs` is too short for the handler or the batch too large to finish inside it.
|
|
12
|
+
|
|
13
|
+
Pair it with `b.queue.init({ sweepIntervalMs: 0 })`, new in this release. `init` otherwise starts a 30-second expired-lease sweep timer, and in a frozen isolate that timer either never fires or fires inside a LATER invocation, touching the backend outside any request the platform is accounting for. `tick` sweeps once per invocation itself, so the timer buys nothing there. The option also lets a resident deployment tune the period; the default is unchanged.
|
|
14
|
+
|
|
15
|
+
It sweeps expired leases before taking a batch. A job whose holder died between leasing and settling stays `inflight` until something re-pends it, and the thing that normally does is the 30-second sweep timer started at `init` — a timer that never fires in the runtime this driver exists for. Without the sweep an invocation killed mid-lease would strand its job permanently, in exactly the deployment `tick` serves.
|
|
16
|
+
|
|
17
|
+
This is the driver a scheduled invocation needs — a Cloudflare `scheduled()` handler, a Lambda on EventBridge, a Kubernetes CronJob. `consume` cannot serve those: it is a resident loop and it starts a background sweep, and such a runtime freezes between invocations, so the timer never fires and the consumer is either blocking the handler or killed mid-lease. Without a per-tick entry point the durable half gets reimplemented by the caller — attempt accounting, backoff schedule, DLQ transition — which is precisely the part that is hard to get right.
|
|
18
|
+
|
|
19
|
+
`consume` and `tick` now run jobs through one shared path, so the retry schedule and the DLQ transition have exactly one implementation rather than two that can drift.
|
|
20
|
+
|
|
21
|
+
It drives the `local` and `redis` protocols, the same boundary `consume` has. An `sqs` backend is refused with `TICK_UNSUPPORTED`: SQS completes and fails by `receiptHandle` and owns redelivery and the dead-letter queue through its own RedrivePolicy, so driving it here would complete without the receipt and let SQS redeliver messages the handler had already processed. Drive SQS directly — lease, handle, complete or fail. · *`b.money.splitUnits(total, weights)` — the largest-remainder split without a currency* — Splits an integer into parts proportional to `weights`, distributing every unit: floor each share, then hand the leftovers to the largest remainders, ties broken by index. The total is preserved by construction and no floating point is involved, so the result is identical across runtimes. Negative totals floor toward negative infinity, so the parts still sum to the total.
|
|
22
|
+
|
|
23
|
+
A `Number` total or weight must be a SAFE integer. Past `Number.MAX_SAFE_INTEGER` the value has already been rounded before the function sees it — `9007199254740993` arrives as `...992` — so exact arithmetic there would faithfully split a total the caller never asked for. Those are refused with a message pointing at BigInt, which carries the value exactly.
|
|
24
|
+
|
|
25
|
+
The algorithm was already there as `Money.allocate`, but reaching it required a `Money`, which requires a currency from `b.money.CURRENCIES` — a table carrying the codes the framework has minor-unit data for rather than all of ISO 4217. A caller holding a currency outside that set could not reach the split at all, and neither could a caller splitting something that is not money: seats, quota, shard weights, a rate limit across workers. `Money.allocate` now composes it, so both doors open onto one implementation. · *`b.fileType.extensionFor(mime)` and `b.fileType.mimeFor(extension)`* — Read-only accessors over the signature table `b.fileType.detect` already consults, in both directions. Unknown returns `null` rather than a guess — a caller minting an object-store key or a `Content-Disposition` filename needs to know when the framework does not recognise the type. Lookups are case-insensitive, `extensionFor` ignores content-type parameters so a `Content-Type` header value can be passed straight in, and `mimeFor` accepts a leading dot because that is what `path.extname` returns.
|
|
26
|
+
|
|
27
|
+
`detect` answers what a file is; the next question is usually what to call it, and until now that had no answer, so consumers kept partial copies of a table the framework maintains — copies that drift the moment a signature is added.
|
|
28
|
+
|
|
29
|
+
`mimeFor` answers a naming question, not a trust question: it reports what an extension claims. Only `detect` establishes what a file IS, by reading the bytes, because an attacker controls the name and never the magic. **References:** [RFC 2045 §5.1 — Content-Type is case-insensitive](https://www.rfc-editor.org/rfc/rfc2045#section-5.1)
|
|
30
|
+
|
|
11
31
|
- v0.18.42 (2026-08-20) — **`b.middleware.botGuard` no longer refuses a request for omitting Accept-Language, which was returning 403 to every search-engine crawler.** `b.middleware.botGuard` treated a missing `Accept-Language` header as grounds for a 403 in its default blocking mode. Google documents that Googlebot sends requests without setting that header, and bingbot behaves the same, so every content page of a site running the default middleware chain answered 403 to a crawler while a browser was served normally. The header's absence is now an advisory signal, matching how the same middleware already treats a missing `Sec-Fetch-Mode`. **Changed:** *Vendored `@blamejs/pki` 0.5.16 to 0.5.17* — A failed integrity check now destroys the plaintext it had recovered rather than returning it, `AuthEnvelopedData` validates the attributes it is asked to authenticate, and the refusal for a content cipher in the wrong container cites the rule it actually applies — RFC 5083 §2 defines `AuthEnvelopedData` in terms of authenticated encryption and never names GCM, so the previous message sent operators to argue with a specification that says the opposite. **Fixed:** *A missing `Accept-Language` tags a request instead of refusing it* — The check was `if (!headers["accept-language"]) return "missing-accept-language"`, and `mode` defaults to `"block"`, so the request was answered 403. Whole client families omit the header — every major search-engine crawler, and with them uptime monitors, link previewers and feed readers — which made a site unreachable to all of them. On a deployment fronted by a cache this is easy to miss from the outside: the cached home page still answers 200, so the site looks partly indexed while every other URL is refused.
|
|
12
32
|
|
|
13
33
|
It now sets `req.suspectedBot` in `mode: "tag"` and never blocks, which is exactly how the middleware already treats a missing `Sec-Fetch-Mode` — absent for Safari before 16.4 and for every plain-HTTP non-localhost origin — and how `b.middleware.fetchMetadata` already treats a missing `Sec-Fetch-Site` through its `allowMissing` default. A header a whole client family omits is not evidence of automation.
|
package/README.md
CHANGED
|
@@ -67,7 +67,7 @@ The framework bundles the surface a typical Node app reaches for. Every primitiv
|
|
|
67
67
|
- **DB lifecycle** — in-memory encrypted snapshot via `b.db.snapshot()`; standalone encrypted-DB-file lifecycle (`b.db.fileLifecycle({ dataDir, vault })` — decrypt-to-tmpfs, periodic re-encrypt flush, graceful shutdown — same envelope as `b.db`, no schema/audit-chain coupling); `db.init` opt-outs `frameworkTables: false` / `auditSigning: false` and path overrides `encryptedDbPath` / `encryptedDbName` / `dbKeyPath`
|
|
68
68
|
- **External RDBMS** — bring-your-own Postgres / MySQL with pool tuning + role-aware connect + read-replica routing (`b.externalDb`); declarative role-narrowed views and Postgres row-level-security migrations (`b.db.declareView`, `b.db.declareRowPolicy`); an opt-in `requireTls` transport posture refuses a non-TLS backend at boot, and query / transaction / read traces carry OpenTelemetry `db.*` attributes. The framework's own data layer — the signed audit chain, cluster leadership and lease fencing, sessions, break-glass, and the local queue / cache / scheduler — is composed through the dialect-aware `b.sql` builder (every identifier quoted by construction, every value bound as a placeholder, dialect-correct SQLite / Postgres / MySQL output), so the framework's tables run on a Postgres or MySQL backend, not only local SQLite; `b.guardSql` validates result rows against NUL bytes, quote-jump sequences, and per-column / total-size boundaries
|
|
69
69
|
- **Object store** — S3 / R2 / B2 / GCS / Azure with multipart upload + SSE + bucket-ops (create / delete / list / lifecycle / CORS); S3 Object Lock + per-object retention + legal hold for write-once-read-many compliance workloads, with versioned delete + `listVersions` for right-to-erasure / crypto-shred against an Object-Lock bucket (`b.storage`, `b.objectStore`)
|
|
70
|
-
- **Queues + cache** — durable queue with priority + cron + flows on local SQLite, shared Redis, OR AWS SQS via SigV4 + AWSJsonProtocol_1.0 (`b.queue`, `b.jobs`) — the local backend can target an operator-supplied database / table / schema; cluster-shared cache (`b.cache`)
|
|
70
|
+
- **Queues + cache** — durable queue with priority + cron + flows on local SQLite, shared Redis, OR AWS SQS via SigV4 + AWSJsonProtocol_1.0 (`b.queue`, `b.jobs`) — the local backend can target an operator-supplied database / table / schema; driven either by a resident consumer (`b.queue.consume`) or one batch at a time for a scheduled runtime that cannot host one (`b.queue.tick`); cluster-shared cache (`b.cache`)
|
|
71
71
|
### Identity & access
|
|
72
72
|
|
|
73
73
|
- **Passwords** — Argon2id + policy primitive (`b.auth.password`); NIST 800-63B / PCI-DSS 4.0 / HIPAA-AAL2 profiles; HaveIBeenPwned k-anonymity breach check; length / context / dictionary / complexity rules; rotation + history
|
|
@@ -156,7 +156,7 @@ The framework bundles the surface a typical Node app reaches for. Every primitiv
|
|
|
156
156
|
- **Mobile credentials (mDL)** — `b.mdoc` ISO/IEC 18013-5 verification: `verifyIssuerSigned` checks the COSE_Sign1 IssuerAuth (issuer cert from the `x5chain` header), the MSO validity window, and every disclosed element's digest against the MSO `valueDigests` (selective-disclosure integrity), with optional issuer-chain verification; `verifyDeviceAuth` proves holder binding (§9.1.3 signature variant) — the device COSE_Sign1 over the `DeviceAuthentication` structure with the MSO device key + protocol `sessionTranscript`. The ISO credential ecosystem alongside `b.vc` and `b.auth.sdJwtVc`. Composes `b.cose` + `b.cbor`
|
|
157
157
|
- **Decentralized Identifiers** — `b.did` W3C DID resolution (DID Core 1.0): `resolve` a `did:key` / `did:jwk` (deterministic, offline — Ed25519 / P-256 / P-384 / secp256k1) or `did:web` (operator-fetched document) to `node:crypto` verification keys, so a credential's issuer DID resolves to the key that verifies it (`b.vc` / `b.mdoc` / `b.scitt`). `keyToDid` names a key as a `did:key` or `did:jwk`; document/JWK keys are kty/crv-allowlisted before import
|
|
158
158
|
- **Document parsers** — `b.parsers` (XML / TOML / YAML / .env); `b.config` (schema-validated env)
|
|
159
|
-
- **File-type detection** — `b.fileType` magic-byte content classification with deny-on-upload categories (image / document / archive / executable / etc.)
|
|
159
|
+
- **File-type detection** — `b.fileType` magic-byte content classification with deny-on-upload categories (image / document / archive / executable / etc.); `extensionFor` / `mimeFor` read the same table in both directions, so naming a detected file does not need a second copy of it
|
|
160
160
|
### Content-safety gates
|
|
161
161
|
|
|
162
162
|
- **Composition contract** — `b.gateContract` uniform mode posture / hooks / forensic snapshot / decision cache / runtime cap
|
package/lib/file-type.js
CHANGED
|
@@ -353,9 +353,89 @@ function assertOneOf(buf, allowlist, opts) {
|
|
|
353
353
|
return detected;
|
|
354
354
|
}
|
|
355
355
|
|
|
356
|
+
// Both directions of the signature table, built once at load. Every row is
|
|
357
|
+
// 1:1 in both directions, so neither map loses an entry; a row added with a
|
|
358
|
+
// duplicate mime or extension would silently shadow, which the test asserts
|
|
359
|
+
// against by round-tripping every row.
|
|
360
|
+
var _MIME_TO_EXT = Object.create(null);
|
|
361
|
+
var _EXT_TO_MIME = Object.create(null);
|
|
362
|
+
SIGNATURES.forEach(function (row) {
|
|
363
|
+
if (row.mime && !(row.mime.toLowerCase() in _MIME_TO_EXT)) {
|
|
364
|
+
_MIME_TO_EXT[row.mime.toLowerCase()] = row.extension;
|
|
365
|
+
}
|
|
366
|
+
if (row.extension && !(row.extension.toLowerCase() in _EXT_TO_MIME)) {
|
|
367
|
+
_EXT_TO_MIME[row.extension.toLowerCase()] = row.mime;
|
|
368
|
+
}
|
|
369
|
+
});
|
|
370
|
+
|
|
371
|
+
/**
|
|
372
|
+
* @primitive b.fileType.extensionFor
|
|
373
|
+
* @signature b.fileType.extensionFor(mime)
|
|
374
|
+
* @since 0.18.43
|
|
375
|
+
* @status stable
|
|
376
|
+
* @related b.fileType.detect, b.fileType.mimeFor
|
|
377
|
+
*
|
|
378
|
+
* The canonical file extension for a MIME type, with no leading dot, or `null`
|
|
379
|
+
* when the framework does not recognise the type. Never a guess: a caller
|
|
380
|
+
* minting an object-store key or a `Content-Disposition` filename needs to
|
|
381
|
+
* know when the answer is unknown rather than receive a plausible-looking one.
|
|
382
|
+
*
|
|
383
|
+
* `detect` answers "what is this"; the next question is usually "what should I
|
|
384
|
+
* call it", and the table that answers it is the same one. Reading it here
|
|
385
|
+
* rather than keeping a private copy is what stops the two drifting when a
|
|
386
|
+
* signature is added.
|
|
387
|
+
*
|
|
388
|
+
* The lookup is case-insensitive and ignores content-type parameters, so the
|
|
389
|
+
* `Content-Type` header value a request arrived with can be passed straight in.
|
|
390
|
+
*
|
|
391
|
+
* @example
|
|
392
|
+
* b.fileType.extensionFor("image/png");
|
|
393
|
+
* // → "png"
|
|
394
|
+
*/
|
|
395
|
+
function extensionFor(mime) {
|
|
396
|
+
if (typeof mime !== "string" || mime.length === 0) return null;
|
|
397
|
+
// MIME types are case-insensitive (RFC 2045 §5.1) and may carry parameters.
|
|
398
|
+
var bare = mime.split(";")[0].trim().toLowerCase();
|
|
399
|
+
if (bare.length === 0) return null;
|
|
400
|
+
return Object.prototype.hasOwnProperty.call(_MIME_TO_EXT, bare) ? _MIME_TO_EXT[bare] : null;
|
|
401
|
+
}
|
|
402
|
+
|
|
403
|
+
/**
|
|
404
|
+
* @primitive b.fileType.mimeFor
|
|
405
|
+
* @signature b.fileType.mimeFor(extension)
|
|
406
|
+
* @since 0.18.43
|
|
407
|
+
* @status stable
|
|
408
|
+
* @related b.fileType.detect, b.fileType.extensionFor
|
|
409
|
+
*
|
|
410
|
+
* The MIME type for a file extension, or `null` when the framework does not
|
|
411
|
+
* recognise it. The inverse of `b.fileType.extensionFor`, over the same table.
|
|
412
|
+
*
|
|
413
|
+
* A leading dot is accepted, because that is the form `path.extname` returns
|
|
414
|
+
* and therefore the form a caller usually has in hand. The lookup is
|
|
415
|
+
* case-insensitive.
|
|
416
|
+
*
|
|
417
|
+
* This answers a naming question, not a trust question. It reports what an
|
|
418
|
+
* extension claims; it does not establish what a file IS. Only
|
|
419
|
+
* `b.fileType.detect` does that, by reading the bytes — an attacker controls
|
|
420
|
+
* the name, never the magic.
|
|
421
|
+
*
|
|
422
|
+
* @example
|
|
423
|
+
* b.fileType.mimeFor(".png");
|
|
424
|
+
* // → "image/png"
|
|
425
|
+
*/
|
|
426
|
+
function mimeFor(extension) {
|
|
427
|
+
if (typeof extension !== "string" || extension.length === 0) return null;
|
|
428
|
+
var ext = extension.charAt(0) === "." ? extension.slice(1) : extension;
|
|
429
|
+
ext = ext.trim().toLowerCase();
|
|
430
|
+
if (ext.length === 0) return null;
|
|
431
|
+
return Object.prototype.hasOwnProperty.call(_EXT_TO_MIME, ext) ? _EXT_TO_MIME[ext] : null;
|
|
432
|
+
}
|
|
433
|
+
|
|
356
434
|
module.exports = {
|
|
357
435
|
detect: detect,
|
|
358
436
|
assertOneOf: assertOneOf,
|
|
437
|
+
extensionFor: extensionFor,
|
|
438
|
+
mimeFor: mimeFor,
|
|
359
439
|
FileTypeError: FileTypeError,
|
|
360
440
|
// Internal — exposed so tests can introspect the registry shape.
|
|
361
441
|
_SIGNATURES: SIGNATURES,
|
package/lib/money.js
CHANGED
|
@@ -444,22 +444,77 @@ function roundMinor(minor, step, mode) {
|
|
|
444
444
|
// method: floor each share, then hand out the leftover units to the
|
|
445
445
|
// shares with the largest fractional remainder. Total preserved by
|
|
446
446
|
// construction; deterministic across runtimes.
|
|
447
|
-
|
|
447
|
+
/**
|
|
448
|
+
* @primitive b.money.splitUnits
|
|
449
|
+
* @signature b.money.splitUnits(total, weights)
|
|
450
|
+
* @since 0.18.43
|
|
451
|
+
* @status stable
|
|
452
|
+
* @related b.money.fromMinorUnits, b.money.roundMinor
|
|
453
|
+
*
|
|
454
|
+
* Split an integer into parts proportional to `weights`, distributing every
|
|
455
|
+
* unit. Largest-remainder method: floor each share, then hand the leftover
|
|
456
|
+
* units to the shares with the largest fractional remainder, ties broken by
|
|
457
|
+
* index. The total is preserved by construction and the result is identical
|
|
458
|
+
* across runtimes — no floating point is involved at any step.
|
|
459
|
+
*
|
|
460
|
+
* This is the algorithm behind `Money.allocate`, reachable without a Money.
|
|
461
|
+
* `Money` requires a currency from `b.money.CURRENCIES`, which carries the
|
|
462
|
+
* codes the framework has minor-unit data for rather than all of ISO 4217, so
|
|
463
|
+
* a caller holding a currency outside that set could not reach the split at
|
|
464
|
+
* all. Nor could a caller splitting something that is not money — seats,
|
|
465
|
+
* quota, shard weights, a rate limit across workers — which is the same
|
|
466
|
+
* arithmetic with the same requirement that nothing be lost to rounding.
|
|
467
|
+
*
|
|
468
|
+
* `total` may be negative: shares floor toward negative infinity so the
|
|
469
|
+
* leftover pass stays positive and the parts still sum to `total`.
|
|
470
|
+
*
|
|
471
|
+
* @example
|
|
472
|
+
* b.money.splitUnits(100n, [1, 1, 1]);
|
|
473
|
+
* // → [34n, 33n, 33n]
|
|
474
|
+
*/
|
|
475
|
+
function splitUnits(total, weights) {
|
|
476
|
+
return _largestRemainder(_toBigIntUnits(total, "total"), weights, "splitUnits");
|
|
477
|
+
}
|
|
478
|
+
|
|
479
|
+
// Coerce a BigInt or SAFE integer Number; anything else is a caller error.
|
|
480
|
+
//
|
|
481
|
+
// Number.isInteger is not enough. Beyond Number.MAX_SAFE_INTEGER a literal has
|
|
482
|
+
// already been rounded to a representable neighbour before this function ever
|
|
483
|
+
// sees it — 9007199254740993 arrives as 9007199254740992, isInteger says yes,
|
|
484
|
+
// and BigInt() faithfully preserves the wrong value. The split would then be
|
|
485
|
+
// exact arithmetic over a total the caller never asked for, which breaks the
|
|
486
|
+
// one guarantee this function makes. Above that range the caller has to pass a
|
|
487
|
+
// BigInt, which carries the value exactly.
|
|
488
|
+
function _toBigIntUnits(v, label) {
|
|
489
|
+
if (typeof v === "bigint") return v;
|
|
490
|
+
if (typeof v === "number" && Number.isSafeInteger(v)) return BigInt(v);
|
|
491
|
+
throw new MoneyError("money/bad-units",
|
|
492
|
+
label + " must be a BigInt or a safe integer Number (beyond " +
|
|
493
|
+
"Number.MAX_SAFE_INTEGER a Number has already lost the value — pass a BigInt); got " +
|
|
494
|
+
(typeof v === "number" ? String(v) : typeof v));
|
|
495
|
+
}
|
|
496
|
+
|
|
497
|
+
// The shared largest-remainder split. `label` names the caller in refusals so
|
|
498
|
+
// an operator sees the API they actually invoked.
|
|
499
|
+
function _largestRemainder(total, weights, label) {
|
|
448
500
|
if (!Array.isArray(weights) || weights.length === 0) {
|
|
449
501
|
throw new MoneyError("money/bad-weights",
|
|
450
|
-
"
|
|
502
|
+
label + " requires a non-empty array of weights");
|
|
451
503
|
}
|
|
452
504
|
var sum = 0n;
|
|
453
505
|
var w = new Array(weights.length);
|
|
454
506
|
for (var i = 0; i < weights.length; i++) {
|
|
455
507
|
var wi = weights[i];
|
|
456
508
|
var wBig;
|
|
509
|
+
// Safe-integer, not merely integer — see _toBigIntUnits. A weight past the
|
|
510
|
+
// safe range has already been rounded, and the shares would be exact
|
|
511
|
+
// arithmetic over proportions the caller did not give.
|
|
457
512
|
if (typeof wi === "bigint") wBig = wi;
|
|
458
|
-
else if (typeof wi === "number" && Number.
|
|
513
|
+
else if (typeof wi === "number" && Number.isSafeInteger(wi)) wBig = BigInt(wi);
|
|
459
514
|
else {
|
|
460
515
|
throw new MoneyError("money/bad-weight",
|
|
461
|
-
"weight[" + i + "] must be BigInt or integer Number; got " +
|
|
462
|
-
(typeof wi));
|
|
516
|
+
"weight[" + i + "] must be a BigInt or a safe integer Number; got " +
|
|
517
|
+
(typeof wi === "number" ? String(wi) : typeof wi));
|
|
463
518
|
}
|
|
464
519
|
if (wBig < 0n) {
|
|
465
520
|
throw new MoneyError("money/bad-weight",
|
|
@@ -470,9 +525,8 @@ Money.prototype.allocate = function (weights) {
|
|
|
470
525
|
}
|
|
471
526
|
if (sum === 0n) {
|
|
472
527
|
throw new MoneyError("money/bad-weights",
|
|
473
|
-
"
|
|
528
|
+
label + " weights sum to zero");
|
|
474
529
|
}
|
|
475
|
-
var total = this._minor;
|
|
476
530
|
var shares = new Array(weights.length);
|
|
477
531
|
var remainders = new Array(weights.length);
|
|
478
532
|
var allocated = 0n;
|
|
@@ -508,6 +562,14 @@ Money.prototype.allocate = function (weights) {
|
|
|
508
562
|
leftover = leftover - 1n;
|
|
509
563
|
k = k + 1;
|
|
510
564
|
}
|
|
565
|
+
return shares;
|
|
566
|
+
}
|
|
567
|
+
|
|
568
|
+
// allocate -- split `this` into `weights.length` parts proportional to the
|
|
569
|
+
// weights, distributing every minor unit. The arithmetic is b.money.splitUnits;
|
|
570
|
+
// this wraps each share back into a Money of the same currency.
|
|
571
|
+
Money.prototype.allocate = function (weights) {
|
|
572
|
+
var shares = _largestRemainder(this._minor, weights, "allocate");
|
|
511
573
|
var out = new Array(shares.length);
|
|
512
574
|
for (var s2 = 0; s2 < shares.length; s2++) {
|
|
513
575
|
out[s2] = new Money(shares[s2], this.currency);
|
|
@@ -800,6 +862,7 @@ module.exports = {
|
|
|
800
862
|
zero: zero,
|
|
801
863
|
convert: convert,
|
|
802
864
|
roundMinor: roundMinor,
|
|
865
|
+
splitUnits: splitUnits,
|
|
803
866
|
CURRENCIES: CURRENCIES,
|
|
804
867
|
Money: Money,
|
|
805
868
|
MoneyError: MoneyError,
|
package/lib/queue.js
CHANGED
|
@@ -136,6 +136,7 @@ var sweepTimer = null;
|
|
|
136
136
|
* },
|
|
137
137
|
* },
|
|
138
138
|
* defaultBackend?: string, // name to use when enqueue/consume omit { backend }
|
|
139
|
+
* sweepIntervalMs?: number, // expired-lease sweep period; default 30s, 0 = no timer
|
|
139
140
|
*
|
|
140
141
|
* @example
|
|
141
142
|
* b.queue.init({
|
|
@@ -153,6 +154,20 @@ function init(opts) {
|
|
|
153
154
|
if (!opts || !opts.backends) {
|
|
154
155
|
throw _err("INVALID_CONFIG", "queue.init({ backends }) is required", true);
|
|
155
156
|
}
|
|
157
|
+
// Validate every option BEFORE touching module state. `initialized` stays
|
|
158
|
+
// false on a throw, so a rejected init is expected to leave nothing behind —
|
|
159
|
+
// but `backends` and `defaultBackend` are module-level and a later init does
|
|
160
|
+
// not clear them, so validating after assignment would leave the refused
|
|
161
|
+
// configuration's queues visible through listBackends() and selectable by
|
|
162
|
+
// name.
|
|
163
|
+
var sweepIntervalMs = opts.sweepIntervalMs === undefined
|
|
164
|
+
? C.TIME.seconds(30)
|
|
165
|
+
: opts.sweepIntervalMs;
|
|
166
|
+
if (sweepIntervalMs !== 0 && !numericChecks.isPositiveInt(sweepIntervalMs)) {
|
|
167
|
+
throw _err("INVALID_CONFIG",
|
|
168
|
+
"queue.init({ sweepIntervalMs }) must be a positive integer, or 0 to run no " +
|
|
169
|
+
"sweep timer; got " + JSON.stringify(opts.sweepIntervalMs), true);
|
|
170
|
+
}
|
|
156
171
|
|
|
157
172
|
// Self-register the _blamejs_jobs sealed-column declaration so payload +
|
|
158
173
|
// lastError seal at rest even when this process never ran db.init (a
|
|
@@ -218,13 +233,21 @@ function init(opts) {
|
|
|
218
233
|
|
|
219
234
|
// Sweep expired leases periodically (every 30s) so crashed-handler jobs
|
|
220
235
|
// get re-pended.
|
|
221
|
-
|
|
236
|
+
//
|
|
237
|
+
// `sweepIntervalMs: 0` starts no timer at all. That is the mode a scheduled
|
|
238
|
+
// runtime wants — a Cloudflare `scheduled()` handler, a Lambda, a CronJob —
|
|
239
|
+
// where a resident timer is not merely useless but wrong: the isolate is
|
|
240
|
+
// frozen between invocations, so it either never fires, or fires inside a
|
|
241
|
+
// LATER invocation and touches the backend outside any request the platform
|
|
242
|
+
// is accounting for. b.queue.tick sweeps once per invocation itself, so
|
|
243
|
+
// nothing is lost by turning the timer off there.
|
|
244
|
+
sweepTimer = sweepIntervalMs === 0 ? null : safeAsync.repeating(function () {
|
|
222
245
|
Object.keys(backends).forEach(function (n) {
|
|
223
246
|
if (backends[n].sweepExpired) {
|
|
224
247
|
backends[n].sweepExpired().catch(function () { /* best effort */ });
|
|
225
248
|
}
|
|
226
249
|
});
|
|
227
|
-
},
|
|
250
|
+
}, sweepIntervalMs, { name: "queue-sweep" });
|
|
228
251
|
|
|
229
252
|
initialized = true;
|
|
230
253
|
}
|
|
@@ -394,10 +417,6 @@ function consume(queueName, handler, opts) {
|
|
|
394
417
|
if (rateLimit) rateLimit.timestamps.push(Date.now());
|
|
395
418
|
}
|
|
396
419
|
|
|
397
|
-
// Progress audit-emit rate-limit — protect the audit chain from a
|
|
398
|
-
// chatty handler that calls progress() every loop iteration.
|
|
399
|
-
var PROGRESS_MIN_INTERVAL_MS = 250;
|
|
400
|
-
|
|
401
420
|
// Each consumer has its own AbortController so cancel() unblocks any
|
|
402
421
|
// in-flight poll-sleep immediately rather than waiting up to
|
|
403
422
|
// pollIntervalMs (default 1s) for the next while-loop iteration.
|
|
@@ -463,101 +482,13 @@ function consume(queueName, handler, opts) {
|
|
|
463
482
|
_emit("system.queue.consume.start", {
|
|
464
483
|
metadata: { queue: queueName, backend: b.name, jobId: job.jobId, attempt: job.attempts, traceId: job.traceId },
|
|
465
484
|
});
|
|
466
|
-
// Consume a rate-limit slot at handler-start so the budget
|
|
467
|
-
//
|
|
468
|
-
//
|
|
485
|
+
// Consume a rate-limit slot at handler-start so the budget tracks
|
|
486
|
+
// invocation rate, not lease rate (a single lease that splits work
|
|
487
|
+
// across many sub-units doesn't double-count).
|
|
469
488
|
_rateLimitConsume();
|
|
470
|
-
|
|
471
|
-
// Handler context — second arg to handler. Carries
|
|
472
|
-
// ctx.extendLease(ms) for long-running handlers and
|
|
473
|
-
// ctx.progress(0..100) for surfacing job progress to the
|
|
474
|
-
// audit chain (rate-limited so chatty handlers don't drown it).
|
|
475
|
-
var lastProgressEmitAt = 0;
|
|
476
|
-
var lastProgressValue = -1;
|
|
477
|
-
var ctx = {
|
|
478
|
-
extendLease: function (additionalMs) {
|
|
479
|
-
if (typeof b.extendLease !== "function") {
|
|
480
|
-
throw _err("EXTEND_LEASE_UNSUPPORTED",
|
|
481
|
-
"queue backend '" + b.name + "' does not support extendLease",
|
|
482
|
-
true);
|
|
483
|
-
}
|
|
484
|
-
return b.extendLease(job.jobId, additionalMs, { attempt: job.attempts }).then(function (ok) {
|
|
485
|
-
if (ok) {
|
|
486
|
-
_emit("system.queue.lease.extended", {
|
|
487
|
-
metadata: { queue: queueName, backend: b.name, jobId: job.jobId, additionalMs: additionalMs },
|
|
488
|
-
});
|
|
489
|
-
}
|
|
490
|
-
return ok;
|
|
491
|
-
});
|
|
492
|
-
},
|
|
493
|
-
progress: function (pct) {
|
|
494
|
-
if (typeof pct !== "number" || !isFinite(pct)) return;
|
|
495
|
-
var clamped = Math.max(0, Math.min(100, Math.floor(pct)));
|
|
496
|
-
var now = Date.now();
|
|
497
|
-
// Always emit 0 and 100 (start/done markers); throttle the rest.
|
|
498
|
-
var isMarker = clamped === 0 || clamped === 100;
|
|
499
|
-
if (!isMarker && (now - lastProgressEmitAt) < PROGRESS_MIN_INTERVAL_MS) return;
|
|
500
|
-
if (clamped === lastProgressValue && !isMarker) return;
|
|
501
|
-
lastProgressEmitAt = now;
|
|
502
|
-
lastProgressValue = clamped;
|
|
503
|
-
observability.event("queue.progress", clamped, { queueName: queueName });
|
|
504
|
-
_emit("system.queue.progress", {
|
|
505
|
-
metadata: {
|
|
506
|
-
queue: queueName, backend: b.name, jobId: job.jobId,
|
|
507
|
-
attempt: job.attempts, traceId: job.traceId,
|
|
508
|
-
percent: clamped,
|
|
509
|
-
},
|
|
510
|
-
});
|
|
511
|
-
},
|
|
512
|
-
};
|
|
513
|
-
observability.tap("queue.consume",
|
|
514
|
-
{ queueName: queueName, backend: b.name, jobId: job.jobId, attempt: job.attempts },
|
|
515
|
-
function () {
|
|
516
|
-
return Promise.resolve()
|
|
517
|
-
.then(function () { return handler(job, ctx); })
|
|
518
|
-
.then(function () {
|
|
519
|
-
return b.complete(job.jobId, { attempt: job.attempts }).then(function () {
|
|
520
|
-
_emit("system.queue.consume.success", {
|
|
521
|
-
metadata: { queue: queueName, backend: b.name, jobId: job.jobId, attempt: job.attempts, traceId: job.traceId },
|
|
522
|
-
});
|
|
523
|
-
observability.event("queue.complete", 1, { queueName: queueName });
|
|
524
|
-
});
|
|
525
|
-
}, function (err) {
|
|
526
|
-
var msg = (err && err.message) || String(err);
|
|
527
|
-
var willRetry = job.attempts < job.maxAttempts;
|
|
528
|
-
return b.fail(job.jobId, msg, { retryDelayMs: _backoffDelay(job.attempts), attempt: job.attempts })
|
|
529
|
-
.then(function () {
|
|
530
|
-
observability.event("queue.fail", 1, { queueName: queueName, willRetry: willRetry });
|
|
531
|
-
_emit("system.queue.consume.failure", {
|
|
532
|
-
metadata: {
|
|
533
|
-
queue: queueName, backend: b.name, jobId: job.jobId,
|
|
534
|
-
attempt: job.attempts, traceId: job.traceId,
|
|
535
|
-
maxAttempts: job.maxAttempts, willRetry: willRetry,
|
|
536
|
-
},
|
|
537
|
-
reason: msg,
|
|
538
|
-
outcome: "failure",
|
|
539
|
-
});
|
|
540
|
-
// DLQ-write event when the job has exhausted its retries.
|
|
541
|
-
// Operators wire this to their alerting / dashboards
|
|
542
|
-
// — failed-after-retries is "needs human review" not
|
|
543
|
-
// "in the normal flow." Audit chain captures the
|
|
544
|
-
// final state for forensics.
|
|
545
|
-
if (!willRetry) {
|
|
546
|
-
_emit("system.queue.dlq.write", {
|
|
547
|
-
metadata: {
|
|
548
|
-
queue: queueName, backend: b.name, jobId: job.jobId,
|
|
549
|
-
attempts: job.attempts, traceId: job.traceId,
|
|
550
|
-
},
|
|
551
|
-
reason: msg,
|
|
552
|
-
outcome: "failure",
|
|
553
|
-
});
|
|
554
|
-
}
|
|
555
|
-
});
|
|
556
|
-
})
|
|
489
|
+
_runJob(b, queueName, job, handler)
|
|
557
490
|
.catch(function (_e) { /* lifecycle errors swallowed — operator sees via audit */ })
|
|
558
491
|
.then(function () { state.inFlight.delete(job.jobId); });
|
|
559
|
-
}
|
|
560
|
-
);
|
|
561
492
|
})(jobs[i]);
|
|
562
493
|
}
|
|
563
494
|
await _pollSleep(fastPollMs);
|
|
@@ -576,6 +507,318 @@ function _backoffDelay(attempt) {
|
|
|
576
507
|
return retryHelper.backoffDelay(attempt, _QUEUE_BACKOFF_OPTS);
|
|
577
508
|
}
|
|
578
509
|
|
|
510
|
+
// The handler context passed as the second argument to a job handler:
|
|
511
|
+
// ctx.extendLease(ms) for a handler that outlives its lease, ctx.progress(0..100)
|
|
512
|
+
// for surfacing progress to the audit chain (rate-limited so a chatty handler
|
|
513
|
+
// cannot drown it).
|
|
514
|
+
var _PROGRESS_MIN_INTERVAL_MS = 250;
|
|
515
|
+
function _handlerContext(backend, queueName, job) {
|
|
516
|
+
var lastProgressEmitAt = 0;
|
|
517
|
+
var lastProgressValue = -1;
|
|
518
|
+
return {
|
|
519
|
+
extendLease: function (additionalMs) {
|
|
520
|
+
if (typeof backend.extendLease !== "function") {
|
|
521
|
+
throw _err("EXTEND_LEASE_UNSUPPORTED",
|
|
522
|
+
"queue backend '" + backend.name + "' does not support extendLease", true);
|
|
523
|
+
}
|
|
524
|
+
return backend.extendLease(job.jobId, additionalMs, { attempt: job.attempts }).then(function (ok) {
|
|
525
|
+
if (ok) {
|
|
526
|
+
_emit("system.queue.lease.extended", {
|
|
527
|
+
metadata: { queue: queueName, backend: backend.name, jobId: job.jobId, additionalMs: additionalMs },
|
|
528
|
+
});
|
|
529
|
+
}
|
|
530
|
+
return ok;
|
|
531
|
+
});
|
|
532
|
+
},
|
|
533
|
+
progress: function (pct) {
|
|
534
|
+
if (typeof pct !== "number" || !isFinite(pct)) return;
|
|
535
|
+
var clamped = Math.max(0, Math.min(100, Math.floor(pct)));
|
|
536
|
+
var now = Date.now();
|
|
537
|
+
// Always emit 0 and 100 (start/done markers); throttle the rest.
|
|
538
|
+
var isMarker = clamped === 0 || clamped === 100;
|
|
539
|
+
if (!isMarker && (now - lastProgressEmitAt) < _PROGRESS_MIN_INTERVAL_MS) return;
|
|
540
|
+
if (clamped === lastProgressValue && !isMarker) return;
|
|
541
|
+
lastProgressEmitAt = now;
|
|
542
|
+
lastProgressValue = clamped;
|
|
543
|
+
observability.event("queue.progress", clamped, { queueName: queueName });
|
|
544
|
+
_emit("system.queue.progress", {
|
|
545
|
+
metadata: {
|
|
546
|
+
queue: queueName, backend: backend.name, jobId: job.jobId,
|
|
547
|
+
attempt: job.attempts, traceId: job.traceId, percent: clamped,
|
|
548
|
+
},
|
|
549
|
+
});
|
|
550
|
+
},
|
|
551
|
+
};
|
|
552
|
+
}
|
|
553
|
+
|
|
554
|
+
// Run one leased job to its terminal state: complete on success, or fail with
|
|
555
|
+
// the deterministic backoff and — once the attempts are exhausted — the DLQ
|
|
556
|
+
// event. Resolves to "succeeded" or "failed"; `movedToDlq` says whether this
|
|
557
|
+
// failure was the last one.
|
|
558
|
+
//
|
|
559
|
+
// Both drivers go through here. b.queue.consume runs it inside a resident
|
|
560
|
+
// polling loop; b.queue.tick runs it over one leased batch and returns. The
|
|
561
|
+
// retry schedule, the attempt accounting and the DLQ transition are the part
|
|
562
|
+
// that is genuinely hard to get right, so there is exactly one copy of it.
|
|
563
|
+
function _runJob(backend, queueName, job, handler) {
|
|
564
|
+
var ctx = _handlerContext(backend, queueName, job);
|
|
565
|
+
return observability.tap("queue.consume",
|
|
566
|
+
{ queueName: queueName, backend: backend.name, jobId: job.jobId, attempt: job.attempts },
|
|
567
|
+
function () {
|
|
568
|
+
return Promise.resolve()
|
|
569
|
+
.then(function () { return handler(job, ctx); })
|
|
570
|
+
.then(function () {
|
|
571
|
+
return backend.complete(job.jobId, { attempt: job.attempts }).then(function (settled) {
|
|
572
|
+
// complete() answers FALSE when the attempt fence no longer
|
|
573
|
+
// matches: this worker's lease expired, the sweep re-pended the
|
|
574
|
+
// job, and someone else owns it now. Nothing was settled here, so
|
|
575
|
+
// claiming success would emit a success event for work this worker
|
|
576
|
+
// did not finish and inflate the caller's tally. Report it as
|
|
577
|
+
// stale and stay quiet.
|
|
578
|
+
if (settled === false) {
|
|
579
|
+
observability.event("queue.stale", 1, { queueName: queueName });
|
|
580
|
+
return { status: "stale", movedToDlq: false };
|
|
581
|
+
}
|
|
582
|
+
_emit("system.queue.consume.success", {
|
|
583
|
+
metadata: { queue: queueName, backend: backend.name, jobId: job.jobId, attempt: job.attempts, traceId: job.traceId },
|
|
584
|
+
});
|
|
585
|
+
observability.event("queue.complete", 1, { queueName: queueName });
|
|
586
|
+
// `status`, not `outcome` — `outcome` is the audit-event
|
|
587
|
+
// vocabulary (success / failure / denied) and this is the job's
|
|
588
|
+
// terminal state for the caller, not an audit row.
|
|
589
|
+
return { status: "succeeded", movedToDlq: false };
|
|
590
|
+
});
|
|
591
|
+
}, function (err) {
|
|
592
|
+
var msg = (err && err.message) || String(err);
|
|
593
|
+
var willRetry = job.attempts < job.maxAttempts;
|
|
594
|
+
return backend.fail(job.jobId, msg, { retryDelayMs: _backoffDelay(job.attempts), attempt: job.attempts })
|
|
595
|
+
.then(function (settled) {
|
|
596
|
+
// Same fence as complete(): a false answer means the job was
|
|
597
|
+
// re-leased under this worker, so this failure did not land.
|
|
598
|
+
// Emitting the failure — or worse, the DLQ write — would report
|
|
599
|
+
// a terminal state for a job that is running elsewhere.
|
|
600
|
+
if (settled === false) {
|
|
601
|
+
observability.event("queue.stale", 1, { queueName: queueName });
|
|
602
|
+
return { status: "stale", movedToDlq: false };
|
|
603
|
+
}
|
|
604
|
+
observability.event("queue.fail", 1, { queueName: queueName, willRetry: willRetry });
|
|
605
|
+
_emit("system.queue.consume.failure", {
|
|
606
|
+
metadata: {
|
|
607
|
+
queue: queueName, backend: backend.name, jobId: job.jobId,
|
|
608
|
+
attempt: job.attempts, traceId: job.traceId,
|
|
609
|
+
maxAttempts: job.maxAttempts, willRetry: willRetry,
|
|
610
|
+
},
|
|
611
|
+
reason: msg,
|
|
612
|
+
outcome: "failure",
|
|
613
|
+
});
|
|
614
|
+
// DLQ-write event when the job has exhausted its retries.
|
|
615
|
+
// Operators wire this to their alerting / dashboards —
|
|
616
|
+
// failed-after-retries is "needs human review" not "in the
|
|
617
|
+
// normal flow." The audit chain captures the final state.
|
|
618
|
+
if (!willRetry) {
|
|
619
|
+
_emit("system.queue.dlq.write", {
|
|
620
|
+
metadata: {
|
|
621
|
+
queue: queueName, backend: backend.name, jobId: job.jobId,
|
|
622
|
+
attempts: job.attempts, traceId: job.traceId,
|
|
623
|
+
},
|
|
624
|
+
reason: msg,
|
|
625
|
+
outcome: "failure",
|
|
626
|
+
});
|
|
627
|
+
}
|
|
628
|
+
return { status: "failed", movedToDlq: !willRetry };
|
|
629
|
+
});
|
|
630
|
+
});
|
|
631
|
+
}
|
|
632
|
+
);
|
|
633
|
+
}
|
|
634
|
+
|
|
635
|
+
/**
|
|
636
|
+
* @primitive b.queue.tick
|
|
637
|
+
* @signature b.queue.tick(opts)
|
|
638
|
+
* @since 0.18.43
|
|
639
|
+
* @status stable
|
|
640
|
+
* @related b.queue.consume, b.queue.enqueue, b.queue.dlqList
|
|
641
|
+
*
|
|
642
|
+
* Drain what is due, then return. Leases at most `max` available jobs, runs
|
|
643
|
+
* `handler` over them, applies the same leasing, deterministic backoff and
|
|
644
|
+
* dead-letter transitions `b.queue.consume` applies, and settles. Nothing is
|
|
645
|
+
* left running when the promise resolves: no resident loop, no timer, no
|
|
646
|
+
* consumer registered for shutdown to wait on.
|
|
647
|
+
*
|
|
648
|
+
* Pair it with `b.queue.init({ sweepIntervalMs: 0 })`. `init` otherwise starts
|
|
649
|
+
* a 30-second expired-lease sweep timer, and in a frozen isolate that timer
|
|
650
|
+
* either never fires or fires inside a later invocation, touching the backend
|
|
651
|
+
* outside any request the platform is accounting for. `tick` sweeps once per
|
|
652
|
+
* invocation itself, so the timer buys nothing there.
|
|
653
|
+
*
|
|
654
|
+
* This is the driver for a scheduled invocation — a Cloudflare `scheduled()`
|
|
655
|
+
* handler, a Lambda on EventBridge, a Kubernetes CronJob. `consume` cannot
|
|
656
|
+
* serve those: it is a resident loop and it starts a background sweep, and such
|
|
657
|
+
* a runtime freezes between invocations, so the timer never fires and the
|
|
658
|
+
* consumer is either blocking the handler or killed mid-lease. Without a
|
|
659
|
+
* per-tick entry point the durable half — attempt accounting, backoff schedule,
|
|
660
|
+
* DLQ transition — gets reimplemented by the caller, which is the part that is
|
|
661
|
+
* genuinely hard to get right.
|
|
662
|
+
*
|
|
663
|
+
* Returns `{ leased, succeeded, failed, stale, unsettled, movedToDlq,
|
|
664
|
+
* mayHaveMore, queueDepth }`.
|
|
665
|
+
*
|
|
666
|
+
* Two of those counts are not terminal states and mean different things.
|
|
667
|
+
* `stale` is benign: the backend answered that this worker no longer owns the
|
|
668
|
+
* job, so someone else is running it. `unsettled` is not: the backend REFUSED
|
|
669
|
+
* to record the outcome — complete or fail rejected after its own retries — so
|
|
670
|
+
* the job is still inflight and this tick does not know whether the handler
|
|
671
|
+
* succeeded. Those are recovered by a later sweep, and if the handler had
|
|
672
|
+
* already succeeded its side effects run a second time, so a non-zero
|
|
673
|
+
* `unsettled` is worth alerting on rather than counting as noise.
|
|
674
|
+
*
|
|
675
|
+
* `mayHaveMore` is the tick-again signal: the batch came back full, so more
|
|
676
|
+
* may be due right now. Loop on that, not on a depth count — `queueDepth` is
|
|
677
|
+
* the queue's total active depth (pending plus inflight) and includes jobs
|
|
678
|
+
* whose `availableAt` is still in the future, so a caller looping on it would
|
|
679
|
+
* spin through empty ticks waiting for a delayed job to come due. `queueDepth`
|
|
680
|
+
* is `null` when the backend could not be asked: unknown is not empty.
|
|
681
|
+
*
|
|
682
|
+
* `stale` counts jobs this tick ran but did not settle: the handler outlived
|
|
683
|
+
* the lease, the job was swept and re-leased, and the backend's attempt fence
|
|
684
|
+
* refused the completion. Those are not failures — another worker owns them —
|
|
685
|
+
* and a non-zero `stale` means `leaseMs` is too short for the handler, or the
|
|
686
|
+
* batch too large to finish inside it.
|
|
687
|
+
*
|
|
688
|
+
* Jobs run sequentially by default. Pass `concurrency` to overlap them; the
|
|
689
|
+
* lease is held for `leaseMs` either way, so a batch that will take longer than
|
|
690
|
+
* the lease should either raise `leaseMs` or call `ctx.extendLease(ms)` from
|
|
691
|
+
* the handler.
|
|
692
|
+
*
|
|
693
|
+
* @opts
|
|
694
|
+
* queue: string, // required — the queue name to drain
|
|
695
|
+
* handler: function, // required — (job, ctx) => Promise, same as consume
|
|
696
|
+
* max: number, // default: 10 — most jobs to lease this tick
|
|
697
|
+
* leaseMs: number, // default: 30s — lease duration for the batch
|
|
698
|
+
* concurrency: number, // default: 1 — jobs run in parallel
|
|
699
|
+
* backend: string, // named backend; default the init default
|
|
700
|
+
*
|
|
701
|
+
* @example
|
|
702
|
+
* var result = await b.queue.tick({
|
|
703
|
+
* queue: "webhooks",
|
|
704
|
+
* max: 50,
|
|
705
|
+
* handler: async function (job) { await deliver(job.payload); },
|
|
706
|
+
* });
|
|
707
|
+
* while (result.mayHaveMore) result = await b.queue.tick({ queue: "webhooks", max: 50, handler: deliver });
|
|
708
|
+
* // → { leased: 50, succeeded: 48, failed: 2, stale: 0, unsettled: 0,
|
|
709
|
+
* // movedToDlq: 1, mayHaveMore: true, queueDepth: 62 }
|
|
710
|
+
*/
|
|
711
|
+
async function tick(opts) {
|
|
712
|
+
_requireInit();
|
|
713
|
+
opts = opts || {};
|
|
714
|
+
var queueName = opts.queue;
|
|
715
|
+
if (!queueName) throw _err("MISSING_QUEUE", "tick requires opts.queue", true);
|
|
716
|
+
if (typeof opts.handler !== "function") {
|
|
717
|
+
throw _err("INVALID_HANDLER", "tick requires opts.handler to be a function", true);
|
|
718
|
+
}
|
|
719
|
+
var max = opts.max === undefined ? 10 : opts.max;
|
|
720
|
+
if (!numericChecks.isPositiveInt(max)) {
|
|
721
|
+
throw _err("BAD_MAX", "tick({ max }) must be a positive integer, got " + JSON.stringify(opts.max), true);
|
|
722
|
+
}
|
|
723
|
+
var leaseMs = opts.leaseMs === undefined ? C.TIME.seconds(30) : opts.leaseMs;
|
|
724
|
+
if (!numericChecks.isPositiveInt(leaseMs)) {
|
|
725
|
+
throw _err("BAD_LEASE", "tick({ leaseMs }) must be a positive integer, got " + JSON.stringify(opts.leaseMs), true);
|
|
726
|
+
}
|
|
727
|
+
var concurrency = opts.concurrency === undefined ? 1 : opts.concurrency;
|
|
728
|
+
if (!numericChecks.isPositiveInt(concurrency)) {
|
|
729
|
+
throw _err("BAD_CONCURRENCY", "tick({ concurrency }) must be a positive integer, got " + JSON.stringify(opts.concurrency), true);
|
|
730
|
+
}
|
|
731
|
+
|
|
732
|
+
var backend = _backendFor(opts);
|
|
733
|
+
// Same boundary b.queue.consume has: the framework-side lifecycle (lease by
|
|
734
|
+
// count, complete/fail by jobId, framework backoff and DLQ) is what `local`
|
|
735
|
+
// and `redis` implement. `sqs` is a different model — lease returns a
|
|
736
|
+
// receiptHandle the caller must thread back, redelivery and the dead-letter
|
|
737
|
+
// queue are the SQS queue's own RedrivePolicy — so driving it through here
|
|
738
|
+
// would call complete/fail without the receipt, leave every handled message
|
|
739
|
+
// in the queue, and let SQS redeliver work that already succeeded. Refuse
|
|
740
|
+
// rather than do that quietly; an SQS consumer drives lease/complete/fail
|
|
741
|
+
// directly, as lib/queue-sqs.js documents.
|
|
742
|
+
if (backend.protocol === "sqs") {
|
|
743
|
+
throw _err("TICK_UNSUPPORTED",
|
|
744
|
+
"queue.tick does not drive the 'sqs' protocol: SQS completes and fails by " +
|
|
745
|
+
"receiptHandle and owns redelivery + DLQ server-side. Drive it directly " +
|
|
746
|
+
"(lease -> handle -> complete/fail) as lib/queue-sqs.js describes.", true);
|
|
747
|
+
}
|
|
748
|
+
// Recover abandoned leases before taking a batch. A job whose holder died
|
|
749
|
+
// between leasing and completing sits in `inflight` until something re-pends
|
|
750
|
+
// it, and the thing that normally does is the 30-second sweep timer started
|
|
751
|
+
// at init — a timer that never fires in the runtime tick exists for, because
|
|
752
|
+
// the isolate is frozen between invocations. Without this, an invocation
|
|
753
|
+
// killed mid-lease strands its job permanently in exactly the deployment
|
|
754
|
+
// this driver serves. Best-effort: a sweep failure must not stop the tick
|
|
755
|
+
// from doing the work it can still do.
|
|
756
|
+
if (backend.sweepExpired) {
|
|
757
|
+
try { await backend.sweepExpired(); }
|
|
758
|
+
catch (e) { log.debug("tick-sweep-failed", { op: "sweepExpired", queue: queueName, error: e.message }); }
|
|
759
|
+
}
|
|
760
|
+
var jobs = await backend.lease(queueName, leaseMs, max);
|
|
761
|
+
jobs = jobs || [];
|
|
762
|
+
|
|
763
|
+
var result = {
|
|
764
|
+
leased: jobs.length,
|
|
765
|
+
succeeded: 0,
|
|
766
|
+
failed: 0,
|
|
767
|
+
stale: 0,
|
|
768
|
+
unsettled: 0,
|
|
769
|
+
movedToDlq: 0,
|
|
770
|
+
// The batch came back full, so there may be more due right now. This is
|
|
771
|
+
// the tick-again signal: it is exact, free, and unlike a depth count it
|
|
772
|
+
// cannot be fooled by jobs that are not yet available.
|
|
773
|
+
mayHaveMore: jobs.length === max,
|
|
774
|
+
queueDepth: 0,
|
|
775
|
+
};
|
|
776
|
+
var next = 0;
|
|
777
|
+
async function worker() {
|
|
778
|
+
while (next < jobs.length) {
|
|
779
|
+
var job = jobs[next++];
|
|
780
|
+
observability.event("queue.lease", 1, { queueName: queueName });
|
|
781
|
+
_emit("system.queue.consume.start", {
|
|
782
|
+
metadata: { queue: queueName, backend: backend.name, jobId: job.jobId, attempt: job.attempts, traceId: job.traceId },
|
|
783
|
+
});
|
|
784
|
+
var settled;
|
|
785
|
+
try { settled = await _runJob(backend, queueName, job, opts.handler); }
|
|
786
|
+
catch (e) {
|
|
787
|
+
// A throw here is the BACKEND refusing to record the outcome —
|
|
788
|
+
// complete or fail rejected after its own retries. The job's state was
|
|
789
|
+
// never changed, so it is still inflight and this tick does not know
|
|
790
|
+
// whether the handler succeeded. Calling that a handler failure would
|
|
791
|
+
// be wrong twice: it reports a terminal state the job never reached,
|
|
792
|
+
// and it hides an infrastructure problem inside an ordinary tally.
|
|
793
|
+
// Report it as unsettled; a later sweep re-pends the job.
|
|
794
|
+
log.debug("tick-settle-failed", { op: "_runJob", queue: queueName, jobId: job.jobId, error: e.message });
|
|
795
|
+
settled = { status: "unsettled", movedToDlq: false };
|
|
796
|
+
}
|
|
797
|
+
if (settled && settled.status === "stale") result.stale += 1;
|
|
798
|
+
else if (settled && settled.status === "unsettled") result.unsettled += 1;
|
|
799
|
+
else if (settled && settled.status === "succeeded") result.succeeded += 1;
|
|
800
|
+
else result.failed += 1;
|
|
801
|
+
if (settled && settled.movedToDlq) result.movedToDlq += 1;
|
|
802
|
+
}
|
|
803
|
+
}
|
|
804
|
+
var workers = [];
|
|
805
|
+
for (var w = 0; w < Math.min(concurrency, jobs.length); w++) workers.push(worker());
|
|
806
|
+
await Promise.all(workers);
|
|
807
|
+
|
|
808
|
+
// Depth AFTER the batch. This is the queue's total active depth — pending
|
|
809
|
+
// plus inflight — and it deliberately does NOT drive the tick-again
|
|
810
|
+
// decision: it counts jobs whose availableAt is still in the future, so a
|
|
811
|
+
// caller looping on it would spin through empty ticks waiting for a delayed
|
|
812
|
+
// job to come due. null when the backend could not be asked; unknown is not
|
|
813
|
+
// the same as empty.
|
|
814
|
+
try { result.queueDepth = await size(queueName, opts); }
|
|
815
|
+
catch (e) {
|
|
816
|
+
log.debug("tick-size-failed", { op: "size", queue: queueName, error: e.message });
|
|
817
|
+
result.queueDepth = null;
|
|
818
|
+
}
|
|
819
|
+
return result;
|
|
820
|
+
}
|
|
821
|
+
|
|
579
822
|
/**
|
|
580
823
|
* @primitive b.queue.size
|
|
581
824
|
* @signature b.queue.size(queueName, opts)
|
|
@@ -1064,6 +1307,7 @@ module.exports = {
|
|
|
1064
1307
|
enqueue: enqueue,
|
|
1065
1308
|
enqueueFlow: enqueueFlow,
|
|
1066
1309
|
consume: consume,
|
|
1310
|
+
tick: tick,
|
|
1067
1311
|
size: size,
|
|
1068
1312
|
purge: purge,
|
|
1069
1313
|
shutdown: shutdown,
|
|
@@ -1074,4 +1318,8 @@ module.exports = {
|
|
|
1074
1318
|
PROTOCOLS: dispatcher.protocols,
|
|
1075
1319
|
DEFERRED_PROTOCOLS: dispatcher.deferred,
|
|
1076
1320
|
_resetForTest: _resetForTest,
|
|
1321
|
+
// Internal — the wrapped backend object, so a test can break one of its
|
|
1322
|
+
// lifecycle calls (an exhausted breaker, an unreachable store) and assert
|
|
1323
|
+
// what the drivers do about it. There is no other way to reach that path.
|
|
1324
|
+
_backendForTest: _backendFor,
|
|
1077
1325
|
};
|
package/package.json
CHANGED
package/sbom.cdx.json
CHANGED
|
@@ -2,10 +2,10 @@
|
|
|
2
2
|
"$schema": "http://cyclonedx.org/schema/bom-1.5.schema.json",
|
|
3
3
|
"bomFormat": "CycloneDX",
|
|
4
4
|
"specVersion": "1.5",
|
|
5
|
-
"serialNumber": "urn:uuid:
|
|
5
|
+
"serialNumber": "urn:uuid:a214f954-8702-46e7-819d-20eb98e5ca08",
|
|
6
6
|
"version": 1,
|
|
7
7
|
"metadata": {
|
|
8
|
-
"timestamp": "2026-08-
|
|
8
|
+
"timestamp": "2026-08-20T20:10:05.952Z",
|
|
9
9
|
"lifecycles": [
|
|
10
10
|
{
|
|
11
11
|
"phase": "build"
|
|
@@ -19,14 +19,14 @@
|
|
|
19
19
|
}
|
|
20
20
|
],
|
|
21
21
|
"component": {
|
|
22
|
-
"bom-ref": "@blamejs/core@0.18.
|
|
22
|
+
"bom-ref": "@blamejs/core@0.18.43",
|
|
23
23
|
"type": "application",
|
|
24
24
|
"name": "blamejs",
|
|
25
|
-
"version": "0.18.
|
|
25
|
+
"version": "0.18.43",
|
|
26
26
|
"scope": "required",
|
|
27
27
|
"author": "blamejs contributors",
|
|
28
28
|
"description": "The Node framework that owns its stack.",
|
|
29
|
-
"purl": "pkg:npm/%40blamejs/core@0.18.
|
|
29
|
+
"purl": "pkg:npm/%40blamejs/core@0.18.43",
|
|
30
30
|
"properties": [],
|
|
31
31
|
"externalReferences": [
|
|
32
32
|
{
|
|
@@ -54,7 +54,7 @@
|
|
|
54
54
|
"components": [],
|
|
55
55
|
"dependencies": [
|
|
56
56
|
{
|
|
57
|
-
"ref": "@blamejs/core@0.18.
|
|
57
|
+
"ref": "@blamejs/core@0.18.43",
|
|
58
58
|
"dependsOn": []
|
|
59
59
|
}
|
|
60
60
|
]
|