@blamejs/core 0.6.22 → 0.6.24
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +2 -0
- package/README.md +1 -1
- package/lib/log-stream-otlp.js +292 -0
- package/lib/log-stream.js +22 -8
- package/lib/queue-local.js +23 -7
- package/package.json +1 -1
- package/sbom.cyclonedx.json +6 -6
package/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,8 @@ upgrading across more than a few patches at a time.
|
|
|
8
8
|
|
|
9
9
|
## v0.6.x
|
|
10
10
|
|
|
11
|
+
- **0.6.24** (2026-05-02) — `b.logStream` gains an OTLP/HTTP-JSON sink. `protocol: "otlp"` now forwards log records to any OpenTelemetry collector via the OTel Logs Data Model — `resourceLogs` → `scopeLogs` → `logRecords` envelope with `severityNumber` (debug=5/info=9/warn=13/error=17), `severityText`, `timeUnixNano` (string-encoded for JSON-safe 64-bit), `body.stringValue`, and OTel-typed attributes. Operator config: `{ url, serviceName, serviceVersion, resourceAttributes, auth, headers, batchSize, retry, onDrop, ... }` — same back-pressure semantics as the webhook sink (per-sink ring buffer, batched flush on size or maxBatchAgeMs, exponential-backoff retry, drop-on-overflow with operator-supplied onDrop callback). URL convention: `/v1/logs` is auto-appended when the operator passes the collector root. JSON not protobuf — operators benchmarking >100K logs/s ship the OTel Collector locally so the framework hands JSON to a sidecar that forwards via gRPC. Removed `otlp` from `DEFERRED_PROTOCOLS`. Wiki observability page documents the new sink with a `b.logStream.init` example pointing at an OTel collector. Tests cover URL resolution, attribute encoding (string / int / float / bool / array / nested object), severity mapping, round-trip via mock collector, auth header pass-through, retry on 5xx + drop on retry-exhaustion, buffer overflow, dispatcher integration. Verified 0 leaks across 291 commits via `gitleaks`.
|
|
12
|
+
- **0.6.23** (2026-05-02) — v0.6.22 follow-up cleanup. The cron-repeat call site in queue-local.js was passing both `availableAt` AND `delaySeconds` (the former was the precise next-fire ms; the latter was a redundant `Math.floor((nextMs - nowMs) / 1000)` computation that only existed to work around the bug v0.6.22 fixed). Dropped — the cron repeat now passes `availableAt` alone, matching the queue's documented precedence rule. The enqueue() docstring gains a 20-line "SCHEDULING PRECEDENCE" header documenting that `opts.availableAt` wins over `opts.delaySeconds` when both are passed, why the framework chose that direction, and which callers should use which form. New round-trip preservation regression test (`testEnqueueRoundTripsAvailableAt`) covers three precise targets, the delaySeconds-only path, and the both-opts-set case — gates against any future "I'll just rederive it from the floored seconds" mistake. Audited the rest of the framework for the same `(absolute-time, relative-time)` opt-overlap shape (cache.set, session.rotate, apiKey, dualControl): queue is the only primitive carrying both forms, so a generalized `b.time.resolveTimePoint` primitive would be premature with one call site.
|
|
11
13
|
- **0.6.22** (2026-05-02) — `b.queue.enqueue({ availableAt })` is now honoured. Previously the local-protocol enqueue() ignored opts.availableAt entirely, recomputing from `Date.now() + delaySeconds*1000`. The cron-repeat path passes both fields (the exact next-fire ms in availableAt + the floored seconds in delaySeconds), and the enqueue's recomputation lost sub-second precision plus drifted on the internal clock-vs-caller delta. Symptoms: cron-scheduled jobs landed up to 999ms off the intended boundary; the queue-flow-repeat smoke test was intermittently flaky on slow CI runners (caught by ubuntu-latest on the v0.6.21 commit). Fix: enqueue() honours opts.availableAt directly when finite; falls back to delaySeconds-based shorthand otherwise. Operators relying on `enqueue({ availableAt: T })` for non-cron scheduled jobs (e.g. "deliver this notification at exactly 09:00 tomorrow") now get the requested time instead of nowMs+0.
|
|
12
14
|
- **0.6.21** (2026-05-02) — closes the medium / low audit findings flagged after v0.6.18. **Cluster cache `invalidateTag`** now actually works — the cluster backend gained a `_blamejs_cache_tags` junction table (`(cacheKey, tag)` PK + index on `tag`), tag-aware `set` / `del` / `clear` / `_sweep`, plus `getTags(key)`. The old NOT_SUPPORTED-on-cluster path is gone; multi-tag rotation, mid-flight tag replacement on update, and namespace-scoped sweeps all covered. **Multi-column cursor pagination** — `b.pagination.cursor({ orderBy: [{column,direction},...] })` accepts a string (single column), an array of strings (multi, all using `opts.direction`), or an array of `{column,direction}` objects (mixed directions). The keyset WHERE expands to the standard OR cascade so successive pages can't skip or repeat rows when ties on the leading columns are broken by trailing ones. `_id` is appended as a tiebreaker if not in the chain. The Query class also gained chained `orderBy(col, dir)` calls — second-and-later calls extend a multi-column ORDER BY in the SQL. Cursor format is bumped to encode `{ orderKey, vals, forward }` instead of the old `{ orderBy, dir, orderByVal, id, forward }` — pre-1.0 break, no compat shim. **DoH POST mode** (RFC 8484 §4.1) — `b.network.dns.useDnsOverHttps({ method })` accepts `"GET" | "POST" | undefined` (auto). Auto switches to POST when the GET URL would exceed 2048 bytes (long DNS names). **DoT connection pooling** — per-`(host:port)` cached TLS socket with a 2-minute idle timeout, serialized in-flight queries per socket. Eliminates the per-query handshake. **INI parser** shipped as `b.parsers.ini` — covers Windows .ini / .gitconfig / systemd-unit / php.ini / tox.ini shape: sections (incl. `[parent.child]` / `[parent "child"]` nesting), `;` and `#` comments (inline + leading), single + double quoting with `\n` `\t` `\\` `\"` `\'` escapes, boolean coercion (`true`/`false`/`yes`/`no`/`on`/`off`), decimal + hex integers + floats. Prototype-pollution defense (`__proto__` / `constructor` / `prototype` rejected). Duplicate-key policy throws by default; `onDuplicate: "first" | "last"` opts in to silent shadowing. Section / per-section key / value-bytes caps configurable. **Cookie-jar file persistence** — `b.httpClient.cookieJar.create({ persist: "file", file: "/abs/path", vault: b.vault })` loads at construct, debounce-flushes on every set/clear, plus `flush()` and `close()` for explicit lifecycle. With `vault`, on-disk bytes are sealed; without, plaintext JSON (operator chooses). **Comment cleanups per rule §4** — `vault/index.js` stale `// later` removed, `mail.js` "future patch" wording rewritten as scope, `bundler.js` "What it does NOT do today (deliberately deferred)" rewritten as "Out of scope", `archive.js` "v1 scope cuts (deferred)" rewritten as "Out of scope", `framework-schema.js` "next release" wording rewritten to describe what's actually shipped, `http-client.js` "out of reach today" / "we'll plumb it through when h3 lands" rewritten as scope statements. **README CLI section** updated to reflect the v0.6.17 + v0.6.18 + v0.6.19 + v0.6.20 additions (security / config-drift / file-type / password / erase / retention) — was missing 6 subcommands.
|
|
13
15
|
- **0.6.20** (2026-05-02) — CI / packaging fix-up. The npm-publish workflow's "Attach SBOM as release asset" step started failing with HTTP 422 ("Cannot upload assets to an immutable release") because the operator's manual `gh release create` had already been published when the workflow ran. Two changes: (1) `sbom.cyclonedx.json` is now bundled into the npm tarball (`files` block in package.json), so `npm install @blamejs/core && cat node_modules/@blamejs/core/sbom.cyclonedx.json` is the canonical SBOM access path. The prepack guard's known-allowed list covers the just-in-time generation. (2) The workflow's GH-release-attach step is now non-fatal: it tries to upload, logs a warning if the release is immutable, and lets the publish proceed regardless. The npm tarball is the load-bearing artifact; the GitHub release attachment was only ever supplementary. `.gitignore` adds `sbom.cyclonedx.json` so a stray local `npm sbom` doesn't pollute the repo.
|
package/README.md
CHANGED
|
@@ -47,7 +47,7 @@ The framework bundles the surface a typical Node app reaches for. Every primitiv
|
|
|
47
47
|
- **HTTP** — router with schema-validated routes + OpenAPI publication; full middleware stack (CSRF, CORS, rate-limit, security headers, CSP nonce, body parser, compression, SSE, request log, request-time DB role binding via `b.middleware.dbRoleFor`, in-process CIDR fence via `b.middleware.networkAllowlist`) wired by `createApp`; HTTP/1.1 + HTTP/2 outbound client with SSRF gate, scheme + userinfo + per-host (wildcard / per-method) destination allowlist, redirects, multipart, interceptors, progress, encrypted cookie jar (`b.httpClient`, `b.ssrfGuard`, `b.safeUrl`); operator-tunable network configurability — env-driven NTP / NTS (RFC 8915 authenticated time), DNS with IPv6 / DoH / DoT / cache / lookup timeout, outbound HTTP proxy (`HTTP_PROXY` / `HTTPS_PROXY` / `NO_PROXY`), runtime DPI trust-store CA additions, application-level heartbeats, TCP socket defaults (`b.network`).
|
|
48
48
|
- **Defensive parsers** — `b.safeJson`, `b.safeBuffer`, `b.safeSql`, `b.safeSchema`, `b.parsers` (XML / TOML / YAML / .env), `b.config` (schema-validated env), `b.fileType` magic-byte content classification with deny-on-upload categories (image / document / archive / executable / etc.).
|
|
49
49
|
- **Communication** — WebSockets with channel/room fan-out across cluster replicas (`b.websocket`, `b.websocketChannels`); mail with multipart + attachments + DKIM + calendar invites + bounce intake (`b.mail`, `b.mailBounce`); generic notification dispatcher with operator-supplied transports (`b.notify`).
|
|
50
|
-
- **Observability** — tamper-evident audit chain with SLH-DSA-signed checkpoints, metrics, tracing (OTel pass-through when wired), PII redaction, log-stream sinks, OTLP/HTTP-JSON exporter for
|
|
50
|
+
- **Observability** — tamper-evident audit chain with SLH-DSA-signed checkpoints, metrics, tracing (OTel pass-through when wired), PII redaction, log-stream sinks (local file rotation, generic webhook, OTLP/HTTP-JSON to an OTel collector), OTLP/HTTP-JSON exporter for traces + metrics (`b.audit`, `b.metrics`, `b.tracing`, `b.redact`, `b.logStream`, `b.otelExport`); operator-callable boot-time security policy assertions (`b.security.assertProduction`) and tamper-evident config-baseline drift detection signed with the audit-signing key (`b.configDrift`).
|
|
51
51
|
- **i18n** — CLDR plural rules, Accept-Language negotiation, Intl formatters, RTL (`b.i18n`).
|
|
52
52
|
- **Format helpers** — RFC 4180 CSV with Excel formula-injection prevention (`b.csv`), RFC 9562 UUID v4 + v7 (`b.uuid`), URL-safe slugs (`b.slug`), TZ-aware datetime (`b.time`), ZIP creation (`b.archive`), HMAC-signed cursor pagination (`b.pagination`), HTML form rendering + validation + CSRF (`b.forms`).
|
|
53
53
|
- **Production** — cluster leader election with fenced leases over Postgres/SQLite (`b.cluster`); cron + interval scheduler that runs exactly-once globally (`b.scheduler`); retry with full-jitter backoff + circuit breaker (`b.retry`); graceful shutdown (`b.appShutdown`); NTP boot check (`b.ntpCheck`); end-to-end-encrypted backup bundles with pre-flush fail-closed mode (`b.backup`); restore with pulled-bundle footprint preflight (`b.restore`); GDPR / PCI / HIPAA-shaped retention rules with multi-stage warn → archive → erase, legal-hold exemptions, dry-run preview, cross-table cascade (`b.retention`).
|
|
@@ -0,0 +1,292 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
/**
|
|
3
|
+
* OTLP/HTTP-JSON log sink — OpenTelemetry Protocol logs over HTTP/JSON.
|
|
4
|
+
*
|
|
5
|
+
* Per the OTLP spec (v1.3+, Logs Data Model), log records are wrapped in
|
|
6
|
+
* a ResourceLogs envelope and POSTed to the operator's collector
|
|
7
|
+
* endpoint. Marshalling is OTel-native: timeUnixNano (string), severity
|
|
8
|
+
* mapping per OTel spec, attribute typing.
|
|
9
|
+
*
|
|
10
|
+
* Operator config:
|
|
11
|
+
*
|
|
12
|
+
* {
|
|
13
|
+
* url: "https://otel-collector.example.com:4318/v1/logs"
|
|
14
|
+
* // OR the collector root — "/v1/logs" auto-appended
|
|
15
|
+
* headers: { ... } // additional headers (api keys, tenant ids)
|
|
16
|
+
* auth: 'none' | 'bearer' | 'basic' | 'header'
|
|
17
|
+
* token / username+password
|
|
18
|
+
* serviceName: "my-service" // emitted as resource service.name
|
|
19
|
+
* serviceVersion: "1.2.3" // resource service.version
|
|
20
|
+
* resourceAttributes: { env: "prod", region: "us-east-1" }
|
|
21
|
+
* // additional resource attributes
|
|
22
|
+
* scopeName: "blamejs" // instrumentation scope name
|
|
23
|
+
* batchSize: 100
|
|
24
|
+
* maxBatchAgeMs: C.TIME.seconds(5)
|
|
25
|
+
* timeoutMs: C.TIME.seconds(30)
|
|
26
|
+
* retry: { maxAttempts, baseDelayMs, ... }
|
|
27
|
+
* bufferLimit: 10000 // ring-buffer cap; drops oldest on overflow
|
|
28
|
+
* onDrop: function ({ reason, batch, error }) { ... }
|
|
29
|
+
* }
|
|
30
|
+
*
|
|
31
|
+
* Severity mapping (per OTel Logs Data Model):
|
|
32
|
+
* debug → 5 (DEBUG)
|
|
33
|
+
* info → 9 (INFO)
|
|
34
|
+
* warn → 13 (WARN)
|
|
35
|
+
* error → 17 (ERROR)
|
|
36
|
+
*
|
|
37
|
+
* Endpoint convention: most OTel collectors expose:
|
|
38
|
+
* <base>:4318/v1/logs — HTTP/JSON path (per OTLP HTTP §4.5)
|
|
39
|
+
* <base>:4317 — gRPC path (NOT supported here)
|
|
40
|
+
*
|
|
41
|
+
* If `url` ends in `/v1/logs` it's used as-is. If not, `/v1/logs` is
|
|
42
|
+
* appended (collector root → logs endpoint). Operators with a custom
|
|
43
|
+
* routing prefix pass the full URL.
|
|
44
|
+
*
|
|
45
|
+
* Why JSON not protobuf: protobuf needs a schema parser (vendored dep
|
|
46
|
+
* or hand-rolled). The OTLP/HTTP spec accepts both Content-Type:
|
|
47
|
+
* application/json and application/x-protobuf; the JSON variant is the
|
|
48
|
+
* "no extra deps" path consistent with the framework's vendoring stance.
|
|
49
|
+
* Operators benchmarking >100K logs/s ship the OTel Collector locally
|
|
50
|
+
* (collector → upstream gRPC) — this sink hands JSON to the local
|
|
51
|
+
* collector which forwards.
|
|
52
|
+
*/
|
|
53
|
+
var C = require("./constants");
|
|
54
|
+
var retryHelper = require("./retry");
|
|
55
|
+
var { LogStreamError } = require("./framework-error");
|
|
56
|
+
var httpClient = require("./http-client");
|
|
57
|
+
var safeUrl = require("./safe-url");
|
|
58
|
+
var authHeader = require("./auth-header");
|
|
59
|
+
|
|
60
|
+
var MAX_RESPONSE_BYTES = C.BYTES.mib(1);
|
|
61
|
+
|
|
62
|
+
// OTel Logs Data Model severity numbers.
|
|
63
|
+
// https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-severitynumber
|
|
64
|
+
var SEVERITY = {
|
|
65
|
+
debug: { number: 5, text: "DEBUG" },
|
|
66
|
+
info: { number: 9, text: "INFO" },
|
|
67
|
+
warn: { number: 13, text: "WARN" },
|
|
68
|
+
error: { number: 17, text: "ERROR" },
|
|
69
|
+
};
|
|
70
|
+
|
|
71
|
+
var DEFAULTS = {
|
|
72
|
+
scopeName: "blamejs",
|
|
73
|
+
batchSize: 100,
|
|
74
|
+
maxBatchAgeMs: C.TIME.seconds(5),
|
|
75
|
+
timeoutMs: C.TIME.seconds(30),
|
|
76
|
+
bufferLimit: 10000,
|
|
77
|
+
};
|
|
78
|
+
|
|
79
|
+
var _err = LogStreamError.factory;
|
|
80
|
+
|
|
81
|
+
function _resolveUrl(url) {
|
|
82
|
+
// If the URL already ends in "/v1/logs" (with or without trailing slash),
|
|
83
|
+
// use as-is. Otherwise treat as the collector root and append.
|
|
84
|
+
var trimmed = url.replace(/\/+$/, "");
|
|
85
|
+
if (/\/v1\/logs$/.test(trimmed)) return trimmed;
|
|
86
|
+
return trimmed + "/v1/logs";
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
function _authHeaders(config) {
|
|
90
|
+
if (config.auth === "header") return Object.assign({}, config.headers || {});
|
|
91
|
+
return authHeader.fromConfig(config);
|
|
92
|
+
}
|
|
93
|
+
|
|
94
|
+
// OTel attribute encoding — JSON form per OTLP spec. Each attribute is
|
|
95
|
+
// { key: <name>, value: { <type>Value: <v> } } where the type field
|
|
96
|
+
// matches the JS type at the boundary.
|
|
97
|
+
function _encodeAttrValue(v) {
|
|
98
|
+
if (v === null || v === undefined) return null;
|
|
99
|
+
var t = typeof v;
|
|
100
|
+
if (t === "string") return { stringValue: v };
|
|
101
|
+
if (t === "boolean") return { boolValue: v };
|
|
102
|
+
if (t === "number") {
|
|
103
|
+
if (Number.isInteger(v)) return { intValue: String(v) }; // OTLP int is string-encoded
|
|
104
|
+
return { doubleValue: v };
|
|
105
|
+
}
|
|
106
|
+
if (Array.isArray(v)) {
|
|
107
|
+
return { arrayValue: { values: v.map(function (e) { return _encodeAttrValue(e); }).filter(Boolean) } };
|
|
108
|
+
}
|
|
109
|
+
if (t === "object") {
|
|
110
|
+
return { kvlistValue: { values: _encodeAttrs(v) } };
|
|
111
|
+
}
|
|
112
|
+
return { stringValue: String(v) };
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
function _encodeAttrs(obj) {
|
|
116
|
+
if (!obj || typeof obj !== "object") return [];
|
|
117
|
+
return Object.keys(obj).map(function (k) {
|
|
118
|
+
var encoded = _encodeAttrValue(obj[k]);
|
|
119
|
+
if (!encoded) return null;
|
|
120
|
+
return { key: k, value: encoded };
|
|
121
|
+
}).filter(Boolean);
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
function _resourceAttrs(cfg) {
|
|
125
|
+
var attrs = {};
|
|
126
|
+
if (cfg.serviceName) attrs["service.name"] = cfg.serviceName;
|
|
127
|
+
if (cfg.serviceVersion) attrs["service.version"] = cfg.serviceVersion;
|
|
128
|
+
if (cfg.resourceAttributes && typeof cfg.resourceAttributes === "object") {
|
|
129
|
+
Object.assign(attrs, cfg.resourceAttributes);
|
|
130
|
+
}
|
|
131
|
+
return attrs;
|
|
132
|
+
}
|
|
133
|
+
|
|
134
|
+
function _toLogRecord(record) {
|
|
135
|
+
var sev = SEVERITY[record.level] || SEVERITY.info;
|
|
136
|
+
// OTel timeUnixNano is a string (JSON can't safely represent 64-bit ints).
|
|
137
|
+
var nanos = String(BigInt(record.ts) * 1000000n);
|
|
138
|
+
var attrs = record.meta ? _encodeAttrs(record.meta) : [];
|
|
139
|
+
return {
|
|
140
|
+
timeUnixNano: nanos,
|
|
141
|
+
observedTimeUnixNano: nanos,
|
|
142
|
+
severityNumber: sev.number,
|
|
143
|
+
severityText: sev.text,
|
|
144
|
+
body: { stringValue: record.message == null ? "" : String(record.message) },
|
|
145
|
+
attributes: attrs,
|
|
146
|
+
};
|
|
147
|
+
}
|
|
148
|
+
|
|
149
|
+
function _serializeBatch(records, cfg, scopeVersion) {
|
|
150
|
+
var resourceAttrs = _resourceAttrs(cfg);
|
|
151
|
+
return Buffer.from(JSON.stringify({
|
|
152
|
+
resourceLogs: [
|
|
153
|
+
{
|
|
154
|
+
resource: {
|
|
155
|
+
attributes: _encodeAttrs(resourceAttrs),
|
|
156
|
+
},
|
|
157
|
+
scopeLogs: [
|
|
158
|
+
{
|
|
159
|
+
scope: {
|
|
160
|
+
name: cfg.scopeName,
|
|
161
|
+
version: scopeVersion,
|
|
162
|
+
},
|
|
163
|
+
logRecords: records.map(_toLogRecord),
|
|
164
|
+
},
|
|
165
|
+
],
|
|
166
|
+
},
|
|
167
|
+
],
|
|
168
|
+
}), "utf8");
|
|
169
|
+
}
|
|
170
|
+
|
|
171
|
+
function _post(url, body, headers, timeoutMs, allowedProtocols, allowInternal) {
|
|
172
|
+
return httpClient.request({
|
|
173
|
+
method: "POST",
|
|
174
|
+
url: url,
|
|
175
|
+
headers: headers,
|
|
176
|
+
body: body,
|
|
177
|
+
idleTimeoutMs: timeoutMs,
|
|
178
|
+
maxResponseBytes: MAX_RESPONSE_BYTES,
|
|
179
|
+
errorClass: LogStreamError,
|
|
180
|
+
allowedProtocols: allowedProtocols,
|
|
181
|
+
allowInternal: allowInternal,
|
|
182
|
+
});
|
|
183
|
+
}
|
|
184
|
+
|
|
185
|
+
function create(config) {
|
|
186
|
+
if (!config || !config.url) throw _err("BAD_OPT", "log-stream otlp requires { url }");
|
|
187
|
+
var cfg = Object.assign({}, DEFAULTS, config);
|
|
188
|
+
var resolvedUrl = _resolveUrl(cfg.url);
|
|
189
|
+
// Validate URL shape + scheme at create time. Default HTTPS-only;
|
|
190
|
+
// operators with an internal cleartext OTel collector pass
|
|
191
|
+
// cfg.allowedProtocols (safeUrl.ALLOW_HTTP_ALL).
|
|
192
|
+
safeUrl.parse(resolvedUrl, {
|
|
193
|
+
allowedProtocols: cfg.allowedProtocols || safeUrl.ALLOW_HTTP_TLS,
|
|
194
|
+
errorClass: LogStreamError,
|
|
195
|
+
});
|
|
196
|
+
var scopeVersion = cfg.scopeVersion || (function () {
|
|
197
|
+
try { return require("../package.json").version; } catch (_e) { return "unknown"; }
|
|
198
|
+
})();
|
|
199
|
+
var headers = Object.assign({
|
|
200
|
+
"Content-Type": "application/json",
|
|
201
|
+
"Accept": "application/json",
|
|
202
|
+
}, _authHeaders(cfg));
|
|
203
|
+
var onDrop = typeof cfg.onDrop === "function" ? cfg.onDrop : null;
|
|
204
|
+
function _emitDrop(reason, batch, err) {
|
|
205
|
+
if (!onDrop) return;
|
|
206
|
+
try { onDrop({ reason: reason, batch: batch, error: err || null }); }
|
|
207
|
+
catch (_e) { /* drop callback is best-effort */ }
|
|
208
|
+
}
|
|
209
|
+
var buffer = [];
|
|
210
|
+
var dropCount = 0;
|
|
211
|
+
var flushTimer = null;
|
|
212
|
+
var inFlight = false;
|
|
213
|
+
var closed = false;
|
|
214
|
+
|
|
215
|
+
function _scheduleFlush() {
|
|
216
|
+
if (flushTimer) return;
|
|
217
|
+
flushTimer = setTimeout(function () { flushTimer = null; _flush(); }, cfg.maxBatchAgeMs);
|
|
218
|
+
flushTimer.unref();
|
|
219
|
+
}
|
|
220
|
+
|
|
221
|
+
async function _flush() {
|
|
222
|
+
if (inFlight) return;
|
|
223
|
+
if (buffer.length === 0) return;
|
|
224
|
+
inFlight = true;
|
|
225
|
+
try {
|
|
226
|
+
while (buffer.length > 0 && !closed) {
|
|
227
|
+
var batch = buffer.splice(0, cfg.batchSize);
|
|
228
|
+
var body = _serializeBatch(batch, cfg, scopeVersion);
|
|
229
|
+
try {
|
|
230
|
+
await retryHelper.withRetry(function () {
|
|
231
|
+
return _post(resolvedUrl, body, headers, cfg.timeoutMs, cfg.allowedProtocols, cfg.allowInternal);
|
|
232
|
+
}, cfg.retry);
|
|
233
|
+
} catch (e) {
|
|
234
|
+
dropCount += batch.length;
|
|
235
|
+
_emitDrop("retry-exhausted", batch, e);
|
|
236
|
+
break;
|
|
237
|
+
}
|
|
238
|
+
}
|
|
239
|
+
} finally {
|
|
240
|
+
inFlight = false;
|
|
241
|
+
if (buffer.length > 0) _scheduleFlush();
|
|
242
|
+
}
|
|
243
|
+
}
|
|
244
|
+
|
|
245
|
+
function emit(record) {
|
|
246
|
+
if (closed) return Promise.resolve({ accepted: false, reason: "sink closed" });
|
|
247
|
+
if (buffer.length >= cfg.bufferLimit) {
|
|
248
|
+
var dropped = buffer.shift();
|
|
249
|
+
dropCount += 1;
|
|
250
|
+
_emitDrop("overflow", [dropped], null);
|
|
251
|
+
}
|
|
252
|
+
buffer.push(record);
|
|
253
|
+
if (buffer.length >= cfg.batchSize) {
|
|
254
|
+
_flush().catch(function () {});
|
|
255
|
+
} else {
|
|
256
|
+
_scheduleFlush();
|
|
257
|
+
}
|
|
258
|
+
return Promise.resolve({ accepted: true, queued: buffer.length });
|
|
259
|
+
}
|
|
260
|
+
|
|
261
|
+
async function close() {
|
|
262
|
+
closed = true;
|
|
263
|
+
if (flushTimer) { clearTimeout(flushTimer); flushTimer = null; }
|
|
264
|
+
await _flush();
|
|
265
|
+
}
|
|
266
|
+
|
|
267
|
+
function stats() {
|
|
268
|
+
return {
|
|
269
|
+
queued: buffer.length,
|
|
270
|
+
dropped: dropCount,
|
|
271
|
+
inFlight: inFlight,
|
|
272
|
+
url: resolvedUrl,
|
|
273
|
+
};
|
|
274
|
+
}
|
|
275
|
+
|
|
276
|
+
return {
|
|
277
|
+
protocol: "otlp",
|
|
278
|
+
emit: emit,
|
|
279
|
+
close: close,
|
|
280
|
+
stats: stats,
|
|
281
|
+
flush: _flush,
|
|
282
|
+
};
|
|
283
|
+
}
|
|
284
|
+
|
|
285
|
+
module.exports = {
|
|
286
|
+
create: create,
|
|
287
|
+
// Exposed for tests + advanced operator wiring.
|
|
288
|
+
_resolveUrl: _resolveUrl,
|
|
289
|
+
_encodeAttrs: _encodeAttrs,
|
|
290
|
+
_toLogRecord: _toLogRecord,
|
|
291
|
+
SEVERITY: SEVERITY,
|
|
292
|
+
};
|
package/lib/log-stream.js
CHANGED
|
@@ -4,12 +4,22 @@
|
|
|
4
4
|
* redaction and bidirectional command channel.
|
|
5
5
|
*
|
|
6
6
|
* Same dispatcher pattern: backends configured per-name with a protocol
|
|
7
|
-
* + protocol-specific options. Built-in protocols
|
|
8
|
-
*
|
|
9
|
-
*
|
|
10
|
-
*
|
|
11
|
-
*
|
|
12
|
-
*
|
|
7
|
+
* + protocol-specific options. Built-in protocols:
|
|
8
|
+
*
|
|
9
|
+
* local — append-only file with rotation
|
|
10
|
+
* webhook — generic HTTP POST; covers Splunk HEC, Datadog, Sumo
|
|
11
|
+
* Logic, Loki, custom collectors that ingest JSON
|
|
12
|
+
* otlp — OpenTelemetry Protocol over HTTP/JSON; ResourceLogs
|
|
13
|
+
* envelope with severity mapping per OTel Logs Data
|
|
14
|
+
* Model. Operators with an OTel collector running
|
|
15
|
+
* (k8s, cloud) get standard log forwarding without a
|
|
16
|
+
* vendor-specific adapter.
|
|
17
|
+
*
|
|
18
|
+
* Adapters listed as deferred and surfacing a clear error when
|
|
19
|
+
* selected:
|
|
20
|
+
*
|
|
21
|
+
* syslog — RFC 5424 syslog over TLS
|
|
22
|
+
* cloudwatch — AWS CloudWatch Logs (PutLogEvents)
|
|
13
23
|
*
|
|
14
24
|
* Every emit goes through lib/redact.js BEFORE any sink sees it. PHI/PCI
|
|
15
25
|
* never reaches operational logs even on a misconfigured field name —
|
|
@@ -34,6 +44,7 @@
|
|
|
34
44
|
*/
|
|
35
45
|
var localProto = require("./log-stream-local");
|
|
36
46
|
var webhookProto = require("./log-stream-webhook");
|
|
47
|
+
var otlpProto = require("./log-stream-otlp");
|
|
37
48
|
var redactor = require("./redact");
|
|
38
49
|
var lazyRequire = require("./lazy-require");
|
|
39
50
|
var protocolDispatcher = require("./protocol-dispatcher");
|
|
@@ -42,10 +53,13 @@ var { LogStreamError } = require("./framework-error");
|
|
|
42
53
|
var dispatcher = protocolDispatcher.create({
|
|
43
54
|
name: "log-stream",
|
|
44
55
|
errorClass: LogStreamError,
|
|
45
|
-
protocols: {
|
|
56
|
+
protocols: {
|
|
57
|
+
"local": localProto,
|
|
58
|
+
"webhook": webhookProto,
|
|
59
|
+
"otlp": otlpProto,
|
|
60
|
+
},
|
|
46
61
|
deferred: {
|
|
47
62
|
"syslog": { description: "RFC 5424 syslog over TLS" },
|
|
48
|
-
"otlp": { description: "OpenTelemetry Logs OTLP/HTTP" },
|
|
49
63
|
"cloudwatch": { description: "AWS CloudWatch Logs (PutLogEvents)" },
|
|
50
64
|
},
|
|
51
65
|
fallbackProtocol: "local",
|
package/lib/queue-local.js
CHANGED
|
@@ -109,12 +109,24 @@ function create(_config) {
|
|
|
109
109
|
cluster.requireLeader();
|
|
110
110
|
opts = opts || {};
|
|
111
111
|
var nowMs = Date.now();
|
|
112
|
-
//
|
|
113
|
-
//
|
|
114
|
-
//
|
|
115
|
-
//
|
|
116
|
-
//
|
|
117
|
-
//
|
|
112
|
+
// ----------------------------------------------------------------------
|
|
113
|
+
// SCHEDULING PRECEDENCE — the contract for every queue.enqueue caller
|
|
114
|
+
// ----------------------------------------------------------------------
|
|
115
|
+
// The queue accepts TWO ways to express "when should this job run":
|
|
116
|
+
// - opts.availableAt (absolute unix-ms) — precise, framework-internal
|
|
117
|
+
// - opts.delaySeconds (integer seconds) — operator shorthand
|
|
118
|
+
//
|
|
119
|
+
// PRECEDENCE: opts.availableAt wins when finite. Operators passing both
|
|
120
|
+
// get the absolute value; the relative form is computed from
|
|
121
|
+
// Date.now() and is therefore strictly less precise (loses sub-second
|
|
122
|
+
// resolution AND drifts on the internal-clock-vs-caller delta).
|
|
123
|
+
//
|
|
124
|
+
// If you're a caller computing a precise time (cron-repeat,
|
|
125
|
+
// scheduler.scheduleAt, notify.deferUntil), pass opts.availableAt
|
|
126
|
+
// ONLY. Don't pass both — the second one is noise (and was the
|
|
127
|
+
// shape that caused the v0.6.21 silent-drift bug). If you only have
|
|
128
|
+
// a relative offset, pass opts.delaySeconds ONLY.
|
|
129
|
+
// ----------------------------------------------------------------------
|
|
118
130
|
var availableAt;
|
|
119
131
|
if (typeof opts.availableAt === "number" && isFinite(opts.availableAt)) {
|
|
120
132
|
availableAt = opts.availableAt;
|
|
@@ -259,8 +271,12 @@ function create(_config) {
|
|
|
259
271
|
await enqueue(unsealedRow.queueName,
|
|
260
272
|
unsealedRow.payload ? safeJson.parse(unsealedRow.payload) : null,
|
|
261
273
|
{
|
|
274
|
+
// availableAt is the precise next-fire ms — pass it alone.
|
|
275
|
+
// Don't also pass delaySeconds (the v0.6.22 / v0.6.23 fix
|
|
276
|
+
// codified that opts.availableAt wins, but mixing both is
|
|
277
|
+
// the shape that masked the silent-drift bug; keep this
|
|
278
|
+
// call site clean as documentation by example).
|
|
262
279
|
availableAt: nextMs,
|
|
263
|
-
delaySeconds: Math.max(0, Math.floor((nextMs - nowMs) / 1000)),
|
|
264
280
|
repeat: { cron: unsealedRow.repeatCron, timezone: unsealedRow.repeatTimezone },
|
|
265
281
|
priority: Number(unsealedRow.priority) || 0,
|
|
266
282
|
classification: unsealedRow.classification || null,
|
package/package.json
CHANGED
package/sbom.cyclonedx.json
CHANGED
|
@@ -2,10 +2,10 @@
|
|
|
2
2
|
"$schema": "http://cyclonedx.org/schema/bom-1.5.schema.json",
|
|
3
3
|
"bomFormat": "CycloneDX",
|
|
4
4
|
"specVersion": "1.5",
|
|
5
|
-
"serialNumber": "urn:uuid:
|
|
5
|
+
"serialNumber": "urn:uuid:aa9c8c0c-504f-4c75-87d4-297f40e4d752",
|
|
6
6
|
"version": 1,
|
|
7
7
|
"metadata": {
|
|
8
|
-
"timestamp": "2026-05-
|
|
8
|
+
"timestamp": "2026-05-02T14:22:45.058Z",
|
|
9
9
|
"lifecycles": [
|
|
10
10
|
{
|
|
11
11
|
"phase": "build"
|
|
@@ -19,14 +19,14 @@
|
|
|
19
19
|
}
|
|
20
20
|
],
|
|
21
21
|
"component": {
|
|
22
|
-
"bom-ref": "@blamejs/core@0.6.
|
|
22
|
+
"bom-ref": "@blamejs/core@0.6.24",
|
|
23
23
|
"type": "library",
|
|
24
24
|
"name": "blamejs",
|
|
25
|
-
"version": "0.6.
|
|
25
|
+
"version": "0.6.24",
|
|
26
26
|
"scope": "required",
|
|
27
27
|
"author": "blamejs contributors",
|
|
28
28
|
"description": "The Node framework that owns its stack.",
|
|
29
|
-
"purl": "pkg:npm/%40blamejs/core@0.6.
|
|
29
|
+
"purl": "pkg:npm/%40blamejs/core@0.6.24",
|
|
30
30
|
"properties": [],
|
|
31
31
|
"externalReferences": [
|
|
32
32
|
{
|
|
@@ -54,7 +54,7 @@
|
|
|
54
54
|
"components": [],
|
|
55
55
|
"dependencies": [
|
|
56
56
|
{
|
|
57
|
-
"ref": "@blamejs/core@0.6.
|
|
57
|
+
"ref": "@blamejs/core@0.6.24",
|
|
58
58
|
"dependsOn": []
|
|
59
59
|
}
|
|
60
60
|
]
|