@blamejs/core 0.6.24 → 0.6.25
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +1 -0
- package/README.md +1 -1
- package/lib/log-stream-cloudwatch.js +309 -0
- package/lib/log-stream.js +82 -11
- package/lib/object-store/sigv4.js +8 -2
- package/package.json +1 -1
- package/sbom.cyclonedx.json +6 -6
package/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,7 @@ upgrading across more than a few patches at a time.
|
|
|
8
8
|
|
|
9
9
|
## v0.6.x
|
|
10
10
|
|
|
11
|
+
- **0.6.25** (2026-05-02) — `b.logStream` gains an AWS CloudWatch Logs sink AND framework-level env wiring. **CloudWatch sink**: `protocol: "cloudwatch"` POSTs `PutLogEvents` over HTTPS with SigV4 signing (service `logs`); operator pre-creates the log group + log stream (the framework does NOT auto-create). Honors IAM role + STS session-token credentials. Respects all three CloudWatch caps automatically: 10,000 events / 1 MiB total payload / 256 KiB-per-event. Per-event oversize dropped at `emit()`-time with `onDrop` fired (truncated message in the drop notification). Per-batch oversize split mid-flush. Permanent AWS errors (`ResourceNotFoundException` / `AccessDeniedException` / `InvalidParameterException` / `UnrecognizedClientException` / `SerializationException`) skip the retry budget. `InvalidSequenceTokenException` (legacy CW accounts) extracts the expected token from the error message and retries with it once. `lib/object-store/sigv4.js` `signRequest()` is now service-agnostic — accepts `opts.service` (default still `"s3"` for back-compat). Removed `cloudwatch` from `DEFERRED_PROTOCOLS`. **`b.logStream.bootFromEnv({ env })`**: framework-level env-driven init mirroring `b.network.bootFromEnv`. Reads `BLAMEJS_LOG_STREAM_PROTOCOL` (`local`/`webhook`/`otlp`/`cloudwatch`), `BLAMEJS_LOG_STREAM_URL`, `BLAMEJS_LOG_STREAM_TOKEN`, `BLAMEJS_LOG_STREAM_SERVICE_NAME`, `BLAMEJS_LOG_STREAM_SERVICE_VERSION`, `BLAMEJS_LOG_STREAM_CLOUDWATCH_LOG_GROUP`, `BLAMEJS_LOG_STREAM_CLOUDWATCH_LOG_STREAM`, `BLAMEJS_LOG_STREAM_PATH`, plus standard AWS_*. Operators get a working log-stream sink without writing build-app code. Wiki app's `build-app.js` replaced its inline env-reading with one `b.logStream.bootFromEnv()` call; both docker-compose configs declare the new env knobs. **Wiki env-snapshot test**: parallel to api-snapshot.json — walks `process.env.X` / `env.X` / `safeEnv.readVar("X")` reads in the wiki app + framework `lib/`, walks docker-compose env declarations, captures the union as `examples/wiki/env-snapshot.json`, fails the e2e gate when env vars are added/removed without updating the snapshot OR when source-only / compose-only gaps appear (env knob declared but unread, env read but undocumented). The validator immediately surfaced 13 real gaps in the wiki app: 5 `WIKI_*` env vars read by source but missing from docker-compose (`WIKI_VAULT_MODE`, `WIKI_DB_AT_REST`, `WIKI_AUDIT_SIGNING_MODE`, `WIKI_BIND`, `WIKI_SITE_URL`), 2 framework env vars (`BLAMEJS_AUDIT_SIGNING_MODE`, `BLAMEJS_TMPDIR`) read by `lib/db.js` via `safeEnv.readVar` but never declared in the wiki's compose configs, and 6 dead env knobs in compose that nothing read. All 13 fixed. Update workflow: `BLAMEJS_UPDATE_ENV_SNAPSHOT=1 node examples/wiki/test/validate-env-snapshot.js` (mirrors the api-snapshot UX). Wiki observability page documents the new sink alongside webhook + otlp; README "What ships in the box" calls out all four log-stream sinks. Tests cover endpoint resolution, event-byte accounting, batch sorting + sequence-token round-trip, permanent-error classifier, validation, round-trip via mock CloudWatch, STS session-token propagation, ResourceNotFoundException no-retry path, 256 KiB per-event hard cap, dispatcher integration, AND batch splitting on the 1-MiB cap (5 quarter-MB events POST as 4 + 1 batches).
|
|
11
12
|
- **0.6.24** (2026-05-02) — `b.logStream` gains an OTLP/HTTP-JSON sink. `protocol: "otlp"` now forwards log records to any OpenTelemetry collector via the OTel Logs Data Model — `resourceLogs` → `scopeLogs` → `logRecords` envelope with `severityNumber` (debug=5/info=9/warn=13/error=17), `severityText`, `timeUnixNano` (string-encoded for JSON-safe 64-bit), `body.stringValue`, and OTel-typed attributes. Operator config: `{ url, serviceName, serviceVersion, resourceAttributes, auth, headers, batchSize, retry, onDrop, ... }` — same back-pressure semantics as the webhook sink (per-sink ring buffer, batched flush on size or maxBatchAgeMs, exponential-backoff retry, drop-on-overflow with operator-supplied onDrop callback). URL convention: `/v1/logs` is auto-appended when the operator passes the collector root. JSON not protobuf — operators benchmarking >100K logs/s ship the OTel Collector locally so the framework hands JSON to a sidecar that forwards via gRPC. Removed `otlp` from `DEFERRED_PROTOCOLS`. Wiki observability page documents the new sink with a `b.logStream.init` example pointing at an OTel collector. Tests cover URL resolution, attribute encoding (string / int / float / bool / array / nested object), severity mapping, round-trip via mock collector, auth header pass-through, retry on 5xx + drop on retry-exhaustion, buffer overflow, dispatcher integration. Verified 0 leaks across 291 commits via `gitleaks`.
|
|
12
13
|
- **0.6.23** (2026-05-02) — v0.6.22 follow-up cleanup. The cron-repeat call site in queue-local.js was passing both `availableAt` AND `delaySeconds` (the former was the precise next-fire ms; the latter was a redundant `Math.floor((nextMs - nowMs) / 1000)` computation that only existed to work around the bug v0.6.22 fixed). Dropped — the cron repeat now passes `availableAt` alone, matching the queue's documented precedence rule. The enqueue() docstring gains a 20-line "SCHEDULING PRECEDENCE" header documenting that `opts.availableAt` wins over `opts.delaySeconds` when both are passed, why the framework chose that direction, and which callers should use which form. New round-trip preservation regression test (`testEnqueueRoundTripsAvailableAt`) covers three precise targets, the delaySeconds-only path, and the both-opts-set case — gates against any future "I'll just rederive it from the floored seconds" mistake. Audited the rest of the framework for the same `(absolute-time, relative-time)` opt-overlap shape (cache.set, session.rotate, apiKey, dualControl): queue is the only primitive carrying both forms, so a generalized `b.time.resolveTimePoint` primitive would be premature with one call site.
|
|
13
14
|
- **0.6.22** (2026-05-02) — `b.queue.enqueue({ availableAt })` is now honoured. Previously the local-protocol enqueue() ignored opts.availableAt entirely, recomputing from `Date.now() + delaySeconds*1000`. The cron-repeat path passes both fields (the exact next-fire ms in availableAt + the floored seconds in delaySeconds), and the enqueue's recomputation lost sub-second precision plus drifted on the internal clock-vs-caller delta. Symptoms: cron-scheduled jobs landed up to 999ms off the intended boundary; the queue-flow-repeat smoke test was intermittently flaky on slow CI runners (caught by ubuntu-latest on the v0.6.21 commit). Fix: enqueue() honours opts.availableAt directly when finite; falls back to delaySeconds-based shorthand otherwise. Operators relying on `enqueue({ availableAt: T })` for non-cron scheduled jobs (e.g. "deliver this notification at exactly 09:00 tomorrow") now get the requested time instead of nowMs+0.
|
package/README.md
CHANGED
|
@@ -47,7 +47,7 @@ The framework bundles the surface a typical Node app reaches for. Every primitiv
|
|
|
47
47
|
- **HTTP** — router with schema-validated routes + OpenAPI publication; full middleware stack (CSRF, CORS, rate-limit, security headers, CSP nonce, body parser, compression, SSE, request log, request-time DB role binding via `b.middleware.dbRoleFor`, in-process CIDR fence via `b.middleware.networkAllowlist`) wired by `createApp`; HTTP/1.1 + HTTP/2 outbound client with SSRF gate, scheme + userinfo + per-host (wildcard / per-method) destination allowlist, redirects, multipart, interceptors, progress, encrypted cookie jar (`b.httpClient`, `b.ssrfGuard`, `b.safeUrl`); operator-tunable network configurability — env-driven NTP / NTS (RFC 8915 authenticated time), DNS with IPv6 / DoH / DoT / cache / lookup timeout, outbound HTTP proxy (`HTTP_PROXY` / `HTTPS_PROXY` / `NO_PROXY`), runtime DPI trust-store CA additions, application-level heartbeats, TCP socket defaults (`b.network`).
|
|
48
48
|
- **Defensive parsers** — `b.safeJson`, `b.safeBuffer`, `b.safeSql`, `b.safeSchema`, `b.parsers` (XML / TOML / YAML / .env), `b.config` (schema-validated env), `b.fileType` magic-byte content classification with deny-on-upload categories (image / document / archive / executable / etc.).
|
|
49
49
|
- **Communication** — WebSockets with channel/room fan-out across cluster replicas (`b.websocket`, `b.websocketChannels`); mail with multipart + attachments + DKIM + calendar invites + bounce intake (`b.mail`, `b.mailBounce`); generic notification dispatcher with operator-supplied transports (`b.notify`).
|
|
50
|
-
- **Observability** — tamper-evident audit chain with SLH-DSA-signed checkpoints, metrics, tracing (OTel pass-through when wired), PII redaction, log-stream sinks (local file rotation, generic webhook, OTLP/HTTP-JSON to an OTel collector), OTLP/HTTP-JSON exporter for traces + metrics (`b.audit`, `b.metrics`, `b.tracing`, `b.redact`, `b.logStream`, `b.otelExport`); operator-callable boot-time security policy assertions (`b.security.assertProduction`) and tamper-evident config-baseline drift detection signed with the audit-signing key (`b.configDrift`).
|
|
50
|
+
- **Observability** — tamper-evident audit chain with SLH-DSA-signed checkpoints, metrics, tracing (OTel pass-through when wired), PII redaction, log-stream sinks (local file rotation, generic webhook, OTLP/HTTP-JSON to an OTel collector, AWS CloudWatch Logs via SigV4), OTLP/HTTP-JSON exporter for traces + metrics (`b.audit`, `b.metrics`, `b.tracing`, `b.redact`, `b.logStream`, `b.otelExport`); operator-callable boot-time security policy assertions (`b.security.assertProduction`) and tamper-evident config-baseline drift detection signed with the audit-signing key (`b.configDrift`).
|
|
51
51
|
- **i18n** — CLDR plural rules, Accept-Language negotiation, Intl formatters, RTL (`b.i18n`).
|
|
52
52
|
- **Format helpers** — RFC 4180 CSV with Excel formula-injection prevention (`b.csv`), RFC 9562 UUID v4 + v7 (`b.uuid`), URL-safe slugs (`b.slug`), TZ-aware datetime (`b.time`), ZIP creation (`b.archive`), HMAC-signed cursor pagination (`b.pagination`), HTML form rendering + validation + CSRF (`b.forms`).
|
|
53
53
|
- **Production** — cluster leader election with fenced leases over Postgres/SQLite (`b.cluster`); cron + interval scheduler that runs exactly-once globally (`b.scheduler`); retry with full-jitter backoff + circuit breaker (`b.retry`); graceful shutdown (`b.appShutdown`); NTP boot check (`b.ntpCheck`); end-to-end-encrypted backup bundles with pre-flush fail-closed mode (`b.backup`); restore with pulled-bundle footprint preflight (`b.restore`); GDPR / PCI / HIPAA-shaped retention rules with multi-stage warn → archive → erase, legal-hold exemptions, dry-run preview, cross-table cascade (`b.retention`).
|
|
@@ -0,0 +1,309 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
/**
|
|
3
|
+
* AWS CloudWatch Logs sink — PutLogEvents over HTTPS with SigV4.
|
|
4
|
+
*
|
|
5
|
+
* Operator config:
|
|
6
|
+
*
|
|
7
|
+
* {
|
|
8
|
+
* region: "us-east-1"
|
|
9
|
+
* accessKeyId: env("AWS_ACCESS_KEY_ID")
|
|
10
|
+
* secretAccessKey: env("AWS_SECRET_ACCESS_KEY")
|
|
11
|
+
* sessionToken: env("AWS_SESSION_TOKEN") // optional, STS creds
|
|
12
|
+
* logGroupName: "my-app-logs" // operator pre-creates
|
|
13
|
+
* logStreamName: "instance-1" // operator pre-creates
|
|
14
|
+
* endpoint: "https://logs.us-east-1.amazonaws.com" // optional
|
|
15
|
+
* batchSize: 100 // CW caps at 10K events / 1 MiB per call
|
|
16
|
+
* maxBatchAgeMs: C.TIME.seconds(5)
|
|
17
|
+
* timeoutMs: C.TIME.seconds(30)
|
|
18
|
+
* retry: { maxAttempts, baseDelayMs, ... }
|
|
19
|
+
* bufferLimit: 10000
|
|
20
|
+
* onDrop: function ({ reason, batch, error }) { ... }
|
|
21
|
+
* }
|
|
22
|
+
*
|
|
23
|
+
* Wire format (Logs_20140328 PutLogEvents — JSON-1.1 over HTTPS):
|
|
24
|
+
*
|
|
25
|
+
* POST /
|
|
26
|
+
* X-Amz-Target: Logs_20140328.PutLogEvents
|
|
27
|
+
* Content-Type: application/x-amz-json-1.1
|
|
28
|
+
* Authorization: AWS4-HMAC-SHA256 Credential=... SignedHeaders=... Signature=...
|
|
29
|
+
* Body: { logGroupName, logStreamName, logEvents: [{ timestamp, message }, ...] }
|
|
30
|
+
*
|
|
31
|
+
* AWS quirks the framework handles:
|
|
32
|
+
* - Events MUST be sorted by timestamp ascending — sink sorts before send.
|
|
33
|
+
* - Per-batch caps: 10,000 events AND <= 1 MiB total payload. Operator
|
|
34
|
+
* batchSize is enforced; the framework also splits batches when the
|
|
35
|
+
* 1 MiB ceiling is reached mid-build.
|
|
36
|
+
* - Per-event 256 KiB hard cap. Oversized events are dropped at emit-time
|
|
37
|
+
* with onDrop fired.
|
|
38
|
+
* - sequenceToken is optional in modern CloudWatch (post-2023). If a
|
|
39
|
+
* legacy account requires it, CloudWatch returns
|
|
40
|
+
* InvalidSequenceTokenException with the expected token; the
|
|
41
|
+
* framework retries with that token transparently.
|
|
42
|
+
* - ResourceNotFoundException -> permanent error (operator forgot to
|
|
43
|
+
* create the log group or stream); surfaced via onDrop with a
|
|
44
|
+
* clear error.
|
|
45
|
+
*
|
|
46
|
+
* SigV4 signing reuses lib/object-store/sigv4.js with service: "logs".
|
|
47
|
+
*/
|
|
48
|
+
var C = require("./constants");
|
|
49
|
+
var nodeCrypto = require("node:crypto");
|
|
50
|
+
var sigv4 = require("./object-store/sigv4");
|
|
51
|
+
var retryHelper = require("./retry");
|
|
52
|
+
var { LogStreamError } = require("./framework-error");
|
|
53
|
+
var httpClient = require("./http-client");
|
|
54
|
+
|
|
55
|
+
var MAX_RESPONSE_BYTES = C.BYTES.mib(1);
|
|
56
|
+
var CW_MAX_EVENTS_PER_BATCH = 10000;
|
|
57
|
+
var CW_MAX_BATCH_BYTES = C.BYTES.mib(1);
|
|
58
|
+
var CW_MAX_EVENT_BYTES = 256 * 1024;
|
|
59
|
+
var CW_EVENT_OVERHEAD_BYTES = 26;
|
|
60
|
+
|
|
61
|
+
var DEFAULTS = {
|
|
62
|
+
batchSize: 100,
|
|
63
|
+
maxBatchAgeMs: C.TIME.seconds(5),
|
|
64
|
+
timeoutMs: C.TIME.seconds(30),
|
|
65
|
+
bufferLimit: 10000,
|
|
66
|
+
};
|
|
67
|
+
|
|
68
|
+
var _err = LogStreamError.factory;
|
|
69
|
+
|
|
70
|
+
function _resolveEndpoint(cfg) {
|
|
71
|
+
if (cfg.endpoint) return cfg.endpoint.replace(/\/+$/, "") + "/";
|
|
72
|
+
return "https://logs." + cfg.region + ".amazonaws.com/";
|
|
73
|
+
}
|
|
74
|
+
|
|
75
|
+
function _eventByteSize(message) {
|
|
76
|
+
return Buffer.byteLength(message, "utf8") + CW_EVENT_OVERHEAD_BYTES;
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
function _serializeBatch(events, cfg, sequenceToken) {
|
|
80
|
+
events.sort(function (a, b) { return a.timestamp - b.timestamp; });
|
|
81
|
+
var body = {
|
|
82
|
+
logGroupName: cfg.logGroupName,
|
|
83
|
+
logStreamName: cfg.logStreamName,
|
|
84
|
+
logEvents: events,
|
|
85
|
+
};
|
|
86
|
+
if (sequenceToken) body.sequenceToken = sequenceToken;
|
|
87
|
+
return Buffer.from(JSON.stringify(body), "utf8");
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
function _signedHeaders(cfg, body) {
|
|
91
|
+
var url = _resolveEndpoint(cfg);
|
|
92
|
+
var payloadHash = nodeCrypto.createHash("sha256").update(body).digest("hex");
|
|
93
|
+
var unsigned = {
|
|
94
|
+
"Content-Type": "application/x-amz-json-1.1",
|
|
95
|
+
"X-Amz-Target": "Logs_20140328.PutLogEvents",
|
|
96
|
+
};
|
|
97
|
+
var signed = sigv4.signRequest({
|
|
98
|
+
method: "POST",
|
|
99
|
+
url: url,
|
|
100
|
+
headers: unsigned,
|
|
101
|
+
payloadHash: payloadHash,
|
|
102
|
+
region: cfg.region,
|
|
103
|
+
service: "logs",
|
|
104
|
+
accessKeyId: cfg.accessKeyId,
|
|
105
|
+
secretAccessKey: cfg.secretAccessKey,
|
|
106
|
+
sessionToken: cfg.sessionToken || null,
|
|
107
|
+
});
|
|
108
|
+
return signed.headers;
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
function _post(cfg, body, headers) {
|
|
112
|
+
return httpClient.request({
|
|
113
|
+
method: "POST",
|
|
114
|
+
url: _resolveEndpoint(cfg),
|
|
115
|
+
headers: headers,
|
|
116
|
+
body: body,
|
|
117
|
+
idleTimeoutMs: cfg.timeoutMs,
|
|
118
|
+
maxResponseBytes: MAX_RESPONSE_BYTES,
|
|
119
|
+
errorClass: LogStreamError,
|
|
120
|
+
allowedProtocols: cfg.allowedProtocols,
|
|
121
|
+
allowInternal: cfg.allowInternal,
|
|
122
|
+
});
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
function _isPermanentAwsError(err) {
|
|
126
|
+
if (!err) return false;
|
|
127
|
+
var msg = err.message || "";
|
|
128
|
+
if (/ResourceNotFoundException/.test(msg)) return true;
|
|
129
|
+
if (/InvalidParameterException/.test(msg)) return true;
|
|
130
|
+
if (/UnrecognizedClientException/.test(msg)) return true;
|
|
131
|
+
if (/AccessDeniedException/.test(msg)) return true;
|
|
132
|
+
if (/SerializationException/.test(msg)) return true;
|
|
133
|
+
return false;
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
function create(config) {
|
|
137
|
+
if (!config || !config.region) {
|
|
138
|
+
throw _err("BAD_OPT", "log-stream cloudwatch requires { region }");
|
|
139
|
+
}
|
|
140
|
+
if (!config.accessKeyId || !config.secretAccessKey) {
|
|
141
|
+
throw _err("BAD_OPT",
|
|
142
|
+
"log-stream cloudwatch requires { accessKeyId, secretAccessKey } " +
|
|
143
|
+
"(IAM role or env-supplied STS credentials)");
|
|
144
|
+
}
|
|
145
|
+
if (!config.logGroupName || !config.logStreamName) {
|
|
146
|
+
throw _err("BAD_OPT",
|
|
147
|
+
"log-stream cloudwatch requires { logGroupName, logStreamName } " +
|
|
148
|
+
"(operator pre-creates both via aws logs create-log-group / create-log-stream " +
|
|
149
|
+
"or CDK / Terraform; the framework does NOT auto-create)");
|
|
150
|
+
}
|
|
151
|
+
var cfg = Object.assign({}, DEFAULTS, config);
|
|
152
|
+
var onDrop = typeof cfg.onDrop === "function" ? cfg.onDrop : null;
|
|
153
|
+
function _emitDrop(reason, batch, err) {
|
|
154
|
+
if (!onDrop) return;
|
|
155
|
+
try { onDrop({ reason: reason, batch: batch, error: err || null }); }
|
|
156
|
+
catch (_e) { /* best-effort */ }
|
|
157
|
+
}
|
|
158
|
+
var buffer = [];
|
|
159
|
+
var dropCount = 0;
|
|
160
|
+
var flushTimer = null;
|
|
161
|
+
var inFlight = false;
|
|
162
|
+
var closed = false;
|
|
163
|
+
var sequenceToken = null;
|
|
164
|
+
|
|
165
|
+
function _scheduleFlush() {
|
|
166
|
+
if (flushTimer) return;
|
|
167
|
+
flushTimer = setTimeout(function () { flushTimer = null; _flush(); }, cfg.maxBatchAgeMs);
|
|
168
|
+
flushTimer.unref();
|
|
169
|
+
}
|
|
170
|
+
|
|
171
|
+
function _takeBatch() {
|
|
172
|
+
var batch = [];
|
|
173
|
+
var totalBytes = 0;
|
|
174
|
+
while (buffer.length > 0) {
|
|
175
|
+
var nextEvent = buffer[0];
|
|
176
|
+
var size = _eventByteSize(nextEvent.message);
|
|
177
|
+
if (batch.length > 0 &&
|
|
178
|
+
(batch.length >= cfg.batchSize ||
|
|
179
|
+
batch.length >= CW_MAX_EVENTS_PER_BATCH ||
|
|
180
|
+
totalBytes + size > CW_MAX_BATCH_BYTES)) {
|
|
181
|
+
break;
|
|
182
|
+
}
|
|
183
|
+
batch.push(buffer.shift());
|
|
184
|
+
totalBytes += size;
|
|
185
|
+
}
|
|
186
|
+
return batch;
|
|
187
|
+
}
|
|
188
|
+
|
|
189
|
+
async function _flush() {
|
|
190
|
+
if (inFlight) return;
|
|
191
|
+
if (buffer.length === 0) return;
|
|
192
|
+
inFlight = true;
|
|
193
|
+
try {
|
|
194
|
+
while (buffer.length > 0 && !closed) {
|
|
195
|
+
var batch = _takeBatch();
|
|
196
|
+
if (batch.length === 0) break;
|
|
197
|
+
try {
|
|
198
|
+
await retryHelper.withRetry(function () {
|
|
199
|
+
return _send(batch);
|
|
200
|
+
}, Object.assign({
|
|
201
|
+
isPermanent: _isPermanentAwsError,
|
|
202
|
+
}, cfg.retry || {}));
|
|
203
|
+
} catch (e) {
|
|
204
|
+
dropCount += batch.length;
|
|
205
|
+
_emitDrop("retry-exhausted", batch, e);
|
|
206
|
+
break;
|
|
207
|
+
}
|
|
208
|
+
}
|
|
209
|
+
} finally {
|
|
210
|
+
inFlight = false;
|
|
211
|
+
if (buffer.length > 0) _scheduleFlush();
|
|
212
|
+
}
|
|
213
|
+
}
|
|
214
|
+
|
|
215
|
+
async function _send(batch) {
|
|
216
|
+
var body = _serializeBatch(batch, cfg, sequenceToken);
|
|
217
|
+
var headers = _signedHeaders(cfg, body);
|
|
218
|
+
var res;
|
|
219
|
+
try {
|
|
220
|
+
res = await _post(cfg, body, headers);
|
|
221
|
+
} catch (e) {
|
|
222
|
+
var match = /expected sequenceToken is:\s*(\S+)/.exec(e.message || "");
|
|
223
|
+
if (match) {
|
|
224
|
+
sequenceToken = match[1];
|
|
225
|
+
var retryBody = _serializeBatch(batch, cfg, sequenceToken);
|
|
226
|
+
var retryHeaders = _signedHeaders(cfg, retryBody);
|
|
227
|
+
res = await _post(cfg, retryBody, retryHeaders);
|
|
228
|
+
} else {
|
|
229
|
+
throw e;
|
|
230
|
+
}
|
|
231
|
+
}
|
|
232
|
+
if (res && res.body) {
|
|
233
|
+
try {
|
|
234
|
+
var parsed = JSON.parse(res.body.toString("utf8"));
|
|
235
|
+
if (parsed && parsed.nextSequenceToken) sequenceToken = parsed.nextSequenceToken;
|
|
236
|
+
} catch (_e) { /* response body not JSON; modern CW returns empty */ }
|
|
237
|
+
}
|
|
238
|
+
return res;
|
|
239
|
+
}
|
|
240
|
+
|
|
241
|
+
function emit(record) {
|
|
242
|
+
if (closed) return Promise.resolve({ accepted: false, reason: "sink closed" });
|
|
243
|
+
var message;
|
|
244
|
+
if (typeof record.message === "string") {
|
|
245
|
+
message = record.message;
|
|
246
|
+
} else {
|
|
247
|
+
message = JSON.stringify(record);
|
|
248
|
+
}
|
|
249
|
+
var size = _eventByteSize(message);
|
|
250
|
+
if (size > CW_MAX_EVENT_BYTES) {
|
|
251
|
+
_emitDrop("event-too-large", [{
|
|
252
|
+
timestamp: record.ts || Date.now(),
|
|
253
|
+
message: message.slice(0, 200) + "...[truncated for drop event]",
|
|
254
|
+
}], new Error("event exceeds 256 KiB CloudWatch hard cap (was " + size + " bytes)"));
|
|
255
|
+
dropCount += 1;
|
|
256
|
+
return Promise.resolve({ accepted: false, reason: "event too large" });
|
|
257
|
+
}
|
|
258
|
+
if (buffer.length >= cfg.bufferLimit) {
|
|
259
|
+
var dropped = buffer.shift();
|
|
260
|
+
dropCount += 1;
|
|
261
|
+
_emitDrop("overflow", [dropped], null);
|
|
262
|
+
}
|
|
263
|
+
buffer.push({
|
|
264
|
+
timestamp: record.ts || Date.now(),
|
|
265
|
+
message: message,
|
|
266
|
+
});
|
|
267
|
+
if (buffer.length >= cfg.batchSize) {
|
|
268
|
+
_flush().catch(function () {});
|
|
269
|
+
} else {
|
|
270
|
+
_scheduleFlush();
|
|
271
|
+
}
|
|
272
|
+
return Promise.resolve({ accepted: true, queued: buffer.length });
|
|
273
|
+
}
|
|
274
|
+
|
|
275
|
+
async function close() {
|
|
276
|
+
closed = true;
|
|
277
|
+
if (flushTimer) { clearTimeout(flushTimer); flushTimer = null; }
|
|
278
|
+
await _flush();
|
|
279
|
+
}
|
|
280
|
+
|
|
281
|
+
function stats() {
|
|
282
|
+
return {
|
|
283
|
+
queued: buffer.length,
|
|
284
|
+
dropped: dropCount,
|
|
285
|
+
inFlight: inFlight,
|
|
286
|
+
sequenceToken: sequenceToken,
|
|
287
|
+
endpoint: _resolveEndpoint(cfg),
|
|
288
|
+
};
|
|
289
|
+
}
|
|
290
|
+
|
|
291
|
+
return {
|
|
292
|
+
protocol: "cloudwatch",
|
|
293
|
+
emit: emit,
|
|
294
|
+
close: close,
|
|
295
|
+
stats: stats,
|
|
296
|
+
flush: _flush,
|
|
297
|
+
};
|
|
298
|
+
}
|
|
299
|
+
|
|
300
|
+
module.exports = {
|
|
301
|
+
create: create,
|
|
302
|
+
_resolveEndpoint: _resolveEndpoint,
|
|
303
|
+
_eventByteSize: _eventByteSize,
|
|
304
|
+
_serializeBatch: _serializeBatch,
|
|
305
|
+
_isPermanentAwsError: _isPermanentAwsError,
|
|
306
|
+
CW_MAX_EVENTS_PER_BATCH: CW_MAX_EVENTS_PER_BATCH,
|
|
307
|
+
CW_MAX_BATCH_BYTES: CW_MAX_BATCH_BYTES,
|
|
308
|
+
CW_MAX_EVENT_BYTES: CW_MAX_EVENT_BYTES,
|
|
309
|
+
};
|
package/lib/log-stream.js
CHANGED
|
@@ -14,12 +14,16 @@
|
|
|
14
14
|
* Model. Operators with an OTel collector running
|
|
15
15
|
* (k8s, cloud) get standard log forwarding without a
|
|
16
16
|
* vendor-specific adapter.
|
|
17
|
+
* cloudwatch — AWS CloudWatch Logs (PutLogEvents) over HTTPS with
|
|
18
|
+
* SigV4. Operator pre-creates the log group + log stream;
|
|
19
|
+
* the framework signs and POSTs batches respecting the
|
|
20
|
+
* 10K-event / 1 MiB / 256 KiB-per-event AWS caps. Honors
|
|
21
|
+
* IAM role + STS session tokens.
|
|
17
22
|
*
|
|
18
23
|
* Adapters listed as deferred and surfacing a clear error when
|
|
19
24
|
* selected:
|
|
20
25
|
*
|
|
21
|
-
* syslog
|
|
22
|
-
* cloudwatch — AWS CloudWatch Logs (PutLogEvents)
|
|
26
|
+
* syslog — RFC 5424 syslog over TLS
|
|
23
27
|
*
|
|
24
28
|
* Every emit goes through lib/redact.js BEFORE any sink sees it. PHI/PCI
|
|
25
29
|
* never reaches operational logs even on a misconfigured field name —
|
|
@@ -42,11 +46,12 @@
|
|
|
42
46
|
* logStream.shutdown()
|
|
43
47
|
* logStream.listSinks() → [{ name, protocol, stats }]
|
|
44
48
|
*/
|
|
45
|
-
var localProto
|
|
46
|
-
var webhookProto
|
|
47
|
-
var otlpProto
|
|
48
|
-
var
|
|
49
|
-
var
|
|
49
|
+
var localProto = require("./log-stream-local");
|
|
50
|
+
var webhookProto = require("./log-stream-webhook");
|
|
51
|
+
var otlpProto = require("./log-stream-otlp");
|
|
52
|
+
var cloudwatchProto = require("./log-stream-cloudwatch");
|
|
53
|
+
var redactor = require("./redact");
|
|
54
|
+
var lazyRequire = require("./lazy-require");
|
|
50
55
|
var protocolDispatcher = require("./protocol-dispatcher");
|
|
51
56
|
var { LogStreamError } = require("./framework-error");
|
|
52
57
|
|
|
@@ -54,13 +59,13 @@ var dispatcher = protocolDispatcher.create({
|
|
|
54
59
|
name: "log-stream",
|
|
55
60
|
errorClass: LogStreamError,
|
|
56
61
|
protocols: {
|
|
57
|
-
"local":
|
|
58
|
-
"webhook":
|
|
59
|
-
"otlp":
|
|
62
|
+
"local": localProto,
|
|
63
|
+
"webhook": webhookProto,
|
|
64
|
+
"otlp": otlpProto,
|
|
65
|
+
"cloudwatch": cloudwatchProto,
|
|
60
66
|
},
|
|
61
67
|
deferred: {
|
|
62
68
|
"syslog": { description: "RFC 5424 syslog over TLS" },
|
|
63
|
-
"cloudwatch": { description: "AWS CloudWatch Logs (PutLogEvents)" },
|
|
64
69
|
},
|
|
65
70
|
fallbackProtocol: "local",
|
|
66
71
|
});
|
|
@@ -197,6 +202,71 @@ function listSinks() {
|
|
|
197
202
|
});
|
|
198
203
|
}
|
|
199
204
|
|
|
205
|
+
// ---- bootFromEnv ----
|
|
206
|
+
//
|
|
207
|
+
// Operator-friendly env-driven init that mirrors b.network.bootFromEnv.
|
|
208
|
+
// Reads BLAMEJS_LOG_STREAM_* env vars and constructs a single-sink
|
|
209
|
+
// configuration matching the operator's choice. Skipped silently when
|
|
210
|
+
// BLAMEJS_LOG_STREAM_PROTOCOL isn't set (operators using the in-code
|
|
211
|
+
// init() path keep their existing wiring).
|
|
212
|
+
//
|
|
213
|
+
// Recognised env vars:
|
|
214
|
+
// BLAMEJS_LOG_STREAM_PROTOCOL "local" | "webhook" | "otlp" | "cloudwatch"
|
|
215
|
+
// BLAMEJS_LOG_STREAM_MIN_LEVEL "debug" | "info" | "warn" | "error"
|
|
216
|
+
//
|
|
217
|
+
// webhook + otlp shared:
|
|
218
|
+
// BLAMEJS_LOG_STREAM_URL
|
|
219
|
+
// BLAMEJS_LOG_STREAM_TOKEN (auth: bearer)
|
|
220
|
+
// otlp-only:
|
|
221
|
+
// BLAMEJS_LOG_STREAM_SERVICE_NAME
|
|
222
|
+
// BLAMEJS_LOG_STREAM_SERVICE_VERSION
|
|
223
|
+
// cloudwatch-only (AWS_* are standard):
|
|
224
|
+
// AWS_REGION, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN
|
|
225
|
+
// BLAMEJS_LOG_STREAM_CLOUDWATCH_LOG_GROUP
|
|
226
|
+
// BLAMEJS_LOG_STREAM_CLOUDWATCH_LOG_STREAM
|
|
227
|
+
// local-only:
|
|
228
|
+
// BLAMEJS_LOG_STREAM_PATH
|
|
229
|
+
function bootFromEnv(opts) {
|
|
230
|
+
opts = opts || {};
|
|
231
|
+
var env = opts.env || process.env;
|
|
232
|
+
var proto = env.BLAMEJS_LOG_STREAM_PROTOCOL;
|
|
233
|
+
if (!proto) return false;
|
|
234
|
+
var sink = { protocol: proto };
|
|
235
|
+
if (proto === "webhook") {
|
|
236
|
+
sink.url = env.BLAMEJS_LOG_STREAM_URL;
|
|
237
|
+
if (env.BLAMEJS_LOG_STREAM_TOKEN) {
|
|
238
|
+
sink.auth = "bearer";
|
|
239
|
+
sink.token = env.BLAMEJS_LOG_STREAM_TOKEN;
|
|
240
|
+
}
|
|
241
|
+
} else if (proto === "otlp") {
|
|
242
|
+
sink.url = env.BLAMEJS_LOG_STREAM_URL;
|
|
243
|
+
sink.serviceName = env.BLAMEJS_LOG_STREAM_SERVICE_NAME || "blamejs";
|
|
244
|
+
sink.serviceVersion = env.BLAMEJS_LOG_STREAM_SERVICE_VERSION || null;
|
|
245
|
+
if (env.BLAMEJS_LOG_STREAM_TOKEN) {
|
|
246
|
+
sink.auth = "bearer";
|
|
247
|
+
sink.token = env.BLAMEJS_LOG_STREAM_TOKEN;
|
|
248
|
+
}
|
|
249
|
+
} else if (proto === "cloudwatch") {
|
|
250
|
+
sink.region = env.AWS_REGION;
|
|
251
|
+
sink.accessKeyId = env.AWS_ACCESS_KEY_ID;
|
|
252
|
+
sink.secretAccessKey = env.AWS_SECRET_ACCESS_KEY;
|
|
253
|
+
sink.sessionToken = env.AWS_SESSION_TOKEN || null;
|
|
254
|
+
sink.logGroupName = env.BLAMEJS_LOG_STREAM_CLOUDWATCH_LOG_GROUP;
|
|
255
|
+
sink.logStreamName = env.BLAMEJS_LOG_STREAM_CLOUDWATCH_LOG_STREAM;
|
|
256
|
+
} else if (proto === "local") {
|
|
257
|
+
sink.path = env.BLAMEJS_LOG_STREAM_PATH;
|
|
258
|
+
} else {
|
|
259
|
+
throw _err("BAD_OPT",
|
|
260
|
+
"BLAMEJS_LOG_STREAM_PROTOCOL='" + proto + "' is not one of " +
|
|
261
|
+
"local | webhook | otlp | cloudwatch (or a custom backend wired via init())");
|
|
262
|
+
}
|
|
263
|
+
init({
|
|
264
|
+
sinks: { primary: sink },
|
|
265
|
+
minLevel: env.BLAMEJS_LOG_STREAM_MIN_LEVEL || undefined,
|
|
266
|
+
});
|
|
267
|
+
return true;
|
|
268
|
+
}
|
|
269
|
+
|
|
200
270
|
function _resetForTest() {
|
|
201
271
|
Object.keys(sinks).forEach(function (n) {
|
|
202
272
|
try { if (sinks[n].raw.close) sinks[n].raw.close(); } catch (_e) {}
|
|
@@ -209,6 +279,7 @@ function _resetForTest() {
|
|
|
209
279
|
|
|
210
280
|
module.exports = {
|
|
211
281
|
init: init,
|
|
282
|
+
bootFromEnv: bootFromEnv,
|
|
212
283
|
emit: emit,
|
|
213
284
|
debug: debug,
|
|
214
285
|
info: info,
|
|
@@ -165,6 +165,12 @@ function signRequest(opts) {
|
|
|
165
165
|
var amzDate = _formatAmzDate(date);
|
|
166
166
|
var dateStamp = _formatDateStamp(date);
|
|
167
167
|
var url = opts.url instanceof URL ? opts.url : new URL(opts.url);
|
|
168
|
+
// service defaults to s3 for back-compat — every call site predating
|
|
169
|
+
// v0.6.25 was object-store / S3. Other AWS services (logs, sqs, sns,
|
|
170
|
+
// kinesis, etc.) pass opts.service explicitly. The credentialScope
|
|
171
|
+
// and signing-key derivation both incorporate the service name, so
|
|
172
|
+
// this MUST match what the target service expects.
|
|
173
|
+
var service = opts.service || SERVICE;
|
|
168
174
|
|
|
169
175
|
var headers = Object.assign({}, opts.headers || {});
|
|
170
176
|
headers["host"] = url.host;
|
|
@@ -177,9 +183,9 @@ function signRequest(opts) {
|
|
|
177
183
|
}
|
|
178
184
|
|
|
179
185
|
var canon = canonicalRequest(opts.method, url, headers, opts.payloadHash);
|
|
180
|
-
var credentialScope = dateStamp + "/" + opts.region + "/" +
|
|
186
|
+
var credentialScope = dateStamp + "/" + opts.region + "/" + service + "/aws4_request";
|
|
181
187
|
var sts = stringToSign(amzDate, credentialScope, canon);
|
|
182
|
-
var signingKey = deriveSigningKey(opts.secretAccessKey, dateStamp, opts.region,
|
|
188
|
+
var signingKey = deriveSigningKey(opts.secretAccessKey, dateStamp, opts.region, service);
|
|
183
189
|
var signature = nodeCrypto.createHmac("sha256", signingKey).update(sts).digest("hex");
|
|
184
190
|
|
|
185
191
|
var canonHeaders = canonicalHeaders(headers);
|
package/package.json
CHANGED
package/sbom.cyclonedx.json
CHANGED
|
@@ -2,10 +2,10 @@
|
|
|
2
2
|
"$schema": "http://cyclonedx.org/schema/bom-1.5.schema.json",
|
|
3
3
|
"bomFormat": "CycloneDX",
|
|
4
4
|
"specVersion": "1.5",
|
|
5
|
-
"serialNumber": "urn:uuid:
|
|
5
|
+
"serialNumber": "urn:uuid:c8d4ecf3-0360-4291-bedc-af54ef2eca2a",
|
|
6
6
|
"version": 1,
|
|
7
7
|
"metadata": {
|
|
8
|
-
"timestamp": "2026-05-02T14:
|
|
8
|
+
"timestamp": "2026-05-02T14:44:53.846Z",
|
|
9
9
|
"lifecycles": [
|
|
10
10
|
{
|
|
11
11
|
"phase": "build"
|
|
@@ -19,14 +19,14 @@
|
|
|
19
19
|
}
|
|
20
20
|
],
|
|
21
21
|
"component": {
|
|
22
|
-
"bom-ref": "@blamejs/core@0.6.
|
|
22
|
+
"bom-ref": "@blamejs/core@0.6.25",
|
|
23
23
|
"type": "library",
|
|
24
24
|
"name": "blamejs",
|
|
25
|
-
"version": "0.6.
|
|
25
|
+
"version": "0.6.25",
|
|
26
26
|
"scope": "required",
|
|
27
27
|
"author": "blamejs contributors",
|
|
28
28
|
"description": "The Node framework that owns its stack.",
|
|
29
|
-
"purl": "pkg:npm/%40blamejs/core@0.6.
|
|
29
|
+
"purl": "pkg:npm/%40blamejs/core@0.6.25",
|
|
30
30
|
"properties": [],
|
|
31
31
|
"externalReferences": [
|
|
32
32
|
{
|
|
@@ -54,7 +54,7 @@
|
|
|
54
54
|
"components": [],
|
|
55
55
|
"dependencies": [
|
|
56
56
|
{
|
|
57
|
-
"ref": "@blamejs/core@0.6.
|
|
57
|
+
"ref": "@blamejs/core@0.6.25",
|
|
58
58
|
"dependsOn": []
|
|
59
59
|
}
|
|
60
60
|
]
|