@foam-ai/node 0.1.0-alpha.11 → 0.1.0-alpha.13

package/README.md

@@ -131,6 +131,7 @@ init({
   additionalResourceAttributes?,
   disableLogSending?,
   networkCapture?,
+  dbCapture?,
   ignoredOutboundHosts?,
   redact?,
   beforeSend?,
@@ -167,6 +168,14 @@ log records is unaffected (see Loggers).
 `networkCapture` defaults to `"basic"`. Upgrade to `"advanced"` unless you
 cannot; it gives much more powerful HTTP and Undici capture. See Network capture.
 
+`dbCapture` defaults to `false` and is accepted but not used yet. It will
+gate database query and result capture from the driver instrumentations
+(statements, row counts, and result payloads where drivers expose them).
+See the TODO on expanded data capture at the end of this document.
+
+TODO(pcga11): Wire dbCapture into the driver instrumentations
+(responseHook/requestHook per driver, opt-in result capture).
+
 `redact` adds keys to Foam's built-in redaction. `secrets` preserves the last
 four characters of longer values; `pii` fully masks values.
 
@@ -211,6 +220,48 @@ TODO(pcga11): Add other common Gen AI providers.
 If another SDK already registered traces, metrics, logs, or the propagator,
 `init()` leaves the signal's slot untouched and only takes what is available. When a slot is taken, use its ingest helper instead.
 
+### `flush()`
+
+Exports all buffered telemetry (spans, metrics, and logs sitting in
+Foam's batch processors) and resolves when the export settles. The SDK
+keeps running.
+
+Use it before a step that may end the process abruptly, or in short-lived
+processes (jobs, scripts, serverless handlers) where waiting for the batch
+interval would lose the final window of telemetry.
+
+```js
+import { flush } from "@foam-ai/node";
+
+await flush();
+```
+
+It never rejects: per-provider export failures are swallowed, matching the
+SDK's fail-open behavior everywhere else. Calling it before `init()` is a
+no-op.
+
+### `shutdown()`
+
+Flushes and permanently stops Foam's providers, releases the global
+OpenTelemetry slots Foam owns (slots owned by another SDK are left
+untouched), and marks the SDK uninitialized so a later `init()` can
+register again.
+
+```js
+import { shutdown } from "@foam-ai/node";
+
+await shutdown();
+```
+
+Use it for a graceful teardown you control (test suites, embedding Foam in
+a tool that starts and stops). Notes:
+
+- Library patches from auto-instrumentation stay in place; after shutdown
+  they emit into unregistered providers, which is inert.
+- `shutdown()` never rejects and is safe to call twice. Foam still shuts
+  its providers down on `beforeExit`, so calling it on a normal exit is
+  optional.
+
 ### `createFoamIngestSpanProcessor(name, environment, token, options?)`
 
 The helper returns a span processor but does not attach it to the
@@ -486,6 +537,7 @@ getState();
 //     metrics: "global",
 //     logs: "global",
 //     baggage: "global",
+//     context: "global",
 //     profile: "none",
 //   },
 // }
@@ -501,14 +553,14 @@ loads.
 Each `signals` value names the source of the Foam export path for that
 signal:
 
-- `"global"` — `init()` registered Foam's provider in the global
+- `"global"`: `init()` registered Foam's provider in the global
   OpenTelemetry slot.
-- `"ingest"` — a `createFoamIngest*` processor/reader was constructed for
+- `"ingest"`: a `createFoamIngest*` processor/reader was constructed for
   another SDK's pipeline. Foam cannot tell whether you actually registered
   the returned object.
-- `"local"` — logs only: another SDK owns the global LoggerProvider, but
+- `"local"` (logs only): another SDK owns the global LoggerProvider, but
   Foam keeps its own local provider so `log()` still delivers to Foam.
-- `"none"` — no Foam export path.
+- `"none"`: no Foam export path.
 
 A value other than `"none"` means telemetry is expected, not that the app
 has already produced or exported it.
@@ -580,9 +632,9 @@ well-known names and any key containing a sensitive term (`legacy_api_key_2`,
 application-specific keys with the `redact` option shown under `init(options)`;
 customer keys match exactly after normalization.
 
-Redaction applies to everything Foam exports — span, span event, and link
-attributes, log bodies and attributes, metric data-point attributes, and
-resource attributes — including the ingest helpers. It does not modify
+Redaction applies to everything Foam exports, including the ingest
+helpers: span, span event, and link attributes, log bodies and attributes,
+metric data-point attributes, and resource attributes. It does not modify
 telemetry sent through other exporters.
 
 ## Loggers
@@ -646,6 +698,85 @@ The official OpenTelemetry JavaScript SDK has no in-process Profiles provider, p
 - [OpenTelemetry Profiles public alpha announcement](https://opentelemetry.io/blog/2026/profiles-alpha/)
 - [OpenTelemetry eBPF profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler)
 
+## TODO(pcga11): Expanded data capture (http2, gRPC, DB results, messaging)
+
+Extension points Foam is not using yet, ordered by how much new data they
+unlock. All payload extraction flows through the same redaction and size
+caps as HTTP network capture.
+
+### Database results: biggest win, zero patching
+
+`responseHook` hands Foam the raw driver result (in memory, no extra query)
+before the span ends. Each package has its own payload shape, so do not
+assume a shared contract: `pg` passes `{ data }`, `mysql2`
+`{ queryResults }`, `mongoose` `{ response }`, `mongodb` the command
+response, `cassandra-driver` only the first result page, and
+`ioredis`/`redis` (v4+) use positional `(cmdName, cmdArgs, response)`
+args. Also `requestHook` (pg, ioredis, oracledb) for full query/args, and
+`maskStatementHook` (mysql2) to wire SQL text into Foam redaction. No
+hooks (spans only): `mysql` v1, `tedious`, `memcached`. Ship as opt-in:
+results are the most PII-dense payload in the system.
+`enhancedDatabaseReporting` (adds bound parameter values) must likewise
+stay opt-in; redaction cannot key-match values inside SQL text.
+
+### http2 + gRPC: currently fully dark
+
+Node ships `http2.client.stream.*` / `http2.server.stream.*` lifecycle
+channels since 24.1 (backported to 22.x), and the request-body channels
+`bodyChunkSent` / `bodySent` since 24.12.0 / 22.22.1; pin those floors in
+`support.ts`, gated on the Node version (core emits these, unlike undici's
+own channels). The adapter mirrors `src/network-capture/undici.ts` but the
+payloads differ: Node publishes `{ stream, writev, data, encoding }` for
+`bodyChunkSent` and `{ stream }` for `bodySent`, vs undici's
+`{ request, chunk }`, so map fields explicitly. No OTel instrumentation
+exists for http2, so Foam creates the spans too. Response bodies have no
+channel and the exposed `ClientHttp2Stream` is live, not replayable:
+collect passively (wrap `push()` as `http.ts` does) without reading the
+stream or switching it into flowing mode, or leave response capture out.
+For gRPC, `instrumentation-grpc` gives spans but zero payload hooks
+(only `metadataToSpanAttributes`; enable it). The http2 body chunks carry
+gRPC framing, not bare protobuf: reassemble the 1-byte compression flag +
+4-byte big-endian length prefix per message, decompressing flagged
+messages, before any protobuf decode, or skip framing entirely with a
+custom patch of `@grpc/grpc-js` serialization.
+
+### Messaging and cloud payloads: hooks receive the actual message
+
+- `kafkajs` `producerHook`/`consumerHook` receive `{ topic, message }`
+  (value, key, headers); `amqplib` `publishHook`/`consumeHook`
+  (+confirm/end variants) receive the message `content` plus
+  routing/options metadata; shapes differ per hook.
+- `socket.io` `emitHook`/`onHook`: event payloads (covers the socket.io
+  slice of WebSocket traffic).
+- `aws-sdk` `preRequestHook`/`responseHook`/`exceptionHook` (the latter
+  gets `(span, requestInfo, err)`): normalized request/response for every
+  AWS call. SDK v3 only.
+
+### Crash and shutdown capture: data we currently lose
+
+Errors outside any span (timer callbacks, unhandled rejections, startup)
+are invisible, and `beforeExit` (Foam's only lifecycle hook) does not
+fire on crashes or signals, so the batch holding the spans that explain a
+crash dies with the process, and every SIGTERM (each deploy) drops the
+final batch window. Use `uncaughtExceptionMonitor` (never plain
+`uncaughtException`, which suppresses the default crash), but note an
+async flush started there is abandoned when the process terminates, so
+crash delivery needs a synchronous durable handoff (sync write to disk, or
+a helper process that ships it) rather than relying on the async exporter.
+SIGTERM/SIGINT get flush-then-resignal handlers with a bounded flush
+timeout that never swallows the signal or changes the exit code. `warning`
+events ship as logs.
+
+### Smaller / later
+
+- `logHook` (bunyan/pino/winston), `graphql` `responseHook`,
+  `express`/`koa`/`restify` `requestHook`: enrichment, little new data.
+- Channels: `worker_threads` (detect uninstrumented workers), 24+ native
+  `console.*` (could replace patch-based console capture),
+  `tracing:module.*` (library-version detection), `child_process`.
+- No hook surface at all: raw `ws` frames (custom patch, frame data model)
+  and Prisma (Rust engine; needs app-enabled Prisma OTel tracing).
+
 ## TODO(pcga11): Anthropic instrumentation
 
 There is no released official OpenTelemetry JavaScript instrumentation for the Anthropic SDK yet, but one is actively in progress in js-contrib, based on the OpenInference donation from Arize. Once `@opentelemetry/instrumentation-anthropic` is released (and picked up by `auto-instrumentations-node`), bundle it here like the OpenAI and aws-sdk ones. Until then, Anthropic coverage is handled case by case with the FDE (see FDE.md). Track upstream progress:

package/dist/constants.d.ts

@@ -4,7 +4,7 @@ export declare const FOAM_INGEST_HOST: string;
 export declare const ATTR_FOAM_INGEST_HOST = "foam.ingest.host";
 export declare const FOAM_IDENTIFIER_NAME = "[foam-otel]";
 export declare const FOAM_DISTRO_NAME = "@foam-ai/node";
-export declare const FOAM_DISTRO_VERSION = "0.1.0-alpha.11";
+export declare const FOAM_DISTRO_VERSION = "0.1.0-alpha.13";
 export declare const FOAM_OTLP_TRACES_PATH = "/v1/traces";
 export declare const FOAM_OTLP_LOGS_PATH = "/v1/logs";
 export declare const FOAM_OTLP_METRICS_PATH = "/v1/metrics";

package/dist/constants.js

@@ -7,7 +7,7 @@ exports.FOAM_INGEST_HOST = new URL(exports.FOAM_ENDPOINT).hostname;
 exports.ATTR_FOAM_INGEST_HOST = "foam.ingest.host";
 exports.FOAM_IDENTIFIER_NAME = "[foam-otel]";
 exports.FOAM_DISTRO_NAME = "@foam-ai/node";
-exports.FOAM_DISTRO_VERSION = "0.1.0-alpha.11";
+exports.FOAM_DISTRO_VERSION = "0.1.0-alpha.13";
 exports.FOAM_OTLP_TRACES_PATH = "/v1/traces";
 exports.FOAM_OTLP_LOGS_PATH = "/v1/logs";
 exports.FOAM_OTLP_METRICS_PATH = "/v1/metrics";

package/dist/index.d.ts

@@ -1,4 +1,5 @@
 export { init, type InitOptions } from "./init.js";
+export { flush, shutdown } from "./lifecycle.js";
 export type { BeforeSendEvent, BeforeSendHook, } from "./before-send.js";
 export type { RedactOptions } from "./redaction.js";
 export { getState } from "./state.js";

package/dist/index.js

@@ -1,8 +1,11 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.recordException = exports.SeverityNumber = exports.log = exports.setMetric = exports.recordHistogram = exports.incrementCounter = exports.addUpDownCounter = exports.setBaggage = exports.injectTraceContext = exports.getBaggage = exports.extractTraceContext = exports.createFoamIngestSpanProcessor = exports.createFoamIngestMetricReader = exports.createFoamIngestLogRecordProcessor = exports.setEndpoint = exports.getState = exports.init = void 0;
+exports.recordException = exports.SeverityNumber = exports.log = exports.setMetric = exports.recordHistogram = exports.incrementCounter = exports.addUpDownCounter = exports.setBaggage = exports.injectTraceContext = exports.getBaggage = exports.extractTraceContext = exports.createFoamIngestSpanProcessor = exports.createFoamIngestMetricReader = exports.createFoamIngestLogRecordProcessor = exports.setEndpoint = exports.getState = exports.shutdown = exports.flush = exports.init = void 0;
 var init_js_1 = require("./init.js");
 Object.defineProperty(exports, "init", { enumerable: true, get: function () { return init_js_1.init; } });
+var lifecycle_js_1 = require("./lifecycle.js");
+Object.defineProperty(exports, "flush", { enumerable: true, get: function () { return lifecycle_js_1.flush; } });
+Object.defineProperty(exports, "shutdown", { enumerable: true, get: function () { return lifecycle_js_1.shutdown; } });
 var state_js_1 = require("./state.js");
 Object.defineProperty(exports, "getState", { enumerable: true, get: function () { return state_js_1.getState; } });
 var endpoint_js_1 = require("./endpoint.js");

package/dist/init.d.ts

@@ -13,6 +13,7 @@ export interface InitOptions {
     sampleRate?: number;
     disableLogSending?: boolean;
     networkCapture?: "off" | "basic" | "advanced";
+    dbCapture?: boolean;
     enabled?: boolean;
     redact?: RedactOptions;
     beforeSend?: BeforeSendHook;

package/dist/init.js

@@ -14,9 +14,12 @@ const report_js_1 = require("./report.js");
 const exporters_js_1 = require("./exporters.js");
 const instrumentations_js_1 = require("./instrumentations.js");
 const before_send_js_1 = require("./before-send.js");
+const lifecycle_js_1 = require("./lifecycle.js");
 const logs_js_1 = require("./logs.js");
 const redaction_js_1 = require("./redaction.js");
 const state_js_1 = require("./state.js");
+const utils_js_1 = require("./utils.js");
+let exitHookInstalled = false;
 // TODO(pcga11): conflicts handling pending.
 function init(options) {
     const { sampleRate = 1, disableLogSending = false, networkCapture = "basic", enabled = true, } = options;
@@ -66,10 +69,6 @@ function init(options) {
         });
         return;
     }
-    // pcga11: Hoisted so the catch block can shut down providers whose batch/periodic
-    // export timers started in their constructors, even when init fails midway.
-    let tracerProvider;
-    let meterProvider;
     try {
         (0, redaction_js_1.setActiveRedactionConfig)(redactionConfig);
         const resource = (0, resource_js_1.createFoamResource)(options.name, options.environment, options.version, options.additionalResourceAttributes);
@@ -79,7 +78,10 @@ function init(options) {
         // is first-wins: if another SDK already installed one, we keep theirs and disable ours.
         const contextManager = new context_async_hooks_1.AsyncLocalStorageContextManager();
         contextManager.enable();
-        if (!api_1.context.setGlobalContextManager(contextManager)) {
+        if (api_1.context.setGlobalContextManager(contextManager)) {
+            (0, state_js_1.setSignal)(state_js_1.Signals.context, state_js_1.SignalSources.global);
+        }
+        else {
             contextManager.disable();
         }
         (0, state_js_1.setSignal)(state_js_1.Signals.baggage, api_1.propagation.setGlobalPropagator(new core_1.CompositePropagator({
@@ -90,7 +92,9 @@ function init(options) {
         })) ? state_js_1.SignalSources.global : state_js_1.SignalSources.none);
         // pcga11: The @opentelemetry/api setters return false when another SDK already owns a slot (registration is first-wins). So we try to register our own and if it fails or already taken, nothing happens.
         // TODO(pcga11): Implement a way to handle piggy backing the context and propagation from another SDK.
-        tracerProvider = new sdk_trace_base_1.BasicTracerProvider({
+        // pcga11: Providers are registered at construction (their export timers
+        // start in the constructor) so the catch's shutdown() always finds them.
+        const tracerProvider = (0, lifecycle_js_1.setFoamTracerProvider)(new sdk_trace_base_1.BasicTracerProvider({
             resource,
             sampler: new sdk_trace_base_1.ParentBasedSampler({
                 root: new sdk_trace_base_1.TraceIdRatioBasedSampler(sampleRate),
@@ -102,8 +106,8 @@ function init(options) {
                 })),
                 ...(options.additionalSpanProcessors ?? []),
             ],
-        });
-        meterProvider = new sdk_metrics_1.MeterProvider({
+        }));
+        const meterProvider = (0, lifecycle_js_1.setFoamMeterProvider)(new sdk_metrics_1.MeterProvider({
             resource,
             readers: [
                 new sdk_metrics_1.PeriodicExportingMetricReader({
@@ -113,7 +117,7 @@ function init(options) {
                 }),
                 ...(options.additionalMetricReaders ?? []),
             ],
-        });
+        }));
         const loggerProvider = new sdk_logs_1.LoggerProvider({
             resource,
             processors: [
@@ -135,13 +139,12 @@ function init(options) {
         // pcga11: registerInstrumentations turns on instrumentation "hooks". OpenTelemetry wraps supported libraries when the app imports them.
         (0, instrumentation_1.registerInstrumentations)({ instrumentations });
         (0, state_js_1.setInstrumentations)(instrumentations.map(instrumentations_js_1.instrumentationLabel));
-        process.once("beforeExit", () => {
-            void Promise.all([
-                (0, state_js_1.getSignal)(state_js_1.Signals.traces) === state_js_1.SignalSources.global ? tracerProvider?.shutdown() : undefined,
-                (0, state_js_1.getSignal)(state_js_1.Signals.metrics) === state_js_1.SignalSources.global ? meterProvider?.shutdown() : undefined,
-                loggerProvider.shutdown(),
-            ]).catch(() => undefined);
-        });
+        // pcga11: shutdown() never rejects, so it doubles as the exit hook.
+        // Installed once per process; re-inits must not stack listeners.
+        if (!exitHookInstalled) {
+            exitHookInstalled = true;
+            process.once("beforeExit", () => void (0, lifecycle_js_1.shutdown)());
+        }
         (0, state_js_1.setInitialized)(true);
         (0, report_js_1.report)({
             token: options.token,
@@ -150,17 +153,10 @@ function init(options) {
         });
     }
     catch (error) {
-        const errorMessage = error instanceof Error ? error.message : String(error);
-        for (const signal of [state_js_1.Signals.traces, state_js_1.Signals.metrics, state_js_1.Signals.logs, state_js_1.Signals.baggage]) {
-            (0, state_js_1.setSignal)(signal, state_js_1.SignalSources.none);
-        }
-        void Promise.all([
-            tracerProvider?.shutdown(),
-            meterProvider?.shutdown(),
-            (0, logs_js_1.getFoamLoggerProvider)()?.shutdown(),
-        ]).catch(() => undefined);
-        (0, logs_js_1.clearFoamLoggerProvider)();
-        (0, state_js_1.setInitialized)(false);
+        // pcga11: Nothing in a catch may throw (it would escape into app code):
+        // shutdown() never rejects, safely() covers throwing toString().
+        void (0, lifecycle_js_1.shutdown)();
+        const errorMessage = (0, utils_js_1.safely)(() => (error instanceof Error ? error.message : String(error)), "unknown");
         (0, report_js_1.report)({
             token: options.token,
             severity: api_logs_1.SeverityNumber.ERROR,

package/dist/lifecycle.d.ts

@@ -0,0 +1,6 @@
+import type { MeterProvider } from "@opentelemetry/sdk-metrics";
+import type { BasicTracerProvider } from "@opentelemetry/sdk-trace-base";
+export declare function setFoamTracerProvider(provider: BasicTracerProvider): BasicTracerProvider;
+export declare function setFoamMeterProvider(provider: MeterProvider): MeterProvider;
+export declare function flush(): Promise<void>;
+export declare function shutdown(): Promise<void>;

package/dist/lifecycle.js

@@ -0,0 +1,53 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.setFoamTracerProvider = setFoamTracerProvider;
+exports.setFoamMeterProvider = setFoamMeterProvider;
+exports.flush = flush;
+exports.shutdown = shutdown;
+const api_1 = require("@opentelemetry/api");
+const api_logs_1 = require("@opentelemetry/api-logs");
+const logs_js_1 = require("./logs.js");
+const utils_js_1 = require("./utils.js");
+const state_js_1 = require("./state.js");
+// pcga11: How flush()/shutdown() reach the providers. init() registers each
+// at construction, so the registry is current even when init() fails midway.
+// The logger provider has its own registry in logs.ts.
+let foamTracerProvider;
+let foamMeterProvider;
+function setFoamTracerProvider(provider) {
+    foamTracerProvider = provider;
+    return provider;
+}
+function setFoamMeterProvider(provider) {
+    foamMeterProvider = provider;
+    return provider;
+}
+// pcga11: Both methods fail open: the async mappers turn synchronous throws
+// into rejections and allSettled swallows them, so neither can ever reject.
+const providers = () => [foamTracerProvider, foamMeterProvider, (0, logs_js_1.getFoamLoggerProvider)()];
+async function flush() {
+    await Promise.allSettled(providers().map(async (provider) => provider?.forceFlush()));
+}
+async function shutdown() {
+    const active = providers();
+    foamTracerProvider = undefined;
+    foamMeterProvider = undefined;
+    (0, logs_js_1.clearFoamLoggerProvider)();
+    // pcga11: Release only the global API slots Foam owns (registration is
+    // first-wins, so a later SDK could not register otherwise). Slots owned
+    // by another SDK are left untouched.
+    const slots = [
+        [state_js_1.Signals.traces, api_1.trace],
+        [state_js_1.Signals.metrics, api_1.metrics],
+        [state_js_1.Signals.logs, api_logs_1.logs],
+        [state_js_1.Signals.baggage, api_1.propagation],
+        [state_js_1.Signals.context, api_1.context],
+    ];
+    for (const [signal, api] of slots) {
+        if ((0, state_js_1.getSignal)(signal) === state_js_1.SignalSources.global)
+            (0, utils_js_1.safely)(() => api.disable());
+        (0, state_js_1.setSignal)(signal, state_js_1.SignalSources.none);
+    }
+    (0, state_js_1.setInitialized)(false);
+    await Promise.allSettled(active.map(async (provider) => provider?.shutdown()));
+}

package/dist/state.d.ts

@@ -3,6 +3,7 @@ export declare enum Signals {
     metrics = "metrics",
     logs = "logs",
     baggage = "baggage",
+    context = "context",
     profile = "profile"
 }
 export declare enum SignalSources {

package/dist/state.js

@@ -7,6 +7,7 @@ var Signals;
     Signals["metrics"] = "metrics";
     Signals["logs"] = "logs";
     Signals["baggage"] = "baggage";
+    Signals["context"] = "context";
     Signals["profile"] = "profile";
 })(Signals || (exports.Signals = Signals = {}));
 var SignalSources;
@@ -24,6 +25,7 @@ const state = {
         metrics: SignalSources.none,
         logs: SignalSources.none,
         baggage: SignalSources.none,
+        context: SignalSources.none,
         profile: SignalSources.none,
     },
     params: {},

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@foam-ai/node",
-  "version": "0.1.0-alpha.11",
+  "version": "0.1.0-alpha.13",
   "description": "Foam JavaScript Node.js SDK",
   "license": "Apache-2.0",
   "repository": {