@modainteract/moda-interact-shared 0.2.0 → 0.4.0

package/README.md

@@ -4,8 +4,8 @@ Shared TypeScript contracts and reusable platform primitives used by multiple
 Moda Interact services.
 
 This package is a **library**, not a deployable service. It does not own
-transport, persistence, background workers, provider SDK integration, or
-application business workflows.
+business transports, persistence, background workers, provider SDK integration,
+or application business workflows.
 
 ## Install
 
@@ -28,10 +28,117 @@ including:
 - canonical queue/job-name constants where multiple repositories must agree;
 - deterministic correlation/job identifier helpers;
 - small pure utilities used by more than one service;
-- reusable structured application logging primitives.
+- reusable structured application logging primitives;
+- the reusable base Node OpenTelemetry runtime, including tracing and metrics
+  providers, exporters, sampling, HTTP/Undici instrumentation, lifecycle, and
+  generic observability helpers.
 
 The package deliberately keeps framework and runtime dependencies small.
 
+## Node observability runtime
+
+The package provides a browser-safe semantic observability entry and a
+Node-only runtime entry:
+
+```ts
+import {
+  getActiveTraceId,
+  withObservedSpan,
+} from "@modainteract/moda-interact-shared/observability";
+
+import {
+  initNodeObservability,
+} from "@modainteract/moda-interact-shared/observability/node";
+```
+
+Import `./observability/node` only from a process preload. It installs tracing,
+metrics, HTTP/HTTPS and Undici/fetch instrumentation before framework or worker
+modules load:
+
+```text
+node --import ./observability.mjs <framework-or-worker-entrypoint>
+```
+
+```js
+// observability.mjs
+import { initNodeObservability } from
+  "@modainteract/moda-interact-shared/observability/node";
+
+initNodeObservability({
+  serviceName: "moda-interact-messaging",
+  environment: process.env.DEPLOYMENT_ENVIRONMENT_NAME,
+  instrument: { http: true, fetch: true, prisma: false },
+});
+```
+
+Set `prisma: true` only for processes that load Prisma, and keep the preload
+before the first Prisma application import.
+
+The runtime uses the canonical `service.namespace`, `service.name`, and
+`deployment.environment.name` resource attributes. It honors
+`OTEL_TRACES_SAMPLER`, ratio arguments for ratio samplers, signal-specific or
+generic OTLP endpoints, and bounded batch/export settings. With no trace or
+metric endpoint those pipelines are no-ops while structured stdout logging
+continues.
+
+`forceFlush()` covers traces, metrics, OpenTelemetry Logs, and direct Loki.
+Initialization, export, flush, and shutdown failures are best-effort and do not
+become application startup or business-operation failures.
+
+### BullMQ telemetry
+
+Create the shared BullMQ telemetry adapter once per Queue or Worker options
+object and pass it through BullMQ's native `telemetry` option:
+
+```ts
+import { createBullMQTelemetry } from
+  "@modainteract/moda-interact-shared/observability/bullmq";
+
+const telemetry = createBullMQTelemetry({
+  serviceName: "moda-messaging-worker",
+  enableMetrics: true,
+});
+
+new Worker(queueName, processor, { connection, telemetry });
+```
+
+The adapter uses the global trace and meter providers installed by the Node
+runtime. BullMQ owns propagation metadata in its job options; applications must
+not copy or mutate queue payloads solely to carry trace context. BullMQ metrics
+use queue name, bounded job name, and job state dimensions, never Moda job,
+shop, checkout, conversation, or message IDs.
+
+### GenAI active spans
+
+Use the GenAI helpers to model one inbound conversation turn as an independent
+trace with active agent, tool, and automatically instrumented child spans:
+
+```ts
+import {
+  observeAgentInvocation,
+  observeAgentTool,
+  observeConversationTurn,
+} from "@modainteract/moda-interact-shared/observability/genai";
+
+await observeConversationTurn("whatsapp", () =>
+  observeAgentInvocation({ agentName: "commerce-agent" }, () =>
+    observeAgentTool("lookup-products", executeTool),
+  ),
+);
+```
+
+The helpers use only the global tracer and meter providers installed by the
+Node runtime. They create no SDK, provider, exporter, or network request, and
+remain lightweight no-ops when providers are absent. Six module-singleton
+instruments record turn, agent, and tool duration and operation outcomes.
+Metric attributes are limited to `outcome=success|error`; turn metrics also use
+the closed `channel=whatsapp|other` vocabulary. Arbitrary agent, provider,
+model, tool, and Moda business identifiers never become metric dimensions.
+
+Agent, provider, model, and tool names are trimmed and bounded on spans only.
+Prompt, completion, message, and tool payload bodies are never accepted or
+captured by default.
+
 ## What this package does not own
 
 This package does **not** own:
@@ -42,12 +149,13 @@ This package does **not** own:
 - Shopify or Meta webhook HTTP handlers;
 - Shopify or Meta SDK integration;
 - application recovery/business logic;
-- OpenTelemetry spans or application-specific metrics;
-- OTLP transport or observability backends;
+- service/domain-specific span names, business attributes, or application
+  metrics;
+- deployment OTLP endpoint/credential wiring or observability backends;
 - Render deployment configuration.
 
-Those capabilities remain in the repository that owns the corresponding
-runtime concern.
+Those service-specific semantics and deployment concerns remain in the
+repository that owns the corresponding runtime.
 
 ---
 
@@ -302,25 +410,297 @@ traces by default.
 Logging is best-effort: a logger sink or serialization failure must not alter
 the success/failure semantics of the business operation being logged.
 
-## Logging and OpenTelemetry are separate
+## Logging destinations
+
+Every call to the shared logger:
+
+```ts
+logger.debug(...)
+logger.info(...)
+logger.warn(...)
+logger.error(...)
+```
 
-The shared logger does **not** automatically create:
+emits the same already-sanitized canonical record to up to three independent,
+best-effort destinations:
 
-- spans;
-- counters;
-- histograms;
-- OTLP exports.
+1. canonical structured JSON to the configured sink (default: console);
+2. the OpenTelemetry Logs API via the global `LoggerProvider`, if one has been
+   installed;
+3. a direct Grafana Loki transport, if the Node process bootstrap has installed
+   the Loki emitter.
 
-Service-specific OpenTelemetry remains in the owning runtime repository.
+Each destination is failure-isolated: a failure in one must not suppress
+another and must never affect application/business correctness.
+
+The OpenTelemetry API is a no-op when no provider is installed, so stdout
+logging continues to work in local/test environments without any backend.
+
+The heavy Node SDK/exporter and the Winston/winston-loki transport are isolated
+in:
+
+```text
+@modainteract/moda-interact-shared/logging/node
+```
 
-For example:
+and must be imported and initialized only from the Node process bootstrap —
+never through Vite/application/browser bundles.
+
+```text
+application module
+    |
+    v
+@modainteract/moda-interact-shared/logging
+    |
+    +--> canonical JSON stdout
+    |
+    +--> @opentelemetry/api-logs
+    |        |
+    |        | global LoggerProvider
+    |        v
+    |   ./logging/node
+    |        |
+    |        +--> LoggerProvider
+    |        +--> BatchLogRecordProcessor
+    |        +--> OTLPLogExporter
+    |                 |
+    |                 v
+    |              /v1/logs
+    |
+    +--> lightweight Loki bridge (globalThis emitter)
+             |
+             v
+        ./logging/node
+             |
+             +--> Winston + winston-loki
+                      |
+                      v
+                  Grafana Loki
+```
+
+Winston and winston-loki exist **only** in the Node-only package graph. The
+normal `./logging` entry contains no Winston, winston-loki or Node networking
+imports.
+
+### Node bootstrap
+
+Process bootstrap code installs the optional destinations once:
+
+```ts
+import {
+  initNodeLokiLogging,
+  initNodeOpenTelemetryLogging,
+} from "@modainteract/moda-interact-shared/logging/node";
+
+initNodeLokiLogging({
+  serviceName: "moda-interact",
+  environment: "production",
+});
+
+initNodeOpenTelemetryLogging({
+  serviceName: "moda-interact",
+  environment: "production",
+});
+```
+
+Each destination is independently configurable, so a deployment may use:
+
+```text
+console only
+
+console + Loki
+
+console + OTel Logs
+
+console + Loki + OTel Logs
+```
+
+Either Node destination may be omitted entirely; without an installed
+provider/emitter the logger remains a safe stdout-only logger.
+
+The bootstrap supports the standard OpenTelemetry environment variables:
+
+```text
+OTEL_SDK_DISABLED
+
+OTEL_LOGS_EXPORTER
+
+OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
+OTEL_EXPORTER_OTLP_ENDPOINT
+
+OTEL_EXPORTER_OTLP_LOGS_HEADERS
+OTEL_EXPORTER_OTLP_HEADERS
+
+OTEL_BLRP_MAX_QUEUE_SIZE
+OTEL_BLRP_MAX_EXPORT_BATCH_SIZE
+OTEL_BLRP_SCHEDULE_DELAY
+OTEL_BLRP_EXPORT_TIMEOUT
+
+OTEL_LOG_EXPORT_CONCURRENCY_LIMIT
+```
+
+Endpoint rules:
+
+```text
+OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
+    exact signal endpoint; do not append /v1/logs
+
+OTEL_EXPORTER_OTLP_ENDPOINT
+    generic endpoint; append /v1/logs
+```
+
+Moda deliberately does **not** export to the OpenTelemetry default localhost
+endpoint. Without an explicitly configured endpoint the logger remains
+stdout-only.
+
+### Direct Loki transport
+
+The direct Loki transport is a lightweight bridge: the normal `./logging` entry
+stores an emitter on `globalThis` and the Node bootstrap installs it. Records
+still pass through the same shared redaction/sanitization used for stdout and
+OpenTelemetry.
+
+The Loki environment contract is:
+
+```text
+LOKI_ENABLED
+
+LOKI_URL
+
+LOKI_USERNAME
+LOKI_PASSWORD
+
+LOKI_BATCHING
+LOKI_BATCH_INTERVAL_SECONDS
+LOKI_TIMEOUT_MS
+LOKI_CLEAR_ON_ERROR
+```
+
+Rules:
+
+```text
+no LOKI_URL
+    -> direct Loki disabled
+
+LOKI_ENABLED=false
+    -> direct Loki disabled
+
+LOKI_URL present
+    -> direct Loki enabled by default
+```
+
+Credentials are Node-bootstrap configuration only. Never log `LOKI_URL`
+credentials, `LOKI_USERNAME`, `LOKI_PASSWORD`, Basic Auth values or connection
+headers.
+
+Default transport behavior:
+
+```text
+batching=true
+bounded timeout (LOKI_TIMEOUT_MS)
+clearOnError=true
+replaceTimestamp=true
+gracefulShutdown=false
+```
+
+The owning process bootstrap is responsible for explicit flush/shutdown via the
+returned runtime (`forceFlush`/`shutdown`). If Loki is unavailable, log records
+may be dropped but application processing must continue. Connection errors are
+not recursively written back through the shared logger.
+
+### Loki labels and cardinality
+
+Moda explicitly configures exactly three canonical low-cardinality labels:
+
+```text
+service_namespace
+service_name
+environment
+```
+
+`winston-loki@6.1.7` additionally adds its own low-cardinality `level` label
+to every pushed stream, so a stream carries the three Moda labels plus
+`level`. This is transport-library behaviour and does not change Moda's label
+configuration.
+
+Dynamic/high-cardinality identifiers must **not** be promoted to Loki stream
+labels. The implementation explicitly retains
+`useWinstonMetaAsLabels: false`, so winston metadata is never promoted to
+labels. In particular, do not label by:
+
+```text
+traceId
+spanId
+requestId
+jobId
+deliveryId
+eventId
+recoveryId
+checkoutToken
+cartToken
+shop
+shopDomain
+customerId
+phone
+email
+```
+
+Safe high-cardinality operational identifiers may remain in the sanitized JSON
+log body where justified, but they must never become indexed stream labels.
+
+### Duplicate-delivery warning
+
+Do not route the OpenTelemetry Logs destination back into the same Loki
+instance while direct Loki is also enabled, unless duplicate storage is
+intentional:
+
+```text
+direct Loki ON
++
+OTel Logs ultimately routed to same Loki
+=
+duplicate log storage
+```
+
+The library does not guess backend topology; deployment configuration owns this
+choice.
+
+### Log-signal data safety and correlation
+
+- The OpenTelemetry log body is `JSON.stringify` of the same canonical
+  sanitized `LogRecord` written to stdout, so the same redaction boundary
+  applies to both destinations.
+- Queryable OTel attributes are limited to:
+
+  ```text
+  event.name
+  log.level
+  service.namespace
+  service.name
+  deployment.environment.name
+  ```
+
+- The shared logger never manufactures trace IDs. If an active
+  trace/span context exists, the OpenTelemetry Logs SDK associates the emitted
+  record with that active context automatically.
+- stdout and OpenTelemetry emission are isolated and best-effort: a failure in
+  either must not change application/business correctness.
+
+### Tracing/metrics ownership
+
+The shared logger does **not** create spans, counters or histograms.
+
+The shared observability runtime owns the generic Node SDK, tracing and metrics
+provider mechanics, exporters, sampling, HTTP/Undici instrumentation, generic
+helpers, and lifecycle. Service repositories own their span names, business
+attributes, application-specific metrics, and domain semantics, for example:
 
 ```text
 Shopify operation
     |
-    +--> shared structured logger
+    +--> shared structured logger (stdout + OTel Logs)
     |
-    +--> Shopify-specific OpenTelemetry
+    +--> Shopify-specific OpenTelemetry (spans/metrics)
 ```
 
 This separation is intentional. Fields that are useful in diagnostic logs can
@@ -344,6 +724,25 @@ The public package exposes independent entry points.
 
 @modainteract/moda-interact-shared/logging
     reusable structured logging API
+    (stdout + OTel Logs + direct Loki fan-out)
+
+@modainteract/moda-interact-shared/logging/node
+    Node-only logging bootstrap
+    (OpenTelemetry LoggerProvider, batching, OTLP HTTP exporter;
+     Winston + winston-loki direct transport)
+
+@modainteract/moda-interact-shared/observability
+    browser-safe generic observability helpers
+
+@modainteract/moda-interact-shared/observability/node
+    reusable Node OpenTelemetry SDK, tracing/metrics providers, exporters,
+    sampling, HTTP/Undici instrumentation, and lifecycle
+
+@modainteract/moda-interact-shared/observability/bullmq
+    Node-only BullMQ native telemetry adapter
+
+@modainteract/moda-interact-shared/observability/genai
+    Node/runtime GenAI conversation-turn, agent, and tool active-span helpers
 ```
 
 Consumers should prefer the narrowest appropriate subpath.
@@ -360,27 +759,31 @@ Examples of repository ownership:
 - normalizes/validates Shopify events;
 - publishes the appropriate queue jobs;
 - uses the shared structured logger for generic log mechanics;
-- owns Shopify-specific observability semantics and OpenTelemetry.
+- uses the shared observability runtime for generic OpenTelemetry mechanics;
+- owns Shopify-specific observability semantics and application metrics.
 
 **moda-interact-background**
 
 - parses shared cross-service contracts before acting;
 - runs BullMQ workers and recovery workflows;
 - uses the shared logger for generic structured logging;
-- owns worker/recovery OpenTelemetry.
+- uses the shared observability runtime for generic OpenTelemetry mechanics;
+- owns worker/recovery observability semantics and application metrics.
 
 **moda-interact-messaging**
 
 - owns Meta/WhatsApp ingress;
 - uses the shared logger for generic structured logging;
-- owns Meta/WhatsApp-specific observability and OpenTelemetry.
+- uses the shared observability runtime for generic OpenTelemetry mechanics;
+- owns Meta/WhatsApp-specific observability semantics and application metrics.
 
 **moda-interact-admin**
 
 - owns the internal admin application;
 - may use the same shared logger where server-side operational logging is
   required;
-- owns admin-specific application telemetry.
+- uses the shared observability runtime for generic OpenTelemetry mechanics;
+- owns admin-specific observability semantics and application metrics.
 
 **moda-interact-database**