@qlibs/apm 0.3.0 → 1.1.0

package/README.md

@@ -19,7 +19,8 @@ What gets traced automatically (no business-logic changes required):
 
 Metrics (RED dashboards — latency, error rate, throughput) are intentionally
 **not** exported separately: SigNoz and Elastic APM both derive these from
-trace data automatically. If you need custom business metrics later, add an
+trace data automatically. The OTel SDK therefore ships traces only
+(`OTEL_METRICS_EXPORTER` and `OTEL_LOGS_EXPORTER` default to `none`). If you need custom business metrics later, add an
 `@opentelemetry/sdk-metrics` + OTLP metrics exporter next to `tracing.ts` —
 the config/resource plumbing here already gives you everything needed to
 wire that in.
@@ -78,12 +79,82 @@ dependency of this package.
 | `OTEL_EXPORTER_OTLP_HEADERS` | _(empty)_ | Comma-separated `key=value` pairs, e.g. ingestion keys / auth tokens. |
 | `LOG_LEVEL` | `info` (`debug` when env is `local`) | Winston log level (`error`, `warn`, `info`, `http`, `verbose`, `debug`). At `info`, Nest `verbose`/`debug` logs are dropped — set `verbose` or `debug` to see them. HTTP access logs and Nest `fatal` (logged as `error` with `fatal: true`) are always visible at `info`. On the console, `error`/`warn` go to stderr and everything else to stdout. |
 | `APM_LOG_FORMAT` | `pretty` for `local`/`development`, `json` otherwise | `pretty` = colorized console; `json` = structured stdout. |
-| `APM_LOG_TRANSPORT` | `stdout` | Comma list of `stdout`, `elasticsearch`, `otlp`. Multiple can be active at once. |
+| `APM_LOG_TRANSPORT` | `stdout` | Comma list of `stdout`, `elasticsearch`, `otlp`. Multiple can be active at once. `none` = the APM writes no log lines of its own (no access log, no extra error line); `ApmLoggerService` then just forwards to Nest's default console logger, so console output appears once, in Nest's format. Traces are unaffected. |
 | `ELASTICSEARCH_NODE` / `_USERNAME` / `_PASSWORD` / `_INDEX` | _(empty)_ | Only used when `APM_LOG_TRANSPORT` includes `elasticsearch`. While ES is unreachable at most 1000 log lines are buffered (the rest are dropped) and each is retried 5 times. Shipping via stdout + Filebeat/Fluent Bit is still preferred in production. |
 | `APM_HTTP_IGNORE_PATHS` | `/health,/favicon.ico` | Comma list of path prefixes excluded from the HTTP access log and from tracing. |
 | `OTEL_NODE_ENABLED_INSTRUMENTATIONS` | `http,undici,express,nestjs-core,pg,ioredis,redis,amqplib` | Which auto-instrumentations to load (names without the `@opentelemetry/instrumentation-` prefix). Add e.g. `mongodb`, `kafkajs` if your app needs them. |
 | `OTEL_TRACES_SAMPLER` / `OTEL_TRACES_SAMPLER_ARG` | `parentbased_always_on` | Standard OTel sampling. For high-traffic services use `parentbased_traceidratio` with e.g. `0.1` (10%). |
 
+## Recommended setup
+
+### Production
+
+```env
+NODE_ENV=production
+APM_SERVICE_NAME=my-app-api
+APM_SERVICE_VERSION=1.4.2
+LOG_LEVEL=warn
+APM_LOG_TRANSPORT=none
+OTEL_EXPORTER_OTLP_ENDPOINT=http://signoz-otel-collector:4318
+# Optional, for high-traffic services (> ~50 req/s):
+# OTEL_TRACES_SAMPLER=parentbased_traceidratio
+# OTEL_TRACES_SAMPLER_ARG=0.2
+```
+
+| Destination | What you get |
+|---|---|
+| **pm2 / docker logs** | Only `ERROR`, `WARN` and `FATAL`, in Nest's own format (`[Nest] 123 - ... ERROR [OrderService] ...`), each line **once**. Errors/fatal go to stderr (`pm2 logs --err`), warnings to stdout. No per-request access log, no `log`/`verbose`/`debug`. Plus one startup line: `[apm] tracing enabled -> ...`. |
+| **SigNoz — Traces** | Every request (except `/health`, `/favicon.ico`) including its Postgres, Redis, RabbitMQ and outbound HTTP calls. Failed requests are marked as errors with message and stack trace. Latency / error-rate / throughput dashboards are derived from these traces. |
+| **SigNoz — Logs** | Nothing is sent. |
+| **SigNoz — Metrics** | Nothing is sent. |
+| **Deploy / restart** | Pending traces are flushed (max 5s), then Nest's graceful shutdown runs normally. |
+
+In short: the terminal is for warnings and errors only; per-request detail
+lives in SigNoz.
+
+Requirements in the app:
+- `import '@qlibs/apm/tracing'` as the first line of `main.ts`.
+- `recordException()` or `ApmExceptionFilter` in place, so errors are marked
+  on the trace.
+- If `main.ts` does **not** call `app.useLogger(app.get(ApmLoggerService))`,
+  `LOG_LEVEL` does not filter Nest's output — use
+  `NestFactory.create(AppModule, { logger: ['fatal', 'error', 'warn'] })`
+  instead.
+
+On the server, enable log rotation as a safety net (`pm2 install
+pm2-logrotate`, or `max-size` / `max-file` for Docker's `json-file` driver).
+
+### Development (shared dev / staging server)
+
+```env
+NODE_ENV=development
+APM_SERVICE_NAME=my-app-api
+LOG_LEVEL=debug
+APM_LOG_TRANSPORT=stdout
+OTEL_EXPORTER_OTLP_ENDPOINT=http://signoz-otel-collector:4318
+```
+
+| Destination | What you get |
+|---|---|
+| **pm2 / docker logs** | **All levels** (error … debug), colorized and human-readable, written by the APM — e.g. `10:21:05 info [HTTP] GET /api/orders 200 (trace=4bf92f...)`. Includes a **per-request access log** with duration and `trace_id`, so any log line can be looked up as a trace in SigNoz. Nest's own logs go through the same logger, so nothing is printed twice. |
+| **SigNoz — Traces** | Every request, kept apart from production by `deployment.environment=development` (filter by environment in SigNoz). |
+| **SigNoz — Logs / Metrics** | Nothing is sent. |
+
+**Developer laptop:** use `NODE_ENV=local` and drop `LOG_LEVEL` (defaults to
+`debug`) and `OTEL_EXPORTER_OTLP_ENDPOINT`. Logs stay colorized in the
+terminal, tracing is off, nothing is sent anywhere.
+
+### At a glance
+
+| | Production | Development |
+|---|---|---|
+| Who prints to the terminal | Nest (APM is silent) | APM |
+| Levels shown | warn, error, fatal | all |
+| Per-request access log | ❌ (use SigNoz) | ✅ |
+| `trace_id` on log lines | ❌ | ✅ |
+| Traces to SigNoz | ✅ | ✅ |
+| Logs / metrics to SigNoz | ❌ | ❌ |
+
 ## Example configurations
 
 `LOG_LEVEL` is a threshold: it keeps that level and everything more severe
@@ -104,7 +175,22 @@ Errors are still marked on the trace span by `recordException()` regardless of
 `LOG_LEVEL`. After changing env vars, restart the app (pm2:
 `pm2 restart <app> --update-env`).
 
-**1. Production, minimal — only warn + error in pm2/docker, traces to SigNoz**
+**1. Traces only — Nest's own console logs, no duplicate APM log lines**
+```env
+NODE_ENV=production
+APM_SERVICE_NAME=my-app-api
+LOG_LEVEL=warn
+APM_LOG_TRANSPORT=none
+OTEL_EXPORTER_OTLP_ENDPOINT=http://signoz-otel-collector:4318
+```
+pm2/docker show only Nest's normal `[Nest] ... ERROR [Context] ...` lines, each
+once. Errors still appear on the trace in SigNoz (status ERROR + exception
+event) via `recordException()`. `LOG_LEVEL` keeps working if `main.ts` calls
+`app.useLogger(app.get(ApmLoggerService))`; if it doesn't, filter Nest's output
+with `NestFactory.create(AppModule, { logger: ['fatal', 'error', 'warn'] })`
+instead. Trade-off: console lines carry no `trace_id`.
+
+**2. Production, minimal — only warn + error in pm2/docker, traces to SigNoz**
 ```env
 NODE_ENV=production
 APM_SERVICE_NAME=my-app-api
@@ -115,7 +201,7 @@ OTEL_EXPORTER_OTLP_ENDPOINT=http://signoz-otel-collector:4318
 Note: at `warn` the HTTP access log (`GET /api/... 200`, level `info`) is
 dropped too.
 
-**2. Production, standard — info + access log in pm2/docker, logs and traces to SigNoz**
+**3. Production, standard — info + access log in pm2/docker, logs and traces to SigNoz**
 ```env
 NODE_ENV=production
 APM_SERVICE_NAME=my-app-api
@@ -124,7 +210,7 @@ APM_LOG_TRANSPORT=stdout,otlp
 OTEL_EXPORTER_OTLP_ENDPOINT=http://signoz-otel-collector:4318
 ```
 
-**3. Production, high traffic — trace only 10% of requests**
+**4. Production, high traffic — trace only 10% of requests**
 ```env
 NODE_ENV=production
 APM_SERVICE_NAME=my-app-api
@@ -135,7 +221,7 @@ OTEL_TRACES_SAMPLER=parentbased_traceidratio
 OTEL_TRACES_SAMPLER_ARG=0.1
 ```
 
-**4. Staging, investigating a bug — everything**
+**5. Staging, investigating a bug — everything**
 ```env
 NODE_ENV=staging
 APM_SERVICE_NAME=my-app-api
@@ -144,7 +230,7 @@ APM_LOG_TRANSPORT=stdout,otlp
 OTEL_EXPORTER_OTLP_ENDPOINT=http://signoz-otel-collector:4318
 ```
 
-**5. Local development, no SigNoz**
+**6. Local development, no SigNoz**
 ```env
 NODE_ENV=local
 APM_SERVICE_NAME=my-app-api
@@ -152,7 +238,7 @@ APM_SERVICE_NAME=my-app-api
 # OTEL_EXPORTER_OTLP_ENDPOINT left empty -> tracing off
 ```
 
-**6. Logs to pm2/docker only, no tracing at all**
+**7. Logs to pm2/docker only, no tracing at all**
 ```env
 NODE_ENV=production
 APM_SERVICE_NAME=my-app-api
@@ -161,13 +247,13 @@ APM_LOG_TRANSPORT=stdout
 # OTEL_EXPORTER_OTLP_ENDPOINT not set -> nothing is sent to SigNoz
 ```
 
-**7. Collector listening on gRPC (port 4317)**
+**8. Collector listening on gRPC (port 4317)**
 ```env
 OTEL_EXPORTER_OTLP_ENDPOINT=http://signoz-otel-collector:4317
 OTEL_EXPORTER_OTLP_PROTOCOL=grpc
 ```
 
-**8. Logs to Elasticsearch / Kibana**
+**9. Logs to Elasticsearch / Kibana**
 ```env
 NODE_ENV=production
 APM_SERVICE_NAME=my-app-api

package/dist/apm.config.d.ts

@@ -1,5 +1,5 @@
 export type LogFormat = 'pretty' | 'json';
-export type LogTransportKind = 'stdout' | 'elasticsearch' | 'otlp';
+export type LogTransportKind = 'stdout' | 'elasticsearch' | 'otlp' | 'none';
 export type OtlpProtocol = 'http/protobuf' | 'grpc';
 export interface ApmConfig {
     enabled: boolean;
@@ -13,6 +13,7 @@ export interface ApmConfig {
         headers: Record<string, string>;
     };
     logging: {
+        enabled: boolean;
         level: string;
         format: LogFormat;
         transports: LogTransportKind[];

package/dist/apm.config.js

@@ -45,6 +45,7 @@ function getApmConfig() {
             headers: parseHeaders(env.OTEL_EXPORTER_OTLP_HEADERS),
         },
         logging: {
+            enabled: !transports.includes('none'),
             level: env.LOG_LEVEL || (nodeEnv === 'local' ? 'debug' : 'info'),
             format: env.APM_LOG_FORMAT || (isLocal ? 'pretty' : 'json'),
             transports: transports.length ? transports : ['stdout'],

package/dist/http/apm-logging.interceptor.js

@@ -14,7 +14,7 @@ const apm_config_1 = require("../apm.config");
 const winston_instance_1 = require("../logger/winston.instance");
 let ApmLoggingInterceptor = class ApmLoggingInterceptor {
     intercept(context, next) {
-        if (context.getType() !== 'http') {
+        if (context.getType() !== 'http' || !(0, apm_config_1.getApmConfig)().logging.enabled) {
             return next.handle();
         }
         const request = context.switchToHttp().getRequest();

package/dist/logger/apm-logger.service.d.ts

@@ -1,5 +1,6 @@
 import { LoggerService } from '@nestjs/common';
 export declare class ApmLoggerService implements LoggerService {
+    private readonly fallback;
     log(message: any, context?: string): void;
     error(message: any, trace?: string, context?: string): void;
     warn(message: any, context?: string): void;

package/dist/logger/apm-logger.service.js

@@ -8,24 +8,57 @@ var __decorate = (this && this.__decorate) || function (decorators, target, key,
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.ApmLoggerService = void 0;
 const common_1 = require("@nestjs/common");
+const apm_config_1 = require("../apm.config");
 const winston_instance_1 = require("./winston.instance");
+const NEST_LEVELS = {
+    error: ['fatal', 'error'],
+    warn: ['fatal', 'error', 'warn'],
+    info: ['fatal', 'error', 'warn', 'log'],
+    http: ['fatal', 'error', 'warn', 'log'],
+    verbose: ['fatal', 'error', 'warn', 'log', 'verbose'],
+    debug: ['fatal', 'error', 'warn', 'log', 'verbose', 'debug'],
+    silly: ['fatal', 'error', 'warn', 'log', 'verbose', 'debug'],
+};
+function buildFallbackLogger() {
+    const { logging } = (0, apm_config_1.getApmConfig)();
+    if (logging.enabled)
+        return undefined;
+    return new common_1.ConsoleLogger(undefined, {
+        logLevels: NEST_LEVELS[logging.level] || NEST_LEVELS.info,
+    });
+}
 let ApmLoggerService = class ApmLoggerService {
+    constructor() {
+        this.fallback = buildFallbackLogger();
+    }
     log(message, context) {
+        if (this.fallback)
+            return this.fallback.log(message, context);
         winston_instance_1.winstonLogger.info(this.stringify(message), { context });
     }
     error(message, trace, context) {
+        if (this.fallback)
+            return this.fallback.error(message, trace, context);
         winston_instance_1.winstonLogger.error(this.stringify(message), { trace, context });
     }
     warn(message, context) {
+        if (this.fallback)
+            return this.fallback.warn(message, context);
         winston_instance_1.winstonLogger.warn(this.stringify(message), { context });
     }
     debug(message, context) {
+        if (this.fallback)
+            return this.fallback.debug(message, context);
         winston_instance_1.winstonLogger.debug(this.stringify(message), { context });
     }
     fatal(message, context) {
+        if (this.fallback)
+            return this.fallback.fatal(message, context);
         winston_instance_1.winstonLogger.error(this.stringify(message), { context, fatal: true });
     }
     verbose(message, context) {
+        if (this.fallback)
+            return this.fallback.verbose(message, context);
         winston_instance_1.winstonLogger.verbose(this.stringify(message), { context });
     }
     stringify(message) {

package/dist/logger/winston.instance.js

@@ -122,11 +122,17 @@ function buildTransports() {
     return transports;
 }
 const config = (0, apm_config_1.getApmConfig)();
-exports.winstonLogger = winston.createLogger({
-    level: config.logging.level,
-    defaultMeta: { service: config.serviceName, environment: config.environment },
-    transports: buildTransports(),
-});
+exports.winstonLogger = config.logging.enabled
+    ? winston.createLogger({
+        level: config.logging.level,
+        defaultMeta: {
+            service: config.serviceName,
+            environment: config.environment,
+        },
+        transports: buildTransports(),
+    })
+    :
+        winston.createLogger({ silent: true });
 exports.winstonLogger.on('error', (err) => {
     console.error('[apm] log transport error:', err instanceof Error ? err.message : err);
 });

package/dist/tracing.js

@@ -1,4 +1,5 @@
 "use strict";
+var _a, _b;
 Object.defineProperty(exports, "__esModule", { value: true });
 const apm_config_1 = require("./apm.config");
 const config = (0, apm_config_1.getApmConfig)();
@@ -7,6 +8,8 @@ if (config.enabled && config.tracing.enabled && config.tracing.otlpEndpoint) {
         process.env.OTEL_NODE_ENABLED_INSTRUMENTATIONS =
             'http,undici,express,nestjs-core,pg,ioredis,redis,amqplib';
     }
+    (_a = process.env).OTEL_METRICS_EXPORTER || (_a.OTEL_METRICS_EXPORTER = 'none');
+    (_b = process.env).OTEL_LOGS_EXPORTER || (_b.OTEL_LOGS_EXPORTER = 'none');
     const { NodeSDK } = require('@opentelemetry/sdk-node');
     const { getNodeAutoInstrumentations, } = require('@opentelemetry/auto-instrumentations-node');
     const { resourceFromAttributes } = require('@opentelemetry/resources');
@@ -32,12 +35,16 @@ if (config.enabled && config.tracing.enabled && config.tracing.otlpEndpoint) {
     sdk.start();
     const SHUTDOWN_TIMEOUT_MS = 5000;
     const onSignal = (signal) => {
+        let timer;
         Promise.race([
             sdk.shutdown(),
-            new Promise((resolve) => setTimeout(resolve, SHUTDOWN_TIMEOUT_MS).unref()),
+            new Promise((resolve) => {
+                timer = setTimeout(resolve, SHUTDOWN_TIMEOUT_MS);
+            }),
         ])
             .catch((err) => console.error('[apm] error shutting down OTel SDK', err))
             .finally(() => {
+            clearTimeout(timer);
             if (process.listenerCount(signal) === 0) {
                 process.kill(process.pid, signal);
             }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@qlibs/apm",
-  "version": "0.3.0",
+  "version": "1.1.0",
   "description": "Vendor-neutral APM for NestJS: OpenTelemetry tracing (OTLP) + structured logging + error monitoring. Works with SigNoz, Elastic/Kibana, Jaeger, or any OTel collector.",
   "author": "QBIT Developer",
   "license": "MIT",