logs-gateway 3.1.2 → 3.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,1263 +1,1255 @@
1
- # logs-gateway
2
-
3
- A standardized logging gateway for Node.js applications. Flexible multi-transport logging with **console**, **file**, and **unified-logger** outputs; **ENV-first** configuration; **PII/credentials sanitization**; **dual correlation trails** (operation & thread); **OpenTelemetry** context; **YAML/JSON/text** formats; **per-run "Shadow Logging"** for test/debug capture; **scoping** with text filters; **story output** powered by `scopeRecord`; and **troubleshooting integration** with `nx-troubleshooting`.
4
-
5
- ## 🚀 Env-Ready Component (ERC 2.0)
6
-
7
- This component supports **zero-config initialization** via environment variables and is compliant with the [Env-Ready Component Standard (ERC 2.0)](https://github.com/xeonox/erc-standard).
8
-
9
- ### ERC 2.0 Compliance
10
-
11
- - ✅ **Auto-Discovery**: Zero-config initialization from environment variables
12
- - ✅ **Complete Documentation**: All environment variables documented (including dependencies)
13
- - ✅ **Type Safety**: Automatic type coercion and validation
14
- - ✅ **Manifest**: Auto-generated `erc-manifest.json` with all requirements
15
- - ✅ **Example File**: Auto-generated `.env.example` with all transitive requirements
16
- - ✅ **Dependency Tracking**: Documents both ERC and non-ERC dependencies
17
-
18
- ### Quick Start (Zero-Config Mode)
19
-
20
- ```bash
21
- # 1. Install the package
22
- npm install logs-gateway
23
-
24
- # 2. Set environment variables (replace MY_APP with your prefix)
25
- export MY_APP_LOG_TO_CONSOLE=true
26
- export MY_APP_LOG_LEVEL=info
27
- export MY_APP_LOG_FORMAT=json
28
-
29
- # 3. Use with zero config!
30
- import { createLogger } from 'logs-gateway';
31
- const logger = createLogger(
32
- { packageName: 'MY_APP', envPrefix: 'MY_APP' }
33
- ); // Auto-discovers from process.env
34
- ```
35
-
36
- ### Advanced Mode (Programmatic Configuration)
37
-
38
- ```typescript
39
- import { createLogger } from 'logs-gateway';
40
-
41
- const logger = createLogger(
42
- { packageName: 'MY_APP', envPrefix: 'MY_APP' },
43
- {
44
- logToFile: true,
45
- logFilePath: '/var/log/myapp.log',
46
- logLevel: 'info'
47
- }
48
- );
49
- ```
50
-
51
- ### Environment Variables
52
-
53
- **Note**: This component uses **dynamic environment variable prefixes** based on `packageConfig.envPrefix`. Replace `{PREFIX}` with your actual prefix (e.g., `MY_APP`, `API_SERVICE`).
54
-
55
- See `.env.example` for the complete list of required and optional variables with descriptions. Generate it by running:
56
-
57
- ```bash
58
- npm run generate-erc
59
- ```
60
-
61
- ### Dependencies
62
-
63
- - ✅ **nx-config2** (ERC 2.0) - Configuration engine
64
- - â„šī¸ **@x-developer/unified-logger** (non-ERC) - Requirements manually documented
65
- - â„šī¸ **nx-troubleshooting** (non-ERC, optional) - Requirements manually documented
66
-
67
- ---
68
-
69
- ## Features
70
-
71
- * ✅ Console output (default)
72
- * ✅ File output (optional)
73
- * ✅ Unified-logger output (Papertrail/UDP/Console via `@x-developer/unified-logger`)
74
- * ✅ Dual/triple output (console + file + unified-logger)
75
- * ✅ Environment variable configuration
76
- * ✅ Custom logger injection (Winston, Pino, etc.)
77
- * ✅ Package-specific prefixes
78
- * ✅ **Five log levels**: `verbose`, `debug`, `info`, `warn`, `error`
79
- * ✅ **Multiple formats**: `text`, `json`, `yaml` (console & file; unified stays JSON)
80
- * ✅ **PII/Credentials sanitization** (auto-detect & mask, opt-in)
81
- * ✅ **Dual trails**: operation (Depth) & thread (Causal), plus `runId`, `jobId`, `correlationId`, `sessionId`
82
- * ✅ **OpenTelemetry context** (`traceId`, `spanId`) when available
83
- * ✅ Routing metadata (control outputs per entry)
84
- * ✅ Recursion safety (prevent circular/unified feedback)
85
- * ✅ TypeScript support
86
- * ✅ **Shadow Logging (per-run capture)** with TTL & forced-verbose, great for tests
87
- * ✅ **Scoping** – Derive focused subsets of logs with text filters and correlation keys
88
- * ✅ **Story Output** – Human-readable narrative format built automatically from log entries
89
- * ✅ **Troubleshooting Integration** – Wire in `nx-troubleshooting` for intelligent error-to-solution matching
90
-
91
- ---
92
-
93
- ## Installation
94
-
95
- ```bash
96
- npm install logs-gateway
97
- ```
98
-
99
- For troubleshooting integration:
100
-
101
- ```bash
102
- npm install logs-gateway nx-troubleshooting
103
- ```
104
-
105
- ---
106
-
107
- ## Quick Start
108
-
109
- ```ts
110
- import { createLogger } from 'logs-gateway';
111
-
112
- const logger = createLogger(
113
- { packageName: 'MY_APP', envPrefix: 'MY_APP' },
114
- {
115
- logToFile: true,
116
- logFilePath: '/var/log/myapp.log',
117
- logLevel: 'info', // verbose|debug|info|warn|error
118
- logFormat: 'json', // text|json|yaml (yaml: console/file only)
119
- enableUnifiedLogger: true,
120
- unifiedLogger: {
121
- transports: { papertrail: true },
122
- service: 'my-app',
123
- env: 'production'
124
- },
125
- // Optional: per-run Shadow Logging defaults (can also be enabled at runtime)
126
- shadow: { enabled: false, ttlMs: 86_400_000 }, // 1 day
127
- // Optional: Scoping & Troubleshooting
128
- scoping: {
129
- enabled: true,
130
- errorScoping: { enabled: true, windowMsBefore: 60_000, windowMsAfter: 30_000 },
131
- buffer: { maxEntries: 5000, preferShadow: true }
132
- },
133
- troubleshooting: {
134
- enabled: true,
135
- narrativesPath: './metadata/troubleshooting.json',
136
- output: { formats: ['markdown'], emitAsLogEntry: true }
137
- }
138
- }
139
- );
140
-
141
- // Standard usage
142
- logger.verbose('Very detailed info', { step: 'init' });
143
- logger.debug('Debug info', { data: 'x' });
144
- logger.info('Application initialized', { version: '1.0.0' });
145
- logger.warn('Deprecated feature used');
146
- logger.error('Error occurred', { error: new Error('boom') });
147
- ```
148
-
149
- ---
150
-
151
- ## Automatic Application Identification
152
-
153
- **logs-gateway** automatically detects and includes your application's package name and version in every log entry. This is done by reading the consuming project's `package.json` file (the project using logs-gateway, not logs-gateway itself).
154
-
155
- ### How It Works
156
-
157
- When you create a logger instance, logs-gateway automatically:
158
- 1. Searches up the directory tree from `process.cwd()` to find the nearest `package.json`
159
- 2. Extracts the `name` and `version` fields from that file
160
- 3. Includes them in all log entries as `appName` and `appVersion`
161
-
162
- This happens **automatically** - no configuration needed! The detection is cached after the first call for performance.
163
-
164
- ### Example
165
-
166
- If your project's `package.json` contains:
167
- ```json
168
- {
169
- "name": "my-awesome-app",
170
- "version": "2.1.0"
171
- }
172
- ```
173
-
174
- Then all log entries will automatically include:
175
- ```json
176
- {
177
- "timestamp": "2025-01-15T10:30:45.123Z",
178
- "package": "MY_APP",
179
- "level": "INFO",
180
- "message": "Application initialized",
181
- "appName": "my-awesome-app",
182
- "appVersion": "2.1.0",
183
- "data": { ... }
184
- }
185
- ```
186
-
187
- ### Benefits
188
-
189
- - **Traceability**: Know exactly which application version generated each log entry
190
- - **Debugging**: Filter logs by application version in centralized logging systems
191
- - **Deployment Tracking**: Identify which deployments are running in production
192
- - **Zero Configuration**: Works automatically without any API changes
193
-
194
- ### Important Notes
195
-
196
- - The detection searches from `process.cwd()` (the working directory where your app runs)
197
- - It stops at the first `package.json` that is not in `node_modules`
198
- - If no `package.json` is found, `appName` and `appVersion` are simply omitted (no error)
199
- - The result is cached after first detection for performance
200
-
201
- ---
202
-
203
- ## Overview: Scoping, Story Output & Troubleshooting
204
-
205
- This extension adds four major capabilities:
206
-
207
- 1. **Runtime Filtering (logger-debug.json)** – Filter logs at the source before they're written. Place `logger-debug.json` at your project root to filter logs by identity or application name. This reduces noise at runtime and complements post-processing scoping.
208
-
209
- 2. **Scoping** – Given a verbose log stream, derive a **focused subset** of logs relevant to a problem or question. Scopes can be:
210
- * Error-centric (anchor on a specific `error` entry)
211
- * Run/Correlation-centric (anchor on `runId`, `correlationId`, etc.)
212
- * Text-based (filter by message/data content)
213
- * Narrative-based (optional, driven by "scoping narratives")
214
-
215
- Scoping **always uses verbose logs** if available, regardless of the current log level.
216
-
217
- 3. **Story vs Full Data Output** – A scope can be returned as:
218
- * **Full data**: structured `ScopedLogView` with all entries
219
- * **Story**: human-readable narrative built automatically from entries using a generic `scopeRecord` helper
220
- * Or **both**
221
-
222
- 4. **Troubleshooting Integration** – Wire in `nx-troubleshooting` so that errors/scopes:
223
- * Are matched to **troubleshooting narratives**, and
224
- * Produce troubleshooting artifacts (Markdown/JSON/text) as **another log output channel** (`troubleshooting`)
225
-
226
- The same `scopeRecord` helper is also exported as a **generic tool** for scoping arbitrary JSON records (not just logs).
227
-
228
- ---
229
-
230
- ## Configuration
231
-
232
- ### Via Constructor
233
-
234
- ```ts
235
- const logger = createLogger(
236
- { packageName: 'MY_PACKAGE', envPrefix: 'MY_PACKAGE', debugNamespace: 'my-pkg' },
237
- {
238
- // Outputs
239
- logToConsole: true, // default: true
240
- logToFile: false, // default: false
241
- logFilePath: '/var/log/app.log', // required if logToFile
242
- enableUnifiedLogger: false, // default: false
243
- unifiedLogger: { /* ... */ },
244
-
245
- // Behavior
246
- logLevel: 'info', // verbose|debug|info|warn|error (default: info)
247
- logFormat: 'text', // text|json|yaml (default: text)
248
- defaultSource: 'application', // fallback source tag
249
-
250
- // Sanitization (opt-in)
251
- sanitization: {
252
- enabled: false, // default: false
253
- maskWith: '[REDACTED]',
254
- keysDenylist: ['authorization','password','secret','api_key'],
255
- fieldsHashInsteadOfMask: ['userId'],
256
- detectJWTs: true
257
- // + other detectors & guardrails
258
- },
259
-
260
- // Trails & tracing (optional; safe no-ops if not used)
261
- trails: {
262
- enableDepthTrail: true, // operation trail
263
- enableThreadTrail: true, // causal/thread trail
264
- injectHeaders: true, // for HTTP/queues adapters
265
- extractHeaders: true
266
- },
267
- tracing: { enableOtelContext: true },
268
-
269
- // Shadow Logging (per-run capture; can also be toggled at runtime)
270
- shadow: {
271
- enabled: false, // default: false
272
- format: 'json', // json|yaml (default: json)
273
- directory: './logs/shadow', // default path
274
- ttlMs: 86_400_000, // 1 day
275
- forceVerbose: true, // capture all levels for the runId
276
- respectRoutingBlocks: true, // honor _routing.blockOutputs: ['shadow'|'file']
277
- includeRaw: false, // also write unsanitized (dangerous; tests only)
278
- rollingBuffer: { maxEntries: 0, maxAgeMs: 0 } // optional retro-capture
279
- },
280
-
281
- // Scoping (opt-in)
282
- scoping: {
283
- enabled: false, // default: false
284
- errorScoping: {
285
- enabled: true, // default: true if scoping.enabled
286
- levels: ['error'], // default: ['error']
287
- windowMsBefore: 30_000, // default: 30_000
288
- windowMsAfter: 30_000 // default: 30_000
289
- },
290
- runScoping: {
291
- enabled: true, // default: true if scoping.enabled
292
- defaultWindowMsBeforeFirstError: 60_000,
293
- defaultWindowMsAfterLastEntry: 30_000
294
- },
295
- narrativeScoping: {
296
- enabled: false, // default: false
297
- narrativesPath: './metadata/log-scopes.json',
298
- envPrefix: 'NX_SCOPE'
299
- },
300
- buffer: {
301
- maxEntries: 0, // default: 0 => disabled if Shadow is enough
302
- maxAgeMs: 0,
303
- includeLevels: ['verbose','debug','info','warn','error'],
304
- preferShadow: true // default: true
305
- }
306
- },
307
-
308
- // Troubleshooting (opt-in, requires nx-troubleshooting)
309
- troubleshooting: {
310
- enabled: false, // default: false
311
- narrativesPath: './metadata/troubleshooting.json',
312
- envPrefix: 'NX_TROUBLE',
313
- loggingConfig: { /* optional */ },
314
- engine: undefined, // optional DI
315
- output: {
316
- formats: ['markdown'], // default: ['markdown']
317
- writeToFileDir: undefined,
318
- attachToShadow: false,
319
- emitAsLogEntry: false,
320
- callback: undefined
321
- }
322
- }
323
- }
324
- );
325
- ```
326
-
327
- ### Via Environment Variables
328
-
329
- **Note**: Environment variable names use a **dynamic prefix** based on `packageConfig.envPrefix`. Replace `{PREFIX}` in the examples below with your actual prefix (e.g., `MY_APP`, `API_SERVICE`).
330
-
331
- ```bash
332
- # Console & file
333
- {PREFIX}_LOG_TO_CONSOLE=true|false
334
- {PREFIX}_LOG_TO_FILE=true|false
335
- {PREFIX}_LOG_FILE=/path/to/log
336
-
337
- # Unified-logger
338
- {PREFIX}_LOG_TO_UNIFIED=true|false
339
-
340
- # Level & format
341
- {PREFIX}_LOG_LEVEL=verbose|debug|info|warn|error
342
- {PREFIX}_LOG_FORMAT=text|json|yaml|table
343
-
344
- # Console output options
345
- {PREFIX}_SHOW_FULL_TIMESTAMP=true|false # Show full ISO timestamp (default: false)
346
- {PREFIX}_CONSOLE_PACKAGES_SHOW=package1,package2 # Only show these packages in console (default: show all)
347
- {PREFIX}_CONSOLE_PACKAGES_HIDE=package1,package2 # Hide these packages in console (default: show all)
348
-
349
- # Debug namespace → enables verbose+debug for that namespace
350
- DEBUG=my-pkg,other-*
351
-
352
- # Sanitization (subset shown)
353
- {PREFIX}_SANITIZE_ENABLED=true|false
354
- {PREFIX}_SANITIZE_KEYS_DENYLIST=authorization,token,secret,api_key,password
355
-
356
- # Trails/tracing
357
- {PREFIX}_TRACE_OTEL=true|false
358
- {PREFIX}_TRAILS_DEPTH=true|false
359
- {PREFIX}_TRAILS_THREAD=true|false
360
- {PREFIX}_TRAILS_INJECT=true|false
361
- {PREFIX}_TRAILS_EXTRACT=true|false
362
-
363
- # Shadow Logging
364
- {PREFIX}_SHADOW_ENABLED=true|false
365
- {PREFIX}_SHADOW_FORMAT=json|yaml
366
- {PREFIX}_SHADOW_DIR=/var/log/myapp/shadow
367
- {PREFIX}_SHADOW_TTL_MS=86400000
368
- {PREFIX}_SHADOW_FORCE_VERBOSE=true|false
369
- {PREFIX}_SHADOW_RESPECT_ROUTING=true|false
370
- {PREFIX}_SHADOW_INCLUDE_RAW=false
371
- {PREFIX}_SHADOW_BUFFER_ENTRIES=0
372
- {PREFIX}_SHADOW_BUFFER_AGE_MS=0
373
-
374
- # Scoping
375
- {PREFIX}_SCOPING_ENABLED=true|false
376
- {PREFIX}_SCOPING_ERROR_ENABLED=true|false
377
- {PREFIX}_SCOPING_ERROR_WINDOW_MS_BEFORE=30000
378
- {PREFIX}_SCOPING_ERROR_WINDOW_MS_AFTER=30000
379
- {PREFIX}_SCOPING_BUFFER_ENTRIES=5000
380
- {PREFIX}_SCOPING_BUFFER_AGE_MS=300000
381
- {PREFIX}_SCOPING_BUFFER_PREFER_SHADOW=true|false
382
-
383
- # Troubleshooting
384
- {PREFIX}_TROUBLESHOOTING_ENABLED=true|false
385
- {PREFIX}_TROUBLESHOOTING_NARRATIVES_PATH=./metadata/troubleshooting.json
386
- {PREFIX}_TROUBLESHOOTING_OUTPUT_FORMATS=markdown,json
387
- {PREFIX}_TROUBLESHOOTING_OUTPUT_EMIT_AS_LOG_ENTRY=true|false
388
-
389
- # Unified-logger dependencies (non-ERC, manually documented)
390
- # Required when unified-logger papertrail transport is enabled:
391
- PAPERTRAIL_HOST=logs.papertrailapp.com
392
- PAPERTRAIL_PORT=12345
393
-
394
- # Required when unified-logger udpRelay transport is enabled:
395
- UDP_RELAY_HOST=127.0.0.1
396
- UDP_RELAY_PORT=514
397
- ```
398
-
399
- > **Default min level:** `info`.
400
- > **`DEBUG=`**: enables **both** `verbose` and `debug` for matching namespaces.
401
- >
402
- > **ERC 2.0 Note**: Generate a complete `.env.example` file with all variables by running `npm run generate-erc`.
403
-
404
- ---
405
-
406
- ## Core Types
407
-
408
- ### LogEntry
409
-
410
- Internal normalized log shape:
411
-
412
- ```ts
413
- export interface LogEntry {
414
- timestamp: string; // ISO-8601
415
- level: 'verbose' | 'debug' | 'info' | 'warn' | 'error';
416
- package: string;
417
- message: string;
418
- source?: string; // e.g. 'application', 'auth-service'
419
- data?: Record<string, any>; // user metadata, error, ids, etc.
420
-
421
- // Automatic application identification (from consuming project's package.json)
422
- appName?: string; // Auto-detected from package.json "name" field
423
- appVersion?: string; // Auto-detected from package.json "version" field
424
-
425
- // Correlation / trails / tracing
426
- runId?: string;
427
- jobId?: string;
428
- correlationId?: string;
429
- sessionId?: string;
430
- operationId?: string;
431
- parentOperationId?: string;
432
- operationName?: string;
433
- threadId?: string;
434
- traceId?: string;
435
- spanId?: string;
436
-
437
- // Routing
438
- _routing?: RoutingMeta;
439
-
440
- // Optional scope metadata (for future/advanced use)
441
- scope?: ScopedMetadata;
442
- }
443
- ```
444
-
445
- ### Runtime Filtering (logger-debug.json)
446
-
447
- Place a `logger-debug.json` file at your project root to filter logs at runtime:
448
-
449
- ```json
450
- {
451
- "scoping": {
452
- "status": "enabled",
453
- "filterIdentities": ["src/auth.ts:login", "src/payment.ts:processPayment"],
454
- "filteredApplications": ["my-app", "other-app"],
455
- "logFrames": [
456
- {
457
- "action": "include",
458
- "exactMatch": false,
459
- "searchLog": false,
460
- "startIdentities": ["src/api.ts:handleRequest"],
461
- "endIdentities": ["src/api.ts:handleRequestEnd"]
462
- }
463
- ]
464
- }
465
- }
466
- ```
467
-
468
- **Behavior:**
469
- - When `status: "enabled"`, logs are filtered before being written to any output (console, file, unified-logger, shadow)
470
- - A log is included if its `identity` matches any entry in `filterIdentities` **OR** its `appName` matches any entry in `filteredApplications` **OR** it falls within an active "logFrames" frame (OR logic)
471
- - If all filters are empty or missing, all logs are included (no filtering)
472
- - File is auto-discovered from `process.cwd()` (searches up directory tree like package.json)
473
- - Configuration is loaded once at startup (not reloaded dynamically)
474
- - If file is not found or invalid, all logs are shown (graceful fallback)
475
-
476
- **Examples:**
477
-
478
- ```json
479
- // Filter by identity only - only show logs from specific code locations
480
- {
481
- "scoping": {
482
- "status": "enabled",
483
- "filterIdentities": ["src/auth.ts:login", "src/payment.ts:processPayment"]
484
- }
485
- }
486
- ```
487
-
488
- ```json
489
- // Filter by application only - only show logs from specific apps
490
- {
491
- "scoping": {
492
- "status": "enabled",
493
- "filteredApplications": ["my-app"]
494
- }
495
- }
496
- ```
497
-
498
- ```json
499
- // Filter by both (OR logic - matches if identity OR appName matches)
500
- {
501
- "scoping": {
502
- "status": "enabled",
503
- "filterIdentities": ["src/auth.ts:login"],
504
- "filteredApplications": ["my-app"]
505
- }
506
- }
507
- ```
508
-
509
- ```json
510
- // Logs Frames - stateful range-based filtering and log level control
511
- {
512
- "scoping": {
513
- "status": "enabled",
514
- "logFrames": [
515
- {
516
- "action": "include",
517
- "exactMatch": false,
518
- "searchLog": false,
519
- "startIdentities": ["src/api.ts:handleRequest"],
520
- "endIdentities": ["src/api.ts:handleRequestEnd"]
521
- },
522
- {
523
- "action": "exclude",
524
- "exactMatch": true,
525
- "searchLog": false,
526
- "startIdentities": ["src/db.ts:query"],
527
- "endIdentities": ["src/db.ts:queryEnd"]
528
- },
529
- {
530
- "action": "setLevel",
531
- "level": "debug",
532
- "startIdentities": ["src/debug.ts:start"],
533
- "endIdentities": ["src/debug.ts:end"]
534
- },
535
- {
536
- "action": "include",
537
- "exactMatch": false,
538
- "searchLog": true,
539
- "startIdentities": [],
540
- "endIdentities": ["src/init.ts:complete"]
541
- },
542
- {
543
- "action": "include",
544
- "exactMatch": true,
545
- "searchLog": true,
546
- "startIdentities": ["Payment started"],
547
- "endIdentities": ["Payment completed"]
548
- }
549
- ]
550
- }
551
- }
552
- ```
553
-
554
- **Logs Frames:**
555
- - **Stateful filtering and log level control**: Tracks active frames across log calls
556
- - **`action`**:
557
- - `"include"` to show logs within frame
558
- - `"exclude"` to hide logs within frame
559
- - `"setLevel"` to change the effective log level within the frame (requires `level` property)
560
- - **`level`**: Log level to use when `action` is `"setLevel"`. Must be one of: `"verbose"`, `"debug"`, `"info"`, `"warn"`, `"error"`
561
- - **`exactMatch`**:
562
- - `true`: Exact string match (case sensitive)
563
- - `false`: Partial substring match (case insensitive, default)
564
- - **`searchLog`**:
565
- - `true`: Search entire log (message + identity + all meta fields stringified)
566
- - `false`: Search only identity field (default)
567
- - **`startIdentities`**: Array of patterns that activate the frame. Empty array means frame starts from the beginning
568
- - **`endIdentities`**: Array of patterns that deactivate the frame. Empty array means frame never ends
569
- - **Multiple frames**: Can overlap and are tracked independently. Uses OR logic (if ANY include rule is active, include; if ANY exclude rule is active, exclude)
570
- - **Log level behavior**:
571
- - When multiple `setLevel` frames are active, the most permissive (lowest priority) level is used
572
- - When a frame with `setLevel` action becomes active, the effective log level changes to the frame's level
573
- - When the frame ends, the log level returns to the configured level
574
- - **Frame behavior**:
575
- - When a log matches a start identity, the frame becomes active
576
- - When a log matches an end identity, the frame becomes inactive
577
- - If a log matches both start and end identities, the frame state toggles
578
- - Frames with empty `startIdentities` are active from the first log
579
- - Frames with empty `endIdentities` never close once activated
580
-
581
- **Integration with Existing Scoping:**
582
- - `logger-debug.json` → Runtime filtering (reduces noise at source, filters before writing)
583
- - `scopeLogs()` → Post-processing scoping (analyzes already-written logs)
584
- - Both can be used together for maximum control
585
-
586
- ### ScopeCriteria
587
-
588
- Scoping criteria defines *which logs* belong to a scope. It supports:
589
-
590
- * Correlation keys
591
- * Time windows
592
- * Levels
593
- * Sources
594
- * **Text matching on message and data**
595
-
596
- ```ts
597
- export interface ScopeCriteria {
598
- // Correlation / keys
599
- runId?: string;
600
- correlationId?: string;
601
- sessionId?: string;
602
- threadId?: string;
603
- traceId?: string;
604
-
605
- // Time window bounds
606
- fromTimestamp?: string; // ISO-8601
607
- toTimestamp?: string; // ISO-8601
608
-
609
- // Window relative to an anchor (error or first/last entry)
610
- windowMsBefore?: number; // relative to anchor timestamp
611
- windowMsAfter?: number;
612
-
613
- // Levels
614
- levelAtLeast?: 'verbose' | 'debug' | 'info' | 'warn' | 'error';
615
- includeLevels?: ('verbose'|'debug'|'info'|'warn'|'error')[];
616
-
617
- // Source filters
618
- sources?: string[]; // e.g. ['api-gateway','payments-service']
619
-
620
- // TEXT FILTERS
621
- /**
622
- * Match logs whose message OR data (stringified) contains ANY of the given strings (case-insensitive).
623
- * - string: single substring
624
- * - string[]: log must contain at least one of them
625
- */
626
- textIncludesAny?: string | string[];
627
-
628
- /**
629
- * Match logs whose message OR data (stringified) contains ALL of the given substrings (case-insensitive).
630
- */
631
- textIncludesAll?: string[];
632
-
633
- /**
634
- * Optional RegExp filter over the combined text (message + JSON-stringified data).
635
- * If provided as string, it is treated as a new RegExp(text, 'i').
636
- */
637
- textMatches?: RegExp | string;
638
-
639
- // Scope metadata filters (if used)
640
- scopeTags?: string[]; // must include all provided tags
641
-
642
- // Custom predicate for in-process advanced filtering
643
- predicate?: (entry: LogEntry) => boolean;
644
- }
645
- ```
646
-
647
- **Text filtering behavior:**
648
-
649
- * Combine `entry.message` and a JSON string of `entry.data` into one string (e.g. `"Payment failed {...}"`).
650
- * Apply:
651
- * `textIncludesAny` – inclusive OR.
652
- * `textIncludesAll` – inclusive AND.
653
- * `textMatches` – regex test.
654
- * All text filtering is **case-insensitive** by default.
655
- * Text filters are **ANDed** with other criteria (correlation, time, etc.).
656
-
657
- ### ScopedLogView (full data)
658
-
659
- ```ts
660
- export interface ScopedLogView {
661
- id: string; // e.g. 'scope:runId:checkout-42'
662
- criteria: ScopeCriteria;
663
- entries: LogEntry[]; // sorted by timestamp ascending
664
- summary: {
665
- firstTimestamp?: string;
666
- lastTimestamp?: string;
667
- totalCount: number;
668
- errorCount: number;
669
- warnCount: number;
670
- infoCount: number;
671
- debugCount: number;
672
- verboseCount: number;
673
- uniqueSources: string[];
674
- };
675
- }
676
- ```
677
-
678
- ### scopeRecord – Generic Tool
679
-
680
- The `scopeRecord` helper is **generic** (not log-specific): given any JSON record, it produces:
681
-
682
- * A human-readable text description ("story" of the record).
683
- * A structured description (fields, truncation info).
684
-
685
- This function is:
686
- * Used internally to build **scope stories** from `LogEntry`s / aggregated records.
687
- * Exported publicly so other code can reuse it for non-log use cases.
688
-
689
- ```ts
690
- import { scopeRecord } from 'logs-gateway';
691
-
692
- // Example usage
693
- const record = {
694
- userId: '123',
695
- action: 'payment',
696
- amount: 100.50,
697
- timestamp: '2025-01-01T10:00:00Z'
698
- };
699
-
700
- const result = scopeRecord(record, {
701
- label: 'Payment Event',
702
- maxFieldStringLength: 200,
703
- excludeKeys: ['password', 'token']
704
- });
705
-
706
- console.log(result.text);
707
- // Output: "Payment Event\n User ID: 123\n Action: payment\n Amount: 100.5\n Timestamp: 2025-01-01T10:00:00Z"
708
-
709
- console.log(result.structured);
710
- // Output: { label: 'Payment Event', fields: [...], ... }
711
- ```
712
-
713
- **Types:**
714
-
715
- ```ts
716
- export interface AutoScopeRecordOptions {
717
- label?: string;
718
- formatting?: ScopingFormattingOptions;
719
- maxFields?: number;
720
- maxFieldStringLength?: number;
721
- includeKeys?: string[];
722
- excludeKeys?: string[];
723
- skipNullish?: boolean;
724
- header?: string;
725
- footer?: string;
726
- }
727
-
728
- export interface ScopeRecordResult {
729
- text: string; // Human-readable text output
730
- structured: StructuredScopedPayload; // Structured representation
731
- }
732
- ```
733
-
734
- ### Scope Story Output
735
-
736
- To let a scope answer with full data or story format:
737
-
738
- ```ts
739
- export type ScopeOutputMode = 'raw' | 'story' | 'both';
740
-
741
- export interface ScopeStoryOptions {
742
- recordOptions?: AutoScopeRecordOptions;
743
- maxEntries?: number;
744
- includeEntryHeader?: boolean;
745
- }
746
-
747
- export interface ScopeLogsResult {
748
- view: ScopedLogView; // always present
749
- story?: ScopedLogStory; // present if mode = 'story' or 'both'
750
- }
751
- ```
752
-
753
- ---
754
-
755
- ## API Reference
756
-
757
- ### `createLogger(packageConfig, userConfig?) → LogsGateway`
758
-
759
- ### `LogsGateway` Methods
760
-
761
- #### Standard Logging
762
-
763
- * `verbose(message, data?)`
764
- * `debug(message, data?)`
765
- * `info(message, data?)`
766
- * `warn(message, data?)`
767
- * `error(message, data?)`
768
- * `isLevelEnabled(level)` – threshold check (namespace DEBUG forces verbose+debug)
769
- * `getConfig()` – effective resolved config
770
-
771
- #### Scoping
772
-
773
- * `scopeLogs(criteria, options?)` – Scope logs by criteria, return full data and/or story
774
-
775
- ```ts
776
- const result = await logger.scopeLogs({
777
- runId: 'checkout-42',
778
- textIncludesAny: 'timeout',
779
- levelAtLeast: 'debug'
780
- }, {
781
- mode: 'both',
782
- storyOptions: {
783
- maxEntries: 50,
784
- includeEntryHeader: true,
785
- recordOptions: {
786
- label: 'Log Entry',
787
- maxFieldStringLength: 200
788
- }
789
- }
790
- });
791
-
792
- console.log(result.view.summary);
793
- console.log(result.story?.text);
794
- ```
795
-
796
- #### Troubleshooting
797
-
798
- * `troubleshootError(error, context?, options?)` – Error-centric troubleshooting
799
-
800
- ```ts
801
- const { scope, reports } = await logger.troubleshootError(
802
- new Error('Missing connections configuration'),
803
- {
804
- config: { /* app config */ },
805
- query: { requestId: req.id },
806
- operation: 'checkout'
807
- },
808
- {
809
- formats: ['markdown'],
810
- generateScopeStory: true,
811
- storyOptions: { /* ... */ }
812
- }
813
- );
814
- ```
815
-
816
- * `troubleshootScope(scope, options?)` – Scope-centric troubleshooting
817
-
818
- ```ts
819
- const { scope: scopeResult, reports } = await logger.troubleshootScope(
820
- scopeView, // or ScopeCriteria or scope id string
821
- {
822
- formats: ['markdown', 'json'],
823
- generateScopeStory: true
824
- }
825
- );
826
- ```
827
-
828
- * `scopeByNarratives(options?)` – Narrative-based scoping (optional)
829
-
830
- ### Shadow Controller
831
-
832
- * `logger.shadow.enable(runId, opts?)`
833
- * `logger.shadow.disable(runId)`
834
- * `logger.shadow.isEnabled(runId)`
835
- * `logger.shadow.listActive()`
836
- * `logger.shadow.export(runId, outPath?, compress?) → Promise<string>`
837
- * `logger.shadow.readIndex(runId) → Promise<ShadowIndex>`
838
- * `logger.shadow.cleanupExpired(now?) → Promise<number>`
839
-
840
- *(Shadow writes sidecar files; primary transports unaffected.)*
841
-
842
- ---
843
-
844
- ## Usage Examples
845
-
846
- ### 1) Error → Scoped logs (text filter) → Story + Troubleshooting
847
-
848
- ```ts
849
- import { createLogger, scopeRecord } from 'logs-gateway';
850
-
851
- const logger = createLogger(
852
- { packageName: 'PAYMENTS', envPrefix: 'PAY' },
853
- {
854
- logToConsole: true,
855
- logFormat: 'json',
856
- shadow: {
857
- enabled: true,
858
- format: 'json',
859
- directory: './logs/shadow',
860
- ttlMs: 86400000,
861
- forceVerbose: true
862
- },
863
- scoping: {
864
- enabled: true,
865
- errorScoping: {
866
- enabled: true,
867
- windowMsBefore: 60_000,
868
- windowMsAfter: 30_000
869
- },
870
- buffer: {
871
- maxEntries: 5000,
872
- maxAgeMs: 300_000,
873
- includeLevels: ['verbose','debug','info','warn','error'],
874
- preferShadow: true
875
- }
876
- },
877
- troubleshooting: {
878
- enabled: true,
879
- narrativesPath: './metadata/troubleshooting.json',
880
- output: {
881
- formats: ['markdown'],
882
- emitAsLogEntry: true,
883
- writeToFileDir: './logs/troubleshooting'
884
- }
885
- }
886
- }
887
- );
888
-
889
- async function handleCheckout(req: any) {
890
- const runId = `checkout-${Date.now()}`;
891
- logger.info('Checkout started', { runId });
892
- logger.verbose('Preparing payment context', { runId });
893
-
894
- try {
895
- // ...
896
- throw new Error('Missing connections configuration');
897
- } catch (error) {
898
- // Direct troubleshooting call
899
- const { scope, reports } = await logger.troubleshootError(error, {
900
- config: { /* app config */ },
901
- query: { requestId: req.id },
902
- operation: 'checkout'
903
- }, {
904
- formats: ['markdown'],
905
- generateScopeStory: true,
906
- storyOptions: {
907
- maxEntries: 50,
908
- includeEntryHeader: true,
909
- recordOptions: {
910
- label: 'Log Entry',
911
- maxFieldStringLength: 200,
912
- excludeKeys: ['password', 'token']
913
- }
914
- }
915
- });
916
-
917
- // Scope includes full data + story
918
- console.log(scope?.view.summary);
919
- console.log(scope?.story?.text);
920
-
921
- // Reports contain troubleshooting text
922
- return {
923
- ok: false,
924
- troubleshooting: reports.map(r => r.rendered)
925
- };
926
- }
927
- }
928
- ```
929
-
930
- ### 2) Manual scoping by text ("include any log that mentions X")
931
-
932
- ```ts
933
- // Scope all logs in the last 5 minutes that mention "timeout" anywhere:
934
- const timeoutScope = await logger.scopeLogs({
935
- levelAtLeast: 'debug',
936
- fromTimestamp: new Date(Date.now() - 5 * 60 * 1000).toISOString(),
937
- textIncludesAny: 'timeout' // message or data, case-insensitive
938
- }, {
939
- mode: 'both',
940
- storyOptions: {
941
- includeEntryHeader: true,
942
- recordOptions: { label: 'Timeout Log' }
943
- }
944
- });
945
-
946
- console.log(timeoutScope.view.summary);
947
- console.log(timeoutScope.story?.text);
948
- ```
949
-
950
- ### 3) Web Application
951
-
952
- ```ts
953
- // src/logger.ts
954
- import { createLogger, LoggingConfig, LogsGateway } from 'logs-gateway';
955
-
956
- export function createAppLogger(config?: LoggingConfig): LogsGateway {
957
- return createLogger(
958
- { packageName: 'WEB_APP', envPrefix: 'WEB_APP', debugNamespace: 'web-app' },
959
- config
960
- );
961
- }
962
-
963
- // src/index.ts
964
- const logger = createAppLogger({ logFormat: 'json' });
965
- logger.info('Web application initialized', { version: '1.0.0' });
966
-
967
- async function handleRequest(request: any) {
968
- logger.debug('Handling request', { requestId: request.id });
969
- // ...
970
- logger.info('Request processed', { responseTime: 42, runId: request.runId });
971
- }
972
- ```
973
-
974
- ### 4) Shadow Logging (per-run capture)
975
-
976
- Capture everything for a **specific `runId`** to a side file (forced-verbose), then fetch it — perfect for tests.
977
-
978
- ```ts
979
- const logger = createLogger(
980
- { packageName: 'API', envPrefix: 'API' },
981
- { shadow: { enabled: true, format: 'yaml', ttlMs: 86_400_000 } }
982
- );
983
-
984
- const runId = `test-${Date.now()}`;
985
-
986
- // Turn on capture for this run (can be mid-execution)
987
- logger.shadow.enable(runId);
988
-
989
- logger.info('test start', { runId });
990
- logger.verbose('deep details', { runId, step: 1 });
991
- logger.debug('more details', { runId, step: 2 });
992
-
993
- // ... run test ...
994
-
995
- // Export & stop capturing
996
- await logger.shadow.export(runId, './artifacts'); // returns exported path
997
- logger.shadow.disable(runId);
998
- ```
999
-
1000
- > Shadow files are stored per-run (JSONL or YAML multi-doc), rotated by size/age, and removed by TTL (default **1 day**).
1001
- > If `forceVerbose=true`, all levels for that `runId` are captured even when global level is higher.
1002
-
1003
- ---
1004
-
1005
- ## Correlation & Trails
1006
-
1007
- Attach correlation fields freely on each call:
1008
-
1009
- * **Job/Correlation:** `runId`, `jobId`, `correlationId`, `sessionId`
1010
- * **Depth / Operation trail:** `operationId`, `parentOperationId`, `operationName`, `operationPath`, `operationStep`
1011
- * **Thread / Causal trail:** `threadId`, `eventId`, `causationId`, `sequenceNo`, `partitionKey`, `attempt`, `shardId`, `workerId`
1012
- * **Tracing:** `traceId`, `spanId` (if OpenTelemetry context is active)
1013
-
1014
- Every log entry automatically merges the current trail context. The library provides optional adapters for HTTP and queue systems to **inject/extract** headers across process boundaries.
1015
-
1016
- ---
1017
-
1018
- ## Routing Metadata & Recursion Safety
1019
-
1020
- ```ts
1021
- interface RoutingMeta {
1022
- allowedOutputs?: string[]; // e.g. ['unified-logger','console']
1023
- blockOutputs?: string[]; // e.g. ['unified-logger','file','shadow','troubleshooting']
1024
- reason?: string;
1025
- tags?: string[];
1026
- }
1027
- ```
1028
-
1029
- * Gateway internal logs (`source: 'logs-gateway-internal'`) never reach unified-logger.
1030
- * `_routing` lets you allow/block specific outputs per entry.
1031
- * Shadow Logging honors `_routing.blockOutputs` by default (configurable).
1032
- * Troubleshooting output can be blocked via `_routing.blockOutputs: ['troubleshooting']`.
1033
-
1034
- ---
1035
-
1036
- ## Log Formats
1037
-
1038
- **Text**
1039
-
1040
- ```
1041
- [2025-01-15T10:30:45.123Z] [MY_APP] [INFO] Application initialized {"version":"1.0.0"}
1042
- ```
1043
-
1044
- **JSON**
1045
-
1046
- ```json
1047
- {
1048
- "timestamp": "2025-01-15T10:30:45.123Z",
1049
- "package": "MY_APP",
1050
- "level": "INFO",
1051
- "message": "Application initialized",
1052
- "source": "application",
1053
- "data": {"version": "1.0.0"}
1054
- }
1055
- ```
1056
-
1057
- **YAML** (console/file; unified stays JSON)
1058
-
1059
- ```yaml
1060
- ---
1061
- timestamp: 2025-01-15T10:30:45.123Z
1062
- package: MY_APP
1063
- level: INFO
1064
- message: Application initialized
1065
- source: application
1066
- appName: my-awesome-app
1067
- appVersion: "2.1.0"
1068
- data:
1069
- version: "1.0.0"
1070
- ```
1071
-
1072
- > **Note:** YAML is human-friendly but slower; prefer JSON for ingestion.
1073
-
1074
- ---
1075
-
1076
- ## PII/Credentials Sanitization (opt-in)
1077
-
1078
- Enable to auto-detect and mask common sensitive data (JWTs, API keys, passwords, emails, credit cards, cloud keys, etc.).
1079
- Supports key-based rules (denylist/allowlist), hashing specific fields, depth/size/time guardrails, and a truncation flag.
1080
-
1081
- ```ts
1082
- const logger = createLogger(
1083
- { packageName: 'MY_APP', envPrefix: 'MY_APP' },
1084
- {
1085
- sanitization: {
1086
- enabled: true,
1087
- maskWith: '[REDACTED]',
1088
- keysDenylist: ['authorization','password','secret','api_key'],
1089
- fieldsHashInsteadOfMask: ['userId'],
1090
- detectJWTs: true
1091
- }
1092
- }
1093
- );
1094
- ```
1095
-
1096
- ---
1097
-
1098
- ## Shadow Logging (Per-Run Debug Capture)
1099
-
1100
- Shadow Logging allows you to capture **all logs for a specific `runId`** to a separate file in JSON or YAML format, with **raw (unsanitized) data**, regardless of the global log level. This is ideal for debugging tests, CI runs, or specific production workflows.
1101
-
1102
- ### Key Features
1103
-
1104
- - **Per-runId capture**: Enable capture for specific run identifiers
1105
- - **Verbose by default**: Captures all log levels for the target runId
1106
- - **Raw data**: Bypasses sanitization to preserve original values (âš ī¸ use with caution)
1107
- - **Optional rolling buffer**: Capture logs from before shadow was enabled ("after-the-fact")
1108
- - **TTL cleanup**: Automatically manage storage with time-to-live expiration
1109
- - **No interference**: Shadow writes are async and won't affect primary logging
1110
-
1111
- ### Configuration
1112
-
1113
- #### Constructor Options
1114
-
1115
- ```typescript
1116
- interface ShadowConfig {
1117
- enabled?: boolean; // default: false
1118
- format?: 'json' | 'yaml'; // default: 'json'
1119
- directory?: string; // default: './logs/shadow'
1120
- ttlMs?: number; // default: 86400000 (1 day)
1121
- respectRoutingBlocks?: boolean; // default: true
1122
- rollingBuffer?: {
1123
- maxEntries?: number; // default: 0 (disabled)
1124
- maxAgeMs?: number; // default: 0 (disabled)
1125
- };
1126
- }
1127
- ```
1128
-
1129
- ### Runtime API
1130
-
1131
- #### `logger.shadow.enable(runId, opts?)`
1132
-
1133
- Enable shadow capture for a specific runId. Optionally override config for this run.
1134
-
1135
- ```typescript
1136
- logger.shadow.enable('test-run-123');
1137
-
1138
- // Or with custom options
1139
- logger.shadow.enable('test-run-yaml', {
1140
- format: 'yaml',
1141
- ttlMs: 3600000 // 1 hour
1142
- });
1143
- ```
1144
-
1145
- #### `logger.shadow.disable(runId)`
1146
-
1147
- Stop capturing logs for a runId and finalize the shadow file.
1148
-
1149
- #### `logger.shadow.isEnabled(runId)`
1150
-
1151
- Check if shadow capture is active for a runId.
1152
-
1153
- #### `logger.shadow.listActive()`
1154
-
1155
- List all currently active shadow captures.
1156
-
1157
- #### `logger.shadow.export(runId, outPath?)`
1158
-
1159
- Copy the shadow file to a destination path.
1160
-
1161
- #### `logger.shadow.cleanupExpired(now?)`
1162
-
1163
- Delete expired shadow files based on TTL. Returns number of deleted runs.
1164
-
1165
- ### Important Considerations
1166
-
1167
- âš ī¸ **Security**: Shadow captures **raw, unsanitized data**. This means passwords, API keys, and PII will be preserved in shadow files. Only use shadow logging in secure environments (development, CI, isolated test systems).
1168
-
1169
- 💾 **Storage**: Shadow files can grow quickly when capturing verbose logs. Configure appropriate `ttlMs` values and regularly run `cleanupExpired()`.
1170
-
1171
- đŸšĢ **Default OFF**: Shadow logging is disabled by default and must be explicitly enabled via config or environment variables.
1172
-
1173
- ---
1174
-
1175
- ## Troubleshooting Integration
1176
-
1177
- The troubleshooting integration uses `nx-troubleshooting` to match errors to solutions. See the [nx-troubleshooting documentation](https://www.npmjs.com/package/nx-troubleshooting) for details on creating troubleshooting narratives.
1178
-
1179
- ### Key Features
1180
-
1181
- - **Intelligent Error Matching** – Matches errors to solutions using multiple strategies
1182
- - **Flexible Probe System** – Built-in probes plus extensible custom probes
1183
- - **Template Variables** – Dynamic solution messages with `{{variable}}` syntax
1184
- - **Multiple Output Formats** – Markdown, JSON, and plain text formatting
1185
- - **Automatic Scoping** – Errors automatically trigger scoped log collection
1186
-
1187
- ### Example Troubleshooting Narrative
1188
-
1189
- ```json
1190
- {
1191
- "id": "missing-connections-config",
1192
- "title": "Missing Connections Configuration",
1193
- "description": "The application config is missing the required 'connections' object...",
1194
- "symptoms": [
1195
- {
1196
- "probe": "config-check",
1197
- "params": { "field": "connections" },
1198
- "condition": "result.exists == false"
1199
- }
1200
- ],
1201
- "solution": [
1202
- {
1203
- "type": "code",
1204
- "message": "Add a 'connections' object to your config:",
1205
- "code": "{\n \"connections\": { ... }\n}"
1206
- }
1207
- ]
1208
- }
1209
- ```
1210
-
1211
- ---
1212
-
1213
- ## Backwards Compatibility
1214
-
1215
- * All new behavior is **opt-in**:
1216
- * `scoping.enabled` and `troubleshooting.enabled` default to `false`.
1217
- * If `nx-troubleshooting` is not installed:
1218
- * `troubleshooting.enabled` must remain `false` or initialization fails clearly.
1219
- * If neither shadow nor buffer is configured:
1220
- * `scopeLogs` can still filter **currently available** logs if in-memory buffer is enabled; otherwise, scopes may be empty.
1221
- * `scopeRecord` is pure and can be used independently anywhere.
1222
-
1223
- ---
1224
-
1225
- ## Environment Variables (summary)
1226
-
1227
- | Key | Description | Default |
1228
- | ---------------------- | -------------------------------------- | --------- |
1229
- | `{P}_LOG_TO_CONSOLE` | Enable console output | `true` |
1230
- | `{P}_LOG_TO_FILE` | Enable file output | `false` |
1231
- | `{P}_LOG_FILE` | Log file path | — |
1232
- | `{P}_LOG_TO_UNIFIED` | Enable unified-logger | `false` |
1233
- | `{P}_LOG_LEVEL` | `verbose\|debug\|info\|warn\|error` | `info` |
1234
- | `{P}_LOG_FORMAT` | `text\|json\|yaml\|table` | `table` |
1235
- | `{P}_SHOW_FULL_TIMESTAMP` | Show full ISO timestamp in console | `false` |
1236
- | `{P}_CONSOLE_PACKAGES_SHOW` | Comma-separated packages to show (console only) | (show all) |
1237
- | `{P}_CONSOLE_PACKAGES_HIDE` | Comma-separated packages to hide (console only) | (show all) |
1238
- | `DEBUG` | Namespace(s) enabling verbose+debug | — |
1239
- | `{P}_SANITIZE_ENABLED` | Turn on sanitization | `false` |
1240
- | `{P}_TRACE_OTEL` | Attach `traceId`/`spanId` if available | `true` |
1241
- | `{P}_TRAILS_*` | Toggle trails/header adapters | see above |
1242
- | `{P}_SHADOW_*` | Shadow Logging controls | see above |
1243
- | `{P}_SCOPING_*` | Scoping controls | see above |
1244
- | `{P}_TROUBLESHOOTING_*` | Troubleshooting controls | see above |
1245
-
1246
- ---
1247
-
1248
- ## Development
1249
-
1250
- ```bash
1251
- npm install
1252
- npm run build
1253
- npm run dev
1254
- npm run clean
1255
- ```
1256
-
1257
- ## License
1258
-
1259
- MIT
1260
-
1261
- ---
1262
-
1263
- **Tip for tests:** set a `runId` at the beginning of each test (e.g., `runId: 'ci-<suite>-<timestamp>'`), enable **Shadow Logging**, run, then `export()` the captured logs for artifacts/triage.
1
+ # logs-gateway
2
+
3
+ A standardized logging gateway for Node.js applications. Flexible multi-transport logging with **console**, **file**, and **unified-logger** outputs; **ENV-first** configuration; **PII/credentials sanitization**; **dual correlation trails** (operation & thread); **OpenTelemetry** context; **YAML/JSON/text** formats; **per-run "Shadow Logging"** for test/debug capture; **scoping** with text filters; **story output** powered by `scopeRecord`; and **troubleshooting integration** with `nx-troubleshooting`.
4
+
5
+ ## 🚀 Env-Ready Component (ERC 2.0)
6
+
7
+ This component supports **zero-config initialization** via environment variables and is compliant with the [Env-Ready Component Standard (ERC 2.0)](https://github.com/xeonox/erc-standard).
8
+
9
+ ### Package-level `.env` contract (libraries)
10
+
11
+ For **per-package** log thresholds, logs-gateway follows **`{PREFIX}_LOGS_LEVEL`** (canonical). **`{PREFIX}_LOG_LEVEL`** remains supported when **`_LOGS_LEVEL`** is not set in the environment. Omitting both keys defaults to **`warn`** (warn and error only); set **`{PREFIX}_LOGS_LEVEL=off`** to silence. Cross-cutting options (console, file, format) stay host-level. Full detail: [`docs/package-usage.md`](./docs/package-usage.md) (published on npm). Helpers: `resolvePackageLogsLevel`, `parsePackageLogsLevelString`, `packageLogsLevelEnvKey`.
12
+
13
+ ### ERC 2.0 Compliance
14
+
15
+ - ✅ **Auto-Discovery**: Zero-config initialization from environment variables
16
+ - ✅ **Complete Documentation**: All environment variables documented (including dependencies)
17
+ - ✅ **Type Safety**: Automatic type coercion and validation
18
+ - ✅ **Manifest**: Auto-generated `erc-manifest.json` with all requirements
19
+ - ✅ **Example File**: Auto-generated `.env.example` with all transitive requirements
20
+ - ✅ **Dependency Tracking**: Documents both ERC and non-ERC dependencies
21
+
22
+ ### Quick Start (Zero-Config Mode)
23
+
24
+ ```bash
25
+ # 1. Install the package
26
+ npm install logs-gateway
27
+
28
+ # 2. Set environment variables (replace MY_APP with your prefix)
29
+ export MY_APP_LOG_TO_CONSOLE=true
30
+ export MY_APP_LOGS_LEVEL=warn
31
+ export MY_APP_LOG_FORMAT=json
32
+
33
+ # 3. Use with zero config!
34
+ import { createLogger } from 'logs-gateway';
35
+ const logger = createLogger(
36
+ { packageName: 'MY_APP', envPrefix: 'MY_APP' }
37
+ ); // Auto-discovers from process.env
38
+ ```
39
+
40
+ ### Advanced Mode (Programmatic Configuration)
41
+
42
+ ```typescript
43
+ import { createLogger } from 'logs-gateway';
44
+
45
+ const logger = createLogger(
46
+ { packageName: 'MY_APP', envPrefix: 'MY_APP' },
47
+ {
48
+ logToFile: true,
49
+ logFilePath: '/var/log/myapp.log',
50
+ logLevel: 'info'
51
+ }
52
+ );
53
+ ```
54
+
55
+ ### Environment Variables
56
+
57
+ **Note**: This component uses **dynamic environment variable prefixes** based on `packageConfig.envPrefix`. Replace `{PREFIX}` with your actual prefix (e.g., `MY_APP`, `API_SERVICE`).
58
+
59
+ See `.env.example` for the complete list of required and optional variables with descriptions. Generate it by running:
60
+
61
+ ```bash
62
+ npm run generate-erc
63
+ ```
64
+
65
+ ### Dependencies
66
+
67
+ - ✅ **nx-config2** (ERC 2.0) - Configuration engine
68
+ - â„šī¸ **@x-developer/unified-logger** (non-ERC) - Requirements manually documented
69
+ - â„šī¸ **nx-troubleshooting** (non-ERC, optional) - Requirements manually documented
70
+
71
+ ---
72
+
73
+ ## Features
74
+
75
+ * ✅ Console output (default)
76
+ * ✅ File output (optional)
77
+ * ✅ Unified-logger output (Papertrail/UDP/Console via `@x-developer/unified-logger`)
78
+ * ✅ Dual/triple output (console + file + unified-logger)
79
+ * ✅ Environment variable configuration
80
+ * ✅ Custom logger injection (Winston, Pino, etc.)
81
+ * ✅ Package-specific prefixes
82
+ * ✅ **Five log levels**: `verbose`, `debug`, `info`, `warn`, `error`
83
+ * ✅ **Multiple formats**: `text`, `json`, `yaml` (console & file; unified stays JSON)
84
+ * ✅ **PII/Credentials sanitization** (auto-detect & mask, opt-in)
85
+ * ✅ **Dual trails**: operation (Depth) & thread (Causal), plus `runId`, `jobId`, `correlationId`, `sessionId`
86
+ * ✅ **OpenTelemetry context** (`traceId`, `spanId`) when available
87
+ * ✅ Routing metadata (control outputs per entry)
88
+ * ✅ Recursion safety (prevent circular/unified feedback)
89
+ * ✅ TypeScript support
90
+ * ✅ **Shadow Logging (per-run capture)** with TTL & forced-verbose, great for tests
91
+ * ✅ **Scoping** – Derive focused subsets of logs with text filters and correlation keys
92
+ * ✅ **Story Output** – Human-readable narrative format built automatically from log entries
93
+ * ✅ **Troubleshooting Integration** – Wire in `nx-troubleshooting` for intelligent error-to-solution matching
94
+
95
+ ---
96
+
97
+ ## Installation
98
+
99
+ ```bash
100
+ npm install logs-gateway
101
+ ```
102
+
103
+ For troubleshooting integration:
104
+
105
+ ```bash
106
+ npm install logs-gateway nx-troubleshooting
107
+ ```
108
+
109
+ ---
110
+
111
+ ## Quick Start
112
+
113
+ ```ts
114
+ import { createLogger } from 'logs-gateway';
115
+
116
+ const logger = createLogger(
117
+ { packageName: 'MY_APP', envPrefix: 'MY_APP' },
118
+ {
119
+ logToFile: true,
120
+ logFilePath: '/var/log/myapp.log',
121
+ logLevel: 'info', // verbose|debug|info|warn|error
122
+ logFormat: 'json', // text|json|yaml (yaml: console/file only)
123
+ enableUnifiedLogger: true,
124
+ unifiedLogger: {
125
+ transports: { papertrail: true },
126
+ service: 'my-app',
127
+ env: 'production'
128
+ },
129
+ // Optional: per-run Shadow Logging defaults (can also be enabled at runtime)
130
+ shadow: { enabled: false, ttlMs: 86_400_000 }, // 1 day
131
+ // Optional: Scoping & Troubleshooting
132
+ scoping: {
133
+ enabled: true,
134
+ errorScoping: { enabled: true, windowMsBefore: 60_000, windowMsAfter: 30_000 },
135
+ buffer: { maxEntries: 5000, preferShadow: true }
136
+ },
137
+ troubleshooting: {
138
+ enabled: true,
139
+ narrativesPath: './metadata/troubleshooting.json',
140
+ output: { formats: ['markdown'], emitAsLogEntry: true }
141
+ }
142
+ }
143
+ );
144
+
145
+ // Standard usage
146
+ logger.verbose('Very detailed info', { step: 'init' });
147
+ logger.debug('Debug info', { data: 'x' });
148
+ logger.info('Application initialized', { version: '1.0.0' });
149
+ logger.warn('Deprecated feature used');
150
+ logger.error('Error occurred', { error: new Error('boom') });
151
+ ```
152
+
153
+ ---
154
+
155
+ ## Automatic Application Identification
156
+
157
+ **logs-gateway** automatically detects and includes your application's package name and version in every log entry. This is done by reading the consuming project's `package.json` file (the project using logs-gateway, not logs-gateway itself).
158
+
159
+ ### How It Works
160
+
161
+ When you create a logger instance, logs-gateway automatically:
162
+ 1. Searches up the directory tree from `process.cwd()` to find the nearest `package.json`
163
+ 2. Extracts the `name` and `version` fields from that file
164
+ 3. Includes them in all log entries as `appName` and `appVersion`
165
+
166
+ This happens **automatically** - no configuration needed! The detection is cached after the first call for performance.
167
+
168
+ ### Example
169
+
170
+ If your project's `package.json` contains:
171
+ ```json
172
+ {
173
+ "name": "my-awesome-app",
174
+ "version": "2.1.0"
175
+ }
176
+ ```
177
+
178
+ Then all log entries will automatically include:
179
+ ```json
180
+ {
181
+ "timestamp": "2025-01-15T10:30:45.123Z",
182
+ "package": "MY_APP",
183
+ "level": "INFO",
184
+ "message": "Application initialized",
185
+ "appName": "my-awesome-app",
186
+ "appVersion": "2.1.0",
187
+ "data": { ... }
188
+ }
189
+ ```
190
+
191
+ ### Benefits
192
+
193
+ - **Traceability**: Know exactly which application version generated each log entry
194
+ - **Debugging**: Filter logs by application version in centralized logging systems
195
+ - **Deployment Tracking**: Identify which deployments are running in production
196
+ - **Zero Configuration**: Works automatically without any API changes
197
+
198
+ ### Important Notes
199
+
200
+ - The detection searches from `process.cwd()` (the working directory where your app runs)
201
+ - It stops at the first `package.json` that is not in `node_modules`
202
+ - If no `package.json` is found, `appName` and `appVersion` are simply omitted (no error)
203
+ - The result is cached after first detection for performance
204
+
205
+ ---
206
+
207
+ ## Overview: Scoping, Story Output & Troubleshooting
208
+
209
+ This extension adds four major capabilities:
210
+
211
+ 1. **Runtime Filtering (logger-debug.json)** – Filter logs at the source before they're written. Place `logger-debug.json` at your project root to filter logs by identity or application name. This reduces noise at runtime and complements post-processing scoping.
212
+
213
+ 2. **Scoping** – Given a verbose log stream, derive a **focused subset** of logs relevant to a problem or question. Scopes can be:
214
+ * Error-centric (anchor on a specific `error` entry)
215
+ * Run/Correlation-centric (anchor on `runId`, `correlationId`, etc.)
216
+ * Text-based (filter by message/data content)
217
+ * Narrative-based (optional, driven by "scoping narratives")
218
+
219
+ Scoping **always uses verbose logs** if available, regardless of the current log level.
220
+
221
+ 3. **Story vs Full Data Output** – A scope can be returned as:
222
+ * **Full data**: structured `ScopedLogView` with all entries
223
+ * **Story**: human-readable narrative built automatically from entries using a generic `scopeRecord` helper
224
+ * Or **both**
225
+
226
+ 4. **Troubleshooting Integration** – Wire in `nx-troubleshooting` so that errors/scopes:
227
+ * Are matched to **troubleshooting narratives**, and
228
+ * Produce troubleshooting artifacts (Markdown/JSON/text) as **another log output channel** (`troubleshooting`)
229
+
230
+ The same `scopeRecord` helper is also exported as a **generic tool** for scoping arbitrary JSON records (not just logs).
231
+
232
+ ---
233
+
234
+ ## Configuration
235
+
236
+ ### Via Constructor
237
+
238
+ ```ts
239
+ const logger = createLogger(
240
+ { packageName: 'MY_PACKAGE', envPrefix: 'MY_PACKAGE', debugNamespace: 'my-pkg' },
241
+ {
242
+ // Outputs
243
+ logToConsole: true, // default: true
244
+ logToFile: false, // default: false
245
+ logFilePath: '/var/log/app.log', // required if logToFile
246
+ enableUnifiedLogger: false, // default: false
247
+ unifiedLogger: { /* ... */ },
248
+
249
+ // Behavior
250
+ logLevel: 'info', // verbose|debug|info|warn|error (default: info)
251
+ logFormat: 'text', // text|json|yaml (default: text)
252
+ defaultSource: 'application', // fallback source tag
253
+
254
+ // Sanitization (opt-in)
255
+ sanitization: {
256
+ enabled: false, // default: false
257
+ maskWith: '[REDACTED]',
258
+ keysDenylist: ['authorization','password','secret','api_key'],
259
+ fieldsHashInsteadOfMask: ['userId'],
260
+ detectJWTs: true
261
+ // + other detectors & guardrails
262
+ },
263
+
264
+ // Trails & tracing (optional; safe no-ops if not used)
265
+ trails: {
266
+ enableDepthTrail: true, // operation trail
267
+ enableThreadTrail: true, // causal/thread trail
268
+ injectHeaders: true, // for HTTP/queues adapters
269
+ extractHeaders: true
270
+ },
271
+ tracing: { enableOtelContext: true },
272
+
273
+ // Shadow Logging (per-run capture; can also be toggled at runtime)
274
+ shadow: {
275
+ enabled: false, // default: false
276
+ format: 'json', // json|yaml (default: json)
277
+ directory: './logs/shadow', // default path
278
+ ttlMs: 86_400_000, // 1 day
279
+ forceVerbose: true, // capture all levels for the runId
280
+ respectRoutingBlocks: true, // honor _routing.blockOutputs: ['shadow'|'file']
281
+ includeRaw: false, // also write unsanitized (dangerous; tests only)
282
+ rollingBuffer: { maxEntries: 0, maxAgeMs: 0 } // optional retro-capture
283
+ },
284
+
285
+ // Scoping (opt-in)
286
+ scoping: {
287
+ enabled: false, // default: false
288
+ errorScoping: {
289
+ enabled: true, // default: true if scoping.enabled
290
+ levels: ['error'], // default: ['error']
291
+ windowMsBefore: 30_000, // default: 30_000
292
+ windowMsAfter: 30_000 // default: 30_000
293
+ },
294
+ runScoping: {
295
+ enabled: true, // default: true if scoping.enabled
296
+ defaultWindowMsBeforeFirstError: 60_000,
297
+ defaultWindowMsAfterLastEntry: 30_000
298
+ },
299
+ narrativeScoping: {
300
+ enabled: false, // default: false
301
+ narrativesPath: './metadata/log-scopes.json',
302
+ envPrefix: 'NX_SCOPE'
303
+ },
304
+ buffer: {
305
+ maxEntries: 0, // default: 0 => disabled if Shadow is enough
306
+ maxAgeMs: 0,
307
+ includeLevels: ['verbose','debug','info','warn','error'],
308
+ preferShadow: true // default: true
309
+ }
310
+ },
311
+
312
+ // Troubleshooting (opt-in, requires nx-troubleshooting)
313
+ troubleshooting: {
314
+ enabled: false, // default: false
315
+ narrativesPath: './metadata/troubleshooting.json',
316
+ envPrefix: 'NX_TROUBLE',
317
+ loggingConfig: { /* optional */ },
318
+ engine: undefined, // optional DI
319
+ output: {
320
+ formats: ['markdown'], // default: ['markdown']
321
+ writeToFileDir: undefined,
322
+ attachToShadow: false,
323
+ emitAsLogEntry: false,
324
+ callback: undefined
325
+ }
326
+ }
327
+ }
328
+ );
329
+ ```
330
+
331
+ ### Via Environment Variables
332
+
333
+ **Note**: Environment variable names use a **dynamic prefix** based on `packageConfig.envPrefix`. Replace `{PREFIX}` in the examples below with your actual prefix (e.g., `MY_APP`, `API_SERVICE`).
334
+
335
+ ```bash
336
+ # Console & file
337
+ {PREFIX}_LOG_TO_CONSOLE=true|false
338
+ {PREFIX}_LOG_TO_FILE=true|false
339
+ {PREFIX}_LOG_FILE=/path/to/log
340
+
341
+ # Unified-logger
342
+ {PREFIX}_LOG_TO_UNIFIED=true|false
343
+
344
+ # Level & format (per-package threshold: canonical _LOGS_LEVEL; legacy _LOG_LEVEL if _LOGS_LEVEL unset)
345
+ {PREFIX}_LOGS_LEVEL=off|none|silent|error|warn|info|debug|verbose
346
+ {PREFIX}_LOG_LEVEL=verbose|debug|info|warn|error
347
+ {PREFIX}_LOG_FORMAT=text|json|yaml|table
348
+
349
+ # Console output options
350
+ {PREFIX}_SHOW_FULL_TIMESTAMP=true|false # Show full ISO timestamp (default: false)
351
+ {PREFIX}_CONSOLE_PACKAGES_SHOW=package1,package2 # Only show these packages in console (default: show all)
352
+ {PREFIX}_CONSOLE_PACKAGES_HIDE=package1,package2 # Hide these packages in console (default: show all)
353
+
354
+ # Debug namespace → enables verbose+debug for that namespace
355
+ DEBUG=my-pkg,other-*
356
+
357
+ # Sanitization (subset shown)
358
+ {PREFIX}_SANITIZE_ENABLED=true|false
359
+ {PREFIX}_SANITIZE_KEYS_DENYLIST=authorization,token,secret,api_key,password
360
+
361
+ # Trails/tracing
362
+ {PREFIX}_TRACE_OTEL=true|false
363
+ {PREFIX}_TRAILS_DEPTH=true|false
364
+ {PREFIX}_TRAILS_THREAD=true|false
365
+ {PREFIX}_TRAILS_INJECT=true|false
366
+ {PREFIX}_TRAILS_EXTRACT=true|false
367
+
368
+ # Shadow Logging
369
+ {PREFIX}_SHADOW_ENABLED=true|false
370
+ {PREFIX}_SHADOW_FORMAT=json|yaml
371
+ {PREFIX}_SHADOW_DIR=/var/log/myapp/shadow
372
+ {PREFIX}_SHADOW_TTL_MS=86400000
373
+ {PREFIX}_SHADOW_FORCE_VERBOSE=true|false
374
+ {PREFIX}_SHADOW_RESPECT_ROUTING=true|false
375
+ {PREFIX}_SHADOW_INCLUDE_RAW=false
376
+ {PREFIX}_SHADOW_BUFFER_ENTRIES=0
377
+ {PREFIX}_SHADOW_BUFFER_AGE_MS=0
378
+
379
+ # Scoping
380
+ {PREFIX}_SCOPING_ENABLED=true|false
381
+ {PREFIX}_SCOPING_ERROR_ENABLED=true|false
382
+ {PREFIX}_SCOPING_ERROR_WINDOW_MS_BEFORE=30000
383
+ {PREFIX}_SCOPING_ERROR_WINDOW_MS_AFTER=30000
384
+ {PREFIX}_SCOPING_BUFFER_ENTRIES=5000
385
+ {PREFIX}_SCOPING_BUFFER_AGE_MS=300000
386
+ {PREFIX}_SCOPING_BUFFER_PREFER_SHADOW=true|false
387
+
388
+ # Troubleshooting
389
+ {PREFIX}_TROUBLESHOOTING_ENABLED=true|false
390
+ {PREFIX}_TROUBLESHOOTING_NARRATIVES_PATH=./metadata/troubleshooting.json
391
+ {PREFIX}_TROUBLESHOOTING_OUTPUT_FORMATS=markdown,json
392
+ {PREFIX}_TROUBLESHOOTING_OUTPUT_EMIT_AS_LOG_ENTRY=true|false
393
+
394
+ # Unified-logger dependencies (non-ERC, manually documented)
395
+ # Required when unified-logger papertrail transport is enabled:
396
+ PAPERTRAIL_HOST=logs.papertrailapp.com
397
+ PAPERTRAIL_PORT=12345
398
+
399
+ # Required when unified-logger udpRelay transport is enabled:
400
+ UDP_RELAY_HOST=127.0.0.1
401
+ UDP_RELAY_PORT=514
402
+ ```
403
+
404
+ > **Default min level:** `info`.
405
+ > **`DEBUG=`**: enables **both** `verbose` and `debug` for matching namespaces.
406
+ >
407
+ > **ERC 2.0 Note**: Generate a complete `.env.example` file with all variables by running `npm run generate-erc`.
408
+
409
+ ---
410
+
411
+ ## Core Types
412
+
413
+ ### LogEntry
414
+
415
+ Internal normalized log shape:
416
+
417
+ ```ts
418
+ export interface LogEntry {
419
+ timestamp: string; // ISO-8601
420
+ level: 'verbose' | 'debug' | 'info' | 'warn' | 'error';
421
+ package: string;
422
+ message: string;
423
+ source?: string; // e.g. 'application', 'auth-service'
424
+ data?: Record<string, any>; // user metadata, error, ids, etc.
425
+
426
+ // Automatic application identification (from consuming project's package.json)
427
+ appName?: string; // Auto-detected from package.json "name" field
428
+ appVersion?: string; // Auto-detected from package.json "version" field
429
+
430
+ // Correlation / trails / tracing
431
+ runId?: string;
432
+ jobId?: string;
433
+ correlationId?: string;
434
+ sessionId?: string;
435
+ operationId?: string;
436
+ parentOperationId?: string;
437
+ operationName?: string;
438
+ threadId?: string;
439
+ traceId?: string;
440
+ spanId?: string;
441
+
442
+ // Routing
443
+ _routing?: RoutingMeta;
444
+
445
+ // Optional scope metadata (for future/advanced use)
446
+ scope?: ScopedMetadata;
447
+ }
448
+ ```
449
+
450
+ ### Runtime Filtering (logger-debug.json)
451
+
452
+ Place a `logger-debug.json` file at your project root to filter logs at runtime:
453
+
454
+ ```json
455
+ {
456
+ "scoping": {
457
+ "status": "enabled",
458
+ "filterIdentities": ["src/auth.ts:login", "src/payment.ts:processPayment"],
459
+ "filteredApplications": ["my-app", "other-app"],
460
+ "between": [
461
+ {
462
+ "action": "include",
463
+ "exactMatch": false,
464
+ "searchLog": false,
465
+ "startIdentities": ["src/api.ts:handleRequest"],
466
+ "endIdentities": ["src/api.ts:handleRequestEnd"]
467
+ }
468
+ ]
469
+ }
470
+ }
471
+ ```
472
+
473
+ **Behavior:**
474
+ - When `status: "enabled"`, logs are filtered before being written to any output (console, file, unified-logger, shadow)
475
+ - A log is included if its `identity` matches any entry in `filterIdentities` **OR** its `appName` matches any entry in `filteredApplications` **OR** it falls within an active "between" range (OR logic)
476
+ - If all filters are empty or missing, all logs are included (no filtering)
477
+ - File is auto-discovered from `process.cwd()` (searches up directory tree like package.json)
478
+ - Configuration is loaded once at startup (not reloaded dynamically)
479
+ - If file is not found or invalid, all logs are shown (graceful fallback)
480
+
481
+ **Examples:**
482
+
483
+ ```json
484
+ // Filter by identity only - only show logs from specific code locations
485
+ {
486
+ "scoping": {
487
+ "status": "enabled",
488
+ "filterIdentities": ["src/auth.ts:login", "src/payment.ts:processPayment"]
489
+ }
490
+ }
491
+ ```
492
+
493
+ ```json
494
+ // Filter by application only - only show logs from specific apps
495
+ {
496
+ "scoping": {
497
+ "status": "enabled",
498
+ "filteredApplications": ["my-app"]
499
+ }
500
+ }
501
+ ```
502
+
503
+ ```json
504
+ // Filter by both (OR logic - matches if identity OR appName matches)
505
+ {
506
+ "scoping": {
507
+ "status": "enabled",
508
+ "filterIdentities": ["src/auth.ts:login"],
509
+ "filteredApplications": ["my-app"]
510
+ }
511
+ }
512
+ ```
513
+
514
+ ```json
515
+ // Between rules - stateful range-based filtering
516
+ {
517
+ "scoping": {
518
+ "status": "enabled",
519
+ "between": [
520
+ {
521
+ "action": "include",
522
+ "exactMatch": false,
523
+ "searchLog": false,
524
+ "startIdentities": ["src/api.ts:handleRequest"],
525
+ "endIdentities": ["src/api.ts:handleRequestEnd"]
526
+ },
527
+ {
528
+ "action": "exclude",
529
+ "exactMatch": true,
530
+ "searchLog": false,
531
+ "startIdentities": ["src/db.ts:query"],
532
+ "endIdentities": ["src/db.ts:queryEnd"]
533
+ },
534
+ {
535
+ "action": "include",
536
+ "exactMatch": false,
537
+ "searchLog": true,
538
+ "startIdentities": [],
539
+ "endIdentities": ["src/init.ts:complete"]
540
+ },
541
+ {
542
+ "action": "include",
543
+ "exactMatch": true,
544
+ "searchLog": true,
545
+ "startIdentities": ["Payment started"],
546
+ "endIdentities": ["Payment completed"]
547
+ }
548
+ ]
549
+ }
550
+ }
551
+ ```
552
+
553
+ **Between Rules:**
554
+ - **Stateful filtering**: Tracks active ranges across log calls
555
+ - **`action`**: `"include"` to show logs within range, `"exclude"` to hide logs within range
556
+ - **`exactMatch`**:
557
+ - `true`: Exact string match (case sensitive)
558
+ - `false`: Partial substring match (case insensitive, default)
559
+ - **`searchLog`**:
560
+ - `true`: Search entire log (message + identity + all meta fields stringified)
561
+ - `false`: Search only identity field (default)
562
+ - **`startIdentities`**: Array of patterns that activate the range. Empty array means range starts from the beginning
563
+ - **`endIdentities`**: Array of patterns that deactivate the range. Empty array means range never ends
564
+ - **Multiple ranges**: Can overlap and are tracked independently. Uses OR logic (if ANY include rule is active, include; if ANY exclude rule is active, exclude)
565
+ - **Range behavior**:
566
+ - When a log matches a start identity, the range becomes active
567
+ - When a log matches an end identity, the range becomes inactive
568
+ - If a log matches both start and end identities, the range state toggles
569
+ - Ranges with empty `startIdentities` are active from the first log
570
+ - Ranges with empty `endIdentities` never close once activated
571
+
572
+ **Integration with Existing Scoping:**
573
+ - `logger-debug.json` → Runtime filtering (reduces noise at source, filters before writing)
574
+ - `scopeLogs()` → Post-processing scoping (analyzes already-written logs)
575
+ - Both can be used together for maximum control
576
+
577
+ ### ScopeCriteria
578
+
579
+ Scoping criteria defines *which logs* belong to a scope. It supports:
580
+
581
+ * Correlation keys
582
+ * Time windows
583
+ * Levels
584
+ * Sources
585
+ * **Text matching on message and data**
586
+
587
+ ```ts
588
+ export interface ScopeCriteria {
589
+ // Correlation / keys
590
+ runId?: string;
591
+ correlationId?: string;
592
+ sessionId?: string;
593
+ threadId?: string;
594
+ traceId?: string;
595
+
596
+ // Time window bounds
597
+ fromTimestamp?: string; // ISO-8601
598
+ toTimestamp?: string; // ISO-8601
599
+
600
+ // Window relative to an anchor (error or first/last entry)
601
+ windowMsBefore?: number; // relative to anchor timestamp
602
+ windowMsAfter?: number;
603
+
604
+ // Levels
605
+ levelAtLeast?: 'verbose' | 'debug' | 'info' | 'warn' | 'error';
606
+ includeLevels?: ('verbose'|'debug'|'info'|'warn'|'error')[];
607
+
608
+ // Source filters
609
+ sources?: string[]; // e.g. ['api-gateway','payments-service']
610
+
611
+ // TEXT FILTERS
612
+ /**
613
+ * Match logs whose message OR data (stringified) contains ANY of the given strings (case-insensitive).
614
+ * - string: single substring
615
+ * - string[]: log must contain at least one of them
616
+ */
617
+ textIncludesAny?: string | string[];
618
+
619
+ /**
620
+ * Match logs whose message OR data (stringified) contains ALL of the given substrings (case-insensitive).
621
+ */
622
+ textIncludesAll?: string[];
623
+
624
+ /**
625
+ * Optional RegExp filter over the combined text (message + JSON-stringified data).
626
+ * If provided as string, it is treated as a new RegExp(text, 'i').
627
+ */
628
+ textMatches?: RegExp | string;
629
+
630
+ // Scope metadata filters (if used)
631
+ scopeTags?: string[]; // must include all provided tags
632
+
633
+ // Custom predicate for in-process advanced filtering
634
+ predicate?: (entry: LogEntry) => boolean;
635
+ }
636
+ ```
637
+
638
+ **Text filtering behavior:**
639
+
640
+ * Combine `entry.message` and a JSON string of `entry.data` into one string (e.g. `"Payment failed {...}"`).
641
+ * Apply:
642
+ * `textIncludesAny` – inclusive OR.
643
+ * `textIncludesAll` – inclusive AND.
644
+ * `textMatches` – regex test.
645
+ * All text filtering is **case-insensitive** by default.
646
+ * Text filters are **ANDed** with other criteria (correlation, time, etc.).
647
+
648
+ ### ScopedLogView (full data)
649
+
650
+ ```ts
651
+ export interface ScopedLogView {
652
+ id: string; // e.g. 'scope:runId:checkout-42'
653
+ criteria: ScopeCriteria;
654
+ entries: LogEntry[]; // sorted by timestamp ascending
655
+ summary: {
656
+ firstTimestamp?: string;
657
+ lastTimestamp?: string;
658
+ totalCount: number;
659
+ errorCount: number;
660
+ warnCount: number;
661
+ infoCount: number;
662
+ debugCount: number;
663
+ verboseCount: number;
664
+ uniqueSources: string[];
665
+ };
666
+ }
667
+ ```
668
+
669
+ ### scopeRecord – Generic Tool
670
+
671
+ The `scopeRecord` helper is **generic** (not log-specific): given any JSON record, it produces:
672
+
673
+ * A human-readable text description ("story" of the record).
674
+ * A structured description (fields, truncation info).
675
+
676
+ This function is:
677
+ * Used internally to build **scope stories** from `LogEntry`s / aggregated records.
678
+ * Exported publicly so other code can reuse it for non-log use cases.
679
+
680
+ ```ts
681
+ import { scopeRecord } from 'logs-gateway';
682
+
683
+ // Example usage
684
+ const record = {
685
+ userId: '123',
686
+ action: 'payment',
687
+ amount: 100.50,
688
+ timestamp: '2025-01-01T10:00:00Z'
689
+ };
690
+
691
+ const result = scopeRecord(record, {
692
+ label: 'Payment Event',
693
+ maxFieldStringLength: 200,
694
+ excludeKeys: ['password', 'token']
695
+ });
696
+
697
+ console.log(result.text);
698
+ // Output: "Payment Event\n User ID: 123\n Action: payment\n Amount: 100.5\n Timestamp: 2025-01-01T10:00:00Z"
699
+
700
+ console.log(result.structured);
701
+ // Output: { label: 'Payment Event', fields: [...], ... }
702
+ ```
703
+
704
+ **Types:**
705
+
706
+ ```ts
707
+ export interface AutoScopeRecordOptions {
708
+ label?: string;
709
+ formatting?: ScopingFormattingOptions;
710
+ maxFields?: number;
711
+ maxFieldStringLength?: number;
712
+ includeKeys?: string[];
713
+ excludeKeys?: string[];
714
+ skipNullish?: boolean;
715
+ header?: string;
716
+ footer?: string;
717
+ }
718
+
719
+ export interface ScopeRecordResult {
720
+ text: string; // Human-readable text output
721
+ structured: StructuredScopedPayload; // Structured representation
722
+ }
723
+ ```
724
+
725
+ ### Scope Story Output
726
+
727
+ To let a scope answer with full data or story format:
728
+
729
+ ```ts
730
+ export type ScopeOutputMode = 'raw' | 'story' | 'both';
731
+
732
+ export interface ScopeStoryOptions {
733
+ recordOptions?: AutoScopeRecordOptions;
734
+ maxEntries?: number;
735
+ includeEntryHeader?: boolean;
736
+ }
737
+
738
+ export interface ScopeLogsResult {
739
+ view: ScopedLogView; // always present
740
+ story?: ScopedLogStory; // present if mode = 'story' or 'both'
741
+ }
742
+ ```
743
+
744
+ ---
745
+
746
+ ## API Reference
747
+
748
+ ### `createLogger(packageConfig, userConfig?) → LogsGateway`
749
+
750
+ ### `LogsGateway` Methods
751
+
752
+ #### Standard Logging
753
+
754
+ * `verbose(message, data?)`
755
+ * `debug(message, data?)`
756
+ * `info(message, data?)`
757
+ * `warn(message, data?)`
758
+ * `error(message, data?)`
759
+ * `isLevelEnabled(level)` – threshold check (namespace DEBUG forces verbose+debug)
760
+ * `getConfig()` – effective resolved config
761
+
762
+ #### Scoping
763
+
764
+ * `scopeLogs(criteria, options?)` – Scope logs by criteria, return full data and/or story
765
+
766
+ ```ts
767
+ const result = await logger.scopeLogs({
768
+ runId: 'checkout-42',
769
+ textIncludesAny: 'timeout',
770
+ levelAtLeast: 'debug'
771
+ }, {
772
+ mode: 'both',
773
+ storyOptions: {
774
+ maxEntries: 50,
775
+ includeEntryHeader: true,
776
+ recordOptions: {
777
+ label: 'Log Entry',
778
+ maxFieldStringLength: 200
779
+ }
780
+ }
781
+ });
782
+
783
+ console.log(result.view.summary);
784
+ console.log(result.story?.text);
785
+ ```
786
+
787
+ #### Troubleshooting
788
+
789
+ * `troubleshootError(error, context?, options?)` – Error-centric troubleshooting
790
+
791
+ ```ts
792
+ const { scope, reports } = await logger.troubleshootError(
793
+ new Error('Missing connections configuration'),
794
+ {
795
+ config: { /* app config */ },
796
+ query: { requestId: req.id },
797
+ operation: 'checkout'
798
+ },
799
+ {
800
+ formats: ['markdown'],
801
+ generateScopeStory: true,
802
+ storyOptions: { /* ... */ }
803
+ }
804
+ );
805
+ ```
806
+
807
+ * `troubleshootScope(scope, options?)` – Scope-centric troubleshooting
808
+
809
+ ```ts
810
+ const { scope: scopeResult, reports } = await logger.troubleshootScope(
811
+ scopeView, // or ScopeCriteria or scope id string
812
+ {
813
+ formats: ['markdown', 'json'],
814
+ generateScopeStory: true
815
+ }
816
+ );
817
+ ```
818
+
819
+ * `scopeByNarratives(options?)` – Narrative-based scoping (optional)
820
+
821
+ ### Shadow Controller
822
+
823
+ * `logger.shadow.enable(runId, opts?)`
824
+ * `logger.shadow.disable(runId)`
825
+ * `logger.shadow.isEnabled(runId)`
826
+ * `logger.shadow.listActive()`
827
+ * `logger.shadow.export(runId, outPath?, compress?) → Promise<string>`
828
+ * `logger.shadow.readIndex(runId) → Promise<ShadowIndex>`
829
+ * `logger.shadow.cleanupExpired(now?) → Promise<number>`
830
+
831
+ *(Shadow writes sidecar files; primary transports unaffected.)*
832
+
833
+ ---
834
+
835
+ ## Usage Examples
836
+
837
+ ### 1) Error → Scoped logs (text filter) → Story + Troubleshooting
838
+
839
+ ```ts
840
+ import { createLogger, scopeRecord } from 'logs-gateway';
841
+
842
+ const logger = createLogger(
843
+ { packageName: 'PAYMENTS', envPrefix: 'PAY' },
844
+ {
845
+ logToConsole: true,
846
+ logFormat: 'json',
847
+ shadow: {
848
+ enabled: true,
849
+ format: 'json',
850
+ directory: './logs/shadow',
851
+ ttlMs: 86400000,
852
+ forceVerbose: true
853
+ },
854
+ scoping: {
855
+ enabled: true,
856
+ errorScoping: {
857
+ enabled: true,
858
+ windowMsBefore: 60_000,
859
+ windowMsAfter: 30_000
860
+ },
861
+ buffer: {
862
+ maxEntries: 5000,
863
+ maxAgeMs: 300_000,
864
+ includeLevels: ['verbose','debug','info','warn','error'],
865
+ preferShadow: true
866
+ }
867
+ },
868
+ troubleshooting: {
869
+ enabled: true,
870
+ narrativesPath: './metadata/troubleshooting.json',
871
+ output: {
872
+ formats: ['markdown'],
873
+ emitAsLogEntry: true,
874
+ writeToFileDir: './logs/troubleshooting'
875
+ }
876
+ }
877
+ }
878
+ );
879
+
880
+ async function handleCheckout(req: any) {
881
+ const runId = `checkout-${Date.now()}`;
882
+ logger.info('Checkout started', { runId });
883
+ logger.verbose('Preparing payment context', { runId });
884
+
885
+ try {
886
+ // ...
887
+ throw new Error('Missing connections configuration');
888
+ } catch (error) {
889
+ // Direct troubleshooting call
890
+ const { scope, reports } = await logger.troubleshootError(error, {
891
+ config: { /* app config */ },
892
+ query: { requestId: req.id },
893
+ operation: 'checkout'
894
+ }, {
895
+ formats: ['markdown'],
896
+ generateScopeStory: true,
897
+ storyOptions: {
898
+ maxEntries: 50,
899
+ includeEntryHeader: true,
900
+ recordOptions: {
901
+ label: 'Log Entry',
902
+ maxFieldStringLength: 200,
903
+ excludeKeys: ['password', 'token']
904
+ }
905
+ }
906
+ });
907
+
908
+ // Scope includes full data + story
909
+ console.log(scope?.view.summary);
910
+ console.log(scope?.story?.text);
911
+
912
+ // Reports contain troubleshooting text
913
+ return {
914
+ ok: false,
915
+ troubleshooting: reports.map(r => r.rendered)
916
+ };
917
+ }
918
+ }
919
+ ```
920
+
921
+ ### 2) Manual scoping by text ("include any log that mentions X")
922
+
923
+ ```ts
924
+ // Scope all logs in the last 5 minutes that mention "timeout" anywhere:
925
+ const timeoutScope = await logger.scopeLogs({
926
+ levelAtLeast: 'debug',
927
+ fromTimestamp: new Date(Date.now() - 5 * 60 * 1000).toISOString(),
928
+ textIncludesAny: 'timeout' // message or data, case-insensitive
929
+ }, {
930
+ mode: 'both',
931
+ storyOptions: {
932
+ includeEntryHeader: true,
933
+ recordOptions: { label: 'Timeout Log' }
934
+ }
935
+ });
936
+
937
+ console.log(timeoutScope.view.summary);
938
+ console.log(timeoutScope.story?.text);
939
+ ```
940
+
941
+ ### 3) Web Application
942
+
943
+ ```ts
944
+ // src/logger.ts
945
+ import { createLogger, LoggingConfig, LogsGateway } from 'logs-gateway';
946
+
947
+ export function createAppLogger(config?: LoggingConfig): LogsGateway {
948
+ return createLogger(
949
+ { packageName: 'WEB_APP', envPrefix: 'WEB_APP', debugNamespace: 'web-app' },
950
+ config
951
+ );
952
+ }
953
+
954
+ // src/index.ts
955
+ const logger = createAppLogger({ logFormat: 'json' });
956
+ logger.info('Web application initialized', { version: '1.0.0' });
957
+
958
+ async function handleRequest(request: any) {
959
+ logger.debug('Handling request', { requestId: request.id });
960
+ // ...
961
+ logger.info('Request processed', { responseTime: 42, runId: request.runId });
962
+ }
963
+ ```
964
+
965
+ ### 4) Shadow Logging (per-run capture)
966
+
967
+ Capture everything for a **specific `runId`** to a side file (forced-verbose), then fetch it — perfect for tests.
968
+
969
+ ```ts
970
+ const logger = createLogger(
971
+ { packageName: 'API', envPrefix: 'API' },
972
+ { shadow: { enabled: true, format: 'yaml', ttlMs: 86_400_000 } }
973
+ );
974
+
975
+ const runId = `test-${Date.now()}`;
976
+
977
+ // Turn on capture for this run (can be mid-execution)
978
+ logger.shadow.enable(runId);
979
+
980
+ logger.info('test start', { runId });
981
+ logger.verbose('deep details', { runId, step: 1 });
982
+ logger.debug('more details', { runId, step: 2 });
983
+
984
+ // ... run test ...
985
+
986
+ // Export & stop capturing
987
+ await logger.shadow.export(runId, './artifacts'); // returns exported path
988
+ logger.shadow.disable(runId);
989
+ ```
990
+
991
+ > Shadow files are stored per-run (JSONL or YAML multi-doc), rotated by size/age, and removed by TTL (default **1 day**).
992
+ > If `forceVerbose=true`, all levels for that `runId` are captured even when global level is higher.
993
+
994
+ ---
995
+
996
+ ## Correlation & Trails
997
+
998
+ Attach correlation fields freely on each call:
999
+
1000
+ * **Job/Correlation:** `runId`, `jobId`, `correlationId`, `sessionId`
1001
+ * **Depth / Operation trail:** `operationId`, `parentOperationId`, `operationName`, `operationPath`, `operationStep`
1002
+ * **Thread / Causal trail:** `threadId`, `eventId`, `causationId`, `sequenceNo`, `partitionKey`, `attempt`, `shardId`, `workerId`
1003
+ * **Tracing:** `traceId`, `spanId` (if OpenTelemetry context is active)
1004
+
1005
+ Every log entry automatically merges the current trail context. The library provides optional adapters for HTTP and queue systems to **inject/extract** headers across process boundaries.
1006
+
1007
+ ---
1008
+
1009
+ ## Routing Metadata & Recursion Safety
1010
+
1011
+ ```ts
1012
+ interface RoutingMeta {
1013
+ allowedOutputs?: string[]; // e.g. ['unified-logger','console']
1014
+ blockOutputs?: string[]; // e.g. ['unified-logger','file','shadow','troubleshooting']
1015
+ reason?: string;
1016
+ tags?: string[];
1017
+ }
1018
+ ```
1019
+
1020
+ * Gateway internal logs (`source: 'logs-gateway-internal'`) never reach unified-logger.
1021
+ * `_routing` lets you allow/block specific outputs per entry.
1022
+ * Shadow Logging honors `_routing.blockOutputs` by default (configurable).
1023
+ * Troubleshooting output can be blocked via `_routing.blockOutputs: ['troubleshooting']`.
1024
+
1025
+ ---
1026
+
1027
+ ## Log Formats
1028
+
1029
+ **Text**
1030
+
1031
+ ```
1032
+ [2025-01-15T10:30:45.123Z] [MY_APP] [INFO] Application initialized {"version":"1.0.0"}
1033
+ ```
1034
+
1035
+ **JSON**
1036
+
1037
+ ```json
1038
+ {
1039
+ "timestamp": "2025-01-15T10:30:45.123Z",
1040
+ "package": "MY_APP",
1041
+ "level": "INFO",
1042
+ "message": "Application initialized",
1043
+ "source": "application",
1044
+ "data": {"version": "1.0.0"}
1045
+ }
1046
+ ```
1047
+
1048
+ **YAML** (console/file; unified stays JSON)
1049
+
1050
+ ```yaml
1051
+ ---
1052
+ timestamp: 2025-01-15T10:30:45.123Z
1053
+ package: MY_APP
1054
+ level: INFO
1055
+ message: Application initialized
1056
+ source: application
1057
+ appName: my-awesome-app
1058
+ appVersion: "2.1.0"
1059
+ data:
1060
+ version: "1.0.0"
1061
+ ```
1062
+
1063
+ > **Note:** YAML is human-friendly but slower; prefer JSON for ingestion.
1064
+
1065
+ ---
1066
+
1067
+ ## PII/Credentials Sanitization (opt-in)
1068
+
1069
+ Enable to auto-detect and mask common sensitive data (JWTs, API keys, passwords, emails, credit cards, cloud keys, etc.).
1070
+ Supports key-based rules (denylist/allowlist), hashing specific fields, depth/size/time guardrails, and a truncation flag.
1071
+
1072
+ ```ts
1073
+ const logger = createLogger(
1074
+ { packageName: 'MY_APP', envPrefix: 'MY_APP' },
1075
+ {
1076
+ sanitization: {
1077
+ enabled: true,
1078
+ maskWith: '[REDACTED]',
1079
+ keysDenylist: ['authorization','password','secret','api_key'],
1080
+ fieldsHashInsteadOfMask: ['userId'],
1081
+ detectJWTs: true
1082
+ }
1083
+ }
1084
+ );
1085
+ ```
1086
+
1087
+ ---
1088
+
1089
+ ## Shadow Logging (Per-Run Debug Capture)
1090
+
1091
+ Shadow Logging allows you to capture **all logs for a specific `runId`** to a separate file in JSON or YAML format, with **raw (unsanitized) data**, regardless of the global log level. This is ideal for debugging tests, CI runs, or specific production workflows.
1092
+
1093
+ ### Key Features
1094
+
1095
+ - **Per-runId capture**: Enable capture for specific run identifiers
1096
+ - **Verbose by default**: Captures all log levels for the target runId
1097
+ - **Raw data**: Bypasses sanitization to preserve original values (âš ī¸ use with caution)
1098
+ - **Optional rolling buffer**: Capture logs from before shadow was enabled ("after-the-fact")
1099
+ - **TTL cleanup**: Automatically manage storage with time-to-live expiration
1100
+ - **No interference**: Shadow writes are async and won't affect primary logging
1101
+
1102
+ ### Configuration
1103
+
1104
+ #### Constructor Options
1105
+
1106
+ ```typescript
1107
+ interface ShadowConfig {
1108
+ enabled?: boolean; // default: false
1109
+ format?: 'json' | 'yaml'; // default: 'json'
1110
+ directory?: string; // default: './logs/shadow'
1111
+ ttlMs?: number; // default: 86400000 (1 day)
1112
+ respectRoutingBlocks?: boolean; // default: true
1113
+ rollingBuffer?: {
1114
+ maxEntries?: number; // default: 0 (disabled)
1115
+ maxAgeMs?: number; // default: 0 (disabled)
1116
+ };
1117
+ }
1118
+ ```
1119
+
1120
+ ### Runtime API
1121
+
1122
+ #### `logger.shadow.enable(runId, opts?)`
1123
+
1124
+ Enable shadow capture for a specific runId. Optionally override config for this run.
1125
+
1126
+ ```typescript
1127
+ logger.shadow.enable('test-run-123');
1128
+
1129
+ // Or with custom options
1130
+ logger.shadow.enable('test-run-yaml', {
1131
+ format: 'yaml',
1132
+ ttlMs: 3600000 // 1 hour
1133
+ });
1134
+ ```
1135
+
1136
+ #### `logger.shadow.disable(runId)`
1137
+
1138
+ Stop capturing logs for a runId and finalize the shadow file.
1139
+
1140
+ #### `logger.shadow.isEnabled(runId)`
1141
+
1142
+ Check if shadow capture is active for a runId.
1143
+
1144
+ #### `logger.shadow.listActive()`
1145
+
1146
+ List all currently active shadow captures.
1147
+
1148
+ #### `logger.shadow.export(runId, outPath?)`
1149
+
1150
+ Copy the shadow file to a destination path.
1151
+
1152
+ #### `logger.shadow.cleanupExpired(now?)`
1153
+
1154
+ Delete expired shadow files based on TTL. Returns number of deleted runs.
1155
+
1156
+ ### Important Considerations
1157
+
1158
+ âš ī¸ **Security**: Shadow captures **raw, unsanitized data**. This means passwords, API keys, and PII will be preserved in shadow files. Only use shadow logging in secure environments (development, CI, isolated test systems).
1159
+
1160
+ 💾 **Storage**: Shadow files can grow quickly when capturing verbose logs. Configure appropriate `ttlMs` values and regularly run `cleanupExpired()`.
1161
+
1162
+ đŸšĢ **Default OFF**: Shadow logging is disabled by default and must be explicitly enabled via config or environment variables.
1163
+
1164
+ ---
1165
+
1166
+ ## Troubleshooting Integration
1167
+
1168
+ The troubleshooting integration uses `nx-troubleshooting` to match errors to solutions. See the [nx-troubleshooting documentation](https://www.npmjs.com/package/nx-troubleshooting) for details on creating troubleshooting narratives.
1169
+
1170
+ ### Key Features
1171
+
1172
+ - **Intelligent Error Matching** – Matches errors to solutions using multiple strategies
1173
+ - **Flexible Probe System** – Built-in probes plus extensible custom probes
1174
+ - **Template Variables** – Dynamic solution messages with `{{variable}}` syntax
1175
+ - **Multiple Output Formats** – Markdown, JSON, and plain text formatting
1176
+ - **Automatic Scoping** – Errors automatically trigger scoped log collection
1177
+
1178
+ ### Example Troubleshooting Narrative
1179
+
1180
+ ```json
1181
+ {
1182
+ "id": "missing-connections-config",
1183
+ "title": "Missing Connections Configuration",
1184
+ "description": "The application config is missing the required 'connections' object...",
1185
+ "symptoms": [
1186
+ {
1187
+ "probe": "config-check",
1188
+ "params": { "field": "connections" },
1189
+ "condition": "result.exists == false"
1190
+ }
1191
+ ],
1192
+ "solution": [
1193
+ {
1194
+ "type": "code",
1195
+ "message": "Add a 'connections' object to your config:",
1196
+ "code": "{\n \"connections\": { ... }\n}"
1197
+ }
1198
+ ]
1199
+ }
1200
+ ```
1201
+
1202
+ ---
1203
+
1204
+ ## Backwards Compatibility
1205
+
1206
+ * All new behavior is **opt-in**:
1207
+ * `scoping.enabled` and `troubleshooting.enabled` default to `false`.
1208
+ * If `nx-troubleshooting` is not installed:
1209
+ * `troubleshooting.enabled` must remain `false` or initialization fails clearly.
1210
+ * If neither shadow nor buffer is configured:
1211
+ * `scopeLogs` can still filter **currently available** logs if in-memory buffer is enabled; otherwise, scopes may be empty.
1212
+ * `scopeRecord` is pure and can be used independently anywhere.
1213
+
1214
+ ---
1215
+
1216
+ ## Environment Variables (summary)
1217
+
1218
+ | Key | Description | Default |
1219
+ | ---------------------- | -------------------------------------- | --------- |
1220
+ | `{P}_LOG_TO_CONSOLE` | Enable console output | `true` |
1221
+ | `{P}_LOG_TO_FILE` | Enable file output | `false` |
1222
+ | `{P}_LOG_FILE` | Log file path | — |
1223
+ | `{P}_LOG_TO_UNIFIED` | Enable unified-logger | `false` |
1224
+ | `{P}_LOGS_LEVEL` | Per-package threshold (canonical); omit both this and `{P}_LOG_LEVEL` → **`warn`** | `warn` |
1225
+ | `{P}_LOG_LEVEL` | Legacy level — used only if `{P}_LOGS_LEVEL` is not set in the environment | — |
1226
+ | `{P}_LOG_FORMAT` | `text\|json\|yaml\|table` | `table` |
1227
+ | `{P}_SHOW_FULL_TIMESTAMP` | Show full ISO timestamp in console | `false` |
1228
+ | `{P}_CONSOLE_PACKAGES_SHOW` | Comma-separated packages to show (console only) | (show all) |
1229
+ | `{P}_CONSOLE_PACKAGES_HIDE` | Comma-separated packages to hide (console only) | (show all) |
1230
+ | `DEBUG` | Namespace(s) enabling verbose+debug | — |
1231
+ | `{P}_SANITIZE_ENABLED` | Turn on sanitization | `false` |
1232
+ | `{P}_TRACE_OTEL` | Attach `traceId`/`spanId` if available | `true` |
1233
+ | `{P}_TRAILS_*` | Toggle trails/header adapters | see above |
1234
+ | `{P}_SHADOW_*` | Shadow Logging controls | see above |
1235
+ | `{P}_SCOPING_*` | Scoping controls | see above |
1236
+ | `{P}_TROUBLESHOOTING_*` | Troubleshooting controls | see above |
1237
+
1238
+ ---
1239
+
1240
+ ## Development
1241
+
1242
+ ```bash
1243
+ npm install
1244
+ npm run build
1245
+ npm run dev
1246
+ npm run clean
1247
+ ```
1248
+
1249
+ ## License
1250
+
1251
+ MIT
1252
+
1253
+ ---
1254
+
1255
+ **Tip for tests:** set a `runId` at the beginning of each test (e.g., `runId: 'ci-<suite>-<timestamp>'`), enable **Shadow Logging**, run, then `export()` the captured logs for artifacts/triage.