@bymax-one/nest-logger 1.1.0 → 1.2.0

package/CHANGELOG.md

@@ -11,6 +11,313 @@ heading here.
 
 ## [Unreleased]
 
+## [1.2.0] - 2026-08-13
+
+Remediation of the P0 findings from the observability audit
+([`docs/observability_audit.md`](./docs/observability_audit.md)): a credential leak in the
+default redaction set, a correlation field silently dropped from every structured entry, a
+published type that never matched the runtime, and reserved log keys that were documented as
+signals but never written. The redaction engine was replaced in the process, which made the
+shipped logging path **~50× faster**.
+
+A second pass moved the HTTP access log from an interceptor to middleware, because an interceptor
+cannot see a request a guard rejected: 401, 403, 429 and 404 produced no log line at all.
+
+### Security
+
+- **Guard rejections and unmatched routes are logged.** NestJS runs
+  middleware → guards → interceptors → handler, so the interceptor-based access log never observed
+  a request rejected by a guard or one that matched no route. Measured against a real backend over
+  20 minutes of traffic: lines existed for 200/201/400, and **zero** for 401 — brute force,
+  credential stuffing and route enumeration were invisible, and invisible without a `requestId` to
+  correlate them by. `HttpAccessLogMiddleware` now records the access log before guards run, so
+  every request produces `HTTP_REQUEST_START` and a terminal entry carrying `ip`, `userAgent` and
+  the correlation context.
+- **The correlation scope no longer skips the prefixed root route.** `applyRequestIdMiddleware`
+  defaulted to the route pattern `'*'`, and under NestJS 11 (Express 5 / path-to-regexp v8) a
+  wildcard is a named parameter with segment-count semantics. Measured: both `'*'` and `'{*splat}'`
+  stop matching once the app calls `setGlobalPrefix`, so `GET /api` silently had no correlation
+  scope and, with the change above, no access log either. The default is now `'/'`, which MOUNTS at
+  the root rather than matching a pattern. The failure mode was absence — no error, just a request
+  that never appeared.
+
+### Fixed
+
+- **An aborted request is no longer logged as a success.** Destroying the socket does not cancel
+  the handler: it runs to completion, writes to a dead connection, the observable completes, and
+  the interceptor reported the `200` it intended. Delivery is now reported on its own axis —
+  `HTTP_REQUEST_ABORTED`, emitted from the response's `'close'` event when `writableFinished` is
+  false — while the status stays whatever the server produced. Inventing one was rejected: nginx's
+  `499` is not an HTTP status (IANA leaves 452–499 unassigned), so recording it would assert a code
+  the protocol has no name for and break any consumer grouping by class. The limit is stated in the
+  README: a fast handler whose client hangs up after the bytes were flushed is still reported as
+  the success it was, because the server cannot know whether the peer read them.
+
+### Breaking
+
+- **`HttpAccessLogMiddleware` must be wired for the new coverage.** It is registered by
+  `applyRequestIdMiddleware(consumer)`, which most consumers already call — no code change needed
+  for them. A consumer who never wired that helper keeps the previous interceptor-based behaviour
+  (including its blind spot) rather than losing HTTP logs. **Migration:** call
+  `applyRequestIdMiddleware(consumer)` in your module's `configure()` hook.
+- **The default middleware route changed from `'*'` to `'/'`.** A consumer passing an explicit
+  route is unaffected. One relying on the default now also gets the prefixed root route, which is
+  the fix. **Migration:** none; if you deliberately excluded the root, pass an explicit route.
+- **`LoggableResponse` gained `writableFinished` and `on('close', …)`.** An Express response
+  satisfies both already. **Migration:** only a consumer who hand-implements the interface — for a
+  test double, say — needs to add them.
+- **`HTTP_REQUEST_START` no longer carries `userId`.** The entry is now emitted before guards run,
+  which is the whole point of the change — and authentication runs in a guard, so at that moment
+  there is no principal to read. The previous interceptor-based START ran after guards and did
+  include it. The acting user is still logged: it is read at the terminal entry
+  (`HTTP_REQUEST_SUCCESS` / `_REDIRECT` / `_CLIENT_ERROR` / `_SERVER_ERROR` / `_ABORTED`), where
+  the guard has populated it. **Migration:** query the terminal entry for `userId`, not START. A
+  dashboard joining on START's `userId` should join on `requestId` instead, which both entries
+  carry.
+- **The logged URL now includes the app's global prefix.** The middleware is mounted, and a mounted
+  middleware sees `url` relative to its mount point, so the entry is built from `originalUrl`.
+  Under `setGlobalPrefix('api')` a request for `/api/users/7` used to be logged by the interceptor
+  as `/api/users/:id`; that is preserved. **Migration:** none — this keeps the previous value. An
+  `excludePaths` pattern is likewise matched against the full path, as before.
+- **New reserved log key `HTTP_REQUEST_ABORTED`.** A consumer asserting an exhaustive list of
+  `RESERVED_LOG_KEYS` must add it. **Migration:** add the key to the assertion.
+
+### Security
+
+- **Credential-bearing HTTP headers are redacted by name.** `authorization`, `cookie`,
+  `set-cookie`, `x-api-key` and `x-auth-token` were covered ONLY by the absolute paths
+  `req.headers.*` / `res.headers.*`. A headers bag logged under any other key — for example
+  `logger.info(key, msg, userId, { headers: req.headers })` — wrote the bearer token in clear.
+  They are now first-class field names, caught wherever they appear.
+- **The four-level nesting cap is gone, and what replaces it fails closed.** The previous wildcard
+  expansion reached `*.*.*.*.field` and anything deeper was emitted in clear. The walk now reaches
+  100 levels, and past that a CONTAINER is DROPPED rather than passed through — a traversal ceiling
+  exists only so a pathological self-similar structure cannot exhaust the call stack, and it can
+  never become a leak the way the old one was. A primitive at the boundary is still emitted, and
+  that is not a gap: its key was matched by the parent at level 100, the last container walked.
+- **Prototype-polluting metadata keys are dropped instead of silently swapping the copy.** A log
+  call's `metadata` was copied with `Reflect.set`, which does NOT create an own property for
+  `__proto__` — the write walks the prototype chain, finds `Object.prototype`'s inherited setter
+  and invokes it. An own `__proto__` (what `JSON.parse` of an untrusted body produces) therefore
+  vanished from the entry AND swapped the copy's prototype for the caller's value. `__proto__`,
+  `constructor` and `prototype` are now dropped, from the same constant the ALS context path
+  already enforced, so the two guards cannot drift. `Object.prototype` itself was never reachable,
+  which is what kept this a correctness bug rather than a pollution vulnerability.
+- **An accessor `toJSON` can no longer smuggle a value through the binary fast path.** That path
+  hands back the ORIGINAL reference, and `JSON.stringify` reads `toJSON` again from it — so a
+  GETTER returning `Buffer.prototype.toJSON` to the walk took the fast path, and the stringifier
+  then read a factory synthesizing `{ password }`, in clear, past both hooks. Identity proves
+  nothing about the next read unless the property is a data property, so an accessor no longer
+  selects any path that returns a caller-controlled reference. `toJSON` is now resolved along the
+  prototype chain exactly ONCE per value, where the walk used to read it twice.
+- **A `toJSON` chain can no longer exhaust the stack.** Following the method's output did not
+  advance the depth counter, so a chain where each `toJSON()` returns a fresh object carrying
+  another one recursed forever — nothing repeats, so the ancestor set never matched, and the
+  ceiling was never reached. The root catch contained it as the fail-closed envelope rather than a
+  crash, but it cost the whole record; the ceiling costs only the pathological value.
+- **A `toJSON` can no longer rename a field around the matcher.** Only the method's OUTPUT was
+  inspected, so `{ password, toJSON: () => ({ value: this.password }) }` emitted the secret under
+  `value` — a name nobody declared sensitive. When the SOURCE carries a sensitive own key the
+  method is no longer trusted and the whole value is censored. This over-redacts an object that
+  holds a sensitive key AND correctly omits it, deliberately: the alternative — invoking `toJSON`
+  against a sanitized copy — throws on every method that reads an internal slot instead of an own
+  property, which is `Date`, `Decimal` and Luxon.
+- **A terminal array index in `redactPaths` no longer covers nothing.** `tokens[0]` fed the leaf
+  `0` to the walk, which compares NAMES and never array positions — so the element stayed raw
+  through the size-bounded `_preview`, while any object key literally named `0` was censored
+  instead. An unquoted numeric segment is now read as an index and skipped like a wildcard, so
+  `tokens[0]` covers `tokens`: broader than the path, in the safe direction. The quoted form
+  `["0"]` stays a name.
+- **Base bindings are redacted.** `service` is consumer-supplied and `applyDefaults` keeps whatever
+  it was handed, so a `{ name, version, apiKey }` reached the sink in clear once base stopped going
+  through the path expansion that had covered it via `*.*.apiKey`. Redacted at
+  `formatters.bindings`, which runs once at logger construction — no per-entry cost — and which
+  preserves a consumer's extra non-sensitive metadata rather than trimming base to the two declared
+  fields.
+- **Child-logger bindings are redacted.** `PinoLoggerService.child(bindings)` accepts any record,
+  and Pino pre-serializes child bindings into the instance's `chindings` fragment before any
+  formatter runs — so no factory hook can reach them. `logger.child({ password })` stamped the
+  value in clear on every entry that child emitted. Redaction is applied in `child()` itself.
+- **An `Error` is redacted wherever it is logged, not only under a key with a serializer.** The
+  walk skipped `Error` instances to preserve the instance Pino's `err` serializer keys off, and
+  the compensating serializer hook only fires for keys that actually have one — so
+  `{ failure: err }` carrying an `apiKey` reached the sink in clear. Errors are now cloned through
+  their prototype and descriptors, which censors the enumerable properties while keeping
+  `instanceof`, `message` and `stack` intact.
+- **Sensitive field names are matched case-insensitively.** HTTP header names are
+  case-insensitive by spec, and only INBOUND Node headers arrive lower-cased — a hand-built or
+  outbound bag carrying `Authorization`, `Cookie` or `X-API-Key` was left in clear while the
+  documentation claimed header coverage. Matching now lower-cases the key, which also covers
+  `Password` / `Email` and errs toward redacting.
+- **An array with a custom `toJSON()` is redacted.** `JSON.stringify` gives the method precedence
+  over array serialization, so an array whose `toJSON()` synthesized a secret was walked as an
+  ordinary array — finding nothing in its elements — and then emitted the secret. The `toJSON`
+  branch now runs before the array branch.
+- **A secret synthesized by `toJSON()` is redacted.** A value with `toJSON` decides its own
+  serialized form, so the walk cannot inspect its own properties — but skipping it let
+  `{ toJSON: () => ({ accessToken }) }` emit the token untouched. The walk now redacts the
+  method's output, substituting it only when something was actually censored so a clean `Date` or
+  `Decimal` is still passed through by reference.
+- **Binary values are fast-pathed by IDENTITY, not by shape.** `ArrayBuffer.isView` was too wide:
+  it also matched extended views, and three shapes went straight through both hooks — an own
+  `toJSON` synthesizing a payload, and an enumerable property on a `Uint8Array` or `DataView`,
+  neither of which has a `toJSON` to hide it. Narrowing it to "a view with no OWN `toJSON`" was
+  still too wide, because a SUBCLASS can define one on its own prototype. The check is now an
+  identity comparison against `Buffer.prototype.toJSON` itself, the one function known to produce a
+  `{ type, data }` output that cannot carry a caller's key — which keeps a real binary payload from
+  being recursed over byte by byte, and inspects everything else.
+- **Arrays are indexed numerically rather than iterated.** `for...of` runs the array's
+  `Symbol.iterator`, which a caller can override, while `JSON.stringify` reads `length` and
+  numeric indices. An iterator that never returned `done` hung the log call — a loop that does not
+  end cannot be caught by the never-throw guard — and one yielding values unrelated to the indices
+  made the walk inspect something the array does not hold. Reading the way the serializer reads
+  also makes holes render identically to the native output.
+- **A callable carrying `toJSON` is inspected.** `JSON.stringify` applies a callable `toJSON` to a
+  FUNCTION object too — the spec runs that step BEFORE the "callable serializes to undefined" rule
+  — so `Object.assign(() => {}, { toJSON: () => ({ accessToken }) })` emitted its payload with
+  nothing having inspected it. An ordinary function is still handed back untouched, so it keeps
+  being omitted from the output rather than becoming `{}`.
+- **The root of a log record never honours `toJSON`.** Pino ITERATES that object rather than
+  serializing it, so honouring the method replaced the whole record with its return value:
+  `logger.info(key, msg, userId, { toJSON: () => 'x' })` emitted `{"0":"x"}` and lost `logKey`,
+  `userId` and every other field. Nested values and serializer outputs do reach `JSON.stringify`
+  and keep it. The same applies to `child()` bindings, which Pino also iterates.
+- **An object that reports no keys is snapshot, not passed through.** The empty-key fast path
+  returned the original reference, which a Proxy exploits: answer `[]` to the walk's
+  `Object.keys`, then expose an enumerable secret when Pino serializes the reference.
+- **A censor that cannot be written fails closed.** `Reflect.defineProperty` reports failure by
+  returning `false` rather than throwing, so an `Error` carrying a non-configurable enumerable
+  secret kept its raw value while the redaction silently no-opped. A failed write is now a
+  traversal failure.
+- **A stateful `toJSON()` or getter cannot differ between inspection and serialization.** The walk
+  probed `toJSON()` once and, when the result was clean, returned the original object — leaving
+  `JSON.stringify` to call the method a second time (with the property key, which the probe does
+  not pass). The inspected result is now what gets serialized, and the same applies to accessors.
+- **Hostile metadata and hostile errors no longer crash the caller.** The never-throw guarantee
+  now starts at the FIRST read of caller-controlled data: `Object.keys` fires a Proxy's `ownKeys`
+  trap and `Reflect.get` fires a getter, both before anything reached the redaction pipeline, so
+  `logger.info(key, msg, undefined, hostileMetadata)` threw outright. Unreadable metadata is
+  dropped whole and marked with the redaction-failure envelope while the entry keeps its real
+  `logKey`, message and correlation ids. The same guard covers the error path — `name`, `message`
+  and `stack` are ordinary properties a caller can redefine as throwing accessors, and `message`
+  is read outside the serializer.
+- **A record whose getter throws is dropped WHOLE, not up to the failing property.** The mixin
+  merge used `Object.assign` on the mixin's own object, which copies key by key — so everything
+  read before the hostile getter was already written into it, and the failure path emitted that
+  prefix while claiming to drop the record. Merging into a disposable target keeps the partial
+  writes in the value that is discarded.
+- **A throwing getter no longer crashes the log call.** Pino merges the mixin result with the
+  caller's object before `formatters.log` runs, and the default strategy's `Object.assign` invokes
+  every own getter — so a hostile getter threw before the redactor's fail-closed envelope could
+  apply. The factory now owns the merge, and the never-throw guarantee holds through the real
+  pipeline.
+- **Bootstrap entries are emitted after destination initialization.** They were written from an
+  eagerly instantiated provider factory, which runs before `DestinationRegistry.onModuleInit()` —
+  so a sink that only accepts writes once its own `onInit()` has run could drop them, including
+  `LOGGER_BOOTSTRAP_WARNING`. They now come from the registry that owns that initialization, which
+  makes the ordering structural. Note for test authors: `Test.createTestingModule().compile()`
+  builds the graph, `init()` runs the lifecycle hooks — the bootstrap entries appear after `init()`.
+- **`LOGGER_BOOTSTRAP_WARNING` is emitted when `shouldDisableDefaultRedact` is on.** The README
+  has always described this entry as the audit trail proving when PII protection was
+  intentionally reduced. It was never written, so a deployment running without redaction was
+  indistinguishable from a protected one.
+
+### Changed
+
+- **Default redaction is now a single name-based recursive walk** instead of 140 compiled
+  `fast-redact` paths. A value is censored when its key name is in `REDACT_COMMON_FIELDS`, at any
+  depth, in one snapshotting traversal that mutates nothing of the caller's and reads every value
+  exactly once.
+  Circular references collapse to `[Circular]`; a record that cannot be walked
+  degrades to a marked, data-free envelope rather than being emitted unredacted.
+
+  Measured on the full production path (`forRoot({ service })`, no other option set):
+  **9,311 → ~274,000 logs/s**, ~107 µs → ~3.6 µs per entry. Every value is read exactly once and
+  pinned into a fresh structure, so what reaches the sink is guaranteed to be what was inspected —
+  an earlier copy-on-write draft returned clean subtrees by reference and let `JSON.stringify`
+  re-evaluate their accessors, which a stateful getter can answer differently.
+
+- **`redactPaths` is unchanged** — consumer paths are still `fast-redact` paths, applied on top of
+  the default coverage. `fast-redact` is now configured only when there are consumer paths to
+  apply.
+- **Bundle-size budget raised** 13.5 → 16.0 KiB brotli for the server subpath, and the benchmark's
+  throughput floor raised 0.004 → 0.20; the old floor was calibrated to the wildcard engine and
+  could no longer fail on anything short of a 100× regression.
+
+### Added
+
+- **`redactStrategy: 'names' | 'paths'`** (default `'names'`). `'paths'` restores the pre-1.2
+  engine for a consumer depending on exact `fast-redact` path semantics — with its four-level
+  ceiling and its cost. Expect it to be removed in a future major.
+- **`PINO_LEVEL_NUMBERS` / `PINO_LEVEL_NAMES` are exported** from the server subpath, so a
+  destination writing a numeric level column can convert the string label without hard-coding the
+  mapping.
+- **`RESERVED_LOG_KEYS.LOGGER_REDACTION_FAILED`** — the marker on the envelope substituted for a
+  record whose traversal threw.
+- **`RESERVED_LOG_KEYS_NOT_EMITTED`** — the keys that are reserved but intentionally never
+  written, each with its reason. A unit test now asserts that every other declared key has a
+  writer in production source, so a key can no longer be declared, documented as a signal, and
+  silently never emitted.
+
+### Fixed
+
+- **AsyncLocalStorage context reaches structured entries.** `emitStructured` / `errorStructured` /
+  the NestJS-variadic path wrote `userId` and `context` as own properties even when they were
+  `undefined`, and Pino's default `mixinMergeStrategy` (`Object.assign(mixinResult, mergeObject)`)
+  let that `undefined` overwrite the value the trace mixin had just read from the ALS store. The
+  key then vanished during serialization, so `logContext.set('userId', …)` — the documented way to
+  attach the authenticated user once per request — never reached a log unless every call site
+  repeated it. `requestId` and `tenantId` only ever survived because those names are not written
+  by this class. Precedence is now explicit argument > ALS store > field absent.
+- **A caller's `metadata` can never occupy `logKey`, `userId` or `context`.** The invariant was
+  previously enforced by overwriting those fields after the spread — the same unconditional write
+  that clobbered the ALS context. Now that they are written only when defined, the owned names are
+  stripped from `metadata` on the way in, so a metadata bag cannot forge the acting user that the
+  mixin read from the authenticated request scope.
+- **`RESERVED_LOG_KEYS_NOT_EMITTED` is exported** from both subpaths, so the `{@link}` in
+  `RESERVED_LOG_KEYS`'s documentation resolves in the published declarations.
+- **`LOGGER_SHUTDOWN_OK` is emitted** at the start of `onApplicationShutdown`, before the
+  destinations are torn down, with an event-loop barrier so an async sink's write is not raced by
+  its own teardown (the authoritative contract remains `ILogDestination.onShutdown`, which MUST
+  flush pending writes). It is the bookend to `LOGGER_BOOTSTRAP_OK`: its absence in a log
+  stream is how an operator tells a graceful shutdown from a killed process.
+- **A consumer path's leaf name reaches the walk, so it cannot leak through a truncation preview.**
+  Consumer `redactPaths` are applied by Pino's stringifier, which runs AFTER the per-field size
+  bound — so a field covered only by a path was still raw when the 200-character `_preview` was
+  built. The leaf name is now matched by the walk as well, which closes that and covers the same
+  name on every other surface the walk reaches. Deliberately broader than the path itself:
+  `redactPaths: ['user.ssn']` censors `ssn` wherever it appears.
+- **An oversized field's truncation `_preview` is redacted.** Redaction now runs before the size
+  bound, so the 200-character preview of a truncated value carries `[REDACTED]` instead of the
+  head of a secret.
+
+### Documentation
+
+- **`README.md` API reference corrected.** The table described `warn` / `debug` / `error` / `fatal`
+  as structured methods taking a log key — they are the NestJS variadic bridge — documented a
+  `fatalStructured` that does not exist, labelled the structured API's third parameter `context`
+  when it is `userId`, and described `@LogContext` as a method decorator that opens a
+  `logContext.run()` scope when it is a class decorator that only records metadata.
+- **The Loki and Prisma destination examples now run.** The Loki example called
+  `BigInt(entry.time)` on an ISO 8601 string, which throws; the Prisma example wrote the string
+  level label into a numeric column. Both conversions are now correct and are exercised by
+  `test/e2e/log-entry-contract.e2e-spec.ts`.
+- **The architecture diagram no longer inverts the pipeline** — `RequestIdMiddleware` runs before
+  `HttpLoggingInterceptor`, which is why the interceptor's entries carry a `requestId` at all.
+
+### Breaking
+
+- **`LogEntry.level` is `LogLevel` (a string) and `LogEntry.time` is `string`.** They were declared
+  `number` and `string | number`; the runtime has always emitted the Pino string label and an ISO
+  8601 string. This is a **type-level** break only — code relying on the old declaration was
+  already failing at runtime — but it can newly fail to compile. Convert with the now-exported
+  `PINO_LEVEL_NUMBERS` and `Date.parse`.
+
+  Shipped as a minor rather than a major deliberately: the library has no consumers yet, and
+  SemVer's major exists to protect the consumers a break would reach. Breaks are documented here
+  under this heading with their migration path, which is what carries the signal while the version
+  number does not.
+
 ## [1.1.0] - 2026-08-11
 
 Coordinated ecosystem release aligning every `@bymax-one/*` package after the ioredis 6 /
@@ -350,7 +657,8 @@ published `dist/` is identical — no runtime behaviour changes for consumers.
 - Professional CI suite: `ci.yml`, `bench.yml`, `codeql.yml`, `scorecard.yml`,
   `release.yml`, Dependabot, and issue templates
 
-[Unreleased]: https://github.com/bymaxone/nest-logger/compare/v1.1.0...HEAD
+[Unreleased]: https://github.com/bymaxone/nest-logger/compare/v1.2.0...HEAD
+[1.2.0]: https://github.com/bymaxone/nest-logger/compare/v1.1.0...v1.2.0
 [1.1.0]: https://github.com/bymaxone/nest-logger/compare/v1.0.8...v1.1.0
 [1.0.8]: https://github.com/bymaxone/nest-logger/compare/v1.0.7...v1.0.8
 [1.0.7]: https://github.com/bymaxone/nest-logger/compare/v1.0.6...v1.0.7