@svayam-opensource/svm-util-log 1.0.1 → 1.1.0

package/README.md

@@ -1,82 +1,396 @@
 # @svayam-opensource/svm-util-log
 
-Svayam's shared logging framework — the logger from `svm-util`, carrying only the
-dependencies a logger needs.
+Structured logging for Node applications and command-line tools.
 
-**Mandated by POL-423.** All Svayam-owned Node code uses this framework and leaves messages
-at the various levels, so a run can be reviewed afterwards for issue and problem resolution.
+Every line carries **where it came from**, so a log you receive from someone else can be read
+back to the code that wrote it. Debug output is **switched on from the environment**, so the
+person in front of the problem can turn it on without a rebuild. Secrets are **masked before
+anything is written**.
 
-## Why this package exists
-
-`@svayam/svm-util` is one package: the logger, the helpers, database access and the API
-server. Taking it whole installs **675 packages, 274 MB** — right for an API service, and the
-whole cost for a command-line tool that wants one log line. `@svayam-opensource/gov` is
-installed with `npm i -g` on adopters' laptops.
-
-This package is **25 packages, 8.4 MB**.
+```bash
+npm install @svayam-opensource/svm-util-log
+```
 
-It is also on the **public** scope, because `@svayam/svm-util` publishes to the internal
-registry, which adopters of a public CLI cannot resolve. A logging mandate that makes the
-governance CLI uninstallable is a mandate nothing can obey.
+Node 20+. TypeScript types included. Works from both `import` and `require`.
 
-First package of the split — see [911#224](https://github.com/Svayamtech/911-SVM-LIB-SVC/issues/224).
+---
 
-## Use
+## Quick start
 
 ```ts
 import { createApplicationLogger, LogLevel } from "@svayam-opensource/svm-util-log";
 
 const logger = createApplicationLogger({
   appId: "gov-work",
-  logDir: "~/.gov/svm/logs",
+  logDir: "/Users/me/.gov/logs",
   methods: [
-    { METHOD: "console", LOGLEVEL: LogLevel.warn },
-    { METHOD: "file",    LOGLEVEL: LogLevel.debug },
+    { METHOD: "console", LOGLEVEL: LogLevel.warn },   // on screen: warnings and errors
+    { METHOD: "file",    LOGLEVEL: LogLevel.debug },  // on disk: everything
   ],
 });
 
-logger.log(LogLevel.debug, "chose the api-key route", "gov-work:cli:sign-in-choice", "captureAgentKey");
+logger.log(LogLevel.info, "starting up", "myapp:cli:main", "start");
+logger.log(LogLevel.error, "could not reach the registry", "myapp:net:client", "fetch", 0, { status: 503 });
+
+logger.close();   // flush before the process exits
+```
+
+That is the whole setup. Everything below is detail you can come back for.
+
+---
+
+## What a line looks like
+
+```
+2026-09-09T03:41:02.240Z warn    gov-work:cli:agent-install    signIn            7    no browser on this machine — offering the key route
+2026-09-09T03:41:02.367Z debug   gov-work:cli:sign-in-choice   captureAgentKey   7    chose the api-key route meta={ user: 'rk', apiKey: '***' }
+2026-09-09T03:41:02.494Z error   gov-work:cli:agent-install    run               7    npm could not install the agent meta={ code: 127 }
+```
+
+Columns are fixed-width and the origin comes early, because the usual thing to do with a log
+is **scan one column for one module**, not read it as prose.
+
+| Column | From | What it is for |
+|---|---|---|
+| `2026-09-09T03:41:02.240Z` | automatic | ISO timestamp, UTC |
+| `warn` | `lvl` | severity — see [Levels](#levels) |
+| `gov-work:cli:agent-install` | `pgm_cd` | **the code path** — which module wrote this |
+| `signIn` | `fn_cd` | which function inside it |
+| `7` | `act_id` | ties one unit of work together across modules |
+| `no browser on this machine…` | `msg` | your message |
+| `meta={ … }` | `meta` | structured detail, masked |
+
+### Naming `pgm_cd` — the one convention worth following
+
+Use the **code path**: `<package>:<directory>:<module>`.
+
+```
+gov-work:cli:sign-in-choice     →     src/cli/sign-in-choice.ts
+myapp:net:client                →     src/net/client.ts
 ```
 
-### `pgm_cd` is the code path
+A reader who receives your log can then open the file without guessing, and — as the next
+section shows — can switch that exact module on by name.
 
-`<package>:<dir>:<module>` — `gov-work:cli:sign-in-choice` is `src/cli/sign-in-choice.ts`.
-The point is that a reader can get from a line back to the code without guessing (POL-426).
+---
 
-### Turning it on in the field
+## Turning on debug output in the field
 
-Errors and warnings always come through. Everything below them is opt-in:
+**Errors and warnings are always written.** Everything below them is off until someone asks
+for it, through the standard Node `NODE_DEBUG` variable:
 
 ```bash
-NODE_DEBUG=gov-work:cli:sign-in-choice   gov work     # one module
-NODE_DEBUG=gov-work:*                    gov work     # the package
+NODE_DEBUG=gov-work:cli:sign-in-choice   gov work    # one module
+NODE_DEBUG=gov-work:cli:*                gov work    # one directory
+NODE_DEBUG=gov-work:*                    gov work    # the whole application
 ```
 
-That is POL-425: **the person who needs the log is usually not the person who can reproduce
-the fault.** They turn it on, reproduce, and send the file.
+This matters because **the person who needs the log is usually not the person who can
+reproduce the fault**. You send them one environment variable; they reproduce it and send back
+the file. No debug build, no release, no code change.
+
+```mermaid
+flowchart TD
+    A["logger.log(level, msg, pgm_cd, …)"] --> B{"level is<br/>error or warn?"}
+    B -- yes --> D["write it"]
+    B -- no --> C{"NODE_DEBUG matches<br/>pgm_cd, fn_cd or act_id?"}
+    C -- yes --> D
+    C -- no --> E["dropped, costs nothing"]
+    D --> F{"for each configured<br/>transport"}
+    F --> G["level within<br/>this transport's LOGLEVEL?"]
+    G -- yes --> H["mask meta → format → write"]
+    G -- no --> E
+```
+
+Two gates, and they are different questions. `NODE_DEBUG` decides **whether this line exists
+at all**; a transport's `LOGLEVEL` decides **where an existing line goes**. That is how the
+quick-start config keeps the screen quiet at `warn` while the file keeps everything.
+
+---
 
-### Secrets
+## Levels
 
-POL-427 is **C01**: no credential, key, token or PII reaches a log, at any level, in any
-transport. `meta` is masked before rendering — by field name, case-insensitively, matching a
-name that *contains* a masked word (`userPassword`, `db_pwd`, `authToken`). Extend
-`DEFAULT_MASK`, never replace it.
+Ordered least to most verbose. A transport set to a level accepts that level and everything
+above it.
 
-## Transports
+| Level | Use it for |
+|---|---|
+| `error` | the operation failed — **always written** |
+| `warn` | something is wrong but the run continues — **always written** |
+| `info` | milestones a user would recognise |
+| `http` | requests and responses |
+| `verbose` | more than `info`, less than a trace |
+| `debug` | what you would want during an incident |
+| `silly` | everything |
 
-`console` and `file` (daily-rotate) ship here. `database` and `api` belong to the packages
-carrying `mysql` and `axios`, and register themselves:
+```ts
+import { withinLevel, LogLevel } from "@svayam-opensource/svm-util-log";
+
+withinLevel(LogLevel.error, LogLevel.info);   // true  — error passes an info threshold
+withinLevel(LogLevel.debug, LogLevel.info);   // false — debug does not
+```
+
+---
+
+## Keeping secrets out of logs
+
+A log is a file. It gets copied and sent to other people — precisely when something has gone
+wrong. So masking happens **in the library, before rendering**, not in your memory at each
+call site.
 
 ```ts
-registerTransport("database", (conf, config) => new MyDbTransport(conf, config));
+logger.log(LogLevel.debug, "signing in", "myapp:auth:login", "run", 0, {
+  user: "rk",
+  apiKey: "sk-live-9f3a",
+  db: { host: "localhost", password: "hunter2" },
+});
 ```
 
-A method declared in config with nothing registered for it is **ignored, not an error** — a
-line that does not go somewhere, never a crash at startup.
+```
+… signing in meta={ user: 'rk', apiKey: '***', db: { host: 'localhost', password: '***' } }
+```
+
+Matching is **by field name, case-insensitive, at any depth**, and matches names that
+*contain* a masked word — so `userPassword`, `db_pwd` and `authToken` are all covered.
+
+Covered out of the box:
+
+```
+password  passwd  pwd  secret  token  apiKey  api_key  authorization
+auth  credential  credentials  privateKey  private_key  sessionId
+session_id  cookie  jwt  bearer
+```
+
+**Extend the default, never replace it:**
+
+```ts
+import { DEFAULT_MASK } from "@svayam-opensource/svm-util-log";
+
+createApplicationLogger({
+  appId: "myapp",
+  logDir: "/var/log/myapp",
+  methods: [{ METHOD: "file", LOGLEVEL: LogLevel.info }],
+  mask: [...DEFAULT_MASK, "panNumber", "aadhaar"],
+});
+```
+
+Passing `mask: ["panNumber"]` on its own would silently switch off every default — the one
+mistake worth watching for here.
+
+> Masking protects values inside `meta`. It cannot see a secret you interpolated into the
+> message string yourself, so put values in `meta`, not in `msg`.
+
+---
+
+## Where lines go
+
+Two transports ship in this package.
+
+### `console`
+
+Writes to **stderr**, coloured by level. stderr because stdout is your program's answer — a
+caller running `myapp | jq` should not receive log lines as data.
+
+```ts
+{ METHOD: "console", LOGLEVEL: LogLevel.warn }
+```
+
+### `file`
+
+A daily-rotating file, `<date>-<appId>.log`, in `logDir`.
+
+```ts
+{
+  METHOD: "file",
+  LOGLEVEL: LogLevel.debug,
+  DATA_STORE: {
+    dirname: "/var/log/myapp",   // defaults to logDir
+    datePattern: "YYYY-MM-DD",
+    zippedArchive: true,
+    maxSize: "20m",
+    maxFiles: "14d",             // older files are deleted
+  },
+}
+```
+
+Defaults are 20 MB per file, gzipped, kept a fortnight — long enough to diagnose something
+reported today, short enough that a forgotten log directory is not a growing disclosure risk.
+
+### Adding your own
+
+`database` and `api` transports are **not** in this package — they would drag a database
+driver and an HTTP client into every install. Register one instead:
+
+```ts
+import { registerTransport } from "@svayam-opensource/svm-util-log";
+import Transport from "winston-transport";
+
+class DbTransport extends Transport { /* … */ }
+
+registerTransport("database", (conf, config) => new DbTransport(conf, config));
+```
+
+Register **before** you call `createApplicationLogger`. A method named in your config with
+nothing registered for it is **ignored, not an error** — the same config file can move between
+a service that has a database and a CLI that does not, and the CLI simply writes fewer places.
+
+```mermaid
+flowchart LR
+    L["logger.log(…)"] --> M["mask meta"] --> F["format one line"]
+    F --> C["console → stderr"]
+    F --> D["file → daily rotate"]
+    F --> X["your transport<br/><i>registerTransport</i>"]
+    style X stroke-dasharray: 4 3
+```
+
+---
+
+## Configuration reference
+
+```ts
+createApplicationLogger({
+  appId,      // string   — names the app in every line and in the filename
+  logDir,     // string   — where file transports write by default
+  methods,    // array    — one entry per destination
+  mask,       // string[] — optional; extend DEFAULT_MASK
+  inspect,    // optional — { depth?: number; maxStringLength?: number } for meta rendering
+});
+```
+
+| Field | Required | Notes |
+|---|---|---|
+| `appId` | yes | appears as `app_cd` and in the log filename |
+| `logDir` | yes | must exist or be creatable; used when a `file` method sets no `dirname` |
+| `methods` | yes | must name at least one destination — see the note below |
+| `mask` | no | defaults to `DEFAULT_MASK`; **extend it, don't replace it** |
+| `inspect` | no | defaults to depth 4, 512-character strings |
+
+> **Give `methods` at least one entry.** An empty array does not throw, but it is not silence
+> either: winston then prints its own `Attempt to write logs with no transports` warning for
+> every line, with the entry as raw JSON. That output bypasses this library's formatting — the
+> masking still applies, because `meta` is masked on the way in, but the result is noisy and
+> not what you configured. `logger.toLogObject().ENABLED_TRANSPORTS` shows what is really
+> attached.
+
+Configuration is **passed in, never read from a global**. Nothing has to be initialised before
+you can log, which matters most on a program's first run — the moment you most want a log and
+are least likely to have configuration.
+
+---
+
+## API
+
+| Export | What it does |
+|---|---|
+| `createApplicationLogger(config)` | returns an `ApplicationLogger` |
+| `registerTransport(method, factory)` | adds a destination this package does not ship |
+| `LogLevel`, `LogLevelIndex`, `logLevelIndex` | the level names, their order, their index |
+| `LogMethod` | `console` · `file` · `database` · `api` |
+| `withinLevel(level, threshold)` | is `level` within `threshold`? |
+| `DEFAULT_MASK` | the field names masked by default |
+| `formatLogMessage(entry, config?)` | render one line — useful in your own transport |
+| `toLogEntry(appId, lvl, msg, …)` | build the entry a transport receives |
+| `maskValues(value, mask?)` | mask an object without logging it |
+| `stringify(value, config?)` | the `meta=` renderer; never throws, handles cycles |
+| `isLogEnabled(pgm_cd, fn_cd?, act_id?)` | is `NODE_DEBUG` on for this origin? |
+
+### `ApplicationLogger`
+
+```ts
+logger.log(level, message, pgm_cd?, fn_cd?, act_id?, meta?);   // write a line
+logger.toLogObject();   // what is actually enabled — for a `doctor`/`status` command
+logger.toString();      // the same, as JSON
+logger.close();         // flush and close every transport
+```
+
+**Always `close()` before a short-lived process exits.** File writes are asynchronous, so
+without it the last lines of a run — the ones explaining why it ended — are the ones most
+likely to be lost.
+
+---
+
+## Recipes
+
+### A command-line tool
+
+Quiet by default, everything on disk, and the user can raise the screen level themselves.
+
+```ts
+const logger = createApplicationLogger({
+  appId: "mycli",
+  logDir: path.join(os.homedir(), ".mycli", "logs"),
+  methods: [
+    { METHOD: "console", LOGLEVEL: LogLevel.warn },
+    { METHOD: "file",    LOGLEVEL: LogLevel.debug },
+  ],
+});
+
+process.on("exit", () => logger.close());
+```
+
+### A long-running service
+
+```ts
+const logger = createApplicationLogger({
+  appId: "orders-api",
+  logDir: "/var/log/orders-api",
+  methods: [
+    { METHOD: "console",  LOGLEVEL: LogLevel.info },
+    { METHOD: "file",     LOGLEVEL: LogLevel.http },
+    { METHOD: "database", LOGLEVEL: LogLevel.error },   // ignored unless registered
+  ],
+});
+```
+
+### Following one request across modules
+
+Give every line of one unit of work the same `act_id`:
+
+```ts
+const actId = nextRequestId();
+logger.log(LogLevel.http,  "POST /orders", "orders:api:routes", "create", actId);
+logger.log(LogLevel.debug, "reserving stock", "orders:domain:stock", "reserve", actId);
+logger.log(LogLevel.error, "payment declined", "orders:domain:pay", "charge", actId, { code: "card_declined" });
+```
+
+Then `grep ' 4471 '` returns that request and nothing else. `NODE_DEBUG` accepts an `act_id`
+too, so a single transaction can be traced without switching on a whole module.
+
+---
+
+## Troubleshooting
+
+| What you see | Why |
+|---|---|
+| Nothing below `warn` appears | Expected. Set `NODE_DEBUG` — see [above](#turning-on-debug-output-in-the-field). |
+| Log lines mixed into piped output | `console` writes to **stderr** by design. Redirect with `2>/dev/null`, or read it with `2>&1`. |
+| No file appears | Check `logDir` exists and is writable, and that a `file` method is in `methods`. |
+| The last lines of a run are missing | Call `logger.close()` before exit. |
+| A field you expected to be masked is not | Masking reads **field names** in `meta`. A secret placed in the message string is not visible to it. |
+| Your custom transport never writes | `registerTransport` must run **before** `createApplicationLogger`. |
+| `Attempt to write logs with no transports` | `methods` is empty, or every entry names a transport nothing registered. Check `logger.toLogObject()`. |
+| `mask` seems to have switched off the defaults | It did. Use `[...DEFAULT_MASK, "yours"]`. |
+
+---
+
+## Notes
+
+Console output uses [winston](https://github.com/winstonjs/winston); file rotation uses
+[winston-daily-rotate-file](https://github.com/winstonjs/winston-daily-rotate-file).
+
+### Design rationale
+
+Why this package is separate from `@svayam/svm-util`, why configuration is injected rather
+than global, why console output goes to stderr, and what belongs in this package versus a
+sibling — all of that is in `docs/architecture.md`.
+
+That file lives in the **source repository, which is private to Svayam**; the link below will
+not open for you unless you have access. Everything in this README stands on its own, and
+nothing in the architecture document changes how the library behaves.
+
+Svayam engineers: [`docs/architecture.md`](./docs/architecture.md).
 
-## Two deliberate departures from `svm-util`
+### Licence and support
 
-1. **Configuration is injected, never global.** `svm-util` reaches a module-level
-   `getSvayamConfiguration()`; a CLI has a first run where no configuration exists at all.
-2. **Console output goes to STDERR.** `stdout` is a program's answer, and a caller piping it
-   should not receive the log as data.
+Copyright © 2026 Svayam Infoware Pvt. Ltd. Published publicly so that Svayam's own public
+tools — and anyone installing them — can resolve it without registry configuration.

package/lib/cjs/logger.js

@@ -82,11 +82,20 @@ function createApplicationLogger(config) {
     });
     const effective = { mask: config_js_1.DEFAULT_MASK, ...config };
     const log = (lvl, msg, pgm_cd, fn_cd, act_id, meta) => {
-        // ERRORS AND WARNINGS ALWAYS PASS. Everything below them is opt-in through NODE_DEBUG, so
-        // an ordinary run is quiet — but a run that went wrong still said so without anybody
-        // having had the foresight to enable anything.
+        // ERRORS AND WARNINGS ALWAYS PASS. Below them, the GATE decides — `debuglog` (the default, opt-in through
+        // NODE_DEBUG) or `level`, where each transport's own LOGLEVEL is the only threshold.
+        //
+        // WHY `level` EXISTS (2026-09-23). A CLI cannot switch `debuglog` on for itself: Node compiles NODE_DEBUG
+        // into a matcher at bootstrap, so setting the variable from inside the process — even on the first line,
+        // before any import — has no effect. An application that has DECIDED to record every run (gov: "info and
+        // above on every run, silent on screen") therefore could not, through this library: its `info` lines were
+        // dropped, and its log folders held nothing but the transport's bookkeeping file. Found when gov's six log
+        // calls turned out never to have written a line.
+        //
+        // The default is unchanged, so nothing that relies on quiet-by-default changes.
         const always = levels_js_1.LogLevelIndex[lvl] <= levels_js_1.LogLevelIndex[levels_js_1.LogLevel.warn];
-        if (!always && !(0, format_js_1.isLogEnabled)(pgm_cd, fn_cd, act_id))
+        const gated = (effective.gate ?? "debuglog") === "debuglog";
+        if (!always && gated && !(0, format_js_1.isLogEnabled)(pgm_cd, fn_cd, act_id))
             return;
         winston.log((0, format_js_1.toLogEntry)(effective.appId, lvl, msg, pgm_cd, fn_cd, act_id, meta));
     };

package/lib/esm/config.d.ts

@@ -64,6 +64,16 @@ export interface LoggerConfig {
      * is copied, and it is sent to other people precisely when something has gone wrong.
      */
     readonly mask?: readonly string[];
+    /**
+     * What decides whether a line BELOW `warn` is written.
+     *
+     *   `debuglog` (default) — opt-in per origin through `NODE_DEBUG` (POL-425): quiet until someone in front of
+     *                          the problem switches it on, with no rebuild and no release.
+     *   `level`              — each transport's own LOGLEVEL is the threshold, and nothing else. For an
+     *                          application that records every run by decision rather than by environment: a
+     *                          process cannot enable `debuglog` for itself (Node reads NODE_DEBUG at bootstrap).
+     */
+    readonly gate?: "debuglog" | "level";
     /** Depth and length limits for `meta`, so one object cannot become the whole file. */
     readonly inspect?: {
         depth?: number;

package/lib/esm/logger.js

@@ -75,11 +75,20 @@ export function createApplicationLogger(config) {
     });
     const effective = { mask: DEFAULT_MASK, ...config };
     const log = (lvl, msg, pgm_cd, fn_cd, act_id, meta) => {
-        // ERRORS AND WARNINGS ALWAYS PASS. Everything below them is opt-in through NODE_DEBUG, so
-        // an ordinary run is quiet — but a run that went wrong still said so without anybody
-        // having had the foresight to enable anything.
+        // ERRORS AND WARNINGS ALWAYS PASS. Below them, the GATE decides — `debuglog` (the default, opt-in through
+        // NODE_DEBUG) or `level`, where each transport's own LOGLEVEL is the only threshold.
+        //
+        // WHY `level` EXISTS (2026-09-23). A CLI cannot switch `debuglog` on for itself: Node compiles NODE_DEBUG
+        // into a matcher at bootstrap, so setting the variable from inside the process — even on the first line,
+        // before any import — has no effect. An application that has DECIDED to record every run (gov: "info and
+        // above on every run, silent on screen") therefore could not, through this library: its `info` lines were
+        // dropped, and its log folders held nothing but the transport's bookkeeping file. Found when gov's six log
+        // calls turned out never to have written a line.
+        //
+        // The default is unchanged, so nothing that relies on quiet-by-default changes.
         const always = LogLevelIndex[lvl] <= LogLevelIndex[LogLevel.warn];
-        if (!always && !isLogEnabled(pgm_cd, fn_cd, act_id))
+        const gated = (effective.gate ?? "debuglog") === "debuglog";
+        if (!always && gated && !isLogEnabled(pgm_cd, fn_cd, act_id))
             return;
         winston.log(toLogEntry(effective.appId, lvl, msg, pgm_cd, fn_cd, act_id, meta));
     };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@svayam-opensource/svm-util-log",
-  "version": "1.0.1",
+  "version": "1.1.0",
   "description": "Svayam's shared logging framework — the logger from svm-util, with only the dependencies a logger needs.",
   "type": "module",
   "exports": {
@@ -50,8 +50,8 @@
     "typescript": "^5.7.2"
   },
   "gov": {
-    "contentSha": "0683398e412c95dc454e7a9f90c2c673a1b6bb6a639215c9ae8274b321c2ea73",
+    "contentSha": "dcd002b72a0afcd5960ce3b6b53bd40a3742b775546893d0847f4fcd34335b71",
     "shaFrom": "built-ref",
-    "commit": "b22df8f24ed4478f57ffcfaebf03a9953de3c375"
+    "commit": "0e172f8bae1d6951b7b86016f8c6cbf76da782ea"
   }
 }