@karmaniverous/get-dotenv 5.1.0 → 5.2.1

package/README.md

@@ -1,4 +1,4 @@
-> **_Load, expand, and compose environment variables from a deterministic dotenv cascade, then execute commands under that context. Use `get-dotenv` as a library, a CLI, or a plugin-first host to build dotenv-aware tooling with cross‑platform shell control, CI‑friendly capture, and clear diagnostics._**
+> Load, expand, and compose environment variables from a deterministic dotenv cascade, then execute commands under that context. Use `get-dotenv` as a library, a CLI, or a plugin-first host to build dotenv-aware tooling with cross‑platform shell control, CI‑friendly capture, and clear diagnostics.
 
 # get-dotenv
 
@@ -34,6 +34,20 @@ Load environment variables with a cascade of environment-aware dotenv files. You
 
 ✅ Set defaults for all options in a `getdotenv.config.json` file in your project root directory.
 
+✅ Validate your final composed environment via config: JSON/YAML `requiredKeys` or a JS/TS Zod `schema`. Validation runs once after Phase C (interpolation). Use `--strict` to fail on issues; otherwise warnings are printed. See the [Config files and overlays](./guides/config.md) guide.
+
+✅ Diagnostics for safer visibility without altering runtime values:
+
+- Redaction at presentation time for secret-like keys (`--redact`, `--redact-pattern`).
+- Entropy warnings (on by default) for high-entropy strings; gated by length/printability and tunable via `--entropy-*` flags. See [Config files and overlays](./guides/config.md).
+
+✅ Clear tracing and CI-friendly capture:
+
+- `--trace [keys...]` shows per-key origin (parent | dotenv | unset) before spawning.
+- Set `GETDOTENV_STDIO=pipe` or use `--capture` to buffer child stdout/stderr deterministically. See [Shell execution behavior](./guides/shell.md).
+
+✅ Cross-platform shells and normalized child environments: defaults to `/bin/bash` on POSIX and `powershell.exe` on Windows; subprocess env is composed once via a unified helper that drops undefineds and normalizes temp/home variables. See [Shell execution behavior](./guides/shell.md).
+
 ✅ [Generate an extensible `getdotenv`-based CLI](https://github.com/karmaniverous/get-dotenv-child) for use in your own projects.
 
 `getdotenv` relies on the excellent [`dotenv`](https://www.npmjs.com/package/dotenv) parser and somewhat improves on [`dotenv-expand`](https://www.npmjs.com/package/dotenv-expand) for recursive variable expansion.
@@ -118,6 +132,42 @@ Options can be passed programmatically or set in a `getdotenv.config.json` file
 
 See the [child CLI example repo](https://github.com/karmaniverous/get-dotenv-child#configuration) for an extensive discussion of the various config options and how & where to set them.
 
+## CLI embedding (createCli)
+
+Prefer the named factory when you want to embed the get‑dotenv CLI in your own tool. It wires the plugin‑first host with the included plugins and returns a small runner.
+
+```ts
+#!/usr/bin/env node
+import { createCli } from '@karmaniverous/get-dotenv';
+
+// Build a CLI and run with your argv; alias appears in help.
+await createCli({
+  alias: 'mycli',
+  // Optional: override the help header (otherwise “mycli v<version>” is used).
+  branding: 'mycli v1.2.3',
+}).run(process.argv.slice(2));
+```
+
+Notes:
+
+- The host resolves dotenv context once per invocation and exposes subcommands: cmd, batch, aws, and init (see Guides below).
+- Use `--trace [keys...]` and `--redact` for diagnostics without altering runtime values.
+- Default shells are normalized across platforms: `/bin/bash` on POSIX and `powershell.exe` on Windows (overridable per‑script or globally).
+- Help/exit behavior (important for embedding and tests):
+  - To keep `-h/--help` deterministic across ESM/CJS and avoid Commander’s default `process.exit`, `createCli().run(['-h'])` prints help and returns without exiting the process.
+  - Because help is printed before the internal `brand()` call runs, the optional branding header may be omitted when `-h/--help` is used at the top level. If you need a branded header, prefer `mycli help` (which runs through normal parsing) or construct and brand a host directly (see “Branding the host CLI” in Guides).
+- Interop matrix (embedding in different module systems):
+  - ESM dynamic:
+    ```ts
+    const { createCli } = await import('@karmaniverous/get-dotenv');
+    await createCli({ alias: 'mycli' }).run(['-h']);
+    ```
+  - CommonJS require (using built outputs for smoke checks):
+    ```js
+    const { createCli } = require('@karmaniverous/get-dotenv/dist/index.cjs');
+    createCli({ alias: 'mycli' }).run(['-h']);
+    ```
+
 ## Dynamic Processing
 
 This package supports the full [`dotenv-expand`](https://www.npmjs.com/package/dotenv-expand) syntax, with some internal performance improvements. Dynamic variables can be authored in JS or TS.

package/dist/cliHost.cjs

@@ -38,6 +38,11 @@ const baseRootOptionDefaults = {
     dotenvToken: '.env',
     loadProcess: true,
     logger: console,
+    // Diagnostics defaults
+    warnEntropy: true,
+    entropyThreshold: 3.8,
+    entropyMinLength: 16,
+    entropyWhitelist: ['^GIT_', '^npm_', '^CI$', 'SHLVL'],
     paths: './',
     pathsDelimiter: ' ',
     privateToken: 'local',
@@ -58,7 +63,7 @@ const baseRootOptionDefaults = {
 const baseGetDotenvCliOptions = baseRootOptionDefaults;
 
 /** @internal */
-const isPlainObject = (value) => value !== null &&
+const isPlainObject$1 = (value) => value !== null &&
     typeof value === 'object' &&
     Object.getPrototypeOf(value) === Object.prototype;
 const mergeInto = (target, source) => {
@@ -66,10 +71,10 @@ const mergeInto = (target, source) => {
         if (sVal === undefined)
             continue; // do not overwrite with undefined
         const tVal = target[key];
-        if (isPlainObject(tVal) && isPlainObject(sVal)) {
+        if (isPlainObject$1(tVal) && isPlainObject$1(sVal)) {
             target[key] = mergeInto({ ...tVal }, sVal);
         }
-        else if (isPlainObject(sVal)) {
+        else if (isPlainObject$1(sVal)) {
             target[key] = mergeInto({}, sVal);
         }
         else {
@@ -229,11 +234,12 @@ const getDotenvOptionsSchemaRaw = zod.z.object({
 const getDotenvOptionsSchemaResolved = getDotenvOptionsSchemaRaw;
 
 /**
- * Zod schemas for configuration files discovered by the new loader. *
+ * Zod schemas for configuration files discovered by the new loader.
+ *
  * Notes:
  * - RAW: all fields optional; shapes are stringly-friendly (paths may be string[] or string).
  * - RESOLVED: normalized shapes (paths always string[]).
- * - For this step (JSON/YAML only), any defined `dynamic` will be rejected by the loader.
+ * - For JSON/YAML configs, the loader rejects "dynamic" and "schema" (JS/TS-only).
  */
 // String-only env value map
 const stringMap = zod.z.record(zod.z.string(), zod.z.string());
@@ -248,6 +254,8 @@ const getDotenvConfigSchemaRaw = zod.z.object({
     log: zod.z.boolean().optional(),
     shell: zod.z.union([zod.z.string(), zod.z.boolean()]).optional(),
     scripts: zod.z.record(zod.z.string(), zod.z.unknown()).optional(), // Scripts validation left wide; generator validates elsewhere
+    requiredKeys: zod.z.array(zod.z.string()).optional(),
+    schema: zod.z.unknown().optional(), // JS/TS-only; loader rejects in JSON/YAML
     vars: stringMap.optional(), // public, global
     envVars: envStringMap.optional(), // public, per-env
     // Dynamic in config (JS/TS only). JSON/YAML loader will reject if set.
@@ -430,9 +438,11 @@ const loadConfigFile = async (filePath) => {
             .join('\n');
         throw new Error(`Invalid config ${filePath}:\n${msgs}`);
     }
-    // Disallow dynamic in JSON/YAML; allow in JS/TS
-    if (!isJsOrTs(filePath) && parsed.data.dynamic !== undefined) {
-        throw new Error(`Config ${filePath} specifies "dynamic"; JSON/YAML configs cannot include dynamic in this step. Use JS/TS config.`);
+    // Disallow dynamic and schema in JSON/YAML; allow both in JS/TS.
+    if (!isJsOrTs(filePath) &&
+        (parsed.data.dynamic !== undefined || parsed.data.schema !== undefined)) {
+        throw new Error(`Config ${filePath} specifies unsupported keys for JSON/YAML. ` +
+            `Use JS/TS config for "dynamic" or "schema".`);
     }
     return getDotenvConfigSchemaResolved.parse(parsed.data);
 };
@@ -620,6 +630,78 @@ const overlayEnv = ({ base, env, configs, programmaticVars, }) => {
     return current;
 };
 
+/** src/diagnostics/entropy.ts
+ * Entropy diagnostics (presentation-only).
+ * - Gated by min length and printable ASCII.
+ * - Warn once per key per run when bits/char \>= threshold.
+ * - Supports whitelist patterns to suppress known-noise keys.
+ */
+const warned = new Set();
+const isPrintableAscii = (s) => /^[\x20-\x7E]+$/.test(s);
+const compile$1 = (patterns) => (patterns ?? []).map((p) => new RegExp(p, 'i'));
+const whitelisted = (key, regs) => regs.some((re) => re.test(key));
+const shannonBitsPerChar = (s) => {
+    const freq = new Map();
+    for (const ch of s)
+        freq.set(ch, (freq.get(ch) ?? 0) + 1);
+    const n = s.length;
+    let h = 0;
+    for (const c of freq.values()) {
+        const p = c / n;
+        h -= p * Math.log2(p);
+    }
+    return h;
+};
+/**
+ * Maybe emit a one-line entropy warning for a key.
+ * Caller supplies an `emit(line)` function; the helper ensures once-per-key.
+ */
+const maybeWarnEntropy = (key, value, origin, opts, emit) => {
+    if (!opts || opts.warnEntropy === false)
+        return;
+    if (warned.has(key))
+        return;
+    const v = value ?? '';
+    const minLen = Math.max(0, opts.entropyMinLength ?? 16);
+    const threshold = opts.entropyThreshold ?? 3.8;
+    if (v.length < minLen)
+        return;
+    if (!isPrintableAscii(v))
+        return;
+    const wl = compile$1(opts.entropyWhitelist);
+    if (whitelisted(key, wl))
+        return;
+    const bpc = shannonBitsPerChar(v);
+    if (bpc >= threshold) {
+        warned.add(key);
+        emit(`[entropy] key=${key} score=${bpc.toFixed(2)} len=${String(v.length)} origin=${origin}`);
+    }
+};
+
+const DEFAULT_PATTERNS = [
+    '\\bsecret\\b',
+    '\\btoken\\b',
+    '\\bpass(word)?\\b',
+    '\\bapi[_-]?key\\b',
+    '\\bkey\\b',
+];
+const compile = (patterns) => (patterns && patterns.length > 0 ? patterns : DEFAULT_PATTERNS).map((p) => new RegExp(p, 'i'));
+const shouldRedactKey = (key, regs) => regs.some((re) => re.test(key));
+const MASK = '[redacted]';
+/**
+ * Produce a shallow redacted copy of an env-like object for display.
+ */
+const redactObject = (obj, opts) => {
+    if (!opts?.redact)
+        return { ...obj };
+    const regs = compile(opts.redactPatterns);
+    const out = {};
+    for (const [k, v] of Object.entries(obj)) {
+        out[k] = v && shouldRedactKey(k, regs) ? MASK : v;
+    }
+    return out;
+};
+
 /**
  * Asynchronously read a dotenv file & parse it into an object.
  *
@@ -869,14 +951,103 @@ const getDotenv = async (options = {}) => {
         resultDotenv = dotenvForOutput;
     }
     // Log result.
-    if (log)
-        logger.log(resultDotenv);
+    if (log) {
+        const redactFlag = options.redact ?? false;
+        const redactPatterns = options.redactPatterns ?? undefined;
+        const redOpts = {};
+        if (redactFlag)
+            redOpts.redact = true;
+        if (redactFlag && Array.isArray(redactPatterns))
+            redOpts.redactPatterns = redactPatterns;
+        const bag = redactFlag
+            ? redactObject(resultDotenv, redOpts)
+            : { ...resultDotenv };
+        logger.log(bag);
+        // Entropy warnings: once-per-key-per-run (presentation only)
+        const warnEntropyVal = options.warnEntropy ?? true;
+        const entropyThresholdVal = options
+            .entropyThreshold;
+        const entropyMinLengthVal = options
+            .entropyMinLength;
+        const entropyWhitelistVal = options
+            .entropyWhitelist;
+        const entOpts = {};
+        if (typeof warnEntropyVal === 'boolean')
+            entOpts.warnEntropy = warnEntropyVal;
+        if (typeof entropyThresholdVal === 'number')
+            entOpts.entropyThreshold = entropyThresholdVal;
+        if (typeof entropyMinLengthVal === 'number')
+            entOpts.entropyMinLength = entropyMinLengthVal;
+        if (Array.isArray(entropyWhitelistVal))
+            entOpts.entropyWhitelist = entropyWhitelistVal;
+        for (const [k, v] of Object.entries(resultDotenv)) {
+            maybeWarnEntropy(k, v, v !== undefined ? 'dotenv' : 'unset', entOpts, (line) => {
+                logger.log(line);
+            });
+        }
+    }
     // Load process.env.
     if (loadProcess)
         Object.assign(process.env, resultDotenv);
     return resultDotenv;
 };
 
+/**
+ * Deep interpolation utility for string leaves.
+ * - Expands string values using dotenv-style expansion against the provided envRef.
+ * - Preserves non-strings as-is.
+ * - Does not recurse into arrays (arrays are returned unchanged).
+ *
+ * Intended for:
+ * - Phase C option/config interpolation after composing ctx.dotenv.
+ * - Per-plugin config slice interpolation before afterResolve.
+ */
+/** @internal */
+const isPlainObject = (v) => v !== null &&
+    typeof v === 'object' &&
+    !Array.isArray(v) &&
+    Object.getPrototypeOf(v) === Object.prototype;
+/**
+ * Deeply interpolate string leaves against envRef.
+ * Arrays are not recursed into; they are returned unchanged.
+ *
+ * @typeParam T - Shape of the input value.
+ * @param value - Input value (object/array/primitive).
+ * @param envRef - Reference environment for interpolation.
+ * @returns A new value with string leaves interpolated.
+ */
+const interpolateDeep = (value, envRef) => {
+    // Strings: expand and return
+    if (typeof value === 'string') {
+        const out = dotenvExpand(value, envRef);
+        // dotenvExpand returns string | undefined; preserve original on undefined
+        return (out ?? value);
+    }
+    // Arrays: return as-is (no recursion)
+    if (Array.isArray(value)) {
+        return value;
+    }
+    // Plain objects: shallow clone and recurse into values
+    if (isPlainObject(value)) {
+        const src = value;
+        const out = {};
+        for (const [k, v] of Object.entries(src)) {
+            // Recurse for strings/objects; keep arrays as-is; preserve other scalars
+            if (typeof v === 'string')
+                out[k] = dotenvExpand(v, envRef) ?? v;
+            else if (Array.isArray(v))
+                out[k] = v;
+            else if (isPlainObject(v))
+                out[k] = interpolateDeep(v, envRef);
+            else
+                out[k] = v;
+        }
+        return out;
+    }
+    // Other primitives/types: return as-is
+    return value;
+};
+
 /**
  * Compute the dotenv context for the host (uses the config loader/overlay path).
  * - Resolves and validates options strictly (host-only).
@@ -960,19 +1131,32 @@ const computeContext = async (customOptions, plugins, hostMetaUrl) => {
         {};
     const mergedPluginConfigs = defaultsDeep({}, packagedPlugins, publicPlugins, localPlugins);
     for (const p of plugins) {
-        if (!p.id || !p.configSchema)
+        if (!p.id)
             continue;
         const slice = mergedPluginConfigs[p.id];
         if (slice === undefined)
             continue;
-        const parsed = p.configSchema.safeParse(slice);
-        if (!parsed.success) {
-            const msgs = parsed.error.issues
-                .map((i) => `${i.path.join('.')}: ${i.message}`)
-                .join('\n');
-            throw new Error(`Invalid config for plugin '${p.id}':\n${msgs}`);
+        // Per-plugin interpolation just before validation/afterResolve:
+        // precedence: process.env wins over ctx.dotenv for slice defaults.
+        const envRef = {
+            ...dotenv,
+            ...process.env,
+        };
+        const interpolated = interpolateDeep(slice, envRef);
+        // Validate if a schema is provided; otherwise accept interpolated slice as-is.
+        if (p.configSchema) {
+            const parsed = p.configSchema.safeParse(interpolated);
+            if (!parsed.success) {
+                const msgs = parsed.error.issues
+                    .map((i) => `${i.path.join('.')}: ${i.message}`)
+                    .join('\n');
+                throw new Error(`Invalid config for plugin '${p.id}':\n${msgs}`);
+            }
+            mergedPluginConfigs[p.id] = parsed.data;
+        }
+        else {
+            mergedPluginConfigs[p.id] = interpolated;
         }
-        mergedPluginConfigs[p.id] = parsed.data;
     }
     return {
         optionsResolved: validated,
@@ -1016,10 +1200,23 @@ class GetDotenvCli extends commander.Command {
             visibleOptions: (cmd) => {
                 const all = cmd.options ??
                     [];
-                return all.filter((opt) => {
+                const base = all.filter((opt) => {
                     const group = opt.__group;
                     return group === 'base';
                 });
+                // Sort: short-aliased options first, then long-only; stable by flags.
+                const hasShort = (opt) => {
+                    const flags = opt.flags ?? '';
+                    // Matches "-x," or starting "-x " before any long
+                    return /(^|\s|,)-[A-Za-z]/.test(flags);
+                };
+                const byFlags = (opt) => opt.flags ?? '';
+                base.sort((a, b) => {
+                    const aS = hasShort(a) ? 1 : 0;
+                    const bS = hasShort(b) ? 1 : 0;
+                    return bS - aS || byFlags(a).localeCompare(byFlags(b));
+                });
+                return base;
             },
         });
         this.addHelpText('beforeAll', () => {
@@ -1208,6 +1405,12 @@ class GetDotenvCli extends commander.Command {
             return '';
         const renderRows = (title, rows) => {
             const width = Math.min(40, rows.reduce((m, r) => Math.max(m, r.flags.length), 0));
+            // Sort within group: short-aliased flags first
+            rows.sort((a, b) => {
+                const aS = /(^|\s|,)-[A-Za-z]/.test(a.flags) ? 1 : 0;
+                const bS = /(^|\s|,)-[A-Za-z]/.test(b.flags) ? 1 : 0;
+                return bS - aS || a.flags.localeCompare(b.flags);
+            });
             const lines = rows
                 .map((r) => {
                 const pad = ' '.repeat(Math.max(2, width - r.flags.length + 2));

package/dist/cliHost.d.cts

@@ -115,6 +115,23 @@ interface GetDotenvCliOptions extends Omit<GetDotenvOptions, 'paths' | 'vars'> {
      * Logs CLI internals when true.
      */
     debug?: boolean;
+    /**
+     * Strict mode: fail the run when env validation issues are detected
+     * (schema or requiredKeys). Warns by default when false or unset.
+     */
+    strict?: boolean;
+    /**
+     * Redaction (presentation): mask secret-like values in logs/trace.
+     */
+    redact?: boolean;
+    /**
+     * Entropy warnings (presentation): emit once-per-key warnings for high-entropy values.
+     */
+    warnEntropy?: boolean;
+    entropyThreshold?: number;
+    entropyMinLength?: number;
+    entropyWhitelist?: string[];
+    redactPatterns?: string[];
     /**
      * When true, capture child stdout/stderr and re-emit after completion.
      * Useful for tests/CI. Default behavior is streaming via stdio: 'inherit'.

package/dist/cliHost.d.mts

@@ -115,6 +115,23 @@ interface GetDotenvCliOptions extends Omit<GetDotenvOptions, 'paths' | 'vars'> {
      * Logs CLI internals when true.
      */
     debug?: boolean;
+    /**
+     * Strict mode: fail the run when env validation issues are detected
+     * (schema or requiredKeys). Warns by default when false or unset.
+     */
+    strict?: boolean;
+    /**
+     * Redaction (presentation): mask secret-like values in logs/trace.
+     */
+    redact?: boolean;
+    /**
+     * Entropy warnings (presentation): emit once-per-key warnings for high-entropy values.
+     */
+    warnEntropy?: boolean;
+    entropyThreshold?: number;
+    entropyMinLength?: number;
+    entropyWhitelist?: string[];
+    redactPatterns?: string[];
     /**
      * When true, capture child stdout/stderr and re-emit after completion.
      * Useful for tests/CI. Default behavior is streaming via stdio: 'inherit'.

package/dist/cliHost.d.ts

@@ -115,6 +115,23 @@ interface GetDotenvCliOptions extends Omit<GetDotenvOptions, 'paths' | 'vars'> {
      * Logs CLI internals when true.
      */
     debug?: boolean;
+    /**
+     * Strict mode: fail the run when env validation issues are detected
+     * (schema or requiredKeys). Warns by default when false or unset.
+     */
+    strict?: boolean;
+    /**
+     * Redaction (presentation): mask secret-like values in logs/trace.
+     */
+    redact?: boolean;
+    /**
+     * Entropy warnings (presentation): emit once-per-key warnings for high-entropy values.
+     */
+    warnEntropy?: boolean;
+    entropyThreshold?: number;
+    entropyMinLength?: number;
+    entropyWhitelist?: string[];
+    redactPatterns?: string[];
     /**
      * When true, capture child stdout/stderr and re-emit after completion.
      * Useful for tests/CI. Default behavior is streaming via stdio: 'inherit'.