@hackylabs/deep-redact 2.2.0 → 3.0.0

package/LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2024 Benjamin Green (https://bengreen.dev)
+Copyright (c) 2025 Benjamin Green (https://bengreen.dev)
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

package/README.md

@@ -3,12 +3,12 @@
 [![npm version](https://badge.fury.io/js/@hackylabs%2Fdeep-redact.svg)](https://badge.fury.io/js/@hackylabs%2Fdeep-redact)
 [![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/hackylabs/deep-redact/blob/main/LICENSE)
 
-Faster than Fast Redact <sup>1</sup> as well as being safer and more configurable than many other redaction solutions,
-Deep Redact is a zero-dependency tool that redacts sensitive information from strings and objects. It is designed to be
-used in a production environment where sensitive information needs to be redacted from logs, error messages, files,
-and other outputs. Supporting both strings and objects or a mix of both, Deep Redact can be used to redact sensitive
-information from more data structures than any other redaction library. Even partially redacting sensitive information
-from strings is supported, by way of custom regex patterns and replacers.
+Safer and more configurable than many other redaction solutions, Deep Redact is a zero-dependency tool that redacts
+sensitive information from strings and objects. It is designed to be used in a production environment where sensitive
+information needs to be redacted from logs, error messages, files, and other outputs. Supporting both strings and objects
+or a mix of both, Deep Redact can be used to redact sensitive information from more data structures than any other
+redaction library. Even partially redacting sensitive information from strings is supported, by way of custom regex
+patterns and replacers.
 
 Circular references and other unsupported values are handled gracefully, and the library is designed to be as fast as
 possible while still being easy to use and configure.
@@ -73,6 +73,30 @@ strRedaction.redact('<email>someone@somewhere.com</email><keepThis>This is fine<
 // '<email>[REDACTED]</email><keepThis>This is fine</keepThis><password>[REDACTED]</password>'
 ```
 
+// Override the `unsupportedTransformer` method to handle unsupported values
+
+```typescript
+class CustomRedaction extends DeepRedact {
+  constructor(options) {
+    super(options)
+    this.rewriteUnsupported = (value) => {
+      if (value instanceof BigInt) return value.toString()
+
+      // Add more conditional statements for unsupported value types here (e.g. Error, Date, Map, Set, etc.)
+
+      // If the value is supported, return it
+      return value
+    }
+  }
+}
+
+const customRedaction = new CustomRedaction({
+  blacklistedKeys: ['sensitive', 'password', /name/i],
+})
+
+customRedaction.redact({ a: BigInt(1) })
+```
+
 ## Configuration
 
 ### Main Options
@@ -80,8 +104,7 @@ strRedaction.redact('<email>someone@somewhere.com</email><keepThis>This is fine<
 | key | description | type | options | default | required |
 | --- | --- | --- | --- | --- | --- |
 | blacklistedKeys | Deeply compare names of these keys against the keys in your object. | array | Array<string￨RegExp￨BlacklistKeyConfig> | [] | N |
-| stringTests | Array of regular expressions to perform against string values, whether that value is a flat string or nested within an object. Will redact whole string values. If you want to redact only part of the string, use `partialStringTests` instead. If a replacer function is provided in the config for the associated test, it will be used to redact the value. | array | Array<RegExp￨StringTestConfig> | [] | N |
-| partialStringTests | Array of regular expressions to perform against string values, whether that value is a flat string or nested within an object. Will redact only the matched part of the string using the replacer function provided in the config for the associated test. | array | StringTestConfig[] | [] | N |
+| stringTests | Array of regular expressions to perform against string values, whether that value is a flat string or nested within an object. Can redact whole or partial string values. If a replacer function is provided in the config for the associated test, it will be used to redact the value if the value matches the test. | array | Array<RegExp￨StringTestConfig> | [] | N |
 | fuzzyKeyMatch | Loosely compare key names by checking if the key name of your unredacted object is included anywhere within the name of your blacklisted key. For example, is "pass" (your key) included in "password" (from config). | boolean |  | false | N |
 | caseSensitiveKeyMatch | Loosely compare key names by normalising the strings. This involves removing non-word characters and transforms the string to lowercase. This means you never have to worry having to list duplicate keys in different formats such as snake_case, camelCase, PascalCase or any other case. | boolean |  | true | N |
 | remove | Determines whether or not to remove the key from the object when it is redacted. | boolean |  | false | N |
@@ -89,8 +112,9 @@ strRedaction.redact('<email>someone@somewhere.com</email><keepThis>This is fine<
 | replacement | When a value is going to be redacted, what would you like to replace it with? | string ￨ function |  | [REDACTED] | N |
 | replaceStringByLength | When a string value is going to be replaced, optionally replace it by repeating the `replacement` to match the length of the value. For example, if `replaceStringByLength` were set to `true` and `replacement` was set to "x", then redacting "secret" would return "xxxxxx". This is sometimes useful for debugging purposes, although it may be less secure as it could give hints to the original value. | boolean |  | false | N |
 | types | JS types (values of `typeof` keyword). Only values with a typeof equal to `string`, `number`, `bigint`, `boolean`, `symbol`, `object`, or `function` will be redacted. Undefined values will never be redacted, although the type `undefined` is included in this list to keep TypeScript happy. | array | Array<'string'￨'number'￨'bigint'￨'boolean'￨'symbol'￨'undefined'￨'object'￨'function'> | ['string'] | N |
-| serialise | Determines whether or not to serialise the object after redacting. Typical use cases for this are when you want to send it over the network or save to a file, both of which are common use cases for redacting sensitive information. | boolean |  | false | N |
-| serialize | Alias of `serialise` for International-English users. | boolean |  | false | N |
+| serialise | Determines whether or not to serialise the object after redacting. Typical use cases for this are when you want to send it over the network or save to a file, both of which are common use cases for redacting sensitive information. | boolean |  | true | N |
+| serialize | Alias of `serialise` for International-English users. | boolean |  | Value of `serialise` | N |
+| transformers | A list of transformers to apply when transforming unsupported values. Each transformer should conditionally transform the value, or return the value unchanged to be passed to the next transformer. Transformers will be run in the order they are provided over the entire object. | array | Array<Transformer> | All standard transformers (bigint, date, error, map, regex, set, and url) | N |
 
 ### BlacklistKeyConfig
 
@@ -100,6 +124,8 @@ strRedaction.redact('<email>someone@somewhere.com</email><keepThis>This is fine<
 | fuzzyKeyMatch | boolean | Main options `fuzzyKeyMatch` | N |
 | caseSensitiveKeyMatch | boolean | Main options `caseSensitiveKeyMatch` | N |
 | remove | boolean | Main options `remove` | N |
+| replacement | string￨function | Main options `replacement` | N |
+| replaceStringByLength | boolean | Main options `replaceStringByLength` | N |
 | retainStructure | boolean | Main options `retainStructure` | N |
 
 ### StringTestConfig
@@ -108,49 +134,3 @@ strRedaction.redact('<email>someone@somewhere.com</email><keepThis>This is fine<
 | --- | --- | --- | --- |
 | pattern | A regular expression to perform against a string value, whether that value is a flat string or nested within an object. | RegExp | Y |
 | replacer | A function that will be called with the value of the string that matched the pattern and the pattern itself. This function should return the new (redacted) value to replace the original value. | function | Y |
-
-### Benchmark
-Comparisons are made against JSON.stringify, Regex.replace, Fast Redact &
-(one of my other creations, [@hackylabs/obglob](https://npmjs.com/package/@hackylabs/obglob)) as well as different
-configurations of Deep Redact, using [this test object](./test/setup/dummyUser.ts). Fast Redact was configured to redact
-the same keys on the same object as Deep Redact without using wildcards.
-
-The benchmark is run on a 2021 iMac with an M1 chip with 16GB memory running macOS Sequoia 15.0.0.
-
-JSON.stringify is included as a benchmark because it is the fastest way to deeply iterate over an object, although it
-doesn't redact any sensitive information.
-
-Regex.replace is included as a benchmark because it is the fastest way to redact sensitive information from a string.
-However, a regex pattern for all keys to be redacted is much harder to configure than a dedicated redaction library,
-especially when dealing with multiple types of values. It also doesn't handle circular references or other unsupported
-values as gracefully as deep-redact unless a third-party library is used to stringify the object beforehand.
-
-Fast-redact is included as a benchmark because it's the next fastest library available specifically for redaction.
-
-Neither JSON.stringify, Regex.replace nor Fast Redact offer the same level of configurability as deep-redact. Both Fast
-Redact and Obglob are slower and rely on dependencies.
-
-![Benchmark](./benchmark.png)
-
-| scenario | ops / sec | op duration (ms) | margin of error | sample count |
-| --- | --- | --- | --- | --- |
-| DeepRedact, partial redaction | 178183.03 | 0.0056122067 | 0.00003 | 89092 |
-| JSON.stringify, large object | 161672.63 | 0.0061853388 | 0.00004 | 80837 |
-| DeepRedact, remove item, single object | 25085.95 | 0.039862959 | 0.00033 | 12543 |
-| DeepRedact, default config, large object | 23164.57 | 0.0431693713 | 0.00023 | 11583 |
-| Regex replace, large object | 22828.32 | 0.043805244 | 0.00031 | 11415 |
-| DeepRedact, custom replacer function, single object | 22520.52 | 0.0444039553 | 0.00039 | 11261 |
-| DeepRedact, replace string by length, single object | 22470.94 | 0.0445019191 | 0.00025 | 11236 |
-| DeepRedact, retain structure, single object | 18315.67 | 0.0545980591 | 0.00026 | 9158 |
-| DeepRedact, fuzzy matching, single object | 18077.45 | 0.0553175285 | 0.00035 | 9039 |
-| DeepRedact, config per key, single object | 16055.96 | 0.0622821745 | 0.00031 | 8028 |
-| DeepRedact, default config, 1000 large objects | 8077.4 | 0.1238022 | 0.00187 | 4039 |
-| fast redact, large object | 5918.61 | 0.1689584628 | 0.00151 | 2960 |
-| ObGlob, large object | 5193.29 | 0.1925563273 | 0.00972 | 2597 |
-| DeepRedact, case insensitive matching, single object | 3477.6 | 0.2875549482 | 0.00324 | 1739 |
-| DeepRedact, fuzzy and case insensitive matching, single object | 3437.73 | 0.2908896131 | 0.00239 | 1719 |
-| ObGlob, 1000 large objects | 237.69 | 4.2072005126 | 0.05005 | 119 |
-| JSON.stringify, 1000 large objects | 230.33 | 4.3416907672 | 0.04373 | 116 |
-| DeepRedact, partial redaction large string | 127.18 | 7.8628400156 | 0.4176 | 64 |
-| fast redact, 1000 large objects | 119.33 | 8.3803393 | 0.31943 | 60 |
-| Regex replace, 1000 large objects | 97.12 | 10.2963537755 | 0.29341 | 49 |

package/dist/cjs/index.js

@@ -14,8 +14,8 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
     return (mod && mod.__esModule) ? mod : { "default": mod };
 };
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.DeepRedact = exports.default = void 0;
-const redactorUtils_1 = __importDefault(require("./utils/redactorUtils"));
+exports.default = exports.DeepRedact = void 0;
+const utils_1 = __importDefault(require("./utils"));
 class DeepRedact {
     /**
      * Create a new DeepRedact instance with the provided configuration.
@@ -24,11 +24,6 @@ class DeepRedact {
      * @param {DeepRedactConfig} config. The configuration for the redaction.
      */
     constructor(config) {
-        /**
-         * A WeakSet to store circular references during redaction. Reset to null after redaction is complete.
-         * @private
-         */
-        this.circularReference = null;
         /**
          * The configuration for the redaction.
          * @private
@@ -36,138 +31,22 @@ class DeepRedact {
         this.config = {
             serialise: false,
         };
-        /**
-         * A transformer for unsupported data types. If `serialise` is false, the value will be returned as is,
-         * otherwise it will transform the value into a format that is supported by JSON.stringify.
-         *
-         * Error, RegExp, and Date instances are technically supported by JSON.stringify,
-         * but they returned as empty objects, therefore they are also transformed here.
-         * @protected
-         * @param {unknown} value The value that is not supported by JSON.stringify.
-         * @returns {unknown} The value in a format that is supported by JSON.stringify.
-         */
-        this.unsupportedTransformer = (value) => {
-            if (!this.config.serialise)
-                return value;
-            if (typeof value === 'bigint') {
-                return {
-                    __unsupported: {
-                        type: 'bigint',
-                        value: value.toString(10),
-                        radix: 10,
-                    },
-                };
-            }
-            if (value instanceof Error) {
-                return {
-                    __unsupported: {
-                        type: 'error',
-                        name: value.name,
-                        message: value.message,
-                        stack: value.stack,
-                    },
-                };
-            }
-            if (value instanceof RegExp) {
-                return {
-                    __unsupported: {
-                        type: 'regexp',
-                        source: value.source,
-                        flags: value.flags,
-                    },
-                };
-            }
-            if (value instanceof Set) {
-                return {
-                    __unsupported: {
-                        type: 'set',
-                        values: Array.from(value),
-                    },
-                };
-            }
-            if (value instanceof Map) {
-                return {
-                    __unsupported: {
-                        type: 'map',
-                        entries: Object.fromEntries(value.entries()),
-                    },
-                };
-            }
-            if (value instanceof URL)
-                return value.toString();
-            if (value instanceof Date)
-                return value.toISOString();
-            return value;
-        };
-        /**
-         * Calls `unsupportedTransformer` on the provided value and rewrites any circular references.
-         *
-         * Circular references will always be removed to avoid infinite recursion.
-         * When a circular reference is found, the value will be replaced with `[[CIRCULAR_REFERENCE: path.to.original.value]]`.
-         * @protected
-         * @param {unknown} value The value to rewrite.
-         * @param {string | undefined} path The path to the value in the object.
-         * @returns {unknown} The rewritten value.
-         */
-        this.rewriteUnsupported = (value, path) => {
-            const safeValue = this.unsupportedTransformer(value);
-            if (!(safeValue instanceof Object))
-                return safeValue;
-            if (this.circularReference === null)
-                this.circularReference = new WeakSet();
-            if (Array.isArray(safeValue)) {
-                return safeValue.map((val, index) => {
-                    var _a, _b;
-                    const newPath = path ? `${path}.[${index}]` : `[${index}]`;
-                    if ((_a = this.circularReference) === null || _a === void 0 ? void 0 : _a.has(val))
-                        return `[[CIRCULAR_REFERENCE: ${newPath}]]`;
-                    if (val instanceof Object) {
-                        (_b = this.circularReference) === null || _b === void 0 ? void 0 : _b.add(val);
-                        return this.rewriteUnsupported(val, newPath);
-                    }
-                    return val;
-                });
-            }
-            return Object.fromEntries(Object.entries(safeValue).map(([key, val]) => {
-                var _a, _b;
-                const newPath = path ? `${path}.${key}` : key;
-                if ((_a = this.circularReference) === null || _a === void 0 ? void 0 : _a.has(val))
-                    return [key, `[[CIRCULAR_REFERENCE: ${newPath}]]`];
-                if (val instanceof Object)
-                    (_b = this.circularReference) === null || _b === void 0 ? void 0 : _b.add(val);
-                return [key, this.rewriteUnsupported(val, path ? `${path}.${key}` : key)];
-            }));
-        };
-        /**
-         * Depending on the value of `serialise`, return the value as a JSON string or as the provided value.
-         *
-         * Also resets the `circularReference` property to null after redaction is complete.
-         * This is to ensure that the WeakSet doesn't cause memory leaks.
-         * @private
-         * @param value
-         */
-        this.maybeSerialise = (value) => {
-            this.circularReference = null;
-            const result = this.redactorUtils.partialStringRedact(value);
-            if (!this.config.serialise)
-                return result;
-            return typeof result === 'string' ? result : JSON.stringify(result);
-        };
         /**
          * Redact the provided value. The value will be stripped of any circular references and other unsupported data types, before being redacted according to the configuration and finally serialised if required.
          * @param {unknown} value The value to redact.
          * @returns {unknown} The redacted value.
+         * @throws {Error} If the value cannot be serialised to JSON and serialise is true.
          */
         this.redact = (value) => {
-            return this.maybeSerialise(this.redactorUtils.recurse(this.rewriteUnsupported(value)));
+            const redacted = this.redactorUtils.traverse(value);
+            return this.config.serialise ? JSON.stringify(redacted) : redacted;
         };
         const { serialise, serialize } = config, rest = __rest(config, ["serialise", "serialize"]);
-        this.redactorUtils = new redactorUtils_1.default(rest);
-        if (serialise !== undefined)
-            this.config.serialise = serialise;
-        if (serialize !== undefined)
-            this.config.serialise = serialize;
+        const englishSerialise = serialise !== null && serialise !== void 0 ? serialise : serialize;
+        if (typeof englishSerialise === 'boolean')
+            this.config.serialise = englishSerialise;
+        this.redactorUtils = new utils_1.default(Object.assign({}, rest));
     }
 }
-exports.default = DeepRedact;
 exports.DeepRedact = DeepRedact;
+exports.default = DeepRedact;