@hackylabs/deep-redact 2.0.1 → 2.1.0

package/README.md

@@ -1,16 +1,21 @@
 # Deep Redact
 
-Faster than Fast Redact <sup>1</sup> as well as being safer and more configurable than many other redaction libraries,
+[![npm version](https://badge.fury.io/js/@hackylabs%2Fdeep-redact.svg)](https://badge.fury.io/js/@hackylabs%2Fdeep-redact)
+[![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/hackylabs/deep-redact/blob/main/LICENSE)
+
+Faster than Fast Redact <sup>1</sup> as well as being safer and more configurable than many other redaction solutions,
 Deep Redact is a zero-dependency tool that redacts sensitive information from strings and objects. It is designed to be
 used in a production environment where sensitive information needs to be redacted from logs, error messages, files,
 and other outputs.
 
 Circular references and other unsupported values are handled gracefully, and the library is designed to be as fast as
-possible while still being configurable.
+possible while still being easy to use and configure.
 
 Supporting both CommonJS and ESM, with named and default exports, Deep Redact is designed to be versatile and easy to
 use in any modern JavaScript or TypeScript project in Node or the browser.
 
+[![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/hackylabs)
+
 ## Installation
 
 ```bash
@@ -26,9 +31,8 @@ library outside of your global logging/error-reporting libraries.</h4>
 // ./src/example.ts
 import {DeepRedact} from '@hackylabs/deep-redact'; // If you're using CommonJS, import with require('@hackylabs/deep-redact') instead. Both CommonJS and ESM support named and default imports.
 
-const redaction = new DeepRedact({
+const objRedaction = new DeepRedact({
   blacklistedKeys: ['sensitive', 'password', /name/i],
-  serialise: false,
 })
 
 const obj = {
@@ -41,7 +45,8 @@ const obj = {
   }
 }
 
-redaction.redact(obj)
+// Recursively redact sensitive information from an object
+objRedaction.redact(obj)
 // {
 //  keepThis: 'This is fine',
 //  sensitive: '[REDACTED]',
@@ -51,6 +56,19 @@ redaction.redact(obj)
 //    firstName: '[REDACTED]'
 //  }
 // }
+
+const strRedaction = new DeepRedact({
+  stringTests: [
+    {
+      pattern: /<(email|password)>([^<]+)<\/\1>/gi,
+      replacer: (value: string, pattern: RegExp) => value.replace(pattern, '<$1>[REDACTED]</$1>'),
+    },
+  ],
+})
+
+// Partially redact sensitive information from a string
+strRedaction.redact('<email>someone@somewhere.com</email><keepThis>This is fine</keepThis><password>secret</password>')
+// '<email>[REDACTED]</email><keepThis>This is fine</keepThis><password>[REDACTED]</password>'
 ```
 
 ## Configuration
@@ -60,7 +78,7 @@ redaction.redact(obj)
 | key | description | type | options | default | required |
 | --- | --- | --- | --- | --- | --- |
 | blacklistedKeys | Deeply compare names of these keys against the keys in your object. | array | Array<string￨RegExp￨BlacklistKeyConfig> | [] | N |
-| stringTests | Array of regular expressions to perform against string values, whether that value is a flat string or nested within an object. | array | RegExp[] | [] | N |
+| stringTests | Array of regular expressions to perform against string values, whether that value is a flat string or nested within an object. | array | Array<RegExp￨StringTestConfig> | [] | N |
 | fuzzyKeyMatch | Loosely compare key names by checking if the key name of your unredacted object is included anywhere within the name of your blacklisted key. For example, is "pass" (your key) included in "password" (from config). | boolean |  | false | N |
 | caseSensitiveKeyMatch | Loosely compare key names by normalising the strings. This involves removing non-word characters and transforms the string to lowercase. This means you never have to worry having to list duplicate keys in different formats such as snake_case, camelCase, PascalCase or any other case. | boolean |  | true | N |
 | remove | Determines whether or not to remove the key from the object when it is redacted. | boolean |  | false | N |
@@ -81,32 +99,54 @@ redaction.redact(obj)
 | remove | boolean | Main options `remove` | N |
 | retainStructure | boolean | Main options `retainStructure` | N |
 
+### StringTestConfig
+
+| key | description | type | required |
+| --- | --- | --- | --- |
+| pattern | A regular expression to perform against a string value, whether that value is a flat string or nested within an object. | RegExp | Y |
+| replacer | A function that will be called with the value of the string that matched the pattern and the pattern itself. This function should return the new (redacted) value to replace the original value. | function | Y |
+
 ### Benchmark
-Comparisons are made against JSON.stringify and Fast Redact as well as different configurations of Deep Redact, using
-[this test object](./test/setup/dummyUser.ts). Fast Redact was configured to redact the same keys on the same object as
-Deep Redact without using wildcards.
+Comparisons are made against JSON.stringify, Regex.replace, Fast Redact &
+(one of my other creations, [@hackylabs/obglob](https://npmjs.com/package/@hackylabs/obglob)) as well as different
+configurations of Deep Redact, using [this test object](./test/setup/dummyUser.ts). Fast Redact was configured to redact
+the same keys on the same object as Deep Redact without using wildcards.
+
+The benchmark is run on a 2021 iMac with an M1 chip with 16GB memory running macOS Sequoia 15.0.0.
+
+JSON.stringify is included as a benchmark because it is the fastest way to deeply iterate over an object, although it
+doesn't redact any sensitive information.
+
+Regex.replace is included as a benchmark because it is the fastest way to redact sensitive information from a string.
+However, a regex pattern for all keys to be redacted is much harder to configure than a dedicated redaction library,
+especially when dealing with multiple types of values. It also doesn't handle circular references or other unsupported
+values as gracefully as deep-redact unless a third-party library is used to stringify the object beforehand.
 
-The benchmark is run on a 2021 iMac with an M1 chip with 16GB memory running Sonoma 14.5.
+Fast-redact is included as a benchmark because it's the next fastest library available specifically for redaction.
 
-JSON.stringify is included as a benchmark because it is the fastest way to deeply iterate over an object although it
-doesn't redact any sensitive information. Fast-redact is included as a benchmark because it's the next fastest redaction
-library available. Neither JSON.stringify nor Fast Redact offer the same level of configurability as deep-redact.
+Neither JSON.stringify, Regex.replace nor Fast Redact offer the same level of configurability as deep-redact. Both Fast
+Redact and Obglob are slower and rely on dependencies.
 
 ![Benchmark](./benchmark.png)
 
 | scenario | ops / sec | op duration (ms) | margin of error | sample count |
 | --- | --- | --- | --- | --- |
-| JSON.stringify, large object | 295500.62 | 0.0033840876 | 0.00002 | 147751 |
-| DeepRedact, remove item, single object | 36272.4 | 0.0275691709 | 0.00016 | 18137 |
-| DeepRedact, custom replacer function, single object | 30314.59 | 0.0329874115 | 0.00028 | 15158 |
-| DeepRedact, default config, large object | 30028.19 | 0.0333020395 | 0.0002 | 15015 |
-| DeepRedact, replace string by length, single object | 28756.9 | 0.0347742688 | 0.00028 | 14379 |
-| DeepRedact, retain structure, single object | 24803.01 | 0.0403176903 | 0.00032 | 12402 |
-| DeepRedact, fuzzy matching, single object | 22243.3 | 0.0449573621 | 0.00038 | 11122 |
-| DeepRedact, config per key, single object | 21603.85 | 0.0462880355 | 0.0013 | 10802 |
-| fast redact, large object | 9529.2 | 0.1049406557 | 0.00064 | 4765 |
-| DeepRedact, case insensitive matching, single object | 6503.72 | 0.1537581959 | 0.00105 | 3252 |
-| DeepRedact, default config, 1000 large objects | 5915.05 | 0.1690602382 | 0.00296 | 2958 |
-| DeepRedact, fuzzy and case insensitive matching, single object | 5591.96 | 0.1788283015 | 0.00184 | 2796 |
-| JSON.stringify, 1000 large objects | 394.41 | 2.5354059248 | 0.01001 | 198 |
-| fast redact, 1000 large objects | 172.23 | 5.8060829174 | 0.06886 | 87 |
+| DeepRedact, XML | 171985.01 | 0.0058144601 | 0.00004 | 85993 |
+| JSON.stringify, large object | 162406.59 | 0.0061573855 | 0.00002 | 81204 |
+| DeepRedact, remove item, single object | 25293.55 | 0.039535775 | 0.00023 | 12647 |
+| Regex replace, large object | 22529.52 | 0.0443862153 | 0.00024 | 11265 |
+| DeepRedact, custom replacer function, single object | 22324.84 | 0.0447931554 | 0.00037 | 11163 |
+| DeepRedact, default config, large object | 21437.04 | 0.046648239 | 0.00028 | 10719 |
+| DeepRedact, replace string by length, single object | 21018.32 | 0.0475775519 | 0.00084 | 10510 |
+| DeepRedact, fuzzy matching, single object | 18000.38 | 0.0555543858 | 0.0003 | 9001 |
+| DeepRedact, retain structure, single object | 17581.5 | 0.056877956 | 0.00037 | 8791 |
+| DeepRedact, config per key, single object | 16914.73 | 0.0591200552 | 0.0004 | 8458 |
+| DeepRedact, default config, 1000 large objects | 7989.05 | 0.1251712531 | 0.0018 | 3995 |
+| fast redact, large object | 5929.58 | 0.1686459096 | 0.00126 | 2965 |
+| ObGlob, large object | 4939.98 | 0.2024300664 | 0.01105 | 2470 |
+| DeepRedact, case insensitive matching, single object | 4721.59 | 0.2117932541 | 0.00378 | 2361 |
+| DeepRedact, fuzzy and case insensitive matching, single object | 4686.29 | 0.2133882948 | 0.0018 | 2344 |
+| JSON.stringify, 1000 large objects | 222.05 | 4.5035912679 | 0.04553 | 112 |
+| ObGlob, 1000 large objects | 164.55 | 6.0771323614 | 0.11829 | 83 |
+| fast redact, 1000 large objects | 120.7 | 8.2848202295 | 0.05702 | 61 |
+| Regex replace, 1000 large objects | 93.5 | 10.6954525319 | 0.39014 | 47 |

package/dist/cjs/index.js

@@ -77,6 +77,24 @@ class DeepRedact {
                     },
                 };
             }
+            if (value instanceof Set) {
+                return {
+                    __unsupported: {
+                        type: 'set',
+                        values: Array.from(value),
+                    },
+                };
+            }
+            if (value instanceof Map) {
+                return {
+                    __unsupported: {
+                        type: 'map',
+                        entries: Object.fromEntries(value.entries()),
+                    },
+                };
+            }
+            if (value instanceof URL)
+                return value.toString();
             if (value instanceof Date)
                 return value.toISOString();
             return value;

package/dist/cjs/utils/redactorUtils.js

@@ -80,11 +80,32 @@ class RedactorUtils {
          * @param shouldRedact
          */
         this.redactString = (value, replacement, remove, shouldRedact) => {
-            if (!value)
+            if (!value || typeof value !== 'string')
                 return value;
             const { stringTests } = this.config;
-            if (!shouldRedact && !(stringTests === null || stringTests === void 0 ? void 0 : stringTests.some((pattern) => pattern.test(value))))
+            if (!shouldRedact) {
+                const result = stringTests === null || stringTests === void 0 ? void 0 : stringTests.map((test) => {
+                    if (test instanceof RegExp) {
+                        if (!test.test(value))
+                            return value;
+                        if (remove)
+                            return undefined;
+                        if (typeof replacement === 'function')
+                            return replacement(value);
+                        if (this.config.replaceStringByLength)
+                            return replacement.repeat(value.length);
+                        return replacement;
+                    }
+                    if (remove && test.pattern.test(value))
+                        return undefined;
+                    return test.replacer(value, test.pattern);
+                }).filter(Boolean)[0];
+                if (result)
+                    return result;
+                if (remove)
+                    return undefined;
                 return value;
+            }
             if (remove)
                 return undefined;
             if (typeof replacement === 'function')

package/dist/esm/index.mjs

@@ -72,6 +72,24 @@ class DeepRedact {
                 },
             };
         }
+        if (value instanceof Set) {
+            return {
+                __unsupported: {
+                    type: 'set',
+                    values: Array.from(value),
+                },
+            };
+        }
+        if (value instanceof Map) {
+            return {
+                __unsupported: {
+                    type: 'map',
+                    entries: Object.fromEntries(value.entries()),
+                },
+            };
+        }
+        if (value instanceof URL)
+            return value.toString();
         if (value instanceof Date)
             return value.toISOString();
         return value;

package/dist/esm/utils/{redactorUtils.js → redactorUtils.mjs}

@@ -131,11 +131,32 @@ class RedactorUtils {
      * @param shouldRedact
      */
     redactString = (value, replacement, remove, shouldRedact) => {
-        if (!value)
+        if (!value || typeof value !== 'string')
             return value;
         const { stringTests } = this.config;
-        if (!shouldRedact && !stringTests?.some((pattern) => pattern.test(value)))
+        if (!shouldRedact) {
+            const result = stringTests?.map((test) => {
+                if (test instanceof RegExp) {
+                    if (!test.test(value))
+                        return value;
+                    if (remove)
+                        return undefined;
+                    if (typeof replacement === 'function')
+                        return replacement(value);
+                    if (this.config.replaceStringByLength)
+                        return replacement.repeat(value.length);
+                    return replacement;
+                }
+                if (remove && test.pattern.test(value))
+                    return undefined;
+                return test.replacer(value, test.pattern);
+            }).filter(Boolean)[0];
+            if (result)
+                return result;
+            if (remove)
+                return undefined;
             return value;
+        }
         if (remove)
             return undefined;
         if (typeof replacement === 'function')

package/dist/types/types.d.ts

@@ -58,7 +58,10 @@ export interface BaseDeepRedactConfig {
      *   /^[\d]{1,3}\.[\d]{1,3}\.[\d]{1,3}\.[\d]{1,3}$/,  // redact any string that looks like an IP address.
      * ]
      */
-    stringTests?: RegExp[];
+    stringTests?: Array<RegExp | {
+        pattern: RegExp;
+        replacer: (value: string, pattern: RegExp) => string;
+    }>;
     /**
      * Perform a fuzzy match on the key. This will match any key that contains the string, rather than a case-sensitive match.
      * @default false

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hackylabs/deep-redact",
-  "version": "2.0.1",
+  "version": "2.1.0",
   "description": "A fast, safe and configurable zero-dependency library for redacting strings or deeply redacting arrays and objects.",
   "private": false,
   "license": "MIT",
@@ -9,7 +9,26 @@
   "types": "./dist/types/index.d.ts",
   "main": "./dist/cjs/index.js",
   "module": "./dist/esm/index.mjs",
+  "exports": {
+    ".": {
+      "import": "./dist/esm/index.mjs",
+      "require": "./dist/cjs/index.js",
+      "types": "./dist/types/index.d.ts"
+    },
+    "./utils/redactorUtils": {
+      "import": "./dist/esm/utils/redactorUtils.mjs",
+      "require": "./dist/cjs/utils/redactorUtils.js",
+      "types": "./dist/types/utils/redactorUtils.d.ts"
+    }
+  },
+  "files": [
+    "dist"
+  ],
   "keywords": [
+    "json",
+    "xml",
+    "document",
+    "fast",
     "redact",
     "redaction",
     "redactor",
@@ -26,19 +45,9 @@
     "GDPR",
     "PII"
   ],
-  "exports": {
-    ".": {
-      "import": "./dist/esm/index.js",
-      "require": "./dist/cjs/index.js",
-      "types": "./dist/types/index.d.ts"
-    }
-  },
-  "files": [
-    "dist"
-  ],
   "repository": {
     "type": "git",
-    "url": "https://github.com/hackylabs/deep-redact"
+    "url": "git+https://github.com/hackylabs/deep-redact"
   },
   "scripts": {
     "lint": "eslint",
@@ -53,10 +62,11 @@
     "update-license": "npx ts-node ./scripts/update-license.ts"
   },
   "//": [
-    "fast-redact installed only as a benchmark comparison and is not used in the library",
+    "fast-redact and obglob are installed only as benchmark comparisons and are not used in the library",
     "all dependencies are for development purposes only"
   ],
   "devDependencies": {
+    "@hackylabs/obglob": "1.1.2",
     "@memlab/core": "1.1.34",
     "@types/fast-redact": "3.0.4",
     "@types/node": "20.14.12",
@@ -71,7 +81,8 @@
     "eslint-plugin-promise": "7.0.0",
     "fast-redact": "3.5.0",
     "image-charts": "6.1.19",
-    "typescript": "5.5.4",
+    "superjson": "2.2.1",
+    "typescript": "5.6.2",
     "vitest": "2.0.4"
   }
 }