@sepiariver/unique-set 3.1.2 → 3.2.0

package/PERF.md

@@ -2,42 +2,44 @@
 
 Benchmarks run with `npm run bench` on Node.js v20.18.1, Apple M2 Pro.
 
+Comparison between **UniqueSet** (`@sepiariver/unique-set`), **DeepSet** (`deep-equality-data-structures`), and the native `Set`.
+
 ## Flat Data (`bench.spec.ts`)
 
 Mixed strings, flat objects (2 keys), and 2-element arrays with ~10-15% duplicate rate.
 
-|   Items | MapSet `add()` | Native Set | Overhead |
-| ------: | -------------: | ---------: | -------: |
-|     400 |        1.23 ms |    0.04 ms |     ~31x |
-|   1,000 |        1.46 ms |    0.18 ms |      ~8x |
-|  20,000 |       13.09 ms |    1.46 ms |      ~9x |
-| 100,000 |       54.08 ms |    6.44 ms |      ~8x |
+|   Items | UniqueSet |   DeepSet | Native Set |
+| ------: | --------: | --------: | ---------: |
+|     400 |   0.96 ms |   8.13 ms |    0.07 ms |
+|   1,000 |   0.60 ms |   9.33 ms |    0.08 ms |
+|  20,000 |   6.12 ms | 116.00 ms |    1.00 ms |
+| 100,000 |  20.83 ms | 517.00 ms |    4.15 ms |
 
 ## Nested Data (`bench-nested.spec.ts`)
 
 Deeply nested objects (3-4 levels), nested arrays with objects, and mixed structures.
 
-### `add()` - insert all items
+### `add()` — insert all items
 
-|   Items |   MapSet | Native Set | Overhead |
-| ------: | -------: | ---------: | -------: |
-|     400 |  1.80 ms |    0.03 ms |     ~60x |
-|   1,000 |  3.01 ms |    0.05 ms |     ~60x |
-|  20,000 | 14.23 ms |    0.89 ms |     ~16x |
-| 100,000 | 80.60 ms |    3.80 ms |     ~21x |
+|   Items | UniqueSet |     DeepSet | Native Set |
+| ------: | --------: | ----------: | ---------: |
+|     400 |   1.90 ms |    14.88 ms |    0.05 ms |
+|   1,000 |   0.72 ms |    22.39 ms |    0.05 ms |
+|  20,000 |   9.73 ms |   423.00 ms |    0.89 ms |
+| 100,000 |  57.18 ms | 2,130.00 ms |    3.88 ms |
 
-### `has()` - query all items (50% hits, 50% misses)
+### `has()` — query all items (50% hits, 50% misses)
 
-|   Items | `has()` time | Queries |   Hits | per query |
-| ------: | -----------: | ------: | -----: | --------: |
-|     400 |      0.96 ms |     457 |    228 |   ~2.1 us |
-|   1,000 |      0.95 ms |   1,144 |    572 |   ~0.8 us |
-|  20,000 |     16.86 ms |  22,892 | 11,446 |   ~0.7 us |
-| 100,000 |     73.89 ms | 114,458 | 57,229 |   ~0.6 us |
+|   Items | UniqueSet.has() | DeepSet.has() | Queries |   Hits |
+| ------: | --------------: | ------------: | ------: | -----: |
+|     400 |         0.63 ms |       8.93 ms |     457 |    228 |
+|   1,000 |         0.73 ms |      21.69 ms |   1,144 |    572 |
+|  20,000 |        12.83 ms |     425.00 ms |  22,892 | 11,446 |
+| 100,000 |        61.26 ms |   2,111.00 ms | 114,458 | 57,229 |
 
 ## Notes
 
-- Native `Set` uses reference equality and cannot deduplicate objects/arrays by value. The overhead shown is the cost of deep value comparison.
-- MapSet uses a streaming 32-bit FNV-1a structural hash with `fast-deep-equal` fallback for hash collisions.
-- At 20,000 items, ~47 hash collisions are expected (birthday paradox). These are handled correctly with no impact on results.
-- The `has()` cost per query decreases at larger sizes due to V8 JIT warmup.
+- **Native `Set`** uses reference equality and cannot deduplicate objects/arrays by value.
+- **UniqueSet** (this package) uses a streaming 32-bit FNV-1a structural hash with `fast-equals` only as fallback for hash collisions. O(1) average for both `add()` and `has()`.
+- **DeepSet** (`deep-equality-data-structures`) hashes values with `object-hash` (MD5 by default) for O(1) lookups. The performance gap comes from MD5 being a cryptographic hash and `object-hash` serializing values before hashing.
+- UniqueSet is roughly **25–35x faster** than DeepSet on nested data at scale, while both produce identical deduplication results.

package/README.md

@@ -1,6 +1,6 @@
 # @sepiariver/unique-set
 
-Uses a streaming structural hash to optimize deep equality checks in a `Set`-like class. Falls back to deeply compare using [fast-deep-equal](https://www.npmjs.com/package/fast-deep-equal) when hash collisions occur.
+Unique set is highly-performant, given the workload. It uses a streaming structural hash to optimize deep equality checks. Falls back to deeply compare using [fast-equals](https://www.npmjs.com/package/fast-equals) only when hash collisions occur.
 
 Supports ESM and CommonJS. Thanks [@sakgoyal](https://github.com/sakgoyal) for contributing to and instigating ESM support.
 
@@ -16,7 +16,7 @@ WARNING: Version 3 includes breaking changes. Older versions are deprecated.
 
 Configuration options from previous versions are no longer supported. Usage is identical to the native `Set` class.
 
-IMPORTANT: `MapSet` and `UniqueSet` are the same class (`UniqueSet` is an alias). The `delete` method uses deep equality, so `delete({a: 1})` will remove a previously added `{a: 1}` even if it's a different reference.
+IMPORTANT: `MapSet` and `UniqueSet` are the same class (`UniqueSet` is an alias). The `delete` method uses deep equality, so `delete({a: 1})` will remove a previously added `{a: 1}` even if it's a different reference. See "Considerations" below for more details on equality semantics.
 
 ## API
 
@@ -63,9 +63,16 @@ set.size; // 1
 
 ### Considerations
 
+- **Performance**: See [PERF.md](PERF.md) for benchmarks. UniqueSet is optimized for deep equality with O(1) average complexity for both `add()` and `has()`, performing _25-35x faster_ than other deep equality `Set`-like implementations, especially on nested data at scale.
 - **Memory**: Each unique value is stored once, bucketed by a 32-bit structural hash. Overhead is minimal: one `Map` entry plus a small array per hash bucket, with >99% of buckets containing exactly one item at typical sizes.
-- **Collisions**: At 20,000 items, roughly 47 hash collisions are expected (birthday paradox on 32-bit). Collisions are handled correctly via `fast-deep-equal`. They add a small cost but never affect correctness.
-- **Equality semantics**: Object key order is ignored. Array element order matters. `NaN === NaN`. `0` and `-0` are treated as equal. Functions and symbols are compared by reference.
+- **Collisions**: At 20,000 items, roughly 47 hash collisions are expected (birthday paradox on 32-bit). Collisions are handled correctly via `fast-equals`. They add a small cost but never affect correctness.
+- **Equality semantics**: Both the structural hash and `fast-equals` use deep value comparison throughout, so they are fully aligned.
+  - **Plain objects**: Key order is ignored.
+  - **Arrays**: Element order matters (hash is sequential; equality is index-by-index).
+  - **`Set` values**: Insertion order is ignored. `new Set([1, 2])` and `new Set([2, 1])` are treated as equal, including Sets containing objects (both layers use deep comparison).
+  - **`Map` values**: Insertion order is ignored. Both keys and values are compared by deep equality.
+  - **Primitives**: `NaN === NaN`. `0` and `-0` are treated as equal.
+  - **Functions and symbols**: Compared by reference. They hash by their string representation (`String(value)`), so same-source functions may land in the same bucket, but `fast-equals` uses `===` for the final check.
 
 ## Installation
 

package/dist/index.js

@@ -1,9 +1,7 @@
 "use strict";
-var __create = Object.create;
 var __defProp = Object.defineProperty;
 var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
 var __getOwnPropNames = Object.getOwnPropertyNames;
-var __getProtoOf = Object.getPrototypeOf;
 var __hasOwnProp = Object.prototype.hasOwnProperty;
 var __export = (target, all) => {
   for (var name in all)
@@ -17,14 +15,6 @@ var __copyProps = (to, from, except, desc) => {
   }
   return to;
 };
-var __toESM = (mod, isNodeMode, target) => (target = mod != null ? __create(__getProtoOf(mod)) : {}, __copyProps(
-  // If the importer is in node compatibility mode or this is not an ESM
-  // file that has been converted to a CommonJS file using a Babel-
-  // compatible transform (i.e. "__esModule" has not been set), then set
-  // "default" to the CommonJS "module.exports" for node compatibility.
-  isNodeMode || !mod || !mod.__esModule ? __defProp(target, "default", { value: mod, enumerable: true }) : target,
-  mod
-));
 var __toCommonJS = (mod) => __copyProps(__defProp({}, "__esModule", { value: true }), mod);
 
 // index.ts
@@ -35,7 +25,7 @@ __export(unique_set_exports, {
   structuralHash: () => structuralHash
 });
 module.exports = __toCommonJS(unique_set_exports);
-var import_es6 = __toESM(require("fast-deep-equal/es6/index.js"));
+var import_fast_equals = require("fast-equals");
 var _f64 = new Float64Array(1);
 var _u8 = new Uint8Array(_f64.buffer);
 var structuralHash = (value) => {
@@ -82,20 +72,20 @@ var _shash = (value, hash) => {
     return hash;
   }
   if (value instanceof Map) {
-    hash = _mix(hash, 17);
-    const entries = Array.from(value.entries()).sort(
-      ([a], [b]) => String(a).localeCompare(String(b))
-    );
-    for (const [k, v] of entries) {
-      hash = _shash(k, hash);
-      hash = _shash(v, hash);
+    let mapHash = 0;
+    for (const [k, v] of value) {
+      let entryHash = _shash(k, 2166136261);
+      entryHash = _shash(v, entryHash);
+      mapHash = mapHash + entryHash | 0;
     }
-    return hash;
+    return _mix(hash, mapHash);
   }
   if (value instanceof Set) {
-    hash = _mix(hash, 18);
-    for (const v of value) hash = _shash(v, hash);
-    return hash;
+    let setHash = 0;
+    for (const v of value) {
+      setHash = setHash + _shash(v, 2166136261) | 0;
+    }
+    return _mix(hash, setHash);
   }
   if (value instanceof Date) {
     hash = _mix(hash, 20);
@@ -108,12 +98,15 @@ var _shash = (value, hash) => {
     return _mixStr(hash, value.toString());
   }
   hash = _mix(hash, 19);
-  const keys = Object.keys(value).sort();
-  for (const key of keys) {
-    hash = _mixStr(hash, key);
-    hash = _shash(value[key], hash);
-  }
-  return hash;
+  let objHash = 0;
+  const keys = Object.keys(value);
+  for (let i = 0; i < keys.length; i++) {
+    const key = keys[i];
+    let pairHash = _mixStr(2166136261, key);
+    pairHash = _shash(value[key], pairHash);
+    objHash = objHash + pairHash | 0;
+  }
+  return _mix(hash, objHash);
 };
 var MapSet = class {
   #map;
@@ -136,7 +129,7 @@ var MapSet = class {
       this.#size++;
     } else {
       for (const item of bucket) {
-        if ((0, import_es6.default)(value, item)) return this;
+        if ((0, import_fast_equals.deepEqual)(value, item)) return this;
       }
       bucket.push(value);
       this.#size++;
@@ -148,7 +141,7 @@ var MapSet = class {
     const bucket = this.#map.get(hash);
     if (!bucket) return false;
     for (const item of bucket) {
-      if ((0, import_es6.default)(value, item)) return true;
+      if ((0, import_fast_equals.deepEqual)(value, item)) return true;
     }
     return false;
   }
@@ -157,7 +150,7 @@ var MapSet = class {
     const bucket = this.#map.get(hash);
     if (!bucket) return false;
     for (let i = 0; i < bucket.length; i++) {
-      if ((0, import_es6.default)(value, bucket[i])) {
+      if ((0, import_fast_equals.deepEqual)(value, bucket[i])) {
         bucket.splice(i, 1);
         if (bucket.length === 0) this.#map.delete(hash);
         this.#size--;

package/dist/index.mjs

@@ -1,5 +1,5 @@
 // index.ts
-import equal from "fast-deep-equal/es6/index.js";
+import { deepEqual as equal } from "fast-equals";
 var _f64 = new Float64Array(1);
 var _u8 = new Uint8Array(_f64.buffer);
 var structuralHash = (value) => {
@@ -46,20 +46,20 @@ var _shash = (value, hash) => {
     return hash;
   }
   if (value instanceof Map) {
-    hash = _mix(hash, 17);
-    const entries = Array.from(value.entries()).sort(
-      ([a], [b]) => String(a).localeCompare(String(b))
-    );
-    for (const [k, v] of entries) {
-      hash = _shash(k, hash);
-      hash = _shash(v, hash);
+    let mapHash = 0;
+    for (const [k, v] of value) {
+      let entryHash = _shash(k, 2166136261);
+      entryHash = _shash(v, entryHash);
+      mapHash = mapHash + entryHash | 0;
     }
-    return hash;
+    return _mix(hash, mapHash);
   }
   if (value instanceof Set) {
-    hash = _mix(hash, 18);
-    for (const v of value) hash = _shash(v, hash);
-    return hash;
+    let setHash = 0;
+    for (const v of value) {
+      setHash = setHash + _shash(v, 2166136261) | 0;
+    }
+    return _mix(hash, setHash);
   }
   if (value instanceof Date) {
     hash = _mix(hash, 20);
@@ -72,12 +72,15 @@ var _shash = (value, hash) => {
     return _mixStr(hash, value.toString());
   }
   hash = _mix(hash, 19);
-  const keys = Object.keys(value).sort();
-  for (const key of keys) {
-    hash = _mixStr(hash, key);
-    hash = _shash(value[key], hash);
-  }
-  return hash;
+  let objHash = 0;
+  const keys = Object.keys(value);
+  for (let i = 0; i < keys.length; i++) {
+    const key = keys[i];
+    let pairHash = _mixStr(2166136261, key);
+    pairHash = _shash(value[key], pairHash);
+    objHash = objHash + pairHash | 0;
+  }
+  return _mix(hash, objHash);
 };
 var MapSet = class {
   #map;

package/index.ts

@@ -1,4 +1,4 @@
-import equal from "fast-deep-equal/es6/index.js";
+import { deepEqual as equal } from "fast-equals";
 
 /** 
  * Streaming structural hash — computes a 32-bit FNV-1a hash by traversing
@@ -56,20 +56,20 @@ const _shash = (value: unknown, hash: number): number => {
     return hash;
   }
   if (value instanceof Map) {
-    hash = _mix(hash, 0x11);
-    const entries = Array.from(value.entries()).sort(([a], [b]) =>
-      String(a).localeCompare(String(b))
-    );
-    for (const [k, v] of entries) {
-      hash = _shash(k, hash);
-      hash = _shash(v, hash);
+    let mapHash = 0;
+    for (const [k, v] of value) {
+      let entryHash = _shash(k, 0x811c9dc5);
+      entryHash = _shash(v, entryHash);
+      mapHash = (mapHash + entryHash) | 0; // order-independent hash by summing entry hashes (32-bit)
     }
-    return hash;
+    return _mix(hash, mapHash);
   }
   if (value instanceof Set) {
-    hash = _mix(hash, 0x12);
-    for (const v of value) hash = _shash(v, hash);
-    return hash;
+    let setHash = 0;
+    for (const v of value) {
+      setHash = (setHash + _shash(v, 0x811c9dc5)) | 0; // order-independent hash by summing element hashes (32-bit)
+    }
+    return _mix(hash, setHash);
   }
   if (value instanceof Date) {
     hash = _mix(hash, 0x14);
@@ -82,14 +82,18 @@ const _shash = (value: unknown, hash: number): number => {
     return _mixStr(hash, value.toString());
   }
 
-  // Plain object — sort keys for order-independence
+  // Plain object: order-independent
   hash = _mix(hash, 0x13);
-  const keys = Object.keys(value as object).sort();
-  for (const key of keys) {
-    hash = _mixStr(hash, key);
-    hash = _shash((value as Record<string, unknown>)[key], hash);
+  let objHash = 0;
+  const keys = Object.keys(value as object);
+  for (let i = 0; i < keys.length; i++) {
+    const key = keys[i]!;
+    let pairHash = _mixStr(0x811c9dc5, key);
+    pairHash = _shash((value as Record<string, unknown>)[key], pairHash);
+    objHash = (objHash + pairHash) | 0;
   }
-  return hash;
+
+  return _mix(hash, objHash);
 };
 
 export class MapSet<T> {

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@sepiariver/unique-set",
-  "version": "3.1.2",
-  "description": "A Set-like collection that deduplicates by deep value equality using a streaming structural hash and fast-deep-equal.",
+  "version": "3.2.0",
+  "description": "A Set-like collection that deduplicates by deep value equality using a streaming structural hash and fast-equals.",
   "main": "dist/index.js",
   "module": "dist/index.mjs",
   "types": "dist/index.d.mts",
@@ -35,9 +35,10 @@
   },
   "homepage": "https://github.com/sepiariver/unique-set#readme",
   "dependencies": {
-    "fast-deep-equal": "^3.1.3"
+    "fast-equals": "^6.0.0"
   },
   "devDependencies": {
+    "deep-equality-data-structures": "^2.0.0",
     "tsup": "^8.3.5",
     "typescript": "^5.7.2",
     "vitest": "^2.1.8"