@sepiariver/unique-set 1.1.0 → 2.0.0

package/README.md

@@ -1,10 +1,76 @@
 # @sepiariver/unique-set
 
-Extends the `has` and `add` methods on the native JavaScript `Set` object to use [fast-deep-equal](https://www.npmjs.com/package/fast-deep-equal) as the equality algorithm.
+Extends the native `Set` class to deeply compare using [fast-deep-equal](https://www.npmjs.com/package/fast-deep-equal), with optional Bloom filter optimization.
 
-The extended methods iterate through the elements of the `UniqueSet` until equality is found. If no elements match, the entire `UniqueSet` would have been iterated to determine so. However fast `fast-deep-equal` is, calling it in a loop like this makes performance many times poorer than the native `Set`. For datasets greater than a thousand elements, there is probably a better way to achieve what you're trying to do. Otherwise, `UniqueSet` is convenient.
+Supports ESM and CommonJS.
 
-Requires @babel/core 7+
+WARNING: This version exports 2 classes instead of a single default class, breaking b/c with version 1.
+
+The extended methods iterate through the elements of the `UniqueSet` until equality is found. If no elements match, the entire `UniqueSet` would have been iterated. However fast `fast-deep-equal` is, calling it in a loop like this makes performance many, many times worse than the native `Set`. For datasets greater than a thousand elements, there is probably a better way to achieve what you're trying to do. Otherwise, `UniqueSet` is convenient.
+
+UPDATE: Version 2 ships with `BloomSet`, in which equality checks are optimized with a Bloom filter. This class is useful for larger datasets, performing about 3-10 times faster than `UniqueSet` for datasets greater than 1000 elements. Less than a few hundred (~400) elements, `UniqueSet` is faster. `BloomSet`'s probabilistic false positives are covered by a fallback to `fast-deep-equal`. BloomSet is still orders of magnitude slower than the native `Set`, but if deep equality is required, this is a decent option.
+
+Experiment with configurations to find the best performance for your use case.
+
+NOTE: The `delete` method is unmodified. In the case of duplicate objects that are equivalent but have different references, the results of `delete` operations may be unexpected.
+
+## Config Options
+
+### Constructor Signature
+
+`new BloomSet(iterable = [], options = { size, hashCount });`
+
+### Options
+
+The options object allows you to customize the behavior and performance of the BloomSet. The following properties can be configured:
+
+#### 1. size (number)
+
+Description: Specifies the size of the bit array used internally by the Bloom filter. This directly impacts the memory usage and false positive probability.
+
+Default: 6,553,577 (a prime number using roughly 800 KB of memory).
+
+Recommendations:
+
+For datasets with ~100,000 elements, this default size provides excellent performance (compared against `UniqueSet`) with minimal (< 1) false positives.
+
+Larger datasets may require increasing the size for lower false positive rates. Remember though, false positives are mitigated by a fallback to `fast-deep-equal`, so you may be able to squeeze more performance from a higher tolerance for false positives, depending on your dataset.
+
+#### 2. hashCount (number)
+
+Description: Specifies the number of hash functions used by the Bloom filter. This impacts both the false positive probability and the computational cost of adding/checking elements.
+
+Default: 7
+
+### Examples
+
+Default Configuration:
+
+```js
+const bloomSet = new BloomSet();
+bloomSet.add("example");
+console.log(bloomSet.has("example")); // true
+```
+
+Custom Configuration for Larger Datasets:
+
+Example 28,755,000 bit array size uses roughly 3.5 MB of memory, but this configuration is robust against datasets of something like 1M elements. The practicality of using BloomSet with that many elements is low, due to the performance hit of deep equality checks.
+
+```js
+const bloomSet = new BloomSet([], { size: 28755000, hashCount: 20 });
+bloomSet.add("custom");
+console.log(bloomSet.has("custom")); // true
+```
+
+### Considerations
+
+- Memory Usage: The bit array uses size / 8 bytes of memory. Even at 800 KB, initializing 1250 BloomSets in the same scope would use a gigabyte of memory.
+- False Positive Rate: The probability of a false positive is influenced by size, hashCount, and the number of elements. Adjust these values to balance performance and accuracy for your dataset.
+
+#### Further Tuning
+
+- Use a larger size for datasets exceeding 100,000 elements.
+- Reduce hashCount if performance is critical and your dataset contains very few duplicates.
 
 ## Installation
 
@@ -15,7 +81,7 @@ npm install @sepiariver/unique-set
 ## Usage
 
 ```js
-const UniqueSet = require('@sepiariver/unique-set');
+const { BloomSet, UniqueSet } = require('@sepiariver/unique-set');
 
 const data = [
   "string",
@@ -38,13 +104,21 @@ const data = [
   [1, 2, 3],
 ];
 
-let unique1 = new UniqueSet();
+const unique1 = new UniqueSet();
 data.forEach((el) => {
   unique1.add(el);
 });
-let unique2 = new UniqueSet(data);
+const unique2 = new UniqueSet(data);
 console.log(unique1.size); // 6 instead of 8 with Set
 console.log(unique2.size); // 6
+
+const bloom1 = new BloomSet();
+data.forEach((el) => {
+  bloom1.add(el);
+});
+const bloom2 = new BloomSet(data);
+console.log(bloom1.size); // 6 instead of 8 with Set
+console.log(bloom2.size); // 6
 ```
 
 ## Testing
@@ -55,4 +129,8 @@ console.log(unique2.size); // 6
 
 ## Contributing
 
-Submit pull requests to https://github.com/sepiariver/unique-set/pulls
+Submit pull requests to [https://github.com/sepiariver/unique-set/pulls]
+
+## Issues
+
+Issue reporting is encouraged: [https://github.com/sepiariver/unique-set/issues]

package/dist/index.js

@@ -1,12 +1,27 @@
 "use strict";
 
-function _typeof(obj) { "@babel/helpers - typeof"; return _typeof = "function" == typeof Symbol && "symbol" == typeof Symbol.iterator ? function (obj) { return typeof obj; } : function (obj) { return obj && "function" == typeof Symbol && obj.constructor === Symbol && obj !== Symbol.prototype ? "symbol" : typeof obj; }, _typeof(obj); }
+Object.defineProperty(exports, "__esModule", {
+  value: true
+});
+exports.UniqueSet = exports.BloomSet = void 0;
 
 var _fastDeepEqual = _interopRequireDefault(require("fast-deep-equal"));
 
 function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { "default": obj }; }
 
-function _createForOfIteratorHelper(o, allowArrayLike) { var it = typeof Symbol !== "undefined" && o[Symbol.iterator] || o["@@iterator"]; if (!it) { if (Array.isArray(o) || (it = _unsupportedIterableToArray(o)) || allowArrayLike && o && typeof o.length === "number") { if (it) o = it; var i = 0; var F = function F() {}; return { s: F, n: function n() { if (i >= o.length) return { done: true }; return { done: false, value: o[i++] }; }, e: function e(_e) { throw _e; }, f: F }; } throw new TypeError("Invalid attempt to iterate non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method."); } var normalCompletion = true, didErr = false, err; return { s: function s() { it = it.call(o); }, n: function n() { var step = it.next(); normalCompletion = step.done; return step; }, e: function e(_e2) { didErr = true; err = _e2; }, f: function f() { try { if (!normalCompletion && it["return"] != null) it["return"](); } finally { if (didErr) throw err; } } }; }
+function _slicedToArray(arr, i) { return _arrayWithHoles(arr) || _iterableToArrayLimit(arr, i) || _unsupportedIterableToArray(arr, i) || _nonIterableRest(); }
+
+function _nonIterableRest() { throw new TypeError("Invalid attempt to destructure non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method."); }
+
+function _iterableToArrayLimit(arr, i) { var _i = arr == null ? null : typeof Symbol !== "undefined" && arr[Symbol.iterator] || arr["@@iterator"]; if (_i == null) return; var _arr = []; var _n = true; var _d = false; var _s, _e; try { for (_i = _i.call(arr); !(_n = (_s = _i.next()).done); _n = true) { _arr.push(_s.value); if (i && _arr.length === i) break; } } catch (err) { _d = true; _e = err; } finally { try { if (!_n && _i["return"] != null) _i["return"](); } finally { if (_d) throw _e; } } return _arr; }
+
+function _arrayWithHoles(arr) { if (Array.isArray(arr)) return arr; }
+
+function _typeof(obj) { "@babel/helpers - typeof"; return _typeof = "function" == typeof Symbol && "symbol" == typeof Symbol.iterator ? function (obj) { return typeof obj; } : function (obj) { return obj && "function" == typeof Symbol && obj.constructor === Symbol && obj !== Symbol.prototype ? "symbol" : typeof obj; }, _typeof(obj); }
+
+function _readOnlyError(name) { throw new TypeError("\"" + name + "\" is read-only"); }
+
+function _createForOfIteratorHelper(o, allowArrayLike) { var it = typeof Symbol !== "undefined" && o[Symbol.iterator] || o["@@iterator"]; if (!it) { if (Array.isArray(o) || (it = _unsupportedIterableToArray(o)) || allowArrayLike && o && typeof o.length === "number") { if (it) o = it; var i = 0; var F = function F() {}; return { s: F, n: function n() { if (i >= o.length) return { done: true }; return { done: false, value: o[i++] }; }, e: function e(_e2) { throw _e2; }, f: F }; } throw new TypeError("Invalid attempt to iterate non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method."); } var normalCompletion = true, didErr = false, err; return { s: function s() { it = it.call(o); }, n: function n() { var step = it.next(); normalCompletion = step.done; return step; }, e: function e(_e3) { didErr = true; err = _e3; }, f: function f() { try { if (!normalCompletion && it["return"] != null) it["return"](); } finally { if (didErr) throw err; } } }; }
 
 function _unsupportedIterableToArray(o, minLen) { if (!o) return; if (typeof o === "string") return _arrayLikeToArray(o, minLen); var n = Object.prototype.toString.call(o).slice(8, -1); if (n === "Object" && o.constructor) n = o.constructor.name; if (n === "Map" || n === "Set") return Array.from(o); if (n === "Arguments" || /^(?:Ui|I)nt(?:8|16|32)(?:Clamped)?Array$/.test(n)) return _arrayLikeToArray(o, minLen); }
 
@@ -18,6 +33,10 @@ function _defineProperties(target, props) { for (var i = 0; i < props.length; i+
 
 function _createClass(Constructor, protoProps, staticProps) { if (protoProps) _defineProperties(Constructor.prototype, protoProps); if (staticProps) _defineProperties(Constructor, staticProps); Object.defineProperty(Constructor, "prototype", { writable: false }); return Constructor; }
 
+function _get() { if (typeof Reflect !== "undefined" && Reflect.get) { _get = Reflect.get; } else { _get = function _get(target, property, receiver) { var base = _superPropBase(target, property); if (!base) return; var desc = Object.getOwnPropertyDescriptor(base, property); if (desc.get) { return desc.get.call(arguments.length < 3 ? target : receiver); } return desc.value; }; } return _get.apply(this, arguments); }
+
+function _superPropBase(object, property) { while (!Object.prototype.hasOwnProperty.call(object, property)) { object = _getPrototypeOf(object); if (object === null) break; } return object; }
+
 function _inherits(subClass, superClass) { if (typeof superClass !== "function" && superClass !== null) { throw new TypeError("Super expression must either be null or a function"); } subClass.prototype = Object.create(superClass && superClass.prototype, { constructor: { value: subClass, writable: true, configurable: true } }); Object.defineProperty(subClass, "prototype", { writable: false }); if (superClass) _setPrototypeOf(subClass, superClass); }
 
 function _createSuper(Derived) { var hasNativeReflectConstruct = _isNativeReflectConstruct(); return function _createSuperInternal() { var Super = _getPrototypeOf(Derived), result; if (hasNativeReflectConstruct) { var NewTarget = _getPrototypeOf(this).constructor; result = Reflect.construct(Super, arguments, NewTarget); } else { result = Super.apply(this, arguments); } return _possibleConstructorReturn(this, result); }; }
@@ -44,33 +63,54 @@ var UniqueSet = /*#__PURE__*/function (_Set) {
   var _super = _createSuper(UniqueSet);
 
   function UniqueSet() {
+    var _this;
+
+    var iterable = arguments.length > 0 && arguments[0] !== undefined ? arguments[0] : [];
+
     _classCallCheck(this, UniqueSet);
 
-    for (var _len = arguments.length, args = new Array(_len), _key = 0; _key < _len; _key++) {
-      args[_key] = arguments[_key];
+    if (!Array.isArray(iterable) && !iterable[Symbol.iterator]) {
+      throw new TypeError("UniqueSet requires an iterable");
+    }
+
+    _this = _super.call(this);
+
+    var _iterator = _createForOfIteratorHelper(iterable),
+        _step;
+
+    try {
+      for (_iterator.s(); !(_step = _iterator.n()).done;) {
+        var item = _step.value;
+
+        _this.add(item);
+      }
+    } catch (err) {
+      _iterator.e(err);
+    } finally {
+      _iterator.f();
     }
 
-    return _super.call.apply(_super, [this].concat(args));
+    return _this;
   }
 
   _createClass(UniqueSet, [{
     key: "has",
     value: function has(o) {
-      var _iterator = _createForOfIteratorHelper(this),
-          _step;
+      var _iterator2 = _createForOfIteratorHelper(this),
+          _step2;
 
       try {
-        for (_iterator.s(); !(_step = _iterator.n()).done;) {
-          var i = _step.value;
+        for (_iterator2.s(); !(_step2 = _iterator2.n()).done;) {
+          var i = _step2.value;
 
           if ((0, _fastDeepEqual["default"])(o, i)) {
             return true;
           }
         }
       } catch (err) {
-        _iterator.e(err);
+        _iterator2.e(err);
       } finally {
-        _iterator.f();
+        _iterator2.f();
       }
 
       return false;
@@ -79,12 +119,271 @@ var UniqueSet = /*#__PURE__*/function (_Set) {
     key: "add",
     value: function add(o) {
       if (!this.has(o)) {
-        Set.prototype.add.call(this, o);
+        _get(_getPrototypeOf(UniqueSet.prototype), "add", this).call(this, o);
       }
+
+      return this;
     }
   }]);
 
   return UniqueSet;
 }( /*#__PURE__*/_wrapNativeSuper(Set));
 
-module.exports = UniqueSet;
+exports.UniqueSet = UniqueSet;
+
+var BloomSet = /*#__PURE__*/function (_Set2) {
+  _inherits(BloomSet, _Set2);
+
+  var _super2 = _createSuper(BloomSet);
+
+  function BloomSet() {
+    var _this2;
+
+    var iterable = arguments.length > 0 && arguments[0] !== undefined ? arguments[0] : [];
+    var options = arguments.length > 1 && arguments[1] !== undefined ? arguments[1] : {};
+
+    _classCallCheck(this, BloomSet);
+
+    if (!Array.isArray(iterable) && !iterable[Symbol.iterator]) {
+      throw new TypeError("BloomSet requires an iterable");
+    }
+
+    _this2 = _super2.call(this);
+
+    if (!options || _typeof(options) !== "object") {
+      options = {};
+    }
+
+    var _options = options,
+        _options$size = _options.size,
+        size = _options$size === void 0 ? 6553577 : _options$size,
+        _options$hashCount = _options.hashCount,
+        hashCount = _options$hashCount === void 0 ? 7 : _options$hashCount;
+
+    if (typeof size !== "number" || size <= 0) {
+      6553577, _readOnlyError("size"); // Targeting < 1 collision per 100,000 elements, ~819 KB memory, needs 7 hashes
+    }
+
+    _this2.aSize = _this2._findNextPrime(size);
+
+    if (typeof hashCount !== "number" || hashCount <= 0) {
+      7, _readOnlyError("hashCount");
+    }
+
+    _this2.hashCount = hashCount;
+    _this2.bitArray = new Uint8Array(Math.ceil(size / 8));
+
+    var _iterator3 = _createForOfIteratorHelper(iterable),
+        _step3;
+
+    try {
+      for (_iterator3.s(); !(_step3 = _iterator3.n()).done;) {
+        var item = _step3.value;
+
+        _this2.add(item);
+      }
+    } catch (err) {
+      _iterator3.e(err);
+    } finally {
+      _iterator3.f();
+    }
+
+    return _this2;
+  }
+
+  _createClass(BloomSet, [{
+    key: "_findNextPrime",
+    value: function _findNextPrime(num) {
+      if (num < 2) return 2;
+      if (num % 2 === 0) num++; // Odd numbers only
+
+      while (!this._isPrime(num)) {
+        num += 2; // Odd numbers only
+      }
+
+      return num;
+    }
+  }, {
+    key: "_isPrime",
+    value: function _isPrime(num) {
+      if (num < 2) return false;
+      if (num === 2 || num === 3) return true;
+      if (num % 2 === 0 || num % 3 === 0) return false;
+      var sqrt = Math.floor(Math.sqrt(num));
+
+      for (var i = 5; i <= sqrt; i += 6) {
+        if (num % i === 0 || num % (i + 2) === 0) return false;
+      }
+
+      return true;
+    }
+  }, {
+    key: "_serialize",
+    value: function _serialize(item) {
+      if (typeof item === "number" && isNaN(item)) {
+        return "NaN";
+      }
+
+      if (item && _typeof(item) === "object") {
+        var serialize = this._serialize.bind(this);
+
+        if (Array.isArray(item)) {
+          return "[".concat(item.map(serialize).join(","), "]");
+        } else {
+          return "{".concat(Object.entries(item).sort(function (_ref, _ref2) {
+            var _ref3 = _slicedToArray(_ref, 1),
+                a = _ref3[0];
+
+            var _ref4 = _slicedToArray(_ref2, 1),
+                b = _ref4[0];
+
+            return a.localeCompare(b);
+          }).map(function (_ref5) {
+            var _ref6 = _slicedToArray(_ref5, 2),
+                k = _ref6[0],
+                v = _ref6[1];
+
+            return "".concat(k, ":").concat(serialize(v));
+          }).join(","), "}");
+        }
+      }
+
+      return String(item);
+    }
+  }, {
+    key: "_hashes",
+    value: function _hashes(item) {
+      var hashes = [];
+
+      var str = this._serialize(item);
+
+      var hash = this._fnv1a(str); // Base hash
+      // Bloom into hashCount hash values
+
+
+      for (var i = 0; i < this.hashCount; i++) {
+        hash %= this.aSize; // Ensure within bounds
+        // Track
+
+        hashes.push(hash); // Modify
+
+        hash = (hash ^ hash >>> 13) * 0xc2b2ae35;
+        hash >>>= 0; // Ensure unsigned 32-bit integer
+      }
+
+      return hashes;
+    }
+  }, {
+    key: "_fnv1a",
+    value: function _fnv1a(str) {
+      if (typeof str !== "string") {
+        str = String(str);
+      }
+
+      var hash = 2166136261; // FNV offset basis for 32-bit
+
+      for (var i = 0; i < str.length; i++) {
+        hash ^= str.charCodeAt(i);
+        hash = hash * 16777619 >>> 0; // Multiply by the FNV prime and ensure 32-bit unsigned
+      }
+
+      return hash >>> 0;
+    }
+  }, {
+    key: "_setBits",
+    value: function _setBits(hashes) {
+      var _iterator4 = _createForOfIteratorHelper(hashes),
+          _step4;
+
+      try {
+        for (_iterator4.s(); !(_step4 = _iterator4.n()).done;) {
+          var hash = _step4.value;
+          var index = Math.floor(hash / 8);
+          var bit = hash % 8;
+          this.bitArray[index] |= 1 << bit;
+        }
+      } catch (err) {
+        _iterator4.e(err);
+      } finally {
+        _iterator4.f();
+      }
+    }
+  }, {
+    key: "_checkBits",
+    value: function _checkBits(hashes) {
+      var _iterator5 = _createForOfIteratorHelper(hashes),
+          _step5;
+
+      try {
+        for (_iterator5.s(); !(_step5 = _iterator5.n()).done;) {
+          var hash = _step5.value;
+          var index = Math.floor(hash / 8);
+          var bit = hash % 8;
+
+          if (!(this.bitArray[index] & 1 << bit)) {
+            return false;
+          }
+        }
+      } catch (err) {
+        _iterator5.e(err);
+      } finally {
+        _iterator5.f();
+      }
+
+      return true;
+    }
+  }, {
+    key: "has",
+    value: function has(o) {
+      var hashes = this._hashes(o);
+
+      if (!this._checkBits(hashes)) {
+        return false; // Definitely not in the set
+      } // Fall back to fast-deep-equal for false positives
+
+
+      var _iterator6 = _createForOfIteratorHelper(this),
+          _step6;
+
+      try {
+        for (_iterator6.s(); !(_step6 = _iterator6.n()).done;) {
+          var i = _step6.value;
+
+          if ((0, _fastDeepEqual["default"])(o, i)) {
+            return true;
+          }
+        }
+      } catch (err) {
+        _iterator6.e(err);
+      } finally {
+        _iterator6.f();
+      }
+
+      return false;
+    }
+  }, {
+    key: "add",
+    value: function add(o) {
+      if (!this.has(o)) {
+        var hashes = this._hashes(o);
+
+        this._setBits(hashes);
+
+        _get(_getPrototypeOf(BloomSet.prototype), "add", this).call(this, o);
+      }
+
+      return this;
+    }
+  }]);
+
+  return BloomSet;
+}( /*#__PURE__*/_wrapNativeSuper(Set));
+
+exports.BloomSet = BloomSet;
+
+if (typeof module !== "undefined" && module.exports) {
+  module.exports = {
+    UniqueSet: UniqueSet,
+    BloomSet: BloomSet
+  };
+}

package/index.d.ts

@@ -0,0 +1,60 @@
+declare module "unique-set" {
+  /**
+   * A `Set` extension that ensures uniqueness of items using deep equality checks.
+   */
+  export class UniqueSet<T> extends Set<T> {
+    /**
+     * Creates a new `UniqueSet` instance.
+     * @param iterable Optional: an iterable with which to initialize the UniqueSet.
+     * @throws TypeError If the input is not iterable.
+     */
+    constructor(iterable?: Iterable<T>);
+
+    /**
+     * Determines whether an object is in the UniqueSet using deep equality.
+     * @param o The object to check for presence in the UniqueSet.
+     * @returns `true` if the object is found, `false` otherwise.
+     */
+    has(o: T): boolean;
+
+    /**
+     * Adds a new object to the UniqueSet if it is not already present.
+     * @param o The object to add to the UniqueSet.
+     * @returns The `UniqueSet` instance, allowing for chaining.
+     */
+    add(o: T): this;
+  }
+
+  /**
+   * A `Set` extension that uses a Bloom filter for fast existence checks combined with deep equality for accuracy.
+   */
+  export class BloomSet<T> extends Set<T> {
+    /**
+     * Creates a new `BloomSet` instance.
+     * @param iterable Optional: an iterable object with which to initialize the BloomSet.
+     * @param options Bloom filter configuration options.
+     * @param options.size The size of the Bloom filter's bit array. Defaults to 6553577.
+     * @param options.hashCount The number of hash functions to use. Defaults to 7.
+     * @throws TypeError If the input is not iterable.
+     */
+    constructor(
+      iterable?: Iterable<T>,
+      options?: { size?: number; hashCount?: number }
+    );
+
+    /**
+     * Determines existence of an object in the BloomSet using the Bloom filter and deep equality.
+     * @param o The object to check for presence in the BloomSet.
+     * @returns `true` if the object is found, `false` otherwise.
+     */
+    has(o: T): boolean;
+
+    /**
+     * Adds a new object to the BloomSet if it is not already present.
+     * @param o The object to add to the BloomSet.
+     * @returns The `BloomSet` instance, allowing for chaining.
+     */
+    add(o: T): this;
+  }
+
+}

package/package.json

@@ -1,8 +1,14 @@
 {
   "name": "@sepiariver/unique-set",
-  "version": "1.1.0",
-  "description": "Extends the has and add methods on the native JavaScript Set object to deeply compare using fast-deep-equal",
+  "version": "2.0.0",
+  "description": "Extends the native Set class to deeply compare using fast-deep-equal, with optional Bloom filter optimization. This version exports 2 classes instead of a default, breaking b/c with version 1.",
   "main": "index.js",
+  "module": "dist/index.js",
+  "types": "index.d.ts",
+  "exports": {
+    "require": "./index.js",
+    "import": "./dist/index.js"
+  },
   "scripts": {
     "test": "jest",
     "build": "babel src -d dist"
@@ -14,7 +20,7 @@
   "keywords": [
     "Set",
     "unique",
-    "object",
+    "bloom",
     "deep",
     "compare",
     "equal"
@@ -29,8 +35,8 @@
     "fast-deep-equal": "^3.1.3"
   },
   "devDependencies": {
-    "@babel/cli": "^7.17.0",
-    "@babel/core": "^7.17.2",
+    "@babel/cli": "^7.17.6",
+    "@babel/core": "^7.17.8",
     "@babel/preset-env": "^7.16.11",
     "babel-jest": "^27.5.1",
     "jest": "^27.5.1"

package/src/index.js

@@ -1,9 +1,16 @@
 import equal from "fast-deep-equal";
 
-class UniqueSet extends Set {
-  constructor(...args) {
-    super(...args);
+export class UniqueSet extends Set {
+  constructor(iterable = []) {
+    if (!Array.isArray(iterable) && !iterable[Symbol.iterator]) {
+      throw new TypeError("UniqueSet requires an iterable");
+    }
+    super();
+    for (const item of iterable) {
+      this.add(item);
+    }
   }
+
   has(o) {
     for (const i of this) {
       if (equal(o, i)) {
@@ -12,11 +19,165 @@ class UniqueSet extends Set {
     }
     return false;
   }
+
   add(o) {
     if (!this.has(o)) {
-      Set.prototype.add.call(this, o);
+      super.add(o);
     }
+    return this;
   }
 }
 
-module.exports = UniqueSet;
+export class BloomSet extends Set {
+  constructor(iterable = [], options = {}) {
+    if (!Array.isArray(iterable) && !iterable[Symbol.iterator]) {
+      throw new TypeError("BloomSet requires an iterable");
+    }
+    super();
+
+    if (!options || typeof options !== "object") {
+      options = {};
+    }
+
+    const { size = 6553577, hashCount = 7 } = options;
+
+    if (typeof size !== "number" || size <= 0) {
+      size = 6553577; // Targeting < 1 collision per 100,000 elements, ~819 KB memory, needs 7 hashes
+    }
+    this.aSize = this._findNextPrime(size);
+
+    if (typeof hashCount !== "number" || hashCount <= 0) {
+      hashCount = 7;
+    }
+    this.hashCount = hashCount;
+
+    this.bitArray = new Uint8Array(Math.ceil(size / 8));
+
+    for (const item of iterable) {
+      this.add(item);
+    }
+  }
+
+  _findNextPrime(num) {
+    if (num < 2) return 2;
+    if (num % 2 === 0) num++; // Odd numbers only
+
+    while (!this._isPrime(num)) {
+      num += 2; // Odd numbers only
+    }
+
+    return num;
+  }
+
+  _isPrime(num) {
+    if (num < 2) return false;
+    if (num === 2 || num === 3) return true;
+    if (num % 2 === 0 || num % 3 === 0) return false;
+
+    const sqrt = Math.floor(Math.sqrt(num));
+    for (let i = 5; i <= sqrt; i += 6) {
+      if (num % i === 0 || num % (i + 2) === 0) return false;
+    }
+
+    return true;
+  }
+
+  _serialize(item) {
+    if (typeof item === "number" && isNaN(item)) {
+      return "NaN";
+    }
+
+    if (item && typeof item === "object") {
+      const serialize = this._serialize.bind(this);
+      if (Array.isArray(item)) {
+        return `[${item.map(serialize).join(",")}]`;
+      } else {
+        return `{${Object.entries(item)
+          .sort(([a], [b]) => a.localeCompare(b))
+          .map(([k, v]) => `${k}:${serialize(v)}`)
+          .join(",")}}`;
+      }
+    }
+
+    return String(item);
+  }
+
+  _hashes(item) {
+    const hashes = [];
+    const str = this._serialize(item);
+    let hash = this._fnv1a(str); // Base hash
+
+    // Bloom into hashCount hash values
+    for (let i = 0; i < this.hashCount; i++) {
+      hash %= this.aSize; // Ensure within bounds
+      // Track
+      hashes.push(hash);
+      // Modify
+      hash = (hash ^ (hash >>> 13)) * 0xc2b2ae35;
+      hash >>>= 0; // Ensure unsigned 32-bit integer
+    }
+
+    return hashes;
+  }
+
+  _fnv1a(str) {
+    if (typeof str !== "string") {
+      str = String(str);
+    }
+    let hash = 2166136261; // FNV offset basis for 32-bit
+    for (let i = 0; i < str.length; i++) {
+      hash ^= str.charCodeAt(i);
+      hash = (hash * 16777619) >>> 0; // Multiply by the FNV prime and ensure 32-bit unsigned
+    }
+    return hash >>> 0;
+  }
+
+  _setBits(hashes) {
+    for (const hash of hashes) {
+      const index = Math.floor(hash / 8);
+      const bit = hash % 8;
+      this.bitArray[index] |= 1 << bit;
+    }
+  }
+
+  _checkBits(hashes) {
+    for (const hash of hashes) {
+      const index = Math.floor(hash / 8);
+      const bit = hash % 8;
+      if (!(this.bitArray[index] & (1 << bit))) {
+        return false;
+      }
+    }
+    return true;
+  }
+
+  has(o) {
+    const hashes = this._hashes(o);
+    if (!this._checkBits(hashes)) {
+      return false; // Definitely not in the set
+    }
+    // Fall back to fast-deep-equal for false positives
+    for (const i of this) {
+      if (equal(o, i)) {
+        return true;
+      }
+    }
+    return false;
+  }
+
+  add(o) {
+    if (!this.has(o)) {
+      const hashes = this._hashes(o);
+      this._setBits(hashes);
+      super.add(o);
+    }
+    return this;
+  }
+}
+
+if (typeof module !== "undefined" && module.exports) {
+  module.exports = {
+    UniqueSet,
+    BloomSet,
+  };
+}