re2 1.22.2 → 1.23.0

package/README.md

@@ -3,12 +3,13 @@
 [npm-img]: https://img.shields.io/npm/v/re2.svg
 [npm-url]: https://npmjs.org/package/re2
 
-This project provides bindings for [RE2](https://github.com/google/re2):
-fast, safe alternative to backtracking regular expression engines written by [Russ Cox](http://swtch.com/~rsc/).
+This project provides Node.js bindings for [RE2](https://github.com/google/re2):
+a fast, safe alternative to backtracking regular expression engines written by [Russ Cox](http://swtch.com/~rsc/) in C++.
 To learn more about RE2, start with an overview
 [Regular Expression Matching in the Wild](http://swtch.com/~rsc/regexp/regexp3.html). More resources can be found
 at his [Implementing Regular Expressions](http://swtch.com/~rsc/regexp/) page.
 
+
 `RE2`'s regular expression language is almost a superset of what is provided by `RegExp`
 (see [Syntax](https://github.com/google/re2/wiki/Syntax)),
 but it lacks two features: backreferences and lookahead assertions. See below for more details.
@@ -24,7 +25,9 @@ See `RE2.unicodeWarningLevel` below for more details.
 It can work with [node.js buffers](http://nodejs.org/api/buffer.html) directly reducing overhead
 on recoding and copying characters, and making processing/parsing long files fast.
 
-All documentation can be found in this README and in the [wiki](https://github.com/uhop/node-re2/wiki).
+This project is implemented in C++ using [nan](https://github.com/nodejs/nan) for Node.js and cannot be used
+with non-compliant runtimes like web browsers. All documentation can be found in this README and in
+the [wiki](https://github.com/uhop/node-re2/wiki).
 
 ## Why use node-re2?
 
@@ -42,6 +45,12 @@ However, neither project is perfect.
 node-re2 can protect your Node.js application from ReDoS.
 node-re2 makes vulnerable regular expression patterns safe by evaluating them in `RE2` instead of the built-in Node.js regex engine.
 
+To run the bundled benchmark, use the following command (make sure that node-re2 is properly built before):
+
+```bash
+npx nano-bench bench/bad-pattern.mjs
+```
+
 ## Standard features
 
 `RE2` object can be created just like `RegExp`:
@@ -165,6 +174,49 @@ RE2("б").replace("абв", bufReplacer);
 This feature works for string and buffer inputs. If a buffer was used as an input, its output will be returned as
 a buffer too, otherwise a string will be returned.
 
+### `RE2.Set`
+
+Starting with 1.23.0, use `RE2.Set` when the same string must be tested against many patterns. It builds a single automaton
+for all of them and frequently beats running a large list of individual regular expressions one by one.
+
+Sets support `test()` and `match()` methods. While `test()` can be simulated by combining patterns with `|` and using a regular expression object,
+`match()` is not because it returns a list of patterns that matched, which is not possible with a regular expression object.
+Parsing data against multiple choices is a frequent operation in the wild and `RE2.Set` is a fast way to do it.
+
+* `new RE2.Set(patterns[, flagsOrOptions][, options])`
+  * `patterns` is any iterable of strings, `Buffer`s, `RegExp`, or `RE2` instances; flags (if provided) apply to the whole set.
+  * `flagsOrOptions` can be a string/`Buffer` with standard flags (`i`, `m`, `s`, `u`, `g`, `y`, `d`).
+  * `options.anchor` can be `'unanchored'` (default), `'start'`, or `'both'`.
+* `set.test(str)` returns `true` if any pattern matches and `false` otherwise.
+* `set.match(str)` returns an array of indexes of matching patterns.
+  * This is an array of integer indices of patterns that matched sorted in ascending order.
+  * If no patterns matched, an empty array is returned.
+* Read-only properties:
+  * `set.size` (number of patterns), `set.flags` (`RegExp` flags as a string), `set.anchor` (anchor mode as a string)
+  * `set.source` (all patterns joined with `|` as a string), `set.sources` (individual pattern sources as an array of strings)
+
+It is based on [RE2::Set](https://github.com/google/re2/blob/main/re2/set.h).
+
+Example:
+
+```js
+const routes = new RE2.Set([
+  '^/users/\\d+$',
+  '^/posts/\\d+$'
+], 'i', {anchor: 'start'});
+
+routes.test('/users/7');     // true
+routes.match('/posts/42');   // [1]
+routes.sources;              // ['^/users/\\d+$', '^/posts/\\d+$']
+routes.toString();           // '/^/users/\\d+$|^/posts/\\d+$/iu'
+```
+
+To run the bundled benchmark, use the following command (make sure that node-re2 is properly built before):
+
+```bash
+npx nano-bench bench/set-match.mjs
+```
+
 ### Calculate length
 
 Two functions to calculate string sizes between
@@ -207,11 +259,13 @@ In the latter case `RE2` can do following actions depending on `RE2.unicodeWarni
 
 Warnings and exceptions help to audit an application for stray non-Unicode regular expressions.
 
+`RE2.unicodeWarningLevel` is a global property. Be careful manipulating it in a multi-threaded environment as it is shared between threads.
+
 ## How to install
 
 Installation:
 
-```
+```bash
 npm install --save re2
 ```
 
@@ -362,6 +416,8 @@ The same applies to `\P{...}`.
 
 ## Release history
 
+- 1.23.0 *Updated all dependencies, upgraded tooling. New feature: `RE2.Set` (thx, [Wes](https://github.com/wrmedford)).*
+- 1.22.3 *Technical release: upgraded QEMU emulations to native ARM runners to speed up the build process.*
 - 1.22.2 *Updated all dependencies and the list of pre-compiled targets: Node 20, 22, 24, 25 (thx, [Jiayu Liu](https://github.com/jimexist)).*
 - 1.22.1 *Added support for translation of scripts as Unicode classes.*
 - 1.22.0 *Added support for translation of Unicode classes (thx, [John Livingston](https://github.com/JohnXLivingston)). Added [attestations](https://github.com/uhop/node-re2/attestations).*

package/binding.gyp

@@ -5,6 +5,7 @@
       "sources": [
         "lib/addon.cc",
         "lib/accessors.cc",
+        "lib/pattern.cc",
         "lib/util.cc",
         "lib/new.cc",
         "lib/exec.cc",
@@ -14,6 +15,7 @@
         "lib/search.cc",
         "lib/split.cc",
         "lib/to_string.cc",
+        "lib/set.cc",
         "vendor/re2/re2/bitmap256.cc",
         "vendor/re2/re2/bitstate.cc",
         "vendor/re2/re2/compile.cc",
@@ -47,6 +49,8 @@
         "vendor/abseil-cpp/absl/base/internal/thread_identity.cc",
         "vendor/abseil-cpp/absl/base/internal/throw_delegate.cc",
         "vendor/abseil-cpp/absl/base/internal/unscaledcycleclock.cc",
+        "vendor/abseil-cpp/absl/container/internal/hashtablez_sampler.cc",
+        "vendor/abseil-cpp/absl/container/internal/hashtablez_sampler_force_weak_definition.cc",
         "vendor/abseil-cpp/absl/container/internal/raw_hash_set.cc",
         "vendor/abseil-cpp/absl/debugging/internal/decode_rust_punycode.cc",
         "vendor/abseil-cpp/absl/debugging/internal/demangle.cc",
@@ -68,7 +72,6 @@
         "vendor/abseil-cpp/absl/flags/usage_config.cc",
         "vendor/abseil-cpp/absl/hash/internal/city.cc",
         "vendor/abseil-cpp/absl/hash/internal/hash.cc",
-        "vendor/abseil-cpp/absl/hash/internal/low_level_hash.cc",
         "vendor/abseil-cpp/absl/log/internal/globals.cc",
         "vendor/abseil-cpp/absl/log/internal/log_format.cc",
         "vendor/abseil-cpp/absl/log/internal/log_message.cc",
@@ -90,6 +93,7 @@
         "vendor/abseil-cpp/absl/strings/internal/str_format/float_conversion.cc",
         "vendor/abseil-cpp/absl/strings/internal/str_format/output.cc",
         "vendor/abseil-cpp/absl/strings/internal/str_format/parser.cc",
+        "vendor/abseil-cpp/absl/strings/internal/utf8.cc",
         "vendor/abseil-cpp/absl/strings/match.cc",
         "vendor/abseil-cpp/absl/strings/numbers.cc",
         "vendor/abseil-cpp/absl/strings/str_cat.cc",

package/lib/addon.cc

@@ -1,4 +1,5 @@
 #include "./wrapped_re2.h"
+#include "./wrapped_re2_set.h"
 
 static NAN_METHOD(GetUtf8Length)
 {
@@ -75,6 +76,9 @@ v8::Local<v8::Function> WrappedRE2::Init()
 	Nan::SetAccessor(instanceTemplate, Nan::New("internalSource").ToLocalChecked(), GetInternalSource);
 
 	auto ctr = Nan::GetFunction(tpl).ToLocalChecked();
+	auto setCtr = WrappedRE2Set::Init();
+
+	Nan::Set(ctr, Nan::New("Set").ToLocalChecked(), setCtr);
 
 	// properties
 

package/lib/new.cc

@@ -1,5 +1,6 @@
 #include "./wrapped_re2.h"
 #include "./util.h"
+#include "./pattern.h"
 
 #include <map>
 #include <memory>
@@ -7,252 +8,6 @@
 #include <unordered_set>
 #include <vector>
 
-static char hex[] = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F'};
-
-inline bool isUpperCaseAlpha(char ch)
-{
-	return 'A' <= ch && ch <= 'Z';
-}
-
-inline bool isHexadecimal(char ch)
-{
-	return ('0' <= ch && ch <= '9') || ('A' <= ch && ch <= 'Z') || ('a' <= ch && ch <= 'z');
-}
-
-static std::map<std::string, std::string> unicodeClasses = {
-	{"Uppercase_Letter", "Lu"},
-	{"Lowercase_Letter", "Ll"},
-	{"Titlecase_Letter", "Lt"},
-	{"Cased_Letter", "LC"},
-	{"Modifier_Letter", "Lm"},
-	{"Other_Letter", "Lo"},
-	{"Letter", "L"},
-	{"Nonspacing_Mark", "Mn"},
-	{"Spacing_Mark", "Mc"},
-	{"Enclosing_Mark", "Me"},
-	{"Mark", "M"},
-	{"Decimal_Number", "Nd"},
-	{"Letter_Number", "Nl"},
-	{"Other_Number", "No"},
-	{"Number", "N"},
-	{"Connector_Punctuation", "Pc"},
-	{"Dash_Punctuation", "Pd"},
-	{"Open_Punctuation", "Ps"},
-	{"Close_Punctuation", "Pe"},
-	{"Initial_Punctuation", "Pi"},
-	{"Final_Punctuation", "Pf"},
-	{"Other_Punctuation", "Po"},
-	{"Punctuation", "P"},
-	{"Math_Symbol", "Sm"},
-	{"Currency_Symbol", "Sc"},
-	{"Modifier_Symbol", "Sk"},
-	{"Other_Symbol", "So"},
-	{"Symbol", "S"},
-	{"Space_Separator", "Zs"},
-	{"Line_Separator", "Zl"},
-	{"Paragraph_Separator", "Zp"},
-	{"Separator", "Z"},
-	{"Control", "Cc"},
-	{"Format", "Cf"},
-	{"Surrogate", "Cs"},
-	{"Private_Use", "Co"},
-	{"Unassigned", "Cn"},
-	{"Other", "C"},
-};
-
-static bool translateRegExp(const char *data, size_t size, bool multiline, std::vector<char> &buffer)
-{
-	std::string result;
-	bool changed = false;
-
-	if (!size)
-	{
-		result = "(?:)";
-		changed = true;
-	}
-	else if (multiline)
-	{
-		result = "(?m)";
-		changed = true;
-	}
-
-	for (size_t i = 0; i < size;)
-	{
-		char ch = data[i];
-		if (ch == '\\')
-		{
-			if (i + 1 < size)
-			{
-				ch = data[i + 1];
-				switch (ch)
-				{
-				case '\\':
-					result += "\\\\";
-					i += 2;
-					continue;
-				case 'c':
-					if (i + 2 < size)
-					{
-						ch = data[i + 2];
-						if (isUpperCaseAlpha(ch))
-						{
-							result += "\\x";
-							result += hex[((ch - '@') / 16) & 15];
-							result += hex[(ch - '@') & 15];
-							i += 3;
-							changed = true;
-							continue;
-						}
-					}
-					result += "\\c";
-					i += 2;
-					continue;
-				case 'u':
-					if (i + 2 < size)
-					{
-						ch = data[i + 2];
-						if (isHexadecimal(ch))
-						{
-							result += "\\x{";
-							result += ch;
-							i += 3;
-							for (size_t j = 0; j < 3 && i < size; ++i, ++j)
-							{
-								ch = data[i];
-								if (!isHexadecimal(ch))
-								{
-									break;
-								}
-								result += ch;
-							}
-							result += '}';
-							changed = true;
-							continue;
-						}
-						else if (ch == '{')
-						{
-							result += "\\x";
-							i += 2;
-							changed = true;
-							continue;
-						}
-					}
-					result += "\\u";
-					i += 2;
-					continue;
-				case 'p':
-				case 'P':
-					if (i + 2 < size) {
-						if (data[i + 2] == '{') {
-							size_t j = i + 3;
-							while (j < size && data[j] != '}') ++j;
-							if (j < size) {
-								result += "\\";
-								result += data[i + 1];
-								std::string name(data + i + 3, j - i - 3);
-								if (unicodeClasses.find(name) != unicodeClasses.end()) {
-									name = unicodeClasses[name];
-								} else if (name.size() > 7 && !strncmp(name.c_str(), "Script=", 7)) {
-									name = name.substr(7);
-								} else if (name.size() > 3 && !strncmp(name.c_str(), "sc=", 3)) {
-									name = name.substr(3);
-								}
-								if (name.size() == 1) {
-									result += name;
-								} else {
-									result += "{";
-									result += name;
-									result += "}";
-								}
-								i = j + 1;
-								changed = true;
-								continue;
-							}
-						}
-					}
-					result += "\\";
-					result += data[i + 1];
-					i += 2;
-					continue;
-				default:
-					result += "\\";
-					size_t sym_size = getUtf8CharSize(ch);
-					result.append(data + i + 1, sym_size);
-					i += sym_size + 1;
-					continue;
-				}
-			}
-		}
-		else if (ch == '/')
-		{
-			result += "\\/";
-			i += 1;
-			changed = true;
-			continue;
-		}
-		else if (ch == '(' && i + 2 < size && data[i + 1] == '?' && data[i + 2] == '<')
-		{
-			if (i + 3 >= size || (data[i + 3] != '=' && data[i + 3] != '!'))
-			{
-				result += "(?P<";
-				i += 3;
-				changed = true;
-				continue;
-			}
-		}
-		size_t sym_size = getUtf8CharSize(ch);
-		result.append(data + i, sym_size);
-		i += sym_size;
-	}
-
-	if (!changed)
-	{
-		return false;
-	}
-
-	buffer.resize(0);
-	buffer.insert(buffer.end(), result.data(), result.data() + result.size());
-	buffer.push_back('\0');
-
-	return true;
-}
-
-static std::string escapeRegExp(const char *data, size_t size)
-{
-	std::string result;
-
-	if (!size)
-	{
-		result = "(?:)";
-	}
-
-	size_t prevBackSlashes = 0;
-	for (size_t i = 0; i < size;)
-	{
-		char ch = data[i];
-		if (ch == '\\')
-		{
-			++prevBackSlashes;
-		}
-		else if (ch == '/' && !(prevBackSlashes & 1))
-		{
-			result += "\\/";
-			i += 1;
-			prevBackSlashes = 0;
-			continue;
-		}
-		else
-		{
-			prevBackSlashes = 0;
-		}
-		size_t sym_size = getUtf8CharSize(ch);
-		result.append(data + i, sym_size);
-		i += sym_size;
-	}
-
-	return result;
-}
-
 bool WrappedRE2::alreadyWarnedAboutUnicode = false;
 
 static const char *deprecationMessage = "BMP patterns aren't supported by node-re2. An implicit \"u\" flag is assumed by the RE2 constructor. In a future major version, calling the RE2 constructor without the \"u\" flag may become forbidden, or cause a different behavior. Please see https://github.com/uhop/node-re2/issues/21 for more information.";

package/lib/pattern.cc

@@ -0,0 +1,252 @@
+#include "./pattern.h"
+#include "./wrapped_re2.h"
+
+#include <cstring>
+#include <map>
+#include <string>
+
+static char hex[] = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F'};
+
+inline bool isUpperCaseAlpha(char ch)
+{
+	return 'A' <= ch && ch <= 'Z';
+}
+
+inline bool isHexadecimal(char ch)
+{
+	return ('0' <= ch && ch <= '9') || ('A' <= ch && ch <= 'Z') || ('a' <= ch && ch <= 'z');
+}
+
+static std::map<std::string, std::string> unicodeClasses = {
+	{"Uppercase_Letter", "Lu"},
+	{"Lowercase_Letter", "Ll"},
+	{"Titlecase_Letter", "Lt"},
+	{"Cased_Letter", "LC"},
+	{"Modifier_Letter", "Lm"},
+	{"Other_Letter", "Lo"},
+	{"Letter", "L"},
+	{"Nonspacing_Mark", "Mn"},
+	{"Spacing_Mark", "Mc"},
+	{"Enclosing_Mark", "Me"},
+	{"Mark", "M"},
+	{"Decimal_Number", "Nd"},
+	{"Letter_Number", "Nl"},
+	{"Other_Number", "No"},
+	{"Number", "N"},
+	{"Connector_Punctuation", "Pc"},
+	{"Dash_Punctuation", "Pd"},
+	{"Open_Punctuation", "Ps"},
+	{"Close_Punctuation", "Pe"},
+	{"Initial_Punctuation", "Pi"},
+	{"Final_Punctuation", "Pf"},
+	{"Other_Punctuation", "Po"},
+	{"Punctuation", "P"},
+	{"Math_Symbol", "Sm"},
+	{"Currency_Symbol", "Sc"},
+	{"Modifier_Symbol", "Sk"},
+	{"Other_Symbol", "So"},
+	{"Symbol", "S"},
+	{"Space_Separator", "Zs"},
+	{"Line_Separator", "Zl"},
+	{"Paragraph_Separator", "Zp"},
+	{"Separator", "Z"},
+	{"Control", "Cc"},
+	{"Format", "Cf"},
+	{"Surrogate", "Cs"},
+	{"Private_Use", "Co"},
+	{"Unassigned", "Cn"},
+	{"Other", "C"},
+};
+
+bool translateRegExp(const char *data, size_t size, bool multiline, std::vector<char> &buffer)
+{
+	std::string result;
+	bool changed = false;
+
+	if (!size)
+	{
+		result = "(?:)";
+		changed = true;
+	}
+	else if (multiline)
+	{
+		result = "(?m)";
+		changed = true;
+	}
+
+	for (size_t i = 0; i < size;)
+	{
+		char ch = data[i];
+		if (ch == '\\')
+		{
+			if (i + 1 < size)
+			{
+				ch = data[i + 1];
+				switch (ch)
+				{
+				case '\\':
+					result += "\\\\";
+					i += 2;
+					continue;
+				case 'c':
+					if (i + 2 < size)
+					{
+						ch = data[i + 2];
+						if (isUpperCaseAlpha(ch))
+						{
+							result += "\\x";
+							result += hex[((ch - '@') / 16) & 15];
+							result += hex[(ch - '@') & 15];
+							i += 3;
+							changed = true;
+							continue;
+						}
+					}
+					result += "\\c";
+					i += 2;
+					continue;
+				case 'u':
+					if (i + 2 < size)
+					{
+						ch = data[i + 2];
+						if (isHexadecimal(ch))
+						{
+							result += "\\x{";
+							result += ch;
+							i += 3;
+							for (size_t j = 0; j < 3 && i < size; ++i, ++j)
+							{
+								ch = data[i];
+								if (!isHexadecimal(ch))
+								{
+									break;
+								}
+								result += ch;
+							}
+							result += '}';
+							changed = true;
+							continue;
+						}
+						else if (ch == '{')
+						{
+							result += "\\x";
+							i += 2;
+							changed = true;
+							continue;
+						}
+					}
+					result += "\\u";
+					i += 2;
+					continue;
+				case 'p':
+				case 'P':
+					if (i + 2 < size) {
+						if (data[i + 2] == '{') {
+							size_t j = i + 3;
+							while (j < size && data[j] != '}') ++j;
+							if (j < size) {
+								result += "\\";
+								result += data[i + 1];
+								std::string name(data + i + 3, j - i - 3);
+								if (unicodeClasses.find(name) != unicodeClasses.end()) {
+									name = unicodeClasses[name];
+								} else if (name.size() > 7 && !strncmp(name.c_str(), "Script=", 7)) {
+									name = name.substr(7);
+								} else if (name.size() > 3 && !strncmp(name.c_str(), "sc=", 3)) {
+									name = name.substr(3);
+								}
+								if (name.size() == 1) {
+									result += name;
+								} else {
+									result += "{";
+									result += name;
+									result += "}";
+								}
+								i = j + 1;
+								changed = true;
+								continue;
+							}
+						}
+					}
+					result += "\\";
+					result += data[i + 1];
+					i += 2;
+					continue;
+				default:
+					result += "\\";
+					size_t sym_size = getUtf8CharSize(ch);
+					result.append(data + i + 1, sym_size);
+					i += sym_size + 1;
+					continue;
+				}
+			}
+		}
+		else if (ch == '/')
+		{
+			result += "\\/";
+			i += 1;
+			changed = true;
+			continue;
+		}
+		else if (ch == '(' && i + 2 < size && data[i + 1] == '?' && data[i + 2] == '<')
+		{
+			if (i + 3 >= size || (data[i + 3] != '=' && data[i + 3] != '!'))
+			{
+				result += "(?P<";
+				i += 3;
+				changed = true;
+				continue;
+			}
+		}
+		size_t sym_size = getUtf8CharSize(ch);
+		result.append(data + i, sym_size);
+		i += sym_size;
+	}
+
+	if (!changed)
+	{
+		return false;
+	}
+
+	buffer.resize(0);
+	buffer.insert(buffer.end(), result.data(), result.data() + result.size());
+	buffer.push_back('\0');
+
+	return true;
+}
+
+std::string escapeRegExp(const char *data, size_t size)
+{
+	std::string result;
+
+	if (!size)
+	{
+		result = "(?:)";
+	}
+
+	size_t prevBackSlashes = 0;
+	for (size_t i = 0; i < size;)
+	{
+		char ch = data[i];
+		if (ch == '\\')
+		{
+			++prevBackSlashes;
+		}
+		else if (ch == '/' && !(prevBackSlashes & 1))
+		{
+			result += "\\/";
+			i += 1;
+			prevBackSlashes = 0;
+			continue;
+		}
+		else
+		{
+			prevBackSlashes = 0;
+		}
+		size_t sym_size = getUtf8CharSize(ch);
+		result.append(data + i, sym_size);
+		i += sym_size;
+	}
+
+	return result;
+}

package/lib/pattern.h

@@ -0,0 +1,10 @@
+#pragma once
+
+#include <string>
+#include <vector>
+
+// Shared helpers for translating JavaScript-style regular expressions
+// into RE2-compatible patterns.
+bool translateRegExp(const char *data, size_t size, bool multiline, std::vector<char> &buffer);
+std::string escapeRegExp(const char *data, size_t size);
+