trilean-sql 1.2.0 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -45,6 +45,9 @@ const columns = {
45
45
 
46
46
  const { sql, params } = compilePredicateNode(rule, {
47
47
  dialect: "postgres",
48
+ // `matches` is refused by default -- see "Regular expressions" -- and opted into here because
49
+ // "@example[.]com$" uses nothing outside the syntax portable between ECMAScript and PostgreSQL.
50
+ postgresRegexpPushdown: true,
48
51
  columnFor: (key) => {
49
52
  const binding = columns[key as keyof typeof columns];
50
53
  if (binding === undefined) throw new Error(`no column for ${key}`);
@@ -73,6 +76,8 @@ Returns `{ sql, params }`. Throws rather than approximating; see [Refusal](#refu
73
76
 
74
77
  `paramType` declares the column's value kind: `"text"`, `"number"`, `"boolean"`, or `"timestamp"` (trilean's `instant`). It is optional and worth supplying — it does not change the emitted SQL, but it is the only thing that lets the compiler detect the operand-kind mismatches described under [Refusal](#refusal). Without it, a comparison the database would coerce into a definite answer where trilean returns `indeterminate` compiles silently.
75
78
 
79
+ `options.postgresRegexpPushdown`, `false` by default, governs whether `matches`/`notMatches` may compile to PostgreSQL's own `~`/`!~` at all. See [Regular expressions](#regular-expressions) for why the default refuses them.
80
+
76
81
  ### `findUnpushableNodeKind(node, options?)`
77
82
 
78
83
  Returns `{ kind, path, reason }` for the first node the compiler will not translate, or `undefined` if the whole tree is pushable. `compilePredicateNode` runs it first and throws on any result, so call it yourself only to *choose* between pushdown and in-process evaluation without provoking an exception:
@@ -97,7 +102,7 @@ Passing `options` widens the check: without them the walk is purely structural;
97
102
  | `and`, `or`, `not` | `AND`, `OR`, `NOT` | same |
98
103
  | `allOf`, `anyOf` | n-ary `AND`, `OR`; empty operands become each connective's identity, `TRUE` and `FALSE`, matching the evaluator's own fold | same |
99
104
  | `compare` | `>`, `>=`, `<`, `<=`, `=`, `<>` | same |
100
- | `textCompare` | `=`, `<>`, and `~` / `!~` for `matches` / `notMatches` | `=`, `<>`, and `REGEXP` / `NOT REGEXP` |
105
+ | `textCompare` | `=`, `<>`; `matches`/`notMatches` only once `postgresRegexpPushdown` opts in, otherwise refused (see [Regular expressions](#regular-expressions)) | `=`, `<>`, and `REGEXP` / `NOT REGEXP` |
101
106
  | `memberOf` | `IN` / `NOT IN`, one parameter per candidate | same |
102
107
  | `exists` | `IS NOT NULL` | same |
103
108
  | `reference` | the mapped column, as a quoted identifier | same |
@@ -136,7 +141,9 @@ The compiler never degrades. There is no best-effort fragment, no silently dropp
136
141
 
137
142
  The check is an allow-list walk rather than a deny-list, so a node kind added to trilean after this version was written is refused by default instead of falling through to whatever branch happened to be last.
138
143
 
139
- That guarantee is about the tree's structure, and it has two limits, both about the *content* of a string operand rather than any node's kind, and neither reachable by a walk over kinds. A `matches`/`notMatches` pattern is matched by the server in PostgreSQL's own regular-expression language, not ECMAScript's — see [Regular expressions](#regular-expressions), which is where the row-for-row claim actually stops. An `instantLiteral` is parsed by PostgreSQL rather than by `Date`, so one carrying no UTC offset is read in the *database session's* time zone where trilean reads it in the Node process's, and one PostgreSQL cannot parse at all raises a query error at execution time rather than this exception (trilean answers `indeterminate` for the same string). Pass instants as offset-bearing ISO-8601, which both read identically.
144
+ That guarantee is about the tree's structure, and it has one remaining limit that is about the *content* of a string operand rather than any node's kind, and is not reachable by a walk over kinds: an `instantLiteral` is parsed by PostgreSQL rather than by `Date`, so one carrying no UTC offset is read in the *database session's* time zone where trilean reads it in the Node process's, and one PostgreSQL cannot parse at all raises a query error at execution time rather than this exception (trilean answers `indeterminate` for the same string). Pass instants as offset-bearing ISO-8601, which both read identically.
145
+
146
+ A `matches`/`notMatches` pattern would be the same kind of limit — matched by the server in PostgreSQL's own regular-expression language, not ECMAScript's — which is why it is refused rather than pushed down by default; see [Regular expressions](#regular-expressions). Setting `postgresRegexpPushdown: true` opts back into that limit deliberately, for patterns already known to be portable.
140
147
 
141
148
  Every refusal below applies to both dialects. What changes with the dialect is the `reason` text, which names the mechanism that actually applies to the engine you are compiling for — a `findUnpushableNodeKind` call given no `options` has no dialect to read and describes PostgreSQL, the one these refusals were first derived against.
142
149
 
@@ -155,7 +162,16 @@ Left undeclared, these compile, and the divergence is real but invisible. That i
155
162
 
156
163
  ## Regular expressions
157
164
 
158
- Under PostgreSQL, `matches` and `notMatches` compile to `~` and `!~`, so the pattern is matched by the server. PostgreSQL's advanced regular expressions and ECMAScript's `RegExp` are close but not the same language: shorthand classes and lookahead exist in both, and much everyday pattern syntax is portable, but they are separate implementations with their own escapes, quantifier subtleties and matching rules. A pattern that relies on ECMAScript-specific behaviour may match differently once pushed down. Keep patterns to portable syntax, or evaluate them in process.
165
+ Under PostgreSQL, `matches` and `notMatches` *can* compile to `~` and `!~`, so the pattern is matched by the server rather than in process — but only once `postgresRegexpPushdown: true` is passed in `SqlCompileOptions`. Left at its default (`false`, or simply omitted), both are refused by `findUnpushableNodeKind`/`compilePredicateNode` and the caller evaluates them in process instead, exactly like any other unpushable node.
166
+
167
+ This is the one node kind refused by default for a reason that is not structural: trilean's evaluator matches a pattern as an ECMAScript `RegExp`, and PostgreSQL's `~`/`!~` match under PostgreSQL's own regular-expression dialect (POSIX Advanced Regular Expressions, "ARE"). The two are close — literals, character classes, basic quantifiers, alternation, non-greedy quantifiers, numbered backreferences, and even lookahead/lookbehind are portable — but they are separate implementations, and measuring them directly (PostgreSQL's own docs against MDN's `RegExp` reference, each point checked against a real PostgreSQL engine via PGlite rather than trusted from documentation alone) turns up real divergence in both directions:
168
+
169
+ - **Silent divergence — the compiled fragment executes cleanly under both engines and still answers a different row set for the same pattern and data, with no error anywhere to catch it.** `.` matches a newline under PostgreSQL's default (newline-insensitive) mode; ECMAScript's `.` does not unless the pattern carries the `s` flag, which trilean's evaluator never sets (`new RegExp(pattern)`, no flags) — so `a.b` matches `"a\nb"` under PostgreSQL and not under trilean. `\w`/`\d`/`\s` (and their negations) are locale-dependent under PostgreSQL — in a UTF8 database, `\w` matches an accented letter like `é` — while ECMAScript's are ASCII-only by default (no `u` flag) — so `^\w+$` matches `"café"` under PostgreSQL and not under trilean. Neither is a syntax difference a validator could reject; the identical pattern text is valid and meaningful in both languages, just with different meaning.
170
+ - **Loud divergence — the pattern compiles clean here and fails only when PostgreSQL executes the query.** Named capture groups (`(?<name>...)`) and named backreferences (`\k<name>`) are ECMAScript syntax with no ARE equivalent (`invalid regular expression: quantifier operand invalid` from PostgreSQL, since ARE reads `(?<` as the start of a lookbehind and then finds no valid lookbehind body). Unicode property escapes (`\p{...}`/`\P{...}`) are likewise ECMAScript-only (`invalid regular expression: invalid escape \ sequence`). Character class subtraction (`[a-z-[aeiou]]`, from other regex dialects, not valid ECMAScript either but sometimes reached for) also fails under ARE.
171
+
172
+ Given the silent class exists at all, a validated "portable subset" of pattern syntax cannot close this gap on its own: the offending constructs here are not unusual syntax to ban, they are two of the most ordinary regex idioms there are (`.` and `\w`), and the difference is in what the database's locale and newline-handling settings mean, not in what characters appear in the pattern text. That is why this compiles to in-process evaluation by default rather than to a syntax-restricted pushdown — and why opting in with `postgresRegexpPushdown: true` is a deliberate, whole-compilation decision rather than a per-pattern one: it is only safe once every pattern reaching this option is known, by the caller, to avoid both the syntax PostgreSQL cannot parse at all and the constructs (`.` against data that may contain newlines, `\w`/`\d`/`\s` against data that may contain non-ASCII text) whose meaning quietly differs.
173
+
174
+ Sources: PostgreSQL's own [pattern-matching documentation](https://www.postgresql.org/docs/current/functions-matching.html) (POSIX ARE support, its embedded-option flags, and its own comparison against XQuery's regular expressions); MDN's [regular expressions reference](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Regular_expressions) for ECMAScript's semantics; the divergence points above were each executed against a real PostgreSQL engine (`@electric-sql/pglite`) and Node's own `RegExp`, not inferred from documentation alone.
159
175
 
160
176
  Under SQLite there is no built-in regular-expression support at all. `REGEXP` is reserved syntax for a `regexp(pattern, value)` function the connection has to register itself — `X REGEXP Y` is exactly `regexp(Y, X)`, pattern first — and the dialect emits `REGEXP` / `NOT REGEXP` for the same two operators. An unregistered one is a query error, `no such function: REGEXP`, rather than a fragment that quietly matches nothing, so this is a documented environment requirement in the same class as the caveats above and not a hole in the compile-time guarantee. The upside is that the pattern is then matched by your own `RegExp`, so the ECMAScript-versus-server-dialect divergence above does not arise.
161
177
 
package/dist/compile.cjs CHANGED
@@ -132,6 +132,7 @@ function compilePredicateNode(node, options) {
132
132
  const bindings = /* @__PURE__ */ new Map();
133
133
  const memoised = {
134
134
  dialect: options.dialect,
135
+ postgresRegexpPushdown: options.postgresRegexpPushdown,
135
136
  columnFor: (referenceKey) => {
136
137
  const cached = bindings.get(referenceKey);
137
138
  if (cached !== void 0) return cached;
package/dist/compile.js CHANGED
@@ -131,6 +131,7 @@ function compilePredicateNode(node, options) {
131
131
  const bindings = /* @__PURE__ */ new Map();
132
132
  const memoised = {
133
133
  dialect: options.dialect,
134
+ postgresRegexpPushdown: options.postgresRegexpPushdown,
134
135
  columnFor: (referenceKey) => {
135
136
  const cached = bindings.get(referenceKey);
136
137
  if (cached !== void 0) return cached;
package/dist/guard.cjs CHANGED
@@ -211,6 +211,11 @@ function findUnpushablePredicate(node, path, options, divergence) {
211
211
  reason: `'textCompare' requires text operands in trilean, and the operand at ${operand.path} is a '${staticKind}' value`
212
212
  };
213
213
  }
214
+ if ((options?.dialect ?? "postgres") === "postgres" && (node.op === "matches" || node.op === "notMatches") && options?.postgresRegexpPushdown !== true) return {
215
+ kind: node.kind,
216
+ path,
217
+ reason: `'${node.op}' matches trilean's own ECMAScript 'RegExp' against the pattern, whereas PostgreSQL's '~'/'!~' would match it under PostgreSQL's own regular-expression dialect (POSIX ARE) -- a different language that can silently answer a different set of rows for the same pattern and data (for example, PostgreSQL's '.' matches a newline by default where ECMAScript's does not, and PostgreSQL's '\\w' follows the database's locale where ECMAScript's is ASCII-only) -- so this is refused unless 'postgresRegexpPushdown' is explicitly set 'true'`
218
+ };
214
219
  return;
215
220
  }
216
221
  case "memberOf": {
package/dist/guard.js CHANGED
@@ -210,6 +210,11 @@ function findUnpushablePredicate(node, path, options, divergence) {
210
210
  reason: `'textCompare' requires text operands in trilean, and the operand at ${operand.path} is a '${staticKind}' value`
211
211
  };
212
212
  }
213
+ if ((options?.dialect ?? "postgres") === "postgres" && (node.op === "matches" || node.op === "notMatches") && options?.postgresRegexpPushdown !== true) return {
214
+ kind: node.kind,
215
+ path,
216
+ reason: `'${node.op}' matches trilean's own ECMAScript 'RegExp' against the pattern, whereas PostgreSQL's '~'/'!~' would match it under PostgreSQL's own regular-expression dialect (POSIX ARE) -- a different language that can silently answer a different set of rows for the same pattern and data (for example, PostgreSQL's '.' matches a newline by default where ECMAScript's does not, and PostgreSQL's '\\w' follows the database's locale where ECMAScript's is ASCII-only) -- so this is refused unless 'postgresRegexpPushdown' is explicitly set 'true'`
217
+ };
213
218
  return;
214
219
  }
215
220
  case "memberOf": {
@@ -24,11 +24,19 @@ interface SqlCompileOptions {
24
24
  */
25
25
  columnFor: (referenceKey: string) => SqlColumnBinding;
26
26
  /**
27
- * Whether the SQLite target can resolve a `regexp(pattern, value)` function for `textCompare`'s `matches`/`notMatches` to compile to. Ignored under the `postgres` dialect, which matches patterns with its own native `~`/`!~` operators and never needs this.
27
+ * Whether the SQLite target can resolve a `regexp(pattern, value)` function for `textCompare`'s `matches`/`notMatches` to compile to. Ignored under the `postgres` dialect, which has its own separate `postgresRegexpPushdown` gate below.
28
28
  *
29
29
  * Defaults to `true`, preserving the historical behaviour of always compiling to `REGEXP`/`NOT REGEXP`, which is correct for a driver a caller can register a function on (better-sqlite3, for instance). Set explicitly to `false` for a SQLite-wire-compatible target with no such registration hook -- Cloudflare D1 is the motivating case -- so `matches`/`notMatches` are refused by `UnsupportedNodeError` at compile time instead of compiling to SQL that fails at query execution with "no such function: REGEXP". See the "Regular expressions" section in README.md.
30
30
  */
31
31
  sqliteRegexpAvailable?: boolean;
32
+ /**
33
+ * Whether `matches`/`notMatches` may compile to PostgreSQL's own `~`/`!~` operators. Ignored under the SQLite dialect, whose `REGEXP` question is `sqliteRegexpAvailable`'s above.
34
+ *
35
+ * trilean's evaluator matches a pattern as an ECMAScript `RegExp`; PostgreSQL matches the same pattern text under its own regular-expression dialect (POSIX ARE plus extensions), a different language that is close but not identical. The divergence is not always a syntax error the compiler could catch ahead of time: `.` matches a newline under PostgreSQL's default but not ECMAScript's, and `\w`/`\d` follow the database's locale under PostgreSQL rather than ASCII -- both compile and execute cleanly under either engine and can still answer a different set of rows for the same data. Named groups (`(?<name>...)`), named backreferences (`\k<name>`) and Unicode property escapes (`\p{...}`) are ECMAScript syntax PostgreSQL's ARE does not have at all, so a pattern using them compiles here but fails as a PostgreSQL query error at execution time rather than as a compile-time refusal.
36
+ *
37
+ * Defaults to `false`: `matches`/`notMatches` are refused as unpushable and fall back to in-process evaluation, which is always correct because that is exactly the language the pattern is defined against. Set it to `true` only once every pattern that reaches this option is known to use nothing outside the portable syntax README.md's "Regular expressions" section describes -- it is one switch for the whole compilation, not a per-pattern one, so setting it accepts that same risk for every `matches`/`notMatches` node the tree contains.
38
+ */
39
+ postgresRegexpPushdown?: boolean;
32
40
  }
33
41
  interface CompiledSql {
34
42
  /** A self-contained boolean expression, always parenthesised, suitable for dropping straight into a `WHERE` clause (or a `CHECK`, a `HAVING`, or a filtered index predicate). It carries no `WHERE` keyword of its own. */
package/dist/options.d.ts CHANGED
@@ -24,11 +24,19 @@ interface SqlCompileOptions {
24
24
  */
25
25
  columnFor: (referenceKey: string) => SqlColumnBinding;
26
26
  /**
27
- * Whether the SQLite target can resolve a `regexp(pattern, value)` function for `textCompare`'s `matches`/`notMatches` to compile to. Ignored under the `postgres` dialect, which matches patterns with its own native `~`/`!~` operators and never needs this.
27
+ * Whether the SQLite target can resolve a `regexp(pattern, value)` function for `textCompare`'s `matches`/`notMatches` to compile to. Ignored under the `postgres` dialect, which has its own separate `postgresRegexpPushdown` gate below.
28
28
  *
29
29
  * Defaults to `true`, preserving the historical behaviour of always compiling to `REGEXP`/`NOT REGEXP`, which is correct for a driver a caller can register a function on (better-sqlite3, for instance). Set explicitly to `false` for a SQLite-wire-compatible target with no such registration hook -- Cloudflare D1 is the motivating case -- so `matches`/`notMatches` are refused by `UnsupportedNodeError` at compile time instead of compiling to SQL that fails at query execution with "no such function: REGEXP". See the "Regular expressions" section in README.md.
30
30
  */
31
31
  sqliteRegexpAvailable?: boolean;
32
+ /**
33
+ * Whether `matches`/`notMatches` may compile to PostgreSQL's own `~`/`!~` operators. Ignored under the SQLite dialect, whose `REGEXP` question is `sqliteRegexpAvailable`'s above.
34
+ *
35
+ * trilean's evaluator matches a pattern as an ECMAScript `RegExp`; PostgreSQL matches the same pattern text under its own regular-expression dialect (POSIX ARE plus extensions), a different language that is close but not identical. The divergence is not always a syntax error the compiler could catch ahead of time: `.` matches a newline under PostgreSQL's default but not ECMAScript's, and `\w`/`\d` follow the database's locale under PostgreSQL rather than ASCII -- both compile and execute cleanly under either engine and can still answer a different set of rows for the same data. Named groups (`(?<name>...)`), named backreferences (`\k<name>`) and Unicode property escapes (`\p{...}`) are ECMAScript syntax PostgreSQL's ARE does not have at all, so a pattern using them compiles here but fails as a PostgreSQL query error at execution time rather than as a compile-time refusal.
36
+ *
37
+ * Defaults to `false`: `matches`/`notMatches` are refused as unpushable and fall back to in-process evaluation, which is always correct because that is exactly the language the pattern is defined against. Set it to `true` only once every pattern that reaches this option is known to use nothing outside the portable syntax README.md's "Regular expressions" section describes -- it is one switch for the whole compilation, not a per-pattern one, so setting it accepts that same risk for every `matches`/`notMatches` node the tree contains.
38
+ */
39
+ postgresRegexpPushdown?: boolean;
32
40
  }
33
41
  interface CompiledSql {
34
42
  /** A self-contained boolean expression, always parenthesised, suitable for dropping straight into a `WHERE` clause (or a `CHECK`, a `HAVING`, or a filtered index predicate). It carries no `WHERE` keyword of its own. */
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "trilean-sql",
3
- "version": "1.2.0",
3
+ "version": "2.0.0",
4
4
  "description": "Compiles a trilean predicate tree into a parameterised SQL WHERE fragment, pushing three-valued logic down onto the database's own NULL propagation instead of evaluating it in process.",
5
5
  "type": "module",
6
6
  "repository": {