@simtlix/simfinity-postgres 3.8.0 → 3.10.0

package/README.md

@@ -31,9 +31,9 @@ import { postgresPlugin } from '@simtlix/simfinity-postgres';
 const simfinity = createSQL({ plugin: postgresPlugin({ pool, schema: 'library' }) });
 ```
 
-Install `@simtlix/simfinity-sql@3.8.0` as a direct dependency when importing its factory. PostgreSQL 15 or later is the only supported SQL plugin. The [SQL package guide](https://github.com/simtlix/simfinity.js/tree/master/packages/sql) documents relational planning and the versioned plugin contract.
+Install `@simtlix/simfinity-sql@3.10.0` as a direct dependency when importing its factory. PostgreSQL 15 or later is the only supported SQL plugin. The [SQL package guide](https://github.com/simtlix/simfinity.js/tree/master/packages/sql) documents relational planning and the versioned plugin contract.
 
-Both factories permanently bind one pool and schema, and five optional settings: `multiRowInserts: false` keeps one `INSERT` per owned row instead of multi-row statements; `explicitNullsOrder: true` keeps the `NULLS` clause on sorts of `id` and required fields, as 3.5.9 did, instead of letting indexes serve them; `batchedCollections: true` reads the generated collection fields of the parents that share their arguments with one statement per 100 parents, in a transaction that turns JIT off for that transaction only, instead of one statement per field as 3.6.0 did and as the default still does, and can return unsorted and tied rows in another order; the opt-in `refreshTriggers: 'perRoot'` refreshes each changed root's owned shape and unique keys once per transaction, while its guard trigger stays deferred, instead of once per changed row; and the opt-in `ownerIndexes: 'minimal'` leaves out the `(__owner_id)` indexes that the primary key or a unique index covers on new owned and guard tables. An unknown `refreshTriggers` or `ownerIndexes` value throws `INVALID_DATABASE_SCHEMA` (400). Read the [upgrade notes](https://simtlix.github.io/simfinity.js/resources/compatibility.html#upgrade-to-3-6-0) before enabling the opt-ins: versions up to 3.5.9 cannot validate databases created with them. The returned runtime supports registration, generated queries and mutations, middleware and scopes, controllers and validators, custom mutations, state machines, nested relationship writes, embedded reconstruction, and native PostgreSQL model handles. Native handles expose `findById`, `find`, `create`, `update`, and `delete`; they are not Mongoose models. `find(args, { session })` accepts the generated GraphQL list query arguments. Existing `pg.Pool` configuration and `pg.PoolClient` query interfaces remain compatible across both factories. A custom pool or client wrapper should pass `pg`'s `command` through in query results: the plugin reads it from the `COMMIT` result to detect a transaction that PostgreSQL rolled back because one of its statements failed, and without it such a transaction reports success with nothing stored.
+Both factories permanently bind one pool and schema, and five optional settings: `multiRowInserts: false` keeps one `INSERT` per owned row instead of multi-row statements; `explicitNullsOrder: true` keeps the `NULLS` clause on sorts of `id` and required fields, as 3.5.9 did, instead of letting indexes serve them; `batchedCollections: true` reads the generated collection fields of the parents that share their arguments with one statement per 100 parents, in a transaction that turns JIT off for that transaction only, instead of one statement per field as 3.6.0 did and as the default still does, and can return unsorted and tied rows in another order; the opt-in `refreshTriggers: 'perRoot'` refreshes each changed root's owned shape and unique keys once per transaction, while its guard trigger stays deferred, instead of once per changed row; and the opt-in `ownerIndexes: 'minimal'` leaves out the `(__owner_id)` indexes that the primary key or a unique index covers on new owned and guard tables. An unknown `refreshTriggers` or `ownerIndexes` value throws `INVALID_DATABASE_SCHEMA` (400). Read the [upgrade notes](https://simtlix.github.io/simfinity.js/resources/compatibility.html#upgrade-to-3-6-0) before enabling the opt-ins: versions up to 3.5.9 cannot validate databases created with them. The returned runtime supports registration, generated queries and mutations, middleware and scopes, controllers and validators, custom queries and mutations, state machines, nested relationship writes, embedded reconstruction, and native PostgreSQL model handles. Native handles expose `findById`, `find`, `create`, `update`, and `delete`; they are not Mongoose models. `find(args, { session })` accepts the generated GraphQL list query arguments. Existing `pg.Pool` configuration and `pg.PoolClient` query interfaces remain compatible across both factories. A custom pool or client wrapper should pass `pg`'s `command` through in query results: the plugin reads it from the `COMMIT` result to detect a transaction that PostgreSQL rolled back because one of its statements failed, and without it such a transaction reports success with nothing stored.
 
 Operations attempted before successful initialization fail with `DATABASE_NOT_INITIALIZED`. Call `createSchema` before `initializeDatabase`. Configuration and registrations become immutable once schema preparation begins. `preventCreatingCollection(true)` makes instance initialization validate existing storage without issuing creation DDL. A rejected initialization request leaves readiness as it was; a ready instance keeps serving while it is initialized again, and becomes unavailable if that initialization fails. `create` mode reads the catalog without table locks and locks nothing when nothing is missing. When it creates objects, writers of the tables that it changes, of existing tables that a new foreign key references, or of roots whose keys it rebuilds wait until it commits, so use `mode: 'validate'` to check a live instance. `simfinity.compileDatabaseSchema()` returns the instance's DDL for review or export.
 
@@ -62,7 +62,7 @@ The module and every `createPostgres` instance expose the shared `auth`, `valida
 MCP integration is an independent opt-in and does not add MCP dependencies to PostgreSQL applications:
 
 ```sh
-npm install @simtlix/simfinity-mcp@3.8.0 @modelcontextprotocol/sdk@^1.31.0
+npm install @simtlix/simfinity-mcp@3.10.0 @modelcontextprotocol/sdk@^1.31.0
 ```
 
 ```javascript
@@ -77,6 +77,10 @@ Entity identities and references use UUIDs. Scalars use native PostgreSQL types,
 
 Queries support nested scalar/reference paths, logical filter groups, scopes, sort, pagination, count, and grouped `SUM`, `COUNT`, `AVG`, `MIN`, and `MAX` facts. The PostgreSQL implementation preserves root multiplicity when joining collection paths.
 
+The text operators take literal values as bound parameters. `LIKE`, `STARTS_WITH` and `ENDS_WITH` compare code points, and `STARTS_WITH` can use a column's btree index. `ILIKE`, `ISTARTS_WITH`, `IENDS_WITH` and `IEQ` fold both sides with the Unicode simple case foldings that MongoDB's case-insensitive matching applies, through one expression, `translate(lower((x) COLLATE "pg_catalog"."und-x-icu"), from, to)`, where `translate()` maps the 22 lowercase variants that ICU `lower()` keeps, such as `ſ`, `µ` and the final sigma `ς`, to their usual letters; so each returns every row that its case-sensitive form returns, and `ISTARTS_WITH "ΚΑΣ"` finds `ΚΑΣΤΡΟ`. The first three match a `LIKE` pattern whose `%`, `_` and `\` are escaped. The two backends still differ on `İ`, which ICU lowers to `i` followed by U+0307, and on letters newer than one side's Unicode tables. The fold needs a `UTF8` database on a server built with ICU: on another, such as one in `LATIN1`, these four operators fail with `UNSUPPORTED_QUERY` (400), and every other operation works. Indexes serve them only on exactly the compiled expression, which the exported `caseFoldExpression(sqlExpression)` returns, such as `caseFoldExpression('"name"')`: a btree with `text_pattern_ops` on it serves `IEQ` and `ISTARTS_WITH`, and a `pg_trgm` GIN index with `gin_trgm_ops` serves `ILIKE`, `ISTARTS_WITH` and `IENDS_WITH`. The [queries guide](https://simtlix.github.io/simfinity.js/guide/queries.html#case-folding) gives the details and the index statements.
+
+`IS_EMPTY` and `IS_NOT_EMPTY` take no value and test the queried record's own fields: a missing, null or `""` String, a missing or null value of any other scalar, a missing, null or `[]` list, and an embedded object that is missing, null or has every stored member empty. They compile from where the path is stored and bind no parameters: `(c IS NULL OR c = '')` for a String column, so that a btree serves it through a `BitmapOr`, `c IS NULL` for any other column, including the uuid identity of a record whose `id` is declared `String`, and `COALESCE(cardinality(c), 0) = 0` for a native array, also on owned rows; a jsonb member is read as text with `->>` and never cast, a jsonb list through a `jsonb_typeof` guard, and a jsonb object as SQL `NULL`, JSON `null` or all of its members empty; an owned single object is empty when its state is not `present` or no owned row has a member that is not empty, and an owned list when its state is not `present` or it has no row, null items included, with a subquery alias of its own at each level of owned storage. `IS_NOT_EMPTY` is the positive form of a leaf and the `NOT` of an object or owned form, so every record matches exactly one of the two. A path that continues inside an embedded list or a collection, such as `lines.qty` or `seasons.year`, fails with the permanent `INVALID_FILTER_PATH` (400), since emptiness of a value inside a list is not defined. In this release, any other path that crosses or ends at a reference, a path that ends at a collection, such as `seasons`, and an embedded object with a reference member fail with `UNSUPPORTED_QUERY` (400). The [queries guide](https://simtlix.github.io/simfinity.js/guide/queries.html) describes the rules.
+
 Errors that PostgreSQL reports become `SimfinityError`s without SQL details, with the driver error as `getCause()`; errors that application code throws propagate unchanged. Text cannot contain NUL and native date-times start at 4714-11-24 BC: such input fails with `INVALID_VALUE` (400).
 
 PostgreSQL enforces required values, real FKs, JSONB/owned embedded shapes, root and embedded scalar/reference uniqueness, and scalar-list multikey uniqueness. Root unique indexes use `NULLS NOT DISTINCT`; private typed owner-key tables allow repeated keys within one owner but reject the same key across owners. Presence markers preserve missing versus explicit null fields. This is stronger and broader than generated Mongoose storage.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@simtlix/simfinity-postgres",
-  "version": "3.8.0",
+  "version": "3.10.0",
   "description": "PostgreSQL GraphQL runtime and schema generation with foreign keys for Simfinity",
   "type": "module",
   "main": "./src/index.js",
@@ -27,8 +27,8 @@
     "directory": "packages/postgres"
   },
   "dependencies": {
-    "@simtlix/simfinity-core": "3.8.0",
-    "@simtlix/simfinity-sql": "3.8.0",
+    "@simtlix/simfinity-core": "3.10.0",
+    "@simtlix/simfinity-sql": "3.10.0",
     "pg": "^8.16.3"
   },
   "peerDependencies": {

package/src/codecs.js

@@ -1,5 +1,6 @@
 import { getNamedType, isEnumType } from 'graphql';
 import { SimfinityError } from '@simtlix/simfinity-core';
+import { isCaseFoldUnsupported } from './query/case-fold.js';
 
 export const storageTypes = { ID: 'uuid', String: 'text', Enum: 'text', Int: 'integer', Float: 'double precision', Boolean: 'boolean', DateTime: 'timestamp with time zone' };
 export const invalidValue = (message) => { throw new SimfinityError(message, 'INVALID_FILTER_VALUE', 400); };
@@ -69,10 +70,14 @@ export const isRetryableDatabaseError = (error) => ['40001', '40P01'].includes(e
 // An application error that only has a `code`, such as ECONNREFUSED or a domain code, is not one.
 export const isPostgresError = (error) => typeof error?.code === 'string' && /^[0-9A-Z]{5}$/.test(error.code) && typeof error.severity === 'string';
 
+// A statement that folds case on a database that cannot evaluate the fold (see isCaseFoldUnsupported): the request
+// uses an operator that this database cannot run, while every other operation keeps working.
+const CASE_FOLD_UNSUPPORTED = ['ILIKE, ISTARTS_WITH, IENDS_WITH and IEQ need a UTF8 database with the ICU collation und-x-icu', 'UNSUPPORTED_QUERY', 400];
+
 /** Maps PostgreSQL errors (and exhausted retries) without SQL details; returns every other value unchanged. */
 export const normalizeDatabaseError = (error) => {
   if (error instanceof SimfinityError || !(isPostgresError(error) || isRetryableDatabaseError(error))) return error;
-  const normalized = new SimfinityError(...(databaseErrors[error.code] || ['Database operation failed', 'DATABASE_ERROR', 500]));
+  const normalized = new SimfinityError(...(isCaseFoldUnsupported(error) ? CASE_FOLD_UNSUPPORTED : databaseErrors[error.code] || ['Database operation failed', 'DATABASE_ERROR', 500]));
   // Kept for server logging, but non-enumerable as on a native Error, so serializing or spreading the
   // client-facing error never copies the driver's detail, table or constraint.
   Object.defineProperty(normalized, 'cause', { value: error, writable: true, configurable: true, enumerable: false });

package/src/index.js

@@ -5,6 +5,7 @@ import { compileDatabaseSchema as compileSchemaSQL } from './schema/ddl.js';
 import { initializeDatabase as initializeSchema } from './schema/initialize.js';
 
 export { postgresPlugin } from './plugin.js';
+export { caseFoldExpression } from './query/case-fold.js';
 export {
   auth,
   buildErrorFormatter,
@@ -19,7 +20,7 @@ export {
 /** Each instance permanently binds one pool/schema; getModel exposes PostgreSQL native helpers. */
 export const createPostgres = (options) => createSQL({ plugin: postgresPlugin(options) });
 const defaultInstance = createPostgres();
-export const { configureQueryLimits, configureMutationLimits, configure, connect, addNoEndpointType, createSchema, getModel, getType, getInputType, getRegistrations, use, registerMutation, saveObject, preventCreatingCollection, withTransaction } = defaultInstance;
+export const { configureQueryLimits, configureMutationLimits, configure, connect, addNoEndpointType, createSchema, getModel, getType, getInputType, getRegistrations, use, registerMutation, registerCustomQuery, saveObject, preventCreatingCollection, withTransaction } = defaultInstance;
 
 /** Preserve the foundation low-level API while allowing initialization of the configured default instance. */
 export const initializeDatabase = (poolOrOptions, description, options) => description

package/src/plugin.js

@@ -5,6 +5,7 @@ import { describeRelationalSchema } from './schema/describe.js';
 import { initializeDatabase } from './schema/initialize.js';
 import { compileDatabaseSchema } from './schema/ddl.js';
 import { compileChildrenQuery, compileQuery } from './query/compiler.js';
+import { markCaseFoldFailure } from './query/case-fold.js';
 import { compileRecord, compileRecordBatch, MAX_RECORD_BATCH_VALUES } from './record-statements.js';
 import { castId, encodeScalar, decodeScalar, encodeParameter, isRetryableDatabaseError, normalizeDatabaseError } from './codecs.js';
 
@@ -17,6 +18,13 @@ export const postgresPlugin = (options) => ({
   // singleStatementReads: one statement reads one snapshot at any isolation level, so a read without owned
   // tables needs no repeatable-read transaction.
   capabilities: ['transactions', 'foreignKeys', 'deferredForeignKeys', 'embeddedValues', 'ownedRecords', 'scalarLists', 'uniqueValues', 'nullableUnique', 'singleStatementReads'],
+  // Every filter operator compileQuery and compileChildrenQuery compile, listed one by one so that a later
+  // QLOperator value is rejected until the compiler supports it. The declaration names the compileQuery it describes,
+  // so a copy of this plugin with another compileQuery compiles the operators of 3.7.0 until it declares its own.
+  queryFeatures: {
+    operators: ['EQ', 'NE', 'LT', 'LTE', 'GT', 'GTE', 'BTW', 'IN', 'NIN', 'LIKE', 'ILIKE', 'STARTS_WITH', 'ISTARTS_WITH', 'ENDS_WITH', 'IENDS_WITH', 'IEQ', 'IS_EMPTY', 'IS_NOT_EMPTY'],
+    compileQuery,
+  },
   naming: { validateIdentifier: identifier, generatedName },
   // The runtime passes its bound configuration: createPostgres(options) or configure(value).
   describeSchema: (plan, configuration) => describeRelationalSchema(plan, { refreshTriggers: configuration?.refreshTriggers, ownerIndexes: configuration?.ownerIndexes }),
@@ -37,7 +45,15 @@ export const postgresPlugin = (options) => ({
     assertConfiguration(configuration) {
       if (!configuration?.pool?.connect || !configuration.pool.query) throw new SimfinityError('Configure a pg.Pool before creating the schema', 'DATABASE_NOT_CONFIGURED', 500);
     },
-    query: (configuration, statement, client) => (client || configuration.pool).query(statement.text, statement.values?.map(encodeParameter)),
+    // The error of a statement that folds case is marked, unchanged, so that normalizeError reports a database that
+    // cannot evaluate the fold as UNSUPPORTED_QUERY.
+    query: async (configuration, statement, client) => {
+      try {
+        return await (client || configuration.pool).query(statement.text, statement.values?.map(encodeParameter));
+      } catch (error) {
+        throw markCaseFoldFailure(statement, error);
+      }
+    },
     acquire: (configuration) => configuration.pool.connect(),
     begin: (client) => client.query('BEGIN ISOLATION LEVEL REPEATABLE READ'),
     // PostgreSQL answers COMMIT with ROLLBACK, and no error, when a statement in the transaction failed

package/src/query/case-fold.js

@@ -0,0 +1,88 @@
+// The case fold of the case-insensitive text operators (ILIKE, ISTARTS_WITH, IENDS_WITH and IEQ), the shared
+// contract of #108 section 3.1: the Unicode simple case foldings that MongoDB's case-insensitive regular expressions
+// apply. lower() under ICU's root collation applies them to every script the same way on every server, whatever the
+// database's default collation (text paths are `C` columns, under which lower() folds ASCII only), except for the
+// lowercase variants below, which it keeps and simple folding maps to another lowercase letter; translate() then
+// maps them. lower() also turns a capital sigma that ends a word into ς and any other into σ, and the value is folded
+// apart from the stored text, so mapping ς to σ also makes a value fold as it does inside the stored text: a value
+// that starts or ends with Σ finds every row that its case-sensitive form finds. The fold is one inline expression,
+// without a database function, so it needs no schema object and no migration, and an application index serves the
+// operators only on exactly this expression, which caseFoldExpression writes.
+
+/**
+ * The one-to-one simple case foldings (CaseFolding.txt, status C) that ICU lower() does not apply and MongoDB 8.0
+ * does, found by comparing both over every code point with a case mapping, on PostgreSQL 16.15 with an ICU of Unicode
+ * 16 and MongoDB 8.0.32: with them, the two differ only on letters newer than MongoDB's Unicode tables. Changing the
+ * list changes every statement and the expression that application indexes must repeat, so it is part of the release
+ * contract.
+ */
+const CASE_FOLD_MAPPINGS = Object.freeze([
+  ['\u00B5', '\u03BC'], // MICRO SIGN, GREEK SMALL LETTER MU
+  ['\u017F', '\u0073'], // LATIN SMALL LETTER LONG S, LATIN SMALL LETTER S
+  ['\u0345', '\u03B9'], // COMBINING GREEK YPOGEGRAMMENI, GREEK SMALL LETTER IOTA
+  ['\u03C2', '\u03C3'], // GREEK SMALL LETTER FINAL SIGMA, GREEK SMALL LETTER SIGMA
+  ['\u03D0', '\u03B2'], // GREEK BETA SYMBOL, GREEK SMALL LETTER BETA
+  ['\u03D1', '\u03B8'], // GREEK THETA SYMBOL, GREEK SMALL LETTER THETA
+  ['\u03D5', '\u03C6'], // GREEK PHI SYMBOL, GREEK SMALL LETTER PHI
+  ['\u03D6', '\u03C0'], // GREEK PI SYMBOL, GREEK SMALL LETTER PI
+  ['\u03F0', '\u03BA'], // GREEK KAPPA SYMBOL, GREEK SMALL LETTER KAPPA
+  ['\u03F1', '\u03C1'], // GREEK RHO SYMBOL, GREEK SMALL LETTER RHO
+  ['\u03F5', '\u03B5'], // GREEK LUNATE EPSILON SYMBOL, GREEK SMALL LETTER EPSILON
+  ['\u1C80', '\u0432'], // CYRILLIC SMALL LETTER ROUNDED VE, CYRILLIC SMALL LETTER VE
+  ['\u1C81', '\u0434'], // CYRILLIC SMALL LETTER LONG-LEGGED DE, CYRILLIC SMALL LETTER DE
+  ['\u1C82', '\u043E'], // CYRILLIC SMALL LETTER NARROW O, CYRILLIC SMALL LETTER O
+  ['\u1C83', '\u0441'], // CYRILLIC SMALL LETTER WIDE ES, CYRILLIC SMALL LETTER ES
+  ['\u1C84', '\u0442'], // CYRILLIC SMALL LETTER TALL TE, CYRILLIC SMALL LETTER TE
+  ['\u1C85', '\u0442'], // CYRILLIC SMALL LETTER THREE-LEGGED TE, CYRILLIC SMALL LETTER TE
+  ['\u1C86', '\u044A'], // CYRILLIC SMALL LETTER TALL HARD SIGN, CYRILLIC SMALL LETTER HARD SIGN
+  ['\u1C87', '\u0463'], // CYRILLIC SMALL LETTER TALL YAT, CYRILLIC SMALL LETTER YAT
+  ['\u1C88', '\uA64B'], // CYRILLIC SMALL LETTER UNBLENDED UK, CYRILLIC SMALL LETTER MONOGRAPH UK
+  ['\u1E9B', '\u1E61'], // LATIN SMALL LETTER LONG S WITH DOT ABOVE, LATIN SMALL LETTER S WITH DOT ABOVE
+  ['\u1FBE', '\u03B9'], // GREEK PROSGEGRAMMENI, GREEK SMALL LETTER IOTA
+]);
+
+const COLLATION = '"pg_catalog"."und-x-icu"';
+// No character of either list is a quote or a backslash, so both are plain string literals.
+const FROM = `'${CASE_FOLD_MAPPINGS.map(([from]) => from).join('')}'`;
+const TO = `'${CASE_FOLD_MAPPINGS.map(([, to]) => to).join('')}'`;
+
+/**
+ * The SQL text that folds `sqlExpression` for the case-insensitive text operators, exactly as the compiler writes it on
+ * both sides of each comparison: `translate(lower((<expression>) COLLATE "pg_catalog"."und-x-icu"), '<22 characters>',
+ * '<their simple case foldings>')`. Applications build their expression indexes with it, such as
+ * `CREATE INDEX ... (${caseFoldExpression('"name"')} text_pattern_ops)`. The expression is trusted SQL, inserted as
+ * given: quote identifiers yourself and never pass request input. It goes in parentheses, because COLLATE binds more
+ * tightly than `->>` and every other binary operator: without them, `"details" ->> 'label'` would collate only
+ * 'label', and an index on that expression would never serve the operators. PostgreSQL keeps no trace of parentheses
+ * in the parse tree, so an expression gives the same index expression with or without parentheses of its own.
+ */
+export const caseFoldExpression = (sqlExpression) => {
+  if (typeof sqlExpression !== 'string' || !sqlExpression.trim()) throw new TypeError('caseFoldExpression expects a non-empty SQL expression string');
+  return `translate(lower((${sqlExpression}) COLLATE ${COLLATION}), ${FROM}, ${TO})`;
+};
+
+// Each statement that folds case contains the list of characters, which a GraphQL name, a generated identifier or a
+// compiled literal never contains, since values are bound parameters.
+const foldsCase = (statement) => typeof statement?.text === 'string' && statement.text.includes(FROM);
+// The errors of statements that fold case, which normalizeDatabaseError reports as unsupported when the database
+// cannot evaluate the fold.
+const failures = new WeakSet();
+
+/**
+ * Records that `error`, which the driver raised for `statement`, comes from a statement that folds case, and returns
+ * it unchanged, so that retries and normalization see the driver's own error.
+ */
+export const markCaseFoldFailure = (statement, error) => {
+  if (error !== null && typeof error === 'object' && foldsCase(statement)) failures.add(error);
+  return error;
+};
+
+/**
+ * Whether the database cannot evaluate the fold of a statement that failed: one whose server encoding lacks the
+ * characters of the fold (22P05, untranslatable character, such as LATIN1, which fails the statement's text before
+ * any row is read, whatever the value), or a server without the ICU collation und-x-icu, built without ICU or with a
+ * database encoding that ICU does not support, such as SQL_ASCII (42704, undefined object). Compiled statements
+ * cannot raise either code otherwise in a UTF8 database with ICU: the collation is their only object that can be
+ * missing, and the database stores no text that UTF8 cannot convert.
+ */
+export const isCaseFoldUnsupported = (error) => failures.has(error) && ['22P05', '42704'].includes(error.code);

package/src/query/compiler.js

@@ -1,9 +1,19 @@
 import { resolveModelPath, SimfinityError } from '@simtlix/simfinity-core';
+import {
+  describeEmptiness, describeModelEmptiness, isEmptinessOperator, textQueryOperator, validateTextFilterValue,
+} from '@simtlix/simfinity-core/internal/query-operators';
 import { identifier as q, literal, qualified } from '../schema/sql.js';
 import { encodeScalar, storageTypes, invalidValue } from '../codecs.js';
+import { caseFoldExpression as fold } from './case-fold.js';
 
 const unsupported = (message) => { throw new SimfinityError(message, 'UNSUPPORTED_QUERY', 400); };
 
+// The case-insensitive text operators compare the case fold of both sides (see case-fold.js), so an application
+// index serves them only on that expression, which caseFoldExpression writes.
+// LIKE's default escape character is the backslash, so escaping it, % and _ leaves only literal characters.
+const likeLiteral = (value) => value.replace(/[\\%_]/g, '\\$&');
+const likePatterns = { contains: (value) => `%${likeLiteral(value)}%`, prefix: (value) => `${likeLiteral(value)}%`, suffix: (value) => `%${likeLiteral(value)}` };
+
 // Alias of each per-parent statement in the batched children statement. The per-parent statement only uses t<n>,
 // f<n> and short subquery aliases, so it never shadows this one.
 const CHILDREN = 'simfinity_children';
@@ -80,6 +90,12 @@ const compileSelect = (models, database, plan, extra, configuration, batched) =>
     }
     return joins.get(key).alias;
   };
+  // The owned table that stores `field` of the row `context`, or undefined when the row stores it; values inside
+  // jsonb never own tables.
+  const ownedOf = (context, field) => context.table && database.tables.find((item) => item.ownership?.ownerTable === context.table.name && item.ownership.field === field.name);
+  // The LEFT JOIN of an owned single object, one per path, which filters and the emptiness forms share. A missed
+  // join, and an object in another state, read as NULL.
+  const joinOwned = (key, context, owned) => join(key, owned.name, (name) => `${col(name, '__owner_id')} = ${col(context.alias, context.table.primaryKey.columns[0])} AND ${col(context.alias, owned.ownership.stateColumn)} = 'present'`);
   const arrayFrom = (vector, expression) => `ARRAY(SELECT ${expression} FROM ${vector.from} WHERE ${vector.where})`;
   const jsonItems = (expression, missing = '[]') => `jsonb_array_elements(CASE WHEN jsonb_typeof(${expression}) = 'array' THEN ${expression} ELSE '${missing}'::jsonb END)`;
   const helper = (name, ...args) => `${qualified(database.schema, `__simfinity_${name}`)}(${args.join(', ')})`;
@@ -138,8 +154,7 @@ const compileSelect = (models, database, plan, extra, configuration, batched) =>
           const state = `${col(ownerAlias, owned.ownership.stateColumn)} = 'present'`;
           const presence = (name) => owned.ownership.nullableItems && usage !== 'sort' ? ` AND ${col(name, '__item_present')}` : '';
           if (!field.list && !context.vector) {
-            const joined = join(key, owned.name, (name) => `${col(name, '__owner_id')} = ${ownerId} AND ${state}`);
-            context = { table: owned, alias: joined, json: null, vector: null };
+            context = { table: owned, alias: joinOwned(key, context, owned), json: null, vector: null };
           } else {
             const name = alias();
             const stored = alias();
@@ -207,14 +222,15 @@ const compileSelect = (models, database, plan, extra, configuration, batched) =>
           const sql = arrayFrom(vector, `${name}.value`);
           return { sql: type === 'text' ? `(${sql}) COLLATE "pg_catalog"."C"` : sql, field, type: dateNumeric ? 'double precision' : type, list: true, vector: true, dateNumeric, sortKey };
         }
-        let sql = context.vector ? arrayFrom(context.vector, expression) : expression;
-        if (type === 'text') sql = `(${sql}) COLLATE "pg_catalog"."C"`;
+        // `uncollated` is the value before the C collation, which the case-insensitive text operators replace.
+        const uncollated = context.vector ? arrayFrom(context.vector, expression) : expression;
+        const sql = type === 'text' ? `(${uncollated}) COLLATE "pg_catalog"."C"` : uncollated;
         // A NOT NULL column of the root row has no nulls to place, so its sort needs no NULLS clause and a
         // default-order btree (the primary key or a declared index) can return it in order. Joined and
         // embedded paths can be NULL even for NOT NULL columns, so they keep the clause.
         const notNull = !context.json && !context.vector && !field.list && context.alias === 't0'
           && context.table.columns.find((item) => item.name === (identity ? 'id' : field.storageName))?.nullable === false;
-        return { sql, field: identity ? { ...field, scalar: 'ID' } : field, type: dateNumeric ? 'double precision' : type, list: field.list || !!context.vector, vector: !!context.vector, dateNumeric, sortKey, notNull };
+        return { sql, uncollated, field: identity ? { ...field, scalar: 'ID' } : field, type: dateNumeric ? 'double precision' : type, list: field.list || !!context.vector, vector: !!context.vector, dateNumeric, sortKey, notNull };
       }
     }
     unsupported('Invalid query path');
@@ -266,19 +282,128 @@ const compileSelect = (models, database, plan, extra, configuration, batched) =>
     }
     if (operator === 'BTW') return `(${predicate(expression, 'GTE', value[0], negated)} AND ${predicate(expression, 'LTE', value[1], negated)})`;
     if (value == null || Array.isArray(value)) invalidValue(`${operator} requires a non-null scalar value`);
+    const text = textQueryOperator(operator);
+    if (text) return textPredicate(expression, operator, text, value, negated);
+    const ordering = { LT: '<', LTE: '<=', GT: '>', GTE: '>=' }[operator];
+    if (!ordering && operator !== 'LIKE') unsupported(`Unsupported filter operator ${operator}`);
     const parameter = scalarBind(value);
     const compare = (left) => {
       if (operator === 'LIKE') {
         return `strpos(${left}, ${parameter}) > 0`;
       }
-      return `${left} ${{ LT: '<', LTE: '<=', GT: '>', GTE: '>=' }[operator]} ${parameter}`;
+      return `${left} ${ordering} ${parameter}`;
     };
     if (list) return `EXISTS (SELECT 1 FROM unnest(${sql}) v(value) WHERE ${compare('v.value')})`;
     return negated ? `COALESCE(${compare(sql)}, FALSE)` : compare(sql);
   };
-  // A negation reaches every predicate below it, including those nested in AND/OR groups.
+  /**
+   * The literal text operators (ILIKE, STARTS_WITH, ISTARTS_WITH, ENDS_WITH, IENDS_WITH and IEQ) on a String leaf or
+   * list, where some element must match. Values stay bound parameters. The case-sensitive forms, STARTS_WITH and
+   * ENDS_WITH, compare code points with starts_with and right under the C collation of text paths, so a column's
+   * btree serves STARTS_WITH. The case-insensitive forms compare the case fold of both sides (see case-fold.js): IEQ
+   * with =, the others with LIKE and a bound pattern whose metacharacters are escaped, so that a btree on the folded
+   * column with text_pattern_ops serves IEQ and ISTARTS_WITH, and a trigram index on it serves ILIKE, ISTARTS_WITH and
+   * IENDS_WITH.
+   */
+  const textPredicate = ({ sql, uncollated, field, list }, operator, { match, ignoreCase }, value, negated) => {
+    // createQueryPlan has validated the value; a plan built by hand gets the same INVALID_FILTER_VALUE.
+    validateTextFilterValue(operator, value, field.scalar);
+    let compare;
+    if (!ignoreCase) {
+      const parameter = bind(value, 'text');
+      compare = match === 'prefix'
+        ? (left) => `starts_with(${left}, ${parameter})`
+        : (left) => `right(${left}, char_length(${parameter})) = ${parameter}`;
+    } else if (match === 'exact') {
+      const parameter = bind(value, 'text');
+      compare = (left) => `${fold(left)} = ${fold(parameter)}`;
+    } else {
+      const pattern = bind(likePatterns[match](value), 'text');
+      compare = (left) => `${fold(left)} LIKE ${fold(pattern)}`;
+    }
+    // fold puts its argument in parentheses, so the uncollated value goes in as it is.
+    if (list) return `EXISTS (SELECT 1 FROM unnest(${sql}) v(value) WHERE ${compare('v.value')})`;
+    const left = ignoreCase ? uncollated : sql;
+    return negated ? `COALESCE(${compare(left)}, FALSE)` : compare(left);
+  };
+  /**
+   * The IS_EMPTY form of `field`, stored on the row or jsonb value `context`, by its emptiness shape, or with `empty`
+   * false the IS_NOT_EMPTY form: the positive form of a leaf, which a btree can serve, and the NOT of an object or an
+   * owned form. Every form is two-valued and binds no parameters. A leaf is a column of any row, or the text of a
+   * jsonb member, never cast, since a cast can fail; a list a native array column, a jsonb list, or an owned list,
+   * whose null items are rows too, so `[null]` is not empty; an object a jsonb value or an owned single object whose
+   * stored members are all empty. An owned table is read in a subquery with an alias of its own, correlated to the
+   * key of the row that owns it: the root's id, or the `__id` of a joined or enclosing owned row.
+   */
+  const emptyForm = (context, field, shape, empty = true) => {
+    const compare = empty ? '=' : '>';
+    const negate = (form) => (empty ? form : `NOT ${form}`);
+    const leaf = (value) => {
+      if (shape.rule === 'string') return empty ? `(${value} IS NULL OR ${value} = '')` : `(${value} IS NOT NULL AND ${value} <> '')`;
+      return `${value} IS ${empty ? '' : 'NOT '}NULL`;
+    };
+    // The CASE keeps jsonb_array_length from failing (22023) on a value that is not an array.
+    const jsonList = (json) => `COALESCE(jsonb_array_length(CASE WHEN jsonb_typeof(${json}) = 'array' THEN ${json} END), 0) ${compare} 0`;
+    // An embedded type without stored members is always empty.
+    const members = (row) => {
+      const forms = shape.members.map((member) => {
+        const stored = field.fields?.find((item) => item.name === member.name);
+        if (!stored) unsupported('Invalid query path');
+        return emptyForm(row, stored, member.shape);
+      });
+      return forms.length ? `(${forms.join(' AND ')})` : 'TRUE';
+    };
+    const jsonObject = (json) => negate(`(${json} IS NULL OR ${json} = 'null'::jsonb OR ${members({ table: null, alias: null, json })})`);
+    if (context.json) {
+      const json = `(${context.json} -> ${literal(field.storageName)})`;
+      if (shape.rule === 'object') return jsonObject(json);
+      if (shape.rule === 'list') return jsonList(json);
+      return leaf(`(${context.json} ->> ${literal(field.storageName)})`);
+    }
+    const owned = ownedOf(context, field);
+    if (owned) {
+      // A missed join leaves the state NULL, which is not 'present'.
+      const state = `${col(context.alias, owned.ownership.stateColumn)} IS DISTINCT FROM 'present'`;
+      const name = alias();
+      const rows = `SELECT 1 FROM ${qualified(database.schema, owned.name)} ${name} WHERE ${col(name, '__owner_id')} = ${col(context.alias, context.table.primaryKey.columns[0])}`;
+      if (shape.rule === 'list') return negate(`(${state} OR NOT EXISTS (${rows}))`);
+      // An object marked present without a row, which the owned-shape check rejects, is empty too.
+      return negate(`(${state} OR NOT EXISTS (${rows} AND NOT ${members({ table: owned, alias: name, json: null })}))`);
+    }
+    // The record identity is the uuid primary key, whatever scalar it is declared with.
+    const identity = !context.table.ownership && ['id', '_id'].includes(field.name);
+    const column = col(context.alias, identity ? 'id' : field.storageName);
+    if (field.kind === 'embedded') return shape.rule === 'object' ? jsonObject(column) : jsonList(column);
+    if (shape.rule === 'list') return `COALESCE(cardinality(${column}), 0) ${compare} 0`;
+    return leaf(column);
+  };
+  /**
+   * IS_EMPTY and IS_NOT_EMPTY (#108, part 2a). Every emptiness predicate, leaves included, compiles here from where
+   * its path is stored, not from path(), whose values cast jsonb members and materialize jsonb lists, and which
+   * rejects paths that end at an embedded field. The checks of createQueryPlan run again, with its codes, on the
+   * entity's GraphQL type or on the field descriptions, so a plan built by hand gets the same answers; they leave
+   * only single embedded objects before the last field, reached through the LEFT JOIN of each owned single object.
+   */
+  const emptiness = ({ path: parts, operator, value }) => {
+    const steps = resolveModelPath(models, plan.entity, parts);
+    const options = { operator, value, path: parts.join('.') };
+    const gqltype = models.entities.find((item) => item.name === plan.entity)?.gqltype;
+    const shape = gqltype ? describeEmptiness(gqltype, parts, options) : describeModelEmptiness(steps, options);
+    let context = { table: root, alias: 't0', json: null };
+    steps.slice(0, -1).forEach((field, index) => {
+      if (field.kind !== 'embedded' || field.list) unsupported('Invalid query path');
+      const owned = ownedOf(context, field);
+      context = owned
+        ? { table: owned, alias: joinOwned(parts.slice(0, index + 1).join('.'), context, owned), json: null }
+        : { table: null, alias: null, json: context.json ? `(${context.json} -> ${literal(field.storageName)})` : col(context.alias, field.storageName) };
+    });
+    return emptyForm(context, steps.at(-1), shape, operator === 'IS_EMPTY');
+  };
+  // A negation reaches every predicate below it, including those nested in AND/OR groups; the emptiness forms are
+  // two-valued, so they need no null-safe form under it.
   const where = (node, negated = false) => {
     if (!node) return 'TRUE';
+    if (node.kind === 'predicate' && isEmptinessOperator(node.operator)) return `(${emptiness(node)})`;
     if (node.kind === 'predicate') return `(${predicate(path(node.path), node.operator, node.value, negated)})`;
     return `(${node.terms.map((term) => where(term, negated)).join(node.kind === 'and' ? ' AND ' : ' OR ')})`;
   };

package/types/index.d.ts

@@ -17,6 +17,7 @@ export type {
   AuthRule,
   AuthRuleFunction,
   AuthSchemaPlugin,
+  CreateSchemaOptions,
   EntityController,
   EnvelopSchemaPlugin,
   FieldValidations,
@@ -230,7 +231,20 @@ export interface PostgresRuntime extends Runtime<PostgresModel, PostgresSession>
   initializeDatabase(options?: InitializationOptions): Promise<InitializationResult>;
   withTransaction<T>(session: PostgresSession | null | undefined, callback: (session: PostgresSession) => Promise<T> | T): Promise<T>;
 }
-/** PostgreSQL 15+ plugin for createSQL; omitted options allow later runtime.configure. */
+/**
+ * PostgreSQL 15+ plugin for createSQL; omitted options allow later runtime.configure. Its `queryFeatures` lists every
+ * `QLOperator` value. The case-insensitive text operators fold case with {@link caseFoldExpression}, which needs a UTF8
+ * database with the ICU collation `und-x-icu`: on another database, such as one in LATIN1 or on a server built without
+ * ICU, they fail with UNSUPPORTED_QUERY (400), and every other operation works.
+ *
+ * `IS_EMPTY` and `IS_NOT_EMPTY` compile from where a path is stored, binding no parameters: `(c IS NULL OR c = '')`
+ * for a String column and `c IS NULL` for any other, the uuid identity included; `COALESCE(cardinality(c), 0) = 0`
+ * for a native array; jsonb members as text without a cast, and jsonb lists through a `jsonb_typeof` guard; an
+ * embedded object as missing, JSON `null`, or every stored member empty; and owned objects and lists through their
+ * state column and a `NOT EXISTS` subquery on the owned table, null list items counting as items. `IS_NOT_EMPTY` is
+ * the positive form of a leaf, which a btree can serve, and the `NOT` of an object or owned form. Declaring the two
+ * operators does not lift the UNSUPPORTED_QUERY (400) that emptiness through references and collections gets.
+ */
 export function postgresPlugin(options?: PostgresConfiguration): SQLPlugin<
   PostgresConfiguration,
   DatabaseQueryable,
@@ -243,6 +257,17 @@ export function postgresPlugin(options?: PostgresConfiguration): SQLPlugin<
   }
 >;
 export function createPostgres(options?: PostgresConfiguration): PostgresRuntime;
+/**
+ * The SQL text with which ILIKE, ISTARTS_WITH, IENDS_WITH and IEQ fold both sides of their comparison, for an SQL
+ * expression such as a quoted column name or a jsonb member: `translate(lower((<expression>) COLLATE
+ * "pg_catalog"."und-x-icu"), '<22 characters>', '<their simple case foldings>')`, the Unicode simple case foldings that
+ * MongoDB's case-insensitive matching applies. Indexes serve these operators only on exactly this expression, so build
+ * them with it, for example `CREATE INDEX ... (${caseFoldExpression('"name"')} text_pattern_ops)` or
+ * `CREATE INDEX ... (${caseFoldExpression('"details" ->> \'label\'')})`. The expression is inserted as given, in
+ * parentheses, so that COLLATE applies to all of it; pass trusted SQL, never request input. Throws a TypeError for
+ * anything other than a non-empty string.
+ */
+export function caseFoldExpression(sqlExpression: string): string;
 export const configure: PostgresRuntime['configure'];
 export const connect: PostgresRuntime['connect'];
 export const addNoEndpointType: PostgresRuntime['addNoEndpointType'];
@@ -253,6 +278,7 @@ export const getInputType: PostgresRuntime['getInputType'];
 export const getRegistrations: PostgresRuntime['getRegistrations'];
 export const use: PostgresRuntime['use'];
 export const registerMutation: PostgresRuntime['registerMutation'];
+export const registerCustomQuery: PostgresRuntime['registerCustomQuery'];
 export const saveObject: PostgresRuntime['saveObject'];
 export const preventCreatingCollection: PostgresRuntime['preventCreatingCollection'];
 export const withTransaction: PostgresRuntime['withTransaction'];