@genesislcap/mock-server 15.59.0 → 15.61.0

package/README.md

@@ -280,7 +280,9 @@ row does (genesis-db `InsertOperation.kt`):
   earlier seed row; an unsafe one (above 2^53, already rounded by
   `JSON.parse`) or a repeat is re-stamped, with one warning per table, and a
   missing `TIMESTAMP` becomes the `RECORD_ID`. An insert always gets fresh ones. `updateRow` moves `TIMESTAMP` on and
-  keeps `RECORD_ID`, even if the patch names either field.
+  keeps `RECORD_ID`, even if the patch names either field. A seed that brings
+  the real server's ids as strings above 2^53 makes later stamps continue from
+  the highest of them, as strings, so a new row is still the newest.
 - **Deviation:** GSF's values are about 7.5e18 (epoch millis shifted left 22
   bits, plus a counter). That is above 2^53, so `JSON.parse` rounds
   neighbouring rows onto one number. The mock's stamps are epoch millis, or the
@@ -721,8 +723,12 @@ createMockServer({ ...config, fidelity: 'legacy' });
 | `JSON_SCHEMA_REQUEST` content | Per resource type, as below | One minimal `{DETAILS: {properties: {F: {type}}, required}}` as both INBOUND and OUTBOUND; a query or request-server name resolves to its source table |
 | Declared schemas (`TableDef.fields`, query and view `fields`, `derivedTypes`) | Used for metadata and rows | Ignored: metadata is inferred, rows go out as stored |
 | Commit events | Committed only on `EVENT_ACK`; `IGNORE_WARNINGS` lets returned warnings commit ([Commit semantics](#commit-semantics-validate-warnings-and-ignore_warnings)) | A handler's writes stay whatever it replies; warnings always NACK |
+| Values of declared columns | Held by type; BigDecimals sent as strings with their scale ([Value encoding](#value-encoding)) | As given |
 | A change to a table a view joins | Pushed only through joins with `backwardsJoin: true`, as the view rows it inserts, deletes or changes ([Views](#views-the-join-model)) | Every view row pushed as a MODIFY, whatever the join |
+| Snapshot row order | Newest first by `RECORD_ID`; `REVERSE` alone gives oldest first; `ORDER_BY` an index ([Dataserver paging](#dataserver-paging)) | The order the table holds the rows in |
 | `DATA_LOGON` `ORDER_BY` / `REVERSE` / `FIELDS`, criteria on unexposed fields | Honoured / `LOGON_NACK` | Ignored |
+| A `CRITERIA_MATCH` that isn't valid Groovy, is in this engine's `AND`/`OR`/`NOT`/`<>`/`=` dialect, calls a function GSF's validator doesn't allow or one no overload takes | `LOGON_NACK INVALID_CRITERIA` with GSF's message, no subscription ([Strict criteria](#strict-criteria)) | Malformed or unknown: fails open, every row, logged. The dialect and the refused functions: evaluated |
+| The criteria functions | GSF's, typed by the declared fields ([Criteria functions](#criteria-functions)) | The same, a date compared with also read as ISO-8601 |
 | `REQ_` | As in [Request servers](#request-servers-req_): `PARAMETRIC_TYPE`, criteria-only paging and ordering, `REQUEST` wildcards, ranges and arrays, a reply block, `MSG_NACK` failures | Every matching row in insertion order; `REQUEST` from the top level or `DETAILS`, `'*'`/`null`/`''` match anything, an array value any of its values; `CRITERIA_MATCH` fails open; no `RECORD_ID`/`TIMESTAMP`; errors as a `REP_` with `ERROR` |
 | Unknown resource | `MSG_NACK` 404 (the client's promise rejects) | A typed reply with `ERROR: [{CODE: 'UNKNOWN_RESOURCE'}]` (the promise resolves) |
 
@@ -766,13 +772,15 @@ eventSchemas: { EVENT_TRADE_BOOK: { className: 'com.acme.BookInput', fields: [/*
 appName: 'showcase', // enum class names: global.genesis.gen.dao.enums.<appName>.<table>.<Field>
 ```
 
-- **`FieldDef`** `{name, type, nullable?, values?, default?, maxSize?, minLength?, description?, title?, optional?}`.
+- **`FieldDef`** `{name, type, nullable?, values?, default?, maxSize?, minLength?, scale?, description?, title?, optional?}`.
   `type` is a dictionary type (`STRING ENUM INT SHORT LONG DOUBLE BIGDECIMAL BOOLEAN DATE
   DATETIME NANO_TIMESTAMP RAW`). Fields are nullable unless declared otherwise; key and
   generated fields default to not null. `description` replaces the JVM class name the real
   server puts there (leave it unset on dates: foundation-forms spots a date by
   `org.joda.time.DateTime`). `optional` decides whether an event payload may leave the field
   out (see events below); an ENUM with no `values` sends no `VALID_VALUES` and no `enum`.
+  `scale` is a BIGDECIMAL column's (default 5); values are held and sent by their declared
+  type, see [Value encoding](#value-encoding).
 - **Views** take their base table's fields and `select` aliases, each join's `select` columns
   typed from the joined table at any depth (nullable unless reached from the base through INNER
   joins only; see [Views: the join model](#views-the-join-model)), and `derivedTypes` for
@@ -824,6 +832,58 @@ Deviations: a bare table name with no request server of that name still gets the
 insert-shaped event schema from `JSON_SCHEMA_REQUEST` (this engine's fallback; the real server
 answers 404). Dataserver rows still carry neither RECORD_ID nor TIMESTAMP.
 
+## Value encoding
+
+A declared column's values are held by its type, and go on the wire the way
+GSF's serializer writes them (`GenesisSetMaskingJsonSerializer.kt`, GSF
+v8.15.29: a `DateTime` as epoch millis, a `BigDecimal` as
+`value.toPlainString()`):
+
+| Type | Held as (seed rows, inserts, update patches) | On the wire |
+|---|---|---|
+| `BIGDECIMAL` | A number rounded HALF_UP to the column's `scale` (a number or a numeric string is accepted) | A string with exactly `scale` decimals: `15` → `"15.00000"`, `1234.56789012` → `"1234.56789"` (recorded) |
+| `DATE` / `DATETIME` | Epoch millis (a number, a `Date`, `yyyyMMdd`, `yyyyMMdd-HH:mm[:ss[.SSS]]` or ISO-8601; ISO without an offset is UTC) | The number |
+| `INT` `SHORT` `LONG` `DOUBLE` `NANO_TIMESTAMP` | A number (a numeric string is read as one) | The number |
+| `BOOLEAN` | `true`/`false` (also the strings, any case) | The boolean |
+| `ENUM` | As given; a value not in `values` is logged once (the real table can't hold it) | The string |
+
+- **The scale.** A SQL-backed GSF table stores a BIGDECIMAL as
+  `NUMERIC(precision, scale)`, `(20, 5)` unless the dictionary says otherwise
+  (`BigDecimalSqlPrecisionConfig`, `SQLBaseTableGenerator.kt`), so every value
+  comes back with exactly that many decimals, and SendIt rounds a seed value
+  HALF_UP before writing it (`BigDecimalCsvTransformer.kt`). `FieldDef.scale`
+  says it (`field("FEE", BIGDECIMAL(20, 2))` is `{type: 'BIGDECIMAL', scale: 2}`).
+  A view's joined column keeps its table's scale; a derived BIGDECIMAL has no
+  column, so it goes out with its own decimals unless its `derivedTypes` entry
+  gives a `scale`.
+- **Numbers inside.** Criteria, `ORDER_BY`, `REQUEST` values and
+  derived fields see numbers, so `COMMISSION > 20` and an index on a BIGDECIMAL
+  compare numerically, as GSF's typed values do. A `REQ_` wildcard matches the
+  scaled text (`'*.50000'`). Precision is a JavaScript number's, about 15
+  significant digits; a `NUMERIC(20, 5)` value beyond that loses its last digits.
+- **Events.** An event's schema already demands a BIGDECIMAL as a string and a
+  date as an integer (a JSON number for a BIGDECIMAL is `VALIDATION_ERROR`, as on
+  GSF, whose validator isn't type-loose). After that check, a declared
+  BIGDECIMAL string reaches the handler as a number (unrounded, as GSF hands it
+  a deserialized `BigDecimal`); the column scale applies when it is written.
+- **What is left as given.** A key column (the table's `pkField`), so lookups by
+  the key a seed or a handler used keep working. An integer beyond 2^53, which
+  no JavaScript number holds exactly. A value that isn't its type (a date that
+  doesn't exist, `'n/a'` in a BIGDECIMAL): nothing is guessed, `Date.parse`
+  included. An undeclared table's values (no types to go by), except that a
+  `fieldTypes` override making a column a BIGDECIMAL sends it as a string with
+  the default scale. A request-server `resolver`'s rows. A view's joined column
+  keeps its table's scale when the view's base table is declared; over an
+  undeclared base the view's columns are inferred like the rest.
+- **Fidelity.** Under `fidelity: 'legacy'` nothing is converted. Seed rows are
+  converted when their table is registered and writes when they happen, so
+  switching `fidelity` on a running server changes how values are sent, not
+  how the rows already stored are held.
+- **`ROW_REF`** is the record's `RECORD_ID` as a string in GSF 8.15
+  (`GenesisSetMaskingJsonSerializer.kt`: `value.toString()`, recorded) and in
+  10.0 alike (it writes a `Long` ROW_REF as a string, v10.0.0-beta4), so there
+  is no option for its type (plan decision D1).
+
 ## Dataserver paging
 
 `DATA_LOGON` pages the way the real dataserver does (genesis-pal-dataserver's
@@ -834,6 +894,22 @@ answers 404). Dataserver rows still carry neither RECORD_ID nor TIMESTAMP.
   is clamped to `MAX_VIEW`, and `0` means "no limit" for either. These are
   also foundation-comms' `DatasourceDefaults`, so a grid that sets neither gets
   250 rows and then pages.
+- **Row order.** Without `ORDER_BY` the dataserver reads its primary index,
+  `RECORD_ID`, **newest first**, and `REVERSE: true` alone makes it oldest
+  first, the order the rows were written in (genesis-pal-dataserver
+  `PrimaryKeyDispenser.kt`: `if (reverse) Order.ASC else Order.DESC`;
+  recorded in `dataserver-order-by`). `ORDER_BY` names one of the query's
+  `indexes` and sorts ascending by its fields, ties by `RECORD_ID`; `REVERSE`
+  flips that; an unknown index is `LOGON_NACK INVALID_INDEX` (checked before
+  the criteria, as `ConnectionManager.verifyClientOptions` does). Without
+  `ORDER_BY`, a `CRITERIA_MATCH` that uses the leading fields of one of the
+  query's `indexes` makes the dataserver read through that index instead, so
+  the rows come in its order (the index matching the most leading fields, the
+  first on a tie: `CriteriaFilterBuilder.findBestMatchingIndex`,
+  `QueryClientFactory.build`). `REVERSE` is read as GSF reads a flag: `true`, or
+  any text `"true"` in any case. Every page follows the same order. A seed brought with the real server's `RECORD_ID`s
+  (as strings, above 2^53) sorts by their integer value. `DETAILS.FIELDS`
+  narrows the rows to those columns, dropping names the query doesn't expose.
 - The snapshot is the first page: at most `MAX_ROWS` rows, and `MORE_ROWS: true`
   when more are waiting. **`ROWS_COUNT`** is on the first page only, and counts
   like the real dataserver's index count: it **ignores `CRITERIA_MATCH` and the
@@ -887,6 +963,12 @@ cut off by `MAX_VIEW`, and the view rows a change to a joined table touches —
 get no MODIFYs or DELETEs at all. A waiting row's page later
 delivers its current values. Nothing absorbed uses a `SEQUENCE_ID`.
 
+A new row is pushed as soon as it arrives, whatever the order — also when it
+sorts among the rows still waiting (under `ORDER_BY`, or oldest first) — and
+no later page sends it again: the real `ClientIndex.processInsertUpdate` sends
+it straight away while the view has room (always, with the default moving
+view), and `NextRowsStrategy` skips rows the client already holds.
+
 An `INSERT` is pushed even when the view already holds `MAX_VIEW` rows. The
 real server would also evict the oldest row in the same frame (the moving
 view, a later item), so here the view can grow past `MAX_VIEW`; the next
@@ -922,7 +1004,7 @@ requestReplies: {
 | | Criteria-only (`criteriaOnly: true`) | Plain |
 |---|---|---|
 | `REQUEST` | Must be a top-level object (missing, `null`, an array or inside `DETAILS`: `MSG_NACK INTERNAL_ERROR`, as recorded); its fields are ignored | Top level only (`DETAILS.REQUEST` is never read); missing means every row. Each field selects rows: an exact value (converted to the field's type; `null` matches nulls, and nothing on a non-null field), a string with `*` (a wildcard, below), or `FIELD_FROM` + `FIELD_TO` (an inclusive range; one alone matches nothing). A field the source doesn't have matches nothing unless its value is `null`. An array runs each object and concatenates the results (`[]`: no rows) |
-| `CRITERIA_MATCH` | Filters the rows. Malformed: `MSG_NACK INTERNAL_ERROR` (recorded). An unknown field: `REPLY: []` (recorded), and `REQUEST_FAILED` "Paging not supported on lazy criteria evaluation" with an `OFFSET` past 0 | Filters the rows. Malformed: `MSG_NACK REQUEST_FAILED` with the parse error. An unknown field matches nothing |
+| `CRITERIA_MATCH` | Filters the rows. Malformed — refused by the Groovy check, or unparseable and not known to be valid Groovy ([Strict criteria](#strict-criteria)): `MSG_NACK INTERNAL_ERROR` (recorded). An unknown field: `REPLY: []` (recorded), and `REQUEST_FAILED` "Paging not supported on lazy criteria evaluation" with an `OFFSET` past 0 | Filters the rows. Malformed (as for criteria-only): `MSG_NACK REQUEST_FAILED` with the parse error. An unknown field matches nothing |
 | `MAX_ROWS` | The page size: `DETAILS.MAX_ROWS`, else `rowLimit`, else 10000. Below 1: `REQUEST_FAILED` "maxRows should be greater than 0" | The most rows each `REQUEST` object returns, same defaults. No signal that more exist |
 | `OFFSET` / `VIEW_NUMBER` | `OFFSET` skips that many rows. `VIEW_NUMBER` n (from 1) is the page at `MAX_ROWS × (n − 1)`, with `NEXT_VIEW: n + 1` while the page has rows. Both, or either negative: `REQUEST_FAILED` with GSF's text | Ignored |
 | `ORDER_BY` | `'FIELD [ASC\|DESC], …'` (grid-pro sends `'NAME DESC'`, `'RECORD_ID ASC'`); an index name or `PK` stands for its fields; ties go by `RECORD_ID` ascending, nulls sort low. Malformed, or a field the source doesn't have: `REQUEST_FAILED` with GSF's text (`Order fields [Asc(field=X)] not found on table T`). Without it, a first page comes in the table's order (oldest first, recorded) and a later page by primary key | Ignored: rows come in primary-key order (recorded: `RIGHT` in `CODE` order) |
@@ -951,8 +1033,11 @@ requestReplies: {
   which foundation-comms rejects the request's promise with. A `resolver`'s
   `NackError` keeps its items; anything else it throws is `GENERIC_ERROR`
   with the message (`AbstractCustomReqRep.kt`).
-- A `CRITERIA_MATCH` calling a function this engine's grammar doesn't model
-  (it may be valid Groovy) still fails open, logged, as on the dataserver.
+- A `CRITERIA_MATCH` this engine's grammar doesn't model but that GSF's
+  validator passes (`-QTY < -5`, `QTY == 5L`) still fails open, logged, as on
+  the dataserver. What the validator refuses (`NAME in ['A']`, `QTY * 2 > 1`,
+  a function off its whitelist), or a call no overload takes, is refused as a
+  malformed criteria ([Strict criteria](#strict-criteria)).
 
 What clients see:
 
@@ -1120,41 +1205,187 @@ extracted from for how each was actually found:
    (e.g. a NOTIONAL that happens to be round in the seed data), say so via
    the `fieldTypes` override on the query/request-reply.
 6. **`CRITERIA_MATCH` needs more than `==`.** `compileCriteria()`
-   (`src/db/criteria.ts`) supports `== != > >= < <= AND OR NOT` (also
-   `&&`/`||`/`!`) and parentheses via a small typed grammar (not
+   (`src/db/criteria.ts`) supports `== != > >= < <= && || !` and
+   parentheses (and, under `fidelity: 'legacy'` only, its own `AND OR NOT`,
+   `<>` and `=`: see [Strict criteria](#strict-criteria)) via a small typed grammar (not
    `eval`/`new Function` — a malformed criteria string can only fail to
-   parse, never execute arbitrary JS). It understands both function-call
-   dialects real clients emit: the `Expr.<fn>(field, value)` form from
-   `foundation-criteria`'s `serialisers.ts` / `foundation-ui`'s date-range
-   filter, and the `<FIELD>.<fn>(value)` method-call form project filter
-   builders generate (e.g. `DEAL_CURRENCIES.containsIgnoreCase('USD')`) —
-   both share the same functions: `contains`, `containsIgnoreCase` (substring
-   on strings, membership on array-valued fields), the Java/Groovy string
-   methods `equals`, `equalsIgnoreCase`, `startsWith`, `endsWith` (pbc-notify-ui's
-   inbox subscribes with `((ALERT_STATUS.equals("NEW")))`, and an unsupported
-   method makes the whole criteria fail to compile — the filter is dropped and
-   the grid shows every row), `dateIsToday`,
-   `dateIsEqual`, `dateIsGreaterEqual`, `dateIsLessEqual`, `dateTimeIsEqual`,
-   `dateTimeIsGreaterEqual`, `dateTimeIsLessEqual`, `dateTimeIsAfter`,
-   `dateTimeIsBefore` (`dateIs*` compares by calendar day, `dateTimeIs*` by
-   full timestamp). Date literals accept `'YYYYMMDD-HH:MM[:SS[.mmm]]'` — the
-   milliseconds matter: `getGroovyDateFormat()` emits them, and a parser that
-   rejects them makes every `dateTimeIs*` comparison silently false, so a
-   grid whose default filter is "Today" renders as an empty screen with no
-   warning. Any criteria this grammar still doesn't cover fails open instead
-   of crashing the handler: `compileCriteria()` logs a warning and returns a
-   match-all predicate, so `DATA_LOGON` still responds (unfiltered) rather
-   than leaving the client's grid stuck on "Loading...". (A request server
-   answers a malformed criteria with a `MSG_NACK`, as GSF does, and fails open
-   only on a function it doesn't model; see
-   [Request servers](#request-servers-req_).) Add new entries to
-   `EXPR_FUNCTIONS` in `criteria.ts` if you hit one that isn't covered yet.
+   parse, never execute arbitrary JS). It evaluates the calls GSF allows, as
+   GSF computes them (see [Criteria functions](#criteria-functions)): every
+   function on GSF's `Expr` whitelist, and the field methods `startsWith`,
+   `equals`, `equalsIgnoreCase`, `contains`, `containsIgnoreCase`,
+   `containsWordsStartingWithIgnoreCase` and `dateIsToday`
+   (pbc-notify-ui's inbox subscribes with `((ALERT_STATUS.equals("NEW")))`).
+   A criteria GSF refuses — not valid Groovy, a function off its whitelist, a
+   call no overload takes — is refused here too: `DATA_LOGON` gets
+   `LOGON_NACK INVALID_CRITERIA` and a request server a `MSG_NACK` (see
+   [Strict criteria](#strict-criteria)). Any other criteria this grammar
+   doesn't cover — it may be valid Groovy — fails open instead of crashing
+   the handler: `compileCriteria()` logs a warning and returns a match-all
+   predicate, so `DATA_LOGON` still responds (unfiltered) rather than leaving
+   the client's grid stuck on "Loading...". The exported `compileCriteria()`
+   and `filterByCriteria()` evaluate as GSF does by default;
+   `{ types }` (a `ReadonlyMap` of field → Genesis type) types the fields as
+   GSF's entity does, and `{ lenient: true }` evaluates as this engine did up
+   to 15.47 (`fidelity: 'legacy'` uses it — see
+   [Strict criteria](#strict-criteria)).
 7. **Composite-key AMEND/DELETE matching needs a fallback beyond the PK.**
    `Store.findRow()` (`src/db/store.ts`) tries the configured PK field, then
    any single field that uniquely identifies one row, then the full
    combination of provided fields — needed for tables where no single field
    is unique on its own.
 
+## Strict criteria
+
+GSF parses every `CRITERIA_MATCH` as Groovy before it opens a subscription
+(`CriteriaFilterBuilder.validateAndParseCriteriaOptions`:
+`AstBuilder().buildFromString(criteria)`, GSF v8.15.29, Groovy 3.0.25). One
+that doesn't parse gets `LOGON_NACK` alone, no `LOGON_ACK` and no subscription,
+so a later `DATA_LOGOFF` is the router's 404 (recorded:
+`dataserver-invalid-criteria`):
+
+```jsonc
+{ "MESSAGE_TYPE": "LOGON_NACK", "WARNING": [], "ERROR": [{
+  "@type": "StandardError", "CODE": "INVALID_CRITERIA", "STATUS_CODE": "400 Bad Request",
+  "TEXT": "MultipleCompilationErrorsException : startup failed:\nScript1791….groovy: 1: Unexpected input: '<EOF>' @ line 1, column 19.\n   NAME == ((( 'oops'\n                     ^\n\n1 error\n" }] }
+```
+
+**The engine's own dialect is refused too.** This grammar also takes `AND`,
+`OR` and `NOT` (any case), `<>` and a single `=`, which Groovy doesn't, and it
+reads a dotted name as a field and a `$` in a double-quoted string as text.
+(foundation-criteria puts every string value in double quotes, so a filter
+value with a `$` in it, such as `US$`, is refused by GSF, and now here.)
+GSF refuses all of them, at one of three stages
+(`CriteriaFilterBuilder.getCriteriaFilter`): Groovy's lexer or parser
+(`A == 1 AND B == 2`, `A <> 1`, `"US$"`), its `CriteriaValidatorVisitor` once
+Groovy has read them some other way (`NOT x` is a call on an implicit `this`,
+`A = 1` an assignment, `A.B` a property, `"a$b"` a GString), or the
+`@CompileStatic` compilation of the filter (`Expr.f(x) AND y` calls `AND` on a
+Boolean). `src/db/criteriaDialect.ts` gives GSF's text for the shapes clients
+write, with every other validator error in the order GSF lists them (an
+unknown field too, when the query's fields are known). It was run side by
+side with Groovy 3.0.25 and genesis-db 8.15.29's validator on about 1,300
+probe criteria: every text GSF's parser or validator gives matches. A
+compilation failure's text is the criteria as GSF compiled it, each field
+replaced by its getter:
+
+```text
+Error compiling criteria expression: Failed to compile criteria expression Expr.containsIgnoreCase(row.getName(), 'a') AND row.getQty() > 1
+```
+
+**So is a call GSF's validator doesn't allow.** Only the `Expr` functions on
+its whitelist (see [Criteria functions](#criteria-functions)) and seven field
+methods pass; anything else — `Expr.foo(X)`, `Expr.stringDateTime`, a call on
+a class (`Math.abs(QTY)`) or on an implicit `this` (`foo(X)`) — is
+`LOGON_NACK INVALID_CRITERIA` with the validator's text, every error in
+source order:
+
+```text
+GenericCriteriaValidationException : Criteria validation failed: Method call Expr.foo is not supported,Variable FOO is not allowed in criteria expression
+```
+
+That includes the functions this engine evaluated before GSF's semantics —
+`Expr.contains`, `Expr.equals`, `Expr.equalsIgnoreCase`, `Expr.startsWith`,
+`Expr.endsWith`, `Expr.dateTimeIsEqual`, `FIELD.endsWith`, and an `Expr`
+function called as a field method (`TRADE_DATE.dateIsEqual(...)`).
+foundation-criteria's `Serialisers.contains` and `Serialisers.endsWith` emit
+two of them. **And a call no overload takes**, which GSF's `@CompileStatic`
+compilation refuses: the wrong number of arguments
+(`Expr.containsIgnoreCase(NAME)`), an ambiguous `null`
+(`Expr.dateIsToday(null)`), and — where the fields' types are declared — an
+argument of the wrong type (`Expr.containsIgnoreCase(QTY, '1')` on a `LONG`),
+a field method the field's type lacks (`QTY.startsWith('1')`), or a
+relational operator between types that don't compare (`QTY > '5'`,
+`TRADE_DATE > 1.5`).
+Names and strings that merely contain a keyword (`NOTES`, `ORDER_ID`,
+`'x AND y'`) are not the dialect, and neither is a keyword used where a value
+goes (`AND == 1`: GSF's unknown-field check answers that). Request servers
+refuse the dialect as any malformed criteria: `MSG_NACK INTERNAL_ERROR`, or
+`REQUEST_FAILED` with GSF's text on a plain server.
+
+**And Groovy the grammar doesn't read, as GSF's validator answers it.**
+`src/db/groovyValidator.ts` parses such a criteria as Groovy does (Groovy
+3.0.25's precedence) and repeats the validator's walk over it: a list
+(`NAME in ['A']`: "List expressions are not supported in criteria"),
+arithmetic (`QTY * PRICE > 10`: "Binary operator * not allowed in criteria
+expression"), a ternary, an Elvis, a range, a map, a cast, a closure, `=~`,
+`===`, `instanceof`, a subscript, `++`, a property, every error in the order
+the validator meets it. A constructor call other than `BigDecimal`,
+`Integer`, `Long`, `Double` and `String` is refused ("Constructor calls for
+java.util.Date are not allowed in criteria expressions"), one of a class
+Groovy can't resolve fails to compile ("unable to resolve class Foo"); those
+five are evaluated (`QTY > new BigDecimal('1.5')`). Checked against Groovy
+3.0.25 and genesis-db 8.15.29's validator (`test/groovyValidator.test.ts`).
+
+This is the default. This engine's criteria grammar is still smaller than
+Groovy's, so it can't refuse whatever it fails to parse: `-QTY < -5` or
+`QTY == 5L` are valid Groovy the validator passes and it doesn't evaluate.
+`src/db/groovySyntax.ts` refuses only the mistakes it can name the way Groovy
+does: an operator with nothing after it (`QTY > 1 &&`), a parenthesis left open
+or closing nothing, an operator where an operand must start (`== 'x'`,
+`A && || B`), an unterminated string (an unescaped `'` in `'O'Brien'`), and a
+`#`. The text, line and column are Groovy's for those cases (checked by running
+Groovy 3.0.25 on them; in a few ANTLR names a different token at the same
+column). Anything else it doesn't understand gets no verdict, and the criteria
+fails open as before: also a `/` anywhere (a slashy string such as
+`/O'Brien/`, or a comment), and nesting too deep to check. Request servers
+refuse what this check refuses, and what their grammar can't parse unless the
+check found it valid Groovy (`-QTY < -5`), which fails open there too
+(see [Request servers](#request-servers-req_)).
+
+Under `fidelity: 'legacy'` criteria are evaluated as up to 15.47
+(`src/db/legacyCriteria.ts`; `compileCriteria(criteria, { lenient: true })`):
+a malformed one, a bare field or an unknown function fails open; the
+dialect is evaluated (`AND` as `&&`, `<>` as `!=`, `=` as `==`); comparisons
+are JavaScript's (`QTY > -1` holds for a null `QTY`, `QTY == null` misses a
+row without the field); a function answers `false` rather than throw, so
+`!STATUS.equals('NEW')` keeps a row with no `STATUS`; `dateTimeIsAfter` and
+`dateTimeIsBefore` are inclusive; dates are also read by `Date.parse`. The
+GSF functions 15.47 didn't have (`containsWordsStartingWithIgnoreCase`,
+`isNullOrBlank`, `notNullOrBlank`, `dateIsAfter`, `dateIsBefore`,
+`dateTimeIsInRange`) are evaluated in that style instead of failing open.
+There is no separate option: a `CRITERIA_MATCH` the real server can't compile
+is a client bug the mock shouldn't hide.
+
+## Criteria functions
+
+What each call computes is GSF's (`Expr.java`, `TimeUtils.java`,
+`ExtraOperators.kt`, `TimeRange.kt`, genesis-commons v8.15.29;
+`src/db/gsfExpr.ts`), with every overload: GSF compiles a criteria against
+the entity's Java types, so a `STRING` or `ENUM` field is a `String`, `DATE`
+and `DATETIME` a Joda `DateTime`, `LONG` a `Long`, `INT` an `Integer` (which
+GSF widens where a function takes a `Long`). The
+evaluator does the same with the declared types; a field without one is read
+by its value (a number as a `Long` of epoch millis, a string as a `String`,
+also read as ISO-8601 when it isn't one of GSF's formats, as mock rows often
+hold). GSF runs in UTC, and so do these.
+
+| Function | What it does |
+|---|---|
+| `Expr.containsIgnoreCase(text, search)`, `FIELD.containsIgnoreCase(search)` | `text` contains `search`, ignoring case. `null` either side: `false`; `''`: `true`. A mock row's list value (`['USD', 'GBP']`): membership |
+| `Expr.containsWordsStartingWithIgnoreCase(text, search[, delimiter])`, and as a field method | A word of `text` — split on the delimiter's first character, `' '` by default — starts with `search`, ignoring case (grid-pro's "Word starts with": `'Tra'` finds `Global Trade Ltd.`, not `Market Solutions`). `null`: `false` |
+| `Expr.isNullOrBlank(text)`, `Expr.notNullOrBlank(text)` | `null`, or only whitespace (`String.isBlank`) |
+| `Expr.dateIsToday(date)`, `FIELD.dateIsToday()` | The date's UTC day is today |
+| `Expr.dateIsEqual / dateIsBefore / dateIsAfter / dateIsGreaterEqual / dateIsLessEqual(date, comparableDate)` | By day. A `DateTime` or `Long` date is compared with a `yyyyMMdd` date only; a `String` date and its comparable may use any of GSF's formats (below) |
+| `Expr.dateTimeIsAfter / dateTimeIsGreaterEqual / dateTimeIsBefore / dateTimeIsLessEqual(date, comparableDate)` | By instant. `After` and `Before` are strict: foundation-ui's date-range filter (`dateTimeIsAfter(F, min) && dateTimeIsBefore(F, max)`) leaves out both bounds |
+| `Expr.dateTimeIsInRange(date, timeRange, timeContext)` | The date is in the `HOUR`, `DAY`, `WEEK` (from Monday), `WEEK_DAY` (a weekend's current one is empty, its previous one Friday), `MONTH` or `YEAR` that is `CURRENT` or `PREVIOUS`. The names are case sensitive |
+| `Expr.longLocalDate(millis)`, `Expr.stringLocalDate(date)`, `Expr.longDateTime(millis)`, `Expr.stringDateTime(date)` | A day or an instant, to compare (`Expr.longLocalDate(D) == Expr.stringLocalDate('20240101')`). `stringDateTime` exists, but GSF's validator refuses it |
+| `FIELD.startsWith(prefix[, offset])`, `FIELD.equals(value)`, `FIELD.equalsIgnoreCase(text)`, `FIELD.contains(text)` | Java's. They dereference the field: a `null` one throws. `equals` wants the same class: a `LONG` field never `equals(5)` (an `Integer`) |
+
+GSF's date formats (`TimeUtils.ALLOWED_DATE_PARSER`) are `yyyyMMdd-HH:mm:ss.SSS`,
+`yyyyMMdd-HH:mm:ss`, `yyyyMMdd-HH:mm`, `yyyyMMdd` and `yyyy-MM-dd`. Joda reads
+fewer digits than a pattern shows (`2024-1-1`, `20240101-9:5`) and up to three
+of a fraction (`.5` is 500 ms); nothing else parses — no `T`, no offset, no
+spaces. A `DateTime` field also compares with a date string, epoch millis or
+another `DateTime` (`TRADE_DATE >= '20240101' && TRADE_DATE < '20240102'`,
+ExtraOperators.kt); `==` takes one naming its instant.
+
+A call that throws in GSF — an unparseable date, a `null` a function
+dereferences, an unknown `TimeRange` — makes the row not match, however the
+criteria goes on: `!Expr.dateIsEqual(TRADE_DATE, '2024-01-01')` matches no
+row. The evaluator's verdicts were checked against GSF's own: about 570
+criteria compiled and run by Groovy 3.0.25 with genesis-commons 8.15.29 and
+Joda-Time 2.14.0 over five rows (`test/gsfCriteriaCases.ts`).
+
 ## Live criteria membership, and known deviations
 
 A live **MODIFY** that moves a row out of a subscription's `CRITERIA_MATCH`, or
@@ -1178,13 +1409,29 @@ Where this engine still differs from the real server:
   nothing then (see [Views](#views-the-join-model)).
 - There is no **moving view**. Once the view holds `MAX_VIEW` rows, live
   INSERTs are still pushed instead of also evicting the oldest row.
-- Without `ORDER_BY`, snapshots are in **insertion order** (with or without
-  `REVERSE`), where the real default is newest first by `RECORD_ID`. `ORDER_BY` names one of
-  the query's `indexes` and sorts ascending by its fields, ties by `RECORD_ID`; `REVERSE`
-
-  flips that; an unknown index is `LOGON_NACK INVALID_INDEX`. Under `ORDER_BY`, a live INSERT
-  that sorts among the rows still waiting for `MORE_ROWS` waits with them, in order (a MODIFY
-  of a waiting row keeps its place). `DETAILS.FIELDS` narrows the rows to those columns.
+- **`ROWS_COUNT` under a criteria-chosen index.** When a criteria makes the
+  dataserver read through an index (see [Dataserver paging](#dataserver-paging)),
+  GSF narrows the read to the range the criteria allows on that index
+  (`IndexBuilderVisitor`), and `ROWS_COUNT` counts that range only. The mock
+  counts the whole query, as it does for any other criteria.
+- A MODIFY of a row still **waiting for `MORE_ROWS`** updates the waiting copy
+  and pushes nothing. GSF's client index doesn't know the row, so it pushes the
+  change as an `INSERT` with the full row (while the view has room, or always
+  with the moving view) and the later page skips it
+  (`ClientIndex.processModifyUpdate` → `putNewRowEntry`, v8.15.29).
+- **Criteria types** come from declared fields (and `fieldTypes` /
+  `derivedTypes`) only. A field without one is read by its value, and a call
+  or comparison it takes part in isn't type-checked: GSF would refuse
+  `Expr.containsIgnoreCase(QTY, '1')` on any `LONG` column; an inferred
+  `QTY` is just never a String, so no row matches. An `ENUM`'s getter in a
+  compilation error's text needs the declared type too.
+- `FIELD.equals(1.50)` on a `BIGDECIMAL` compares the values; GSF's
+  `BigDecimal.equals` also compares the scale, which depends on its database.
+- GSF's `Expr.java` (v8.15.29) caches `singlePatternLocalDate` and
+  `stringLocalDate` in one `STRING_LOCAL_DATE_CACHE`, so for up to 60 s after
+  `stringLocalDate('2024-01-01')` ran, a DateTime's `dateIsEqual(F,
+  '2024-01-01')` gets the cached day instead of throwing. Not modelled: each
+  call parses as its own overload does.
 - A `CRITERIA_MATCH` naming a field the query doesn't expose is `LOGON_NACK
   INVALID_CRITERIA`, as on GSF — when the query's fields are known (declared, or a `fields`
   block). An inferred query can't tell, and evaluates the criteria on the stored row.

package/dist/db/criteria.d.ts

@@ -1,18 +1,37 @@
 import type { Row } from '../types.ts';
 export type Predicate = (record: Row) => boolean;
-export declare class CriteriaSyntaxError extends Error {
-    name: string;
+export { CriteriaSyntaxError, type CriteriaToken, parseGenesisDateTime, tokenizeCriteria, UnsupportedCriteriaError, } from './criteriaTokens.ts';
+/**
+ * How {@link compileCriteria} and {@link filterByCriteria} read a criteria.
+ * Without options they evaluate it as GSF 8.15 does, each field read by its
+ * value.
+ * @public
+ */
+export interface CriteriaOptions {
+    /**
+     * Each field's Genesis type (`STRING`, `DATETIME`, ...), where it is
+     * declared: GSF compiles a criteria against these, which picks each
+     * function's overload (a `DATETIME` is a Joda DateTime, a `STRING` a
+     * String).
+     */
+    types?: ReadonlyMap<string, string>;
+    /**
+     * `fidelity: 'legacy'`: evaluate the criteria as this engine did up to
+     * 15.47 — its comparisons, its function semantics (inclusive
+     * `dateTimeIsAfter`, `false` instead of an exception, dates read by
+     * `Date.parse` too), its `AND`/`OR`/`NOT` dialect — plus the GSF functions
+     * it didn't have. `types` is ignored.
+     */
+    lenient?: boolean;
 }
-export declare class UnsupportedCriteriaError extends Error {
-    name: string;
-}
-export declare function parseGenesisDateTime(value: unknown): number | null;
 export interface ParsedCriteria {
     predicate: Predicate;
     fields: string[];
 }
-export declare function parseCriteria(criteria?: string | null): ParsedCriteria;
+export declare function parseCriteria(criteria?: string | null, options?: CriteriaOptions): ParsedCriteria;
 /** Compiles a CRITERIA_MATCH string into a predicate function(record) => boolean. */
-export declare function compileCriteria(criteria?: string | null): Predicate;
+export declare function compileCriteria(criteria?: string | null, options?: CriteriaOptions): Predicate;
 export declare function criteriaFields(criteria?: string | null): string[] | undefined;
-export declare function filterByCriteria(rows: Row[], criteria?: string | null): Row[];
+export declare function filterByCriteria(rows: Row[], criteria?: string | null, options?: CriteriaOptions): Row[];
+export declare function compileFailureText(criteria: string, fields: Iterable<string>, types?: ReadonlyMap<string, string>): string;
+export declare function criteriaCompileRefusal(criteria: string, types?: ReadonlyMap<string, string>): string | undefined;