@lossless.org/client 1.2.0 → 1.4.0

package/readme.md

@@ -17,11 +17,11 @@ Document access works immediately: `mongodb` is a dependency because the NoSQLDB
 | Family | Additional install |
 | --- | --- |
 | `nosqldb` connections, `@lossless.org/client/nosqldb`, `@lossless.org/client/testsupport` | none |
-| `sqldb` connections with backend `sqldb` or `mariadb` | `pnpm add mariadb` |
-| `sqldb` connections with backend `clickhouse` | `pnpm add @clickhouse/client` |
+| `sqldb` connections with backend `sqldb` or `mariadb`, `@lossless.org/client/sqldb/mariadb` | `pnpm add mariadb` |
+| `sqldb` connections with backend `clickhouse`, `@lossless.org/client/sqldb/clickhouse` | `pnpm add @clickhouse/client` |
 | `objectstorage` connections with backend `s3` | `pnpm add @aws-sdk/client-s3` |
 
-The aggregate entry point imports a family only when the configuration names it, so an absent driver stays absent until it is used; `connect()` then fails with `driver_missing`, naming the package, the entry point and the install command. Importing `@lossless.org/client/sqldb` directly needs both relational drivers, because that entry point exports `SqlConnection` and `ClickHouseConnection` as values. `optionalDrivers` exposes the same table to tooling.
+The aggregate entry point imports a family only when the configuration names it, so an absent driver stays absent until it is used; `connect()` then fails with `driver_missing`, naming the package, the entry point and the install command. The two relational families also have an entry point each: `@lossless.org/client/sqldb/mariadb` needs `mariadb` only and `@lossless.org/client/sqldb/clickhouse` needs `@clickhouse/client` only, while `@lossless.org/client/sqldb` is their union and needs both, because it exports `SqlConnection` and `ClickHouseConnection` as values. Import the family you use. A direct family import resolves its driver as the module loads, so an absent driver fails there as Node's `ERR_MODULE_NOT_FOUND` naming the package; `driver_missing` is the aggregate entry point's refusal, and it names the family entry point that needs the driver. `optionalDrivers` exposes the same table to tooling.
 
 ```ts
 import { LosslessOrgClient } from '@lossless.org/client';
@@ -72,7 +72,9 @@ Literal connection names and backend discriminators determine the return types.
 | --- | --- |
 | `@lossless.org/client` | `LosslessOrgClient`, configuration, capabilities, readiness, `LosslessClientError` |
 | `@lossless.org/client/nosqldb` | Migrated SmartData models, decorators, collections, cursors, sessions, exact persistence and administration; `NoSqlConnection` |
-| `@lossless.org/client/sqldb` | `SqlConnection`, `SqlTransaction`, `SqlTable`, `ClickHouseConnection`, `SmartClickHouseDb`, tables, query builders, `TimeDataTable`, `MetricWriter` |
+| `@lossless.org/client/sqldb` | Both relational families: `SqlConnection`, `SqlTransaction`, `SqlTable`, `ClickHouseConnection`, `SmartClickHouseDb`, tables, query builders, `TimeDataTable`, `MetricWriter` |
+| `@lossless.org/client/sqldb/mariadb` | `SqlConnection`, `SqlTransaction`, `SqlTable`, their options and `quoteIdentifier` |
+| `@lossless.org/client/sqldb/clickhouse` | `ClickHouseConnection`, `SmartClickHouseDb`, tables, query builders, `TimeDataTable`, `MetricWriter`, their options and `quoteClickHouseIdentifier` |
 | `@lossless.org/client/objectstorage` | Migrated `SmartBucket`, buckets, directories, files, metadata, watchers, exact operations; `ObjectStorageConnection` |
 | `@lossless.org/client/testsupport` | Explicit disposable-database testing helpers |
 
@@ -85,9 +87,33 @@ Family imports preserve the established SmartData and SmartBucket names and cons
 | Bounded plural upsert | `Model.atomicUpsertMany(operations, opts?)` — 1..1000 per-document filter/update pairs in one unordered round trip |
 | Content-addressed primary keys | model option `identityAsDocumentId` on `defineCollectionModel()` and `@Collection()` |
 | Identities on a migrated unique index | `@unI({ indexName })` names the index that backs the identity instead of `<field>_1` |
+| Typed `null` predicates | read and atomic filters accept `null` for a field declared optional, so a value stored as an explicit null stays addressable |
+| Owned sessions on cursors | `getCursor()` takes the handle from `db.createSession()` and keeps it leased until the cursor is closed |
+| Named plural selectors | a non-empty `$in` anchors `atomicUpdateMany()`, `atomicDeleteMany()` and `atomicUpsertMany()`; singular operations are unchanged |
+| Fields only atomic writes may update | `@svDb({ atomicOnly: true })` and `atomicOnlyFields` — seeded by every write that creates the document, never overwritten by a later `save()` |
+| Defaults inside `$expr` | `{ $ifNull: ['$field', literal] }` operands, so a guard covers rows written before the field existed |
+| Fenced instance saves | `instance.saveIf(fence, opts?)` — one non-upserting `updateOne`; `matchedCount === 0` means another writer moved the document first |
+| Identity rotation | `Model.atomicFindOneAndReplace(filter, replacement, opts?)` — the one write that moves an `@unI()` identity, under the matched document's `_id` |
+| Fail-fast on unprepared collections | a session-carrying call inside a transaction refuses instead of stalling on lazy index DDL until the client deadline |
+| Migration-owned indexes | `toleratedIndexNames` on the model — named undeclared indexes are observed, never created, dropped or verified, and no longer make the topology divergent |
 
 `docs/source/smartdata/readme.md` documents each one with its refusals.
 
+### Document APIs that need a MongoDB backend
+
+The document model runs on both NoSQLDB and MongoDB, but four APIs do not mean the same thing on both. The NoSQLDB engine is qualified at 8.0.2 and 10.2.0 (see [Verification](#verification)); both behave identically on every row below.
+
+| Document API | On the NoSQLDB engine | On MongoDB |
+| --- | --- | --- |
+| `getGroupedTotals()` / `getGroupedTotalsPage()` with one `groupBy` field | exact, including `sums` | exact |
+| the same with **two** `groupBy` fields | **returns a wrong answer without raising** — the engine leaves the composite group key unresolved and answers with a single row whose group values are the literal field paths (`'$owner'`) and whose count is the whole filtered collection. Do not call the two-field form against NoSQLDB | exact |
+| guarded `$expr` atomic filters (`$add`, `$lte`, `$ifNull`) | refused by the engine: the fail-closed compilation uses `$cond`, which the engine rejects with `invalid argument: $cond`. The document is left untouched | exact |
+| `watch()` change streams | unsupported. `NoSqlConnection` refuses with `unsupported_capability`; a plain `SmartdataDb` carries no backend declaration, so the refusal instead arrives from the engine as `CommandNotFound` once the change stream is read | supported on replica sets |
+| model-declared partial indexes (`partialFilterExpression`) | not expressible: the index-option type rejects it and the runtime refuses it with `invalid_configuration`; the engine cannot create one either. Migration-owned partial indexes stay reachable through `toleratedIndexNames` | not expressible, but a migration may own one |
+| aggregation-pipeline (array) updates | refused by the client with `invalid_argument` before any backend is contacted | same client refusal |
+
+`test/nosqldb/smartdb/test.engine-capability-boundary.node.ts` asserts the grouped-totals, guarded-`$expr`, partial-index and pipeline-update rows against both qualified engines, so a boundary that moves shows up as a failing suite rather than as a changed number in a consumer. The `watch()` row is the contract refusal `ts/nosqldb/classes.collection.ts` states; the engine suite opens no change stream.
+
 ## Relational SQL
 
 `SqlConnection` uses the official MariaDB connector for parameters, protocol, TLS and decoding. This client owns cancellable admission and TCP sockets from authentication onward. A transaction holds one physical session; releasing a session resets its state before reuse.
@@ -160,9 +186,20 @@ Fixes land here. @push.rocks/smartdata is no longer a release channel for this i
 
 ## Verification
 
-`pnpm build` builds production declarations. `pnpm test` checks aggregate lifecycle and writers. `pnpm run test:nosqldb`, `test:mongodb`, `test:objectstorage` and `test:sqldb` exercise isolated backends. `pnpm run test:package` packs the package and resolves every entry point; `pnpm run test:install` installs the packed tarball into a scratch project and proves that a document-only consumer receives no relational or object driver, and that the aggregate client refuses a missing one by name. Wrappers own and remove their disposable servers, containers and data directories. Never supply a production database or bucket to these destructive suites.
+`pnpm build` builds production declarations. `pnpm test` checks aggregate lifecycle and writers. `pnpm run test:nosqldb`, `test:nosqldb:qualified`, `test:mongodb`, `test:objectstorage` and `test:sqldb` exercise isolated backends. `pnpm run test:qualification` runs the aggregate, both engine suites and the MongoDB suite in one sequence; it is what the release preflight runs. `pnpm run test:package` packs the package and resolves every entry point; `pnpm run test:install` installs the packed tarball into a scratch project and proves that a document-only consumer receives no relational or object driver, and that the aggregate client refuses a missing one by name. Wrappers own and remove their disposable servers, containers and data directories. Never supply a production database or bucket to these destructive suites.
+
+Qualified engines:
 
-Qualification includes NoSQLDB 8.0.2, MongoDB 8.0.26 replica sets, ObjectStorage 10.1.0, SQLDB 0.2.3, MariaDB 11.8 and ClickHouse 25.8. `pnpm run test:minio` uses a digest-pinned MinIO image: `sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e`. That MinIO version enforces the tested conditional uploads but does not enforce conditional deletion; its exact-purge capability is correctly unavailable. ObjectStorage passes both live probes. AWS S3 has not been qualified against a live account in this migration; no universal S3 exact-operation guarantee is inferred from the SDK.
+| Component | Qualified versions |
+| --- | --- |
+| NoSQLDB engine | **8.0.2 and 10.2.0** — the same `test/nosqldb/smartdb/` suite runs against both, through the single engine seam in `test/nosqldb/helpers/smartdb.ts`; `NOSQLDB_ENGINE=qualified` selects 10.2.0 |
+| MongoDB | 8.0.26 replica sets |
+| ObjectStorage | 10.1.0 |
+| SQLDB | 0.2.3 |
+| MariaDB | 11.8 |
+| ClickHouse | 25.8 |
+
+Both engine versions answer every assertion in the document suite identically, including the asserted boundaries in [Document APIs that need a MongoDB backend](#document-apis-that-need-a-mongodb-backend). Text search, the Lucene adapter and the broad read-filter surface are exercised against MongoDB only. `pnpm run test:minio` uses a digest-pinned MinIO image: `sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e`. That MinIO version enforces the tested conditional uploads but does not enforce conditional deletion; its exact-purge capability is correctly unavailable. ObjectStorage passes both live probes. AWS S3 has not been qualified against a live account in this migration; no universal S3 exact-operation guarantee is inferred from the SDK.
 
 ## License and Legal Information
 

package/ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@lossless.org/client',
-  version: '1.2.0',
+  version: '1.4.0',
   description: 'One typed client for NoSQLDB, MongoDB, SQLDB, MariaDB, ClickHouse and S3 object storage.'
 }

package/ts/classes.losslessorgclient.ts

@@ -1,9 +1,9 @@
 import * as plugins from './plugins.js';
 import type { NoSqlConnection, INoSqlConnectionOptions } from './nosqldb/classes.connection.js';
 import type { ObjectStorageConnection, IObjectStorageConnectionOptions } from './objectstorage/classes.connection.js';
-import type { SqlConnection } from './sqldb/classes.sqlconnection.js';
+import type { SqlConnection, ISqlConnectionOptions } from './sqldb/classes.sqlconnection.js';
 import type { ClickHouseConnection } from './sqldb/classes.clickhouseconnection.js';
-import type { ISqlConnectionOptions, IClickHouseConnectionOptions } from './sqldb/interfaces.js';
+import type { IClickHouseConnectionOptions } from './sqldb/interfaces.js';
 import type { IOperationOptions, IReadiness } from './core/interfaces.js';
 import { LosslessClientError } from './core/classes.error.js';
 import { Operation } from './core/classes.operation.js';

package/ts/core/drivers.ts

@@ -14,9 +14,9 @@ export interface IOptionalDriver {
 }
 
 export const optionalDrivers: Readonly<Record<TOptionalDriverBackend, IOptionalDriver>> = Object.freeze({
-  mariadb: { package: 'mariadb', subpath: '@lossless.org/client/sqldb',
+  mariadb: { package: 'mariadb', subpath: '@lossless.org/client/sqldb/mariadb',
     usage: 'sqldb connections with backend "sqldb" or "mariadb"' },
-  clickhouse: { package: '@clickhouse/client', subpath: '@lossless.org/client/sqldb',
+  clickhouse: { package: '@clickhouse/client', subpath: '@lossless.org/client/sqldb/clickhouse',
     usage: 'sqldb connections with backend "clickhouse"' },
   s3: { package: '@aws-sdk/client-s3', subpath: '@lossless.org/client/objectstorage',
     usage: 'objectstorage connections with backend "s3"' },

package/ts/nosqldb/classes.atomicdelete.ts

@@ -26,8 +26,7 @@ export const executeAtomicDelete = async <T>(
     {
       ordinaryWrite: true,
       prepared: collectionArg.isInitializedForCurrentDatabase(),
-      preparationMessage:
-        `Initialize collection "${collectionArg.collectionName}" before using an owned SmartData session.`,
+      collectionName: collectionArg.collectionName,
     },
     async (rawSessionArg) => {
       await collectionArg.init();
@@ -64,8 +63,7 @@ export const executeAtomicDeleteMany = async <T>(
     {
       ordinaryWrite: true,
       prepared: collectionArg.isInitializedForCurrentDatabase(),
-      preparationMessage:
-        `Initialize collection "${collectionArg.collectionName}" before using an owned SmartData session.`,
+      collectionName: collectionArg.collectionName,
     },
     async (rawSessionArg) => {
       await collectionArg.init();

package/ts/nosqldb/classes.atomicfindoneandupdate.ts

@@ -20,6 +20,10 @@ export type TAtomicFindOneAndUpdateExecutionResult =
   | { status: 'upserted'; document: TRawDocument | null }
   | { status: 'not_matched'; document: null };
 
+export type TAtomicFindOneAndReplaceExecutionResult =
+  | { status: 'matched'; document: TRawDocument }
+  | { status: 'not_matched'; document: null };
+
 const unsupportedResult = (): never => {
   throw new SmartdataPersistenceError(
     'unsupported_operation',
@@ -104,8 +108,7 @@ export const executeAtomicFindOneAndUpdate = async <T>(
     {
       ordinaryWrite: true,
       prepared: collectionArg.isInitializedForCurrentDatabase(),
-      preparationMessage:
-        `Initialize collection "${collectionArg.collectionName}" before using an owned SmartData session.`,
+      collectionName: collectionArg.collectionName,
     },
     async (rawSessionArg) => {
       if (policy && !rawSessionArg?.inTransaction()) {
@@ -150,3 +153,67 @@ export const executeAtomicFindOneAndUpdate = async <T>(
     },
   );
 };
+
+/**
+ * Executes a model-validated atomic find-one-and-replace against MongoDB.
+ *
+ * This helper is package-internal. `SmartDataDbDoc.atomicFindOneAndReplace()`
+ * owns the public API and must validate the filter and the replacement body
+ * before calling this raw persistence boundary. The replacement never carries
+ * a stored `_id`, so the matched document keeps its own primary key while its
+ * declared identity changes.
+ */
+export const executeAtomicFindOneAndReplace = async <T>(
+  collectionArg: SmartdataCollection<T>,
+  filterArg: plugins.mongodb.Filter<plugins.mongodb.Document>,
+  replacementArg: plugins.mongodb.WithoutId<plugins.mongodb.Document>,
+  optionsArg: {
+    returnDocument: 'before' | 'after';
+    session?: TSmartdataOrdinarySession;
+  },
+): Promise<TAtomicFindOneAndReplaceExecutionResult> => {
+  if (collectionArg.getBoundModelSchema()?.ordinaryPersistence) {
+    // A validated ordinary document owns its stored shape and primary key, so
+    // it is written through its own postimage path, never replaced wholesale.
+    throw new SmartdataPersistenceError('unsupported_operation',
+      'Validated ordinary models require transactional postimage updates.');
+  }
+  return runWithOrdinarySmartdataSession(
+    optionsArg.session,
+    collectionArg.smartdataDb,
+    {
+      ordinaryWrite: true,
+      prepared: collectionArg.isInitializedForCurrentDatabase(),
+      collectionName: collectionArg.collectionName,
+    },
+    async (rawSessionArg) => {
+      await collectionArg.init();
+      try {
+        const result = await collectionArg.mongoDbCollection.findOneAndReplace(
+          filterArg,
+          replacementArg,
+          {
+            includeResultMetadata: true,
+            returnDocument: optionsArg.returnDocument,
+            upsert: false,
+            session: rawSessionArg,
+          },
+        );
+        const classified = classifyResult(result, {
+          returnDocument: optionsArg.returnDocument,
+          upsert: false,
+        });
+        if (classified.status === 'upserted') {
+          // Unreachable: this operation never requests an upsert.
+          return unsupportedResult();
+        }
+        return classified;
+      } catch (errorArg) {
+        return normalizeOrdinaryPersistenceError(
+          errorArg,
+          `Atomic find-one-and-replace conflicts with a unique index in collection "${collectionArg.collectionName}".`,
+        );
+      }
+    },
+  );
+};

package/ts/nosqldb/classes.atomicupdate.ts

@@ -28,8 +28,7 @@ export const executeAtomicUpdate = async <T>(
     {
       ordinaryWrite: true,
       prepared: collectionArg.isInitializedForCurrentDatabase(),
-      preparationMessage:
-        `Initialize collection "${collectionArg.collectionName}" before using an owned SmartData session.`,
+      collectionName: collectionArg.collectionName,
     },
     async (rawSessionArg) => {
       await collectionArg.init();
@@ -83,8 +82,7 @@ export const executeAtomicUpsertMany = async <T>(
     {
       ordinaryWrite: true,
       prepared: collectionArg.isInitializedForCurrentDatabase(),
-      preparationMessage:
-        `Initialize collection "${collectionArg.collectionName}" before using an owned SmartData session.`,
+      collectionName: collectionArg.collectionName,
     },
     async (rawSessionArg) => {
       await collectionArg.init();
@@ -142,8 +140,7 @@ export const executeAtomicUpdateMany = async <T>(
     {
       ordinaryWrite: true,
       prepared: collectionArg.isInitializedForCurrentDatabase(),
-      preparationMessage:
-        `Initialize collection "${collectionArg.collectionName}" before using an owned SmartData session.`,
+      collectionName: collectionArg.collectionName,
     },
     async (rawSessionArg) => {
       await collectionArg.init();