@lossless.org/nosqldb 10.7.3 → 10.8.1

package/readme.md

@@ -250,11 +250,17 @@ Each entry contains:
 
 ---
 
+## 🔢 Document Field Order
+
+Like MongoDB, every stored document leads with `_id`. Inserts, upserts, replacements, `findAndModify`, `$out` and `$merge` move a client-supplied `_id` to the front (the Node.js driver appends its generated `_id` after the caller's fields) and generate a missing one there. Every other field keeps its written order: update operators change existing fields in place and append the fields they create, and a replacement keeps its own field order after `_id`. Documents stored with a trailing `_id` by 10.7.3 and earlier keep it until an update, replacement or `$merge` changes them.
+
+---
+
 ## ✋ No-Op Write Detection
 
 The engine detects document rewrites that change nothing and skips them entirely — no storage write, no WAL append, no index update, and no oplog entry. A skipped rewrite still counts as matched (`matchedCount: 1`) but reports `modifiedCount: 0`.
 
-Only identical field values are skipped; top-level field order does not cause a write. Explicit changes to `_updatedAt` are persisted and returned just like changes to any other field. Applications decide when to advance their timestamps.
+As in MongoDB, a rewrite is skipped only when the document stays identical: the same fields in the same order with the same values and BSON types, at every depth. Only the position of `_id` is ignored, so a document stored with a trailing `_id` by 10.7.3 and earlier is not rewritten by a write that changes nothing. A replacement or `$set` that only reorders fields is written. Explicit changes to `_updatedAt` are persisted and returned just like changes to any other field. Applications decide when to advance their timestamps.
 
 Counters are exposed through `serverStatus`:
 
@@ -390,9 +396,9 @@ semantics. A range is built from `$gt`, `$gte`, `$lt` and `$lte`, and from
 `$ne: null` or a `$nin` list holding null, which read only the non-null keys as
 MongoDB's `[MinKey, undefined) ∪ (null, MaxKey]` bounds do: documents without
 the field or with an explicit null are not examined. Ranges are built only on
-complete, non-multikey indexes whose values all have a scalar order; other
-indexes are not narrowed. The full predicate is still checked for every
-selected candidate.
+complete indexes whose values all have a scalar order, and on a list field from
+one side of the range only (see **List fields** below); other indexes are not
+narrowed. The full predicate is still checked for every selected candidate.
 
 For a query whose filter references only `_id` and whose projection is exactly
 `{ _id: 1 }`, the native primary index can supply the original BSON identity
@@ -419,6 +425,17 @@ lifetimes remain held through durability. History publication also waits for the
 flush. DDL, transactions, index builds, imports and maintenance retain their
 existing independently durable publication paths.
 
+The Rust engine registers SIGTERM and SIGINT before reporting readiness. The
+first signal closes request admission, drains admitted commands and native
+workers, and closes storage only after their durable publications finish. An
+admitted transaction commit finishes atomically or aborts; idle transactions
+discard their uncommitted changes. Successful shutdown releases storage-root
+ownership and exits with code 0 even when the parent's management pipe remains
+open. A second shutdown signal, a drain failure, or the overall 40-second
+deadline forces exit with code 70; forced exit does not promise drain completion.
+`NoSqlDbServer.stop()` also succeeds after a confirmed engine exit without
+sending another management request to the dead child.
+
 **Storage compatibility:** version 10 adds a 40-byte certified document-ordinal
 record alongside the legacy 8-byte ordinal. It reads existing stores directly;
 startup and reads perform no format rewrite. After version 10 writes certified
@@ -594,7 +611,8 @@ unique index key wins, and the later writer is told at its own statement.
   publish every document.
 - A transaction open for longer than 60 seconds is aborted, as by MongoDB's
   transaction reaper: its staged writes are discarded and its cursors, write
-  claims, transaction capacity and database gates are released. The server
+  claims, transaction capacity and database and collection gates are
+  released. The server
   checks idle sessions once a second; a statement still executing when the
   limit passes is not interrupted, and the transaction aborts when that
   statement has finished (MongoDB instead interrupts that statement with
@@ -630,11 +648,12 @@ reports `TransactionCommitted` (256), commit after abort reports
 `$out`/`$merge` reject before transaction admission or mutation.
 
 Outside a transaction, `createIndexes` for already installed, identical indexes
-completes without draining active transactions or publishing a new storage
+completes without waiting for any transaction or publishing a new storage
 position. A repeated physical `create` preserves `NamespaceExists` (48) under
 the same shared admission. Both paths validate current authorization and catalog
-options. New collections and changed index catalogs still drain transactions
-and revalidate under exclusive admission before any publication.
+options. A new collection or a changed index catalog waits only for the
+transactions that named that collection, and revalidates under its exclusive
+collection gate before any publication (see "Database waits" below).
 
 Logical sessions survive socket disconnects. Expiry, `endSessions` and
 `killSessions` drain active work and release cursors and namespace gates.
@@ -1599,6 +1618,23 @@ only its members. Engines older than this release refuse a storage root that
 holds a partial index instead of enforcing its uniqueness over every
 document.
 
+**List fields.** An index on a field that holds lists has an entry per element,
+as a MongoDB multikey index, and a compound index an entry per combination of
+its list fields' elements. Equality, `$in` and one-sided ranges on the elements
+read their bounds, and a document with several entries within them is returned
+once. A list matches a range bounded on both sides when one element meets the
+lower bound and one the upper, so such a read takes the side with fewer
+entries. Equality to a whole list, such as `{ tags: ['a', 'b'] }`, is not
+answered from element entries, and a list nested in a list stops range reads
+through the index. An index with element entries returns no order, so a sorted
+read sorts its matches, and a `hint` naming it is refused. A unique index on a
+list field is unique per element: a document that shares an element with
+another is a `DuplicateKey` (11000), while one document may repeat an element.
+An index whose field already held a list under 10.7.3 or earlier keeps its
+whole-list entries and uniqueness and is maintained but never selected until it
+is dropped and created again. Engines up to 10.7.3 refuse an index with element
+entries instead of maintaining it with whole lists.
+
 `find()` and collection `aggregate()` accept `hint` as an index name (including
 `_id_`) or an exact ascending/descending key pattern. Key-pattern order matters;
 if several indexes share a key pattern, use a name. The selected active scalar
@@ -1619,8 +1655,8 @@ const count = await collection.countDocuments({ organizationId: 'org-1' }, {
 
 Unknown or quarantined indexes, malformed/ambiguous patterns, `$natural`, and
 hints on missing collections or views fail explicitly. Forced multikey traversal
-is unsupported and rejects; unhinted array queries keep their existing scan
-behavior. Sparse hints intentionally exclude documents absent from that index,
+is unsupported and rejects; unhinted queries on list fields use element entries
+as described under **List fields**. Sparse hints intentionally exclude documents absent from that index,
 including for empty filters and counts. A hint is never silently replaced by a
 collection scan or another index.
 
@@ -1775,9 +1811,9 @@ restores a `mongodump` of ordinary collections with its default flags: the
 documents, validators and supported indexes (unique, compound, sparse,
 partial and TTL). It sends this `idIndex` with every `create` and
 `createIndexes` with each index's `ns` and `ignoreUnknownIndexOptions: true`,
-and it builds several collections' indexes in parallel, which queue on the
-database (see "Database waits" below). Exclude the `admin` and `config`
-namespaces (`--nsExclude 'admin.*' --nsExclude 'config.*'`) and create users
+and it builds several collections' indexes in parallel, which wait for one
+another instead of failing (see "Database waits" below). Exclude the `admin`
+and `config` namespaces (`--nsExclude 'admin.*' --nsExclude 'config.*'`) and create users
 through NoSQLDB's authentication options. Collections or indexes that use
 unsupported features (collation, text, geospatial, hashed or wildcard
 indexes, capped or time-series collections) fail the restore with the
@@ -2095,27 +2131,43 @@ fields:
   transaction. `getMore` rejects `maxTimeMS` with `BadValue` because NoSQLDB
   has no awaitData cursors, and a literal `maxCommitTimeMS` field is rejected
   with `InvalidOptions`.
-- **Database waits.** DDL (`create`, `createIndexes`, `drop`, `dropIndexes`,
-  `collMod`, `dropDatabase`, `renameCollection`, `$out`/`$merge`) holds its
-  database exclusively; other commands on that database wait for it, as they
-  wait for a MongoDB collection or database lock. A command outside a
+- **Database waits.** DDL that drops or reshapes collections (`drop`,
+  `dropIndexes`, `collMod`, `dropDatabase`, `renameCollection`,
+  `$out`/`$merge`) holds its database exclusively; other commands on that
+  database wait for it, as they wait for a MongoDB database lock. Creating a
+  collection or its indexes (`create`, `createIndexes`) holds only that
+  collection exclusively, as MongoDB locks the collection: it waits for the
+  open transactions whose statements targeted that collection (a collection
+  a transaction reads only through `$lookup` or `$unionWith` is not held; the
+  transaction reads its snapshot of it), transaction statements on that
+  collection that arrive after it queue behind it, and everything else
+  proceeds. Commands outside a transaction take no collection
+  gate; writes still queue for the storage writer, which an index build holds
+  until its index is ready. A command outside a
   transaction waits first in, first out for as long as it may run: until
   `maxTimeMS` (`MaxTimeMSExpired`, code 50), until its client disconnects or
   shuts down its sending side (the wait ends with `ClientDisconnect`, code
   279, which a client that still reads receives before the connection closes)
-  or until the server shuts down (`InterruptedAtShutdown`, code 91). A DDL
-  command likewise waits for the open transactions on its database to commit
-  or abort, and every command on that database that arrives after the DDL
-  queues behind it, including reads of other collections. An open
-  transaction therefore delays DDL and the commands behind it at most until
-  the transaction lifetime limit (60 seconds, MongoDB's
+  or until the server shuts down (`InterruptedAtShutdown`, code 91). A
+  database DDL command likewise waits for the open transactions on its
+  database to commit or abort, and every command on that database that
+  arrives after it queues behind it, including reads of other collections. An
+  open transaction therefore delays DDL and the commands behind it at most
+  until the transaction lifetime limit (60 seconds, MongoDB's
   `transactionLifetimeLimitSeconds`) aborts it.
-  A transaction statement that needs a further database keeps a fixed
-  five-second limit and then fails with `IllegalOperation` (code 20,
+  A transaction statement that needs a further database or collection keeps a
+  fixed five-second limit and then fails with `IllegalOperation` (code 20,
   `EFENCE_BUSY: …`), so a transaction never waits without bound while it
-  holds its databases; like any failed statement, it aborts the transaction
-  and releases its databases. The service operations (export,
-  import, tenant and publication fencing) keep the same limit and report
+  holds its databases and collections; like any failed statement, it aborts
+  the transaction and releases them. One transaction targets at most 4,096
+  distinct collections across all its databases; statements that revisit a
+  collection retain no extra gate. This matches the process-wide collection-lock
+  registry's 4,096-live-identity bound; concurrent transactions and collection
+  DDL share that registry. It bounds process-local locks, not document staging,
+  WAL record size or atomic publication. A statement targeting one more fails
+  with `TooManyRequests`
+  (code 16500) and aborts it. The service operations (export,
+  import, tenant and publication fencing) keep the five-second gate-wait limit and report
   `EFENCE_BUSY`.
 
 Callers may therefore rely on a successful NoSQLDB response as a durable,

package/third_party_licenses/inventory.json

@@ -246,7 +246,7 @@
     "zmij@1.0.21"
   ],
   "files": {
-    ".smartconfig.json": "3024d7ff0fea774ba57f7f1e64d801947bc24becc10299cdceddcb0c39305dd9",
+    ".smartconfig.json": "a6b17fae7da13c9b5f6eef7049cfd13b497ab7e80739e17446becc0edd5ccefe",
     "license.md": "cfa51b1d8428e453099273832dc2ec1cebdf23d7ba4eda307883158cfc77da2d",
     "rust-toolchain.toml": "8c3afbe90fa0feff766dffd92e6728a569c0a804cf49f1ecd47bf5ac262499b3",
     "rust/Cargo.lock": "aa9659174e579bc943719a5c92ab343d6505a5432e6375254e39abdb1792110a",

package/ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@lossless.org/nosqldb',
-  version: '10.7.3',
+  version: '10.8.1',
   description: 'A MongoDB-compatible embedded database server with wire protocol support, backed by a high-performance Rust engine.'
 }

package/ts/ts_nosqldb/server/classes.nosqldbserver.ts

@@ -134,6 +134,7 @@ export class NoSqlDbServer {
   private bridgeOwned = false;
   private startInProgress = false;
   private startCompletionPromise: Promise<void> | null = null;
+  private stopCompletionPromise: Promise<void> | null = null;
   private isRunning = false;
   private resolvedConnectionUri = '';
   private resolvedPort: number | undefined;
@@ -157,9 +158,10 @@ export class NoSqlDbServer {
     };
     this.bridge = new RustDbBridge({ binaryPath: this.options.binaryPath });
     this.bridge.on('exit', (code: number | null, signal: string | null) => {
-      const exitedUnexpectedly = this.isRunning;
+      const exitedUnexpectedly = this.isRunning
+        && !this.stopCompletionPromise
+        && (code !== 0 || signal !== null);
       this.isRunning = false;
-      this.bridgeOwned = false;
       this.resolvedConnectionUri = '';
       this.resolvedPort = undefined;
       if (exitedUnexpectedly) {
@@ -174,7 +176,7 @@ export class NoSqlDbServer {
   async start(optionsArg?: INoSqlDbManagementOperationOptions): Promise<void> {
     validateNoSqlDbManagementOperationOptions(optionsArg);
     optionsArg?.signal?.throwIfAborted();
-    if (this.isRunning || this.startInProgress || this.bridgeOwned) {
+    if (this.isRunning || this.startInProgress || this.bridgeOwned || this.stopCompletionPromise) {
       throw new Error('Server is already running or still owns a Rust bridge');
     }
 
@@ -333,7 +335,23 @@ export class NoSqlDbServer {
   /**
    * Stop the server
    */
-  async stop(): Promise<void> {
+  stop(): Promise<void> {
+    if (this.stopCompletionPromise) {
+      return this.stopCompletionPromise;
+    }
+
+    const stopCompletionPromise = this.stopInternal();
+    this.stopCompletionPromise = stopCompletionPromise;
+    const releaseStopCompletion = () => {
+      if (this.stopCompletionPromise === stopCompletionPromise) {
+        this.stopCompletionPromise = null;
+      }
+    };
+    void stopCompletionPromise.then(releaseStopCompletion, releaseStopCompletion);
+    return stopCompletionPromise;
+  }
+
+  private async stopInternal(): Promise<void> {
     const startCompletionPromise = this.startCompletionPromise;
     if (startCompletionPromise) {
       await startCompletionPromise;
@@ -344,9 +362,6 @@ export class NoSqlDbServer {
     }
 
     const requestGracefulStop = this.isRunning && this.bridge.running;
-    this.isRunning = false;
-    this.resolvedConnectionUri = '';
-    this.resolvedPort = undefined;
     if (requestGracefulStop) {
       try {
         await this.bridge.stopDb();
@@ -357,6 +372,9 @@ export class NoSqlDbServer {
 
     await this.bridge.terminate();
     this.bridgeOwned = false;
+    this.isRunning = false;
+    this.resolvedConnectionUri = '';
+    this.resolvedPort = undefined;
   }
 
   /**