monsqlize 3.5.2 → 3.5.3

package/CHANGELOG.md

@@ -1,18 +1,19 @@
 # CHANGELOG
 
-> Summary index — the 3.5.2 versioned details are in [changelogs/v3.5.2.md](./changelogs/v3.5.2.md). A versioned note does not establish npm publication; check the registry and immutable tag before deployment.
+> Summary index — the 3.5.3 versioned details are in [changelogs/v3.5.3.md](./changelogs/v3.5.3.md). A versioned note does not establish npm publication; check the registry and immutable tag before deployment.
 > **Last updated**: 2026-11-02
 
 ---
 
 ## Unreleased
 
-Vector search guidance, Search docs regrouping and the vector example are assigned to **3.5.2** ([versioned notes](./changelogs/v3.5.2.md)); README, docs-site share card and release tooling fixes are in [3.5.1](./changelogs/v3.5.1.md); GridFS buckets are in [3.5.0](./changelogs/v3.5.0.md); the Search index lifecycle API is in [3.4.0](./changelogs/v3.4.0.md); the weekly-scan cache-key, health-check and lockfile security fixes are in [3.3.4](./changelogs/v3.3.4.md); the schema-dsl 3.0.5 upgrade is in [3.3.3](./changelogs/v3.3.3.md); the audited 3.3.2 repairs are in its [versioned notes](./changelogs/v3.3.2.md) and the [next-change staging log](https://github.com/devcodex-labs/monSQLize/blob/main/changelogs/unreleased.md). Untagged checkouts remain release candidates.
+GridFS and Search index fixes from the 2026-11-02 weekly scan, including the idempotent `finalizeResumableUpload` retry, are assigned to **3.5.3** ([versioned notes](./changelogs/v3.5.3.md)); vector search guidance, Search docs regrouping and the vector example are in [3.5.2](./changelogs/v3.5.2.md); README, docs-site share card and release tooling fixes are in [3.5.1](./changelogs/v3.5.1.md); GridFS buckets are in [3.5.0](./changelogs/v3.5.0.md); the Search index lifecycle API is in [3.4.0](./changelogs/v3.4.0.md); the weekly-scan cache-key, health-check and lockfile security fixes are in [3.3.4](./changelogs/v3.3.4.md); the schema-dsl 3.0.5 upgrade is in [3.3.3](./changelogs/v3.3.3.md); the audited 3.3.2 repairs are in its [versioned notes](./changelogs/v3.3.2.md) and the [next-change staging log](https://github.com/devcodex-labs/monSQLize/blob/main/changelogs/unreleased.md). Untagged checkouts remain release candidates.
 
 ## Version Overview
 
 | Version | Date | Summary | Details |
 |---------|------|---------|---------|
+| [v3.5.3](./changelogs/v3.5.3.md) | 2026-11-02 | Patch: GridFS upload/download/resumable-session fixes, idempotent finalize retry (contract change), long Search index wait timeouts and appendChunks cost | [View](./changelogs/v3.5.3.md) |
 | [v3.5.2](./changelogs/v3.5.2.md) | 2026-11-02 | Patch: corrected vector search guidance, regrouped Search docs and a new vector search example; no runtime changes | [View](./changelogs/v3.5.2.md) |
 | [v3.5.1](./changelogs/v3.5.1.md) | 2026-10-28 | Patch: README support wording and native-driver example, docs-site share card and locale meta; no runtime changes | [View](./changelogs/v3.5.1.md) |
 | [v3.5.0](./changelogs/v3.5.0.md) | 2026-10-27 | Minor: GridFS buckets with upload, download, ranges, ensureIndexes and phase-1 resumable upload | [View](./changelogs/v3.5.0.md) |

package/README.md

@@ -14,7 +14,7 @@ Upgrade and security: [Migration Guide](./MIGRATION.md) · [Private Vulnerabilit
 
 The npm `latest` dist-tag and GitHub Pages are the stable channel; `main` may contain the next release. Pages deployment accepts only a versioned Git tag whose package version is already present on npm, so unpublished APIs are not promoted as stable documentation. The Google site ownership verification file is copied separately from the workflow commit, with its source commit and content hash recorded in the deployment summary.
 
-This source tree targets **3.5.2**, a documentation patch with no runtime changes (see the [3.5.2 notes](./changelogs/v3.5.2.md)); 3.5.0 added GridFS buckets on `msq.gridfs()` and `model.gridfs()`, and 3.4.0 added the Search index lifecycle API for Atlas Search and Vector Search. The 3.3.2 cache, Model, pagination and sync fixes, including cold-cache upgrades and checkpoint rollback limits, are described in the [3.3.2 notes](./changelogs/v3.3.2.md) and [migration guide](./MIGRATION.md). Read the upgrade guide in [English](https://github.com/devcodex-labs/monSQLize/blob/main/docs/en/upgrade-3.3.2.md) or [简体中文](https://github.com/devcodex-labs/monSQLize/blob/main/docs/zh/upgrade-3.3.2.md). Untagged checkouts remain release candidates; verify the immutable tag and requested registry version before deployment. Candidate preparation does not publish npm or update stable Pages.
+This source tree targets **3.5.3**, a GridFS and Search index fix patch that also makes `finalizeResumableUpload` retries idempotent (see the [3.5.3 notes](./changelogs/v3.5.3.md)); 3.5.0 added GridFS buckets on `msq.gridfs()` and `model.gridfs()`, and 3.4.0 added the Search index lifecycle API for Atlas Search and Vector Search. The 3.3.2 cache, Model, pagination and sync fixes, including cold-cache upgrades and checkpoint rollback limits, are described in the [3.3.2 notes](./changelogs/v3.3.2.md) and [migration guide](./MIGRATION.md). Read the upgrade guide in [English](https://github.com/devcodex-labs/monSQLize/blob/main/docs/en/upgrade-3.3.2.md) or [简体中文](https://github.com/devcodex-labs/monSQLize/blob/main/docs/zh/upgrade-3.3.2.md). Untagged checkouts remain release candidates; verify the immutable tag and requested registry version before deployment. Candidate preparation does not publish npm or update stable Pages.
 
 This development source is pinned to verified registry `schema-dsl@3.0.5`. The v3 package remains subject to the full release gate before an immutable tag and npm publication.
 

package/changelogs/v3.5.3.md

@@ -0,0 +1,32 @@
+# monSQLize v3.5.3
+
+> Release date: 2026-11-02
+> Publication authority: the immutable tag and npm registry. An untagged checkout is a release candidate; the date above does not establish registry availability.
+
+## Highlights
+
+Patch release with GridFS and Search index fixes found by the 2026-11-02 weekly scan, plus one GridFS contract change: retrying `finalizeResumableUpload` is now idempotent.
+
+## Contract change
+
+- `finalizeResumableUpload` retried while its upload session still exists, for an id whose file is already visible with the same `metadata.sha256` and length as the session, now **returns that committed file and deletes the session** instead of throwing `GRIDFS_FILE_EXISTS`. A visible file with a different sha256 or length still throws `GRIDFS_FILE_EXISTS`. A retry after the session is gone still throws `GRIDFS_FILE_NOT_FOUND`; use `getFile(uploadId)` to check the result. Callers that treated `GRIDFS_FILE_EXISTS` on retry as "already finalized" now receive the file info instead.
+
+## Fixes
+
+- `uploadStream` with an `id` that already has a file now rejects with `GRIDFS_FILE_EXISTS` before writing; a duplicate-key write failure no longer runs cleanup that deleted the original file's chunks and files document.
+- `downloadStream` closes the driver stream and its server cursor when the consumer destroys it early; `download` does the same on a read error.
+- `appendChunks` that loses a race with `abortResumableUpload` now rejects with `GRIDFS_FILE_NOT_FOUND` and removes the chunks it wrote.
+- `drop()` resets the bucket's index latch, so `ensureIndexes()` and later writes on the same bucket object recreate the indexes.
+- `uploadStream` failures before the driver stream opens now destroy the returned `Writable` with the mapped error; `upload` with an existing `id` releases a `Readable` source.
+- `finalizeResumableUpload` no longer fails after the files document is committed when only the session cleanup fails (logged as a warning; the TTL removes the session); post-insert reads and filename lookups map driver errors. Retrying `finalizeResumableUpload` while the session exists is now idempotent: a visible file with the same id, sha256 and length is returned and the session deleted; a different file still throws `GRIDFS_FILE_EXISTS`, and a retry after the session is gone still throws `GRIDFS_FILE_NOT_FOUND`.
+- `abortResumableUpload` deletes leftover chunks even when the session expired; `beginResumableUpload` with a reused explicit `id` clears leftover chunks; `GRIDFS_RESUME_MISMATCH` details report the real `expectedNextChunk`.
+- `waitForSearchIndex` arms its deadline in segments, so `timeoutMs` above 2,147,483,647 ms no longer times out after 1 ms.
+- `appendChunks` reads only chunks at or after the session's `nextChunk`, restoring a near-constant cost per append (3,000 chunks of 1 KiB on a local memory server: the last 500 appends averaged 16.76 ms before and 3.69 ms after).
+
+## Packaging
+
+The runtime bundle ceilings in `config/package-budget.json` were raised for these fixes. No dependency versions changed.
+
+## Upgrade and rollback
+
+No migration is required. Rolling back to 3.5.2 restores the previous defects, including data loss when `uploadStream` reuses an existing id, and the old `GRIDFS_FILE_EXISTS` result on finalize retries.

package/dist/cjs/cli/data-task.cjs

@@ -31,7 +31,7 @@ var import_node_url = require("node:url");
 // package.json
 var package_default = {
   name: "monsqlize",
-  version: "3.5.2",
+  version: "3.5.3",
   description: "TypeScript production data runtime layer for MongoDB today, with cache, transactions, pools, models, sync, and observability",
   type: "commonjs",
   main: "./dist/cjs/index.cjs",
@@ -52,6 +52,7 @@ var package_default = {
     "./package.json": "./package.json"
   },
   files: [
+    "changelogs/v3.5.3.md",
     "changelogs/v3.5.2.md",
     "changelogs/v3.5.1.md",
     "changelogs/v3.5.0.md",

package/dist/cjs/index.cjs

@@ -6406,6 +6406,7 @@ async function uploadFile(ctx, filename, data, opts) {
   }
   await ctx.indexes.ensure();
   if (options.id && await getFile(ctx, options.id)) {
+    if (source) releaseReadable(source);
     throw fileExists("upload", ctx, { id: options.id, filename, sha256: options.sha256 });
   }
   if (ifExists !== "duplicate") {
@@ -6441,6 +6442,9 @@ function openUploadStream(ctx, filename, opts) {
     let target;
     try {
       await ctx.indexes.ensure();
+      if (await getFile(ctx, id)) {
+        throw fileExists("uploadStream", ctx, { id, filename, sha256: options.sha256 });
+      }
       target = ctx.driver.openUploadStream(id, filename, {
         chunkSizeBytes,
         metadata,
@@ -6452,8 +6456,10 @@ function openUploadStream(ctx, filename, opts) {
       if (!info) throw fileNotFound("uploadStream", ctx, { id, filename });
       return info;
     } catch (error) {
-      if (target) await cleanupFailedWrite(ctx, id, target);
-      throw mapDriverError(error, "uploadStream", ctx, { id, filename });
+      if (target) await cleanupFailedWrite(ctx, id, target, error);
+      const mapped = mapDriverError(error, "uploadStream", ctx, { id, filename });
+      if (!input.destroyed) input.destroy(mapped);
+      throw mapped;
     }
   })();
   void done.catch(() => void 0);
@@ -6466,6 +6472,7 @@ async function downloadFile(ctx, idOrName) {
   try {
     return await readReadable(source);
   } catch (error) {
+    abortDriverRead(source);
     throw mapDriverError(error, "download", ctx, { id: file.id, filename: file.filename });
   }
 }
@@ -6478,8 +6485,12 @@ function openDownloadStream(ctx, idOrName, range) {
         const file = await resolveFile(ctx, idOrName, "downloadStream");
         if (!file) throw fileNotFound("downloadStream", ctx, targetOf(idOrName));
         const bounds = rangeBounds(shape, file.length, "downloadStream", ctx.bucketName);
+        if (output.destroyed) return;
         const source = ctx.driver.openDownloadStream(file.id, bounds);
         source.on("error", (error) => output.destroy(mapDriverError(error, "downloadStream", ctx, { id: file.id, filename: file.filename })));
+        output.once("close", () => {
+          if (!source.readableEnded) abortDriverRead(source);
+        });
         source.pipe(output);
       } catch (error) {
         output.destroy(mapDriverError(error, "downloadStream", ctx, targetOf(idOrName)));
@@ -6544,6 +6555,7 @@ async function dropBucket(ctx) {
   await dropIfExists(ctx, chunksCollectionName(ctx.bucketName));
   await dropIfExists(ctx, filesCollectionName(ctx.bucketName));
   await dropIfExists(ctx, uploadsCollectionName(ctx.bucketName));
+  ctx.indexes.reset();
 }
 async function dropIfExists(ctx, name) {
   try {
@@ -6555,7 +6567,12 @@ async function dropIfExists(ctx, name) {
 async function findConflict(ctx, filename, sha256) {
   const files = ctx.db.collection(filesCollectionName(ctx.bucketName));
   const filter = sha256 !== void 0 ? { "metadata.sha256": sha256 } : { filename };
-  const docs = await files.find(filter).sort({ uploadDate: -1 }).limit(1).toArray();
+  let docs;
+  try {
+    docs = await files.find(filter).sort({ uploadDate: -1 }).limit(1).toArray();
+  } catch (error) {
+    throw mapDriverError(error, "upload", ctx, { filename });
+  }
   return docs[0] ? toFileInfo(docs[0]) : null;
 }
 async function resolveFile(ctx, idOrName, operation) {
@@ -6570,15 +6587,28 @@ async function resolveFile(ctx, idOrName, operation) {
   }
   const files = ctx.db.collection(filesCollectionName(ctx.bucketName));
   const cursor = revision >= 0 ? files.find({ filename: idOrName.filename }).sort({ uploadDate: 1 }).skip(revision).limit(1) : files.find({ filename: idOrName.filename }).sort({ uploadDate: -1 }).skip(-revision - 1).limit(1);
-  const docs = await cursor.toArray();
+  let docs;
+  try {
+    docs = await cursor.toArray();
+  } catch (error) {
+    throw mapDriverError(error, operation, ctx, { filename: idOrName.filename });
+  }
   return docs[0] ? toFileInfo(docs[0]) : null;
 }
+function abortDriverRead(source) {
+  const abortable = source;
+  if (typeof abortable.abort === "function") {
+    void abortable.abort().catch(() => void 0);
+  } else if (!source.destroyed) {
+    source.destroy();
+  }
+}
 async function writeBuffer(ctx, id, filename, payload, options) {
   const stream = ctx.driver.openUploadStream(id, filename, options);
   try {
     await endWritable(stream, payload);
   } catch (error) {
-    await cleanupFailedWrite(ctx, id, stream);
+    await cleanupFailedWrite(ctx, id, stream, error);
     throw mapDriverError(error, "upload", ctx, { id, filename });
   }
 }
@@ -6588,7 +6618,7 @@ async function writeReadable(ctx, id, filename, source, options) {
     const { pipeline } = await import("node:stream/promises");
     await pipeline(source, stream);
   } catch (error) {
-    await cleanupFailedWrite(ctx, id, stream);
+    await cleanupFailedWrite(ctx, id, stream, error);
     throw mapDriverError(error, "upload", ctx, { id, filename });
   }
 }
@@ -6603,7 +6633,8 @@ function endWritable(stream, payload) {
     stream.end(payload);
   });
 }
-async function cleanupFailedWrite(ctx, id, stream) {
+async function cleanupFailedWrite(ctx, id, stream, cause) {
+  if (isDuplicateKey(cause)) return;
   try {
     if (typeof stream.abort === "function") await stream.abort();
   } catch {
@@ -6657,6 +6688,11 @@ async function beginResumableUpload(ctx, filename, opts) {
   } catch (error) {
     throw mapDriverError(error, "beginResumableUpload", ctx, { uploadId, filename });
   }
+  try {
+    await ctx.db.collection(chunksCollectionName(ctx.bucketName)).deleteMany({ files_id: uploadId });
+  } catch (error) {
+    throw mapDriverError(error, "beginResumableUpload", ctx, { uploadId, filename });
+  }
   return { uploadId, nextChunk: 0, chunkSizeBytes };
 }
 async function appendChunks(ctx, uploadId, fromChunk, data) {
@@ -6665,7 +6701,7 @@ async function appendChunks(ctx, uploadId, fromChunk, data) {
   const payload = await payloadBytes(data);
   if (payload.length === 0) throw badChunkSize();
   const session = await requirePending(ctx, uploadId, "appendChunks");
-  const nextChunk = await contiguousNext(ctx, uploadId);
+  const nextChunk = await contiguousNext(ctx, uploadId, session.nextChunk);
   if (fromChunk > nextChunk) {
     throw resumeMismatch("appendChunks", ctx.bucketName, { uploadId, fromChunk, expectedNextChunk: nextChunk });
   }
@@ -6675,23 +6711,32 @@ async function appendChunks(ctx, uploadId, fromChunk, data) {
     throw resumeMismatch("appendChunks", ctx.bucketName, { uploadId, fromChunk, expectedNextChunk: nextChunk });
   }
   for (const slice of slices) {
-    if (slice.index < nextChunk) await assertStoredChunk(ctx, uploadId, slice.index, slice.bytes);
-    else await writeChunk(ctx, uploadId, slice.index, slice.bytes);
+    if (slice.index < nextChunk) await assertStoredChunk(ctx, uploadId, slice.index, slice.bytes, nextChunk);
+    else await writeChunk(ctx, uploadId, slice.index, slice.bytes, nextChunk);
+  }
+  const advanced = await contiguousNext(ctx, uploadId, nextChunk);
+  let matched;
+  try {
+    const updated = await ctx.db.collection(uploadsCollectionName(ctx.bucketName)).updateOne(
+      { _id: uploadId, status: "pending" },
+      { $max: { nextChunk: advanced }, $set: { updatedAt: /* @__PURE__ */ new Date() } }
+    );
+    matched = updated.matchedCount;
+  } catch (error) {
+    throw mapDriverError(error, "appendChunks", ctx, { uploadId });
+  }
+  if (matched === 0) {
+    await removeInvisibleChunks(ctx, uploadId, "appendChunks");
+    throw fileNotFound("appendChunks", ctx, { id: uploadId });
   }
-  const advanced = await contiguousNext(ctx, uploadId);
-  await ctx.db.collection(uploadsCollectionName(ctx.bucketName)).updateOne(
-    { _id: uploadId },
-    { $set: { nextChunk: advanced, updatedAt: /* @__PURE__ */ new Date() } }
-  );
   return { nextChunk: advanced, bytesAccepted: payload.length };
 }
 async function finalizeResumableUpload(ctx, uploadId) {
   assertObjectId(uploadId, "finalizeResumableUpload", "uploadId");
   const session = await requirePending(ctx, uploadId, "finalizeResumableUpload");
   const files = ctx.db.collection(filesCollectionName(ctx.bucketName));
-  if (await files.findOne({ _id: uploadId }, { projection: { _id: 1 } })) {
-    throw fileExists("finalizeResumableUpload", ctx, { id: uploadId, filename: session.filename, sha256: session.sha256 });
-  }
+  const committed = await committedFile(ctx, uploadId, session);
+  if (committed) return committed;
   const expected = chunkCount(session.length, session.chunkSizeBytes);
   const hash = (0, import_node_crypto4.createHash)("sha256");
   let total = 0;
@@ -6748,12 +6793,22 @@ async function finalizeResumableUpload(ctx, uploadId) {
     });
   } catch (error) {
     if (isDuplicateKey(error)) {
-      throw fileExists("finalizeResumableUpload", ctx, { id: uploadId, filename: session.filename, sha256: session.sha256 });
+      const raced = await committedFile(ctx, uploadId, session);
+      if (raced) return raced;
     }
     throw mapDriverError(error, "finalizeResumableUpload", ctx, { uploadId, filename: session.filename });
   }
-  await ctx.db.collection(uploadsCollectionName(ctx.bucketName)).deleteOne({ _id: uploadId });
-  const info = await files.findOne({ _id: uploadId });
+  try {
+    await ctx.db.collection(uploadsCollectionName(ctx.bucketName)).deleteOne({ _id: uploadId });
+  } catch (error) {
+    ctx.logger?.warn?.("[gridfs] finalizeResumableUpload could not delete the upload session", { bucket: ctx.bucketName, uploadId: String(uploadId), error });
+  }
+  let info;
+  try {
+    info = await files.findOne({ _id: uploadId });
+  } catch (error) {
+    throw mapDriverError(error, "finalizeResumableUpload", ctx, { uploadId, filename: session.filename });
+  }
   if (!info) throw fileNotFound("finalizeResumableUpload", ctx, { id: uploadId, filename: session.filename });
   return toFileInfo(info);
 }
@@ -6762,17 +6817,37 @@ async function abortResumableUpload(ctx, uploadId) {
   try {
     const uploads = ctx.db.collection(uploadsCollectionName(ctx.bucketName));
     const session = await uploads.findOne({ _id: uploadId });
-    if (!session || session.status === "aborted" || session.status === "finalized") return;
+    if (session?.status === "finalized") return;
+    await removeInvisibleChunks(ctx, uploadId, "abortResumableUpload");
+    if (session) await uploads.deleteOne({ _id: uploadId });
+  } catch (error) {
+    throw mapDriverError(error, "abortResumableUpload", ctx, { uploadId });
+  }
+}
+async function committedFile(ctx, uploadId, session) {
+  const target = { uploadId, filename: session.filename };
+  try {
+    const doc = await ctx.db.collection(filesCollectionName(ctx.bucketName)).findOne({ _id: uploadId });
+    if (!doc) return null;
+    const info = toFileInfo(doc);
+    if (info.metadata?.sha256 !== session.sha256 || info.length !== session.length) {
+      throw fileExists("finalizeResumableUpload", ctx, { id: uploadId, filename: session.filename, sha256: session.sha256 });
+    }
+    await ctx.db.collection(uploadsCollectionName(ctx.bucketName)).deleteOne({ _id: uploadId, status: "pending" });
+    return info;
+  } catch (error) {
+    throw mapDriverError(error, "finalizeResumableUpload", ctx, target);
+  }
+}
+async function removeInvisibleChunks(ctx, uploadId, operation) {
+  try {
     const visible = await ctx.db.collection(filesCollectionName(ctx.bucketName)).findOne(
       { _id: uploadId },
       { projection: { _id: 1 } }
     );
-    if (!visible) {
-      await ctx.db.collection(chunksCollectionName(ctx.bucketName)).deleteMany({ files_id: uploadId });
-    }
-    await uploads.deleteOne({ _id: uploadId });
+    if (!visible) await ctx.db.collection(chunksCollectionName(ctx.bucketName)).deleteMany({ files_id: uploadId });
   } catch (error) {
-    throw mapDriverError(error, "abortResumableUpload", ctx, { uploadId });
+    throw mapDriverError(error, operation, ctx, { uploadId });
   }
 }
 async function payloadBytes(data) {
@@ -6803,27 +6878,33 @@ async function requirePending(ctx, uploadId, operation) {
   if (!doc || doc.status !== "pending") throw fileNotFound(operation, ctx, { id: uploadId });
   return doc;
 }
-async function contiguousNext(ctx, uploadId) {
-  const docs = await ctx.db.collection(chunksCollectionName(ctx.bucketName)).find({ files_id: uploadId }).sort({ n: 1 }).project({ n: 1 }).toArray();
-  let expected = 0;
+async function contiguousNext(ctx, uploadId, from) {
+  const start = Number.isInteger(from) && from > 0 ? from : 0;
+  let docs;
+  try {
+    docs = await ctx.db.collection(chunksCollectionName(ctx.bucketName)).find({ files_id: uploadId, n: { $gte: start } }).sort({ n: 1 }).project({ n: 1 }).toArray();
+  } catch (error) {
+    throw mapDriverError(error, "appendChunks", ctx, { uploadId });
+  }
+  let expected = start;
   for (const doc of docs) {
     if (doc.n !== expected) break;
     expected += 1;
   }
   return expected;
 }
-async function assertStoredChunk(ctx, uploadId, n, slice) {
+async function assertStoredChunk(ctx, uploadId, n, slice, expectedNextChunk) {
   const existing = await ctx.db.collection(chunksCollectionName(ctx.bucketName)).findOne({ files_id: uploadId, n });
   if (!existing || !sameBytes(binaryToBuffer(existing.data, "appendChunks"), slice)) {
-    throw resumeMismatch("appendChunks", ctx.bucketName, { uploadId, fromChunk: n, expectedNextChunk: n });
+    throw resumeMismatch("appendChunks", ctx.bucketName, { uploadId, fromChunk: n, expectedNextChunk });
   }
 }
-async function writeChunk(ctx, uploadId, n, slice) {
+async function writeChunk(ctx, uploadId, n, slice, expectedNextChunk) {
   try {
     await ctx.db.collection(chunksCollectionName(ctx.bucketName)).insertOne({ files_id: uploadId, n, data: slice });
   } catch (error) {
     if (!isDuplicateKey(error)) throw mapDriverError(error, "appendChunks", ctx, { uploadId });
-    await assertStoredChunk(ctx, uploadId, n, slice);
+    await assertStoredChunk(ctx, uploadId, n, slice, expectedNextChunk);
   }
 }
 
@@ -6958,18 +7039,25 @@ var GridFSIndexLatch = class {
     this.bucketName = bucketName;
     this.inflight = null;
     this.ready = false;
+    this.generation = 0;
   }
   ensure() {
     if (this.ready) return Promise.resolve();
     if (!this.inflight) {
+      const generation = this.generation;
       this.inflight = this.create().then(() => {
-        this.ready = true;
+        if (generation === this.generation) this.ready = true;
       }).finally(() => {
         this.inflight = null;
       });
     }
     return this.inflight;
   }
+  /** Forget a completed ensure, e.g. after the bucket collections were dropped, so the next call recreates indexes. */
+  reset() {
+    this.ready = false;
+    this.generation += 1;
+  }
   async create() {
     const files = this.db.collection(filesCollectionName(this.bucketName));
     const chunks2 = this.db.collection(chunksCollectionName(this.bucketName));
@@ -10536,15 +10624,24 @@ function sleep2(ms, signal) {
     signal?.addEventListener("abort", onAbort, { once: true });
   });
 }
+var MAX_TIMER_MS = 2147483647;
 function linkSignal(userSignal, remainingMs) {
   const controller = new AbortController();
   let expired = false;
   const onUserAbort = () => controller.abort(userSignal?.reason);
   userSignal?.addEventListener("abort", onUserAbort, { once: true });
-  const timer = setTimeout(() => {
-    expired = true;
-    controller.abort(new Error("search index wait deadline reached"));
-  }, Math.max(0, remainingMs));
+  let timer;
+  const arm = (ms) => {
+    timer = setTimeout(() => {
+      if (ms > MAX_TIMER_MS) {
+        arm(ms - MAX_TIMER_MS);
+        return;
+      }
+      expired = true;
+      controller.abort(new Error("search index wait deadline reached"));
+    }, Math.min(ms, MAX_TIMER_MS));
+  };
+  arm(Math.max(0, remainingMs));
   return {
     signal: controller.signal,
     timedOut: () => expired,

package/dist/esm/index.mjs

@@ -6389,6 +6389,7 @@ async function uploadFile(ctx, filename, data, opts) {
   }
   await ctx.indexes.ensure();
   if (options.id && await getFile(ctx, options.id)) {
+    if (source) releaseReadable(source);
     throw fileExists("upload", ctx, { id: options.id, filename, sha256: options.sha256 });
   }
   if (ifExists !== "duplicate") {
@@ -6424,6 +6425,9 @@ function openUploadStream(ctx, filename, opts) {
     let target;
     try {
       await ctx.indexes.ensure();
+      if (await getFile(ctx, id)) {
+        throw fileExists("uploadStream", ctx, { id, filename, sha256: options.sha256 });
+      }
       target = ctx.driver.openUploadStream(id, filename, {
         chunkSizeBytes,
         metadata,
@@ -6435,8 +6439,10 @@ function openUploadStream(ctx, filename, opts) {
       if (!info) throw fileNotFound("uploadStream", ctx, { id, filename });
       return info;
     } catch (error) {
-      if (target) await cleanupFailedWrite(ctx, id, target);
-      throw mapDriverError(error, "uploadStream", ctx, { id, filename });
+      if (target) await cleanupFailedWrite(ctx, id, target, error);
+      const mapped = mapDriverError(error, "uploadStream", ctx, { id, filename });
+      if (!input.destroyed) input.destroy(mapped);
+      throw mapped;
     }
   })();
   void done.catch(() => void 0);
@@ -6449,6 +6455,7 @@ async function downloadFile(ctx, idOrName) {
   try {
     return await readReadable(source);
   } catch (error) {
+    abortDriverRead(source);
     throw mapDriverError(error, "download", ctx, { id: file.id, filename: file.filename });
   }
 }
@@ -6461,8 +6468,12 @@ function openDownloadStream(ctx, idOrName, range) {
         const file = await resolveFile(ctx, idOrName, "downloadStream");
         if (!file) throw fileNotFound("downloadStream", ctx, targetOf(idOrName));
         const bounds = rangeBounds(shape, file.length, "downloadStream", ctx.bucketName);
+        if (output.destroyed) return;
         const source = ctx.driver.openDownloadStream(file.id, bounds);
         source.on("error", (error) => output.destroy(mapDriverError(error, "downloadStream", ctx, { id: file.id, filename: file.filename })));
+        output.once("close", () => {
+          if (!source.readableEnded) abortDriverRead(source);
+        });
         source.pipe(output);
       } catch (error) {
         output.destroy(mapDriverError(error, "downloadStream", ctx, targetOf(idOrName)));
@@ -6527,6 +6538,7 @@ async function dropBucket(ctx) {
   await dropIfExists(ctx, chunksCollectionName(ctx.bucketName));
   await dropIfExists(ctx, filesCollectionName(ctx.bucketName));
   await dropIfExists(ctx, uploadsCollectionName(ctx.bucketName));
+  ctx.indexes.reset();
 }
 async function dropIfExists(ctx, name) {
   try {
@@ -6538,7 +6550,12 @@ async function dropIfExists(ctx, name) {
 async function findConflict(ctx, filename, sha256) {
   const files = ctx.db.collection(filesCollectionName(ctx.bucketName));
   const filter = sha256 !== void 0 ? { "metadata.sha256": sha256 } : { filename };
-  const docs = await files.find(filter).sort({ uploadDate: -1 }).limit(1).toArray();
+  let docs;
+  try {
+    docs = await files.find(filter).sort({ uploadDate: -1 }).limit(1).toArray();
+  } catch (error) {
+    throw mapDriverError(error, "upload", ctx, { filename });
+  }
   return docs[0] ? toFileInfo(docs[0]) : null;
 }
 async function resolveFile(ctx, idOrName, operation) {
@@ -6553,15 +6570,28 @@ async function resolveFile(ctx, idOrName, operation) {
   }
   const files = ctx.db.collection(filesCollectionName(ctx.bucketName));
   const cursor = revision >= 0 ? files.find({ filename: idOrName.filename }).sort({ uploadDate: 1 }).skip(revision).limit(1) : files.find({ filename: idOrName.filename }).sort({ uploadDate: -1 }).skip(-revision - 1).limit(1);
-  const docs = await cursor.toArray();
+  let docs;
+  try {
+    docs = await cursor.toArray();
+  } catch (error) {
+    throw mapDriverError(error, operation, ctx, { filename: idOrName.filename });
+  }
   return docs[0] ? toFileInfo(docs[0]) : null;
 }
+function abortDriverRead(source) {
+  const abortable = source;
+  if (typeof abortable.abort === "function") {
+    void abortable.abort().catch(() => void 0);
+  } else if (!source.destroyed) {
+    source.destroy();
+  }
+}
 async function writeBuffer(ctx, id, filename, payload, options) {
   const stream = ctx.driver.openUploadStream(id, filename, options);
   try {
     await endWritable(stream, payload);
   } catch (error) {
-    await cleanupFailedWrite(ctx, id, stream);
+    await cleanupFailedWrite(ctx, id, stream, error);
     throw mapDriverError(error, "upload", ctx, { id, filename });
   }
 }
@@ -6571,7 +6601,7 @@ async function writeReadable(ctx, id, filename, source, options) {
     const { pipeline } = await import("node:stream/promises");
     await pipeline(source, stream);
   } catch (error) {
-    await cleanupFailedWrite(ctx, id, stream);
+    await cleanupFailedWrite(ctx, id, stream, error);
     throw mapDriverError(error, "upload", ctx, { id, filename });
   }
 }
@@ -6586,7 +6616,8 @@ function endWritable(stream, payload) {
     stream.end(payload);
   });
 }
-async function cleanupFailedWrite(ctx, id, stream) {
+async function cleanupFailedWrite(ctx, id, stream, cause) {
+  if (isDuplicateKey(cause)) return;
   try {
     if (typeof stream.abort === "function") await stream.abort();
   } catch {
@@ -6640,6 +6671,11 @@ async function beginResumableUpload(ctx, filename, opts) {
   } catch (error) {
     throw mapDriverError(error, "beginResumableUpload", ctx, { uploadId, filename });
   }
+  try {
+    await ctx.db.collection(chunksCollectionName(ctx.bucketName)).deleteMany({ files_id: uploadId });
+  } catch (error) {
+    throw mapDriverError(error, "beginResumableUpload", ctx, { uploadId, filename });
+  }
   return { uploadId, nextChunk: 0, chunkSizeBytes };
 }
 async function appendChunks(ctx, uploadId, fromChunk, data) {
@@ -6648,7 +6684,7 @@ async function appendChunks(ctx, uploadId, fromChunk, data) {
   const payload = await payloadBytes(data);
   if (payload.length === 0) throw badChunkSize();
   const session = await requirePending(ctx, uploadId, "appendChunks");
-  const nextChunk = await contiguousNext(ctx, uploadId);
+  const nextChunk = await contiguousNext(ctx, uploadId, session.nextChunk);
   if (fromChunk > nextChunk) {
     throw resumeMismatch("appendChunks", ctx.bucketName, { uploadId, fromChunk, expectedNextChunk: nextChunk });
   }
@@ -6658,23 +6694,32 @@ async function appendChunks(ctx, uploadId, fromChunk, data) {
     throw resumeMismatch("appendChunks", ctx.bucketName, { uploadId, fromChunk, expectedNextChunk: nextChunk });
   }
   for (const slice of slices) {
-    if (slice.index < nextChunk) await assertStoredChunk(ctx, uploadId, slice.index, slice.bytes);
-    else await writeChunk(ctx, uploadId, slice.index, slice.bytes);
+    if (slice.index < nextChunk) await assertStoredChunk(ctx, uploadId, slice.index, slice.bytes, nextChunk);
+    else await writeChunk(ctx, uploadId, slice.index, slice.bytes, nextChunk);
+  }
+  const advanced = await contiguousNext(ctx, uploadId, nextChunk);
+  let matched;
+  try {
+    const updated = await ctx.db.collection(uploadsCollectionName(ctx.bucketName)).updateOne(
+      { _id: uploadId, status: "pending" },
+      { $max: { nextChunk: advanced }, $set: { updatedAt: /* @__PURE__ */ new Date() } }
+    );
+    matched = updated.matchedCount;
+  } catch (error) {
+    throw mapDriverError(error, "appendChunks", ctx, { uploadId });
+  }
+  if (matched === 0) {
+    await removeInvisibleChunks(ctx, uploadId, "appendChunks");
+    throw fileNotFound("appendChunks", ctx, { id: uploadId });
   }
-  const advanced = await contiguousNext(ctx, uploadId);
-  await ctx.db.collection(uploadsCollectionName(ctx.bucketName)).updateOne(
-    { _id: uploadId },
-    { $set: { nextChunk: advanced, updatedAt: /* @__PURE__ */ new Date() } }
-  );
   return { nextChunk: advanced, bytesAccepted: payload.length };
 }
 async function finalizeResumableUpload(ctx, uploadId) {
   assertObjectId(uploadId, "finalizeResumableUpload", "uploadId");
   const session = await requirePending(ctx, uploadId, "finalizeResumableUpload");
   const files = ctx.db.collection(filesCollectionName(ctx.bucketName));
-  if (await files.findOne({ _id: uploadId }, { projection: { _id: 1 } })) {
-    throw fileExists("finalizeResumableUpload", ctx, { id: uploadId, filename: session.filename, sha256: session.sha256 });
-  }
+  const committed = await committedFile(ctx, uploadId, session);
+  if (committed) return committed;
   const expected = chunkCount(session.length, session.chunkSizeBytes);
   const hash = createHash3("sha256");
   let total = 0;
@@ -6731,12 +6776,22 @@ async function finalizeResumableUpload(ctx, uploadId) {
     });
   } catch (error) {
     if (isDuplicateKey(error)) {
-      throw fileExists("finalizeResumableUpload", ctx, { id: uploadId, filename: session.filename, sha256: session.sha256 });
+      const raced = await committedFile(ctx, uploadId, session);
+      if (raced) return raced;
     }
     throw mapDriverError(error, "finalizeResumableUpload", ctx, { uploadId, filename: session.filename });
   }
-  await ctx.db.collection(uploadsCollectionName(ctx.bucketName)).deleteOne({ _id: uploadId });
-  const info = await files.findOne({ _id: uploadId });
+  try {
+    await ctx.db.collection(uploadsCollectionName(ctx.bucketName)).deleteOne({ _id: uploadId });
+  } catch (error) {
+    ctx.logger?.warn?.("[gridfs] finalizeResumableUpload could not delete the upload session", { bucket: ctx.bucketName, uploadId: String(uploadId), error });
+  }
+  let info;
+  try {
+    info = await files.findOne({ _id: uploadId });
+  } catch (error) {
+    throw mapDriverError(error, "finalizeResumableUpload", ctx, { uploadId, filename: session.filename });
+  }
   if (!info) throw fileNotFound("finalizeResumableUpload", ctx, { id: uploadId, filename: session.filename });
   return toFileInfo(info);
 }
@@ -6745,17 +6800,37 @@ async function abortResumableUpload(ctx, uploadId) {
   try {
     const uploads = ctx.db.collection(uploadsCollectionName(ctx.bucketName));
     const session = await uploads.findOne({ _id: uploadId });
-    if (!session || session.status === "aborted" || session.status === "finalized") return;
+    if (session?.status === "finalized") return;
+    await removeInvisibleChunks(ctx, uploadId, "abortResumableUpload");
+    if (session) await uploads.deleteOne({ _id: uploadId });
+  } catch (error) {
+    throw mapDriverError(error, "abortResumableUpload", ctx, { uploadId });
+  }
+}
+async function committedFile(ctx, uploadId, session) {
+  const target = { uploadId, filename: session.filename };
+  try {
+    const doc = await ctx.db.collection(filesCollectionName(ctx.bucketName)).findOne({ _id: uploadId });
+    if (!doc) return null;
+    const info = toFileInfo(doc);
+    if (info.metadata?.sha256 !== session.sha256 || info.length !== session.length) {
+      throw fileExists("finalizeResumableUpload", ctx, { id: uploadId, filename: session.filename, sha256: session.sha256 });
+    }
+    await ctx.db.collection(uploadsCollectionName(ctx.bucketName)).deleteOne({ _id: uploadId, status: "pending" });
+    return info;
+  } catch (error) {
+    throw mapDriverError(error, "finalizeResumableUpload", ctx, target);
+  }
+}
+async function removeInvisibleChunks(ctx, uploadId, operation) {
+  try {
     const visible = await ctx.db.collection(filesCollectionName(ctx.bucketName)).findOne(
       { _id: uploadId },
       { projection: { _id: 1 } }
     );
-    if (!visible) {
-      await ctx.db.collection(chunksCollectionName(ctx.bucketName)).deleteMany({ files_id: uploadId });
-    }
-    await uploads.deleteOne({ _id: uploadId });
+    if (!visible) await ctx.db.collection(chunksCollectionName(ctx.bucketName)).deleteMany({ files_id: uploadId });
   } catch (error) {
-    throw mapDriverError(error, "abortResumableUpload", ctx, { uploadId });
+    throw mapDriverError(error, operation, ctx, { uploadId });
   }
 }
 async function payloadBytes(data) {
@@ -6786,27 +6861,33 @@ async function requirePending(ctx, uploadId, operation) {
   if (!doc || doc.status !== "pending") throw fileNotFound(operation, ctx, { id: uploadId });
   return doc;
 }
-async function contiguousNext(ctx, uploadId) {
-  const docs = await ctx.db.collection(chunksCollectionName(ctx.bucketName)).find({ files_id: uploadId }).sort({ n: 1 }).project({ n: 1 }).toArray();
-  let expected = 0;
+async function contiguousNext(ctx, uploadId, from) {
+  const start = Number.isInteger(from) && from > 0 ? from : 0;
+  let docs;
+  try {
+    docs = await ctx.db.collection(chunksCollectionName(ctx.bucketName)).find({ files_id: uploadId, n: { $gte: start } }).sort({ n: 1 }).project({ n: 1 }).toArray();
+  } catch (error) {
+    throw mapDriverError(error, "appendChunks", ctx, { uploadId });
+  }
+  let expected = start;
   for (const doc of docs) {
     if (doc.n !== expected) break;
     expected += 1;
   }
   return expected;
 }
-async function assertStoredChunk(ctx, uploadId, n, slice) {
+async function assertStoredChunk(ctx, uploadId, n, slice, expectedNextChunk) {
   const existing = await ctx.db.collection(chunksCollectionName(ctx.bucketName)).findOne({ files_id: uploadId, n });
   if (!existing || !sameBytes(binaryToBuffer(existing.data, "appendChunks"), slice)) {
-    throw resumeMismatch("appendChunks", ctx.bucketName, { uploadId, fromChunk: n, expectedNextChunk: n });
+    throw resumeMismatch("appendChunks", ctx.bucketName, { uploadId, fromChunk: n, expectedNextChunk });
   }
 }
-async function writeChunk(ctx, uploadId, n, slice) {
+async function writeChunk(ctx, uploadId, n, slice, expectedNextChunk) {
   try {
     await ctx.db.collection(chunksCollectionName(ctx.bucketName)).insertOne({ files_id: uploadId, n, data: slice });
   } catch (error) {
     if (!isDuplicateKey(error)) throw mapDriverError(error, "appendChunks", ctx, { uploadId });
-    await assertStoredChunk(ctx, uploadId, n, slice);
+    await assertStoredChunk(ctx, uploadId, n, slice, expectedNextChunk);
   }
 }
 
@@ -6941,18 +7022,25 @@ var GridFSIndexLatch = class {
     this.bucketName = bucketName;
     this.inflight = null;
     this.ready = false;
+    this.generation = 0;
   }
   ensure() {
     if (this.ready) return Promise.resolve();
     if (!this.inflight) {
+      const generation = this.generation;
       this.inflight = this.create().then(() => {
-        this.ready = true;
+        if (generation === this.generation) this.ready = true;
       }).finally(() => {
         this.inflight = null;
       });
     }
     return this.inflight;
   }
+  /** Forget a completed ensure, e.g. after the bucket collections were dropped, so the next call recreates indexes. */
+  reset() {
+    this.ready = false;
+    this.generation += 1;
+  }
   async create() {
     const files = this.db.collection(filesCollectionName(this.bucketName));
     const chunks2 = this.db.collection(chunksCollectionName(this.bucketName));
@@ -10519,15 +10607,24 @@ function sleep2(ms, signal) {
     signal?.addEventListener("abort", onAbort, { once: true });
   });
 }
+var MAX_TIMER_MS = 2147483647;
 function linkSignal(userSignal, remainingMs) {
   const controller = new AbortController();
   let expired = false;
   const onUserAbort = () => controller.abort(userSignal?.reason);
   userSignal?.addEventListener("abort", onUserAbort, { once: true });
-  const timer = setTimeout(() => {
-    expired = true;
-    controller.abort(new Error("search index wait deadline reached"));
-  }, Math.max(0, remainingMs));
+  let timer;
+  const arm = (ms) => {
+    timer = setTimeout(() => {
+      if (ms > MAX_TIMER_MS) {
+        arm(ms - MAX_TIMER_MS);
+        return;
+      }
+      expired = true;
+      controller.abort(new Error("search index wait deadline reached"));
+    }, Math.min(ms, MAX_TIMER_MS));
+  };
+  arm(Math.max(0, remainingMs));
   return {
     signal: controller.signal,
     timedOut: () => expired,

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "monsqlize",
-    "version": "3.5.2",
+    "version": "3.5.3",
     "description": "TypeScript production data runtime layer for MongoDB today, with cache, transactions, pools, models, sync, and observability",
     "type": "commonjs",
     "main": "./dist/cjs/index.cjs",
@@ -21,6 +21,7 @@
         "./package.json": "./package.json"
     },
     "files": [
+        "changelogs/v3.5.3.md",
         "changelogs/v3.5.2.md",
         "changelogs/v3.5.1.md",
         "changelogs/v3.5.0.md",