@dynamic-entity/server 1.14.0 → 2.0.1

package/README.md

@@ -129,6 +129,16 @@ console.log(`imported ${result.imported}, skipped ${result.skipped}`);
 **awaits** `onBatch`, and drops the batch. Peak memory is a function of `batchSize`, not of the
 file's size — which is the entire reason this package exists.
 
+**One qualification, for xlsx.** That statement is exact for CSV, which is read as a byte
+stream from first row to last. An `.xlsx` file is a zip archive, and exceljs cannot be handed
+a stream of one: `guardZip` validates the archive and rebuilds it in memory before exceljs is
+constructed, so the *compressed* file is resident for the length of the import. It is bounded
+by `maxBytes` rather than by `batchSize`, and it is the compressed size — a fifty-megabyte
+workbook is fifty megabytes here, not the several hundred its rows would occupy expanded. The
+row-by-row half above still holds: what scales with `batchSize` is everything downstream of
+the reader. `xlsx-source.ts` has always said this in its own header; it belongs where a
+reader meets the claim first.
+
 ---
 
 ## Four things to know before you deploy this
@@ -203,7 +213,35 @@ catastrophic backtrack. See [SECURITY.md](../../SECURITY.md).
 
 ## What it holds in memory
 
-Stated rather than left to be discovered:
+```mermaid
+flowchart LR
+  U["the browser<br>uploads a file"]
+  R["readSheet()<br>streamed, limits enforced"]
+  M["applyMapping()<br>one batch of 500"]
+  Y["your onImport<br>awaited"]
+  C["imported: 50000<br>records: empty"]
+
+  U -->|"multipart POST"| R
+  R --> M
+  M -->|"records"| Y
+  Y -.->|"resolves: drop the batch, pull the next 500"| R
+  R -.->|"when the file ends"| C
+
+  classDef yours stroke-width:3px,stroke-dasharray:5 3
+  class Y yours
+```
+
+The dashed edge back to `readSheet` is the whole package. A batch is mapped, handed to your
+writer, **awaited**, and dropped before the next is pulled — so peak memory downstream of the
+reader is a function of `batchSize` and not of the file. Not awaiting would turn bounded memory
+into an unbounded queue of pending writes, which is the same failure arrived at from the other
+side. For xlsx, add the rebuilt archive `guardZip` holds — see the qualification above.
+
+Measured on a fifty-thousand-row, thirty-five-column sheet: **2.7 MB** of collected heap, against
+163 MB with the per-batch flush removed. `stress.spec.ts` asserts it at 16 MB, chosen to
+discriminate rather than to be comfortable.
+
+The rest, stated rather than left to be discovered:
 
 - **CSV streams properly.** Rows are read incrementally and never accumulated.
 - **Exactly one worksheet is read** — the lowest-numbered `sheetN.xml`, not whichever the

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@dynamic-entity/server",
-  "version": "1.14.0",
+  "version": "2.0.1",
   "description": "Streaming spreadsheet import for @dynamic-entity/core, with an Express adapter.",
   "keywords": [
     "dynamic-entity",
@@ -47,7 +47,7 @@
     "exceljs": "4.4.0"
   },
   "peerDependencies": {
-    "@dynamic-entity/core": "^1.14.0",
+    "@dynamic-entity/core": "^2.0.1",
     "express": "^4.18.0 || ^5.0.0"
   },
   "peerDependenciesMeta": {