@cloudbitmaps/roaring 0.18.0 → 0.18.1

package/PRIVACY.md

@@ -45,7 +45,7 @@ from constructing a cross-region topology. The points where personal data moves
 | **Storage** (object store) | immutable `.crbm` generations — every generation a segment has had, until a superseded one is collected. Azure Blob objects, and GCS objects above the simple-upload threshold, also carry a random write id in their metadata (`cbwid`): 128 random bits, no data from the bitmap or the source | the region of the bucket you wire |
 | **Registry** (S3 / GCS / Azure Blob / local) | one row per segment: the current-generation pointer, wrapped keys, retention metadata, and the current generation's id count and your metadata (sealed when the segment is encrypted) — no IDs, unless you put one in the metadata, which you must not. An Azure Blob row also carries the random write id in its metadata | the region of the bucket you wire |
 | **cache** (process RAM) | decoded chunks, and the stored chunk bytes (ciphertext on an encrypted segment) a reader keeps of a small generation it read whole; bounded LRU | **wherever your process/Lambda runs** — an EU segment queried from a US function is processed in the US |
-| **Loads and rewrites** (`store.load()`, the `*Into` verbs, `eraseSubject`) | read your source (or existing generations), write a new generation | run wherever you run them — a loader in one region writing to a bucket in another is a transfer |
+| **Loads and rewrites** (`store.load()`, the `*Into` verbs, `store.materializeMany()`, `eraseSubject`) | read your source (or existing generations), write a new generation; a fed or held operand of `materializeMany` is read from your own process's memory | run wherever you run them — a loader in one region writing to a bucket in another is a transfer |
 | **Intersection** | pulls chunks from N segments into one process | co-locates those segments in one region |
 
 **Guidance (not enforced by the library):** to keep EU data in EU infrastructure, wire region-local drivers
@@ -203,7 +203,7 @@ single-subject erasure is a rewrite (`eraseSubject`), while crypto-shred (`destr
 handles segment/tenant-level erasure and is the only erasure that reaches immutable backups / WORM copies of the
 objects — once no copy of the segment's registry row still holds its wrapped key, or every KEK that wrapped it
 is destroyed (below). And a
-**materialized segment** (`intersectInto` / `unionInto` / `andNotInto`) is a point-in-time snapshot of its
+**materialized segment** (`intersectInto` / `unionInto` / `andNotInto`, or an output of `materializeMany`) is a point-in-time snapshot of its
 inputs: erasing a subject from a source does not touch a destination computed earlier — which is exactly why
 `eraseSubject` scans *every* registered segment, destinations included, rather than erasing per source. What that
 costs is one unit of the per-op `budget` for each segment, and one more for each generation it opens in a segment

package/README.md

@@ -7,7 +7,7 @@ file, then ask `has`, `count`, `iterate`, `intersect`, `union` and `andNot`, wit
 > **ESM-only, Node ≥ 22.12.** Use `import`; for `require()`, Jest and TypeScript, see
 > [CommonJS, Jest and TypeScript](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/getting-started.md#commonjs-jest-and-typescript).
 
-> Pre-1.0: the API and the on-disk format can still change. These docs describe `main`, ahead of the npm release.
+> Pre-1.0: the API and the on-disk format can still change. On npm these docs match the version they ship in; on GitHub they describe `main`, and the changelog says what each release changed.
 
 ## Install
 
@@ -52,6 +52,11 @@ request handler. `store.exists(ref)` says whether a segment has been loaded. A l
 hold: `store.load(ref, { bitmap })` for a `RoaringBitmap32`, or `{ serialized }` for portable Roaring bytes, checked
 before anything is written and loaded with no per-id work, into the same bytes its ids would write.
 
+To write many results over the same stored operands, `store.materializeMany` reads each operand once for a whole batch of
+`and` / `or` / `andNot` outputs instead of once per `*Into` call, and `store.memory(ids)` holds a set you already have in
+memory as an operand of that batch. [Many outputs from one pass](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/loading.md#many-outputs-from-one-pass-materializemany) and
+[held operands](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/loading.md#operands-held-in-memory-storememory) cover both.
+
 ## Options
 
 `new CloudRoaring({ storage, ...options })`. Only `storage` is required. A key the store does not take is refused by
@@ -93,7 +98,8 @@ The [production checklist](https://github.com/cloudbitmaps/cloudbitmaps/blob/mai
 ## Coming from Redis bitmaps?
 
 `GETBIT` is `has`, `BITCOUNT` is `count`, and `BITOP AND` / `OR` / `DIFF` are `intersect` / `union` / `andNot`, with
-`intersectInto`, `unionInto` and `andNotInto` publishing the result as a new generation. `EXPIRE` becomes
+`intersectInto`, `unionInto` and `andNotInto` publishing the result as a new generation, and `materializeMany` publishing
+many such results from one pass. `EXPIRE` becomes
 `setRetention` plus a `retireExpired` sweep you schedule, per segment, never per id. `SETBIT` has no equivalent:
 there is no per-id write. Build the set upstream and load it, and do not loop one id at a time. Remove one id
 everywhere with `eraseSubject`, a rewrite for compliance, not a hot-path verb. This is not a drop-in replacement, and

package/dist/bin/export-segments.js

@@ -1131,8 +1131,9 @@ var CloudRoaring = class _CloudRoaring {
    *
    * **Every pinned operand an output subtracts is re-checked.** An output subtracts an operand when it is in its `exclude`
    * or under any entry of an `andNot` after the first, at any depth. Immediately before the publishes the call re-reads the
-   * registry row of each such operand and compares its generation and incarnation with what was pinned (a name deleted and
-   * created again is a replacement though it is generation 0 again). If one was replaced, each output that subtracts it is
+   * registry row of each such operand and compares its generation and incarnation, and the fingerprint of the current
+   * object where both reads carry one, with what was pinned (a name deleted and created again is a replacement though it
+   * is generation 0 again). If one was replaced, each output that subtracts it is
    * not published and carries {@link StaleOperandError}; the others publish. It costs one registry read per such operand per
    * group, and `stats.operands` carries every operand's pinned, start and end generation and whether it moved.
    *
@@ -1278,7 +1279,7 @@ var CloudRoaring = class _CloudRoaring {
         }
         fedMayBeEmpty = rest;
       }
-      if (feedIn === void 0 && (heldHandles.size === 0 || !Array.isArray(fedMayBeEmpty) || fedMayBeEmpty.length > 0)) {
+      if (feedIn === void 0 && (!Array.isArray(fedMayBeEmpty) || fedMayBeEmpty.length > 0)) {
         throw new ValidationError2(
           "materializeMany: mayBeEmpty names fed operands, and the call has no feed"
         );