@cloudbitmaps/roaring 0.17.0 → 0.18.0

package/PRIVACY.md

@@ -121,6 +121,14 @@ ledger is not by itself proof the id is gone.
 generation the erasure did not rewrite, so the id can be current again. Roll back before the erasure starts or
 after it returns, and re-run any erasure a rollback interrupted.
 
+### A batch of materializations
+
+A `store.materializeMany` call computes many `*Into` outputs in passes that read each operand once per group, and its window is the call's length, not one read's. An output of a `materializeMany` call can carry an id that was erased while the call ran, for as long as the call ran: the call reads each operand at the generation it pinned, and an output's publish is a load, so an erasure that lands after the chunks were read does not reach what the call holds, unless the output subtracts a pinned operand that the erasure rewrote, which the call re-reads just before the publishes and refuses. An erasure that rewrites a destination before that output's publish starts does not stop the publish, which, subject to the output's own `guard` and the refusal of an empty result over a non-empty destination, writes on top of the erasure's generation; an erasure that lands inside the publish's own write, between its pointer read and its pointer write, makes the publish lose with `WriteConflictError`; after any call that overlapped an erasure, re-run `eraseSubject` and keep both ledgers. `eraseSubject` deletes the generation it rewrites even when a call holds it pinned, so a pin does not outlive an erasure, and an output that still needs a deleted operand generation fails with `NotFoundError`.
+
+A call with a feed reads operands the library never stores, which no erasure can reach, so the guarantee above does not describe it, and it is refused instead. A `materializeMany` call with a feed is refused at its next record, and immediately before each fed output's publish (after the output has waited for its room in the memory budget), once `eraseSubject` has started in this store, one already running when the call began included: nothing is published from operands the call read before the erasure returned, apart from the window below, and the one record in hand when the erasure lands is processed and discarded. The counter it reads is moved at the start and at the end of each `eraseSubject` this store runs, so an erasure that began and finished between two of the call's checks is seen too; `rollback`, `retireExpired` and `dropSegment` do not move it. An erasure that starts after a fed output's last check and finishes before that publish's pointer write is not caught by the counter, and its window is the publish's own object write and pointer write, which lasts as long as the object takes to upload; the fences above still apply to what it touched. Only this store's `eraseSubject` moves the counter such a call reads, so an erasure by any other path (another store, another process, or the free function `eraseIdFromSegment` in this process) is not seen by it, and is bounded by nothing. An id erased from the source of a feed has to be erased there: the library holds none of it.
+
+A held operand, made by `store.memory`, is the caller's memory too, so no erasure can reach it either, and it is refused instead. A `materializeMany` call that reads an in-memory operand from `store.memory` fails the outputs that read it, before any request and immediately before each of their publishes, once `eraseSubject` has started in this store since the operand began to be made, one already running when it began included; an erasure by any path but this store's `eraseSubject` is not seen by such an operand, and is bounded by nothing. An erasure that starts after an output's last check and finishes before its publish writes is not caught by the counter, as for a feed, and the fences above still apply to what it touched. Build a held operand from your source of truth at the start of each refresh and release it at its end: the library keeps none of it once released.
+
 ### One process, and the rest of your fleet
 
 Erasure and crypto-shred are **immediate in storage and immediate in the store whose verb performed them**, for
@@ -137,6 +145,8 @@ bucket.
 | another store, with a registry and a `cache.genTtlMs` above 0 | within `cache.genTtlMs` (default 2 s), when its snapshot re-resolves, **while the registry can be read** (see below) |
 | another store with **no registry** (a bare `IStorageDriver`), with `cache: { genTtlMs: 0 }`, or on a storage source built with **no clock** | **no bound** — only when its caches happen to let the segment go, or something tells it |
 | `seg.pinAt` of a generation the erasure rewrote, in any store | on return it fails with `NotFoundError`, since the object is deleted (and, with a registry, the row has moved on); during the erasure, between the rewrite's publish and its delete of the old generation, it can still open that generation, and a handle opened then is a pin, covered by the row below |
+| a `materializeMany` call with a feed, and an erasure by any path but this store's `eraseSubject` | **no bound**: the call is refused only by this store's `eraseSubject` |
+| a held in-memory operand (`store.memory`), and an erasure by any path but this store's `eraseSubject` | **no bound**: the operand is refused only by this store's `eraseSubject`, started after the operand began to be made |
 | a pinned handle (`seg.pin()`) in another store | **no bound** — until that store's reader cache evicts the pin's reader (in a store with a timed pointer refresh, a small generation's reader holds all of its chunks, decoded or not) and its chunk cache evicts the chunks the pin decoded, or something tells it |
 | a leased pinned handle (`seg.pin({ leaseUntil })`) in another store | until its lease ends, at most 14 days after it was taken: its reads then throw `LeaseExpiredError`, whether or not its reader still holds the chunks. Until then, as for any pinned handle in the row above |
 

package/README.md

@@ -69,10 +69,10 @@ name.
 
 ## What it costs
 
-A cold `intersect` of two 500,000-id segments sharing 100 of their 1,999 chunks made 6 GETs and took 92.63 ms at the median, **$2.40 per
-million** at list prices: measured on S3 in-region, in the run of 2026-10-05, whose client had 128 sockets. On a spread layout it reads more
+A cold `intersect` of two 500,000-id segments sharing 100 of their 1,999 chunks made 6 GETs and took 99.80 ms at the median, **$2.40 per
+million** at list prices: measured on S3 in-region, in the run of 2026-10-06, whose client had 128 sockets. On a spread layout it reads more
 bytes than it needs, which cross-region can cost more than the requests it saves. A segment's first `store.load()` made
-2 PUT + 3 GET, **$11.20 per million** at the default prices, pointer included: the requests are measured on S3 in-region, in the same run. A cold `count()` is one
+2 PUT + 3 GET, **$11.20 per million** at the default prices, pointer included: the requests are measured on S3 in-region, in the same run. A steady `store.load()` at `keep: 12` that deletes a generation by name made 2 PUT + 4 GET and a free delete, measured: **$11.60 per million**. A cold `count()` is one
 pointer read: it reads no payload and no object.
 
 The trade is stated plainly. A membership check that misses the cache costs a ranged GET against object storage,

package/dist/bin/export-segments.js

@@ -38,6 +38,10 @@ import {
   groundedReport,
   loadSegment,
   loadSegmentChunks,
+  compileCombineMany,
+  prepareHeld,
+  rebindCombineMany,
+  runCombineMany,
   mapWithConcurrency,
   resolveBudget,
   resolvePerOpBudget,
@@ -635,6 +639,7 @@ import {
   BudgetExceededError,
   LeaseExpiredError as LeaseExpiredError2,
   LeaseLimitError,
+  StaleOperandError,
   isCloudRoaringError,
   isWriteConflictError,
   isTransientError,
@@ -643,6 +648,7 @@ import {
   isValidationError,
   isLeaseExpiredError,
   isLeaseLimitError,
+  isStaleOperandError,
   CrbmReader,
   BufferReader,
   CountingMetricsSink,
@@ -673,6 +679,36 @@ function requireScope(options, op) {
     );
   }
 }
+var viewOf;
+var MATERIALIZE_MANY_KEYS = [
+  "operands",
+  "outputs",
+  "keep",
+  "pin",
+  "maxBufferedBytes",
+  "budget",
+  "publishConcurrency",
+  "concurrency",
+  "allowAbsentOperands",
+  "after",
+  "through",
+  "feed",
+  "mayBeEmpty"
+];
+var MATERIALIZE_MANY_OUTPUT_KEYS = [
+  "dest",
+  "expr",
+  "exclude",
+  "guard",
+  "allowEmpty",
+  "metadata",
+  "audit",
+  "keep"
+];
+var DEFAULT_MAX_BUFFERED_BYTES = 256 * 1024 * 1024;
+var DEFAULT_PUBLISH_CONCURRENCY = 8;
+var DEFAULT_BATCH_CONCURRENCY = 1;
+var PIN_PARALLELISM = 16;
 function resolveStorageSource(options, clock) {
   const storage = options.storage;
   if (storage === null || typeof storage !== "object") {
@@ -785,6 +821,13 @@ var CloudRoaring = class _CloudRoaring {
   requireEncryption;
   /** Resolved store-level per-op budget (null = disabled); the admin scans use it, with a per-op override. */
   budget;
+  /**
+   * Moved at the start and at the end of each `eraseSubject` this store runs. A `materializeMany` call with a feed records
+   * it when it begins and is refused once it has moved: it reads caller memory, which no erasure can reach.
+   */
+  erasureEpoch = 0;
+  /** How many `eraseSubject` calls of this store are running: while any is, every check of the counter counts as moved. */
+  erasuresRunning = 0;
   /**
    * Refuse any option key the store does not take, at the top level or inside a group, naming each one.
    *
@@ -1024,6 +1067,440 @@ var CloudRoaring = class _CloudRoaring {
       collected: result.collected
     };
   }
+  /**
+   * **Hold a set of ids in memory**, to be an operand of {@link CloudRoaring.materializeMany} and of no other verb.
+   *
+   * ```ts
+   * const vip = await store.memory([7, 42, 99_001]);
+   * await store.materializeMany({ operands: { vip, engaged: store.segment('cond-engaged') }, outputs: [...], keep: 12 });
+   * vip.release();
+   * ```
+   *
+   * It takes what {@link CloudRoaring.load} takes as input (ids as a sync or async iterable or a typed array, `{ bitmap }`
+   * or `{ serialized }`) and checks it the same way, with the same errors: the size cap, the structure, the safe
+   * deserializer and each id's range. **Only a real `Uint32Array` skips the per-id range check**; any other typed array goes
+   * through it, so `Int32Array([-1])` and `Float64Array([1.5])` throw {@link ValidationError}. An object with an `ascending`
+   * key is refused: ids that arrive in key order are a `feed`. An empty set is accepted here, and refused by
+   * `materializeMany` unless its name is in `mayBeEmpty`.
+   *
+   * The ids are held as the chunks a stored generation would hold, until {@link MemoryOperand.release}; between calls that
+   * memory is the caller's, and during a call it counts against `maxBufferedBytes`. There is no size option: the load's
+   * own cap applies to a serialized input.
+   *
+   * **Erasure.** Only `eraseSubject` moves the store's counter (at its start and at its end). One that started in this store
+   * since the handle began to be made, or is still running, makes a call that reads the handle fail the outputs that read it with
+   * {@link StaleOperandError} (`reason: 'erased'`), before any request and again just before each publish. An erasure in
+   * another store or another process is not seen, nor are `dropSegment`, `eraseNamespace`, `destroySegment` and the free
+   * function `eraseIdFromSegment`, which do not move it: the handle is the caller's copy, so build it from the source of truth at
+   * the start of each refresh and release it at the end.
+   *
+   * @throws {ValidationError} where a load of the same input would.
+   */
+  async memory(input) {
+    const epochAt = this.epochNow();
+    const chunks = await prepareHeld(input, roaringCodec, this.clock);
+    return makeMemoryOperand({ store: this, chunks, epochAt });
+  }
+  /**
+   * **Many outputs, each operand read once per group.** Each output is an expression over named stored operands (`and`, `or`, `andNot`, nested
+   * to depth 64) published as a new generation of its own `dest`, exactly as an `*Into` would publish it, but computed
+   * together: each operand's chunks are read once for all the outputs that use them, instead of once per output and
+   * once more for every scratch segment a nested expression would otherwise need.
+   *
+   * ```ts
+   * const run = await store.materializeMany({
+   *   operands: { us: store.segment('us'), engaged: store.segment('engaged'), optOut: store.segment('opt-out') },
+   *   outputs: [
+   *     { dest: store.segment('send-1'), expr: { and: ['us', 'engaged'] }, exclude: ['optOut'] },
+   *     { dest: store.segment('send-2'), expr: { andNot: ['us', 'engaged'] } },
+   *   ],
+   *   keep: 3,
+   * });
+   * run.outputs[0]; // a MaterializeResult, or { published: false, error }
+   * ```
+   *
+   * **Results.** `run.outputs[i]` is index-aligned with `outputs` and is exactly what output `i`'s `*Into` would have
+   * returned (a {@link MaterializeResult}: published, or refused with its `reason`), or `{ published: false, error }` for
+   * what it would have thrown. One output's refusal or error never stops another. The call itself throws only for bad
+   * input ({@link ValidationError} naming the output index and path, before any request), an operand that names no
+   * segment (unless `allowAbsentOperands`), and a `budget` its own plan exceeds, the last two before any chunk is read.
+   *
+   * **One generation per operand.** By default every stored operand is pinned for the call: read at the generation it had
+   * when the call started, however long the call runs and whatever loads land meanwhile. That is one generation per
+   * operand, not one instant across operands (two pins are two registry reads). `pin: false` reads live, as a combine does.
+   *
+   * **Every pinned operand an output subtracts is re-checked.** An output subtracts an operand when it is in its `exclude`
+   * or under any entry of an `andNot` after the first, at any depth. Immediately before the publishes the call re-reads the
+   * registry row of each such operand and compares its generation and incarnation with what was pinned (a name deleted and
+   * created again is a replacement though it is generation 0 again). If one was replaced, each output that subtracts it is
+   * not published and carries {@link StaleOperandError}; the others publish. It costs one registry read per such operand per
+   * group, and `stats.operands` carries every operand's pinned, start and end generation and whether it moved.
+   *
+   * **Memory.** Everything the pass holds resident is counted against `maxBufferedBytes` (default 256 MiB): each output's
+   * buffered chunks, the operand streams' ranges, the chunks being evaluated, and the plan itself, and a call whose plan alone
+   * passes it is refused before any chunk is read. When the outputs do not fit, they run in groups, each reading its operands
+   * once (so the operand reads grow with the total output size over the budget), and a group's buffers are released only as
+   * its publishes settle. A chunk count the index understated does not grow a group past the budget: the largest buffer is
+   * dropped and that output re-runs alone. An output that cannot fit alone has `BudgetExceededError` as its result. The count
+   * is the pass's own: process memory is more, measured at 1.6 times the ledger's high water on Linux and 2.5 to 5 times on
+   * macOS on the in-memory backend (the guide has the shapes and what is not counted).
+   *
+   * **Untrusted bytes.** Every operand chunk is decoded through the checks on untrusted tier data. A key an operand's index
+   * lists whose bytes are missing is an error for the outputs that read the operand, never an empty chunk.
+   *
+   * **Handles.** An expired or released handle anywhere in the call (an operand or a `dest`) is refused before any request,
+   * as on the `*Into` verbs. A leased operand's lease, and an operand's deadline, are checked before each chunk key
+   * is read: a lapse fails only the outputs that read that operand, never reads empty. Each `dest`'s lease and deadline
+   * are checked again just before its publish.
+   *
+   * **The chunk cache is bypassed:** the pass reads through the storage source and never writes the decoded-chunk cache, so
+   * a batch does not evict other readers' hot chunks. Each publish still invalidates its `dest` as an `*Into` does.
+   *
+   * **A limit of the form.** An output cannot name another output of the same call: a `dest` that is also an operand is
+   * refused. Chain two calls.
+   *
+   * **Held operands.** An operand may be a {@link MemoryOperand} from {@link CloudRoaring.memory}: ids the caller holds,
+   * read beside the stored ones and giving the same output byte for byte. It is read from memory (no request, and the
+   * shared chunk cache never sees it), counts against `maxBufferedBytes` while the call runs, and is refused when empty
+   * unless its name is in `mayBeEmpty`. A handle from another store, or a released one, is a {@link ValidationError}. An
+   * erasure in this store since the handle was made fails the outputs that read it with {@link StaleOperandError}
+   * (`reason: 'erased'`).
+   *
+   * **A feed.** `feed: { names, records, counts }` adds operands that arrive as records in chunk-key order, for conditions
+   * too many to hold or store. Each record is checked before the pass sees it (see {@link MaterializeManyFeedRecord}): a bad
+   * feed is refused, never read as fewer members, and **a fed output is published only after the whole feed was read and every
+   * end-of-feed check passed**, so a bad feed refuses every fed output and publishes none of them; outputs that name only
+   * stored operands are unaffected. At the end of the feed `counts` must equal the ids the feed held for each name, and a
+   * name that appeared in no record is refused unless it is in `mayBeEmpty`. A feed is read once, so the call runs its outputs
+   * as one group and **`maxBufferedBytes` is required**: each record is converted to compressed bitmaps as it arrives, the key
+   * in hand and its fed outputs are counted against the budget, and the call fails as soon as the count passes it, with
+   * {@link BudgetExceededError} for every fed output. **Erasure:** a fed call is refused at its next record, and before each
+   * fed publish, once `eraseSubject` has started in this store ({@link StaleOperandError}, `reason: 'erased'`).
+   *
+   * ```ts
+   * await store.materializeMany({
+   *   operands: { unsub: store.segment('global-unsub') },
+   *   feed: {
+   *     names: ['us', 'engaged'],
+   *     records: warehouseRecords(), // yields { key, operands: { us: Uint32Array, engaged: Uint32Array } } by key
+   *     counts: () => warehouseCounts(),
+   *   },
+   *   maxBufferedBytes: 512 * 1024 * 1024,
+   *   outputs: [{ dest: store.segment('send-1'), expr: { and: ['us', 'engaged'] }, exclude: ['unsub'] }],
+   *   keep: 12,
+   * });
+   * ```
+   */
+  async materializeMany(options) {
+    if (typeof options !== "object" || options === null) {
+      throw new ValidationError2(
+        "materializeMany needs an options object with `operands`, `outputs` and `keep`"
+      );
+    }
+    const epochAt = options.feed === void 0 ? 0 : this.epochNow();
+    const unknown = Object.keys(options).filter(
+      (k) => !MATERIALIZE_MANY_KEYS.includes(k)
+    );
+    if (unknown.length > 0) {
+      throw new ValidationError2(
+        `materializeMany: unknown option${unknown.length > 1 ? "s" : ""} ${unknown.map((k) => `"${k}"`).join(", ")}`
+      );
+    }
+    const operandsIn = options.operands;
+    const outputsIn = options.outputs;
+    const keep = options.keep;
+    const after = options.after;
+    const through = options.through;
+    const pinOption = options.pin ?? true;
+    const feedIn = options.feed;
+    const mayBeEmptyIn = options.mayBeEmpty;
+    if (feedIn !== void 0 && options.maxBufferedBytes === void 0) {
+      throw new ValidationError2(
+        "materializeMany: maxBufferedBytes is required with a feed: the feed is read once, so the call runs as one group and the budget is what bounds it"
+      );
+    }
+    const maxBufferedBytes = options.maxBufferedBytes ?? DEFAULT_MAX_BUFFERED_BYTES;
+    const publishConcurrency = options.publishConcurrency ?? DEFAULT_PUBLISH_CONCURRENCY;
+    const concurrency = options.concurrency ?? DEFAULT_BATCH_CONCURRENCY;
+    const budgetOption = options.budget;
+    const allowAbsentOperands = options.allowAbsentOperands;
+    if (typeof pinOption !== "boolean")
+      throw new ValidationError2("materializeMany: pin must be a boolean");
+    if (allowAbsentOperands !== void 0 && typeof allowAbsentOperands !== "boolean") {
+      throw new ValidationError2("materializeMany: allowAbsentOperands must be a boolean");
+    }
+    if (typeof operandsIn !== "object" || operandsIn === null || Array.isArray(operandsIn)) {
+      throw new ValidationError2("materializeMany: operands must be an object of name to segment");
+    }
+    if (!Array.isArray(outputsIn) || outputsIn.length === 0) {
+      throw new ValidationError2("materializeMany: outputs must be a non-empty array");
+    }
+    const names = Object.keys(operandsIn);
+    const handles = /* @__PURE__ */ new Map();
+    const heldHandles = /* @__PURE__ */ new Map();
+    for (const name of names) {
+      const handle = operandsIn[name];
+      if (handle instanceof MemoryOperand) {
+        const view = heldOf(handle);
+        if (view === void 0) {
+          throw new ValidationError2(
+            `materializeMany: operands["${name}"] is not a memory operand from store.memory()`
+          );
+        }
+        if (view.store !== this) {
+          throw new ValidationError2(
+            `materializeMany: operands["${name}"] was made by another store: a memory operand belongs to the store that made it`
+          );
+        }
+        if (view.chunks.released) {
+          throw new ValidationError2(
+            `materializeMany: operands["${name}"] was released: make another with store.memory()`
+          );
+        }
+        heldHandles.set(name, view);
+        continue;
+      }
+      if (!(handle instanceof Segment)) {
+        throw new ValidationError2(
+          `materializeMany: operands["${name}"] must be a segment from store.segment() or a memory operand from store.memory()`
+        );
+      }
+      handles.set(name, handle);
+    }
+    const heldMayBeEmpty = /* @__PURE__ */ new Set();
+    let fedMayBeEmpty = mayBeEmptyIn;
+    if (mayBeEmptyIn !== void 0) {
+      if (Array.isArray(mayBeEmptyIn)) {
+        const rest = [];
+        for (const name of mayBeEmptyIn) {
+          if (typeof name === "string" && heldHandles.has(name)) heldMayBeEmpty.add(name);
+          else rest.push(name);
+        }
+        fedMayBeEmpty = rest;
+      }
+      if (feedIn === void 0 && (heldHandles.size === 0 || !Array.isArray(fedMayBeEmpty) || fedMayBeEmpty.length > 0)) {
+        throw new ValidationError2(
+          "materializeMany: mayBeEmpty names fed operands, and the call has no feed"
+        );
+      }
+    }
+    const operandKeys = new Set([...handles.values()].map((h) => h.key()));
+    const outputs = [];
+    const destKeys = /* @__PURE__ */ new Set();
+    outputsIn.forEach((raw, i) => {
+      if (typeof raw !== "object" || raw === null) {
+        throw new ValidationError2(`outputs[${i}] must be an object`);
+      }
+      const bad = Object.keys(raw).filter(
+        (k) => !MATERIALIZE_MANY_OUTPUT_KEYS.includes(k)
+      );
+      if (bad.length > 0) {
+        throw new ValidationError2(
+          `outputs[${i}]: unknown option${bad.length > 1 ? "s" : ""} ${bad.map((k) => `"${k}"`).join(", ")}`
+        );
+      }
+      const spec = raw;
+      const dest = spec.dest;
+      if (!(dest instanceof Segment)) {
+        throw new ValidationError2(`outputs[${i}].dest must be a segment from store.segment()`);
+      }
+      if (operandKeys.has(dest.key())) {
+        throw new ValidationError2(
+          `outputs[${i}].dest is also an operand of the call: one call is one pass over operands that do not change under it, so chain two calls`
+        );
+      }
+      if (destKeys.has(dest.key())) {
+        throw new ValidationError2(
+          `outputs[${i}].dest names a segment another output of the call writes`
+        );
+      }
+      destKeys.add(dest.key());
+      outputs.push({ spec, dest });
+    });
+    const storeCeiling = this.budget === null || this.budget.maxRequests !== DEFAULT_BUDGET.maxRequests ? this.budget : void 0;
+    const budget = budgetOption === void 0 ? storeCeiling : resolvePerOpBudget(budgetOption, this.budget);
+    const operandSpecs = (pins2) => names.map((name) => {
+      const view = heldHandles.get(name);
+      if (view !== void 0) {
+        return {
+          name,
+          ref: { segment: name },
+          held: {
+            chunks: view.chunks,
+            moved: () => this.epochMoved(view.epochAt),
+            mayBeEmpty: heldMayBeEmpty.has(name)
+          }
+        };
+      }
+      const handle = handles.get(name);
+      const seg = viewOf(handle);
+      const pinnedAt = pins2.get(handle.key());
+      const guarded = seg.expiresAt !== void 0 || handle.lease !== void 0;
+      return {
+        name,
+        ref: seg.ref,
+        ...guarded ? { check: () => this.assertReadable("materializeMany", handle) } : {},
+        ...pinnedAt === void 0 ? {} : {
+          pinnedGeneration: pinnedAt.generation,
+          pinnedVersion: pinnedAt.version,
+          pinnedFingerprint: pinnedAt.fingerprint ?? null
+        },
+        current: (read) => this.currentRowOf(seg.ref, read?.fingerprint === true)
+      };
+    });
+    const request = (pins2) => ({
+      operands: operandSpecs(pins2),
+      outputs: outputs.map(({ spec, dest }, i) => ({
+        expr: spec.expr,
+        ...spec.exclude === void 0 ? {} : { exclude: spec.exclude },
+        ...spec.allowEmpty === void 0 ? {} : { allowEmpty: spec.allowEmpty },
+        ...spec.guard === void 0 ? {} : { guard: spec.guard },
+        ...spec.metadata === void 0 ? {} : { metadata: spec.metadata },
+        ...spec.keep === void 0 ? {} : { keep: spec.keep },
+        beforePublish: () => this.assertWritable(`outputs[${i}]`, dest),
+        // What a settled publish proves it sent: a published one wrote its object and moved the pointer, a refused one
+        // wrote its object and deleted it.
+        requestsOf: (result) => ({ get: 0, put: result.published ? 2 : 1 }),
+        publish: (chunks, write) => this.materialize(viewOf(dest).ref, new CombineChunks(chunks), "materializeMany", {
+          ...write,
+          ...spec.audit === void 0 ? {} : { audit: spec.audit }
+        })
+      })),
+      keep,
+      ...after === void 0 ? {} : { after },
+      ...through === void 0 ? {} : { through },
+      maxBufferedBytes,
+      publishConcurrency,
+      concurrency,
+      ...budget === void 0 ? {} : { budget },
+      ...allowAbsentOperands === void 0 ? {} : { allowAbsentOperands },
+      ...feedIn === void 0 ? {} : { feed: this.feedOf(feedIn, fedMayBeEmpty, epochAt) }
+    });
+    const compiled = compileCombineMany(request(/* @__PURE__ */ new Map()));
+    for (const { dest } of outputs) this.assertWritable("materializeMany", dest);
+    for (const handle of handles.values()) this.assertReadable("materializeMany", handle);
+    this.lifecycleDeps("materializeMany");
+    const crbm = this.crbmSource;
+    if (crbm === void 0) {
+      throw new UnsupportedError(
+        "materializeMany needs the `.crbm` storage source \u2014 pass a backend or a raw IStorageDriver as `storage`"
+      );
+    }
+    const held = /* @__PURE__ */ new Map();
+    for (const [name, handle] of handles) {
+      const key = handle.key();
+      const at = handle.pinnedAt;
+      const prior = held.get(key);
+      if (prior === void 0) {
+        held.set(key, { at, handle });
+      } else if (prior.at !== void 0 && at !== void 0) {
+        if (!samePin(prior.at, at)) {
+          throw new ValidationError2(
+            `materializeMany: operand "${name}" holds a segment another operand holds, ${twoPins(prior.at, at)}: one call reads a segment at one generation`
+          );
+        }
+      } else if (prior.at !== void 0 || at !== void 0) {
+        if (!pinOption) {
+          throw new ValidationError2(
+            `materializeMany: operand "${name}" holds a segment another operand holds, ${twoPins(prior.at, at)}: one call reads a segment at one generation`
+          );
+        }
+        if (at !== void 0) held.set(key, { at, handle });
+      }
+    }
+    const pins = /* @__PURE__ */ new Map();
+    const toPin = [];
+    for (const [key, { at, handle }] of held) {
+      if (at !== void 0) pins.set(key, at);
+      else if (pinOption) toPin.push([key, handle]);
+      else pins.set(key, void 0);
+    }
+    const taken = await mapWithConcurrency(
+      toPin,
+      PIN_PARALLELISM,
+      async ([key, handle]) => [key, (await handle.pin()).pinnedAt]
+    );
+    for (const [key, at] of taken) pins.set(key, at);
+    const live = new Map([...pins].filter((e) => e[1] !== void 0));
+    const pinnedSource = new PinnedStorageChunkSource(crbm, live);
+    const source = this.retryOptions === void 0 ? pinnedSource : new RetryingStorageChunkSource(pinnedSource, this.retryOptions);
+    const final = rebindCombineMany(compiled, operandSpecs(pins));
+    const run = await runCombineMany(final, { source, codec: roaringCodec, clock: this.clock });
+    return {
+      outputs: run.outputs.map(
+        (o) => o.ok ? o.value : {
+          published: false,
+          error: o.error instanceof Error ? o.error : new Error(String(o.error))
+        }
+      ),
+      stats: run.stats
+    };
+  }
+  epochNow() {
+    return this.erasureEpoch;
+  }
+  /**
+   * Whether an erasure has started in this store since `at` was read, or was running when it was: one that is still
+   * running is seen by the count of those running, and one that has ended since by the counter, which its end moved.
+   * The one check every call that reads caller memory uses.
+   */
+  epochMoved(at) {
+    return this.erasuresRunning > 0 || this.epochNow() !== at;
+  }
+  /** A call's feed with the store's erasure counter: read once here, checked by the combine before any record is read. */
+  feedOf(feed, mayBeEmpty, epochAt) {
+    if (typeof feed !== "object" || feed === null || Array.isArray(feed)) {
+      throw new ValidationError2(
+        "materializeMany: feed must be an object with names, records and counts"
+      );
+    }
+    const f = feed;
+    return {
+      names: f.names,
+      records: f.records,
+      counts: f.counts,
+      ...mayBeEmpty === void 0 ? {} : { mayBeEmpty },
+      epoch: { moved: () => this.epochMoved(epochAt) }
+    };
+  }
+  /**
+   * The registry row of `ref`, read straight from the registry, not from a snapshot: a re-check must see a move. With
+   * `fingerprint`, the current generation's object is opened too (one tail read, as a pin of it makes), and its
+   * fingerprint is returned beside the row: a generation's number is identified by its row and its object, since a number
+   * can be taken again once its object is gone.
+   */
+  async currentRowOf(ref, fingerprint) {
+    const registry = this.requireRegistry("materializeMany");
+    if (fingerprint && this.crbmSource !== void 0) {
+      const crbm = this.crbmSource;
+      const at = await this.withRetries(() => crbm.pinGeneration(ref));
+      const colon = at?.version.indexOf(":") ?? -1;
+      if (at !== null && colon >= 0) {
+        return {
+          generation: at.generation,
+          token: at.version.slice(colon + 1),
+          fingerprint: at.fingerprint
+        };
+      }
+    }
+    const row = await this.withRetries(() => registry.get(ref));
+    return row === null ? null : { generation: row.currentGen, token: row.token };
+  }
+  /** Throw what a read of `handle` must throw now: its lease error, or a refusal for a deadline that has passed. */
+  assertReadable(op, handle) {
+    const view = viewOf(handle);
+    const lease = view.leaseError();
+    if (lease !== void 0) throw lease;
+    if (view.expired()) {
+      throw new ValidationError2(
+        `${op}: refusing to read "${view.ref.segment}", whose handle has expired \u2014 an expired handle reads as empty, so a generation built from it would be short. Open the handle without \`expiresAt\` if you meant to materialise.`
+      );
+    }
+  }
+  /** {@link assertReadable} for a destination, named by where it sits in the call. */
+  assertWritable(where, dest) {
+    this.assertReadable(where, dest);
+  }
   /**
    * **Subject access (GDPR Art. 15 / CCPA right-to-know): which segments is this id a member of?**
    *
@@ -1132,6 +1609,16 @@ var CloudRoaring = class _CloudRoaring {
    * is `dropSegment` / the `destroySegment`/`eraseNamespace` free functions.
    */
   async eraseSubject(id, options = {}) {
+    this.erasureEpoch++;
+    this.erasuresRunning++;
+    try {
+      return await this.eraseSubjectRun(id, options);
+    } finally {
+      this.erasuresRunning--;
+      this.erasureEpoch++;
+    }
+  }
+  async eraseSubjectRun(id, options) {
     const deps = this.lifecycleDeps("eraseSubject");
     requireScope(options, "eraseSubject");
     validateConcurrency(options.concurrency);
@@ -1948,6 +2435,39 @@ var AS_CHUNKS = {
   andNot: (engine, base, refs, opts) => engine.andNotChunks(base, refs, opts)
 };
 var KEEP_EVERY_GENERATION = Number.MAX_SAFE_INTEGER;
+var heldOf;
+var makeMemoryOperand;
+var mintingMemory = false;
+var MemoryOperand = class _MemoryOperand {
+  view;
+  static {
+    heldOf = (operand) => operand.view;
+    makeMemoryOperand = (view) => {
+      mintingMemory = true;
+      try {
+        return new _MemoryOperand(view);
+      } finally {
+        mintingMemory = false;
+      }
+    };
+  }
+  /** Not constructible: a handle comes from {@link CloudRoaring.memory}. */
+  constructor(view) {
+    if (!mintingMemory) {
+      throw new ValidationError2(
+        "a MemoryOperand is not constructed directly; call `await store.memory(ids)`"
+      );
+    }
+    this.view = view;
+  }
+  /**
+   * Zero and drop the bytes it holds. Any later use throws {@link ValidationError}; releasing twice does nothing. A call
+   * still reading it fails the outputs that read it.
+   */
+  release() {
+    this.view.chunks.release();
+  }
+};
 var makeSegment;
 var minting = false;
 var Segment = class _Segment {
@@ -1982,6 +2502,13 @@ var Segment = class _Segment {
         minting = false;
       }
     };
+    viewOf = (segment) => ({
+      ref: segment.ref,
+      pinnedAt: segment.pinnedAt,
+      expiresAt: segment.expiresAt,
+      leaseError: () => segment.leaseError(),
+      expired: () => segment.expired()
+    });
   }
   /**
    * Not constructible: a handle comes from {@link CloudRoaring.segment}, which wires it to the store's engine,