@cloudbitmaps/roaring 0.21.0 → 0.23.0

package/PRIVACY.md

@@ -42,8 +42,8 @@ from constructing a cross-region topology. The points where personal data moves
 
 | Location | What's there | Residency note |
 |---|---|---|
-| **Storage** (object store) | immutable `.crbm` generations — every generation a segment has had, until a superseded one is collected. Azure Blob objects, and GCS objects above the simple-upload threshold, also carry a random write id in their metadata (`cbwid`): 128 random bits, no data from the bitmap or the source | the region of the bucket you wire |
-| **Registry** (S3 / GCS / Azure Blob / local) | one row per segment: its namespace and name, its status and timestamps, the current-generation pointer, wrapped keys and any key id, its `retention` and `residency` metadata, the current generation's id count, its object's size and footer checksum and your metadata (those three sealed when the segment is encrypted), the generations its loads keep, any leases, and a token and `pointerId` — no IDs, unless you put one in the metadata or a name, which you must not. An Azure Blob row also carries the random write id in its metadata | the region of the bucket you wire |
+| **Storage** (object store) | immutable `.crbm` generations — every generation a segment has had, until a superseded one is collected. Every generation object a cloud driver writes (S3, GCS and Azure Blob) also carries a random write id in its metadata (`cbwid`; `x-amz-meta-cbwid` on S3): 128 random bits, no data from the bitmap or the source. A write that fails or is killed part way can leave the uploaded part of a generation behind (S3 multipart parts, Azure Blob uncommitted blocks, an unfinished GCS resumable session, a `LocalFsStorage` `.tmp` file); no listing shows it, so no erasure or drop reaches it. What each backend leaves, and when it is removed, is in [the backup checklist](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/disaster-recovery.md#backup-checklist) | the region of the bucket you wire |
+| **Registry** (S3 / GCS / Azure Blob / local) | one row per segment: its namespace and name, its status and timestamps, the current-generation pointer, wrapped keys and any key id, its `retention` and `residency` metadata, the current generation's id count, its object's size, footer checksum and tail size, and your metadata (those sealed when the segment is encrypted), a ledger of the generations the segment knows (each one's number and its object's footer checksum, in the clear even on an encrypted segment), any leases, and a token and `pointerId` — no IDs, unless you put one in the metadata or a name, which you must not. An Azure Blob row also carries the random write id in its metadata. On `LocalFsStorage`, a write killed part way leaves a `.tmp` copy of the row beside it, wrapped keys included, until a later write in that directory removes it once it is more than a day old, so a crypto-shred there can be outlived by that copy for that long | the region of the bucket you wire |
 | **cache** (process RAM) | decoded chunks, and the stored chunk bytes (ciphertext on an encrypted segment) a reader keeps of a small generation it read whole; bounded LRU | **wherever your process/Lambda runs** — an EU segment queried from a US function is processed in the US |
 | **Loads and rewrites** (`store.load()`, the `*Into` verbs, `store.materializeMany()`, `eraseSubject`) | read your source (or existing generations), write a new generation; a fed operand of `materializeMany` is read from your own process's memory | run wherever you run them — a loader in one region writing to a bucket in another is a transfer |
 | **Intersection** | pulls chunks from N segments into one process | co-locates those segments in one region |
@@ -60,16 +60,18 @@ CloudBitmaps gives you three levers with different guarantees. Use them delibera
 
 | Lever | API | Guarantee | Use for |
 |---|---|---|---|
-| **Subject erasure** | `store.eraseSubject(id, { namespace })` | *Physical on return* — the segment's current generation is rewritten without the id (every chunk streamed through, one bit cleared), published **fenced on the generation it streamed**, and **the generation that held the bit is deleted before the call returns**. So is every other generation still holding the id: a retained *superseded* one (an ex-member dropped by a re-seed), and each one above the pointer after a `rollback` — generations a rollback can move the pointer onto again — while, when the current generation does not hold the id, the ones there that never held it stay as rollback targets, but for any object no read of the segment can open, which goes whatever the id. (When the current generation holds the id, the rewrite is numbered above everything, so its `keep: 0` collection takes every older generation, above the pointer or below.) Before it says `erased: true` the call lists the bucket and reads what is left, so **`erased: true` means no generation of the segment holds the id**, above the pointer or below it. `eraseSubject` reports a per-segment fault as an `error: …` ledger entry rather than throwing it, and a call that a racing writer overtook (a load, another erasure, or a rollback) as `erased: false, note: 'superseded'`, so "on return" is a claim about every segment whose entry says **`erased: true`** (a fault is an `error: …` entry, never `erased: true`). The store that performs the call stops serving the id immediately; **other processes converge on their own read TTL** — see [One process, and the rest of your fleet](#one-process-and-the-rest-of-your-fleet). Does **not** reach backups, replicas or noncurrent object versions — those hold the old generation until their own lifecycle removes it. It carries a generation's metadata over unchanged and does **not** scan it, so **never put a subject's id in metadata**. | "forget this person" — GDPR Art. 17 |
+| **Subject erasure** | `store.eraseSubject(id, { namespace })` | *Physical on return* — the segment's current generation is rewritten without the id (every chunk streamed through, one bit cleared), published **fenced on the generation it streamed**, and **the generation that held the bit is deleted before the call returns**. So is every other generation still holding the id: a retained *superseded* one (an ex-member dropped by a re-seed), and each one above the pointer after a `rollback` — generations a rollback can move the pointer onto again — while, when the current generation does not hold the id, the ones there that never held it stay as rollback targets, but for any object no read of the segment can open, which goes whatever the id. (When the current generation holds the id, the rewrite is numbered above everything, so its `keep: 0` collection takes every older generation, above the pointer or below. A holder found only above the pointer likewise takes every generation below the pointer with it, clean ones included ([every holder goes](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/correctness.md#every-holder-goes)); above the pointer, a generation with no copy of the id stays, and so does the current one.) Before it says `erased: true` the call lists the bucket and reads what is left, so **`erased: true` means no generation of the segment holds the id**, above the pointer or below it. `eraseSubject` reports a per-segment fault as an `error: …` ledger entry rather than throwing it, and a call that a racing writer overtook (a load, another erasure, or a rollback) as `erased: false, note: 'superseded'`, so "on return" is a claim about every segment whose entry says **`erased: true`** (a fault is an `error: …` entry, never `erased: true`). The store that performs the call stops serving the id immediately; **other processes converge on their own read TTL** — see [One process, and the rest of your fleet](#one-process-and-the-rest-of-your-fleet). Does **not** reach backups, replicas or noncurrent object versions — those hold the old generation until their own lifecycle removes it. It carries a generation's metadata over unchanged and does **not** scan it, so **never put a subject's id in metadata**. The scope is one `namespace` or `{ allNamespaces: true }`, never both (both throws `ValidationError`); a segment loaded with no `namespace` is reached only by `{ allNamespaces: true }`, and `'_default'` is an ordinary namespace name, not those segments. | "forget this person" — GDPR Art. 17 |
 | **Dispose** | `store.dropSegment(ref, { confirmSegment })` | *Immediate* — the segment is tombstoned and its Storage generations deleted, reclaiming the space. **Check `generationsRemaining`:** if it is non-empty the space was *not* fully reclaimed and the drop should be re-run (a load that had read the segment before the tombstone landed, even one still consuming its ids, or an `eraseSubject` rewrite already under way, can write its object after the drop's sweep; it deletes that object itself once its publish is refused, and only one whose process stops in between, or whose publish fails without a definite answer (a lost response, a timeout), leaves it, for a re-run of the drop). Works on cleartext; on an encrypted segment it *also* crypto-shreds, on the terms in the next row. Does **not** reach noncurrent versions / replicas / PITR snapshots — deleting an object is weaker than removing a key. | retiring a dated bucket; rolling-window retention |
 | **Crypto-shred** | `destroySegment` / `eraseNamespace` | *At rest, complete once no copy of the wrapped key survives* — removes the segment's wrapped DEK(s) from its registry row in one compare-and-swap, without touching the bytes or any KEK. **Every** copy of the objects (current, prior generations, backups, WORM-locked objects) is then unreadable once no retained copy of that row still holds the wrapped key — noncurrent object versions, backups and PITR copies of the row included — or once every KEK that wrapped it is destroyed. Until then, a retained copy of the row and a KEK that wrapped it decrypt the segment, and a registry restore to a point before the shred makes it readable again; see *Erasure vs. backups / WORM* below, and [an optional registry expiry rule](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/disaster-recovery.md#optional-make-a-shred-durable-with-a-registry-expiry-rule) that makes a shred durable after a set number of days. Requires the segment to be encrypted. A process that had already opened the segment holds the **unwrapped** key in memory and keeps reading until it is told — call `store.invalidate(ref)` there; see [One process, and the rest of your fleet](#one-process-and-the-rest-of-your-fleet). | whole-segment / tenant offboarding; erasure under immutable backups (see below) |
 
 **Subject-wide erasure** (GDPR Art. 17 — "forget this person everywhere") is
-`store.eraseSubject(id, { namespace })` — or `{ allNamespaces: true }` to sweep every tenant deliberately. For
-every **registered** segment the id is in, it rewrites the current generation without the id, publishes the
-rewrite, and deletes every generation that held the bit — the one it replaced, a retained superseded one, and
-any above the pointer that a rollback left — so the bit is physically gone from the bucket on return, for idle
-and archival segments as much as busy ones. It checks rather than assumes: before an entry says `erased: true`,
+`store.eraseSubject(id, { namespace })` — or `{ allNamespaces: true }` to sweep every tenant deliberately; giving both throws `ValidationError`. A segment loaded with no `namespace` is in no namespace you can name: only `{ allNamespaces: true }` reaches it, and `'_default'`, the label the export tool and the disaster-recovery samples print, and the bucket's object keys use, for those segments, is an ordinary namespace name here, so passing it scans a namespace of that name, not them. For
+every **registered** segment the id is in, it [rewrites the current generation without the
+id](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/correctness.md#erasure-is-a-rewrite), publishes the rewrite, and deletes [every generation that held
+the bit](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/correctness.md#every-holder-goes) — the one it replaced, a retained superseded one, and any
+above the pointer that a rollback left — so the bit is physically gone from the bucket on return, for idle
+and archival segments as much as busy ones (a noncurrent version, a replica or a backup keeps the old object until
+its own lifecycle removes it; see *Honest limits* below). It checks rather than assumes: before an entry says `erased: true`,
 the call lists the segment's generations and reads each one it has not already seen without the id. There is no logical-then-physical gap and no
 scheduled step to wait for: an erasure *is* a new generation, the same shape as every other write in the library.
 It reuses the store's own drivers (so build the store with a backend). It returns an
@@ -86,11 +88,11 @@ not in are not listed. A segment where the erasure found the id in nothing it co
 no read of it can open, is not listed either: the audit sink gets `segment.collect { collected }` for it, with no
 `fromGeneration`, and that event is the only record of the deletion. `store.subjectReport(id, { namespace })` answers
 the read side (Art. 15 — which segments
-an id is in).
+an id is in, and which destroyed segments still hold it in objects left in the bucket).
 
 An erasure that finds the id in no generation but an older one also writes the segment's registry row once, before it
 deletes anything, when the row's ledger of known generations names a generation the call is about to delete: it takes
-those names out of the ledger first (see [how it stays correct](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/erasure.md#how-it-stays-correct)). That write
+those names out of the ledger first (see [how it stays correct](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/correctness.md#how-an-erasure-stays-correct)). That write
 is a timing signal. Someone who can read the registry and knows when an erasure ran can tell that this segment's row
 changed then, and so that the segment held the subject in a generation older than its current one. The row also
 changes at erasure time when the segment's current generation held the id, or one above the pointer, a first load's
@@ -111,7 +113,7 @@ replaced, so it still holds the winner's id, and when it sits above the winner's
 returning, unless writes other than lease writes and other erasures' renewals beat its renewal of the segment's
 row 8 times: it then keeps that object, one `rollback({ allowUnknown: true })` from being served, until a later call
 removes it: an erasure that rewrites the segment (the loser's re-run, while the loser's id is still in the current
-generation), an erasure of the winner's id, or the collection of loads that number past it ([how it stays correct](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/erasure.md#how-it-stays-correct)). **Read the ledger**: an `erased: false` entry means *this run* did not erase the id from that segment,
+generation), an erasure of the winner's id, or the collection of loads that number past it ([how it stays correct](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/correctness.md#how-an-erasure-stays-correct)). **Read the ledger**: an `erased: false` entry means *this run* did not erase the id from that segment,
 which is not the same as the id still being present — if the racing writer was another erasure of the same id, it
 is already gone. For `'superseded'` re-run `eraseSubject`: it is idempotent, it erases the id if the id is still
 there, and a segment the id is no longer in is simply not listed — though "not listed" alone does not prove the
@@ -131,7 +133,7 @@ same one, a generation still holding the id — can also come from the collect-o
 only outside the current generation and nothing was published; there the pointer has not moved, and a re-run
 simply repeats the attempt.) **Read what the re-run says.** Usually it reports `erased: true` against the
 generation it found the id in, which is the receipt the failed call could not give you. If a racing collector took
-that generation first it reports nothing for the segment: the bit is gone, but no run holds a receipt for it, so
+that generation first, or this call's own delete of it landed after a fault the call could not settle, it reports nothing for the segment: the bit is gone, but no run holds a receipt for it, so
 keep the failed call's error alongside your ledger. And if the segment's registry row has since been purged, it is
 not a segment any more and `eraseSubject` does not scan it at all — anything left in its bucket is an **orphan**,
 which neither `store.checkConsistency()` (it visits registry rows only) nor the collection a load and the retention
@@ -146,7 +148,7 @@ object is refused at its publish and never names an object that is gone. Each de
 read: on a storage driver that reports `conditionalDelete`, another erasure of the id that deletes it first, followed
 by a load that takes the number freed and publishes, does not lose that load's generation to this erasure's delete; on
 one that does not, it can, and the row then names a missing object, which `store.checkConsistency()` reports
-([a number taken again](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/erasure.md#how-it-stays-correct)). On an encrypted store such an object cannot
+([a number taken again](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/correctness.md#how-an-erasure-stays-correct)). On an encrypted store such an object cannot
 be searched until its load publishes the key, so it is deleted whatever id is erased: an erasure of any id refuses an
 encrypted first load in flight that has written its object onto a segment whose row was made ahead of its data, and
 that load is re-run ([the erasure guide](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/erasure.md#three-rules-while-you-erase)). On an encrypted segment,
@@ -180,7 +182,7 @@ bucket.
 | storage | on return — the generation holding it is deleted, or the wrapped DEK is removed from the segment's current registry row (a crypto-shred is complete once no retained copy of that row holds it, or every KEK that wrapped it is destroyed; see *Erasure vs. backups / WORM* below) |
 | the store whose verb made the call (`eraseSubject`, `dropSegment`, `retireExpired`) | on return, for every read that starts after it — it invalidates what it cached, and its pins then fail; a read already in progress there can still yield it from a chunk it had requested before |
 | another store, with a registry and a `cache.genTtlMs` above 0 | within `cache.genTtlMs` (default 2 s), when its snapshot re-resolves, **while the registry can be read** (see below) |
-| another store with **no registry** (a bare `IStorageDriver`), with `cache: { genTtlMs: 0 }`, or on a storage source built with **no clock** | **no bound** — only when its caches happen to let the segment go, or something tells it |
+| another store with **no registry** (a bare `IStorageDriver`), with `cache: { genTtlMs: 0 }`, or on a `StorageChunkSource` of your own, which the store times no refresh on | **no bound** — only when its caches happen to let the segment go, or something tells it |
 | `seg.pinAt` of a generation the erasure rewrote, in any store | on return it fails with `NotFoundError`, since the object is deleted (and, with a registry, the row has moved on); during the erasure, between the rewrite's publish and its delete of the old generation, it can still open that generation, and a handle opened then is a pin, covered by the rows for the store whose verb made the call and for a pinned handle in another store |
 | a `materializeMany` call with a feed, and an erasure by any path but this store's `eraseSubject` | **no bound**: the call is refused only by this store's `eraseSubject` |
 | a pinned handle (`seg.pin()`) in another store | **no bound** — until that store's reader cache evicts the pin's reader (in a store with a timed pointer refresh, a small generation's reader holds all of its chunks, decoded or not) and its chunk cache evicts the chunks the pin decoded, or something tells it |
@@ -229,7 +231,7 @@ registry row is a random holder id, a generation number and an instant, with no
 
 **Your exit path** (and a building block for a **data-portability / Art. 20** response): `store.exportSegments(sink,
 { format })` (and the `export-segments` CLI) dumps the current generation of every registered segment that is not
-destroyed (crypto-shredded or dropped) to a portable
+destroyed (crypto-shredded, destroyed in the clear or dropped) to a portable
 file — `roaring` (loadable by any roaring library) or `ndjson` (newline ids) — readable without CloudBitmaps. A destroyed segment resolves no generation, so it is not exported: the manifest's `skipped` lists it, and `segments`, `failed` and `skipped` together account for every segment row the registry listed.
 It's a **controller-side bulk dump** (all segments, opaque ids), not a per-subject deliverable — the per-subject
 rights are `subjectReport` (Art. 15) / `eraseSubject` (Art. 17). Encrypted segments are decrypted transparently
@@ -360,7 +362,9 @@ So the two are complements, not alternatives: **`dropSegment` for "stop paying f
 them included), its timestamps and its token. It holds no id, and on an encrypted segment no wrapped key, since the
 drop shreds it. The due index holds a pointer to it whose name spells out the same namespace and name. Both stay
 for `tombstoneGraceMs` (24 h by default) after the retirement, and for as long as the segment's storage cannot be
-proven gone. Then the sweep's purge removes the row, and the pointers it read to it, from the bucket, where the registry
+proven gone, except a segment that held no generation, whose row the same pass deletes (with `purgeTombstones` on, the default; from the bucket where the
+registry reports `conditionalDelete`; elsewhere as a deleted marker that keeps the same fields), so it never waits out
+the grace. Then the sweep's purge removes the row, and the pointers it read to it, from the bucket, where the registry
 reports `conditionalDelete`: the default for S3 when its client sends to an AWS S3 host,
 for Azure Blob, the local filesystem and memory. **A pointer can outlive its row**, and with it the name in its key: a
 purge that ran with another `tombstoneGraceMs` than the sweep that filed the pointer, a delete that landed and lost its
@@ -369,13 +373,15 @@ response, a removal the registry refused (`purgeFaults` counts it), or a pointer
 it reads, and an unscoped fleet scan, one given no `namespace`, from every day. A deployment that scopes every sweep to a
 namespace, or runs only index scans, keeps such a pointer until an unscoped fleet scan runs. Where the registry does not
 report `conditionalDelete` (an S3 client that sends to an S3-compatible store or an emulator, or a GCS client, by default, or
-`conditionalDelete: false`), the purge leaves a tombstone instead: every one of those fields, in the bucket, indefinitely, read by every full listing. A registry that does not
-report it files no pointer to begin with. A removed
+`conditionalDelete: false`), the purge leaves a tombstone instead: every one of those fields, in the bucket, indefinitely, read by every full listing. Such a
+registry files no pointer for a retirement's purge, but `setRetention` files its expiry-day pointer on every registry,
+and removing it there (at the retirement, a `clearRetention`, or a moved expiry) leaves a deleted marker under the
+reserved `cbm.due.` namespace whose key spells out the segment's namespace and name, also indefinitely. A removed
 row stays recoverable wherever the storage keeps a copy of it: with object versioning on, its earlier versions stay until
 a noncurrent-version rule expires them, as an overwritten row's do; with soft delete on, it stays for the retention
 window (GCS: on by default for a new bucket, 7 days; Azure Blob: where enabled, for the days set), whatever the registry
-did. A tombstone a hand-run `dropSegment` or a
-crypto-shred left is never purged by the library.
+did. A [tombstone a hand-run `dropSegment`, a
+crypto-shred or a cleartext destroy left is never purged by the library](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/correctness.md#tombstones-purged-narrowly).
 
 Full detail, including the dated-bucket pattern and the pitfalls, is in the retention section of the
 [retention guide](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/retention.md).
@@ -422,13 +428,13 @@ rewrite: `fromGeneration` → `generation`), `segment.collect` (a subject erasur
 no reader resolves: the generations holding the id, when it found the id only outside the current generation, or, with
 no `fromGeneration`, only objects no read of the segment can open, which it deletes whatever the id), `segment.erase`
 (a genuine
-crypto-shred), `segment.dispose` (a `dropSegment`, including every retirement the sweep performs) and
+crypto-shred), `segment.tombstone` (a cleartext tombstone from `allowCleartext`, which deletes nothing), `segment.dispose` (a `dropSegment`, including every retirement the sweep performs) and
 `namespace.erase` — for your audit log / SIEM. Each `segment.*` event names the segment's `incarnation` when its
 registry token carries one, so a segment created again under the same name is told apart from the one before.
 It is off by default and exception-safe. See the [dashboards guide](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/dashboards.md), which also says
 which event is the receipt for which claim. An erasure *ledger* — per-subject-request proof of physical deletion
-— is returned by **`eraseSubject`**. `subjectReport` is the read side and returns only which segments an id is
-in; it performs no deletion and issues no ledger.
+— is returned by **`eraseSubject`**. `subjectReport` is the read side and returns which segments an id is
+in, and which destroyed segments' leftover objects hold it; it performs no deletion and issues no ledger.
 
 **Log hygiene:** `segment` / `namespace` are *your* strings and may encode sensitive purpose; IDs are personal
 data. The library never logs bitmap contents or raw IDs, but **you** should treat segment names and IDs as PII
@@ -437,6 +443,15 @@ segment names for sensitive segments (keeping the human label in your own classi
 address, free text, or anything else an upstream system hands you. That makes this advice stronger rather than
 weaker: the library will store what you name, and will not sanitise it for you.
 
+The library's own error messages name the segment and namespace; some messages also give a chunk key (the high 16 bits
+of the ids in that chunk), id counts, metadata keys, key ids, the bucket or container name, or the account host, and an
+object's key (the store's `prefix`, the namespace and the encoded segment name); an error the cloud SDK raised reaches
+you as the SDK wrote it, except that the S3, GCS and Azure Blob drivers first remove the credentials they recognise in
+it (the request and response it kept, and a credential the service quotes back or its text quotes), and a credential
+in another form reaches you as the SDK wrote it; and `eraseSubject` copies each message into its ledger's `note`.
+Treat an error message, and the ledger, as carrying those names wherever you report or keep them.
+
+
 ## Shared-responsibility matrix
 
 | Concern | CloudBitmaps provides | You (the integrator) must |

package/README.md

@@ -17,7 +17,16 @@ pnpm add @cloudbitmaps/s3                   # the storage you have: or @cloudbit
 pnpm add @cloudbitmaps/tools                # optional: price a store, offline
 ```
 
-On npm 12 and pnpm 10 and later, allow `roaring`'s one install script first, or the install exits 0 and the package throws at `import`. Put this in your `package.json`: `{ "allowScripts": { "roaring": true }, "pnpm": { "onlyBuiltDependencies": ["roaring"] } }`. npm 11 runs the script but warns until you allow it the same way; pnpm 9 needs nothing extra. [Details](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/getting-started.md#cannot-find-module-buildreleaseroaringnode-after-a-successful-install).
+Install the same version of every `@cloudbitmaps/*` package: they release together, and each pins the exact version of `core` it was built with.
+
+`roaring` needs its one install script to fetch its native binary, and newer clients run a dependency's script only if you allow it. Allow it before you install:
+
+- **npm 12:** add `{ "allowScripts": { "roaring": true } }` to your `package.json`. Without it the install exits 0 and the package throws at `import`. npm 11 runs the script but warns until you allow it the same way.
+- **pnpm 11 and later:** put `allowBuilds: { roaring: true }` in a `pnpm-workspace.yaml` beside your `package.json`. Those versions no longer read the `package.json` `pnpm` key. Without it the install exits 1 with `ERR_PNPM_IGNORED_BUILDS`.
+- **pnpm 10:** add `{ "pnpm": { "onlyBuiltDependencies": ["roaring"] } }` to your `package.json`. Without it the install exits 0 and the package throws at `import`.
+- **pnpm 9:** nothing extra.
+
+[Details](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/getting-started.md#cannot-find-module-buildreleaseroaringnode-after-a-successful-install).
 
 ## Use
 
@@ -51,7 +60,7 @@ To persist, pass a different **backend** as `storage`. Nothing else changes:
 There is no `add` or `remove`: a segment changes only by loading a new generation. A load is a batch job, not a
 request handler. `store.exists(ref)` says whether a segment has been loaded. A load takes ids, or a bitmap you already
 hold: `store.load(ref, { bitmap })` for a `RoaringBitmap32`, or `{ serialized }` for portable Roaring bytes, checked
-before anything is written and loaded with no per-id work, into the same bytes its ids would write.
+before anything is written and loaded with no per-id work, into a generation holding chunk for chunk what its ids would write (its footer carries its own write identity).
 
 To write many results over the same stored operands, `store.materializeMany` reads each operand once for a whole batch of
 `and` / `or` / `andNot` outputs instead of once per `*Into` call, and a set you already hold in memory streams in as its
@@ -71,14 +80,14 @@ name.
 | `retry` | tune the retry of reads, or `false` to turn it off |
 | `metrics` | a metrics sink; off by default |
 | `budget` | `{ maxRequests }` or `false`: the per-call request ceiling |
-| `seams` | `{ clock, rng }`: determinism, for tests |
+| `seams` | `{ clock, rng, entropy }`: determinism, for tests and replayable jobs; the source of each generation's write identity is the last |
 
 ## What it costs
 
-A cold `intersect` of two 500,000-id segments sharing 100 of their 1,999 chunks made 6 GETs and took 99.80 ms at the median, **$2.40 per
-million** at list prices: measured on S3 in-region, in the run of 2026-10-06, whose client had 128 sockets. On a spread layout it reads more
-bytes than it needs, which cross-region can cost more than the requests it saves. A segment's first `store.load()` made
-2 PUT + 3 GET, **$11.20 per million** at the default prices, pointer included: the requests are measured on S3 in-region, in the same run. A steady `store.load()` at `keep: 12` that deletes a generation by name costs **$12.00 per million**, for 2 PUT + 5 GET and a free delete, counted by the engine's tests. A cold `count()` is one
+A cold `intersect` of two 500,000-id segments sharing 100 of their 1,999 chunks made 6 GETs and took 104.88 ms at the median, **$2.40 per
+million** at list prices: measured on S3 in-region, in the run of 2026-10-13, whose client had 128 sockets. On a spread layout it reads more
+bytes than it needs, which cross-region can cost more than the requests it saves. A segment's first single-part `store.load()` made
+2 PUT + 3 GET, **$11.20 per million** at the default prices, pointer included: the requests are measured on S3 in-region, in the same run. A steady single-part `store.load()` at `keep: 12` that deletes a generation by name costs **$12.00 per million**, for 2 PUT + 5 GET and a free delete, the requests measured on S3 in-region, in the same run. A cold `count()` is one
 pointer read: it reads no payload and no object.
 
 The trade is stated plainly. A membership check that misses the cache costs a ranged GET against object storage,
@@ -89,8 +98,16 @@ give the method and what the numbers do not establish. To price your own store,
 
 ## Before production
 
-- Grant the permissions the library issues and add a lifecycle rule to abort incomplete uploads.
-- Use a bucket that honors conditional writes, and `@aws-sdk/client-s3` 3.700.0 or later if you pass your own S3 client.
+- Run `await store.preflight()` once before the first load. It reads the registry, writes, tries to write again, reads, lists and
+  deletes one probe object and checks the backend's lifecycle rules, and answers `{ ok, checks }`; see [`store.preflight()`](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/production.md#check-the-setup-with-storepreflight).
+  It does not check `s3:AbortMultipartUpload`, the registry's `If-Match`, or a read after a delete, among others: [the full list](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/production.md#what-preflight-does-not-check).
+- Grant the permissions the library issues and add a lifecycle rule to abort incomplete uploads. The S3 and GCS
+  backends' `checkLifecycle()` read the bucket's rules and say whether one would delete or archive what the store
+  keeps; Azure Blob's answers `unchecked` and says how to check by hand; `MemoryStorage` and `LocalFsStorage` have no
+  rules and answer `ok`.
+- Use a bucket that honors conditional writes and is strongly consistent (a read or a listing made after a write or a
+  delete sees it), and `@aws-sdk/client-s3` 3.700.0 or later if you pass your own S3 client. Preflight reports a second write that succeeds (which is how an SDK of 3.640.0 or older shows) and a read or listing
+  that lags a write, but not an SDK from 3.641.0 to 3.699.0, which keeps `If-None-Match` but drops `If-Match`: the SDK floor stays yours to check.
 - Set a request timeout on your storage client. Reads retry by themselves, and a read can be timed on every backend (`readTimeoutMs`); a write that fails transiently is yours to re-run.
 - Back up the registry with the data, and the keystore too if you encrypt. If you lose the key, the data is gone.
 - Schedule your loads, and `retireExpired` if you use retention. Run loads off the request path.
@@ -115,6 +132,7 @@ says what else differs.
 - [API reference](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/api-reference.md)
 - [Changelog](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/CHANGELOG.md)
 - [Privacy and shared responsibility](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/PRIVACY.md)
+
 ## License
 
 Apache-2.0