@cloudbitmaps/roaring 0.15.0 → 0.17.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/PRIVACY.md CHANGED
@@ -136,7 +136,9 @@ bucket.
136
136
  | the store whose verb made the call (`eraseSubject`, `dropSegment`, `retireExpired`) | on return, for every read that starts after it — it invalidates what it cached, and its pins then fail; a read already in progress there can still yield it from a chunk it had requested before |
137
137
  | another store, with a registry and a `cache.genTtlMs` above 0 | within `cache.genTtlMs` (default 2 s), when its snapshot re-resolves, **while the registry can be read** (see below) |
138
138
  | another store with **no registry** (a bare `IStorageDriver`), with `cache: { genTtlMs: 0 }`, or on a storage source built with **no clock** | **no bound** — only when its caches happen to let the segment go, or something tells it |
139
+ | `seg.pinAt` of a generation the erasure rewrote, in any store | on return it fails with `NotFoundError`, since the object is deleted (and, with a registry, the row has moved on); during the erasure, between the rewrite's publish and its delete of the old generation, it can still open that generation, and a handle opened then is a pin, covered by the row below |
139
140
  | a pinned handle (`seg.pin()`) in another store | **no bound** — until that store's reader cache evicts the pin's reader (in a store with a timed pointer refresh, a small generation's reader holds all of its chunks, decoded or not) and its chunk cache evicts the chunks the pin decoded, or something tells it |
141
+ | a leased pinned handle (`seg.pin({ leaseUntil })`) in another store | until its lease ends, at most 14 days after it was taken: its reads then throw `LeaseExpiredError`, whether or not its reader still holds the chunks. Until then, as for any pinned handle in the row above |
140
142
 
141
143
  **An outage of the registry extends that bound.** A refresh that cannot read the row because of a transient fault
142
144
  (throttling, a 5xx, a dropped connection) keeps serving the generation the reader already holds, and the key it
@@ -161,6 +163,20 @@ yours.
161
163
  A read served from a stale cache is bounded by the same window and answers from memory, so nothing on the
162
164
  storage side — a bucket policy, a lifecycle rule, the object's own deletion — can shorten it.
163
165
 
166
+ **A lease keeps superseded generations, and erasure ignores it.** A pinned handle taken with `pin({ leaseUntil })` keeps
167
+ its generation out of a load's collection until the lease ends: at most 14 days after it was taken, and a 60-second margin
168
+ after that. While it does, the bucket holds a generation that a newer load has superseded, including ids that load
169
+ removed. `eraseSubject`, `eraseIdFromSegment`, `dropSegment`, `destroySegment` and a retention expiry never read a lease:
170
+ they delete the generations they must, a leased one included, so a lease never holds an erased subject's data past the
171
+ bounds above, and erasure, shred, drop and retention always win over it. A rewrite, a shred and a drop clear the row's
172
+ leases; an erasure that deletes a generation without rewriting leaves an entry for it in the row, which spares nothing and
173
+ is pruned by the next publish or lease write after it ends. A lease write does not delay an erasure: a row that differs
174
+ from the one the erasure read only in its leases does not refuse it. What a lease holds against is an ordinary load's
175
+ collection, and the generation is collected by the next listing pass after the lease ends: within 16 later loads of the
176
+ segment, and a segment that is never loaded again keeps it until an erasure, a drop or a retention expiry takes it. A
177
+ leased handle stops answering when its lease ends, and its reads then throw `LeaseExpiredError`. The lease entry in the
178
+ registry row is a random holder id, a generation number and an instant, with no id and no count.
179
+
164
180
  **Your exit path** (and a building block for a **data-portability / Art. 20** response): `store.exportSegments(sink,
165
181
  { format })` (and the `export-segments` CLI) dumps every registered segment's current generation to a portable
166
182
  file — `roaring` (loadable by any roaring library) or `ndjson` (newline ids) — readable without CloudBitmaps.
package/README.md CHANGED
@@ -69,11 +69,10 @@ name.
69
69
 
70
70
  ## What it costs
71
71
 
72
- A cold `intersect` of two 500,000-id segments sharing 100 of their 1,999 chunks is expected to make 6 GETs, **$2.40 per
73
- million**: counted by running the engine and priced at list, not yet measured on a cloud. On a spread layout it reads more
74
- bytes than it needs, which cross-region can cost more than the requests it saves. A segment's first `store.load()` is
75
- **$11.60 per million**, pointer included, from its measured requests at list prices (run of 2026-10-04, in-region, whose
76
- client had 50 sockets; the client this release builds allows 128, and its effect is not measured). A cold `count()` is one
72
+ A cold `intersect` of two 500,000-id segments sharing 100 of their 1,999 chunks made 6 GETs and took 92.63 ms at the median, **$2.40 per
73
+ million** at list prices: measured on S3 in-region, in the run of 2026-10-05, whose client had 128 sockets. On a spread layout it reads more
74
+ bytes than it needs, which cross-region can cost more than the requests it saves. A segment's first `store.load()` made
75
+ 2 PUT + 3 GET, **$11.20 per million** at the default prices, pointer included: the requests are measured on S3 in-region, in the same run. A cold `count()` is one
77
76
  pointer read: it reads no payload and no object.
78
77
 
79
78
  The trade is stated plainly. A membership check that misses the cache costs a ranged GET against object storage,