@cloudbitmaps/s3 0.10.0 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,41 +1,23 @@
1
1
  # @cloudbitmaps/s3
2
2
 
3
- **S3 and S3-compatible object storage for [CloudBitmaps](https://github.com/cloudbitmaps/cloudbitmaps).**
3
+ **S3 storage for [CloudBitmaps](https://github.com/cloudbitmaps/cloudbitmaps).** One bucket holds your generations and their pointers, so there is no second
4
+ service to run.
4
5
 
5
- > **ESM-only, Node ≥ 22.12.** This package ships as ES modules; there is no CommonJS bundle.
6
- > `require()` works on Node 22.12+ through `require(esm)`, but a runner with its own CommonJS loader
7
- > (notably Jest in its default configuration) does not get that and needs `import` instead. On TypeScript,
8
- > a CommonJS project needs `"module": "nodenext"` or `"node20"`. See the
9
- > [repository README](https://github.com/cloudbitmaps/cloudbitmaps#install--entry-points) for the details.
6
+ > **ESM-only, Node ≥ 22.12.** Use `import`; for `require()`, Jest and TypeScript, see
7
+ > [CommonJS, Jest and TypeScript](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/getting-started.md#commonjs-jest-and-typescript).
10
8
 
11
- AWS S3 — and every S3-compatible service: Cloudflare R2, MinIO, Ceph, Wasabi, Backblaze B2.
9
+ > Pre-1.0: the API and the on-disk format can still change. These docs describe `main`, ahead of the npm release.
12
10
 
13
11
  ## Install
14
12
 
15
13
  ```bash
16
14
  pnpm add @cloudbitmaps/roaring @cloudbitmaps/s3
17
- # npm i @cloudbitmaps/roaring @cloudbitmaps/s3 # the same, with npm
18
15
  ```
19
16
 
20
- > **On pnpm 10+, allow the one build script.** pnpm 10 skips dependency build scripts by default, so
21
- > the `roaring` native addon never downloads and the package throws at `import` — while the install
22
- > itself prints a warning and **exits 0**. Add this to your `package.json`, then install:
23
- >
24
- > ```json
25
- > { "pnpm": { "onlyBuiltDependencies": ["roaring"] } }
26
- > ```
27
- >
28
- > pnpm 9 and npm run it already. [Full symptoms and fixes](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/getting-started.md#cannot-find-module-buildreleaseroaringnode-after-a-successful-install).
29
-
30
- Two packages: the **codec** you want and the **storage** you have. `@aws-sdk/client-s3` (`>=3.645.0 <4`) is a real dependency of
31
- this package, so installing it is the whole step — there is no optional peer to remember. `@cloudbitmaps/core` is one
32
- too, so the engine lands in your tree without you installing it — you never name it yourself.
33
-
34
- > **The `>=3.645.0` floor is a correctness floor, not a preference.** Below it the SDK does not model the
35
- > conditional write this library's write-once guarantee is built on: measured against MinIO, **3.640.0
36
- > silently overwrites** an existing object instead of refusing, which loses a published generation without
37
- > an error; 3.641.0 rejects correctly. If you pin `@aws-sdk/client-s3` yourself, raise the pin to at least
38
- > `3.645.0` — an older pin will fail to resolve against this package rather than quietly downgrading you.
17
+ On pnpm 10 and later, allow the one build script first, or the package throws at `import` while the install exits 0. Put this in your `package.json`: `{ "pnpm": { "onlyBuiltDependencies": ["roaring"] } }`. npm and pnpm 9 need nothing extra. [Details](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/getting-started.md#cannot-find-module-buildreleaseroaringnode-after-a-successful-install).
18
+
19
+ `@aws-sdk/client-s3` (`>=3.645.0 <4`) is a real dependency of this package, so installing it is the whole step.
20
+ `@cloudbitmaps/core`, the engine underneath, arrives with it and you never name it.
39
21
 
40
22
  ## Use
41
23
 
@@ -48,26 +30,43 @@ const store = new CloudRoaring({
48
30
  });
49
31
 
50
32
  await store.load({ segment: 'active-users' }, [1, 2, 3]);
51
- const seg = store.segment('active-users');
52
- await seg.has(2); // true
33
+ await store.segment('active-users').has(2); // true
53
34
  ```
54
35
 
55
- `S3Storage` configures both halves — the immutable generation objects and the registry pointer row — from one set
56
- of values. Need them apart? This package also exports the two drivers and their option types; see the
57
- [API reference](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/api-reference.md).
58
-
59
- ## What this package is
60
-
61
- These drivers move opaque payload bytes, so they are codec-agnostic: the same package serves every codec
62
- flavor. That is why storage is a package rather than a subpath of one — as a subpath, each flavor needed a
63
- re-export barrel per service, and the count multiplied with every codec added.
64
-
65
- Built against `@cloudbitmaps/core/driver-kit`, the declared contract for a driver — the same surface a
66
- third-party driver would use.
36
+ It builds its own client from your usual AWS credentials. Any other key is refused by name.
37
+
38
+ | Option | What it does |
39
+ |---|---|
40
+ | `bucket` (required) | the bucket |
41
+ | `prefix` | a key prefix for everything this store writes |
42
+ | `client` | your own `S3Client`; it carries its own region, endpoint and credentials |
43
+ | `region`, `endpoint`, `pathStyle`, `credentials` | build a client for you, such as one for MinIO; refused beside `client` |
44
+ | `partBytes`, `maxObjectBytes` | multipart sizing: part size (default 8 MiB; the upload buffers one part at a time) and the largest object (about 80 GiB by default, up to S3's 5 TiB) |
45
+
46
+ ## Before production
47
+
48
+ - **The bucket must honor conditional writes**: `If-None-Match: *` for a write-once generation and `If-Match` for the
49
+ pointer. AWS S3 does, and the test suite runs against MinIO. Another S3-compatible service that ignores the headers
50
+ turns write-once into overwrite, so check yours before you depend on it.
51
+ - **Use `@aws-sdk/client-s3` 3.645.0 or later** if you pin the SDK or pass your own `client`. This floor is a
52
+ correctness floor, not a preference. The SDK models the conditional write this library's write-once guarantee is
53
+ built on from 3.641.0: measured against MinIO, 3.640.0 silently overwrites an existing object, which loses a
54
+ published generation without an error, and 3.641.0 rejects correctly. The floor sits a small margin above that.
55
+ This package's own range never resolves below it.
56
+ - **Grant `s3:GetObject`, `s3:PutObject`, `s3:DeleteObject`, `s3:AbortMultipartUpload` and `s3:ListBucket`.** Without
57
+ `s3:ListBucket`, S3 answers a missing key with `403` instead of `404`.
58
+ - **Add a lifecycle rule that aborts incomplete multipart uploads**, and never one that expires current objects or the
59
+ `registry/` prefix.
60
+ - **A transient failure of a write throws `TransientError`, and the write may or may not have landed.** Re-run the
61
+ call, or check `store.generations(ref)`. The SDK's own retry is off for conditional writes, so a write that landed
62
+ is not reported as a conflict.
63
+
64
+ The [production checklist](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/production.md) covers each item, with a sample IAM policy, and the ones every backend shares: a request timeout on your client, backups of the data and the registry, and a schedule for your loads.
67
65
 
68
66
  ## Documentation
69
67
 
70
68
  - [Getting started](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/getting-started.md)
69
+ - [Before production](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/production.md)
71
70
  - [API reference](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/docs/guide/api-reference.md)
72
71
  - [Changelog](https://github.com/cloudbitmaps/cloudbitmaps/blob/main/CHANGELOG.md)
73
72
 
@@ -0,0 +1,17 @@
1
+ /**
2
+ * SDK-free helpers for classifying AWS-style errors (used by the S3 drivers).
3
+ *
4
+ * These only read structural shapes an AWS SDK v3 error carries — `name`, `$metadata.httpStatusCode`, a
5
+ * lower-level `code`/`errno`, and the SDK's own `$retryable` marker — so the (subtle, easy-to-get-wrong)
6
+ * transient-vs-fatal decision is unit-testable without a live backend or even the SDK installed. They import no SDK.
7
+ */
8
+ export declare function httpStatus(err: unknown): number | undefined;
9
+ export declare function errorName(err: unknown): string | undefined;
10
+ /** A lower-level transport code (e.g. `ECONNRESET`) — the Node networking layer sets `code`. */
11
+ export declare function errorCode(err: unknown): string | undefined;
12
+ /** The AWS SDK v3 tags retryable errors with a `$retryable` object (throttling faults carry `.throttling`). */
13
+ export declare function isSdkRetryable(err: unknown): boolean;
14
+ /** Any 5xx is a server-side fault that's safe to retry (the request didn't deterministically fail). */
15
+ export declare function isServerSide(err: unknown): boolean;
16
+ /** A dropped/timed-out connection — transient by nature; a retry on a fresh connection usually succeeds. */
17
+ export declare function isNetworkOrTimeout(err: unknown): boolean;
package/dist/backend.d.ts CHANGED
@@ -10,7 +10,8 @@
10
10
  * ambient credential chain and region exactly as the SDK would. Pass `client` instead when you need a
11
11
  * credential chain the SDK cannot infer (SSO, an assumed role, a custom retry strategy); pass `endpoint` +
12
12
  * `pathStyle` + `credentials` for an S3-compatible store (MinIO, Ceph, R2). Both halves stay reachable as `.storage` and
13
- * `.registry` for anyone wiring something the facade does not cover.
13
+ * `.registry` for anyone wiring something the facade does not cover. `maxObjectBytes` and `partBytes` size the
14
+ * multipart upload.
14
15
  */
15
16
  import { STORAGE_BACKEND } from '@cloudbitmaps/core/driver-kit';
16
17
  import type { IRegistryDriver, IStorageDriver, StorageBackend } from '@cloudbitmaps/core/driver-kit';
@@ -20,13 +21,16 @@ export interface S3StorageOptions {
20
21
  readonly bucket: string;
21
22
  /** Optional key prefix under which everything lives — generations and the registry alike. */
22
23
  readonly prefix?: string;
23
- /** A constructed client. Supply one for a credential chain the SDK cannot infer; otherwise one is built. */
24
+ /**
25
+ * A constructed client. Supply one for a credential chain the SDK cannot infer; otherwise one is built. Its retry
26
+ * applies to every request except the conditional writes, which are sent once whatever it is configured to do.
27
+ */
24
28
  readonly client?: S3Client;
25
- /** Region for the client built when `client` is absent. Falls back to the SDK's own resolution. */
29
+ /** Region for the client built when `client` is absent (refused beside `client`). Falls back to the SDK's own resolution. */
26
30
  readonly region?: string;
27
- /** Endpoint for an S3-compatible store (MinIO, Ceph, R2). Ignored when `client` is supplied. */
31
+ /** Endpoint for an S3-compatible store (MinIO, Ceph, R2). Refused beside `client`, which carries its own. */
28
32
  readonly endpoint?: string;
29
- /** Path-style addressing, which most S3-compatible stores require. Ignored when `client` is supplied. */
33
+ /** Path-style addressing, which most S3-compatible stores require. Refused beside `client`, which carries its own. */
30
34
  readonly pathStyle?: boolean;
31
35
  /**
32
36
  * Static credentials, for the S3-compatible stores that issue them (MinIO, Ceph, R2).
@@ -34,16 +38,32 @@ export interface S3StorageOptions {
34
38
  * On AWS itself, leave this unset — the SDK's own chain (instance role, SSO, environment, profile) is what
35
39
  * you want, and hard-coding keys to reach it would be a downgrade. It exists because the alternative for a
36
40
  * MinIO user was to construct an `S3Client` purely to carry two strings, which is the ergonomics this class
37
- * is here to remove. Ignored when `client` is supplied.
41
+ * is here to remove. Refused beside `client`, which carries its own.
38
42
  */
39
43
  readonly credentials?: {
40
44
  readonly accessKeyId: string;
41
45
  readonly secretAccessKey: string;
42
46
  readonly sessionToken?: string;
43
47
  };
48
+ /**
49
+ * Largest object the backend will write and advertise. Default = `partBytes × 10,000` (≈ 80 GiB at the default
50
+ * 8 MiB part) — the honest ceiling reachable within S3's 10,000-part limit. Set it higher and `partBytes`
51
+ * auto-grows so 10,000 parts still cover it (raising peak write memory to ~one part); up to the 5 TiB S3 max.
52
+ * Must be a positive safe integer.
53
+ */
54
+ readonly maxObjectBytes?: number;
55
+ /** Multipart part size in bytes (default 8 MiB; a smaller value is raised to the S3 5 MiB minimum). Must be a
56
+ * positive safe integer. Tunes peak write memory. */
57
+ readonly partBytes?: number;
44
58
  /** Injected clock for the registry's `createdAt`/`updatedAt`; defaults to `Date.now`. */
45
59
  readonly now?: () => number;
46
60
  }
61
+ /**
62
+ * The keys `new S3Storage(options)` takes. Any other is refused by name rather than ignored: an ignored client or
63
+ * endpoint key builds a client from ambient credentials against the **public** endpoint, and for a store pointed at
64
+ * MinIO that is production traffic from a wiring typo.
65
+ */
66
+ export declare const S3_STORAGE_OPTION_KEYS: readonly ["bucket", "prefix", "client", "region", "endpoint", "pathStyle", "credentials", "maxObjectBytes", "partBytes", "now"];
47
67
  export declare class S3Storage implements StorageBackend {
48
68
  /** Cross-bundle brand, stamped non-enumerably in the constructor so a spread cannot carry it. */
49
69
  readonly [STORAGE_BACKEND]: true;
package/dist/index.d.ts CHANGED
@@ -12,17 +12,13 @@
12
12
  * ```
13
13
  *
14
14
  * These drivers move opaque payload bytes, so they are codec-agnostic: the same package serves every
15
- * flavor. That is why they are a package rather than a subpath of one — as a subpath, each flavor needed a
16
- * re-export barrel per service, and the count multiplied with every new codec.
15
+ * flavor. That is why they are a package rather than a subpath of one — as a subpath, each flavor would need a
16
+ * re-export barrel per service, and the count would multiply with every new codec.
17
17
  *
18
18
  * The engine, the `.crbm` format and the ports live in `@cloudbitmaps/core`, which is a real dependency of
19
19
  * THIS package: it lands in your tree without you installing it, and you never name it yourself. What this
20
20
  * package builds against is `@cloudbitmaps/core/driver-kit`, the declared contract for a driver — the same
21
21
  * surface a third-party driver would use.
22
22
  */
23
- export { S3StorageDriver } from './storage.js';
24
- export type { S3StorageDriverOptions } from './storage.js';
25
- export { S3RegistryDriver } from './registry.js';
26
- export type { S3RegistryDriverOptions } from './registry.js';
27
23
  export { S3Storage } from './backend.js';
28
24
  export type { S3StorageOptions } from './backend.js';
package/dist/index.js CHANGED
@@ -1,3 +1,7 @@
1
+ // src/backend.ts
2
+ import { ValidationError as ValidationError3, brandAsBackend } from "@cloudbitmaps/core/driver-kit";
3
+ import { S3Client } from "@aws-sdk/client-s3";
4
+
1
5
  // src/storage.ts
2
6
  import {
3
7
  NotFoundError,
@@ -30,12 +34,6 @@ import {
30
34
  validateSegmentRef
31
35
  } from "@cloudbitmaps/core/driver-kit";
32
36
  import { normalizeObjectPrefix } from "@cloudbitmaps/core/driver-kit";
33
- import {
34
- registryPrefix,
35
- registryObjectKey,
36
- registryListPrefix,
37
- parseRegistryKey
38
- } from "@cloudbitmaps/core/driver-kit";
39
37
  var SUFFIX = ".crbm";
40
38
  function segmentObjectPrefix(prefix, ref) {
41
39
  validateSegmentRef(ref);
@@ -55,14 +53,47 @@ function parseGenerationFromKey(segmentPrefix, objectKey) {
55
53
  return Number.isSafeInteger(generation) ? generation : null;
56
54
  }
57
55
 
56
+ // src/aws-errors.ts
57
+ function httpStatus(err) {
58
+ return err?.$metadata?.httpStatusCode;
59
+ }
60
+ function errorName(err) {
61
+ return err?.name;
62
+ }
63
+ function errorCode(err) {
64
+ return err?.code;
65
+ }
66
+ function isSdkRetryable(err) {
67
+ return err?.$retryable != null;
68
+ }
69
+ function isServerSide(err) {
70
+ const status = httpStatus(err);
71
+ return status !== void 0 && status >= 500 && status <= 599;
72
+ }
73
+ var NETWORK_NAMES = /* @__PURE__ */ new Set([
74
+ "TimeoutError",
75
+ "RequestTimeout",
76
+ "RequestTimeoutException",
77
+ "NetworkingError",
78
+ "AbortError"
79
+ ]);
80
+ var NETWORK_CODES = /* @__PURE__ */ new Set([
81
+ "ETIMEDOUT",
82
+ "ECONNRESET",
83
+ "ECONNREFUSED",
84
+ "EPIPE",
85
+ "ENOTFOUND",
86
+ "EAI_AGAIN",
87
+ "ECONNABORTED"
88
+ ]);
89
+ var NETWORK_MESSAGE = /socket hang up|network (error|failure)|(connection|request|socket|operation|read|write)\s+tim(e|ed)\s?out|connection (reset|refused|aborted|closed)/i;
90
+ function isNetworkOrTimeout(err) {
91
+ if (NETWORK_NAMES.has(errorName(err) ?? "")) return true;
92
+ if (NETWORK_CODES.has(errorCode(err) ?? "")) return true;
93
+ return NETWORK_MESSAGE.test(err?.message ?? "");
94
+ }
95
+
58
96
  // src/s3-errors.ts
59
- import {
60
- errorName,
61
- httpStatus,
62
- isNetworkOrTimeout,
63
- isSdkRetryable,
64
- isServerSide
65
- } from "@cloudbitmaps/core/driver-kit";
66
97
  function isPreconditionFailed(err) {
67
98
  return errorName(err) === "PreconditionFailed" || httpStatus(err) === 412;
68
99
  }
@@ -76,9 +107,13 @@ function isNotFound(err) {
76
107
  function isInvalidRange(err) {
77
108
  return errorName(err) === "InvalidRange" || httpStatus(err) === 416;
78
109
  }
110
+ function isClockSkewCorrected(err) {
111
+ const e = err;
112
+ return e?.$metadata?.clockSkewCorrected === true;
113
+ }
79
114
  function isTransient(err) {
80
115
  if (isConditionalConflict(err) || isNotFound(err) || isInvalidRange(err)) return false;
81
- return errorName(err) === "SlowDown" || isServerSide(err) || isNetworkOrTimeout(err) || isSdkRetryable(err);
116
+ return errorName(err) === "SlowDown" || isServerSide(err) || isNetworkOrTimeout(err) || isSdkRetryable(err) || isClockSkewCorrected(err);
82
117
  }
83
118
  function totalFromContentRange(contentRange) {
84
119
  if (contentRange === void 0) return void 0;
@@ -88,6 +123,18 @@ function totalFromContentRange(contentRange) {
88
123
  return Number.isSafeInteger(total) ? total : void 0;
89
124
  }
90
125
 
126
+ // src/send-once.ts
127
+ var NO_RETRY = {
128
+ name: "retryMiddleware",
129
+ step: "finalizeRequest",
130
+ priority: "high",
131
+ override: true
132
+ };
133
+ function sendOnce(client, command) {
134
+ command.middlewareStack.add((next) => next, NO_RETRY);
135
+ return client.send(command, {});
136
+ }
137
+
91
138
  // src/storage.ts
92
139
  var S3_PART_BYTES = 8 * 1024 * 1024;
93
140
  var S3_MAX_PARTS = 1e4;
@@ -101,6 +148,14 @@ var S3StorageDriver = class {
101
148
  this.client = options.client;
102
149
  this.bucket = options.bucket;
103
150
  this.prefix = normalizeObjectPrefix(options.prefix);
151
+ for (const [name, value] of [
152
+ ["partBytes", options.partBytes],
153
+ ["maxObjectBytes", options.maxObjectBytes]
154
+ ]) {
155
+ if (value !== void 0 && (!Number.isSafeInteger(value) || value < 1)) {
156
+ throw new ValidationError2(`${name} must be a positive safe integer; got ${value}`);
157
+ }
158
+ }
104
159
  const requestedPart = Math.max(options.partBytes ?? S3_PART_BYTES, 5 * 1024 * 1024);
105
160
  this.maxObjectBytes = options.maxObjectBytes ?? requestedPart * S3_MAX_PARTS;
106
161
  this.partBytes = Math.max(requestedPart, Math.ceil(this.maxObjectBytes / S3_MAX_PARTS));
@@ -238,9 +293,10 @@ var S3StorageDriver = class {
238
293
  return this.mapError(err);
239
294
  }
240
295
  /**
241
- * Reclassify a transient S3 fault (throttle/5xx/dropped connection) as a retryable {@link TransientError}
242
- * so the retry decorator can ride it out; everything else propagates unchanged. The final fallback at every
243
- * `client.send` site, so callers and the decorator only ever see typed errors.
296
+ * Reclassify a transient S3 fault (throttle/5xx/dropped connection) as a retryable {@link TransientError},
297
+ * so the store's read retry can ride it out and a write's caller can tell it from a deterministic failure;
298
+ * everything else propagates unchanged. The final fallback at every `client.send` site, so callers and the
299
+ * read retry only ever see typed errors.
244
300
  */
245
301
  mapError(err) {
246
302
  if (isTransient(err)) {
@@ -325,7 +381,8 @@ var S3MultipartSink = class {
325
381
  async finish() {
326
382
  const sha256 = this.hash.digest("hex");
327
383
  if (this.uploadId === void 0) {
328
- await this.client.send(
384
+ await sendOnce(
385
+ this.client,
329
386
  new PutObjectCommand({
330
387
  Bucket: this.bucket,
331
388
  Key: this.objectKey,
@@ -337,7 +394,8 @@ var S3MultipartSink = class {
337
394
  return { size: this.total, sha256 };
338
395
  }
339
396
  if (this.pendingLen > 0) await this.flushPart();
340
- await this.client.send(
397
+ await sendOnce(
398
+ this.client,
341
399
  new CompleteMultipartUploadCommand({
342
400
  Bucket: this.bucket,
343
401
  Key: this.objectKey,
@@ -414,7 +472,8 @@ var S3Store = class {
414
472
  }
415
473
  async write(key, body, expect) {
416
474
  try {
417
- await this.client.send(
475
+ await sendOnce(
476
+ this.client,
418
477
  new PutObjectCommand2({
419
478
  Bucket: this.bucket,
420
479
  Key: key,
@@ -473,26 +532,66 @@ var S3RegistryDriver = class extends ObjectStoreRegistry {
473
532
  };
474
533
 
475
534
  // src/backend.ts
476
- import { brandAsBackend } from "@cloudbitmaps/core/driver-kit";
477
- import { S3Client } from "@aws-sdk/client-s3";
535
+ var S3_STORAGE_OPTION_KEYS = [
536
+ "bucket",
537
+ "prefix",
538
+ "client",
539
+ "region",
540
+ "endpoint",
541
+ "pathStyle",
542
+ "credentials",
543
+ "maxObjectBytes",
544
+ "partBytes",
545
+ "now"
546
+ ];
547
+ var CLIENT_SETTINGS = ["region", "endpoint", "pathStyle", "credentials"];
548
+ function refuseUnknown(name, options, keys, hint) {
549
+ if (options === null || typeof options !== "object") {
550
+ throw new ValidationError3(
551
+ `${name} needs an options object \u2014 got ${options === null ? "null" : typeof options}`
552
+ );
553
+ }
554
+ const unknown = Object.keys(options).filter((k) => !keys.includes(k));
555
+ if (unknown.length > 0) {
556
+ const list = (ks) => ks.map((k) => `\`${k}\``).join(", ");
557
+ throw new ValidationError3(
558
+ `${name} does not take ${list(unknown)}. It takes ${list(keys)}; ${hint}.`
559
+ );
560
+ }
561
+ }
478
562
  var S3Storage = class {
479
563
  storage;
480
564
  registry;
481
565
  /** The client both halves share — built here unless one was supplied. */
482
566
  client;
483
567
  constructor(options) {
484
- this.client = options.client ?? new S3Client({
485
- ...options.region === void 0 ? {} : { region: options.region },
486
- ...options.endpoint === void 0 ? {} : { endpoint: options.endpoint },
487
- ...options.pathStyle === void 0 ? {} : { forcePathStyle: options.pathStyle },
488
- ...options.credentials === void 0 ? {} : { credentials: options.credentials }
489
- });
568
+ refuseUnknown("S3Storage", options, S3_STORAGE_OPTION_KEYS, "an S3 client goes in `client`");
569
+ if (options.client !== void 0 && options.client !== null) {
570
+ const ignored = CLIENT_SETTINGS.filter((k) => options[k] !== void 0);
571
+ if (ignored.length > 0) {
572
+ throw new ValidationError3(
573
+ `S3Storage takes \`client\` OR ${CLIENT_SETTINGS.map((k) => `\`${k}\``).join(" / ")}, not both \u2014 got \`client\` with ${ignored.map((k) => `\`${k}\``).join(", ")}; the \`client\` already carries them, so configure them on the client, or drop \`client\``
574
+ );
575
+ }
576
+ this.client = options.client;
577
+ } else {
578
+ this.client = new S3Client({
579
+ ...options.region === void 0 ? {} : { region: options.region },
580
+ ...options.endpoint === void 0 ? {} : { endpoint: options.endpoint },
581
+ ...options.pathStyle === void 0 ? {} : { forcePathStyle: options.pathStyle },
582
+ ...options.credentials === void 0 ? {} : { credentials: options.credentials }
583
+ });
584
+ }
490
585
  const shared = {
491
586
  client: this.client,
492
587
  bucket: options.bucket,
493
588
  ...options.prefix === void 0 ? {} : { prefix: options.prefix }
494
589
  };
495
- this.storage = new S3StorageDriver(shared);
590
+ this.storage = new S3StorageDriver({
591
+ ...shared,
592
+ ...options.maxObjectBytes === void 0 ? {} : { maxObjectBytes: options.maxObjectBytes },
593
+ ...options.partBytes === void 0 ? {} : { partBytes: options.partBytes }
594
+ });
496
595
  this.registry = new S3RegistryDriver({
497
596
  ...shared,
498
597
  ...options.now === void 0 ? {} : { now: options.now }
@@ -501,8 +600,6 @@ var S3Storage = class {
501
600
  }
502
601
  };
503
602
  export {
504
- S3RegistryDriver,
505
- S3Storage,
506
- S3StorageDriver
603
+ S3Storage
507
604
  };
508
605
  //# sourceMappingURL=index.js.map
package/dist/index.js.map CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "version": 3,
3
- "sources": ["../src/storage.ts", "../src/keys.ts", "../src/s3-errors.ts", "../src/registry.ts", "../src/backend.ts"],
4
- "sourcesContent": ["/**\n * `S3StorageDriver` \u2014 an {@link IStorageDriver} over S3-compatible object storage.\n *\n * Works with AWS S3 and any compatible backend (MinIO, etc.) via the official `@aws-sdk/client-s3`, a real\n * dependency of this package \u2014 installing `@cloudbitmaps/s3` is what installs it. The client is\n * **injected** (dependency injection): the driver owns no credential/region/endpoint logic, so it's thin,\n * testable against MinIO (point a client at its endpoint), and reuses the caller's existing client.\n *\n * Generations are write-once immutable objects: a conditional `PutObject` with `If-None-Match: *` makes the\n * publish atomic \u2014 a second write to the same key fails with `WriteConflictError`, never a silent overwrite\n * (hard invariant 2: storage objects are immutable and never overwritten in place), the cloud analogue of\n * the LocalFs atomic `link`. **This requires a backend that honors\n * `If-None-Match: *`** (AWS S3 \u2014 GA Aug 2024; recent MinIO): a backend that silently ignored the\n * precondition would break write-once immutability. **Writes stream:** the object is uploaded in\n * constant memory \u2014 a small object is a single conditional `PutObject`; a large one is a **multipart upload**\n * (parts flushed as the codec writes, freed as they go) finished with a conditional `CompleteMultipartUpload`,\n * so a load's footprint stays ~one part regardless of segment size, up to the advertised `maxObjectBytes`\n * (default `partBytes \u00D7 10,000` \u2014 S3's per-upload part limit). Drivers may use `node:crypto`; only `core/`\n * is bound by the determinism lint.\n */\nimport {\n NotFoundError,\n TransientError,\n ValidationError,\n WriteConflictError,\n isNotFoundError,\n isValidationError,\n isWriteConflictError,\n} from '@cloudbitmaps/core/driver-kit';\nimport type {\n BlobSink,\n GenKey,\n IStorageDriver,\n SegmentRef,\n StorageCaps,\n} from '@cloudbitmaps/core/driver-kit';\nimport { createHash, type Hash } from 'node:crypto';\nimport {\n AbortMultipartUploadCommand,\n CompleteMultipartUploadCommand,\n CreateMultipartUploadCommand,\n DeleteObjectCommand,\n GetObjectCommand,\n HeadObjectCommand,\n ListObjectsV2Command,\n PutObjectCommand,\n UploadPartCommand,\n type S3Client,\n} from '@aws-sdk/client-s3';\nimport {\n storageObjectKey,\n normalizeS3Prefix,\n parseGenerationFromKey,\n segmentObjectPrefix,\n} from './keys';\nimport {\n isConditionalConflict,\n isInvalidRange,\n isNotFound,\n isTransient,\n totalFromContentRange,\n} from './s3-errors';\n\n/** Part size for multipart uploads. \u2265 the S3 5 MiB minimum; an object that fits in one part uses a single\n * conditional PUT instead (no multipart overhead, strongest write-once). Peak write memory \u2248 one part. */\nconst S3_PART_BYTES = 8 * 1024 * 1024;\n/** S3 hard limit: a multipart upload has at most 10,000 parts. This \u00D7 the part size is the real object ceiling. */\nconst S3_MAX_PARTS = 10_000;\n\nexport interface S3StorageDriverOptions {\n /** A constructed S3 client (point its `endpoint` at MinIO for local/integration use). */\n readonly client: S3Client;\n /** Target bucket (must already exist). */\n readonly bucket: string;\n /** Optional key prefix under which all objects live (e.g. `cloudroaring/`). */\n readonly prefix?: string;\n /**\n * Largest object this driver will write/advertise. Default = `partBytes \u00D7 10,000` (\u2248 80 GiB at the default\n * 8 MiB part) \u2014 the honest ceiling reachable within S3's 10,000-part limit. Set it higher and `partBytes`\n * auto-grows so 10,000 parts still cover it (raising peak write memory to ~one part); up to the 5 TiB S3 max.\n */\n readonly maxObjectBytes?: number;\n /** Multipart part size in bytes (default 8 MiB; clamped to the S3 5 MiB minimum). Tunes peak write memory. */\n readonly partBytes?: number;\n}\n\nexport class S3StorageDriver implements IStorageDriver {\n private readonly client: S3Client;\n private readonly bucket: string;\n private readonly prefix: string | undefined;\n private readonly maxObjectBytes: number;\n private readonly partBytes: number;\n\n constructor(options: S3StorageDriverOptions) {\n this.client = options.client;\n this.bucket = options.bucket;\n this.prefix = normalizeS3Prefix(options.prefix);\n const requestedPart = Math.max(options.partBytes ?? S3_PART_BYTES, 5 * 1024 * 1024);\n // Default the object cap to what the requested part size can actually cover within S3's 10,000-part limit;\n // if a larger cap is requested, grow the part size to keep it reachable (so the advertised cap is honest).\n this.maxObjectBytes = options.maxObjectBytes ?? requestedPart * S3_MAX_PARTS;\n this.partBytes = Math.max(requestedPart, Math.ceil(this.maxObjectBytes / S3_MAX_PARTS));\n }\n\n capabilities(): StorageCaps {\n return { rangeRead: true, maxObjectBytes: this.maxObjectBytes, conditionalPut: true };\n }\n\n async putImmutable(\n key: GenKey,\n write: (sink: BlobSink) => Promise<void>,\n ): Promise<{ size: number; sha256: string }> {\n const objectKey = storageObjectKey(this.prefix, key); // validates ref + generation\n const sink = new S3MultipartSink(\n this.client,\n this.bucket,\n objectKey,\n this.partBytes,\n this.maxObjectBytes,\n );\n try {\n await write(sink);\n return await sink.finish();\n } catch (err) {\n await sink.abort(); // best-effort cleanup of any in-flight multipart upload\n // A lost conditional-write race \u2014 the precondition failed (412) or S3 rejected concurrent conditional\n // writes to the key (409) \u2014 is the write-once conflict, never a silent overwrite.\n if (isConditionalConflict(err)) {\n throw new WriteConflictError(\n `generation already exists (write-once): ${key.segment}.${key.generation}`,\n );\n }\n if (isValidationError(err) || isWriteConflictError(err) || isNotFoundError(err)) {\n throw err;\n }\n throw this.mapError(err);\n }\n }\n\n async getRange(key: GenKey, offset: number, length: number): Promise<Uint8Array> {\n if (!Number.isInteger(offset) || !Number.isInteger(length) || offset < 0 || length < 0) {\n throw new ValidationError(`invalid range offset=${offset} length=${length}`);\n }\n const objectKey = storageObjectKey(this.prefix, key);\n if (length === 0) return new Uint8Array(0);\n try {\n const res = await this.client.send(\n new GetObjectCommand({\n Bucket: this.bucket,\n Key: objectKey,\n Range: `bytes=${offset}-${offset + length - 1}`,\n }),\n );\n const bytes = await collect(res.Body);\n // A short read means the range ran past EOF \u2014 treat as out-of-bounds, never a partial result.\n if (bytes.length !== length) {\n throw new ValidationError(\n `range [${offset}, ${offset + length}) out of bounds (got ${bytes.length}B)`,\n );\n }\n return bytes;\n } catch (err) {\n throw this.mapReadError(err, key);\n }\n }\n\n async getTail(key: GenKey, maxBytes: number): Promise<{ bytes: Uint8Array; size: number }> {\n const objectKey = storageObjectKey(this.prefix, key);\n if (maxBytes <= 0) {\n // No tail bytes wanted \u2014 just resolve the size via a HEAD.\n try {\n const head = await this.client.send(\n new HeadObjectCommand({ Bucket: this.bucket, Key: objectKey }),\n );\n return { bytes: new Uint8Array(0), size: head.ContentLength ?? 0 };\n } catch (err) {\n throw this.mapReadError(err, key);\n }\n }\n try {\n const res = await this.client.send(\n new GetObjectCommand({ Bucket: this.bucket, Key: objectKey, Range: `bytes=-${maxBytes}` }),\n );\n const bytes = await collect(res.Body);\n let size = totalFromContentRange(res.ContentRange);\n if (size === undefined) {\n // A spec-compliant backend omits Content-Range only on a 200 (whole object), where bytes.length\n // IS the size. If the body is exactly maxBytes we can't rule out a clamped partial from a\n // non-compliant backend \u2014 confirm the true size with a HEAD rather than trust a possibly-short read.\n if (bytes.length === maxBytes) {\n const head = await this.client.send(\n new HeadObjectCommand({ Bucket: this.bucket, Key: objectKey }),\n );\n size = head.ContentLength ?? bytes.length;\n } else {\n size = bytes.length;\n }\n }\n return { bytes, size };\n } catch (err) {\n throw this.mapReadError(err, key);\n }\n }\n\n async delete(key: GenKey): Promise<void> {\n // Idempotent: S3 DeleteObject succeeds even if the key is absent (GC may race / retry).\n try {\n await this.client.send(\n new DeleteObjectCommand({ Bucket: this.bucket, Key: storageObjectKey(this.prefix, key) }),\n );\n } catch (err) {\n throw this.mapError(err);\n }\n }\n\n async *list(ref: SegmentRef): AsyncIterable<GenKey> {\n const prefix = segmentObjectPrefix(this.prefix, ref); // validates ref\n let token: string | undefined;\n do {\n let res;\n try {\n res = await this.client.send(\n new ListObjectsV2Command({\n Bucket: this.bucket,\n Prefix: prefix,\n ContinuationToken: token,\n }),\n );\n } catch (err) {\n throw this.mapError(err);\n }\n for (const obj of res.Contents ?? []) {\n if (obj.Key === undefined) continue;\n const generation = parseGenerationFromKey(prefix, obj.Key);\n if (generation !== null) {\n yield { namespace: ref.namespace, segment: ref.segment, generation };\n }\n }\n token = res.IsTruncated === true ? res.NextContinuationToken : undefined;\n } while (token !== undefined);\n }\n\n /** Map S3 read errors to the driver vocabulary; pass everything else through {@link mapError}. */\n private mapReadError(err: unknown, key: GenKey): unknown {\n if (isValidationError(err)) return err;\n if (isNotFound(err)) {\n return new NotFoundError(`no such generation: ${key.segment}.${key.generation}`);\n }\n // A fully out-of-range request (start past EOF) \u2014 the BlobReader contract treats range errors as\n // ValidationError, never a short/empty read.\n if (isInvalidRange(err)) {\n return new ValidationError(`range out of bounds for ${key.segment}.${key.generation}`);\n }\n return this.mapError(err);\n }\n\n /**\n * Reclassify a transient S3 fault (throttle/5xx/dropped connection) as a retryable {@link TransientError}\n * so the retry decorator can ride it out; everything else propagates unchanged. The final fallback at every\n * `client.send` site, so callers and the decorator only ever see typed errors.\n */\n private mapError(err: unknown): unknown {\n if (isTransient(err)) {\n return new TransientError(\n `transient S3 fault: ${(err as { name?: string } | null)?.name ?? 'unknown'}`,\n { cause: err },\n );\n }\n return err;\n }\n}\n\n/** Concatenate a list of byte chunks of known total length into one buffer. */\nfunction concatBytes(parts: readonly Uint8Array[], total: number): Uint8Array {\n const out = new Uint8Array(total);\n let offset = 0;\n for (const p of parts) {\n out.set(p, offset);\n offset += p.length;\n }\n return out;\n}\n\n/**\n * Streaming {@link BlobSink} that uploads one S3 object in **constant memory**. It buffers at most\n * one part: as the codec writes, full parts are flushed via `UploadPart` and freed. A small object that never\n * reaches one part is committed as a single conditional `PutObject`; a larger one is finished with a\n * conditional `CompleteMultipartUpload` \u2014 **both enforce write-once** via `If-None-Match: *`. SHA-256 is hashed\n * incrementally. On any error the caller invokes {@link abort} to clean up the in-flight multipart upload.\n */\nclass S3MultipartSink implements BlobSink {\n private readonly hash: Hash = createHash('sha256');\n private readonly pending: Uint8Array[] = [];\n private pendingLen = 0;\n private total = 0;\n private uploadId: string | undefined;\n private partNumber = 0;\n private readonly parts: { ETag: string | undefined; PartNumber: number }[] = [];\n\n constructor(\n private readonly client: S3Client,\n private readonly bucket: string,\n private readonly objectKey: string,\n private readonly partBytes: number,\n private readonly maxObjectBytes: number,\n ) {}\n\n async write(bytes: Uint8Array): Promise<void> {\n if (bytes.length === 0) return;\n this.total += bytes.length;\n if (this.total > this.maxObjectBytes) {\n // Fail fast + typed, rather than a late opaque S3 error (and abort the in-flight upload via the caller).\n throw new ValidationError(`object exceeds maxObjectBytes ${this.maxObjectBytes}`);\n }\n this.hash.update(bytes);\n this.pending.push(bytes);\n this.pendingLen += bytes.length;\n if (this.pendingLen >= this.partBytes) await this.flushPart();\n }\n\n /** Upload the buffered bytes (\u2265 one part) as a single part, freeing them. Starts the upload on first call. */\n private async flushPart(): Promise<void> {\n if (this.uploadId === undefined) {\n const res = await this.client.send(\n new CreateMultipartUploadCommand({ Bucket: this.bucket, Key: this.objectKey }),\n );\n if (res.UploadId === undefined) {\n throw new TransientError('S3 CreateMultipartUpload returned no UploadId');\n }\n this.uploadId = res.UploadId;\n }\n const body = concatBytes(this.pending, this.pendingLen);\n this.pending.length = 0;\n this.pendingLen = 0;\n this.partNumber += 1;\n if (this.partNumber > S3_MAX_PARTS) {\n // Unreachable for valid input (the maxObjectBytes byte-cap, sized to \u2264 S3_MAX_PARTS parts, fires first) \u2014\n // a typed guard so the S3 hard limit is never a raw 400.\n throw new ValidationError(`multipart upload exceeded the S3 ${S3_MAX_PARTS}-part limit`);\n }\n const res = await this.client.send(\n new UploadPartCommand({\n Bucket: this.bucket,\n Key: this.objectKey,\n UploadId: this.uploadId,\n PartNumber: this.partNumber,\n Body: body,\n }),\n );\n this.parts.push({ ETag: res.ETag, PartNumber: this.partNumber });\n }\n\n /** Commit the object: a single conditional PUT if it fit in one part, else complete the multipart upload. */\n async finish(): Promise<{ size: number; sha256: string }> {\n const sha256 = this.hash.digest('hex');\n if (this.uploadId === undefined) {\n await this.client.send(\n new PutObjectCommand({\n Bucket: this.bucket,\n Key: this.objectKey,\n Body: concatBytes(this.pending, this.pendingLen),\n IfNoneMatch: '*', // write-once\n }),\n );\n return { size: this.total, sha256 };\n }\n if (this.pendingLen > 0) await this.flushPart(); // the final part may be < partBytes (allowed)\n await this.client.send(\n new CompleteMultipartUploadCommand({\n Bucket: this.bucket,\n Key: this.objectKey,\n UploadId: this.uploadId,\n MultipartUpload: { Parts: this.parts },\n IfNoneMatch: '*', // write-once: fail if the object already exists\n }),\n );\n this.uploadId = undefined; // completed \u2014 nothing left to abort\n return { size: this.total, sha256 };\n }\n\n /** Best-effort cleanup of an in-flight multipart upload after an error (a leaked MPU is reaped by a bucket\n * lifecycle rule; never a correctness issue). No-op if nothing was started or it already completed. */\n async abort(): Promise<void> {\n if (this.uploadId === undefined) return;\n const id = this.uploadId;\n this.uploadId = undefined;\n try {\n await this.client.send(\n new AbortMultipartUploadCommand({ Bucket: this.bucket, Key: this.objectKey, UploadId: id }),\n );\n } catch {\n // swallow \u2014 best-effort\n }\n }\n}\n\n/**\n * Collect an S3 response body into a `Uint8Array`. `transformToByteArray` is added at runtime to the SDK's\n * Node stream by `@aws-sdk`'s sdk-stream-mixin, so the structural cast is sound on Node.\n */\nasync function collect(body: GetObjectCommandBody): Promise<Uint8Array> {\n if (body === undefined) {\n throw new NotFoundError('S3 GetObject returned an empty body');\n }\n return body.transformToByteArray();\n}\n\n/** The S3 `GetObject` Body type, narrowed to the part we use (`transformToByteArray`). */\ntype GetObjectCommandBody = { transformToByteArray(): Promise<Uint8Array> } | undefined;\n", "/**\n * Logical-ref \u2192 S3 object-key mapping for {@link S3StorageDriver}.\n *\n * Pure string logic with no SDK dependency, so it's unit-testable without S3/MinIO. Mirrors the LocalFs\n * layout (`<namespace>/segments/<segment>.<gen>.crbm`) under an optional caller prefix, and re-validates\n * names at the boundary \u2014 defense in depth, because a driver can be constructed and driven directly rather\n * than through the engine that would otherwise have validated for it. The default\n * (absent) namespace maps to `_default`, which cannot collide with a real namespace because a caller's\n * `_default` encodes to `%5Fdefault` while the sentinel is emitted literally.\n */\n// `prefixPart` is imported, never redefined: the storage and registry layouts sit under the SAME caller\n// prefix, so they must normalize it identically \u2014 a second copy of that three-line function is how the two\n// halves of one bucket drift apart.\nimport {\n ValidationError,\n encodeNameForKey,\n namespaceKeyPart,\n prefixPart,\n validateSegmentRef,\n} from '@cloudbitmaps/core/driver-kit';\nimport type { GenKey, SegmentRef } from '@cloudbitmaps/core/driver-kit';\n\nconst SUFFIX = '.crbm';\n\n/** Validate a caller-supplied key prefix. The rule is shared with every other object store. */\nexport { normalizeObjectPrefix as normalizeS3Prefix } from '@cloudbitmaps/core/driver-kit';\n\n/**\n * The S3 key prefix shared by all of a segment's generations: `<prefix><ns>/segments/<segment>.`. Used\n * both as the `ListObjectsV2` prefix and as the string stripped by {@link parseGenerationFromKey}.\n */\nexport function segmentObjectPrefix(prefix: string | undefined, ref: SegmentRef): string {\n validateSegmentRef(ref);\n return `${prefixPart(prefix)}${namespaceKeyPart(ref.namespace)}/segments/${encodeNameForKey(ref.segment)}.`;\n}\n\n/** The full S3 key of one `.crbm` generation: `<segmentPrefix><gen>.crbm`. */\nexport function storageObjectKey(prefix: string | undefined, key: GenKey): string {\n if (!Number.isInteger(key.generation) || key.generation < 0) {\n throw new ValidationError(`generation must be a non-negative integer; got ${key.generation}`);\n }\n return `${segmentObjectPrefix(prefix, key)}${key.generation}${SUFFIX}`;\n}\n\n// \u2500\u2500\u2500 Registry keys \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n// The registry layout is identical across S3, GCS and Azure Blob \u2014 all three encode names the same way and\n// build byte-identical keys \u2014 so it lives in `_shared/object-registry-keys` and is re-exported here for the\n// callers (and tests) that already name it through this module.\nexport {\n registryPrefix,\n registryObjectKey,\n registryListPrefix,\n parseRegistryKey,\n} from '@cloudbitmaps/core/driver-kit';\n\n/**\n * Parse a generation number out of a full object key, given its segment prefix, or `null` if it doesn't\n * match. Canonical decimal only \u2014 no leading zeros (so `\u2026s.07.crbm` can't alias `\u2026s.7.crbm`) and within\n * safe-integer range. This also rejects a *different* segment whose name merely shares the prefix (e.g. a\n * key for segment `s.x` won't parse under segment `s`'s prefix, since its middle isn't all digits).\n */\nexport function parseGenerationFromKey(segmentPrefix: string, objectKey: string): number | null {\n if (!objectKey.startsWith(segmentPrefix) || !objectKey.endsWith(SUFFIX)) return null;\n const middle = objectKey.slice(segmentPrefix.length, objectKey.length - SUFFIX.length);\n if (!/^(0|[1-9]\\d*)$/.test(middle)) return null;\n const generation = Number(middle);\n return Number.isSafeInteger(generation) ? generation : null;\n}\n", "/**\n * Pure helpers for classifying S3 SDK errors + parsing response headers (conflict and transient classification).\n *\n * Kept SDK-free and side-effect-free (they only read structural shapes \u2014 `err.name`,\n * `$metadata.httpStatusCode`, a `Content-Range` string) so the subtle S3-specific translation logic is\n * unit-testable without a live MinIO/S3 or even the AWS SDK. Shared AWS shapes come from `_shared/aws-errors`.\n */\n\nimport {\n errorName,\n httpStatus,\n isNetworkOrTimeout,\n isSdkRetryable,\n isServerSide,\n} from '@cloudbitmaps/core/driver-kit';\n\n/** A conditional `If-None-Match: *` PUT lost the write-once race (the object already existed). */\nexport function isPreconditionFailed(err: unknown): boolean {\n return errorName(err) === 'PreconditionFailed' || httpStatus(err) === 412;\n}\n\n/**\n * A conditional write (`If-None-Match: *` / `If-Match: <etag>`) lost the race \u2014 **either** outcome S3 uses:\n * the precondition evaluated false (`412 PreconditionFailed`), **or** S3 rejected concurrent conditional\n * writes to the same key to prevent a lost update (`409 ConditionalRequestConflict`, which AWS documents and\n * asks you to retry). Both mean \"you lost; re-read and retry\" \u2014 so both must map to `WriteConflictError` and\n * route through the caller's OCC path, never a blind transient retry (which would just replay a doomed PUT).\n */\nexport function isConditionalConflict(err: unknown): boolean {\n return (\n isPreconditionFailed(err) ||\n errorName(err) === 'ConditionalRequestConflict' ||\n httpStatus(err) === 409\n );\n}\n\n/** The object / generation does not exist (GetObject \u2192 `NoSuchKey`, HeadObject \u2192 `NotFound`; both 404). */\nexport function isNotFound(err: unknown): boolean {\n const name = errorName(err);\n return name === 'NoSuchKey' || name === 'NotFound' || httpStatus(err) === 404;\n}\n\n/** A range request started past EOF (HTTP 416). */\nexport function isInvalidRange(err: unknown): boolean {\n return errorName(err) === 'InvalidRange' || httpStatus(err) === 416;\n}\n\n/**\n * A transient S3 fault that is safe to retry: throttling (`SlowDown` / 503), any 5xx, a dropped/timed-out\n * connection, or anything the SDK itself marks retryable. Excludes the deterministic outcomes above\n * (412/404/416) \u2014 those are caller-meaningful and must never be retried/reclassified.\n */\nexport function isTransient(err: unknown): boolean {\n // A conditional-write conflict (412/409) is caller-meaningful OCC, not a blind-retryable transient.\n if (isConditionalConflict(err) || isNotFound(err) || isInvalidRange(err)) return false;\n return (\n errorName(err) === 'SlowDown' ||\n isServerSide(err) ||\n isNetworkOrTimeout(err) ||\n isSdkRetryable(err)\n );\n}\n\n/**\n * Parse the total object size out of a `Content-Range: bytes <start>-<end>/<total>` header, or `undefined`\n * if absent/unparseable/unsafe. The total is the part after the final `/`.\n */\nexport function totalFromContentRange(contentRange: string | undefined): number | undefined {\n if (contentRange === undefined) return undefined;\n const match = /\\/(\\d+)\\s*$/.exec(contentRange);\n if (match === null) return undefined;\n const total = Number(match[1]);\n return Number.isSafeInteger(total) ? total : undefined;\n}\n", "/**\n * `S3RegistryDriver` \u2014 an {@link IRegistryDriver} over S3-compatible object storage.\n *\n * Lets a **read-mostly deployment run on S3 alone** \u2014 storage `.crbm` generations + the registry in one bucket,\n * no separate database. The protocol (an ABA-safe OCC counter, tombstoning delete, the bounded retry, the key layout)\n * lives once in {@link ObjectStoreRegistry}; this file is only the three I/O calls S3 makes, so the S3, GCS\n * and Azure registries cannot drift from one another.\n *\n * **The atomic swap is offloaded to S3's conditional writes** (GA Nov 2024): `If-None-Match: *` for\n * create-only and `If-Match: <etag>` for compare-and-swap, so a concurrent writer between our read and our\n * PUT loses with a `412` \u2192 {@link WriteConflictError}. Reads are strongly consistent (S3, since 2020),\n * satisfying the registry's `strongRead` contract. The client is **injected**, exactly like\n * {@link S3StorageDriver}.\n *\n * **Deployment requirements** (a backend/policy that violates these silently corrupts the registry):\n * - The backend **must honor `If-Match`** (AWS S3; recent MinIO). One that returns ETags but ignores the\n * precondition degrades compare-and-swap to last-write-wins \u2192 lost `currentGen` swaps. Verified against\n * real S3 semantics by the MinIO integration lane.\n * - The IAM principal needs **`s3:ListBucket`** on the bucket. Without it, `GetObject` on a missing key\n * returns `403` (not `404`), so the \"absent segment \u2192 `null`\" contract (and `create`'s bootstrap read)\n * breaks \u2014 and `list()` needs it regardless.\n * - **Do not apply an S3 lifecycle-expiration rule to the `registry/` prefix.** See {@link ObjectStoreRegistry}.\n */\nimport {\n IntegrityError,\n MAX_ROW_BYTES,\n ObjectStoreRegistry,\n TransientError,\n WriteConflictError,\n normalizeObjectPrefix,\n} from '@cloudbitmaps/core/driver-kit';\nimport type { ObjectRegistryStore, ObjectRow } from '@cloudbitmaps/core/driver-kit';\nimport {\n GetObjectCommand,\n ListObjectsV2Command,\n PutObjectCommand,\n type S3Client,\n} from '@aws-sdk/client-s3';\nimport { isConditionalConflict, isNotFound, isTransient } from './s3-errors';\n\nexport interface S3RegistryDriverOptions {\n /** A constructed S3 client (point its `endpoint` at MinIO for local/integration use). */\n readonly client: S3Client;\n /** Target bucket (must already exist). */\n readonly bucket: string;\n /** Optional key prefix under which all registry objects live (e.g. `cloudroaring/`). */\n readonly prefix?: string;\n /** Injected clock for `createdAt`/`updatedAt`; defaults to `Date.now`. */\n readonly now?: () => number;\n}\n\n/** The three calls {@link ObjectStoreRegistry} needs, in S3's dialect. */\nclass S3Store implements ObjectRegistryStore {\n readonly label = 'S3';\n\n constructor(\n private readonly client: S3Client,\n private readonly bucket: string,\n ) {}\n\n async read(key: string): Promise<ObjectRow | null> {\n let res;\n try {\n res = await this.client.send(new GetObjectCommand({ Bucket: this.bucket, Key: key }));\n } catch (err) {\n if (isNotFound(err)) return null;\n throw mapError(err);\n }\n // Check the advertised length BEFORE allocating, so a hostile object cannot make us buffer it first.\n if ((res.ContentLength ?? 0) > MAX_ROW_BYTES) {\n throw new IntegrityError(\n `registry object ${res.ContentLength}B exceeds cap ${MAX_ROW_BYTES}B`,\n );\n }\n if (res.Body === undefined) {\n throw new IntegrityError(`registry object has an empty body: ${key}`);\n }\n const bytes = await (\n res.Body as { transformToByteArray(): Promise<Uint8Array> }\n ).transformToByteArray();\n return { bytes, version: res.ETag ?? '' };\n }\n\n async write(\n key: string,\n body: Uint8Array,\n expect: 'absent' | { version: string },\n ): Promise<void> {\n try {\n await this.client.send(\n new PutObjectCommand({\n Bucket: this.bucket,\n Key: key,\n Body: body,\n ContentType: 'application/json',\n IfNoneMatch: expect === 'absent' ? '*' : undefined,\n IfMatch: expect === 'absent' ? undefined : expect.version,\n }),\n );\n } catch (err) {\n // A lost conditional-write race (412 precondition, or 409 concurrent-conflict) is an OCC conflict.\n if (isConditionalConflict(err)) {\n throw new WriteConflictError(`registry OCC conflict for ${key}`);\n }\n throw mapError(err);\n }\n }\n\n async *listKeys(prefix: string): AsyncIterable<string> {\n let token: string | undefined;\n do {\n let res;\n try {\n res = await this.client.send(\n new ListObjectsV2Command({\n Bucket: this.bucket,\n Prefix: prefix,\n ContinuationToken: token,\n }),\n );\n } catch (err) {\n throw mapError(err);\n }\n for (const obj of res.Contents ?? []) {\n if (obj.Key !== undefined) yield obj.Key;\n }\n token = res.IsTruncated === true ? res.NextContinuationToken : undefined;\n } while (token !== undefined);\n }\n}\n\n/** Reclassify a transient S3 fault as a retryable {@link TransientError}; pass everything else through. */\nfunction mapError(err: unknown): unknown {\n if (isTransient(err)) {\n return new TransientError(\n `transient S3 fault: ${(err as { name?: string } | null)?.name ?? 'unknown'}`,\n { cause: err },\n );\n }\n return err;\n}\n\nexport class S3RegistryDriver extends ObjectStoreRegistry {\n constructor(options: S3RegistryDriverOptions) {\n super(\n new S3Store(options.client, options.bucket),\n normalizeObjectPrefix(options.prefix),\n options.now ?? ((): number => Date.now()),\n );\n }\n}\n", "/**\n * `S3Storage` \u2014 the S3 backend as one object: the generations and the pointer, in one bucket, stated once.\n *\n * Replaces two constructor calls that each repeated `client`, `bucket` and `prefix`. Repeating them is how\n * they come apart: point the registry at one prefix and the objects at another and the store answers *empty*\n * rather than *misconfigured*, which is the hardest kind of wrong answer to debug. Here the location is written\n * once and shared, so the mismatch cannot be expressed.\n *\n * **It will build a client for you**, which is the common case \u2014 `new S3Storage({ bucket })` picks up the\n * ambient credential chain and region exactly as the SDK would. Pass `client` instead when you need a\n * credential chain the SDK cannot infer (SSO, an assumed role, a custom retry strategy); pass `endpoint` +\n * `pathStyle` + `credentials` for an S3-compatible store (MinIO, Ceph, R2). Both halves stay reachable as `.storage` and\n * `.registry` for anyone wiring something the facade does not cover.\n */\nimport { STORAGE_BACKEND, brandAsBackend } from '@cloudbitmaps/core/driver-kit';\nimport type {\n IRegistryDriver,\n IStorageDriver,\n StorageBackend,\n} from '@cloudbitmaps/core/driver-kit';\nimport { S3Client } from '@aws-sdk/client-s3';\nimport { S3StorageDriver } from './storage';\nimport { S3RegistryDriver } from './registry';\n\nexport interface S3StorageOptions {\n /** Target bucket (must already exist). */\n readonly bucket: string;\n /** Optional key prefix under which everything lives \u2014 generations and the registry alike. */\n readonly prefix?: string;\n /** A constructed client. Supply one for a credential chain the SDK cannot infer; otherwise one is built. */\n readonly client?: S3Client;\n /** Region for the client built when `client` is absent. Falls back to the SDK's own resolution. */\n readonly region?: string;\n /** Endpoint for an S3-compatible store (MinIO, Ceph, R2). Ignored when `client` is supplied. */\n readonly endpoint?: string;\n /** Path-style addressing, which most S3-compatible stores require. Ignored when `client` is supplied. */\n readonly pathStyle?: boolean;\n /**\n * Static credentials, for the S3-compatible stores that issue them (MinIO, Ceph, R2).\n *\n * On AWS itself, leave this unset \u2014 the SDK's own chain (instance role, SSO, environment, profile) is what\n * you want, and hard-coding keys to reach it would be a downgrade. It exists because the alternative for a\n * MinIO user was to construct an `S3Client` purely to carry two strings, which is the ergonomics this class\n * is here to remove. Ignored when `client` is supplied.\n */\n readonly credentials?: {\n readonly accessKeyId: string;\n readonly secretAccessKey: string;\n readonly sessionToken?: string;\n };\n /** Injected clock for the registry's `createdAt`/`updatedAt`; defaults to `Date.now`. */\n readonly now?: () => number;\n}\n\nexport class S3Storage implements StorageBackend {\n /** Cross-bundle brand, stamped non-enumerably in the constructor so a spread cannot carry it. */\n declare readonly [STORAGE_BACKEND]: true;\n readonly storage: IStorageDriver;\n readonly registry: IRegistryDriver;\n /** The client both halves share \u2014 built here unless one was supplied. */\n readonly client: S3Client;\n\n constructor(options: S3StorageOptions) {\n this.client =\n options.client ??\n new S3Client({\n ...(options.region === undefined ? {} : { region: options.region }),\n ...(options.endpoint === undefined ? {} : { endpoint: options.endpoint }),\n ...(options.pathStyle === undefined ? {} : { forcePathStyle: options.pathStyle }),\n ...(options.credentials === undefined ? {} : { credentials: options.credentials }),\n });\n const shared = {\n client: this.client,\n bucket: options.bucket,\n ...(options.prefix === undefined ? {} : { prefix: options.prefix }),\n };\n this.storage = new S3StorageDriver(shared);\n this.registry = new S3RegistryDriver({\n ...shared,\n ...(options.now === undefined ? {} : { now: options.now }),\n });\n brandAsBackend(this);\n }\n}\n"],
5
- "mappings": ";AAoBA;AAAA,EACE;AAAA,EACA;AAAA,EACA,mBAAAA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAQP,SAAS,kBAA6B;AACtC;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OAEK;;;ACnCP;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAMP,SAAkC,6BAAyB;AAuB3D;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AA/BP,IAAM,SAAS;AASR,SAAS,oBAAoB,QAA4B,KAAyB;AACvF,qBAAmB,GAAG;AACtB,SAAO,GAAG,WAAW,MAAM,CAAC,GAAG,iBAAiB,IAAI,SAAS,CAAC,aAAa,iBAAiB,IAAI,OAAO,CAAC;AAC1G;AAGO,SAAS,iBAAiB,QAA4B,KAAqB;AAChF,MAAI,CAAC,OAAO,UAAU,IAAI,UAAU,KAAK,IAAI,aAAa,GAAG;AAC3D,UAAM,IAAI,gBAAgB,kDAAkD,IAAI,UAAU,EAAE;AAAA,EAC9F;AACA,SAAO,GAAG,oBAAoB,QAAQ,GAAG,CAAC,GAAG,IAAI,UAAU,GAAG,MAAM;AACtE;AAmBO,SAAS,uBAAuB,eAAuB,WAAkC;AAC9F,MAAI,CAAC,UAAU,WAAW,aAAa,KAAK,CAAC,UAAU,SAAS,MAAM,EAAG,QAAO;AAChF,QAAM,SAAS,UAAU,MAAM,cAAc,QAAQ,UAAU,SAAS,OAAO,MAAM;AACrF,MAAI,CAAC,iBAAiB,KAAK,MAAM,EAAG,QAAO;AAC3C,QAAM,aAAa,OAAO,MAAM;AAChC,SAAO,OAAO,cAAc,UAAU,IAAI,aAAa;AACzD;;;AC3DA;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAGA,SAAS,qBAAqB,KAAuB;AAC1D,SAAO,UAAU,GAAG,MAAM,wBAAwB,WAAW,GAAG,MAAM;AACxE;AASO,SAAS,sBAAsB,KAAuB;AAC3D,SACE,qBAAqB,GAAG,KACxB,UAAU,GAAG,MAAM,gCACnB,WAAW,GAAG,MAAM;AAExB;AAGO,SAAS,WAAW,KAAuB;AAChD,QAAM,OAAO,UAAU,GAAG;AAC1B,SAAO,SAAS,eAAe,SAAS,cAAc,WAAW,GAAG,MAAM;AAC5E;AAGO,SAAS,eAAe,KAAuB;AACpD,SAAO,UAAU,GAAG,MAAM,kBAAkB,WAAW,GAAG,MAAM;AAClE;AAOO,SAAS,YAAY,KAAuB;AAEjD,MAAI,sBAAsB,GAAG,KAAK,WAAW,GAAG,KAAK,eAAe,GAAG,EAAG,QAAO;AACjF,SACE,UAAU,GAAG,MAAM,cACnB,aAAa,GAAG,KAChB,mBAAmB,GAAG,KACtB,eAAe,GAAG;AAEtB;AAMO,SAAS,sBAAsB,cAAsD;AAC1F,MAAI,iBAAiB,OAAW,QAAO;AACvC,QAAM,QAAQ,cAAc,KAAK,YAAY;AAC7C,MAAI,UAAU,KAAM,QAAO;AAC3B,QAAM,QAAQ,OAAO,MAAM,CAAC,CAAC;AAC7B,SAAO,OAAO,cAAc,KAAK,IAAI,QAAQ;AAC/C;;;AFRA,IAAM,gBAAgB,IAAI,OAAO;AAEjC,IAAM,eAAe;AAmBd,IAAM,kBAAN,MAAgD;AAAA,EACpC;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EAEjB,YAAY,SAAiC;AAC3C,SAAK,SAAS,QAAQ;AACtB,SAAK,SAAS,QAAQ;AACtB,SAAK,SAAS,sBAAkB,QAAQ,MAAM;AAC9C,UAAM,gBAAgB,KAAK,IAAI,QAAQ,aAAa,eAAe,IAAI,OAAO,IAAI;AAGlF,SAAK,iBAAiB,QAAQ,kBAAkB,gBAAgB;AAChE,SAAK,YAAY,KAAK,IAAI,eAAe,KAAK,KAAK,KAAK,iBAAiB,YAAY,CAAC;AAAA,EACxF;AAAA,EAEA,eAA4B;AAC1B,WAAO,EAAE,WAAW,MAAM,gBAAgB,KAAK,gBAAgB,gBAAgB,KAAK;AAAA,EACtF;AAAA,EAEA,MAAM,aACJ,KACA,OAC2C;AAC3C,UAAM,YAAY,iBAAiB,KAAK,QAAQ,GAAG;AACnD,UAAM,OAAO,IAAI;AAAA,MACf,KAAK;AAAA,MACL,KAAK;AAAA,MACL;AAAA,MACA,KAAK;AAAA,MACL,KAAK;AAAA,IACP;AACA,QAAI;AACF,YAAM,MAAM,IAAI;AAChB,aAAO,MAAM,KAAK,OAAO;AAAA,IAC3B,SAAS,KAAK;AACZ,YAAM,KAAK,MAAM;AAGjB,UAAI,sBAAsB,GAAG,GAAG;AAC9B,cAAM,IAAI;AAAA,UACR,2CAA2C,IAAI,OAAO,IAAI,IAAI,UAAU;AAAA,QAC1E;AAAA,MACF;AACA,UAAI,kBAAkB,GAAG,KAAK,qBAAqB,GAAG,KAAK,gBAAgB,GAAG,GAAG;AAC/E,cAAM;AAAA,MACR;AACA,YAAM,KAAK,SAAS,GAAG;AAAA,IACzB;AAAA,EACF;AAAA,EAEA,MAAM,SAAS,KAAa,QAAgB,QAAqC;AAC/E,QAAI,CAAC,OAAO,UAAU,MAAM,KAAK,CAAC,OAAO,UAAU,MAAM,KAAK,SAAS,KAAK,SAAS,GAAG;AACtF,YAAM,IAAIC,iBAAgB,wBAAwB,MAAM,WAAW,MAAM,EAAE;AAAA,IAC7E;AACA,UAAM,YAAY,iBAAiB,KAAK,QAAQ,GAAG;AACnD,QAAI,WAAW,EAAG,QAAO,IAAI,WAAW,CAAC;AACzC,QAAI;AACF,YAAM,MAAM,MAAM,KAAK,OAAO;AAAA,QAC5B,IAAI,iBAAiB;AAAA,UACnB,QAAQ,KAAK;AAAA,UACb,KAAK;AAAA,UACL,OAAO,SAAS,MAAM,IAAI,SAAS,SAAS,CAAC;AAAA,QAC/C,CAAC;AAAA,MACH;AACA,YAAM,QAAQ,MAAM,QAAQ,IAAI,IAAI;AAEpC,UAAI,MAAM,WAAW,QAAQ;AAC3B,cAAM,IAAIA;AAAA,UACR,UAAU,MAAM,KAAK,SAAS,MAAM,wBAAwB,MAAM,MAAM;AAAA,QAC1E;AAAA,MACF;AACA,aAAO;AAAA,IACT,SAAS,KAAK;AACZ,YAAM,KAAK,aAAa,KAAK,GAAG;AAAA,IAClC;AAAA,EACF;AAAA,EAEA,MAAM,QAAQ,KAAa,UAAgE;AACzF,UAAM,YAAY,iBAAiB,KAAK,QAAQ,GAAG;AACnD,QAAI,YAAY,GAAG;AAEjB,UAAI;AACF,cAAM,OAAO,MAAM,KAAK,OAAO;AAAA,UAC7B,IAAI,kBAAkB,EAAE,QAAQ,KAAK,QAAQ,KAAK,UAAU,CAAC;AAAA,QAC/D;AACA,eAAO,EAAE,OAAO,IAAI,WAAW,CAAC,GAAG,MAAM,KAAK,iBAAiB,EAAE;AAAA,MACnE,SAAS,KAAK;AACZ,cAAM,KAAK,aAAa,KAAK,GAAG;AAAA,MAClC;AAAA,IACF;AACA,QAAI;AACF,YAAM,MAAM,MAAM,KAAK,OAAO;AAAA,QAC5B,IAAI,iBAAiB,EAAE,QAAQ,KAAK,QAAQ,KAAK,WAAW,OAAO,UAAU,QAAQ,GAAG,CAAC;AAAA,MAC3F;AACA,YAAM,QAAQ,MAAM,QAAQ,IAAI,IAAI;AACpC,UAAI,OAAO,sBAAsB,IAAI,YAAY;AACjD,UAAI,SAAS,QAAW;AAItB,YAAI,MAAM,WAAW,UAAU;AAC7B,gBAAM,OAAO,MAAM,KAAK,OAAO;AAAA,YAC7B,IAAI,kBAAkB,EAAE,QAAQ,KAAK,QAAQ,KAAK,UAAU,CAAC;AAAA,UAC/D;AACA,iBAAO,KAAK,iBAAiB,MAAM;AAAA,QACrC,OAAO;AACL,iBAAO,MAAM;AAAA,QACf;AAAA,MACF;AACA,aAAO,EAAE,OAAO,KAAK;AAAA,IACvB,SAAS,KAAK;AACZ,YAAM,KAAK,aAAa,KAAK,GAAG;AAAA,IAClC;AAAA,EACF;AAAA,EAEA,MAAM,OAAO,KAA4B;AAEvC,QAAI;AACF,YAAM,KAAK,OAAO;AAAA,QAChB,IAAI,oBAAoB,EAAE,QAAQ,KAAK,QAAQ,KAAK,iBAAiB,KAAK,QAAQ,GAAG,EAAE,CAAC;AAAA,MAC1F;AAAA,IACF,SAAS,KAAK;AACZ,YAAM,KAAK,SAAS,GAAG;AAAA,IACzB;AAAA,EACF;AAAA,EAEA,OAAO,KAAK,KAAwC;AAClD,UAAM,SAAS,oBAAoB,KAAK,QAAQ,GAAG;AACnD,QAAI;AACJ,OAAG;AACD,UAAI;AACJ,UAAI;AACF,cAAM,MAAM,KAAK,OAAO;AAAA,UACtB,IAAI,qBAAqB;AAAA,YACvB,QAAQ,KAAK;AAAA,YACb,QAAQ;AAAA,YACR,mBAAmB;AAAA,UACrB,CAAC;AAAA,QACH;AAAA,MACF,SAAS,KAAK;AACZ,cAAM,KAAK,SAAS,GAAG;AAAA,MACzB;AACA,iBAAW,OAAO,IAAI,YAAY,CAAC,GAAG;AACpC,YAAI,IAAI,QAAQ,OAAW;AAC3B,cAAM,aAAa,uBAAuB,QAAQ,IAAI,GAAG;AACzD,YAAI,eAAe,MAAM;AACvB,gBAAM,EAAE,WAAW,IAAI,WAAW,SAAS,IAAI,SAAS,WAAW;AAAA,QACrE;AAAA,MACF;AACA,cAAQ,IAAI,gBAAgB,OAAO,IAAI,wBAAwB;AAAA,IACjE,SAAS,UAAU;AAAA,EACrB;AAAA;AAAA,EAGQ,aAAa,KAAc,KAAsB;AACvD,QAAI,kBAAkB,GAAG,EAAG,QAAO;AACnC,QAAI,WAAW,GAAG,GAAG;AACnB,aAAO,IAAI,cAAc,uBAAuB,IAAI,OAAO,IAAI,IAAI,UAAU,EAAE;AAAA,IACjF;AAGA,QAAI,eAAe,GAAG,GAAG;AACvB,aAAO,IAAIA,iBAAgB,2BAA2B,IAAI,OAAO,IAAI,IAAI,UAAU,EAAE;AAAA,IACvF;AACA,WAAO,KAAK,SAAS,GAAG;AAAA,EAC1B;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOQ,SAAS,KAAuB;AACtC,QAAI,YAAY,GAAG,GAAG;AACpB,aAAO,IAAI;AAAA,QACT,uBAAwB,KAAkC,QAAQ,SAAS;AAAA,QAC3E,EAAE,OAAO,IAAI;AAAA,MACf;AAAA,IACF;AACA,WAAO;AAAA,EACT;AACF;AAGA,SAAS,YAAY,OAA8B,OAA2B;AAC5E,QAAM,MAAM,IAAI,WAAW,KAAK;AAChC,MAAI,SAAS;AACb,aAAW,KAAK,OAAO;AACrB,QAAI,IAAI,GAAG,MAAM;AACjB,cAAU,EAAE;AAAA,EACd;AACA,SAAO;AACT;AASA,IAAM,kBAAN,MAA0C;AAAA,EASxC,YACmB,QACA,QACA,WACA,WACA,gBACjB;AALiB;AACA;AACA;AACA;AACA;AAAA,EAChB;AAAA,EALgB;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EAbF,OAAa,WAAW,QAAQ;AAAA,EAChC,UAAwB,CAAC;AAAA,EAClC,aAAa;AAAA,EACb,QAAQ;AAAA,EACR;AAAA,EACA,aAAa;AAAA,EACJ,QAA4D,CAAC;AAAA,EAU9E,MAAM,MAAM,OAAkC;AAC5C,QAAI,MAAM,WAAW,EAAG;AACxB,SAAK,SAAS,MAAM;AACpB,QAAI,KAAK,QAAQ,KAAK,gBAAgB;AAEpC,YAAM,IAAIA,iBAAgB,iCAAiC,KAAK,cAAc,EAAE;AAAA,IAClF;AACA,SAAK,KAAK,OAAO,KAAK;AACtB,SAAK,QAAQ,KAAK,KAAK;AACvB,SAAK,cAAc,MAAM;AACzB,QAAI,KAAK,cAAc,KAAK,UAAW,OAAM,KAAK,UAAU;AAAA,EAC9D;AAAA;AAAA,EAGA,MAAc,YAA2B;AACvC,QAAI,KAAK,aAAa,QAAW;AAC/B,YAAMC,OAAM,MAAM,KAAK,OAAO;AAAA,QAC5B,IAAI,6BAA6B,EAAE,QAAQ,KAAK,QAAQ,KAAK,KAAK,UAAU,CAAC;AAAA,MAC/E;AACA,UAAIA,KAAI,aAAa,QAAW;AAC9B,cAAM,IAAI,eAAe,+CAA+C;AAAA,MAC1E;AACA,WAAK,WAAWA,KAAI;AAAA,IACtB;AACA,UAAM,OAAO,YAAY,KAAK,SAAS,KAAK,UAAU;AACtD,SAAK,QAAQ,SAAS;AACtB,SAAK,aAAa;AAClB,SAAK,cAAc;AACnB,QAAI,KAAK,aAAa,cAAc;AAGlC,YAAM,IAAID,iBAAgB,oCAAoC,YAAY,aAAa;AAAA,IACzF;AACA,UAAM,MAAM,MAAM,KAAK,OAAO;AAAA,MAC5B,IAAI,kBAAkB;AAAA,QACpB,QAAQ,KAAK;AAAA,QACb,KAAK,KAAK;AAAA,QACV,UAAU,KAAK;AAAA,QACf,YAAY,KAAK;AAAA,QACjB,MAAM;AAAA,MACR,CAAC;AAAA,IACH;AACA,SAAK,MAAM,KAAK,EAAE,MAAM,IAAI,MAAM,YAAY,KAAK,WAAW,CAAC;AAAA,EACjE;AAAA;AAAA,EAGA,MAAM,SAAoD;AACxD,UAAM,SAAS,KAAK,KAAK,OAAO,KAAK;AACrC,QAAI,KAAK,aAAa,QAAW;AAC/B,YAAM,KAAK,OAAO;AAAA,QAChB,IAAI,iBAAiB;AAAA,UACnB,QAAQ,KAAK;AAAA,UACb,KAAK,KAAK;AAAA,UACV,MAAM,YAAY,KAAK,SAAS,KAAK,UAAU;AAAA,UAC/C,aAAa;AAAA;AAAA,QACf,CAAC;AAAA,MACH;AACA,aAAO,EAAE,MAAM,KAAK,OAAO,OAAO;AAAA,IACpC;AACA,QAAI,KAAK,aAAa,EAAG,OAAM,KAAK,UAAU;AAC9C,UAAM,KAAK,OAAO;AAAA,MAChB,IAAI,+BAA+B;AAAA,QACjC,QAAQ,KAAK;AAAA,QACb,KAAK,KAAK;AAAA,QACV,UAAU,KAAK;AAAA,QACf,iBAAiB,EAAE,OAAO,KAAK,MAAM;AAAA,QACrC,aAAa;AAAA;AAAA,MACf,CAAC;AAAA,IACH;AACA,SAAK,WAAW;AAChB,WAAO,EAAE,MAAM,KAAK,OAAO,OAAO;AAAA,EACpC;AAAA;AAAA;AAAA,EAIA,MAAM,QAAuB;AAC3B,QAAI,KAAK,aAAa,OAAW;AACjC,UAAM,KAAK,KAAK;AAChB,SAAK,WAAW;AAChB,QAAI;AACF,YAAM,KAAK,OAAO;AAAA,QAChB,IAAI,4BAA4B,EAAE,QAAQ,KAAK,QAAQ,KAAK,KAAK,WAAW,UAAU,GAAG,CAAC;AAAA,MAC5F;AAAA,IACF,QAAQ;AAAA,IAER;AAAA,EACF;AACF;AAMA,eAAe,QAAQ,MAAiD;AACtE,MAAI,SAAS,QAAW;AACtB,UAAM,IAAI,cAAc,qCAAqC;AAAA,EAC/D;AACA,SAAO,KAAK,qBAAqB;AACnC;;;AG9XA;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA,kBAAAE;AAAA,EACA,sBAAAC;AAAA,EACA,yBAAAC;AAAA,OACK;AAEP;AAAA,EACE,oBAAAC;AAAA,EACA,wBAAAC;AAAA,EACA,oBAAAC;AAAA,OAEK;AAeP,IAAM,UAAN,MAA6C;AAAA,EAG3C,YACmB,QACA,QACjB;AAFiB;AACA;AAAA,EAChB;AAAA,EAFgB;AAAA,EACA;AAAA,EAJV,QAAQ;AAAA,EAOjB,MAAM,KAAK,KAAwC;AACjD,QAAI;AACJ,QAAI;AACF,YAAM,MAAM,KAAK,OAAO,KAAK,IAAIC,kBAAiB,EAAE,QAAQ,KAAK,QAAQ,KAAK,IAAI,CAAC,CAAC;AAAA,IACtF,SAAS,KAAK;AACZ,UAAI,WAAW,GAAG,EAAG,QAAO;AAC5B,YAAM,SAAS,GAAG;AAAA,IACpB;AAEA,SAAK,IAAI,iBAAiB,KAAK,eAAe;AAC5C,YAAM,IAAI;AAAA,QACR,mBAAmB,IAAI,aAAa,iBAAiB,aAAa;AAAA,MACpE;AAAA,IACF;AACA,QAAI,IAAI,SAAS,QAAW;AAC1B,YAAM,IAAI,eAAe,sCAAsC,GAAG,EAAE;AAAA,IACtE;AACA,UAAM,QAAQ,MACZ,IAAI,KACJ,qBAAqB;AACvB,WAAO,EAAE,OAAO,SAAS,IAAI,QAAQ,GAAG;AAAA,EAC1C;AAAA,EAEA,MAAM,MACJ,KACA,MACA,QACe;AACf,QAAI;AACF,YAAM,KAAK,OAAO;AAAA,QAChB,IAAIC,kBAAiB;AAAA,UACnB,QAAQ,KAAK;AAAA,UACb,KAAK;AAAA,UACL,MAAM;AAAA,UACN,aAAa;AAAA,UACb,aAAa,WAAW,WAAW,MAAM;AAAA,UACzC,SAAS,WAAW,WAAW,SAAY,OAAO;AAAA,QACpD,CAAC;AAAA,MACH;AAAA,IACF,SAAS,KAAK;AAEZ,UAAI,sBAAsB,GAAG,GAAG;AAC9B,cAAM,IAAIC,oBAAmB,6BAA6B,GAAG,EAAE;AAAA,MACjE;AACA,YAAM,SAAS,GAAG;AAAA,IACpB;AAAA,EACF;AAAA,EAEA,OAAO,SAAS,QAAuC;AACrD,QAAI;AACJ,OAAG;AACD,UAAI;AACJ,UAAI;AACF,cAAM,MAAM,KAAK,OAAO;AAAA,UACtB,IAAIC,sBAAqB;AAAA,YACvB,QAAQ,KAAK;AAAA,YACb,QAAQ;AAAA,YACR,mBAAmB;AAAA,UACrB,CAAC;AAAA,QACH;AAAA,MACF,SAAS,KAAK;AACZ,cAAM,SAAS,GAAG;AAAA,MACpB;AACA,iBAAW,OAAO,IAAI,YAAY,CAAC,GAAG;AACpC,YAAI,IAAI,QAAQ,OAAW,OAAM,IAAI;AAAA,MACvC;AACA,cAAQ,IAAI,gBAAgB,OAAO,IAAI,wBAAwB;AAAA,IACjE,SAAS,UAAU;AAAA,EACrB;AACF;AAGA,SAAS,SAAS,KAAuB;AACvC,MAAI,YAAY,GAAG,GAAG;AACpB,WAAO,IAAIC;AAAA,MACT,uBAAwB,KAAkC,QAAQ,SAAS;AAAA,MAC3E,EAAE,OAAO,IAAI;AAAA,IACf;AAAA,EACF;AACA,SAAO;AACT;AAEO,IAAM,mBAAN,cAA+B,oBAAoB;AAAA,EACxD,YAAY,SAAkC;AAC5C;AAAA,MACE,IAAI,QAAQ,QAAQ,QAAQ,QAAQ,MAAM;AAAA,MAC1CC,uBAAsB,QAAQ,MAAM;AAAA,MACpC,QAAQ,QAAQ,MAAc,KAAK,IAAI;AAAA,IACzC;AAAA,EACF;AACF;;;ACxIA,SAA0B,sBAAsB;AAMhD,SAAS,gBAAgB;AAkClB,IAAM,YAAN,MAA0C;AAAA,EAGtC;AAAA,EACA;AAAA;AAAA,EAEA;AAAA,EAET,YAAY,SAA2B;AACrC,SAAK,SACH,QAAQ,UACR,IAAI,SAAS;AAAA,MACX,GAAI,QAAQ,WAAW,SAAY,CAAC,IAAI,EAAE,QAAQ,QAAQ,OAAO;AAAA,MACjE,GAAI,QAAQ,aAAa,SAAY,CAAC,IAAI,EAAE,UAAU,QAAQ,SAAS;AAAA,MACvE,GAAI,QAAQ,cAAc,SAAY,CAAC,IAAI,EAAE,gBAAgB,QAAQ,UAAU;AAAA,MAC/E,GAAI,QAAQ,gBAAgB,SAAY,CAAC,IAAI,EAAE,aAAa,QAAQ,YAAY;AAAA,IAClF,CAAC;AACH,UAAM,SAAS;AAAA,MACb,QAAQ,KAAK;AAAA,MACb,QAAQ,QAAQ;AAAA,MAChB,GAAI,QAAQ,WAAW,SAAY,CAAC,IAAI,EAAE,QAAQ,QAAQ,OAAO;AAAA,IACnE;AACA,SAAK,UAAU,IAAI,gBAAgB,MAAM;AACzC,SAAK,WAAW,IAAI,iBAAiB;AAAA,MACnC,GAAG;AAAA,MACH,GAAI,QAAQ,QAAQ,SAAY,CAAC,IAAI,EAAE,KAAK,QAAQ,IAAI;AAAA,IAC1D,CAAC;AACD,mBAAe,IAAI;AAAA,EACrB;AACF;",
6
- "names": ["ValidationError", "ValidationError", "res", "TransientError", "WriteConflictError", "normalizeObjectPrefix", "GetObjectCommand", "ListObjectsV2Command", "PutObjectCommand", "GetObjectCommand", "PutObjectCommand", "WriteConflictError", "ListObjectsV2Command", "TransientError", "normalizeObjectPrefix"]
3
+ "sources": ["../src/backend.ts", "../src/storage.ts", "../src/keys.ts", "../src/aws-errors.ts", "../src/s3-errors.ts", "../src/send-once.ts", "../src/registry.ts"],
4
+ "sourcesContent": ["/**\n * `S3Storage` \u2014 the S3 backend as one object: the generations and the pointer, in one bucket, stated once.\n *\n * Replaces two constructor calls that each repeated `client`, `bucket` and `prefix`. Repeating them is how\n * they come apart: point the registry at one prefix and the objects at another and the store answers *empty*\n * rather than *misconfigured*, which is the hardest kind of wrong answer to debug. Here the location is written\n * once and shared, so the mismatch cannot be expressed.\n *\n * **It will build a client for you**, which is the common case \u2014 `new S3Storage({ bucket })` picks up the\n * ambient credential chain and region exactly as the SDK would. Pass `client` instead when you need a\n * credential chain the SDK cannot infer (SSO, an assumed role, a custom retry strategy); pass `endpoint` +\n * `pathStyle` + `credentials` for an S3-compatible store (MinIO, Ceph, R2). Both halves stay reachable as `.storage` and\n * `.registry` for anyone wiring something the facade does not cover. `maxObjectBytes` and `partBytes` size the\n * multipart upload.\n */\nimport { STORAGE_BACKEND, ValidationError, brandAsBackend } from '@cloudbitmaps/core/driver-kit';\nimport type {\n IRegistryDriver,\n IStorageDriver,\n StorageBackend,\n} from '@cloudbitmaps/core/driver-kit';\nimport { S3Client } from '@aws-sdk/client-s3';\nimport { S3StorageDriver } from './storage';\nimport { S3RegistryDriver } from './registry';\n\nexport interface S3StorageOptions {\n /** Target bucket (must already exist). */\n readonly bucket: string;\n /** Optional key prefix under which everything lives \u2014 generations and the registry alike. */\n readonly prefix?: string;\n /**\n * A constructed client. Supply one for a credential chain the SDK cannot infer; otherwise one is built. Its retry\n * applies to every request except the conditional writes, which are sent once whatever it is configured to do.\n */\n readonly client?: S3Client;\n /** Region for the client built when `client` is absent (refused beside `client`). Falls back to the SDK's own resolution. */\n readonly region?: string;\n /** Endpoint for an S3-compatible store (MinIO, Ceph, R2). Refused beside `client`, which carries its own. */\n readonly endpoint?: string;\n /** Path-style addressing, which most S3-compatible stores require. Refused beside `client`, which carries its own. */\n readonly pathStyle?: boolean;\n /**\n * Static credentials, for the S3-compatible stores that issue them (MinIO, Ceph, R2).\n *\n * On AWS itself, leave this unset \u2014 the SDK's own chain (instance role, SSO, environment, profile) is what\n * you want, and hard-coding keys to reach it would be a downgrade. It exists because the alternative for a\n * MinIO user was to construct an `S3Client` purely to carry two strings, which is the ergonomics this class\n * is here to remove. Refused beside `client`, which carries its own.\n */\n readonly credentials?: {\n readonly accessKeyId: string;\n readonly secretAccessKey: string;\n readonly sessionToken?: string;\n };\n /**\n * Largest object the backend will write and advertise. Default = `partBytes \u00D7 10,000` (\u2248 80 GiB at the default\n * 8 MiB part) \u2014 the honest ceiling reachable within S3's 10,000-part limit. Set it higher and `partBytes`\n * auto-grows so 10,000 parts still cover it (raising peak write memory to ~one part); up to the 5 TiB S3 max.\n * Must be a positive safe integer.\n */\n readonly maxObjectBytes?: number;\n /** Multipart part size in bytes (default 8 MiB; a smaller value is raised to the S3 5 MiB minimum). Must be a\n * positive safe integer. Tunes peak write memory. */\n readonly partBytes?: number;\n /** Injected clock for the registry's `createdAt`/`updatedAt`; defaults to `Date.now`. */\n readonly now?: () => number;\n}\n\n/**\n * The keys `new S3Storage(options)` takes. Any other is refused by name rather than ignored: an ignored client or\n * endpoint key builds a client from ambient credentials against the **public** endpoint, and for a store pointed at\n * MinIO that is production traffic from a wiring typo.\n */\nexport const S3_STORAGE_OPTION_KEYS = [\n 'bucket',\n 'prefix',\n 'client',\n 'region',\n 'endpoint',\n 'pathStyle',\n 'credentials',\n 'maxObjectBytes',\n 'partBytes',\n 'now',\n] as const;\n\n/** The settings that build a client, which a supplied `client` already carries and so cannot be given beside. */\nconst CLIENT_SETTINGS = ['region', 'endpoint', 'pathStyle', 'credentials'] as const;\n\n/** Refuse an options bag that is not an object, or that holds a key not in `keys`, naming each such key. */\nfunction refuseUnknown(\n name: string,\n options: unknown,\n keys: readonly string[],\n hint: string,\n): void {\n if (options === null || typeof options !== 'object') {\n throw new ValidationError(\n `${name} needs an options object \u2014 got ${options === null ? 'null' : typeof options}`,\n );\n }\n const unknown = Object.keys(options).filter((k) => !keys.includes(k));\n if (unknown.length > 0) {\n const list = (ks: readonly string[]): string => ks.map((k) => `\\`${k}\\``).join(', ');\n throw new ValidationError(\n `${name} does not take ${list(unknown)}. It takes ${list(keys)}; ${hint}.`,\n );\n }\n}\n\nexport class S3Storage implements StorageBackend {\n /** Cross-bundle brand, stamped non-enumerably in the constructor so a spread cannot carry it. */\n declare readonly [STORAGE_BACKEND]: true;\n readonly storage: IStorageDriver;\n readonly registry: IRegistryDriver;\n /** The client both halves share \u2014 built here unless one was supplied. */\n readonly client: S3Client;\n\n constructor(options: S3StorageOptions) {\n refuseUnknown('S3Storage', options, S3_STORAGE_OPTION_KEYS, 'an S3 client goes in `client`');\n if (options.client !== undefined && options.client !== null) {\n // A supplied client already carries its region, endpoint, addressing style and credentials, so a setting\n // beside it is ignored, and ignoring it leaves the store talking to somewhere the caller did not mean:\n // an `endpoint` meant for MinIO, silently dropped, is production traffic from a client that was built\n // for AWS. Refuse instead of picking one.\n const ignored = CLIENT_SETTINGS.filter((k) => options[k] !== undefined);\n if (ignored.length > 0) {\n throw new ValidationError(\n `S3Storage takes \\`client\\` OR ${CLIENT_SETTINGS.map((k) => `\\`${k}\\``).join(' / ')}, not both \u2014 ` +\n `got \\`client\\` with ${ignored.map((k) => `\\`${k}\\``).join(', ')}; ` +\n 'the `client` already carries them, so configure them on the client, or drop `client`',\n );\n }\n this.client = options.client;\n } else {\n this.client = new S3Client({\n ...(options.region === undefined ? {} : { region: options.region }),\n ...(options.endpoint === undefined ? {} : { endpoint: options.endpoint }),\n ...(options.pathStyle === undefined ? {} : { forcePathStyle: options.pathStyle }),\n ...(options.credentials === undefined ? {} : { credentials: options.credentials }),\n });\n }\n const shared = {\n client: this.client,\n bucket: options.bucket,\n ...(options.prefix === undefined ? {} : { prefix: options.prefix }),\n };\n this.storage = new S3StorageDriver({\n ...shared,\n ...(options.maxObjectBytes === undefined ? {} : { maxObjectBytes: options.maxObjectBytes }),\n ...(options.partBytes === undefined ? {} : { partBytes: options.partBytes }),\n });\n this.registry = new S3RegistryDriver({\n ...shared,\n ...(options.now === undefined ? {} : { now: options.now }),\n });\n brandAsBackend(this);\n }\n}\n", "/**\n * `S3StorageDriver` \u2014 an {@link IStorageDriver} over S3-compatible object storage.\n *\n * Works with AWS S3 and any compatible backend (MinIO, etc.) via the official `@aws-sdk/client-s3`, a real\n * dependency of this package \u2014 installing `@cloudbitmaps/s3` is what installs it. The client is\n * **injected** (dependency injection): the driver owns no credential/region/endpoint logic, so it's thin,\n * testable against MinIO (point a client at its endpoint), and reuses the caller's existing client.\n *\n * Generations are write-once immutable objects: a conditional `PutObject` with `If-None-Match: *` makes the\n * publish atomic \u2014 a second write to the same key fails with `WriteConflictError`, never a silent overwrite\n * (hard invariant 2: storage objects are immutable and never overwritten in place), the cloud analogue of\n * the LocalFs atomic `link`. **This requires a backend that honors\n * `If-None-Match: *`** (AWS S3 \u2014 GA Aug 2024; recent MinIO): a backend that silently ignored the\n * precondition would break write-once immutability. **Writes stream:** the object is uploaded in\n * constant memory \u2014 a small object is a single conditional `PutObject`; a large one is a **multipart upload**\n * (parts flushed as the codec writes, freed as they go) finished with a conditional `CompleteMultipartUpload`,\n * so a load's footprint stays ~one part regardless of segment size, up to the advertised `maxObjectBytes`\n * (default `partBytes \u00D7 10,000` \u2014 S3's per-upload part limit). **Each conditional request is sent once**, with the\n * SDK's retry off for it ({@link sendOnce}): a replay of a write that landed and lost its response would find its own\n * object and read as a lost race. A transient failure there throws {@link TransientError}, and the object may or may\n * not exist. The unconditional requests \u2014 the reads, the delete, and a multipart upload's own start, parts and abort \u2014\n * keep the SDK's retry. Drivers may use `node:crypto`; only `core/` is bound by the determinism lint.\n */\nimport {\n NotFoundError,\n TransientError,\n ValidationError,\n WriteConflictError,\n isNotFoundError,\n isValidationError,\n isWriteConflictError,\n} from '@cloudbitmaps/core/driver-kit';\nimport type {\n BlobSink,\n GenKey,\n IStorageDriver,\n SegmentRef,\n StorageCaps,\n} from '@cloudbitmaps/core/driver-kit';\nimport { createHash, type Hash } from 'node:crypto';\nimport {\n AbortMultipartUploadCommand,\n CompleteMultipartUploadCommand,\n CreateMultipartUploadCommand,\n DeleteObjectCommand,\n GetObjectCommand,\n HeadObjectCommand,\n ListObjectsV2Command,\n PutObjectCommand,\n UploadPartCommand,\n type S3Client,\n} from '@aws-sdk/client-s3';\nimport {\n storageObjectKey,\n normalizeS3Prefix,\n parseGenerationFromKey,\n segmentObjectPrefix,\n} from './keys';\nimport {\n isConditionalConflict,\n isInvalidRange,\n isNotFound,\n isTransient,\n totalFromContentRange,\n} from './s3-errors';\nimport { sendOnce } from './send-once';\n\n/** Part size for multipart uploads. \u2265 the S3 5 MiB minimum; an object that fits in one part uses a single\n * conditional PUT instead (no multipart overhead, strongest write-once). Peak write memory \u2248 one part. */\nconst S3_PART_BYTES = 8 * 1024 * 1024;\n/** S3 hard limit: a multipart upload has at most 10,000 parts. This \u00D7 the part size is the real object ceiling. */\nconst S3_MAX_PARTS = 10_000;\n\nexport interface S3StorageDriverOptions {\n /** A constructed S3 client (point its `endpoint` at MinIO for local/integration use). */\n readonly client: S3Client;\n /** Target bucket (must already exist). */\n readonly bucket: string;\n /** Optional key prefix under which all objects live (e.g. `cloudbitmaps/`). */\n readonly prefix?: string;\n /**\n * Largest object this driver will write/advertise. Default = `partBytes \u00D7 10,000` (\u2248 80 GiB at the default\n * 8 MiB part) \u2014 the honest ceiling reachable within S3's 10,000-part limit. Set it higher and `partBytes`\n * auto-grows so 10,000 parts still cover it (raising peak write memory to ~one part); up to the 5 TiB S3 max.\n */\n readonly maxObjectBytes?: number;\n /** Multipart part size in bytes (default 8 MiB; a smaller value is raised to the S3 5 MiB minimum). Must be a\n * positive safe integer. Tunes peak write memory. */\n readonly partBytes?: number;\n}\n\nexport class S3StorageDriver implements IStorageDriver {\n private readonly client: S3Client;\n private readonly bucket: string;\n private readonly prefix: string | undefined;\n private readonly maxObjectBytes: number;\n private readonly partBytes: number;\n\n constructor(options: S3StorageDriverOptions) {\n this.client = options.client;\n this.bucket = options.bucket;\n this.prefix = normalizeS3Prefix(options.prefix);\n // Fail fast at the boundary: `??` only guards `undefined`, so NaN, 0, a negative or a fraction would otherwise\n // reach the arithmetic below and size every part (and the advertised cap) from garbage.\n for (const [name, value] of [\n ['partBytes', options.partBytes],\n ['maxObjectBytes', options.maxObjectBytes],\n ] as const) {\n if (value !== undefined && (!Number.isSafeInteger(value) || value < 1)) {\n throw new ValidationError(`${name} must be a positive safe integer; got ${value}`);\n }\n }\n const requestedPart = Math.max(options.partBytes ?? S3_PART_BYTES, 5 * 1024 * 1024);\n // Default the object cap to what the requested part size can actually cover within S3's 10,000-part limit;\n // if a larger cap is requested, grow the part size to keep it reachable (so the advertised cap is honest).\n this.maxObjectBytes = options.maxObjectBytes ?? requestedPart * S3_MAX_PARTS;\n this.partBytes = Math.max(requestedPart, Math.ceil(this.maxObjectBytes / S3_MAX_PARTS));\n }\n\n capabilities(): StorageCaps {\n return { rangeRead: true, maxObjectBytes: this.maxObjectBytes, conditionalPut: true };\n }\n\n async putImmutable(\n key: GenKey,\n write: (sink: BlobSink) => Promise<void>,\n ): Promise<{ size: number; sha256: string }> {\n const objectKey = storageObjectKey(this.prefix, key); // validates ref + generation\n const sink = new S3MultipartSink(\n this.client,\n this.bucket,\n objectKey,\n this.partBytes,\n this.maxObjectBytes,\n );\n try {\n await write(sink);\n return await sink.finish();\n } catch (err) {\n await sink.abort(); // best-effort cleanup of any in-flight multipart upload\n // A lost conditional-write race \u2014 the precondition failed (412) or S3 rejected concurrent conditional\n // writes to the key (409) \u2014 is the write-once conflict, never a silent overwrite.\n if (isConditionalConflict(err)) {\n throw new WriteConflictError(\n `generation already exists (write-once): ${key.segment}.${key.generation}`,\n );\n }\n if (isValidationError(err) || isWriteConflictError(err) || isNotFoundError(err)) {\n throw err;\n }\n throw this.mapError(err);\n }\n }\n\n async getRange(key: GenKey, offset: number, length: number): Promise<Uint8Array> {\n if (!Number.isInteger(offset) || !Number.isInteger(length) || offset < 0 || length < 0) {\n throw new ValidationError(`invalid range offset=${offset} length=${length}`);\n }\n const objectKey = storageObjectKey(this.prefix, key);\n if (length === 0) return new Uint8Array(0);\n try {\n const res = await this.client.send(\n new GetObjectCommand({\n Bucket: this.bucket,\n Key: objectKey,\n Range: `bytes=${offset}-${offset + length - 1}`,\n }),\n );\n const bytes = await collect(res.Body);\n // A short read means the range ran past EOF \u2014 treat as out-of-bounds, never a partial result.\n if (bytes.length !== length) {\n throw new ValidationError(\n `range [${offset}, ${offset + length}) out of bounds (got ${bytes.length}B)`,\n );\n }\n return bytes;\n } catch (err) {\n throw this.mapReadError(err, key);\n }\n }\n\n async getTail(key: GenKey, maxBytes: number): Promise<{ bytes: Uint8Array; size: number }> {\n const objectKey = storageObjectKey(this.prefix, key);\n if (maxBytes <= 0) {\n // No tail bytes wanted \u2014 just resolve the size via a HEAD.\n try {\n const head = await this.client.send(\n new HeadObjectCommand({ Bucket: this.bucket, Key: objectKey }),\n );\n return { bytes: new Uint8Array(0), size: head.ContentLength ?? 0 };\n } catch (err) {\n throw this.mapReadError(err, key);\n }\n }\n try {\n const res = await this.client.send(\n new GetObjectCommand({ Bucket: this.bucket, Key: objectKey, Range: `bytes=-${maxBytes}` }),\n );\n const bytes = await collect(res.Body);\n let size = totalFromContentRange(res.ContentRange);\n if (size === undefined) {\n // A spec-compliant backend omits Content-Range only on a 200 (whole object), where bytes.length\n // IS the size. If the body is exactly maxBytes we can't rule out a clamped partial from a\n // non-compliant backend \u2014 confirm the true size with a HEAD rather than trust a possibly-short read.\n if (bytes.length === maxBytes) {\n const head = await this.client.send(\n new HeadObjectCommand({ Bucket: this.bucket, Key: objectKey }),\n );\n size = head.ContentLength ?? bytes.length;\n } else {\n size = bytes.length;\n }\n }\n return { bytes, size };\n } catch (err) {\n throw this.mapReadError(err, key);\n }\n }\n\n async delete(key: GenKey): Promise<void> {\n // Idempotent: S3 DeleteObject succeeds even if the key is absent (GC may race / retry).\n try {\n await this.client.send(\n new DeleteObjectCommand({ Bucket: this.bucket, Key: storageObjectKey(this.prefix, key) }),\n );\n } catch (err) {\n throw this.mapError(err);\n }\n }\n\n async *list(ref: SegmentRef): AsyncIterable<GenKey> {\n const prefix = segmentObjectPrefix(this.prefix, ref); // validates ref\n let token: string | undefined;\n do {\n let res;\n try {\n res = await this.client.send(\n new ListObjectsV2Command({\n Bucket: this.bucket,\n Prefix: prefix,\n ContinuationToken: token,\n }),\n );\n } catch (err) {\n throw this.mapError(err);\n }\n for (const obj of res.Contents ?? []) {\n if (obj.Key === undefined) continue;\n const generation = parseGenerationFromKey(prefix, obj.Key);\n if (generation !== null) {\n yield { namespace: ref.namespace, segment: ref.segment, generation };\n }\n }\n token = res.IsTruncated === true ? res.NextContinuationToken : undefined;\n } while (token !== undefined);\n }\n\n /** Map S3 read errors to the driver vocabulary; pass everything else through {@link mapError}. */\n private mapReadError(err: unknown, key: GenKey): unknown {\n if (isValidationError(err)) return err;\n if (isNotFound(err)) {\n return new NotFoundError(`no such generation: ${key.segment}.${key.generation}`);\n }\n // A fully out-of-range request (start past EOF) \u2014 the BlobReader contract treats range errors as\n // ValidationError, never a short/empty read.\n if (isInvalidRange(err)) {\n return new ValidationError(`range out of bounds for ${key.segment}.${key.generation}`);\n }\n return this.mapError(err);\n }\n\n /**\n * Reclassify a transient S3 fault (throttle/5xx/dropped connection) as a retryable {@link TransientError},\n * so the store's read retry can ride it out and a write's caller can tell it from a deterministic failure;\n * everything else propagates unchanged. The final fallback at every `client.send` site, so callers and the\n * read retry only ever see typed errors.\n */\n private mapError(err: unknown): unknown {\n if (isTransient(err)) {\n return new TransientError(\n `transient S3 fault: ${(err as { name?: string } | null)?.name ?? 'unknown'}`,\n { cause: err },\n );\n }\n return err;\n }\n}\n\n/** Concatenate a list of byte chunks of known total length into one buffer. */\nfunction concatBytes(parts: readonly Uint8Array[], total: number): Uint8Array {\n const out = new Uint8Array(total);\n let offset = 0;\n for (const p of parts) {\n out.set(p, offset);\n offset += p.length;\n }\n return out;\n}\n\n/**\n * Streaming {@link BlobSink} that uploads one S3 object in **constant memory**. It buffers at most\n * one part: as the codec writes, full parts are flushed via `UploadPart` and freed. A small object that never\n * reaches one part is committed as a single conditional `PutObject`; a larger one is finished with a\n * conditional `CompleteMultipartUpload` \u2014 **both enforce write-once** via `If-None-Match: *`, and both are sent\n * once. SHA-256 is hashed incrementally. On any error the caller invokes {@link abort} to clean up the in-flight\n * multipart upload.\n */\nclass S3MultipartSink implements BlobSink {\n private readonly hash: Hash = createHash('sha256');\n private readonly pending: Uint8Array[] = [];\n private pendingLen = 0;\n private total = 0;\n private uploadId: string | undefined;\n private partNumber = 0;\n private readonly parts: { ETag: string | undefined; PartNumber: number }[] = [];\n\n constructor(\n private readonly client: S3Client,\n private readonly bucket: string,\n private readonly objectKey: string,\n private readonly partBytes: number,\n private readonly maxObjectBytes: number,\n ) {}\n\n async write(bytes: Uint8Array): Promise<void> {\n if (bytes.length === 0) return;\n this.total += bytes.length;\n if (this.total > this.maxObjectBytes) {\n // Fail fast + typed, rather than a late opaque S3 error (and abort the in-flight upload via the caller).\n throw new ValidationError(`object exceeds maxObjectBytes ${this.maxObjectBytes}`);\n }\n this.hash.update(bytes);\n this.pending.push(bytes);\n this.pendingLen += bytes.length;\n if (this.pendingLen >= this.partBytes) await this.flushPart();\n }\n\n /** Upload the buffered bytes (\u2265 one part) as a single part, freeing them. Starts the upload on first call. */\n private async flushPart(): Promise<void> {\n if (this.uploadId === undefined) {\n const res = await this.client.send(\n new CreateMultipartUploadCommand({ Bucket: this.bucket, Key: this.objectKey }),\n );\n if (res.UploadId === undefined) {\n throw new TransientError('S3 CreateMultipartUpload returned no UploadId');\n }\n this.uploadId = res.UploadId;\n }\n const body = concatBytes(this.pending, this.pendingLen);\n this.pending.length = 0;\n this.pendingLen = 0;\n this.partNumber += 1;\n if (this.partNumber > S3_MAX_PARTS) {\n // Unreachable for valid input (the maxObjectBytes byte-cap, sized to \u2264 S3_MAX_PARTS parts, fires first) \u2014\n // a typed guard so the S3 hard limit is never a raw 400.\n throw new ValidationError(`multipart upload exceeded the S3 ${S3_MAX_PARTS}-part limit`);\n }\n const res = await this.client.send(\n new UploadPartCommand({\n Bucket: this.bucket,\n Key: this.objectKey,\n UploadId: this.uploadId,\n PartNumber: this.partNumber,\n Body: body,\n }),\n );\n this.parts.push({ ETag: res.ETag, PartNumber: this.partNumber });\n }\n\n /** Commit the object: a single conditional PUT if it fit in one part, else complete the multipart upload. */\n async finish(): Promise<{ size: number; sha256: string }> {\n const sha256 = this.hash.digest('hex');\n if (this.uploadId === undefined) {\n await sendOnce(\n this.client,\n new PutObjectCommand({\n Bucket: this.bucket,\n Key: this.objectKey,\n Body: concatBytes(this.pending, this.pendingLen),\n IfNoneMatch: '*', // write-once\n }),\n );\n return { size: this.total, sha256 };\n }\n if (this.pendingLen > 0) await this.flushPart(); // the final part may be < partBytes (allowed)\n await sendOnce(\n this.client,\n new CompleteMultipartUploadCommand({\n Bucket: this.bucket,\n Key: this.objectKey,\n UploadId: this.uploadId,\n MultipartUpload: { Parts: this.parts },\n IfNoneMatch: '*', // write-once: fail if the object already exists\n }),\n );\n this.uploadId = undefined; // completed \u2014 nothing left to abort\n return { size: this.total, sha256 };\n }\n\n /** Best-effort cleanup of an in-flight multipart upload after an error (a leaked MPU is reaped by a bucket\n * lifecycle rule; never a correctness issue). No-op if nothing was started or it already completed. */\n async abort(): Promise<void> {\n if (this.uploadId === undefined) return;\n const id = this.uploadId;\n this.uploadId = undefined;\n try {\n await this.client.send(\n new AbortMultipartUploadCommand({ Bucket: this.bucket, Key: this.objectKey, UploadId: id }),\n );\n } catch {\n // swallow \u2014 best-effort\n }\n }\n}\n\n/**\n * Collect an S3 response body into a `Uint8Array`. `transformToByteArray` is added at runtime to the SDK's\n * Node stream by `@aws-sdk`'s sdk-stream-mixin, so the structural cast is sound on Node.\n */\nasync function collect(body: GetObjectCommandBody): Promise<Uint8Array> {\n if (body === undefined) {\n throw new NotFoundError('S3 GetObject returned an empty body');\n }\n return body.transformToByteArray();\n}\n\n/** The S3 `GetObject` Body type, narrowed to the part we use (`transformToByteArray`). */\ntype GetObjectCommandBody = { transformToByteArray(): Promise<Uint8Array> } | undefined;\n", "/**\n * Logical-ref \u2192 S3 object-key mapping for {@link S3StorageDriver}.\n *\n * Pure string logic with no SDK dependency, so it's unit-testable without S3/MinIO. Mirrors the LocalFs\n * layout (`<namespace>/segments/<segment>.<gen>.crbm`) under an optional caller prefix, and re-validates\n * names at the boundary \u2014 defense in depth, because a driver can be constructed and driven directly rather\n * than through the engine that would otherwise have validated for it. The default\n * (absent) namespace maps to `_default`, which cannot collide with a real namespace because a caller's\n * `_default` encodes to `%5Fdefault` while the sentinel is emitted literally.\n */\n// `prefixPart` is imported, never redefined: the storage and registry layouts sit under the SAME caller\n// prefix, so they must normalize it identically \u2014 a second copy of that three-line function is how the two\n// halves of one bucket drift apart.\nimport {\n ValidationError,\n encodeNameForKey,\n namespaceKeyPart,\n prefixPart,\n validateSegmentRef,\n} from '@cloudbitmaps/core/driver-kit';\nimport type { GenKey, SegmentRef } from '@cloudbitmaps/core/driver-kit';\n\nconst SUFFIX = '.crbm';\n\n/** Validate a caller-supplied key prefix. The rule is shared with every other object store. */\nexport { normalizeObjectPrefix as normalizeS3Prefix } from '@cloudbitmaps/core/driver-kit';\n\n/**\n * The S3 key prefix shared by all of a segment's generations: `<prefix><ns>/segments/<segment>.`. Used\n * both as the `ListObjectsV2` prefix and as the string stripped by {@link parseGenerationFromKey}.\n */\nexport function segmentObjectPrefix(prefix: string | undefined, ref: SegmentRef): string {\n validateSegmentRef(ref);\n return `${prefixPart(prefix)}${namespaceKeyPart(ref.namespace)}/segments/${encodeNameForKey(ref.segment)}.`;\n}\n\n/** The full S3 key of one `.crbm` generation: `<segmentPrefix><gen>.crbm`. */\nexport function storageObjectKey(prefix: string | undefined, key: GenKey): string {\n if (!Number.isInteger(key.generation) || key.generation < 0) {\n throw new ValidationError(`generation must be a non-negative integer; got ${key.generation}`);\n }\n return `${segmentObjectPrefix(prefix, key)}${key.generation}${SUFFIX}`;\n}\n\n/**\n * Parse a generation number out of a full object key, given its segment prefix, or `null` if it doesn't\n * match. Canonical decimal only \u2014 no leading zeros (so `\u2026s.07.crbm` can't alias `\u2026s.7.crbm`) and within\n * safe-integer range. This also rejects a *different* segment whose name merely shares the prefix (e.g. a\n * key for segment `s.x` won't parse under segment `s`'s prefix, since its middle isn't all digits).\n */\nexport function parseGenerationFromKey(segmentPrefix: string, objectKey: string): number | null {\n if (!objectKey.startsWith(segmentPrefix) || !objectKey.endsWith(SUFFIX)) return null;\n const middle = objectKey.slice(segmentPrefix.length, objectKey.length - SUFFIX.length);\n if (!/^(0|[1-9]\\d*)$/.test(middle)) return null;\n const generation = Number(middle);\n return Number.isSafeInteger(generation) ? generation : null;\n}\n", "/**\n * SDK-free helpers for classifying AWS-style errors (used by the S3 drivers).\n *\n * These only read structural shapes an AWS SDK v3 error carries \u2014 `name`, `$metadata.httpStatusCode`, a\n * lower-level `code`/`errno`, and the SDK's own `$retryable` marker \u2014 so the (subtle, easy-to-get-wrong)\n * transient-vs-fatal decision is unit-testable without a live backend or even the SDK installed. They import no SDK.\n */\n\nexport function httpStatus(err: unknown): number | undefined {\n return (err as { $metadata?: { httpStatusCode?: number } } | null)?.$metadata?.httpStatusCode;\n}\n\nexport function errorName(err: unknown): string | undefined {\n return (err as { name?: string } | null)?.name;\n}\n\n/** A lower-level transport code (e.g. `ECONNRESET`) \u2014 the Node networking layer sets `code`. */\nexport function errorCode(err: unknown): string | undefined {\n return (err as { code?: string } | null)?.code;\n}\n\n/** The AWS SDK v3 tags retryable errors with a `$retryable` object (throttling faults carry `.throttling`). */\nexport function isSdkRetryable(err: unknown): boolean {\n return (err as { $retryable?: unknown } | null)?.$retryable != null;\n}\n\n/** Any 5xx is a server-side fault that's safe to retry (the request didn't deterministically fail). */\nexport function isServerSide(err: unknown): boolean {\n const status = httpStatus(err);\n return status !== undefined && status >= 500 && status <= 599;\n}\n\nconst NETWORK_NAMES = new Set([\n 'TimeoutError',\n 'RequestTimeout',\n 'RequestTimeoutException',\n 'NetworkingError',\n 'AbortError',\n]);\nconst NETWORK_CODES = new Set([\n 'ETIMEDOUT',\n 'ECONNRESET',\n 'ECONNREFUSED',\n 'EPIPE',\n 'ENOTFOUND',\n 'EAI_AGAIN',\n 'ECONNABORTED',\n]);\n\n// Message-text fallback for when the structural signals (name/code/$metadata/$retryable) are absent. Kept\n// SPECIFIC on purpose: a loose `/timed? ?out/` matches deterministic messages like \"value timed out of\n// range\" and would wrongly retry them, so we only match timeout/network phrases anchored to a transport word\n// (connection/request/socket/read/write) plus the unambiguous standalone phrases.\nconst NETWORK_MESSAGE =\n /socket hang up|network (error|failure)|(connection|request|socket|operation|read|write)\\s+tim(e|ed)\\s?out|connection (reset|refused|aborted|closed)/i;\n\n/** A dropped/timed-out connection \u2014 transient by nature; a retry on a fresh connection usually succeeds. */\nexport function isNetworkOrTimeout(err: unknown): boolean {\n if (NETWORK_NAMES.has(errorName(err) ?? '')) return true;\n if (NETWORK_CODES.has(errorCode(err) ?? '')) return true;\n return NETWORK_MESSAGE.test((err as { message?: string } | null)?.message ?? '');\n}\n", "/**\n * Pure helpers for classifying S3 SDK errors + parsing response headers (conflict and transient classification).\n *\n * Kept SDK-free and side-effect-free (they only read structural shapes \u2014 `err.name`,\n * `$metadata.httpStatusCode`, a `Content-Range` string) so the subtle S3-specific translation logic is\n * unit-testable without a live MinIO/S3 or even the AWS SDK. The AWS error shapes come from `./aws-errors`.\n */\n\nimport {\n errorName,\n httpStatus,\n isNetworkOrTimeout,\n isSdkRetryable,\n isServerSide,\n} from './aws-errors';\n\n/** A conditional `If-None-Match: *` PUT lost the write-once race (the object already existed). */\nexport function isPreconditionFailed(err: unknown): boolean {\n return errorName(err) === 'PreconditionFailed' || httpStatus(err) === 412;\n}\n\n/**\n * A conditional write (`If-None-Match: *` / `If-Match: <etag>`) lost the race \u2014 **either** outcome S3 uses:\n * the precondition evaluated false (`412 PreconditionFailed`), **or** S3 rejected concurrent conditional\n * writes to the same key to prevent a lost update (`409 ConditionalRequestConflict`, which AWS documents and\n * asks you to retry). Both mean \"you lost; re-read and retry\" \u2014 so both must map to `WriteConflictError` and\n * route through the caller's OCC path, never a blind transient retry (which would just replay a doomed PUT).\n */\nexport function isConditionalConflict(err: unknown): boolean {\n return (\n isPreconditionFailed(err) ||\n errorName(err) === 'ConditionalRequestConflict' ||\n httpStatus(err) === 409\n );\n}\n\n/** The object / generation does not exist (GetObject \u2192 `NoSuchKey`, HeadObject \u2192 `NotFound`; both 404). */\nexport function isNotFound(err: unknown): boolean {\n const name = errorName(err);\n return name === 'NoSuchKey' || name === 'NotFound' || httpStatus(err) === 404;\n}\n\n/** A range request started past EOF (HTTP 416). */\nexport function isInvalidRange(err: unknown): boolean {\n return errorName(err) === 'InvalidRange' || httpStatus(err) === 416;\n}\n\n/**\n * S3 refused the request's signature because the client's clock was off by minutes, and the SDK has corrected the\n * clock for the next request. Nothing was applied, and a second request is signed right. The SDK's own retry treats\n * this as transient; a conditional write is sent once without that retry, so the driver has to say so itself.\n */\nfunction isClockSkewCorrected(err: unknown): boolean {\n const e = err as { $metadata?: { clockSkewCorrected?: unknown } } | null;\n return e?.$metadata?.clockSkewCorrected === true;\n}\n\n/**\n * A transient S3 fault that is safe to retry: throttling (`SlowDown` / 503), any 5xx, a dropped/timed-out\n * connection, a clock-skew refusal the SDK has corrected for, or anything the SDK itself marks retryable. Excludes\n * the deterministic outcomes above (412/404/416) \u2014 those are caller-meaningful and must never be\n * retried/reclassified.\n */\nexport function isTransient(err: unknown): boolean {\n // A conditional-write conflict (412/409) is caller-meaningful OCC, not a blind-retryable transient.\n if (isConditionalConflict(err) || isNotFound(err) || isInvalidRange(err)) return false;\n return (\n errorName(err) === 'SlowDown' ||\n isServerSide(err) ||\n isNetworkOrTimeout(err) ||\n isSdkRetryable(err) ||\n isClockSkewCorrected(err)\n );\n}\n\n/**\n * Parse the total object size out of a `Content-Range: bytes <start>-<end>/<total>` header, or `undefined`\n * if absent/unparseable/unsafe. The total is the part after the final `/`.\n */\nexport function totalFromContentRange(contentRange: string | undefined): number | undefined {\n if (contentRange === undefined) return undefined;\n const match = /\\/(\\d+)\\s*$/.exec(contentRange);\n if (match === null) return undefined;\n const total = Number(match[1]);\n return Number.isSafeInteger(total) ? total : undefined;\n}\n", "/**\n * `sendOnce` \u2014 send a conditional write exactly once, with the SDK's retry off for that one command.\n *\n * The SDK re-sends a request whose response it did not get: a timeout, a reset connection, a 5xx. For a conditional\n * write that is the wrong thing to do. When the write landed and only its response was lost, the second send meets\n * the first \u2014 `If-None-Match: *` finds the object it created, `If-Match` finds the ETag it replaced \u2014 and fails with\n * `412`, which the driver can only report as a lost race. The caller is told its write lost when it won. Only the\n * caller can find out which happened, by reading the pointer or listing the generations, so the write is sent once\n * and a transient failure reaches it as `TransientError`.\n *\n * **How.** The client runs its retry as one middleware, `retryMiddleware`, at high priority in the `finalizeRequest`\n * step, whatever retry strategy or `maxAttempts` it was built with. When a command is sent, its own middleware stack\n * is merged over the client's, and an entry with the same name, step and priority that sets `override` replaces the\n * client's. So this command runs with a pass-through where the retry was, and nothing about the client changes: a\n * caller's own client keeps its configuration, and every other command sent through it keeps its retry.\n *\n * The options object passed to `send` matters too. A client built with `cacheMiddleware: true` reuses the handler it\n * resolved for the first command of a class, and that handler holds the retry; `send` resolves afresh whenever it is\n * given options, so the replacement always takes effect.\n */\nimport type {\n $Command,\n S3Client,\n S3ClientResolvedConfig,\n ServiceInputTypes,\n ServiceOutputTypes,\n} from '@aws-sdk/client-s3';\n\n/** Where the client registers its retry. Overriding an entry takes the same name, step and priority. */\nconst NO_RETRY = {\n name: 'retryMiddleware',\n step: 'finalizeRequest',\n priority: 'high',\n override: true,\n} as const;\n\n/** Send `command` once: its retry step is a pass-through, and the client's stays as it is for every other command. */\nexport function sendOnce<Input extends ServiceInputTypes, Output extends ServiceOutputTypes>(\n client: S3Client,\n command: $Command<Input, Output, S3ClientResolvedConfig, ServiceInputTypes, ServiceOutputTypes>,\n): Promise<Output> {\n command.middlewareStack.add((next) => next, NO_RETRY);\n return client.send(command, {});\n}\n", "/**\n * `S3RegistryDriver` \u2014 an {@link IRegistryDriver} over S3-compatible object storage.\n *\n * Lets a **read-mostly deployment run on S3 alone** \u2014 storage `.crbm` generations + the registry in one bucket,\n * no separate database. The protocol (an ABA-safe OCC counter, tombstoning delete, the bounded retry, the key layout)\n * lives once in {@link ObjectStoreRegistry}; this file is only the three I/O calls S3 makes, so the S3, GCS\n * and Azure registries cannot drift from one another.\n *\n * **The atomic swap is offloaded to S3's conditional writes** (GA Nov 2024): `If-None-Match: *` for\n * create-only and `If-Match: <etag>` for compare-and-swap, so a concurrent writer between our read and our\n * PUT loses with a `412` \u2192 {@link WriteConflictError}. Each conditional PUT is sent once, with the SDK's retry off\n * for it ({@link sendOnce}), so a `412` means another write got there first, never this one meeting itself after a\n * lost response. A transient failure reaches the caller as {@link TransientError}: the write may or may not have\n * landed, and the caller re-reads the row to learn where it stands. Reads are strongly consistent (S3, since 2020),\n * satisfying the registry's `strongRead` contract. The client is **injected**, exactly like {@link S3StorageDriver}.\n *\n * **Deployment requirements** (a backend/policy that violates these silently corrupts the registry):\n * - The backend **must honor `If-Match`** (AWS S3; recent MinIO). One that returns ETags but ignores the\n * precondition degrades compare-and-swap to last-write-wins \u2192 lost `currentGen` swaps. Verified against\n * real S3 semantics by the MinIO integration lane.\n * - The IAM principal needs **`s3:ListBucket`** on the bucket. Without it, `GetObject` on a missing key\n * returns `403` (not `404`), so the \"absent segment \u2192 `null`\" contract (and `create`'s bootstrap read)\n * breaks \u2014 and `list()` needs it regardless.\n * - **Do not apply an S3 lifecycle-expiration rule to the `registry/` prefix that expires a current version**\n * (`NoncurrentVersionExpiration` is safe). See {@link ObjectStoreRegistry}.\n */\nimport {\n IntegrityError,\n MAX_ROW_BYTES,\n ObjectStoreRegistry,\n TransientError,\n WriteConflictError,\n normalizeObjectPrefix,\n} from '@cloudbitmaps/core/driver-kit';\nimport type { ObjectRegistryStore, ObjectRow } from '@cloudbitmaps/core/driver-kit';\nimport {\n GetObjectCommand,\n ListObjectsV2Command,\n PutObjectCommand,\n type S3Client,\n} from '@aws-sdk/client-s3';\nimport { isConditionalConflict, isNotFound, isTransient } from './s3-errors';\nimport { sendOnce } from './send-once';\n\nexport interface S3RegistryDriverOptions {\n /** A constructed S3 client (point its `endpoint` at MinIO for local/integration use). */\n readonly client: S3Client;\n /** Target bucket (must already exist). */\n readonly bucket: string;\n /** Optional key prefix under which all registry objects live (e.g. `cloudbitmaps/`). */\n readonly prefix?: string;\n /** Injected clock for `createdAt`/`updatedAt`; defaults to `Date.now`. */\n readonly now?: () => number;\n}\n\n/** The three calls {@link ObjectStoreRegistry} needs, in S3's dialect. */\nclass S3Store implements ObjectRegistryStore {\n readonly label = 'S3';\n\n constructor(\n private readonly client: S3Client,\n private readonly bucket: string,\n ) {}\n\n async read(key: string): Promise<ObjectRow | null> {\n let res;\n try {\n res = await this.client.send(new GetObjectCommand({ Bucket: this.bucket, Key: key }));\n } catch (err) {\n if (isNotFound(err)) return null;\n throw mapError(err);\n }\n // Check the advertised length BEFORE allocating, so a hostile object cannot make us buffer it first.\n if ((res.ContentLength ?? 0) > MAX_ROW_BYTES) {\n throw new IntegrityError(\n `registry object ${res.ContentLength}B exceeds cap ${MAX_ROW_BYTES}B`,\n );\n }\n if (res.Body === undefined) {\n throw new IntegrityError(`registry object has an empty body: ${key}`);\n }\n const bytes = await (\n res.Body as { transformToByteArray(): Promise<Uint8Array> }\n ).transformToByteArray();\n return { bytes, version: res.ETag ?? '' };\n }\n\n async write(\n key: string,\n body: Uint8Array,\n expect: 'absent' | { version: string },\n ): Promise<void> {\n try {\n // Sent once: a replay of a write that landed would fail its own precondition, and read as a lost race.\n await sendOnce(\n this.client,\n new PutObjectCommand({\n Bucket: this.bucket,\n Key: key,\n Body: body,\n ContentType: 'application/json',\n IfNoneMatch: expect === 'absent' ? '*' : undefined,\n IfMatch: expect === 'absent' ? undefined : expect.version,\n }),\n );\n } catch (err) {\n // A lost conditional-write race (412 precondition, or 409 concurrent-conflict) is an OCC conflict.\n if (isConditionalConflict(err)) {\n throw new WriteConflictError(`registry OCC conflict for ${key}`);\n }\n throw mapError(err);\n }\n }\n\n async *listKeys(prefix: string): AsyncIterable<string> {\n let token: string | undefined;\n do {\n let res;\n try {\n res = await this.client.send(\n new ListObjectsV2Command({\n Bucket: this.bucket,\n Prefix: prefix,\n ContinuationToken: token,\n }),\n );\n } catch (err) {\n throw mapError(err);\n }\n for (const obj of res.Contents ?? []) {\n if (obj.Key !== undefined) yield obj.Key;\n }\n token = res.IsTruncated === true ? res.NextContinuationToken : undefined;\n } while (token !== undefined);\n }\n}\n\n/** Reclassify a transient S3 fault as a retryable {@link TransientError}; pass everything else through. */\nfunction mapError(err: unknown): unknown {\n if (isTransient(err)) {\n return new TransientError(\n `transient S3 fault: ${(err as { name?: string } | null)?.name ?? 'unknown'}`,\n { cause: err },\n );\n }\n return err;\n}\n\nexport class S3RegistryDriver extends ObjectStoreRegistry {\n constructor(options: S3RegistryDriverOptions) {\n super(\n new S3Store(options.client, options.bucket),\n normalizeObjectPrefix(options.prefix),\n options.now ?? ((): number => Date.now()),\n );\n }\n}\n"],
5
+ "mappings": ";AAeA,SAA0B,mBAAAA,kBAAiB,sBAAsB;AAMjE,SAAS,gBAAgB;;;ACEzB;AAAA,EACE;AAAA,EACA;AAAA,EACA,mBAAAC;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAQP,SAAS,kBAA6B;AACtC;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OAEK;;;ACtCP;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAMP,SAAkC,6BAAyB;AAH3D,IAAM,SAAS;AASR,SAAS,oBAAoB,QAA4B,KAAyB;AACvF,qBAAmB,GAAG;AACtB,SAAO,GAAG,WAAW,MAAM,CAAC,GAAG,iBAAiB,IAAI,SAAS,CAAC,aAAa,iBAAiB,IAAI,OAAO,CAAC;AAC1G;AAGO,SAAS,iBAAiB,QAA4B,KAAqB;AAChF,MAAI,CAAC,OAAO,UAAU,IAAI,UAAU,KAAK,IAAI,aAAa,GAAG;AAC3D,UAAM,IAAI,gBAAgB,kDAAkD,IAAI,UAAU,EAAE;AAAA,EAC9F;AACA,SAAO,GAAG,oBAAoB,QAAQ,GAAG,CAAC,GAAG,IAAI,UAAU,GAAG,MAAM;AACtE;AAQO,SAAS,uBAAuB,eAAuB,WAAkC;AAC9F,MAAI,CAAC,UAAU,WAAW,aAAa,KAAK,CAAC,UAAU,SAAS,MAAM,EAAG,QAAO;AAChF,QAAM,SAAS,UAAU,MAAM,cAAc,QAAQ,UAAU,SAAS,OAAO,MAAM;AACrF,MAAI,CAAC,iBAAiB,KAAK,MAAM,EAAG,QAAO;AAC3C,QAAM,aAAa,OAAO,MAAM;AAChC,SAAO,OAAO,cAAc,UAAU,IAAI,aAAa;AACzD;;;AChDO,SAAS,WAAW,KAAkC;AAC3D,SAAQ,KAA4D,WAAW;AACjF;AAEO,SAAS,UAAU,KAAkC;AAC1D,SAAQ,KAAkC;AAC5C;AAGO,SAAS,UAAU,KAAkC;AAC1D,SAAQ,KAAkC;AAC5C;AAGO,SAAS,eAAe,KAAuB;AACpD,SAAQ,KAAyC,cAAc;AACjE;AAGO,SAAS,aAAa,KAAuB;AAClD,QAAM,SAAS,WAAW,GAAG;AAC7B,SAAO,WAAW,UAAa,UAAU,OAAO,UAAU;AAC5D;AAEA,IAAM,gBAAgB,oBAAI,IAAI;AAAA,EAC5B;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AACF,CAAC;AACD,IAAM,gBAAgB,oBAAI,IAAI;AAAA,EAC5B;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AACF,CAAC;AAMD,IAAM,kBACJ;AAGK,SAAS,mBAAmB,KAAuB;AACxD,MAAI,cAAc,IAAI,UAAU,GAAG,KAAK,EAAE,EAAG,QAAO;AACpD,MAAI,cAAc,IAAI,UAAU,GAAG,KAAK,EAAE,EAAG,QAAO;AACpD,SAAO,gBAAgB,KAAM,KAAqC,WAAW,EAAE;AACjF;;;AC5CO,SAAS,qBAAqB,KAAuB;AAC1D,SAAO,UAAU,GAAG,MAAM,wBAAwB,WAAW,GAAG,MAAM;AACxE;AASO,SAAS,sBAAsB,KAAuB;AAC3D,SACE,qBAAqB,GAAG,KACxB,UAAU,GAAG,MAAM,gCACnB,WAAW,GAAG,MAAM;AAExB;AAGO,SAAS,WAAW,KAAuB;AAChD,QAAM,OAAO,UAAU,GAAG;AAC1B,SAAO,SAAS,eAAe,SAAS,cAAc,WAAW,GAAG,MAAM;AAC5E;AAGO,SAAS,eAAe,KAAuB;AACpD,SAAO,UAAU,GAAG,MAAM,kBAAkB,WAAW,GAAG,MAAM;AAClE;AAOA,SAAS,qBAAqB,KAAuB;AACnD,QAAM,IAAI;AACV,SAAO,GAAG,WAAW,uBAAuB;AAC9C;AAQO,SAAS,YAAY,KAAuB;AAEjD,MAAI,sBAAsB,GAAG,KAAK,WAAW,GAAG,KAAK,eAAe,GAAG,EAAG,QAAO;AACjF,SACE,UAAU,GAAG,MAAM,cACnB,aAAa,GAAG,KAChB,mBAAmB,GAAG,KACtB,eAAe,GAAG,KAClB,qBAAqB,GAAG;AAE5B;AAMO,SAAS,sBAAsB,cAAsD;AAC1F,MAAI,iBAAiB,OAAW,QAAO;AACvC,QAAM,QAAQ,cAAc,KAAK,YAAY;AAC7C,MAAI,UAAU,KAAM,QAAO;AAC3B,QAAM,QAAQ,OAAO,MAAM,CAAC,CAAC;AAC7B,SAAO,OAAO,cAAc,KAAK,IAAI,QAAQ;AAC/C;;;ACxDA,IAAM,WAAW;AAAA,EACf,MAAM;AAAA,EACN,MAAM;AAAA,EACN,UAAU;AAAA,EACV,UAAU;AACZ;AAGO,SAAS,SACd,QACA,SACiB;AACjB,UAAQ,gBAAgB,IAAI,CAAC,SAAS,MAAM,QAAQ;AACpD,SAAO,OAAO,KAAK,SAAS,CAAC,CAAC;AAChC;;;AJ0BA,IAAM,gBAAgB,IAAI,OAAO;AAEjC,IAAM,eAAe;AAoBd,IAAM,kBAAN,MAAgD;AAAA,EACpC;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EAEjB,YAAY,SAAiC;AAC3C,SAAK,SAAS,QAAQ;AACtB,SAAK,SAAS,QAAQ;AACtB,SAAK,SAAS,sBAAkB,QAAQ,MAAM;AAG9C,eAAW,CAAC,MAAM,KAAK,KAAK;AAAA,MAC1B,CAAC,aAAa,QAAQ,SAAS;AAAA,MAC/B,CAAC,kBAAkB,QAAQ,cAAc;AAAA,IAC3C,GAAY;AACV,UAAI,UAAU,WAAc,CAAC,OAAO,cAAc,KAAK,KAAK,QAAQ,IAAI;AACtE,cAAM,IAAIC,iBAAgB,GAAG,IAAI,yCAAyC,KAAK,EAAE;AAAA,MACnF;AAAA,IACF;AACA,UAAM,gBAAgB,KAAK,IAAI,QAAQ,aAAa,eAAe,IAAI,OAAO,IAAI;AAGlF,SAAK,iBAAiB,QAAQ,kBAAkB,gBAAgB;AAChE,SAAK,YAAY,KAAK,IAAI,eAAe,KAAK,KAAK,KAAK,iBAAiB,YAAY,CAAC;AAAA,EACxF;AAAA,EAEA,eAA4B;AAC1B,WAAO,EAAE,WAAW,MAAM,gBAAgB,KAAK,gBAAgB,gBAAgB,KAAK;AAAA,EACtF;AAAA,EAEA,MAAM,aACJ,KACA,OAC2C;AAC3C,UAAM,YAAY,iBAAiB,KAAK,QAAQ,GAAG;AACnD,UAAM,OAAO,IAAI;AAAA,MACf,KAAK;AAAA,MACL,KAAK;AAAA,MACL;AAAA,MACA,KAAK;AAAA,MACL,KAAK;AAAA,IACP;AACA,QAAI;AACF,YAAM,MAAM,IAAI;AAChB,aAAO,MAAM,KAAK,OAAO;AAAA,IAC3B,SAAS,KAAK;AACZ,YAAM,KAAK,MAAM;AAGjB,UAAI,sBAAsB,GAAG,GAAG;AAC9B,cAAM,IAAI;AAAA,UACR,2CAA2C,IAAI,OAAO,IAAI,IAAI,UAAU;AAAA,QAC1E;AAAA,MACF;AACA,UAAI,kBAAkB,GAAG,KAAK,qBAAqB,GAAG,KAAK,gBAAgB,GAAG,GAAG;AAC/E,cAAM;AAAA,MACR;AACA,YAAM,KAAK,SAAS,GAAG;AAAA,IACzB;AAAA,EACF;AAAA,EAEA,MAAM,SAAS,KAAa,QAAgB,QAAqC;AAC/E,QAAI,CAAC,OAAO,UAAU,MAAM,KAAK,CAAC,OAAO,UAAU,MAAM,KAAK,SAAS,KAAK,SAAS,GAAG;AACtF,YAAM,IAAIA,iBAAgB,wBAAwB,MAAM,WAAW,MAAM,EAAE;AAAA,IAC7E;AACA,UAAM,YAAY,iBAAiB,KAAK,QAAQ,GAAG;AACnD,QAAI,WAAW,EAAG,QAAO,IAAI,WAAW,CAAC;AACzC,QAAI;AACF,YAAM,MAAM,MAAM,KAAK,OAAO;AAAA,QAC5B,IAAI,iBAAiB;AAAA,UACnB,QAAQ,KAAK;AAAA,UACb,KAAK;AAAA,UACL,OAAO,SAAS,MAAM,IAAI,SAAS,SAAS,CAAC;AAAA,QAC/C,CAAC;AAAA,MACH;AACA,YAAM,QAAQ,MAAM,QAAQ,IAAI,IAAI;AAEpC,UAAI,MAAM,WAAW,QAAQ;AAC3B,cAAM,IAAIA;AAAA,UACR,UAAU,MAAM,KAAK,SAAS,MAAM,wBAAwB,MAAM,MAAM;AAAA,QAC1E;AAAA,MACF;AACA,aAAO;AAAA,IACT,SAAS,KAAK;AACZ,YAAM,KAAK,aAAa,KAAK,GAAG;AAAA,IAClC;AAAA,EACF;AAAA,EAEA,MAAM,QAAQ,KAAa,UAAgE;AACzF,UAAM,YAAY,iBAAiB,KAAK,QAAQ,GAAG;AACnD,QAAI,YAAY,GAAG;AAEjB,UAAI;AACF,cAAM,OAAO,MAAM,KAAK,OAAO;AAAA,UAC7B,IAAI,kBAAkB,EAAE,QAAQ,KAAK,QAAQ,KAAK,UAAU,CAAC;AAAA,QAC/D;AACA,eAAO,EAAE,OAAO,IAAI,WAAW,CAAC,GAAG,MAAM,KAAK,iBAAiB,EAAE;AAAA,MACnE,SAAS,KAAK;AACZ,cAAM,KAAK,aAAa,KAAK,GAAG;AAAA,MAClC;AAAA,IACF;AACA,QAAI;AACF,YAAM,MAAM,MAAM,KAAK,OAAO;AAAA,QAC5B,IAAI,iBAAiB,EAAE,QAAQ,KAAK,QAAQ,KAAK,WAAW,OAAO,UAAU,QAAQ,GAAG,CAAC;AAAA,MAC3F;AACA,YAAM,QAAQ,MAAM,QAAQ,IAAI,IAAI;AACpC,UAAI,OAAO,sBAAsB,IAAI,YAAY;AACjD,UAAI,SAAS,QAAW;AAItB,YAAI,MAAM,WAAW,UAAU;AAC7B,gBAAM,OAAO,MAAM,KAAK,OAAO;AAAA,YAC7B,IAAI,kBAAkB,EAAE,QAAQ,KAAK,QAAQ,KAAK,UAAU,CAAC;AAAA,UAC/D;AACA,iBAAO,KAAK,iBAAiB,MAAM;AAAA,QACrC,OAAO;AACL,iBAAO,MAAM;AAAA,QACf;AAAA,MACF;AACA,aAAO,EAAE,OAAO,KAAK;AAAA,IACvB,SAAS,KAAK;AACZ,YAAM,KAAK,aAAa,KAAK,GAAG;AAAA,IAClC;AAAA,EACF;AAAA,EAEA,MAAM,OAAO,KAA4B;AAEvC,QAAI;AACF,YAAM,KAAK,OAAO;AAAA,QAChB,IAAI,oBAAoB,EAAE,QAAQ,KAAK,QAAQ,KAAK,iBAAiB,KAAK,QAAQ,GAAG,EAAE,CAAC;AAAA,MAC1F;AAAA,IACF,SAAS,KAAK;AACZ,YAAM,KAAK,SAAS,GAAG;AAAA,IACzB;AAAA,EACF;AAAA,EAEA,OAAO,KAAK,KAAwC;AAClD,UAAM,SAAS,oBAAoB,KAAK,QAAQ,GAAG;AACnD,QAAI;AACJ,OAAG;AACD,UAAI;AACJ,UAAI;AACF,cAAM,MAAM,KAAK,OAAO;AAAA,UACtB,IAAI,qBAAqB;AAAA,YACvB,QAAQ,KAAK;AAAA,YACb,QAAQ;AAAA,YACR,mBAAmB;AAAA,UACrB,CAAC;AAAA,QACH;AAAA,MACF,SAAS,KAAK;AACZ,cAAM,KAAK,SAAS,GAAG;AAAA,MACzB;AACA,iBAAW,OAAO,IAAI,YAAY,CAAC,GAAG;AACpC,YAAI,IAAI,QAAQ,OAAW;AAC3B,cAAM,aAAa,uBAAuB,QAAQ,IAAI,GAAG;AACzD,YAAI,eAAe,MAAM;AACvB,gBAAM,EAAE,WAAW,IAAI,WAAW,SAAS,IAAI,SAAS,WAAW;AAAA,QACrE;AAAA,MACF;AACA,cAAQ,IAAI,gBAAgB,OAAO,IAAI,wBAAwB;AAAA,IACjE,SAAS,UAAU;AAAA,EACrB;AAAA;AAAA,EAGQ,aAAa,KAAc,KAAsB;AACvD,QAAI,kBAAkB,GAAG,EAAG,QAAO;AACnC,QAAI,WAAW,GAAG,GAAG;AACnB,aAAO,IAAI,cAAc,uBAAuB,IAAI,OAAO,IAAI,IAAI,UAAU,EAAE;AAAA,IACjF;AAGA,QAAI,eAAe,GAAG,GAAG;AACvB,aAAO,IAAIA,iBAAgB,2BAA2B,IAAI,OAAO,IAAI,IAAI,UAAU,EAAE;AAAA,IACvF;AACA,WAAO,KAAK,SAAS,GAAG;AAAA,EAC1B;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAQQ,SAAS,KAAuB;AACtC,QAAI,YAAY,GAAG,GAAG;AACpB,aAAO,IAAI;AAAA,QACT,uBAAwB,KAAkC,QAAQ,SAAS;AAAA,QAC3E,EAAE,OAAO,IAAI;AAAA,MACf;AAAA,IACF;AACA,WAAO;AAAA,EACT;AACF;AAGA,SAAS,YAAY,OAA8B,OAA2B;AAC5E,QAAM,MAAM,IAAI,WAAW,KAAK;AAChC,MAAI,SAAS;AACb,aAAW,KAAK,OAAO;AACrB,QAAI,IAAI,GAAG,MAAM;AACjB,cAAU,EAAE;AAAA,EACd;AACA,SAAO;AACT;AAUA,IAAM,kBAAN,MAA0C;AAAA,EASxC,YACmB,QACA,QACA,WACA,WACA,gBACjB;AALiB;AACA;AACA;AACA;AACA;AAAA,EAChB;AAAA,EALgB;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EAbF,OAAa,WAAW,QAAQ;AAAA,EAChC,UAAwB,CAAC;AAAA,EAClC,aAAa;AAAA,EACb,QAAQ;AAAA,EACR;AAAA,EACA,aAAa;AAAA,EACJ,QAA4D,CAAC;AAAA,EAU9E,MAAM,MAAM,OAAkC;AAC5C,QAAI,MAAM,WAAW,EAAG;AACxB,SAAK,SAAS,MAAM;AACpB,QAAI,KAAK,QAAQ,KAAK,gBAAgB;AAEpC,YAAM,IAAIA,iBAAgB,iCAAiC,KAAK,cAAc,EAAE;AAAA,IAClF;AACA,SAAK,KAAK,OAAO,KAAK;AACtB,SAAK,QAAQ,KAAK,KAAK;AACvB,SAAK,cAAc,MAAM;AACzB,QAAI,KAAK,cAAc,KAAK,UAAW,OAAM,KAAK,UAAU;AAAA,EAC9D;AAAA;AAAA,EAGA,MAAc,YAA2B;AACvC,QAAI,KAAK,aAAa,QAAW;AAC/B,YAAMC,OAAM,MAAM,KAAK,OAAO;AAAA,QAC5B,IAAI,6BAA6B,EAAE,QAAQ,KAAK,QAAQ,KAAK,KAAK,UAAU,CAAC;AAAA,MAC/E;AACA,UAAIA,KAAI,aAAa,QAAW;AAC9B,cAAM,IAAI,eAAe,+CAA+C;AAAA,MAC1E;AACA,WAAK,WAAWA,KAAI;AAAA,IACtB;AACA,UAAM,OAAO,YAAY,KAAK,SAAS,KAAK,UAAU;AACtD,SAAK,QAAQ,SAAS;AACtB,SAAK,aAAa;AAClB,SAAK,cAAc;AACnB,QAAI,KAAK,aAAa,cAAc;AAGlC,YAAM,IAAID,iBAAgB,oCAAoC,YAAY,aAAa;AAAA,IACzF;AACA,UAAM,MAAM,MAAM,KAAK,OAAO;AAAA,MAC5B,IAAI,kBAAkB;AAAA,QACpB,QAAQ,KAAK;AAAA,QACb,KAAK,KAAK;AAAA,QACV,UAAU,KAAK;AAAA,QACf,YAAY,KAAK;AAAA,QACjB,MAAM;AAAA,MACR,CAAC;AAAA,IACH;AACA,SAAK,MAAM,KAAK,EAAE,MAAM,IAAI,MAAM,YAAY,KAAK,WAAW,CAAC;AAAA,EACjE;AAAA;AAAA,EAGA,MAAM,SAAoD;AACxD,UAAM,SAAS,KAAK,KAAK,OAAO,KAAK;AACrC,QAAI,KAAK,aAAa,QAAW;AAC/B,YAAM;AAAA,QACJ,KAAK;AAAA,QACL,IAAI,iBAAiB;AAAA,UACnB,QAAQ,KAAK;AAAA,UACb,KAAK,KAAK;AAAA,UACV,MAAM,YAAY,KAAK,SAAS,KAAK,UAAU;AAAA,UAC/C,aAAa;AAAA;AAAA,QACf,CAAC;AAAA,MACH;AACA,aAAO,EAAE,MAAM,KAAK,OAAO,OAAO;AAAA,IACpC;AACA,QAAI,KAAK,aAAa,EAAG,OAAM,KAAK,UAAU;AAC9C,UAAM;AAAA,MACJ,KAAK;AAAA,MACL,IAAI,+BAA+B;AAAA,QACjC,QAAQ,KAAK;AAAA,QACb,KAAK,KAAK;AAAA,QACV,UAAU,KAAK;AAAA,QACf,iBAAiB,EAAE,OAAO,KAAK,MAAM;AAAA,QACrC,aAAa;AAAA;AAAA,MACf,CAAC;AAAA,IACH;AACA,SAAK,WAAW;AAChB,WAAO,EAAE,MAAM,KAAK,OAAO,OAAO;AAAA,EACpC;AAAA;AAAA;AAAA,EAIA,MAAM,QAAuB;AAC3B,QAAI,KAAK,aAAa,OAAW;AACjC,UAAM,KAAK,KAAK;AAChB,SAAK,WAAW;AAChB,QAAI;AACF,YAAM,KAAK,OAAO;AAAA,QAChB,IAAI,4BAA4B,EAAE,QAAQ,KAAK,QAAQ,KAAK,KAAK,WAAW,UAAU,GAAG,CAAC;AAAA,MAC5F;AAAA,IACF,QAAQ;AAAA,IAER;AAAA,EACF;AACF;AAMA,eAAe,QAAQ,MAAiD;AACtE,MAAI,SAAS,QAAW;AACtB,UAAM,IAAI,cAAc,qCAAqC;AAAA,EAC/D;AACA,SAAO,KAAK,qBAAqB;AACnC;;;AK9YA;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA,kBAAAE;AAAA,EACA,sBAAAC;AAAA,EACA,yBAAAC;AAAA,OACK;AAEP;AAAA,EACE,oBAAAC;AAAA,EACA,wBAAAC;AAAA,EACA,oBAAAC;AAAA,OAEK;AAgBP,IAAM,UAAN,MAA6C;AAAA,EAG3C,YACmB,QACA,QACjB;AAFiB;AACA;AAAA,EAChB;AAAA,EAFgB;AAAA,EACA;AAAA,EAJV,QAAQ;AAAA,EAOjB,MAAM,KAAK,KAAwC;AACjD,QAAI;AACJ,QAAI;AACF,YAAM,MAAM,KAAK,OAAO,KAAK,IAAIC,kBAAiB,EAAE,QAAQ,KAAK,QAAQ,KAAK,IAAI,CAAC,CAAC;AAAA,IACtF,SAAS,KAAK;AACZ,UAAI,WAAW,GAAG,EAAG,QAAO;AAC5B,YAAM,SAAS,GAAG;AAAA,IACpB;AAEA,SAAK,IAAI,iBAAiB,KAAK,eAAe;AAC5C,YAAM,IAAI;AAAA,QACR,mBAAmB,IAAI,aAAa,iBAAiB,aAAa;AAAA,MACpE;AAAA,IACF;AACA,QAAI,IAAI,SAAS,QAAW;AAC1B,YAAM,IAAI,eAAe,sCAAsC,GAAG,EAAE;AAAA,IACtE;AACA,UAAM,QAAQ,MACZ,IAAI,KACJ,qBAAqB;AACvB,WAAO,EAAE,OAAO,SAAS,IAAI,QAAQ,GAAG;AAAA,EAC1C;AAAA,EAEA,MAAM,MACJ,KACA,MACA,QACe;AACf,QAAI;AAEF,YAAM;AAAA,QACJ,KAAK;AAAA,QACL,IAAIC,kBAAiB;AAAA,UACnB,QAAQ,KAAK;AAAA,UACb,KAAK;AAAA,UACL,MAAM;AAAA,UACN,aAAa;AAAA,UACb,aAAa,WAAW,WAAW,MAAM;AAAA,UACzC,SAAS,WAAW,WAAW,SAAY,OAAO;AAAA,QACpD,CAAC;AAAA,MACH;AAAA,IACF,SAAS,KAAK;AAEZ,UAAI,sBAAsB,GAAG,GAAG;AAC9B,cAAM,IAAIC,oBAAmB,6BAA6B,GAAG,EAAE;AAAA,MACjE;AACA,YAAM,SAAS,GAAG;AAAA,IACpB;AAAA,EACF;AAAA,EAEA,OAAO,SAAS,QAAuC;AACrD,QAAI;AACJ,OAAG;AACD,UAAI;AACJ,UAAI;AACF,cAAM,MAAM,KAAK,OAAO;AAAA,UACtB,IAAIC,sBAAqB;AAAA,YACvB,QAAQ,KAAK;AAAA,YACb,QAAQ;AAAA,YACR,mBAAmB;AAAA,UACrB,CAAC;AAAA,QACH;AAAA,MACF,SAAS,KAAK;AACZ,cAAM,SAAS,GAAG;AAAA,MACpB;AACA,iBAAW,OAAO,IAAI,YAAY,CAAC,GAAG;AACpC,YAAI,IAAI,QAAQ,OAAW,OAAM,IAAI;AAAA,MACvC;AACA,cAAQ,IAAI,gBAAgB,OAAO,IAAI,wBAAwB;AAAA,IACjE,SAAS,UAAU;AAAA,EACrB;AACF;AAGA,SAAS,SAAS,KAAuB;AACvC,MAAI,YAAY,GAAG,GAAG;AACpB,WAAO,IAAIC;AAAA,MACT,uBAAwB,KAAkC,QAAQ,SAAS;AAAA,MAC3E,EAAE,OAAO,IAAI;AAAA,IACf;AAAA,EACF;AACA,SAAO;AACT;AAEO,IAAM,mBAAN,cAA+B,oBAAoB;AAAA,EACxD,YAAY,SAAkC;AAC5C;AAAA,MACE,IAAI,QAAQ,QAAQ,QAAQ,QAAQ,MAAM;AAAA,MAC1CC,uBAAsB,QAAQ,MAAM;AAAA,MACpC,QAAQ,QAAQ,MAAc,KAAK,IAAI;AAAA,IACzC;AAAA,EACF;AACF;;;ANnFO,IAAM,yBAAyB;AAAA,EACpC;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AACF;AAGA,IAAM,kBAAkB,CAAC,UAAU,YAAY,aAAa,aAAa;AAGzE,SAAS,cACP,MACA,SACA,MACA,MACM;AACN,MAAI,YAAY,QAAQ,OAAO,YAAY,UAAU;AACnD,UAAM,IAAIC;AAAA,MACR,GAAG,IAAI,uCAAkC,YAAY,OAAO,SAAS,OAAO,OAAO;AAAA,IACrF;AAAA,EACF;AACA,QAAM,UAAU,OAAO,KAAK,OAAO,EAAE,OAAO,CAAC,MAAM,CAAC,KAAK,SAAS,CAAC,CAAC;AACpE,MAAI,QAAQ,SAAS,GAAG;AACtB,UAAM,OAAO,CAAC,OAAkC,GAAG,IAAI,CAAC,MAAM,KAAK,CAAC,IAAI,EAAE,KAAK,IAAI;AACnF,UAAM,IAAIA;AAAA,MACR,GAAG,IAAI,kBAAkB,KAAK,OAAO,CAAC,cAAc,KAAK,IAAI,CAAC,KAAK,IAAI;AAAA,IACzE;AAAA,EACF;AACF;AAEO,IAAM,YAAN,MAA0C;AAAA,EAGtC;AAAA,EACA;AAAA;AAAA,EAEA;AAAA,EAET,YAAY,SAA2B;AACrC,kBAAc,aAAa,SAAS,wBAAwB,+BAA+B;AAC3F,QAAI,QAAQ,WAAW,UAAa,QAAQ,WAAW,MAAM;AAK3D,YAAM,UAAU,gBAAgB,OAAO,CAAC,MAAM,QAAQ,CAAC,MAAM,MAAS;AACtE,UAAI,QAAQ,SAAS,GAAG;AACtB,cAAM,IAAIA;AAAA,UACR,iCAAiC,gBAAgB,IAAI,CAAC,MAAM,KAAK,CAAC,IAAI,EAAE,KAAK,KAAK,CAAC,yCAC1D,QAAQ,IAAI,CAAC,MAAM,KAAK,CAAC,IAAI,EAAE,KAAK,IAAI,CAAC;AAAA,QAEpE;AAAA,MACF;AACA,WAAK,SAAS,QAAQ;AAAA,IACxB,OAAO;AACL,WAAK,SAAS,IAAI,SAAS;AAAA,QACzB,GAAI,QAAQ,WAAW,SAAY,CAAC,IAAI,EAAE,QAAQ,QAAQ,OAAO;AAAA,QACjE,GAAI,QAAQ,aAAa,SAAY,CAAC,IAAI,EAAE,UAAU,QAAQ,SAAS;AAAA,QACvE,GAAI,QAAQ,cAAc,SAAY,CAAC,IAAI,EAAE,gBAAgB,QAAQ,UAAU;AAAA,QAC/E,GAAI,QAAQ,gBAAgB,SAAY,CAAC,IAAI,EAAE,aAAa,QAAQ,YAAY;AAAA,MAClF,CAAC;AAAA,IACH;AACA,UAAM,SAAS;AAAA,MACb,QAAQ,KAAK;AAAA,MACb,QAAQ,QAAQ;AAAA,MAChB,GAAI,QAAQ,WAAW,SAAY,CAAC,IAAI,EAAE,QAAQ,QAAQ,OAAO;AAAA,IACnE;AACA,SAAK,UAAU,IAAI,gBAAgB;AAAA,MACjC,GAAG;AAAA,MACH,GAAI,QAAQ,mBAAmB,SAAY,CAAC,IAAI,EAAE,gBAAgB,QAAQ,eAAe;AAAA,MACzF,GAAI,QAAQ,cAAc,SAAY,CAAC,IAAI,EAAE,WAAW,QAAQ,UAAU;AAAA,IAC5E,CAAC;AACD,SAAK,WAAW,IAAI,iBAAiB;AAAA,MACnC,GAAG;AAAA,MACH,GAAI,QAAQ,QAAQ,SAAY,CAAC,IAAI,EAAE,KAAK,QAAQ,IAAI;AAAA,IAC1D,CAAC;AACD,mBAAe,IAAI;AAAA,EACrB;AACF;",
6
+ "names": ["ValidationError", "ValidationError", "ValidationError", "res", "TransientError", "WriteConflictError", "normalizeObjectPrefix", "GetObjectCommand", "ListObjectsV2Command", "PutObjectCommand", "GetObjectCommand", "PutObjectCommand", "WriteConflictError", "ListObjectsV2Command", "TransientError", "normalizeObjectPrefix", "ValidationError"]
7
7
  }
package/dist/keys.d.ts CHANGED
@@ -8,7 +8,6 @@ export { normalizeObjectPrefix as normalizeS3Prefix } from '@cloudbitmaps/core/d
8
8
  export declare function segmentObjectPrefix(prefix: string | undefined, ref: SegmentRef): string;
9
9
  /** The full S3 key of one `.crbm` generation: `<segmentPrefix><gen>.crbm`. */
10
10
  export declare function storageObjectKey(prefix: string | undefined, key: GenKey): string;
11
- export { registryPrefix, registryObjectKey, registryListPrefix, parseRegistryKey, } from '@cloudbitmaps/core/driver-kit';
12
11
  /**
13
12
  * Parse a generation number out of a full object key, given its segment prefix, or `null` if it doesn't
14
13
  * match. Canonical decimal only — no leading zeros (so `…s.07.crbm` can't alias `…s.7.crbm`) and within
@@ -8,9 +8,11 @@
8
8
  *
9
9
  * **The atomic swap is offloaded to S3's conditional writes** (GA Nov 2024): `If-None-Match: *` for
10
10
  * create-only and `If-Match: <etag>` for compare-and-swap, so a concurrent writer between our read and our
11
- * PUT loses with a `412` → {@link WriteConflictError}. Reads are strongly consistent (S3, since 2020),
12
- * satisfying the registry's `strongRead` contract. The client is **injected**, exactly like
13
- * {@link S3StorageDriver}.
11
+ * PUT loses with a `412` → {@link WriteConflictError}. Each conditional PUT is sent once, with the SDK's retry off
12
+ * for it ({@link sendOnce}), so a `412` means another write got there first, never this one meeting itself after a
13
+ * lost response. A transient failure reaches the caller as {@link TransientError}: the write may or may not have
14
+ * landed, and the caller re-reads the row to learn where it stands. Reads are strongly consistent (S3, since 2020),
15
+ * satisfying the registry's `strongRead` contract. The client is **injected**, exactly like {@link S3StorageDriver}.
14
16
  *
15
17
  * **Deployment requirements** (a backend/policy that violates these silently corrupts the registry):
16
18
  * - The backend **must honor `If-Match`** (AWS S3; recent MinIO). One that returns ETags but ignores the
@@ -19,7 +21,8 @@
19
21
  * - The IAM principal needs **`s3:ListBucket`** on the bucket. Without it, `GetObject` on a missing key
20
22
  * returns `403` (not `404`), so the "absent segment → `null`" contract (and `create`'s bootstrap read)
21
23
  * breaks — and `list()` needs it regardless.
22
- * - **Do not apply an S3 lifecycle-expiration rule to the `registry/` prefix.** See {@link ObjectStoreRegistry}.
24
+ * - **Do not apply an S3 lifecycle-expiration rule to the `registry/` prefix that expires a current version**
25
+ * (`NoncurrentVersionExpiration` is safe). See {@link ObjectStoreRegistry}.
23
26
  */
24
27
  import { ObjectStoreRegistry } from '@cloudbitmaps/core/driver-kit';
25
28
  import { type S3Client } from '@aws-sdk/client-s3';
@@ -28,7 +31,7 @@ export interface S3RegistryDriverOptions {
28
31
  readonly client: S3Client;
29
32
  /** Target bucket (must already exist). */
30
33
  readonly bucket: string;
31
- /** Optional key prefix under which all registry objects live (e.g. `cloudroaring/`). */
34
+ /** Optional key prefix under which all registry objects live (e.g. `cloudbitmaps/`). */
32
35
  readonly prefix?: string;
33
36
  /** Injected clock for `createdAt`/`updatedAt`; defaults to `Date.now`. */
34
37
  readonly now?: () => number;
@@ -3,7 +3,7 @@
3
3
  *
4
4
  * Kept SDK-free and side-effect-free (they only read structural shapes — `err.name`,
5
5
  * `$metadata.httpStatusCode`, a `Content-Range` string) so the subtle S3-specific translation logic is
6
- * unit-testable without a live MinIO/S3 or even the AWS SDK. Shared AWS shapes come from `_shared/aws-errors`.
6
+ * unit-testable without a live MinIO/S3 or even the AWS SDK. The AWS error shapes come from `./aws-errors`.
7
7
  */
8
8
  /** A conditional `If-None-Match: *` PUT lost the write-once race (the object already existed). */
9
9
  export declare function isPreconditionFailed(err: unknown): boolean;
@@ -21,8 +21,9 @@ export declare function isNotFound(err: unknown): boolean;
21
21
  export declare function isInvalidRange(err: unknown): boolean;
22
22
  /**
23
23
  * A transient S3 fault that is safe to retry: throttling (`SlowDown` / 503), any 5xx, a dropped/timed-out
24
- * connection, or anything the SDK itself marks retryable. Excludes the deterministic outcomes above
25
- * (412/404/416) — those are caller-meaningful and must never be retried/reclassified.
24
+ * connection, a clock-skew refusal the SDK has corrected for, or anything the SDK itself marks retryable. Excludes
25
+ * the deterministic outcomes above (412/404/416) — those are caller-meaningful and must never be
26
+ * retried/reclassified.
26
27
  */
27
28
  export declare function isTransient(err: unknown): boolean;
28
29
  /**
@@ -0,0 +1,23 @@
1
+ /**
2
+ * `sendOnce` — send a conditional write exactly once, with the SDK's retry off for that one command.
3
+ *
4
+ * The SDK re-sends a request whose response it did not get: a timeout, a reset connection, a 5xx. For a conditional
5
+ * write that is the wrong thing to do. When the write landed and only its response was lost, the second send meets
6
+ * the first — `If-None-Match: *` finds the object it created, `If-Match` finds the ETag it replaced — and fails with
7
+ * `412`, which the driver can only report as a lost race. The caller is told its write lost when it won. Only the
8
+ * caller can find out which happened, by reading the pointer or listing the generations, so the write is sent once
9
+ * and a transient failure reaches it as `TransientError`.
10
+ *
11
+ * **How.** The client runs its retry as one middleware, `retryMiddleware`, at high priority in the `finalizeRequest`
12
+ * step, whatever retry strategy or `maxAttempts` it was built with. When a command is sent, its own middleware stack
13
+ * is merged over the client's, and an entry with the same name, step and priority that sets `override` replaces the
14
+ * client's. So this command runs with a pass-through where the retry was, and nothing about the client changes: a
15
+ * caller's own client keeps its configuration, and every other command sent through it keeps its retry.
16
+ *
17
+ * The options object passed to `send` matters too. A client built with `cacheMiddleware: true` reuses the handler it
18
+ * resolved for the first command of a class, and that handler holds the retry; `send` resolves afresh whenever it is
19
+ * given options, so the replacement always takes effect.
20
+ */
21
+ import type { $Command, S3Client, S3ClientResolvedConfig, ServiceInputTypes, ServiceOutputTypes } from '@aws-sdk/client-s3';
22
+ /** Send `command` once: its retry step is a pass-through, and the client's stays as it is for every other command. */
23
+ export declare function sendOnce<Input extends ServiceInputTypes, Output extends ServiceOutputTypes>(client: S3Client, command: $Command<Input, Output, S3ClientResolvedConfig, ServiceInputTypes, ServiceOutputTypes>): Promise<Output>;
package/dist/storage.d.ts CHANGED
@@ -5,7 +5,7 @@ export interface S3StorageDriverOptions {
5
5
  readonly client: S3Client;
6
6
  /** Target bucket (must already exist). */
7
7
  readonly bucket: string;
8
- /** Optional key prefix under which all objects live (e.g. `cloudroaring/`). */
8
+ /** Optional key prefix under which all objects live (e.g. `cloudbitmaps/`). */
9
9
  readonly prefix?: string;
10
10
  /**
11
11
  * Largest object this driver will write/advertise. Default = `partBytes × 10,000` (≈ 80 GiB at the default
@@ -13,7 +13,8 @@ export interface S3StorageDriverOptions {
13
13
  * auto-grows so 10,000 parts still cover it (raising peak write memory to ~one part); up to the 5 TiB S3 max.
14
14
  */
15
15
  readonly maxObjectBytes?: number;
16
- /** Multipart part size in bytes (default 8 MiB; clamped to the S3 5 MiB minimum). Tunes peak write memory. */
16
+ /** Multipart part size in bytes (default 8 MiB; a smaller value is raised to the S3 5 MiB minimum). Must be a
17
+ * positive safe integer. Tunes peak write memory. */
17
18
  readonly partBytes?: number;
18
19
  }
19
20
  export declare class S3StorageDriver implements IStorageDriver {
@@ -38,9 +39,10 @@ export declare class S3StorageDriver implements IStorageDriver {
38
39
  /** Map S3 read errors to the driver vocabulary; pass everything else through {@link mapError}. */
39
40
  private mapReadError;
40
41
  /**
41
- * Reclassify a transient S3 fault (throttle/5xx/dropped connection) as a retryable {@link TransientError}
42
- * so the retry decorator can ride it out; everything else propagates unchanged. The final fallback at every
43
- * `client.send` site, so callers and the decorator only ever see typed errors.
42
+ * Reclassify a transient S3 fault (throttle/5xx/dropped connection) as a retryable {@link TransientError},
43
+ * so the store's read retry can ride it out and a write's caller can tell it from a deterministic failure;
44
+ * everything else propagates unchanged. The final fallback at every `client.send` site, so callers and the
45
+ * read retry only ever see typed errors.
44
46
  */
45
47
  private mapError;
46
48
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@cloudbitmaps/s3",
3
- "version": "0.10.0",
3
+ "version": "0.11.0",
4
4
  "description": "S3 and S3-compatible storage (R2, MinIO, Ceph, Wasabi, B2) for CloudBitmaps",
5
5
  "keywords": [
6
6
  "cloudbitmaps",
@@ -53,7 +53,7 @@
53
53
  },
54
54
  "dependencies": {
55
55
  "@aws-sdk/client-s3": ">=3.645.0 <4",
56
- "@cloudbitmaps/core": "^0.10.0"
56
+ "@cloudbitmaps/core": "^0.11.0"
57
57
  },
58
58
  "devDependencies": {
59
59
  "@types/node": "^22.0.0",