git-fs-s3 0.15.0 → 0.15.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +5 -3
- package/dist/fetch-s3.js +137 -160
- package/dist/fetch-s3.js.map +1 -1
- package/dist/http.js +905 -849
- package/dist/http.js.map +1 -1
- package/dist/index.js +578 -658
- package/dist/index.js.map +1 -1
- package/dist/memory-M2cCuVF-.js +61 -0
- package/dist/memory-M2cCuVF-.js.map +1 -0
- package/dist/node.js +75 -71
- package/dist/node.js.map +1 -1
- package/dist/object-size-BJAsnZzw.js +191 -0
- package/dist/object-size-BJAsnZzw.js.map +1 -0
- package/dist/ops.js +662 -692
- package/dist/ops.js.map +1 -1
- package/dist/refs-Cb_NfKWI.js +550 -0
- package/dist/refs-Cb_NfKWI.js.map +1 -0
- package/dist/s3.js +111 -116
- package/dist/s3.js.map +1 -1
- package/dist/semaphore-BviNoQuJ.js +646 -0
- package/dist/semaphore-BviNoQuJ.js.map +1 -0
- package/dist/types/git-errors.d.ts +6 -0
- package/dist/types/http/repack.d.ts +7 -0
- package/dist/types/index.d.ts +1 -1
- package/package.json +18 -23
- package/dist/chunk-HLYCSNJ2.js +0 -53
- package/dist/chunk-HLYCSNJ2.js.map +0 -1
- package/dist/chunk-IAKZRWBZ.js +0 -598
- package/dist/chunk-IAKZRWBZ.js.map +0 -1
- package/dist/chunk-OKBQISUC.js +0 -442
- package/dist/chunk-OKBQISUC.js.map +0 -1
- package/dist/chunk-T5M5EYCZ.js +0 -143
- package/dist/chunk-T5M5EYCZ.js.map +0 -1
package/README.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# git-fs-s3
|
|
2
2
|
|
|
3
3
|
[](https://www.npmjs.com/package/git-fs-s3)
|
|
4
|
-
[](https://github.com/nandan-varma/pushstack/actions/workflows/ci.yml)
|
|
5
5
|
[](LICENSE)
|
|
6
6
|
|
|
7
7
|
An [isomorphic-git](https://isomorphic-git.org) filesystem backend for S3-compatible object storage — AWS S3, Cloudflare R2, MinIO, Backblaze B2. Run git repositories on serverless platforms with **no disk, no git binary, and no state between invocations**.
|
|
@@ -16,6 +16,8 @@ Extracted from the git layer of a production git-hosting service, where it serve
|
|
|
16
16
|
- **`/http`** — a git smart-HTTP protocol handler (`upload-pack`/`receive-pack`) built on it.
|
|
17
17
|
- **`/ops`** — higher-level git-hosting operations (branches, commits, diffs, history, merge) built on top of that.
|
|
18
18
|
|
|
19
|
+
[Documentation](https://docs.git.nandan.fyi/docs/libraries/git-fs-s3) — guides, runtime support and API reference.
|
|
20
|
+
|
|
19
21
|
## Why
|
|
20
22
|
|
|
21
23
|
isomorphic-git removed the need for a native `git` binary, but it still expects a filesystem. On Lambda, Vercel, or Cloudflare Workers you don't have a durable one. This package maps the fs contract git actually uses onto object-storage keys, so a bare repository lives entirely in a bucket:
|
|
@@ -195,7 +197,7 @@ Wraps any store with an in-process LRU read cache (git objects are content-addre
|
|
|
195
197
|
|
|
196
198
|
- `maxBytes` — cache budget, default 50 MiB. `maxEntryBytes` — largest admissible entry, default a tenth of `maxBytes`, so one huge pack can't evict the working set.
|
|
197
199
|
- `ttlMs` — entry TTL, default 60 s
|
|
198
|
-
- `ttlForKey(key)` — per-key/prefix TTL override in ms (applies to `get`/`head`/`list`); return `undefined` to fall back to `ttlMs`. A single long `ttlMs` is right for content-addressed object keys (the same key's bytes never change) but wrong for mutable ones — a ref's value moves on every push, and a directory listing (`objects/pack/`) grows on every push, while the *key* naming them stays the same. This cache is in-process with no cross-instance invalidation, so without an override a warm instance that already cached a ref, or a pack directory's listing, can keep serving that pre-push view for the rest of `ttlMs` even though nothing is wrong with its own invalidation logic — it just never re-checked. See the [caching guide](https://nandan
|
|
200
|
+
- `ttlForKey(key)` — per-key/prefix TTL override in ms (applies to `get`/`head`/`list`); return `undefined` to fall back to `ttlMs`. A single long `ttlMs` is right for content-addressed object keys (the same key's bytes never change) but wrong for mutable ones — a ref's value moves on every push, and a directory listing (`objects/pack/`) grows on every push, while the *key* naming them stays the same. This cache is in-process with no cross-instance invalidation, so without an override a warm instance that already cached a ref, or a pack directory's listing, can keep serving that pre-push view for the rest of `ttlMs` even though nothing is wrong with its own invalidation logic — it just never re-checked. See the [caching guide](https://docs.git.nandan.fyi/docs/libraries/git-fs-s3/guides/caching/) for the full reasoning.
|
|
199
201
|
- `cacheMisses` — also cache "not found" results. Big win for loose-object probes on packed repos; only safe when this process is the sole writer.
|
|
200
202
|
- `cacheLists` — also cache `list()` results (readdir + existence probes). Writes through the store keep listings consistent, with one deliberate asymmetry: a non-empty `limit: 1` probe ("this directory exists") survives writes underneath it, since a write can't make a directory stop existing. After external writes, call `invalidate(prefix)`.
|
|
201
203
|
- `coalesce` — collapse concurrent `get`/`head`/`list` calls for the same key into one backend request. Default on.
|
|
@@ -265,7 +267,7 @@ Every client-supplied ref name in a push is validated with `isSafeFullRefName` (
|
|
|
265
267
|
|
|
266
268
|
`HttpHooks` (`{ step?, onWarn? }`), accepted by every handler, is the instrumentation seam: `step(label, fn)` wraps a timed sub-step — drop in an app's own request-scoped timer — and `onWarn(message, error)` surfaces non-fatal problems (a missing object during a reachability walk, a failed repack). `handleUploadPack`'s `beforeWalk` option is where to wire loose-object detection (`GitFs.detectLooseObjects`, above) so a fully packed repo's reachability walk doesn't pay a doomed loose-object probe per object.
|
|
267
269
|
|
|
268
|
-
`repackRepository(repo, options?, hooks?)` — also exported standalone for out-of-band maintenance (clearing a backlog outside a live push) — consolidates all packs into one once `objects/pack/` crosses `options.threshold` (default `REPACK_PACK_COUNT_THRESHOLD = 4`) packs, verifying every object's SHA-1 independently before writing (isomorphic-git's *packed*-object read path never checks a resolved delta's content against the requested oid — only the loose-object branch does). Returns the gitdir-relative paths of the `.pack`/`.idx` files it removed locally; if this repo's storage is also synced elsewhere, delete those same paths there too, or a reader served by the stale copy re-fetches objects that are gone from the pack it expects them in.
|
|
270
|
+
`repackRepository(repo, options?, hooks?)` — also exported standalone for out-of-band maintenance (clearing a backlog outside a live push) — consolidates all packs into one once `objects/pack/` crosses `options.threshold` (default `REPACK_PACK_COUNT_THRESHOLD = 4`) packs, verifying every object's SHA-1 independently before writing (isomorphic-git's *packed*-object read path never checks a resolved delta's content against the requested oid — only the loose-object branch does). Returns the gitdir-relative paths of the `.pack`/`.idx` files it removed locally; if this repo's storage is also synced elsewhere, delete those same paths there too, or a reader served by the stale copy re-fetches objects that are gone from the pack it expects them in. A failed consolidation rejects — deciding whether that is fatal (an explicit maintenance call) or best-effort (a repack after a write that already landed, as `applyReceivePack` does) is the caller's.
|
|
269
271
|
|
|
270
272
|
## App-layer git operations
|
|
271
273
|
|
package/dist/fetch-s3.js
CHANGED
|
@@ -1,174 +1,151 @@
|
|
|
1
|
-
// src/stores/fetch-s3.ts
|
|
2
1
|
import { AwsClient } from "aws4fetch";
|
|
2
|
+
//#region src/stores/fetch-s3.ts
|
|
3
|
+
/**
|
|
4
|
+
* An S3-compatible store (AWS S3, Cloudflare R2, MinIO, …) over the REST API,
|
|
5
|
+
* signed with `aws4fetch` (optional peer dependency) and sent with plain
|
|
6
|
+
* `fetch` — no AWS SDK, so it runs on any runtime with Web APIs (Node, Vercel
|
|
7
|
+
* Functions, Workers). Node's built-in fetch pools keep-alive connections.
|
|
8
|
+
*
|
|
9
|
+
* Retries and the circuit breaker are not here: wrap the store in
|
|
10
|
+
* `createRetryStore`. Every failed request throws an {@link S3HttpError}
|
|
11
|
+
* carrying the status in the `$metadata.httpStatusCode` shape that
|
|
12
|
+
* `isRetryableError` classifies (5xx/429 retry); a stalled request aborts with
|
|
13
|
+
* a `TimeoutError`, which is retried too.
|
|
14
|
+
*/
|
|
15
|
+
/** A non-2xx response (404s never throw — they map to `null`). */
|
|
3
16
|
var S3HttpError = class extends Error {
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
this.$metadata = { httpStatusCode: status };
|
|
11
|
-
}
|
|
17
|
+
$metadata;
|
|
18
|
+
constructor(operation, key, status, detail) {
|
|
19
|
+
super(`S3 ${operation} ${key} failed: HTTP ${status}${detail ? ` — ${detail}` : ""}`);
|
|
20
|
+
this.name = status === 429 ? "SlowDown" : "S3HttpError";
|
|
21
|
+
this.$metadata = { httpStatusCode: status };
|
|
22
|
+
}
|
|
12
23
|
};
|
|
13
24
|
function encodeKey(key) {
|
|
14
|
-
|
|
25
|
+
return key.split("/").map(encodeURIComponent).join("/");
|
|
15
26
|
}
|
|
16
27
|
function decodeXml(text) {
|
|
17
|
-
|
|
18
|
-
/&#(\d+);/g,
|
|
19
|
-
(_, code) => String.fromCodePoint(Number(code))
|
|
20
|
-
).replace(
|
|
21
|
-
/&#x([0-9a-f]+);/gi,
|
|
22
|
-
(_, code) => String.fromCodePoint(Number.parseInt(code, 16))
|
|
23
|
-
).replace(/&/g, "&");
|
|
28
|
+
return text.replace(/</g, "<").replace(/>/g, ">").replace(/"/g, "\"").replace(/'/g, "'").replace(/&#(\d+);/g, (_, code) => String.fromCodePoint(Number(code))).replace(/&#x([0-9a-f]+);/gi, (_, code) => String.fromCodePoint(Number.parseInt(code, 16))).replace(/&/g, "&");
|
|
24
29
|
}
|
|
25
30
|
function xmlValues(xml, tag) {
|
|
26
|
-
|
|
27
|
-
|
|
31
|
+
const pattern = new RegExp(`<${tag}>([\\s\\S]*?)</${tag}>`, "g");
|
|
32
|
+
return [...xml.matchAll(pattern)].map((match) => decodeXml(match[1] ?? ""));
|
|
28
33
|
}
|
|
34
|
+
/** Parses one ListObjectsV2 response page (exported for tests). */
|
|
29
35
|
function parseListObjectsV2(xml) {
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
const nextToken = truncated ? xmlValues(xml, "NextContinuationToken")[0] : void 0;
|
|
39
|
-
return { objects, prefixes, nextToken };
|
|
36
|
+
return {
|
|
37
|
+
objects: xmlValues(xml, "Contents").map((entry) => ({
|
|
38
|
+
key: xmlValues(entry, "Key")[0] ?? "",
|
|
39
|
+
size: Number(xmlValues(entry, "Size")[0] ?? 0)
|
|
40
|
+
})),
|
|
41
|
+
prefixes: xmlValues(xml, "CommonPrefixes").flatMap((entry) => xmlValues(entry, "Prefix")),
|
|
42
|
+
nextToken: xmlValues(xml, "IsTruncated")[0] === "true" ? xmlValues(xml, "NextContinuationToken")[0] : void 0
|
|
43
|
+
};
|
|
40
44
|
}
|
|
41
45
|
function createFetchS3Store(options) {
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
for (const p of page.prefixes) prefixes.add(p);
|
|
144
|
-
if (limit !== void 0 && objects.length + prefixes.size >= limit)
|
|
145
|
-
break;
|
|
146
|
-
token = page.nextToken;
|
|
147
|
-
} while (token);
|
|
148
|
-
return { objects, prefixes: [...prefixes] };
|
|
149
|
-
},
|
|
150
|
-
async copy(fromKey, toKey) {
|
|
151
|
-
const response = await send(
|
|
152
|
-
objectUrl(toKey),
|
|
153
|
-
{
|
|
154
|
-
method: "PUT",
|
|
155
|
-
headers: {
|
|
156
|
-
"x-amz-copy-source": `/${encodeURIComponent(options.bucket)}/${encodeKey(fromKey)}`
|
|
157
|
-
}
|
|
158
|
-
},
|
|
159
|
-
BODY_TIMEOUT_MS
|
|
160
|
-
);
|
|
161
|
-
if (!response.ok) return fail("COPY", toKey, response);
|
|
162
|
-
const body = await response.text();
|
|
163
|
-
if (body.includes("<Error>")) {
|
|
164
|
-
throw new S3HttpError("COPY", toKey, 500, body.slice(0, 300));
|
|
165
|
-
}
|
|
166
|
-
}
|
|
167
|
-
};
|
|
46
|
+
const BODY_TIMEOUT_MS = options.bodyTimeoutMs ?? 3e4;
|
|
47
|
+
const METADATA_TIMEOUT_MS = options.metadataTimeoutMs ?? 1e4;
|
|
48
|
+
const client = new AwsClient({
|
|
49
|
+
accessKeyId: options.accessKeyId,
|
|
50
|
+
secretAccessKey: options.secretAccessKey,
|
|
51
|
+
service: "s3",
|
|
52
|
+
region: options.region ?? "auto",
|
|
53
|
+
retries: 0
|
|
54
|
+
});
|
|
55
|
+
const doFetch = options.fetch ?? fetch;
|
|
56
|
+
const bucketUrl = `${options.endpoint.replace(/\/+$/, "")}/${encodeURIComponent(options.bucket)}`;
|
|
57
|
+
async function send(url, init, timeoutMs) {
|
|
58
|
+
const request = await client.sign(url, init);
|
|
59
|
+
return doFetch(request, { signal: AbortSignal.timeout(timeoutMs) });
|
|
60
|
+
}
|
|
61
|
+
async function fail(operation, key, response) {
|
|
62
|
+
const detail = (await response.text().catch(() => "")).slice(0, 300);
|
|
63
|
+
throw new S3HttpError(operation, key, response.status, detail);
|
|
64
|
+
}
|
|
65
|
+
const objectUrl = (key) => `${bucketUrl}/${encodeKey(key)}`;
|
|
66
|
+
return {
|
|
67
|
+
async get(key) {
|
|
68
|
+
const response = await send(objectUrl(key), { method: "GET" }, BODY_TIMEOUT_MS);
|
|
69
|
+
if (response.status === 404) {
|
|
70
|
+
await response.body?.cancel();
|
|
71
|
+
return null;
|
|
72
|
+
}
|
|
73
|
+
if (!response.ok) return fail("GET", key, response);
|
|
74
|
+
return new Uint8Array(await response.arrayBuffer());
|
|
75
|
+
},
|
|
76
|
+
async getRange(key, start, end) {
|
|
77
|
+
const response = await send(objectUrl(key), {
|
|
78
|
+
method: "GET",
|
|
79
|
+
headers: { Range: `bytes=${start}-${end}` }
|
|
80
|
+
}, METADATA_TIMEOUT_MS);
|
|
81
|
+
if (response.status === 404) {
|
|
82
|
+
await response.body?.cancel();
|
|
83
|
+
return null;
|
|
84
|
+
}
|
|
85
|
+
if (!response.ok) return fail("GET range", key, response);
|
|
86
|
+
return new Uint8Array(await response.arrayBuffer());
|
|
87
|
+
},
|
|
88
|
+
async put(key, data) {
|
|
89
|
+
const headers = {};
|
|
90
|
+
const type = options.contentType?.(key);
|
|
91
|
+
if (type) headers["Content-Type"] = type;
|
|
92
|
+
const response = await send(objectUrl(key), {
|
|
93
|
+
method: "PUT",
|
|
94
|
+
headers,
|
|
95
|
+
body: data
|
|
96
|
+
}, BODY_TIMEOUT_MS);
|
|
97
|
+
if (!response.ok) return fail("PUT", key, response);
|
|
98
|
+
await response.body?.cancel();
|
|
99
|
+
},
|
|
100
|
+
async delete(key) {
|
|
101
|
+
const response = await send(objectUrl(key), { method: "DELETE" }, METADATA_TIMEOUT_MS);
|
|
102
|
+
if (!response.ok && response.status !== 404) return fail("DELETE", key, response);
|
|
103
|
+
await response.body?.cancel();
|
|
104
|
+
},
|
|
105
|
+
async head(key) {
|
|
106
|
+
const response = await send(objectUrl(key), { method: "HEAD" }, METADATA_TIMEOUT_MS);
|
|
107
|
+
if (response.status === 404) return null;
|
|
108
|
+
if (!response.ok) return fail("HEAD", key, response);
|
|
109
|
+
return { size: Number(response.headers.get("content-length") ?? 0) };
|
|
110
|
+
},
|
|
111
|
+
async list(prefix, listOptions) {
|
|
112
|
+
const limit = listOptions?.limit;
|
|
113
|
+
const objects = [];
|
|
114
|
+
const prefixes = /* @__PURE__ */ new Set();
|
|
115
|
+
let token;
|
|
116
|
+
do {
|
|
117
|
+
const params = new URLSearchParams({
|
|
118
|
+
"list-type": "2",
|
|
119
|
+
prefix
|
|
120
|
+
});
|
|
121
|
+
if (listOptions?.delimiter) params.set("delimiter", listOptions.delimiter);
|
|
122
|
+
if (limit !== void 0) params.set("max-keys", String(Math.min(limit, 1e3)));
|
|
123
|
+
if (token) params.set("continuation-token", token);
|
|
124
|
+
const response = await send(`${bucketUrl}?${params}`, { method: "GET" }, METADATA_TIMEOUT_MS);
|
|
125
|
+
if (!response.ok) return fail("LIST", prefix, response);
|
|
126
|
+
const page = parseListObjectsV2(await response.text());
|
|
127
|
+
objects.push(...page.objects);
|
|
128
|
+
for (const p of page.prefixes) prefixes.add(p);
|
|
129
|
+
if (limit !== void 0 && objects.length + prefixes.size >= limit) break;
|
|
130
|
+
token = page.nextToken;
|
|
131
|
+
} while (token);
|
|
132
|
+
return {
|
|
133
|
+
objects,
|
|
134
|
+
prefixes: [...prefixes]
|
|
135
|
+
};
|
|
136
|
+
},
|
|
137
|
+
async copy(fromKey, toKey) {
|
|
138
|
+
const response = await send(objectUrl(toKey), {
|
|
139
|
+
method: "PUT",
|
|
140
|
+
headers: { "x-amz-copy-source": `/${encodeURIComponent(options.bucket)}/${encodeKey(fromKey)}` }
|
|
141
|
+
}, BODY_TIMEOUT_MS);
|
|
142
|
+
if (!response.ok) return fail("COPY", toKey, response);
|
|
143
|
+
const body = await response.text();
|
|
144
|
+
if (body.includes("<Error>")) throw new S3HttpError("COPY", toKey, 500, body.slice(0, 300));
|
|
145
|
+
}
|
|
146
|
+
};
|
|
168
147
|
}
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
parseListObjectsV2
|
|
173
|
-
};
|
|
148
|
+
//#endregion
|
|
149
|
+
export { S3HttpError, createFetchS3Store, parseListObjectsV2 };
|
|
150
|
+
|
|
174
151
|
//# sourceMappingURL=fetch-s3.js.map
|
package/dist/fetch-s3.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"sources":["../src/stores/fetch-s3.ts"],"sourcesContent":["import { AwsClient } from \"aws4fetch\";\nimport type {\n\tListOptions,\n\tListResult,\n\tObjectStat,\n\tObjectStore,\n} from \"../types.js\";\n\n/**\n * An S3-compatible store (AWS S3, Cloudflare R2, MinIO, …) over the REST API,\n * signed with `aws4fetch` (optional peer dependency) and sent with plain\n * `fetch` — no AWS SDK, so it runs on any runtime with Web APIs (Node, Vercel\n * Functions, Workers). Node's built-in fetch pools keep-alive connections.\n *\n * Retries and the circuit breaker are not here: wrap the store in\n * `createRetryStore`. Every failed request throws an {@link S3HttpError}\n * carrying the status in the `$metadata.httpStatusCode` shape that\n * `isRetryableError` classifies (5xx/429 retry); a stalled request aborts with\n * a `TimeoutError`, which is retried too.\n */\n\n/** A non-2xx response (404s never throw — they map to `null`). */\nexport class S3HttpError extends Error {\n\treadonly $metadata: { httpStatusCode: number };\n\n\tconstructor(operation: string, key: string, status: number, detail: string) {\n\t\tsuper(\n\t\t\t`S3 ${operation} ${key} failed: HTTP ${status}${detail ? ` — ${detail}` : \"\"}`,\n\t\t);\n\t\tthis.name = status === 429 ? \"SlowDown\" : \"S3HttpError\";\n\t\tthis.$metadata = { httpStatusCode: status };\n\t}\n}\n\nexport interface FetchS3ObjectStore extends ObjectStore {\n\t/** Server-side copy within the bucket: no bytes pass through this process. */\n\tcopy(fromKey: string, toKey: string): Promise<void>;\n\t/**\n\t * Required, not optional as on {@link ObjectStore}: this store always serves\n\t * ranges (S3's `Range` header). Declaring it required is what makes a wrapper\n\t * that forgets to forward it a compile error — dropping it does not fail\n\t * loudly, it makes `getObjectSize` answer \"unknown\", so every size gate would\n\t * quietly read the whole object instead of refusing it.\n\t */\n\tgetRange(key: string, start: number, end: number): Promise<Uint8Array | null>;\n}\n\nfunction encodeKey(key: string): string {\n\treturn key.split(\"/\").map(encodeURIComponent).join(\"/\");\n}\n\nfunction decodeXml(text: string): string {\n\treturn text\n\t\t.replace(/</g, \"<\")\n\t\t.replace(/>/g, \">\")\n\t\t.replace(/"/g, '\"')\n\t\t.replace(/'/g, \"'\")\n\t\t.replace(/&#(\\d+);/g, (_, code: string) =>\n\t\t\tString.fromCodePoint(Number(code)),\n\t\t)\n\t\t.replace(/&#x([0-9a-f]+);/gi, (_, code: string) =>\n\t\t\tString.fromCodePoint(Number.parseInt(code, 16)),\n\t\t)\n\t\t.replace(/&/g, \"&\");\n}\n\nfunction xmlValues(xml: string, tag: string): string[] {\n\tconst pattern = new RegExp(`<${tag}>([\\\\s\\\\S]*?)</${tag}>`, \"g\");\n\treturn [...xml.matchAll(pattern)].map((match) => decodeXml(match[1] ?? \"\"));\n}\n\n/** Parses one ListObjectsV2 response page (exported for tests). */\nexport function parseListObjectsV2(xml: string): {\n\tobjects: ListResult[\"objects\"];\n\tprefixes: string[];\n\tnextToken: string | undefined;\n} {\n\tconst objects = xmlValues(xml, \"Contents\").map((entry) => ({\n\t\tkey: xmlValues(entry, \"Key\")[0] ?? \"\",\n\t\tsize: Number(xmlValues(entry, \"Size\")[0] ?? 0),\n\t}));\n\tconst prefixes = xmlValues(xml, \"CommonPrefixes\").flatMap((entry) =>\n\t\txmlValues(entry, \"Prefix\"),\n\t);\n\tconst truncated = xmlValues(xml, \"IsTruncated\")[0] === \"true\";\n\tconst nextToken = truncated\n\t\t? xmlValues(xml, \"NextContinuationToken\")[0]\n\t\t: undefined;\n\treturn { objects, prefixes, nextToken };\n}\n\nexport function createFetchS3Store(options: {\n\t/** Service endpoint, e.g. `https://<account>.r2.cloudflarestorage.com`. */\n\tendpoint: string;\n\tbucket: string;\n\taccessKeyId: string;\n\tsecretAccessKey: string;\n\t/** Signing region. Default `\"auto\"` (R2); use e.g. `\"us-east-1\"` for AWS. */\n\tregion?: string;\n\t/** Per-attempt timeout for body transfers (get/put/copy). Default 30 000 ms. */\n\tbodyTimeoutMs?: number;\n\t/** Per-attempt timeout for metadata calls (head/list/range). Default 10 000 ms. */\n\tmetadataTimeoutMs?: number;\n\t/** Derive a Content-Type for uploads from the key. */\n\tcontentType?: (key: string) => string | undefined;\n\t/** Custom `fetch` (tests, proxies). Default: the global one. */\n\tfetch?: typeof fetch;\n}): FetchS3ObjectStore {\n\tconst BODY_TIMEOUT_MS = options.bodyTimeoutMs ?? 30_000;\n\tconst METADATA_TIMEOUT_MS = options.metadataTimeoutMs ?? 10_000;\n\tconst client = new AwsClient({\n\t\taccessKeyId: options.accessKeyId,\n\t\tsecretAccessKey: options.secretAccessKey,\n\t\tservice: \"s3\",\n\t\tregion: options.region ?? \"auto\",\n\t\t// createRetryStore owns retries; aws4fetch's own would stack under it.\n\t\tretries: 0,\n\t});\n\tconst doFetch = options.fetch ?? fetch;\n\tconst bucketUrl = `${options.endpoint.replace(/\\/+$/, \"\")}/${encodeURIComponent(options.bucket)}`;\n\n\tasync function send(\n\t\turl: string,\n\t\tinit: RequestInit,\n\t\ttimeoutMs: number,\n\t): Promise<Response> {\n\t\tconst request = await client.sign(url, init);\n\t\treturn doFetch(request, { signal: AbortSignal.timeout(timeoutMs) });\n\t}\n\n\tasync function fail(\n\t\toperation: string,\n\t\tkey: string,\n\t\tresponse: Response,\n\t): Promise<never> {\n\t\tconst detail = (await response.text().catch(() => \"\")).slice(0, 300);\n\t\tthrow new S3HttpError(operation, key, response.status, detail);\n\t}\n\n\tconst objectUrl = (key: string) => `${bucketUrl}/${encodeKey(key)}`;\n\n\treturn {\n\t\tasync get(key) {\n\t\t\tconst response = await send(\n\t\t\t\tobjectUrl(key),\n\t\t\t\t{ method: \"GET\" },\n\t\t\t\tBODY_TIMEOUT_MS,\n\t\t\t);\n\t\t\tif (response.status === 404) {\n\t\t\t\tawait response.body?.cancel();\n\t\t\t\treturn null;\n\t\t\t}\n\t\t\tif (!response.ok) return fail(\"GET\", key, response);\n\t\t\treturn new Uint8Array(await response.arrayBuffer());\n\t\t},\n\n\t\tasync getRange(key, start, end) {\n\t\t\tconst response = await send(\n\t\t\t\tobjectUrl(key),\n\t\t\t\t{ method: \"GET\", headers: { Range: `bytes=${start}-${end}` } },\n\t\t\t\tMETADATA_TIMEOUT_MS,\n\t\t\t);\n\t\t\tif (response.status === 404) {\n\t\t\t\tawait response.body?.cancel();\n\t\t\t\treturn null;\n\t\t\t}\n\t\t\tif (!response.ok) return fail(\"GET range\", key, response);\n\t\t\treturn new Uint8Array(await response.arrayBuffer());\n\t\t},\n\n\t\tasync put(key, data) {\n\t\t\tconst headers: Record<string, string> = {};\n\t\t\tconst type = options.contentType?.(key);\n\t\t\tif (type) headers[\"Content-Type\"] = type;\n\t\t\tconst response = await send(\n\t\t\t\tobjectUrl(key),\n\t\t\t\t{ method: \"PUT\", headers, body: data as Uint8Array<ArrayBuffer> },\n\t\t\t\tBODY_TIMEOUT_MS,\n\t\t\t);\n\t\t\tif (!response.ok) return fail(\"PUT\", key, response);\n\t\t\tawait response.body?.cancel();\n\t\t},\n\n\t\tasync delete(key) {\n\t\t\tconst response = await send(\n\t\t\t\tobjectUrl(key),\n\t\t\t\t{ method: \"DELETE\" },\n\t\t\t\tMETADATA_TIMEOUT_MS,\n\t\t\t);\n\t\t\t// S3 answers 204 for a missing key; tolerate 404 from compatibles.\n\t\t\tif (!response.ok && response.status !== 404) {\n\t\t\t\treturn fail(\"DELETE\", key, response);\n\t\t\t}\n\t\t\tawait response.body?.cancel();\n\t\t},\n\n\t\tasync head(key): Promise<ObjectStat | null> {\n\t\t\tconst response = await send(\n\t\t\t\tobjectUrl(key),\n\t\t\t\t{ method: \"HEAD\" },\n\t\t\t\tMETADATA_TIMEOUT_MS,\n\t\t\t);\n\t\t\tif (response.status === 404) return null;\n\t\t\tif (!response.ok) return fail(\"HEAD\", key, response);\n\t\t\treturn { size: Number(response.headers.get(\"content-length\") ?? 0) };\n\t\t},\n\n\t\tasync list(prefix, listOptions?: ListOptions): Promise<ListResult> {\n\t\t\tconst limit = listOptions?.limit;\n\t\t\tconst objects: ListResult[\"objects\"] = [];\n\t\t\tconst prefixes = new Set<string>();\n\t\t\tlet token: string | undefined;\n\n\t\t\tdo {\n\t\t\t\tconst params = new URLSearchParams({ \"list-type\": \"2\", prefix });\n\t\t\t\tif (listOptions?.delimiter)\n\t\t\t\t\tparams.set(\"delimiter\", listOptions.delimiter);\n\t\t\t\tif (limit !== undefined)\n\t\t\t\t\tparams.set(\"max-keys\", String(Math.min(limit, 1000)));\n\t\t\t\tif (token) params.set(\"continuation-token\", token);\n\t\t\t\tconst response = await send(\n\t\t\t\t\t`${bucketUrl}?${params}`,\n\t\t\t\t\t{ method: \"GET\" },\n\t\t\t\t\tMETADATA_TIMEOUT_MS,\n\t\t\t\t);\n\t\t\t\tif (!response.ok) return fail(\"LIST\", prefix, response);\n\t\t\t\tconst page = parseListObjectsV2(await response.text());\n\t\t\t\tobjects.push(...page.objects);\n\t\t\t\tfor (const p of page.prefixes) prefixes.add(p);\n\t\t\t\tif (limit !== undefined && objects.length + prefixes.size >= limit)\n\t\t\t\t\tbreak;\n\t\t\t\ttoken = page.nextToken;\n\t\t\t} while (token);\n\n\t\t\treturn { objects, prefixes: [...prefixes] };\n\t\t},\n\n\t\tasync copy(fromKey, toKey) {\n\t\t\tconst response = await send(\n\t\t\t\tobjectUrl(toKey),\n\t\t\t\t{\n\t\t\t\t\tmethod: \"PUT\",\n\t\t\t\t\theaders: {\n\t\t\t\t\t\t\"x-amz-copy-source\": `/${encodeURIComponent(options.bucket)}/${encodeKey(fromKey)}`,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tBODY_TIMEOUT_MS,\n\t\t\t);\n\t\t\tif (!response.ok) return fail(\"COPY\", toKey, response);\n\t\t\t// A copy can answer 200 with an <Error> body (S3 semantics).\n\t\t\tconst body = await response.text();\n\t\t\tif (body.includes(\"<Error>\")) {\n\t\t\t\tthrow new S3HttpError(\"COPY\", toKey, 500, body.slice(0, 300));\n\t\t\t}\n\t\t},\n\t};\n}\n"],"mappings":";AAAA,SAAS,iBAAiB;AAsBnB,IAAM,cAAN,cAA0B,MAAM;AAAA,EAC7B;AAAA,EAET,YAAY,WAAmB,KAAa,QAAgB,QAAgB;AAC3E;AAAA,MACC,MAAM,SAAS,IAAI,GAAG,iBAAiB,MAAM,GAAG,SAAS,WAAM,MAAM,KAAK,EAAE;AAAA,IAC7E;AACA,SAAK,OAAO,WAAW,MAAM,aAAa;AAC1C,SAAK,YAAY,EAAE,gBAAgB,OAAO;AAAA,EAC3C;AACD;AAeA,SAAS,UAAU,KAAqB;AACvC,SAAO,IAAI,MAAM,GAAG,EAAE,IAAI,kBAAkB,EAAE,KAAK,GAAG;AACvD;AAEA,SAAS,UAAU,MAAsB;AACxC,SAAO,KACL,QAAQ,SAAS,GAAG,EACpB,QAAQ,SAAS,GAAG,EACpB,QAAQ,WAAW,GAAG,EACtB,QAAQ,WAAW,GAAG,EACtB;AAAA,IAAQ;AAAA,IAAa,CAAC,GAAG,SACzB,OAAO,cAAc,OAAO,IAAI,CAAC;AAAA,EAClC,EACC;AAAA,IAAQ;AAAA,IAAqB,CAAC,GAAG,SACjC,OAAO,cAAc,OAAO,SAAS,MAAM,EAAE,CAAC;AAAA,EAC/C,EACC,QAAQ,UAAU,GAAG;AACxB;AAEA,SAAS,UAAU,KAAa,KAAuB;AACtD,QAAM,UAAU,IAAI,OAAO,IAAI,GAAG,kBAAkB,GAAG,KAAK,GAAG;AAC/D,SAAO,CAAC,GAAG,IAAI,SAAS,OAAO,CAAC,EAAE,IAAI,CAAC,UAAU,UAAU,MAAM,CAAC,KAAK,EAAE,CAAC;AAC3E;AAGO,SAAS,mBAAmB,KAIjC;AACD,QAAM,UAAU,UAAU,KAAK,UAAU,EAAE,IAAI,CAAC,WAAW;AAAA,IAC1D,KAAK,UAAU,OAAO,KAAK,EAAE,CAAC,KAAK;AAAA,IACnC,MAAM,OAAO,UAAU,OAAO,MAAM,EAAE,CAAC,KAAK,CAAC;AAAA,EAC9C,EAAE;AACF,QAAM,WAAW,UAAU,KAAK,gBAAgB,EAAE;AAAA,IAAQ,CAAC,UAC1D,UAAU,OAAO,QAAQ;AAAA,EAC1B;AACA,QAAM,YAAY,UAAU,KAAK,aAAa,EAAE,CAAC,MAAM;AACvD,QAAM,YAAY,YACf,UAAU,KAAK,uBAAuB,EAAE,CAAC,IACzC;AACH,SAAO,EAAE,SAAS,UAAU,UAAU;AACvC;AAEO,SAAS,mBAAmB,SAgBZ;AACtB,QAAM,kBAAkB,QAAQ,iBAAiB;AACjD,QAAM,sBAAsB,QAAQ,qBAAqB;AACzD,QAAM,SAAS,IAAI,UAAU;AAAA,IAC5B,aAAa,QAAQ;AAAA,IACrB,iBAAiB,QAAQ;AAAA,IACzB,SAAS;AAAA,IACT,QAAQ,QAAQ,UAAU;AAAA;AAAA,IAE1B,SAAS;AAAA,EACV,CAAC;AACD,QAAM,UAAU,QAAQ,SAAS;AACjC,QAAM,YAAY,GAAG,QAAQ,SAAS,QAAQ,QAAQ,EAAE,CAAC,IAAI,mBAAmB,QAAQ,MAAM,CAAC;AAE/F,iBAAe,KACd,KACA,MACA,WACoB;AACpB,UAAM,UAAU,MAAM,OAAO,KAAK,KAAK,IAAI;AAC3C,WAAO,QAAQ,SAAS,EAAE,QAAQ,YAAY,QAAQ,SAAS,EAAE,CAAC;AAAA,EACnE;AAEA,iBAAe,KACd,WACA,KACA,UACiB;AACjB,UAAM,UAAU,MAAM,SAAS,KAAK,EAAE,MAAM,MAAM,EAAE,GAAG,MAAM,GAAG,GAAG;AACnE,UAAM,IAAI,YAAY,WAAW,KAAK,SAAS,QAAQ,MAAM;AAAA,EAC9D;AAEA,QAAM,YAAY,CAAC,QAAgB,GAAG,SAAS,IAAI,UAAU,GAAG,CAAC;AAEjE,SAAO;AAAA,IACN,MAAM,IAAI,KAAK;AACd,YAAM,WAAW,MAAM;AAAA,QACtB,UAAU,GAAG;AAAA,QACb,EAAE,QAAQ,MAAM;AAAA,QAChB;AAAA,MACD;AACA,UAAI,SAAS,WAAW,KAAK;AAC5B,cAAM,SAAS,MAAM,OAAO;AAC5B,eAAO;AAAA,MACR;AACA,UAAI,CAAC,SAAS,GAAI,QAAO,KAAK,OAAO,KAAK,QAAQ;AAClD,aAAO,IAAI,WAAW,MAAM,SAAS,YAAY,CAAC;AAAA,IACnD;AAAA,IAEA,MAAM,SAAS,KAAK,OAAO,KAAK;AAC/B,YAAM,WAAW,MAAM;AAAA,QACtB,UAAU,GAAG;AAAA,QACb,EAAE,QAAQ,OAAO,SAAS,EAAE,OAAO,SAAS,KAAK,IAAI,GAAG,GAAG,EAAE;AAAA,QAC7D;AAAA,MACD;AACA,UAAI,SAAS,WAAW,KAAK;AAC5B,cAAM,SAAS,MAAM,OAAO;AAC5B,eAAO;AAAA,MACR;AACA,UAAI,CAAC,SAAS,GAAI,QAAO,KAAK,aAAa,KAAK,QAAQ;AACxD,aAAO,IAAI,WAAW,MAAM,SAAS,YAAY,CAAC;AAAA,IACnD;AAAA,IAEA,MAAM,IAAI,KAAK,MAAM;AACpB,YAAM,UAAkC,CAAC;AACzC,YAAM,OAAO,QAAQ,cAAc,GAAG;AACtC,UAAI,KAAM,SAAQ,cAAc,IAAI;AACpC,YAAM,WAAW,MAAM;AAAA,QACtB,UAAU,GAAG;AAAA,QACb,EAAE,QAAQ,OAAO,SAAS,MAAM,KAAgC;AAAA,QAChE;AAAA,MACD;AACA,UAAI,CAAC,SAAS,GAAI,QAAO,KAAK,OAAO,KAAK,QAAQ;AAClD,YAAM,SAAS,MAAM,OAAO;AAAA,IAC7B;AAAA,IAEA,MAAM,OAAO,KAAK;AACjB,YAAM,WAAW,MAAM;AAAA,QACtB,UAAU,GAAG;AAAA,QACb,EAAE,QAAQ,SAAS;AAAA,QACnB;AAAA,MACD;AAEA,UAAI,CAAC,SAAS,MAAM,SAAS,WAAW,KAAK;AAC5C,eAAO,KAAK,UAAU,KAAK,QAAQ;AAAA,MACpC;AACA,YAAM,SAAS,MAAM,OAAO;AAAA,IAC7B;AAAA,IAEA,MAAM,KAAK,KAAiC;AAC3C,YAAM,WAAW,MAAM;AAAA,QACtB,UAAU,GAAG;AAAA,QACb,EAAE,QAAQ,OAAO;AAAA,QACjB;AAAA,MACD;AACA,UAAI,SAAS,WAAW,IAAK,QAAO;AACpC,UAAI,CAAC,SAAS,GAAI,QAAO,KAAK,QAAQ,KAAK,QAAQ;AACnD,aAAO,EAAE,MAAM,OAAO,SAAS,QAAQ,IAAI,gBAAgB,KAAK,CAAC,EAAE;AAAA,IACpE;AAAA,IAEA,MAAM,KAAK,QAAQ,aAAgD;AAClE,YAAM,QAAQ,aAAa;AAC3B,YAAM,UAAiC,CAAC;AACxC,YAAM,WAAW,oBAAI,IAAY;AACjC,UAAI;AAEJ,SAAG;AACF,cAAM,SAAS,IAAI,gBAAgB,EAAE,aAAa,KAAK,OAAO,CAAC;AAC/D,YAAI,aAAa;AAChB,iBAAO,IAAI,aAAa,YAAY,SAAS;AAC9C,YAAI,UAAU;AACb,iBAAO,IAAI,YAAY,OAAO,KAAK,IAAI,OAAO,GAAI,CAAC,CAAC;AACrD,YAAI,MAAO,QAAO,IAAI,sBAAsB,KAAK;AACjD,cAAM,WAAW,MAAM;AAAA,UACtB,GAAG,SAAS,IAAI,MAAM;AAAA,UACtB,EAAE,QAAQ,MAAM;AAAA,UAChB;AAAA,QACD;AACA,YAAI,CAAC,SAAS,GAAI,QAAO,KAAK,QAAQ,QAAQ,QAAQ;AACtD,cAAM,OAAO,mBAAmB,MAAM,SAAS,KAAK,CAAC;AACrD,gBAAQ,KAAK,GAAG,KAAK,OAAO;AAC5B,mBAAW,KAAK,KAAK,SAAU,UAAS,IAAI,CAAC;AAC7C,YAAI,UAAU,UAAa,QAAQ,SAAS,SAAS,QAAQ;AAC5D;AACD,gBAAQ,KAAK;AAAA,MACd,SAAS;AAET,aAAO,EAAE,SAAS,UAAU,CAAC,GAAG,QAAQ,EAAE;AAAA,IAC3C;AAAA,IAEA,MAAM,KAAK,SAAS,OAAO;AAC1B,YAAM,WAAW,MAAM;AAAA,QACtB,UAAU,KAAK;AAAA,QACf;AAAA,UACC,QAAQ;AAAA,UACR,SAAS;AAAA,YACR,qBAAqB,IAAI,mBAAmB,QAAQ,MAAM,CAAC,IAAI,UAAU,OAAO,CAAC;AAAA,UAClF;AAAA,QACD;AAAA,QACA;AAAA,MACD;AACA,UAAI,CAAC,SAAS,GAAI,QAAO,KAAK,QAAQ,OAAO,QAAQ;AAErD,YAAM,OAAO,MAAM,SAAS,KAAK;AACjC,UAAI,KAAK,SAAS,SAAS,GAAG;AAC7B,cAAM,IAAI,YAAY,QAAQ,OAAO,KAAK,KAAK,MAAM,GAAG,GAAG,CAAC;AAAA,MAC7D;AAAA,IACD;AAAA,EACD;AACD;","names":[]}
|
|
1
|
+
{"version":3,"file":"fetch-s3.js","names":[],"sources":["../src/stores/fetch-s3.ts"],"sourcesContent":["import { AwsClient } from \"aws4fetch\";\nimport type {\n\tListOptions,\n\tListResult,\n\tObjectStat,\n\tObjectStore,\n} from \"../types.js\";\n\n/**\n * An S3-compatible store (AWS S3, Cloudflare R2, MinIO, …) over the REST API,\n * signed with `aws4fetch` (optional peer dependency) and sent with plain\n * `fetch` — no AWS SDK, so it runs on any runtime with Web APIs (Node, Vercel\n * Functions, Workers). Node's built-in fetch pools keep-alive connections.\n *\n * Retries and the circuit breaker are not here: wrap the store in\n * `createRetryStore`. Every failed request throws an {@link S3HttpError}\n * carrying the status in the `$metadata.httpStatusCode` shape that\n * `isRetryableError` classifies (5xx/429 retry); a stalled request aborts with\n * a `TimeoutError`, which is retried too.\n */\n\n/** A non-2xx response (404s never throw — they map to `null`). */\nexport class S3HttpError extends Error {\n\treadonly $metadata: { httpStatusCode: number };\n\n\tconstructor(operation: string, key: string, status: number, detail: string) {\n\t\tsuper(\n\t\t\t`S3 ${operation} ${key} failed: HTTP ${status}${detail ? ` — ${detail}` : \"\"}`,\n\t\t);\n\t\tthis.name = status === 429 ? \"SlowDown\" : \"S3HttpError\";\n\t\tthis.$metadata = { httpStatusCode: status };\n\t}\n}\n\nexport interface FetchS3ObjectStore extends ObjectStore {\n\t/** Server-side copy within the bucket: no bytes pass through this process. */\n\tcopy(fromKey: string, toKey: string): Promise<void>;\n\t/**\n\t * Required, not optional as on {@link ObjectStore}: this store always serves\n\t * ranges (S3's `Range` header). Declaring it required is what makes a wrapper\n\t * that forgets to forward it a compile error — dropping it does not fail\n\t * loudly, it makes `getObjectSize` answer \"unknown\", so every size gate would\n\t * quietly read the whole object instead of refusing it.\n\t */\n\tgetRange(key: string, start: number, end: number): Promise<Uint8Array | null>;\n}\n\nfunction encodeKey(key: string): string {\n\treturn key.split(\"/\").map(encodeURIComponent).join(\"/\");\n}\n\nfunction decodeXml(text: string): string {\n\treturn text\n\t\t.replace(/</g, \"<\")\n\t\t.replace(/>/g, \">\")\n\t\t.replace(/"/g, '\"')\n\t\t.replace(/'/g, \"'\")\n\t\t.replace(/&#(\\d+);/g, (_, code: string) =>\n\t\t\tString.fromCodePoint(Number(code)),\n\t\t)\n\t\t.replace(/&#x([0-9a-f]+);/gi, (_, code: string) =>\n\t\t\tString.fromCodePoint(Number.parseInt(code, 16)),\n\t\t)\n\t\t.replace(/&/g, \"&\");\n}\n\nfunction xmlValues(xml: string, tag: string): string[] {\n\tconst pattern = new RegExp(`<${tag}>([\\\\s\\\\S]*?)</${tag}>`, \"g\");\n\treturn [...xml.matchAll(pattern)].map((match) => decodeXml(match[1] ?? \"\"));\n}\n\n/** Parses one ListObjectsV2 response page (exported for tests). */\nexport function parseListObjectsV2(xml: string): {\n\tobjects: ListResult[\"objects\"];\n\tprefixes: string[];\n\tnextToken: string | undefined;\n} {\n\tconst objects = xmlValues(xml, \"Contents\").map((entry) => ({\n\t\tkey: xmlValues(entry, \"Key\")[0] ?? \"\",\n\t\tsize: Number(xmlValues(entry, \"Size\")[0] ?? 0),\n\t}));\n\tconst prefixes = xmlValues(xml, \"CommonPrefixes\").flatMap((entry) =>\n\t\txmlValues(entry, \"Prefix\"),\n\t);\n\tconst truncated = xmlValues(xml, \"IsTruncated\")[0] === \"true\";\n\tconst nextToken = truncated\n\t\t? xmlValues(xml, \"NextContinuationToken\")[0]\n\t\t: undefined;\n\treturn { objects, prefixes, nextToken };\n}\n\nexport function createFetchS3Store(options: {\n\t/** Service endpoint, e.g. `https://<account>.r2.cloudflarestorage.com`. */\n\tendpoint: string;\n\tbucket: string;\n\taccessKeyId: string;\n\tsecretAccessKey: string;\n\t/** Signing region. Default `\"auto\"` (R2); use e.g. `\"us-east-1\"` for AWS. */\n\tregion?: string;\n\t/** Per-attempt timeout for body transfers (get/put/copy). Default 30 000 ms. */\n\tbodyTimeoutMs?: number;\n\t/** Per-attempt timeout for metadata calls (head/list/range). Default 10 000 ms. */\n\tmetadataTimeoutMs?: number;\n\t/** Derive a Content-Type for uploads from the key. */\n\tcontentType?: (key: string) => string | undefined;\n\t/** Custom `fetch` (tests, proxies). Default: the global one. */\n\tfetch?: typeof fetch;\n}): FetchS3ObjectStore {\n\tconst BODY_TIMEOUT_MS = options.bodyTimeoutMs ?? 30_000;\n\tconst METADATA_TIMEOUT_MS = options.metadataTimeoutMs ?? 10_000;\n\tconst client = new AwsClient({\n\t\taccessKeyId: options.accessKeyId,\n\t\tsecretAccessKey: options.secretAccessKey,\n\t\tservice: \"s3\",\n\t\tregion: options.region ?? \"auto\",\n\t\t// createRetryStore owns retries; aws4fetch's own would stack under it.\n\t\tretries: 0,\n\t});\n\tconst doFetch = options.fetch ?? fetch;\n\tconst bucketUrl = `${options.endpoint.replace(/\\/+$/, \"\")}/${encodeURIComponent(options.bucket)}`;\n\n\tasync function send(\n\t\turl: string,\n\t\tinit: RequestInit,\n\t\ttimeoutMs: number,\n\t): Promise<Response> {\n\t\tconst request = await client.sign(url, init);\n\t\treturn doFetch(request, { signal: AbortSignal.timeout(timeoutMs) });\n\t}\n\n\tasync function fail(\n\t\toperation: string,\n\t\tkey: string,\n\t\tresponse: Response,\n\t): Promise<never> {\n\t\tconst detail = (await response.text().catch(() => \"\")).slice(0, 300);\n\t\tthrow new S3HttpError(operation, key, response.status, detail);\n\t}\n\n\tconst objectUrl = (key: string) => `${bucketUrl}/${encodeKey(key)}`;\n\n\treturn {\n\t\tasync get(key) {\n\t\t\tconst response = await send(\n\t\t\t\tobjectUrl(key),\n\t\t\t\t{ method: \"GET\" },\n\t\t\t\tBODY_TIMEOUT_MS,\n\t\t\t);\n\t\t\tif (response.status === 404) {\n\t\t\t\tawait response.body?.cancel();\n\t\t\t\treturn null;\n\t\t\t}\n\t\t\tif (!response.ok) return fail(\"GET\", key, response);\n\t\t\treturn new Uint8Array(await response.arrayBuffer());\n\t\t},\n\n\t\tasync getRange(key, start, end) {\n\t\t\tconst response = await send(\n\t\t\t\tobjectUrl(key),\n\t\t\t\t{ method: \"GET\", headers: { Range: `bytes=${start}-${end}` } },\n\t\t\t\tMETADATA_TIMEOUT_MS,\n\t\t\t);\n\t\t\tif (response.status === 404) {\n\t\t\t\tawait response.body?.cancel();\n\t\t\t\treturn null;\n\t\t\t}\n\t\t\tif (!response.ok) return fail(\"GET range\", key, response);\n\t\t\treturn new Uint8Array(await response.arrayBuffer());\n\t\t},\n\n\t\tasync put(key, data) {\n\t\t\tconst headers: Record<string, string> = {};\n\t\t\tconst type = options.contentType?.(key);\n\t\t\tif (type) headers[\"Content-Type\"] = type;\n\t\t\tconst response = await send(\n\t\t\t\tobjectUrl(key),\n\t\t\t\t{ method: \"PUT\", headers, body: data as Uint8Array<ArrayBuffer> },\n\t\t\t\tBODY_TIMEOUT_MS,\n\t\t\t);\n\t\t\tif (!response.ok) return fail(\"PUT\", key, response);\n\t\t\tawait response.body?.cancel();\n\t\t},\n\n\t\tasync delete(key) {\n\t\t\tconst response = await send(\n\t\t\t\tobjectUrl(key),\n\t\t\t\t{ method: \"DELETE\" },\n\t\t\t\tMETADATA_TIMEOUT_MS,\n\t\t\t);\n\t\t\t// S3 answers 204 for a missing key; tolerate 404 from compatibles.\n\t\t\tif (!response.ok && response.status !== 404) {\n\t\t\t\treturn fail(\"DELETE\", key, response);\n\t\t\t}\n\t\t\tawait response.body?.cancel();\n\t\t},\n\n\t\tasync head(key): Promise<ObjectStat | null> {\n\t\t\tconst response = await send(\n\t\t\t\tobjectUrl(key),\n\t\t\t\t{ method: \"HEAD\" },\n\t\t\t\tMETADATA_TIMEOUT_MS,\n\t\t\t);\n\t\t\tif (response.status === 404) return null;\n\t\t\tif (!response.ok) return fail(\"HEAD\", key, response);\n\t\t\treturn { size: Number(response.headers.get(\"content-length\") ?? 0) };\n\t\t},\n\n\t\tasync list(prefix, listOptions?: ListOptions): Promise<ListResult> {\n\t\t\tconst limit = listOptions?.limit;\n\t\t\tconst objects: ListResult[\"objects\"] = [];\n\t\t\tconst prefixes = new Set<string>();\n\t\t\tlet token: string | undefined;\n\n\t\t\tdo {\n\t\t\t\tconst params = new URLSearchParams({ \"list-type\": \"2\", prefix });\n\t\t\t\tif (listOptions?.delimiter)\n\t\t\t\t\tparams.set(\"delimiter\", listOptions.delimiter);\n\t\t\t\tif (limit !== undefined)\n\t\t\t\t\tparams.set(\"max-keys\", String(Math.min(limit, 1000)));\n\t\t\t\tif (token) params.set(\"continuation-token\", token);\n\t\t\t\tconst response = await send(\n\t\t\t\t\t`${bucketUrl}?${params}`,\n\t\t\t\t\t{ method: \"GET\" },\n\t\t\t\t\tMETADATA_TIMEOUT_MS,\n\t\t\t\t);\n\t\t\t\tif (!response.ok) return fail(\"LIST\", prefix, response);\n\t\t\t\tconst page = parseListObjectsV2(await response.text());\n\t\t\t\tobjects.push(...page.objects);\n\t\t\t\tfor (const p of page.prefixes) prefixes.add(p);\n\t\t\t\tif (limit !== undefined && objects.length + prefixes.size >= limit)\n\t\t\t\t\tbreak;\n\t\t\t\ttoken = page.nextToken;\n\t\t\t} while (token);\n\n\t\t\treturn { objects, prefixes: [...prefixes] };\n\t\t},\n\n\t\tasync copy(fromKey, toKey) {\n\t\t\tconst response = await send(\n\t\t\t\tobjectUrl(toKey),\n\t\t\t\t{\n\t\t\t\t\tmethod: \"PUT\",\n\t\t\t\t\theaders: {\n\t\t\t\t\t\t\"x-amz-copy-source\": `/${encodeURIComponent(options.bucket)}/${encodeKey(fromKey)}`,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tBODY_TIMEOUT_MS,\n\t\t\t);\n\t\t\tif (!response.ok) return fail(\"COPY\", toKey, response);\n\t\t\t// A copy can answer 200 with an <Error> body (S3 semantics).\n\t\t\tconst body = await response.text();\n\t\t\tif (body.includes(\"<Error>\")) {\n\t\t\t\tthrow new S3HttpError(\"COPY\", toKey, 500, body.slice(0, 300));\n\t\t\t}\n\t\t},\n\t};\n}\n"],"mappings":";;;;;;;;;;;;;;;AAsBA,IAAa,cAAb,cAAiC,MAAM;CACtC;CAEA,YAAY,WAAmB,KAAa,QAAgB,QAAgB;EAC3E,MACC,MAAM,UAAU,GAAG,IAAI,gBAAgB,SAAS,SAAS,MAAM,WAAW,IAC3E;EACA,KAAK,OAAO,WAAW,MAAM,aAAa;EAC1C,KAAK,YAAY,EAAE,gBAAgB,OAAO;CAC3C;AACD;AAeA,SAAS,UAAU,KAAqB;CACvC,OAAO,IAAI,MAAM,GAAG,CAAC,CAAC,IAAI,kBAAkB,CAAC,CAAC,KAAK,GAAG;AACvD;AAEA,SAAS,UAAU,MAAsB;CACxC,OAAO,KACL,QAAQ,SAAS,GAAG,CAAC,CACrB,QAAQ,SAAS,GAAG,CAAC,CACrB,QAAQ,WAAW,IAAG,CAAC,CACvB,QAAQ,WAAW,GAAG,CAAC,CACvB,QAAQ,cAAc,GAAG,SACzB,OAAO,cAAc,OAAO,IAAI,CAAC,CAClC,CAAC,CACA,QAAQ,sBAAsB,GAAG,SACjC,OAAO,cAAc,OAAO,SAAS,MAAM,EAAE,CAAC,CAC/C,CAAC,CACA,QAAQ,UAAU,GAAG;AACxB;AAEA,SAAS,UAAU,KAAa,KAAuB;CACtD,MAAM,UAAU,IAAI,OAAO,IAAI,IAAI,iBAAiB,IAAI,IAAI,GAAG;CAC/D,OAAO,CAAC,GAAG,IAAI,SAAS,OAAO,CAAC,CAAC,CAAC,KAAK,UAAU,UAAU,MAAM,MAAM,EAAE,CAAC;AAC3E;;AAGA,SAAgB,mBAAmB,KAIjC;CAYD,OAAO;EAAE,SAXO,UAAU,KAAK,UAAU,CAAC,CAAC,KAAK,WAAW;GAC1D,KAAK,UAAU,OAAO,KAAK,CAAC,CAAC,MAAM;GACnC,MAAM,OAAO,UAAU,OAAO,MAAM,CAAC,CAAC,MAAM,CAAC;EAC9C,EAQe;EAAG,UAPD,UAAU,KAAK,gBAAgB,CAAC,CAAC,SAAS,UAC1D,UAAU,OAAO,QAAQ,CAMD;EAAG,WAJV,UAAU,KAAK,aAAa,CAAC,CAAC,OAAO,SAEpD,UAAU,KAAK,uBAAuB,CAAC,CAAC,KACxC,KAAA;CACmC;AACvC;AAEA,SAAgB,mBAAmB,SAgBZ;CACtB,MAAM,kBAAkB,QAAQ,iBAAiB;CACjD,MAAM,sBAAsB,QAAQ,qBAAqB;CACzD,MAAM,SAAS,IAAI,UAAU;EAC5B,aAAa,QAAQ;EACrB,iBAAiB,QAAQ;EACzB,SAAS;EACT,QAAQ,QAAQ,UAAU;EAE1B,SAAS;CACV,CAAC;CACD,MAAM,UAAU,QAAQ,SAAS;CACjC,MAAM,YAAY,GAAG,QAAQ,SAAS,QAAQ,QAAQ,EAAE,EAAE,GAAG,mBAAmB,QAAQ,MAAM;CAE9F,eAAe,KACd,KACA,MACA,WACoB;EACpB,MAAM,UAAU,MAAM,OAAO,KAAK,KAAK,IAAI;EAC3C,OAAO,QAAQ,SAAS,EAAE,QAAQ,YAAY,QAAQ,SAAS,EAAE,CAAC;CACnE;CAEA,eAAe,KACd,WACA,KACA,UACiB;EACjB,MAAM,UAAU,MAAM,SAAS,KAAK,CAAC,CAAC,YAAY,EAAE,EAAA,CAAG,MAAM,GAAG,GAAG;EACnE,MAAM,IAAI,YAAY,WAAW,KAAK,SAAS,QAAQ,MAAM;CAC9D;CAEA,MAAM,aAAa,QAAgB,GAAG,UAAU,GAAG,UAAU,GAAG;CAEhE,OAAO;EACN,MAAM,IAAI,KAAK;GACd,MAAM,WAAW,MAAM,KACtB,UAAU,GAAG,GACb,EAAE,QAAQ,MAAM,GAChB,eACD;GACA,IAAI,SAAS,WAAW,KAAK;IAC5B,MAAM,SAAS,MAAM,OAAO;IAC5B,OAAO;GACR;GACA,IAAI,CAAC,SAAS,IAAI,OAAO,KAAK,OAAO,KAAK,QAAQ;GAClD,OAAO,IAAI,WAAW,MAAM,SAAS,YAAY,CAAC;EACnD;EAEA,MAAM,SAAS,KAAK,OAAO,KAAK;GAC/B,MAAM,WAAW,MAAM,KACtB,UAAU,GAAG,GACb;IAAE,QAAQ;IAAO,SAAS,EAAE,OAAO,SAAS,MAAM,GAAG,MAAM;GAAE,GAC7D,mBACD;GACA,IAAI,SAAS,WAAW,KAAK;IAC5B,MAAM,SAAS,MAAM,OAAO;IAC5B,OAAO;GACR;GACA,IAAI,CAAC,SAAS,IAAI,OAAO,KAAK,aAAa,KAAK,QAAQ;GACxD,OAAO,IAAI,WAAW,MAAM,SAAS,YAAY,CAAC;EACnD;EAEA,MAAM,IAAI,KAAK,MAAM;GACpB,MAAM,UAAkC,CAAC;GACzC,MAAM,OAAO,QAAQ,cAAc,GAAG;GACtC,IAAI,MAAM,QAAQ,kBAAkB;GACpC,MAAM,WAAW,MAAM,KACtB,UAAU,GAAG,GACb;IAAE,QAAQ;IAAO;IAAS,MAAM;GAAgC,GAChE,eACD;GACA,IAAI,CAAC,SAAS,IAAI,OAAO,KAAK,OAAO,KAAK,QAAQ;GAClD,MAAM,SAAS,MAAM,OAAO;EAC7B;EAEA,MAAM,OAAO,KAAK;GACjB,MAAM,WAAW,MAAM,KACtB,UAAU,GAAG,GACb,EAAE,QAAQ,SAAS,GACnB,mBACD;GAEA,IAAI,CAAC,SAAS,MAAM,SAAS,WAAW,KACvC,OAAO,KAAK,UAAU,KAAK,QAAQ;GAEpC,MAAM,SAAS,MAAM,OAAO;EAC7B;EAEA,MAAM,KAAK,KAAiC;GAC3C,MAAM,WAAW,MAAM,KACtB,UAAU,GAAG,GACb,EAAE,QAAQ,OAAO,GACjB,mBACD;GACA,IAAI,SAAS,WAAW,KAAK,OAAO;GACpC,IAAI,CAAC,SAAS,IAAI,OAAO,KAAK,QAAQ,KAAK,QAAQ;GACnD,OAAO,EAAE,MAAM,OAAO,SAAS,QAAQ,IAAI,gBAAgB,KAAK,CAAC,EAAE;EACpE;EAEA,MAAM,KAAK,QAAQ,aAAgD;GAClE,MAAM,QAAQ,aAAa;GAC3B,MAAM,UAAiC,CAAC;GACxC,MAAM,2BAAW,IAAI,IAAY;GACjC,IAAI;GAEJ,GAAG;IACF,MAAM,SAAS,IAAI,gBAAgB;KAAE,aAAa;KAAK;IAAO,CAAC;IAC/D,IAAI,aAAa,WAChB,OAAO,IAAI,aAAa,YAAY,SAAS;IAC9C,IAAI,UAAU,KAAA,GACb,OAAO,IAAI,YAAY,OAAO,KAAK,IAAI,OAAO,GAAI,CAAC,CAAC;IACrD,IAAI,OAAO,OAAO,IAAI,sBAAsB,KAAK;IACjD,MAAM,WAAW,MAAM,KACtB,GAAG,UAAU,GAAG,UAChB,EAAE,QAAQ,MAAM,GAChB,mBACD;IACA,IAAI,CAAC,SAAS,IAAI,OAAO,KAAK,QAAQ,QAAQ,QAAQ;IACtD,MAAM,OAAO,mBAAmB,MAAM,SAAS,KAAK,CAAC;IACrD,QAAQ,KAAK,GAAG,KAAK,OAAO;IAC5B,KAAK,MAAM,KAAK,KAAK,UAAU,SAAS,IAAI,CAAC;IAC7C,IAAI,UAAU,KAAA,KAAa,QAAQ,SAAS,SAAS,QAAQ,OAC5D;IACD,QAAQ,KAAK;GACd,SAAS;GAET,OAAO;IAAE;IAAS,UAAU,CAAC,GAAG,QAAQ;GAAE;EAC3C;EAEA,MAAM,KAAK,SAAS,OAAO;GAC1B,MAAM,WAAW,MAAM,KACtB,UAAU,KAAK,GACf;IACC,QAAQ;IACR,SAAS,EACR,qBAAqB,IAAI,mBAAmB,QAAQ,MAAM,EAAE,GAAG,UAAU,OAAO,IACjF;GACD,GACA,eACD;GACA,IAAI,CAAC,SAAS,IAAI,OAAO,KAAK,QAAQ,OAAO,QAAQ;GAErD,MAAM,OAAO,MAAM,SAAS,KAAK;GACjC,IAAI,KAAK,SAAS,SAAS,GAC1B,MAAM,IAAI,YAAY,QAAQ,OAAO,KAAK,KAAK,MAAM,GAAG,GAAG,CAAC;EAE9D;CACD;AACD"}
|