@nxgt/s3 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +28 -11
- package/dist/bucket/guards.d.ts +6 -0
- package/dist/bucket/guards.d.ts.map +1 -1
- package/dist/bucket/operations/presign.d.ts.map +1 -1
- package/dist/bucket/operations/writes.d.ts.map +1 -1
- package/dist/errors/s3-error.d.ts +6 -1
- package/dist/errors/s3-error.d.ts.map +1 -1
- package/dist/index.js +108 -52
- package/dist/index.js.map +7 -7
- package/docs/README.md +16 -0
- package/docs/guide/buckets.md +170 -0
- package/docs/guide/presigned-urls.md +180 -0
- package/docs/guide/reads.md +186 -0
- package/docs/guide/writes.md +273 -0
- package/docs/roadmap.md +64 -0
- package/docs/troubleshooting.md +328 -0
- package/package.json +2 -1
|
@@ -0,0 +1,273 @@
|
|
|
1
|
+
# Writing
|
|
2
|
+
|
|
3
|
+
Storing an object, saying what it is and how it should be served back, and
|
|
4
|
+
getting a refusal **before** anything goes over the wire.
|
|
5
|
+
|
|
6
|
+
## The smallest thing that works
|
|
7
|
+
|
|
8
|
+
```ts
|
|
9
|
+
import { bindBucket, defineBucket } from '@nxgt/s3';
|
|
10
|
+
|
|
11
|
+
const avatars = defineBucket({
|
|
12
|
+
bucket: 'avatars',
|
|
13
|
+
key: (p: { userId: string }) => `${p.userId}.png`,
|
|
14
|
+
contentType: ['image/png', 'image/jpeg'],
|
|
15
|
+
maxSize: 2 * 1024 * 1024,
|
|
16
|
+
});
|
|
17
|
+
|
|
18
|
+
const store = bindBucket(avatars);
|
|
19
|
+
|
|
20
|
+
await store.put({ userId: 'u1' }, png, { type: 'image/png' });
|
|
21
|
+
await store.delete({ userId: 'u1' });
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Describing the bucket is [Buckets](buckets.md). `put` resolves when the
|
|
25
|
+
object is stored, and gives back nothing: S3 has nothing to say beyond that.
|
|
26
|
+
|
|
27
|
+
## The signatures
|
|
28
|
+
|
|
29
|
+
```ts
|
|
30
|
+
import type { S3Client, S3Options } from 'bun';
|
|
31
|
+
|
|
32
|
+
/** A string, bytes, a Blob, a Bun.file, a stream, a Response… */
|
|
33
|
+
type PutBody = Parameters<S3Client['write']>[1];
|
|
34
|
+
|
|
35
|
+
type PutOptions = Pick<
|
|
36
|
+
S3Options,
|
|
37
|
+
'type' | 'acl' | 'storageClass' | 'contentDisposition' | 'contentEncoding'
|
|
38
|
+
>;
|
|
39
|
+
|
|
40
|
+
put(params: P, body: PutBody, options?: PutOptions): Promise<void>;
|
|
41
|
+
delete(params: P): Promise<void>;
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
## What a body may be
|
|
45
|
+
|
|
46
|
+
Anything Bun's own `write` takes:
|
|
47
|
+
|
|
48
|
+
```ts
|
|
49
|
+
await store.put({ userId: 'u1' }, 'a,b\n1,2\n', { type: 'text/csv' });
|
|
50
|
+
await store.put({ userId: 'u1' }, new Uint8Array([1, 2, 3]), { type: 'image/png' });
|
|
51
|
+
await store.put({ userId: 'u1' }, Bun.file('ada.png')); // the type comes from the file
|
|
52
|
+
await store.put({ userId: 'u1' }, await fetch(url)); // only without `maxSize`
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
A `Blob` — `Bun.file` included — carries its own content type, so a write
|
|
56
|
+
that names none is still checked against one. A string is measured in
|
|
57
|
+
**bytes**, not characters.
|
|
58
|
+
|
|
59
|
+
## What a write may say about the object
|
|
60
|
+
|
|
61
|
+
Every option here describes **the object being stored**. A write says nothing
|
|
62
|
+
about where it goes or how it gets there: the bucket, the endpoint, the
|
|
63
|
+
region and the credentials belong to the bound bucket.
|
|
64
|
+
|
|
65
|
+
| Option | Type | Default | Effect |
|
|
66
|
+
| --- | --- | --- | --- |
|
|
67
|
+
| `type` | `string` | the body's own, for a `Blob` | the content type. It is the one the guard checks **and** the one the service receives — they can never be two different answers |
|
|
68
|
+
| `acl` | `'private' \| 'public-read' \| 'public-read-write' \| 'aws-exec-read' \| 'authenticated-read' \| 'bucket-owner-read' \| 'bucket-owner-full-control' \| 'log-delivery-write'` | the client's, else the service's | who may read the object, where the service implements ACLs |
|
|
69
|
+
| `storageClass` | `'STANDARD' \| 'STANDARD_IA' \| 'ONEZONE_IA' \| 'INTELLIGENT_TIERING' \| 'GLACIER' \| 'GLACIER_IR' \| 'DEEP_ARCHIVE' \| 'REDUCED_REDUNDANCY' \| 'EXPRESS_ONEZONE' \| 'OUTPOSTS' \| 'SNOW'` | the client's, else `STANDARD` | what it costs to keep it; comes back as `x-amz-storage-class` |
|
|
70
|
+
| `contentDisposition` | `string` | none | how a reader should present it; comes back on a `GET` |
|
|
71
|
+
| `contentEncoding` | `string` | none | how the body is encoded; comes back on a `GET` |
|
|
72
|
+
|
|
73
|
+
```ts
|
|
74
|
+
await store.put({ userId: 'u1' }, png, {
|
|
75
|
+
type: 'image/png',
|
|
76
|
+
contentDisposition: 'attachment; filename="ada.png"',
|
|
77
|
+
contentEncoding: 'identity',
|
|
78
|
+
storageClass: 'STANDARD_IA',
|
|
79
|
+
acl: 'public-read',
|
|
80
|
+
});
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
Measured against a real service: `contentDisposition` and `contentEncoding`
|
|
84
|
+
come back as the headers of the same name on a read, and `storageClass` as
|
|
85
|
+
`x-amz-storage-class`.
|
|
86
|
+
|
|
87
|
+
These are Bun's own names, picked out of its `S3Options`, so a Bun release
|
|
88
|
+
that changes one is a compile error here rather than a silent drift. Given
|
|
89
|
+
both here and to `bindBucket`, the one on the write wins: it is the last
|
|
90
|
+
thing handed to the client.
|
|
91
|
+
|
|
92
|
+
### An option's *value* is checked here too
|
|
93
|
+
|
|
94
|
+
`acl` and `storageClass` are unions, and the values listed above are the only
|
|
95
|
+
ones the service takes. A bag off a request body never met those types, so
|
|
96
|
+
the value is checked at run time as well — before anything is sent, with the
|
|
97
|
+
content type and the size, and as the same `S3Error`:
|
|
98
|
+
|
|
99
|
+
```ts
|
|
100
|
+
import { S3Error, type PutOptions } from '@nxgt/s3';
|
|
101
|
+
|
|
102
|
+
const bad = { type: 'text/plain', storageClass: 'CHEAP' } as unknown as PutOptions;
|
|
103
|
+
|
|
104
|
+
const error = (await store
|
|
105
|
+
.put({ userId: 'u1' }, 'a,b\n', bad)
|
|
106
|
+
.catch((reason: unknown) => reason)) as S3Error;
|
|
107
|
+
|
|
108
|
+
error instanceof S3Error; // true
|
|
109
|
+
error.code; // 'WRONG_OPTION'
|
|
110
|
+
error.message; // 'storageClass must be one of STANDARD, DEEP_ARCHIVE, …; got "CHEAP"'
|
|
111
|
+
error.key; // 'u1.png' — the key, never the body
|
|
112
|
+
// Nothing was stored: `await store.exists({ userId: 'u1' })` is still false.
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
Bun checks both values as well, and refuses with a plain `TypeError`
|
|
116
|
+
(measured on bun 1.4.2). That left one `put` with two classes of refusal —
|
|
117
|
+
an `S3Error` with a code for the content type and the size, a `TypeError`
|
|
118
|
+
with only a sentence for these two. Checking here means one `put` has one
|
|
119
|
+
class of refusal, with a code to switch on. `contentDisposition` and `contentEncoding` are free strings
|
|
120
|
+
and are never refused, by this package or by Bun.
|
|
121
|
+
|
|
122
|
+
The same allowlist holds `acl` on a
|
|
123
|
+
[presigned URL](presigned-urls.md#options), so a wrong value is the same
|
|
124
|
+
`WRONG_OPTION` whichever call a consumer reached for.
|
|
125
|
+
|
|
126
|
+
### A write cannot change where it goes
|
|
127
|
+
|
|
128
|
+
`PutOptions` carries no `bucket`, `endpoint`, `region` or credential, and the
|
|
129
|
+
run time forwards only the five keys above — a key that is not in that list
|
|
130
|
+
is dropped, never sent.
|
|
131
|
+
|
|
132
|
+
```ts
|
|
133
|
+
// A bag off a request body never met the types.
|
|
134
|
+
const smuggled = {
|
|
135
|
+
type: 'text/plain',
|
|
136
|
+
bucket: 'somewhere-else',
|
|
137
|
+
accessKeyId: 'someone-else',
|
|
138
|
+
} as PutOptions;
|
|
139
|
+
|
|
140
|
+
await store.put({ userId: 'u1' }, 'x', smuggled);
|
|
141
|
+
// Stored in `avatars`, with the credentials the bucket was bound with.
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
Measured before that filter existed, spreading the caller's options straight
|
|
145
|
+
through let a `bucket` key store the object in another bucket — and report
|
|
146
|
+
success.
|
|
147
|
+
|
|
148
|
+
### A `put` is one PUT
|
|
149
|
+
|
|
150
|
+
`partSize`, `queueSize` and `retry` are not `PutOptions`: they belong to
|
|
151
|
+
[`bindBucket`](buckets.md), where they are the client's own. Measured on bun
|
|
152
|
+
1.4.2 against a 12 MiB body, a `put` with `partSize` set and one without come
|
|
153
|
+
back with the **same** ETag, and neither carries the `-<parts>` suffix a
|
|
154
|
+
multipart upload leaves. For a body that wants parts, use
|
|
155
|
+
`store.file(params).writer()`.
|
|
156
|
+
|
|
157
|
+
## The guards
|
|
158
|
+
|
|
159
|
+
`contentType` and `maxSize` on the definition are checked in `put`, before
|
|
160
|
+
the request goes out. A refused body is never sent.
|
|
161
|
+
|
|
162
|
+
```ts
|
|
163
|
+
import { S3Error } from '@nxgt/s3';
|
|
164
|
+
|
|
165
|
+
try {
|
|
166
|
+
await store.put({ userId: 'u1' }, pdf, { type: 'application/pdf' });
|
|
167
|
+
} catch (error) {
|
|
168
|
+
if (error instanceof S3Error) {
|
|
169
|
+
error.code; // 'WRONG_TYPE'
|
|
170
|
+
error.key; // 'u1.png' — the key, never the body
|
|
171
|
+
}
|
|
172
|
+
}
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
| `S3ErrorCode` | When |
|
|
176
|
+
| --- | --- |
|
|
177
|
+
| `WRONG_TYPE` | the type is not one this bucket accepts — or the write named none and the body carries none, while the bucket names some |
|
|
178
|
+
| `TOO_LARGE` | the body is bigger than `maxSize`. The limit is inclusive: with `maxSize: 1024`, 1024 bytes passes and 1025 does not |
|
|
179
|
+
| `UNMEASURABLE` | `maxSize` is set and the body's size cannot be known before sending |
|
|
180
|
+
| `WRONG_OPTION` | the write's `acl` or `storageClass` is not a value the service accepts — see [above](#an-options-value-is-checked-here-too) |
|
|
181
|
+
|
|
182
|
+
A content type is compared on its **essence**: parameters and case are
|
|
183
|
+
ignored, and nothing else is. A bucket that accepts `text/csv` accepts
|
|
184
|
+
`text/csv;charset=utf-8` and `TEXT/CSV`, because that is what real bodies
|
|
185
|
+
carry — measured, `Bun.file` labels `.txt` as `text/plain;charset=utf-8` and
|
|
186
|
+
`.csv` as the bare `text/csv`.
|
|
187
|
+
|
|
188
|
+
```ts
|
|
189
|
+
const reports = defineBucket({
|
|
190
|
+
bucket: 'reports',
|
|
191
|
+
key: (id: string) => `${id}.csv`,
|
|
192
|
+
contentType: 'text/csv',
|
|
193
|
+
});
|
|
194
|
+
const csv = bindBucket(reports);
|
|
195
|
+
|
|
196
|
+
await csv.put('q1', Bun.file('q1.csv')); // stored
|
|
197
|
+
await csv.put('q1', Bun.file('note.txt')); // S3Error WRONG_TYPE, nothing sent
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
`UNMEASURABLE` is the one that surprises: with `maxSize` set, a `Response`, a
|
|
201
|
+
`Request`, a stream or another `S3File` is **refused, not streamed** —
|
|
202
|
+
nothing can check a length it has not read, and an `S3File` reports its size
|
|
203
|
+
as `NaN` until the service has been asked. Read it into memory first, or
|
|
204
|
+
leave `maxSize` out and let the service refuse an oversized body.
|
|
205
|
+
|
|
206
|
+
```ts
|
|
207
|
+
const answer = await fetch(url);
|
|
208
|
+
const body = new Uint8Array(await answer.arrayBuffer()); // now it has a length
|
|
209
|
+
await store.put({ userId: 'u1' }, body, { type: 'image/png' });
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
The guards are `put`'s, not the bucket's: `store.file(params).writer()`,
|
|
213
|
+
`store.client` and anyone holding a [presigned PUT](presigned-urls.md) write
|
|
214
|
+
whatever they are given. Set the service's own policy too where it matters.
|
|
215
|
+
|
|
216
|
+
## Deleting
|
|
217
|
+
|
|
218
|
+
```ts
|
|
219
|
+
await store.delete({ userId: 'u1' });
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
S3 does not say whether anything was there, and nor does this. Call `exists`
|
|
223
|
+
first when it matters — see [Reading](reads.md).
|
|
224
|
+
|
|
225
|
+
## A real one: an upload route
|
|
226
|
+
|
|
227
|
+
```ts
|
|
228
|
+
import { Hono } from 'hono';
|
|
229
|
+
import { bindBucket, defineBucket, S3Error } from '@nxgt/s3';
|
|
230
|
+
|
|
231
|
+
const avatars = defineBucket({
|
|
232
|
+
bucket: 'avatars',
|
|
233
|
+
key: (p: { userId: string }) => `${p.userId}.png`,
|
|
234
|
+
contentType: ['image/png', 'image/jpeg'],
|
|
235
|
+
maxSize: 2 * 1024 * 1024,
|
|
236
|
+
});
|
|
237
|
+
|
|
238
|
+
const store = bindBucket(avatars);
|
|
239
|
+
const app = new Hono();
|
|
240
|
+
|
|
241
|
+
app.put('/users/:id/avatar', async (c) => {
|
|
242
|
+
const userId = c.req.param('id');
|
|
243
|
+
const body = await c.req.blob(); // a Blob: it carries its own type
|
|
244
|
+
|
|
245
|
+
try {
|
|
246
|
+
await store.put({ userId }, body, {
|
|
247
|
+
contentDisposition: `inline; filename="${userId}.png"`,
|
|
248
|
+
});
|
|
249
|
+
} catch (error) {
|
|
250
|
+
if (error instanceof S3Error && error.code === 'WRONG_TYPE') {
|
|
251
|
+
return c.json({ error: 'Send a PNG or a JPEG' }, 415);
|
|
252
|
+
}
|
|
253
|
+
if (error instanceof S3Error && error.code === 'TOO_LARGE') {
|
|
254
|
+
return c.json({ error: 'That file is too big' }, 413);
|
|
255
|
+
}
|
|
256
|
+
throw error;
|
|
257
|
+
}
|
|
258
|
+
|
|
259
|
+
return c.json({ key: store.keyFor({ userId }) }, 201);
|
|
260
|
+
});
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
Every `S3Error` is thrown before the request goes out, so a 413 costs no
|
|
264
|
+
bandwidth. Bun names **its own** S3 failures `S3Error` too, with S3's codes:
|
|
265
|
+
discriminate with `instanceof S3Error` on this package's class, never with
|
|
266
|
+
`error.name`.
|
|
267
|
+
|
|
268
|
+
## Next
|
|
269
|
+
|
|
270
|
+
- [Reading](reads.md) — reading back what was written.
|
|
271
|
+
- [Presigned URLs](presigned-urls.md) — letting a browser upload directly,
|
|
272
|
+
and what that does *not* constrain.
|
|
273
|
+
- [Troubleshooting](../troubleshooting.md) — each error with its fix.
|
package/docs/roadmap.md
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
# Roadmap
|
|
2
|
+
|
|
3
|
+
Where `@nxgt/s3` is going. A direction, not a commitment: the version an item
|
|
4
|
+
shipped in is the only number on this page.
|
|
5
|
+
|
|
6
|
+
## Now
|
|
7
|
+
|
|
8
|
+
_Nothing in progress._
|
|
9
|
+
|
|
10
|
+
## Next
|
|
11
|
+
|
|
12
|
+
_Nothing queued._
|
|
13
|
+
|
|
14
|
+
## Later
|
|
15
|
+
|
|
16
|
+
- **Copy and move** — once Bun's own `S3Client` has them. Until then
|
|
17
|
+
`store.client` is where they live.
|
|
18
|
+
|
|
19
|
+
## Not planned
|
|
20
|
+
|
|
21
|
+
- **Running on Node** — the client is Bun's own `S3Client`, which is why this
|
|
22
|
+
package installs no AWS SDK at all. There is nothing to swap for Node, and a
|
|
23
|
+
build for it is not coming: it needs Bun 1.4 or later.
|
|
24
|
+
- **Multipart upload** — a body goes out in one `put`. For anything bigger,
|
|
25
|
+
`store.file(params)` hands back Bun's own file handle and its `writer()`.
|
|
26
|
+
- **Bucket administration** — creating, deleting or configuring a bucket is
|
|
27
|
+
not this package's, and Bun's client has no `createBucket` either.
|
|
28
|
+
- **Retries or a cache of its own** — Bun's client already retries (three
|
|
29
|
+
attempts by default, and `retry` is passed through on `bindBucket`), and a
|
|
30
|
+
failed request comes back with S3's own error, as `S3Error`.
|
|
31
|
+
- **Upload tuning on a `put`** — a `put` is a single PUT: measured against a
|
|
32
|
+
12 MiB body, `partSize` changes nothing, right down to the ETag carrying no
|
|
33
|
+
`-<parts>` suffix. `PutOptions` describes the object, not the transfer.
|
|
34
|
+
- **An option that says where a write goes** — `bucket`, `endpoint`, `region`
|
|
35
|
+
and the credentials belong to the bound bucket, and are refused by the types
|
|
36
|
+
and again at run time. Measured: passing them through let a bag store the
|
|
37
|
+
object in a different bucket and report success, and redirected a signed URL
|
|
38
|
+
the same way.
|
|
39
|
+
|
|
40
|
+
## Shipped
|
|
41
|
+
|
|
42
|
+
- **An option's own value is refused before anything is sent or signed** —
|
|
43
|
+
`code: 'WRONG_OPTION'` for an `acl` or a `storageClass` a write names, for
|
|
44
|
+
an `acl` on a presigned URL, and for an `expiresIn` outside the seven days
|
|
45
|
+
S3 itself allows. Every refusal of a write or a signed URL is one class with
|
|
46
|
+
a code to switch on, rather than an `S3Error` for the content type and the
|
|
47
|
+
size and the client's own `TypeError` for the rest — 0.3.0.
|
|
48
|
+
- **Documentation that travels with the package** — a guide page for the
|
|
49
|
+
bucket definition, writes, reads and presigned URLs, a troubleshooting page
|
|
50
|
+
whose headings are the exact error text, and this roadmap, installed in
|
|
51
|
+
`docs/` rather than left on GitHub — 0.2.1.
|
|
52
|
+
- **What a write may say about the object** — `contentDisposition`,
|
|
53
|
+
`contentEncoding`, `acl` and `storageClass` on `put`, with Bun's own names;
|
|
54
|
+
an option naming another bucket, endpoint, region or credential is refused,
|
|
55
|
+
on `put` and on both `presign` calls — 0.2.0.
|
|
56
|
+
- **First release** — `defineBucket` naming the bucket, the key-building
|
|
57
|
+
function, the content types it accepts and the biggest body it takes, and
|
|
58
|
+
`bindBucket` giving `put`, `bytes`, `text`, `exists`, `stat`, `delete`, a
|
|
59
|
+
cursor `list` and `presignGet` / `presignPut`; the content type and the size
|
|
60
|
+
are checked before the request goes out, and its one error is `S3Error` —
|
|
61
|
+
0.1.0.
|
|
62
|
+
|
|
63
|
+
Everything released is in [`CHANGELOG.md`](https://github.com/softistx/nxgt-data/blob/develop/packages/s3/CHANGELOG.md) — it is not in
|
|
64
|
+
the published package, only in the repository.
|
|
@@ -0,0 +1,328 @@
|
|
|
1
|
+
# Troubleshooting
|
|
2
|
+
|
|
3
|
+
This package throws one error of its own, `S3Error`, with a `code` of
|
|
4
|
+
`WRONG_TYPE`, `TOO_LARGE`, `UNMEASURABLE` or `WRONG_OPTION`, and the object
|
|
5
|
+
`key` it was about — never the body. Every one of them is raised **before**
|
|
6
|
+
anything is sent, so a refused write stored nothing. The service's own
|
|
7
|
+
failures come back as Bun raises them; the entries below say which is which.
|
|
8
|
+
A wrong `acl` is this package's own refusal on a `put` **and** on a
|
|
9
|
+
`presign`, so one class and one code cover both. Bun names its own *service*
|
|
10
|
+
failures `S3Error` as well, so it is `instanceof S3Error` against the class
|
|
11
|
+
this package exports that tells those apart — never `error.name`. (Bun's
|
|
12
|
+
refusal of a wrong argument is a different thing again: a plain `TypeError`,
|
|
13
|
+
named `"TypeError"`.) The Bun messages were measured on Bun 1.4.2.
|
|
14
|
+
|
|
15
|
+
- **Install and import**
|
|
16
|
+
- [`Cannot find package 'bun'`](#cannot-find-package-bun)
|
|
17
|
+
- [`Cannot find module 'bun' or its corresponding type declarations.`](#cannot-find-module-bun-or-its-corresponding-type-declarations)
|
|
18
|
+
- **Configuration**
|
|
19
|
+
- [`defineBucket: a bucket definition needs a bucket`](#definebucket-a-bucket-definition-needs-a-bucket)
|
|
20
|
+
- [`defineBucket: "avatars" has a maxSize of 0; it is a number of bytes, and must be above zero`](#definebucket-avatars-has-a-maxsize-of-0-it-is-a-number-of-bytes-and-must-be-above-zero)
|
|
21
|
+
- [`defineBucket: "avatars" accepts an empty list of content types, so nothing could ever be written. …`](#definebucket-avatars-accepts-an-empty-list-of-content-types-so-nothing-could-ever-be-written-)
|
|
22
|
+
- **Writes refused before they are sent**
|
|
23
|
+
- [`"avatars" accepts image/png, image/jpeg, not application/pdf`](#avatars-accepts-imagepng-imagejpeg-not-applicationpdf)
|
|
24
|
+
- [``"avatars" accepts image/png, image/jpeg, and this write names no content type. Pass `type` ``](#avatars-accepts-imagepng-imagejpeg-and-this-write-names-no-content-type-pass-type-)
|
|
25
|
+
- [`"avatars" accepts 2097152 bytes at most, and this body is 5242880`](#avatars-accepts-2097152-bytes-at-most-and-this-body-is-5242880)
|
|
26
|
+
- [`"avatars" has a maxSize, and this body's size cannot be known before sending it. …`](#avatars-has-a-maxsize-and-this-bodys-size-cannot-be-known-before-sending-it-)
|
|
27
|
+
- [`storageClass must be one of STANDARD, DEEP_ARCHIVE, EXPRESS_ONEZONE, …; got "CHEAP"`](#storageclass-must-be-one-of-standard-deep_archive-express_onezone--got-cheap)
|
|
28
|
+
- [`acl must be one of private, public-read, public-read-write, …; got "everyone"`](#acl-must-be-one-of-private-public-read-public-read-write--got-everyone)
|
|
29
|
+
- [`expiresIn is seconds, and must be above 0 and at most 604800 …`](#expiresin-is-seconds-and-must-be-above-0-and-at-most-604800-seven-days-which-is-s3s-own-limit-got-1000000000000)
|
|
30
|
+
- **The service**
|
|
31
|
+
- [`Missing S3 credentials. 'accessKeyId', 'secretAccessKey', 'bucket', and 'endpoint' are required`](#missing-s3-credentials-accesskeyid-secretaccesskey-bucket-and-endpoint-are-required)
|
|
32
|
+
- [`The AWS Access Key Id you provided does not exist in our records.`](#the-aws-access-key-id-you-provided-does-not-exist-in-our-records)
|
|
33
|
+
- [A presigned upload stored a body the bucket would have refused](#a-presigned-upload-stored-a-body-the-bucket-would-have-refused)
|
|
34
|
+
- [A listing came back short with a cursor still set](#a-listing-came-back-short-with-a-cursor-still-set)
|
|
35
|
+
|
|
36
|
+
## Install and import
|
|
37
|
+
|
|
38
|
+
### `Cannot find package 'bun'`
|
|
39
|
+
|
|
40
|
+
**When:** importing `@nxgt/s3` under Node — the full line names the file it
|
|
41
|
+
was imported from.
|
|
42
|
+
**Why:** the client is Bun's own `S3Client`, which is why there is no AWS SDK
|
|
43
|
+
to install and why this package does not run on Node. Measured on Node 22.
|
|
44
|
+
**Fix:**
|
|
45
|
+
|
|
46
|
+
```sh
|
|
47
|
+
bun run ./src/index.ts # Bun 1.4 or later
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
### `Cannot find module 'bun' or its corresponding type declarations.`
|
|
51
|
+
|
|
52
|
+
**When:** typechecking, on the first file that imports `@nxgt/s3`.
|
|
53
|
+
**Why:** the shipped declarations import `S3Client`, `S3File` and `S3Options`
|
|
54
|
+
from `bun`, so your project needs Bun's types. They are not a dependency of
|
|
55
|
+
this package.
|
|
56
|
+
**Fix:**
|
|
57
|
+
|
|
58
|
+
```sh
|
|
59
|
+
bun add -d @types/bun
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
## Configuration
|
|
63
|
+
|
|
64
|
+
### `defineBucket: a bucket definition needs a bucket`
|
|
65
|
+
|
|
66
|
+
**When:** at `defineBucket`, with an empty `bucket`.
|
|
67
|
+
**Why:** the bucket name is what every key is written under; an empty one
|
|
68
|
+
cannot be meant.
|
|
69
|
+
**Fix:**
|
|
70
|
+
|
|
71
|
+
```ts
|
|
72
|
+
export const avatars = defineBucket({
|
|
73
|
+
bucket: 'avatars',
|
|
74
|
+
key: (p: { userId: string }) => `${p.userId}.png`,
|
|
75
|
+
});
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
### `defineBucket: "avatars" has a maxSize of 0; it is a number of bytes, and must be above zero`
|
|
79
|
+
|
|
80
|
+
**When:** at `defineBucket`.
|
|
81
|
+
**Why:** `maxSize` is a number of **bytes**, and it is inclusive: 1024 passes
|
|
82
|
+
and 1025 does not. A string body is measured in bytes, not in characters.
|
|
83
|
+
**Fix:**
|
|
84
|
+
|
|
85
|
+
```ts
|
|
86
|
+
defineBucket({ bucket: 'avatars', key, maxSize: 2 * 1024 * 1024 });
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
### `defineBucket: "avatars" accepts an empty list of content types, so nothing could ever be written. …`
|
|
90
|
+
|
|
91
|
+
**When:** at `defineBucket`, with `contentType: []`.
|
|
92
|
+
**Why:** an empty list refuses every write. The message ends with the way out,
|
|
93
|
+
``Leave `contentType` out to accept anything``: leaving the option out is what
|
|
94
|
+
accepts anything, and `[]` is never what was meant.
|
|
95
|
+
**Fix:**
|
|
96
|
+
|
|
97
|
+
```ts
|
|
98
|
+
defineBucket({ bucket: 'avatars', key, contentType: ['image/png', 'image/jpeg'] });
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
## Writes refused before they are sent
|
|
102
|
+
|
|
103
|
+
### `"avatars" accepts image/png, image/jpeg, not application/pdf`
|
|
104
|
+
|
|
105
|
+
**When:** `put`, with a `type` the definition does not list. A presigned URL
|
|
106
|
+
is never checked against it — `PresignOptions` has no `type`, and a signed
|
|
107
|
+
PUT constrains the key and the deadline and nothing else.
|
|
108
|
+
**Why:** an `S3Error` with `code: 'WRONG_TYPE'`. Nothing was sent. The type
|
|
109
|
+
is compared on its **essence**: `text/csv` accepts `text/csv;charset=utf-8`
|
|
110
|
+
and `TEXT/CSV`, because that is what real bodies carry — parameters and case
|
|
111
|
+
are ignored, nothing else is.
|
|
112
|
+
**Fix:**
|
|
113
|
+
|
|
114
|
+
```ts
|
|
115
|
+
import { S3Error } from '@nxgt/s3';
|
|
116
|
+
|
|
117
|
+
try {
|
|
118
|
+
await avatars.put({ userId }, body, { type: file.type });
|
|
119
|
+
} catch (error) {
|
|
120
|
+
if (error instanceof S3Error && error.code === 'WRONG_TYPE') return badRequest();
|
|
121
|
+
throw error;
|
|
122
|
+
}
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
### ``"avatars" accepts image/png, image/jpeg, and this write names no content type. Pass `type` ``
|
|
126
|
+
|
|
127
|
+
**When:** `put` on a bucket with `contentType`, for a body that carries no
|
|
128
|
+
type of its own — a string, a typed array, a stream.
|
|
129
|
+
**Why:** the guard cannot accept what it cannot read, so an unnamed type is
|
|
130
|
+
refused rather than guessed. `code: 'WRONG_TYPE'`.
|
|
131
|
+
**Fix:**
|
|
132
|
+
|
|
133
|
+
```ts
|
|
134
|
+
await avatars.put({ userId }, bytes, { type: 'image/png' });
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
`Bun.file(path)` carries its own type, and does not need the option.
|
|
138
|
+
|
|
139
|
+
### `"avatars" accepts 2097152 bytes at most, and this body is 5242880`
|
|
140
|
+
|
|
141
|
+
**When:** `put`, for a body whose size is known and above `maxSize`.
|
|
142
|
+
**Why:** an `S3Error` with `code: 'TOO_LARGE'`, raised before the request goes
|
|
143
|
+
out.
|
|
144
|
+
**Fix:**
|
|
145
|
+
|
|
146
|
+
```ts
|
|
147
|
+
if (error instanceof S3Error && error.code === 'TOO_LARGE') return payloadTooLarge();
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
### `"avatars" has a maxSize, and this body's size cannot be known before sending it. …`
|
|
151
|
+
|
|
152
|
+
**When:** `put` with `maxSize` set, for a `Response`, a `Request`, a stream or
|
|
153
|
+
another `S3File`.
|
|
154
|
+
**Why:** nothing can check a length it has not read — an `S3File` reports its
|
|
155
|
+
size as `NaN` until the service has been asked — so the write is refused
|
|
156
|
+
rather than streamed unchecked. `code: 'UNMEASURABLE'`. The message ends with
|
|
157
|
+
the two ways out, ``Read it into memory first, or drop `maxSize` and let the
|
|
158
|
+
service refuse it``.
|
|
159
|
+
**Fix:**
|
|
160
|
+
|
|
161
|
+
```ts
|
|
162
|
+
await avatars.put({ userId }, await response.bytes()); // read it in first
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
Or drop `maxSize` from the definition and let the service refuse an oversized
|
|
166
|
+
body.
|
|
167
|
+
|
|
168
|
+
### `storageClass must be one of STANDARD, DEEP_ARCHIVE, EXPRESS_ONEZONE, …; got "CHEAP"`
|
|
169
|
+
|
|
170
|
+
The whole line names every class the service takes, then the one it was
|
|
171
|
+
given:
|
|
172
|
+
|
|
173
|
+
```text
|
|
174
|
+
storageClass must be one of STANDARD, DEEP_ARCHIVE, EXPRESS_ONEZONE, GLACIER,
|
|
175
|
+
GLACIER_IR, INTELLIGENT_TIERING, ONEZONE_IA, OUTPOSTS, REDUCED_REDUNDANCY,
|
|
176
|
+
SNOW, STANDARD_IA; got "CHEAP"
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
**When:** `put` with a `storageClass` that is not one of those — usually a
|
|
180
|
+
value read off a request body or an environment variable, where the types
|
|
181
|
+
were not there to refuse it.
|
|
182
|
+
**Why:** an `S3Error` with `code: 'WRONG_OPTION'`, raised before the request
|
|
183
|
+
goes out, like the content type and the size. Since 0.3.0 this package
|
|
184
|
+
checks the two guarded options itself rather than letting the client refuse
|
|
185
|
+
them, so one `catch` covers every refusal of a `put`.
|
|
186
|
+
**Fix:** narrow the value against the option's own type before it reaches the
|
|
187
|
+
call:
|
|
188
|
+
|
|
189
|
+
```ts
|
|
190
|
+
import { S3Error, type PutOptions } from '@nxgt/s3';
|
|
191
|
+
|
|
192
|
+
// the classes this application allows, proved against the option's type
|
|
193
|
+
const CLASSES = ['STANDARD', 'STANDARD_IA'] as const satisfies readonly NonNullable<
|
|
194
|
+
PutOptions['storageClass']
|
|
195
|
+
>[];
|
|
196
|
+
|
|
197
|
+
function storageClassOf(value: string | undefined): PutOptions['storageClass'] {
|
|
198
|
+
return CLASSES.find((known) => known === value); // undefined: the bucket's default
|
|
199
|
+
}
|
|
200
|
+
|
|
201
|
+
try {
|
|
202
|
+
await avatars.put({ userId }, bytes, { storageClass: storageClassOf(input) });
|
|
203
|
+
} catch (error) {
|
|
204
|
+
if (error instanceof S3Error && error.code === 'WRONG_OPTION') return badRequest();
|
|
205
|
+
throw error;
|
|
206
|
+
}
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
### `acl must be one of private, public-read, public-read-write, …; got "everyone"`
|
|
210
|
+
|
|
211
|
+
```text
|
|
212
|
+
acl must be one of private, public-read, public-read-write, aws-exec-read,
|
|
213
|
+
authenticated-read, bucket-owner-read, bucket-owner-full-control,
|
|
214
|
+
log-delivery-write; got "everyone"
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
**When:** `put`, `presignGet` or `presignPut` with an `acl` that is not one
|
|
218
|
+
of those. Both paths go through the same allowlist since 0.3.0; before it,
|
|
219
|
+
`presign` handed the value to the client, which refused it with a plain
|
|
220
|
+
`TypeError` that quoted **every accepted value** (`must be one of "private",
|
|
221
|
+
"public-read", …`). This one quotes only the value it was given, so a message
|
|
222
|
+
with the allowed values in quotes means the package is older than 0.3.0.
|
|
223
|
+
**Why:** the same `code: 'WRONG_OPTION'`, before anything is sent or signed.
|
|
224
|
+
The two guarded options are `acl` and `storageClass`; `contentDisposition`
|
|
225
|
+
and `contentEncoding` are plain strings to the service and accept anything.
|
|
226
|
+
`presign` takes no `storageClass` — nothing is stored by signing a URL.
|
|
227
|
+
**Fix:**
|
|
228
|
+
|
|
229
|
+
```ts
|
|
230
|
+
await avatars.put({ userId }, bytes, { acl: 'public-read' });
|
|
231
|
+
const url = avatars.presignPut({ userId }, { acl: 'private', expiresIn: 300 });
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
### `expiresIn is seconds, and must be above 0 and at most 604800 (seven days, which is S3's own limit); got 1000000000000`
|
|
235
|
+
|
|
236
|
+
**When:** `presignGet` or `presignPut` with an `expiresIn` that is not a
|
|
237
|
+
finite number of seconds inside S3's range.
|
|
238
|
+
**Why:** an `S3Error` with `code: 'WRONG_OPTION'`, raised before anything is
|
|
239
|
+
signed. Measured on bun 1.4.2, the client refuses `0` and below with a
|
|
240
|
+
`TypeError` of its own and **signs** `1e12` happily — a URL the service then
|
|
241
|
+
rejects when somebody uses it, long after this package reported success.
|
|
242
|
+
**Fix:** pass seconds, and no more than a week:
|
|
243
|
+
|
|
244
|
+
```ts
|
|
245
|
+
const url = avatars.presignPut({ userId }, { expiresIn: 300 }); // five minutes
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
Sign for the time the page actually needs: a day is a long life for a URL
|
|
249
|
+
anyone can forward.
|
|
250
|
+
|
|
251
|
+
## The service
|
|
252
|
+
|
|
253
|
+
### `Missing S3 credentials. 'accessKeyId', 'secretAccessKey', 'bucket', and 'endpoint' are required`
|
|
254
|
+
|
|
255
|
+
**When:** the first call, when neither `bindBucket`'s options nor the
|
|
256
|
+
environment gave the client credentials. Bun's error, `code:
|
|
257
|
+
'ERR_S3_MISSING_CREDENTIALS'`.
|
|
258
|
+
**Why:** `bindBucket` creates the `S3Client` for you and passes your options
|
|
259
|
+
through; with none, Bun falls back to the environment, and to AWS's endpoint.
|
|
260
|
+
**Fix:**
|
|
261
|
+
|
|
262
|
+
```ts
|
|
263
|
+
export const avatars = bindBucket(avatarsDefinition, {
|
|
264
|
+
endpoint: process.env.S3_ENDPOINT,
|
|
265
|
+
accessKeyId: process.env.S3_KEY,
|
|
266
|
+
secretAccessKey: process.env.S3_SECRET,
|
|
267
|
+
virtualHostedStyle: false, // for a service that is not AWS
|
|
268
|
+
});
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
### `The AWS Access Key Id you provided does not exist in our records.`
|
|
272
|
+
|
|
273
|
+
**When:** the first call, with credentials the service refuses — or with no
|
|
274
|
+
`endpoint`, which sends the request to AWS whatever your service is.
|
|
275
|
+
**Why:** it is the **service's** answer, raised by Bun as an error whose
|
|
276
|
+
`name` is `S3Error` and whose `code` is the S3 code (`InvalidAccessKeyId`,
|
|
277
|
+
`SignatureDoesNotMatch`, `NoSuchBucket`, `AccessDenied`). It is **not** this
|
|
278
|
+
package's `S3Error` class: `instanceof S3Error` is false for it.
|
|
279
|
+
**Fix:**
|
|
280
|
+
|
|
281
|
+
```ts
|
|
282
|
+
try {
|
|
283
|
+
await avatars.put({ userId }, bytes);
|
|
284
|
+
} catch (error) {
|
|
285
|
+
if (error instanceof S3Error) return badRequest(error.code); // ours: the guards
|
|
286
|
+
throw error; // the service's, or Bun's: log the code it carries
|
|
287
|
+
}
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
`bytes`, `text` and `stat` are the exception: they turn S3's own `NoSuchKey`
|
|
291
|
+
into `undefined`, so `undefined` means "no such object" and nothing else.
|
|
292
|
+
Every other failure comes back as the error it is.
|
|
293
|
+
|
|
294
|
+
### A presigned upload stored a body the bucket would have refused
|
|
295
|
+
|
|
296
|
+
**When:** after handing out a `presignPut` URL. No error anywhere.
|
|
297
|
+
**Why:** a presigned PUT constrains the key and the deadline, and nothing
|
|
298
|
+
else. Measured on Bun 1.4: `X-Amz-SignedHeaders` stays `host`, so the
|
|
299
|
+
uploader's `Content-Type` is never signed and the size is never checked —
|
|
300
|
+
which is why `presignPut` takes no `type` at all. The guards are `put`'s;
|
|
301
|
+
`file(params).writer()` and `client` are Bun's own and write whatever they
|
|
302
|
+
are given.
|
|
303
|
+
**Fix:**
|
|
304
|
+
|
|
305
|
+
```ts
|
|
306
|
+
const url = avatars.presignPut({ userId }, { expiresIn: 300 });
|
|
307
|
+
// after the upload, check what actually landed
|
|
308
|
+
const stat = await avatars.stat({ userId });
|
|
309
|
+
if (!stat || stat.size > maxSize) await avatars.delete({ userId });
|
|
310
|
+
```
|
|
311
|
+
|
|
312
|
+
Set the service's own bucket policy too where it matters.
|
|
313
|
+
|
|
314
|
+
### A listing came back short with a cursor still set
|
|
315
|
+
|
|
316
|
+
**When:** `list`, on an eventually consistent service.
|
|
317
|
+
**Why:** `limit` is a maximum, not a promise.
|
|
318
|
+
**Fix:**
|
|
319
|
+
|
|
320
|
+
```ts
|
|
321
|
+
let cursor: string | null = null;
|
|
322
|
+
do {
|
|
323
|
+
const page = await avatars.list({ prefix, limit: 100, cursor });
|
|
324
|
+
cursor = page.nextCursor;
|
|
325
|
+
} while (cursor !== null);
|
|
326
|
+
```
|
|
327
|
+
|
|
328
|
+
Page until `nextCursor` is `null`, never until a page is short.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@nxgt/s3",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.0",
|
|
4
4
|
"description": "S3 on Bun's own client: buckets described once, keys built by a typed function, uploads refused before they are sent, and presigned URLs from the same definition",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -8,6 +8,7 @@
|
|
|
8
8
|
"types": "./dist/index.d.ts",
|
|
9
9
|
"files": [
|
|
10
10
|
"dist",
|
|
11
|
+
"docs",
|
|
11
12
|
"README.md",
|
|
12
13
|
"package.json",
|
|
13
14
|
"LICENSE"
|