@nxgt/s3 0.2.0 → 0.3.0

package/docs/guide/buckets.md

@@ -0,0 +1,170 @@
+# Buckets
+
+Describing a bucket once — which bucket, how a key is built, and what this
+application is willing to put there — then binding that description to
+credentials.
+
+## The smallest thing that works
+
+```ts
+import { bindBucket, defineBucket } from '@nxgt/s3';
+
+export const avatars = defineBucket({
+	bucket: 'avatars',
+	key: (p: { userId: string }) => `${p.userId}.png`,
+	contentType: ['image/png', 'image/jpeg'],
+	maxSize: 2 * 1024 * 1024, // bytes
+});
+
+const store = bindBucket(avatars, {
+	endpoint: process.env.S3_ENDPOINT,
+	accessKeyId: process.env.S3_KEY,
+	secretAccessKey: process.env.S3_SECRET,
+});
+
+store.keyFor({ userId: 'u1' }); // 'u1.png'
+```
+
+`defineBucket` talks to nothing, so the definition is what a server, a worker
+and a test import; `bindBucket` is the half that needs credentials.
+`S3Client` is built into Bun, which is why there is no SDK in `bun add` — and
+why this package does not run on Node.
+
+## The signatures
+
+```ts
+import type { S3Client, S3File, S3Options, S3Stats } from 'bun';
+
+interface BucketDefinition<P> {
+	readonly bucket: string;
+	readonly key: (params: P) => string;
+	readonly contentType?: string | readonly string[];
+	readonly maxSize?: number;
+}
+
+function defineBucket<P>(definition: BucketDefinition<P>): BucketDefinition<P>;
+
+function bindBucket<P>(
+	definition: BucketDefinition<P>,
+	options?: Omit<S3Options, 'bucket'>,
+): BoundBucket<P>;
+
+interface BoundBucket<P> {
+	readonly client: S3Client;
+	keyFor(params: P): string;
+	file(params: P): S3File;
+	put(params: P, body: PutBody, options?: PutOptions): Promise<void>;
+	bytes(params: P): Promise<Uint8Array | undefined>;
+	text(params: P): Promise<string | undefined>;
+	exists(params: P): Promise<boolean>;
+	stat(params: P): Promise<S3Stats | undefined>;
+	delete(params: P): Promise<void>;
+	list(options?: {
+		prefix?: string;
+		limit?: number;
+		cursor?: string | null;
+	}): Promise<ObjectPage>;
+	presignGet(params: P, options?: PresignOptions): string;
+	presignPut(params: P, options?: PresignOptions): string;
+}
+```
+
+## The definition
+
+| Option | Type | Default | Effect |
+| --- | --- | --- | --- |
+| `bucket` | `string` | required | the bucket's name on the service |
+| `key` | `(params: P) => string` | required | the object's key, from whatever identifies it |
+| `contentType` | `string \| readonly string[]` | anything goes | the content types this bucket accepts; a write of anything else is refused **before it is sent** |
+| `maxSize` | `number` | anything goes | the biggest body, in **bytes**, refused before it is sent |
+
+The key is a **function**, not a template, so nothing is spelled by hand at a
+call site and a renamed parameter is a compile error. `P` is whatever that
+function takes:
+
+```ts
+const uploads = defineBucket({
+	bucket: 'uploads',
+	key: (p: { folder: string; name: string }) => `${p.folder}/${p.name}`,
+});
+
+bindBucket(uploads).keyFor({ folder: 'a', name: 'b.txt' }); // 'a/b.txt'
+```
+
+`keyFor` exists so a caller that needs the string gets *the* string. Building
+one by hand somewhere else is how a bucket ends up with two spellings of the
+same object.
+
+What `contentType` and `maxSize` do is [Writing](writes.md); they are guards
+on `put`, and nothing else on the object goes through them.
+
+`defineBucket` refuses a definition that could never work, at import time:
+
+```ts
+defineBucket({ bucket: '', key: () => 'k' });
+// TypeError: defineBucket: a bucket definition needs a bucket
+
+defineBucket({ bucket: 'b', key: () => 'k', maxSize: 0 });
+// TypeError: defineBucket: "b" has a maxSize of 0; it is a number of bytes…
+
+defineBucket({ bucket: 'b', key: () => 'k', contentType: [] });
+// TypeError: … accepts an empty list of content types, so nothing could ever
+// be written. Leave `contentType` out to accept anything
+```
+
+The definition it gives back is frozen, so it cannot drift once it is shared.
+
+## Binding it
+
+`options` is Bun's own `S3Options` without `bucket` — the bucket comes from
+the definition, and a call may never change it. It is optional: given none,
+Bun reads its own `S3_*` / `AWS_*` environment variables.
+
+```ts
+const store = bindBucket(avatars); // credentials from the environment
+```
+
+| Option | Type | Default | Effect |
+| --- | --- | --- | --- |
+| `endpoint` | `string` | AWS | the service's URL; anything S3-compatible Bun can sign for |
+| `accessKeyId` / `secretAccessKey` | `string` | `$S3_*` / `$AWS_*` | the credentials |
+| `sessionToken` | `string` | `$AWS_SESSION_TOKEN` | for temporary credentials |
+| `region` | `string` | `$S3_REGION` / `$AWS_REGION` | the region to sign for |
+| `virtualHostedStyle` | `boolean` | `false` | `bucket.host` URLs instead of `host/bucket`; a service that is not AWS usually wants it left off |
+| `acl`, `storageClass` | see [Writing](writes.md) | — | a default for every object this client writes |
+| `retry`, `partSize`, `queueSize` | `number` | Bun's | the client's own transfer tuning. A [`put`](writes.md) takes none of these: it is one PUT |
+
+Each bound bucket holds an `S3Client` of its own. S3 is stateless HTTP —
+there is no connection to share, and **nothing to close**.
+
+## Everything this package does not wrap
+
+```ts
+store.client;               // Bun's S3Client
+store.file({ userId: 'u1' }); // Bun's lazy S3File
+```
+
+`file(params)` is the handle for `.stream()`, `.slice()`, `.writer()` and the
+rest, keyed by the definition rather than by a string you typed. It is also
+the way to write a body in parts, which `put` does not do.
+
+## Types a caller names
+
+```ts
+import type { BoundBucket, ParamsOf } from '@nxgt/s3';
+
+type AvatarParams = ParamsOf<typeof avatars>; // { userId: string }
+
+async function purge(bucket: BoundBucket<AvatarParams>, userId: string) {
+	await bucket.delete({ userId });
+}
+```
+
+`stat` gives back Bun's own `S3Stats`, which this package does not re-export:
+import it from `bun` where you need to name it.
+
+## Next
+
+- [Writing](writes.md) — `put`, its options, and the guards.
+- [Reading](reads.md) — `bytes`, `text`, `stat`, `exists` and `list`.
+- [Presigned URLs](presigned-urls.md) — the same definition, signed.

package/docs/guide/presigned-urls.md

@@ -0,0 +1,180 @@
+# Presigned URLs
+
+Handing a browser or another service a URL that reads or writes one object,
+for a while, without ever handing over a credential.
+
+## The smallest thing that works
+
+```ts
+import { bindBucket, defineBucket } from '@nxgt/s3';
+
+const avatars = defineBucket({
+	bucket: 'avatars',
+	key: (p: { userId: string }) => `${p.userId}.png`,
+	contentType: ['image/png', 'image/jpeg'],
+	maxSize: 2 * 1024 * 1024,
+});
+
+const store = bindBucket(avatars);
+
+const download = store.presignGet({ userId: 'u1' }, { expiresIn: 300 }); // seconds
+const upload = store.presignPut({ userId: 'u1' }, { expiresIn: 300 });
+```
+
+Both are synchronous: signing is arithmetic, not a request. The key comes
+from the definition, so a signed URL and a `put` can never disagree about
+which object they mean — see [Buckets](buckets.md).
+
+## The signatures
+
+```ts
+import type { S3Options } from 'bun';
+
+presignGet(params: P, options?: PresignOptions): string;
+presignPut(params: P, options?: PresignOptions): string;
+
+interface PresignOptions {
+	expiresIn?: number;
+	acl?: S3Options['acl'];
+}
+```
+
+## Options
+
+| Option | Type | Default | Effect |
+| --- | --- | --- | --- |
+| `expiresIn` | `number` | Bun's, one day | **seconds** until the URL expires. Always pass one. Above 604 800 (seven days, S3's own limit), at or below zero, or anything that is not a finite number is an `S3Error` with `code: 'WRONG_OPTION'` |
+| `acl` | `'private' \| 'public-read' \| …` | none | the ACL the URL is signed for, where the service honours it |
+
+`acl` goes through the same allowlist a `put` is held to, so a value the
+service does not accept is an `S3Error` with `code: 'WRONG_OPTION'` and no
+URL is signed:
+
+```ts
+import { S3Error } from '@nxgt/s3';
+
+try {
+	store.presignPut({ userId: 'u1' }, { acl: 'everyone' as never });
+} catch (error) {
+	if (error instanceof S3Error && error.code === 'WRONG_OPTION') {
+		error.key; // 'u1.png' — the object it was about
+	}
+}
+```
+
+`expiresIn` goes through the same check. Measured on bun 1.4.2, the client
+refuses `0` and below itself, and **signs** an `expiresIn` of `1e12`
+happily — a URL S3 then rejects at use time, which is the one thing this
+package exists not to do. Both ends are refused here now, before anything is
+signed.
+
+Before 0.3.0 the values went straight to the client: the same mistake was two
+different classes depending on whether it reached a `put` or a `presign`.
+There is no `storageClass` here — signing a URL stores nothing, so there is
+no class to name.
+
+A day is a long time for a URL that anyone can forward. Sign for the time the
+page actually needs:
+
+```ts
+const url = store.presignGet({ userId: 'u1' }, { expiresIn: 60 });
+const answer = await fetch(url);
+answer.status; // 200 — it really reads the object
+```
+
+## A presigned PUT constrains the key and the deadline, and nothing else
+
+Not the size. Not the content type.
+
+```ts
+const url = store.presignPut({ userId: 'u1' }, { expiresIn: 60 });
+
+await fetch(url, {
+	method: 'PUT',
+	body: Bun.file('archive.zip'),
+	headers: { 'content-type': 'application/zip' },
+});
+// 200 — stored, in a bucket whose definition says image/png only
+```
+
+That is why `presignPut` takes no `type`: it would read as a guarantee it
+cannot make. Measured on bun 1.4.2, `presign`'s `type` only adds
+`response-content-type` — S3's override for what a *download* is labelled —
+and `X-Amz-SignedHeaders` stays `host`, so the uploader's `Content-Type` is
+never signed.
+
+The bucket's `contentType` and `maxSize` are [`put`'s guards](writes.md),
+and whoever holds a signed URL is not going through `put`. Where it matters:
+
+- check with `stat` after the upload and delete what does not belong, or
+- enforce it in the service's own bucket policy.
+
+```ts
+const found = await store.stat({ userId: 'u1' });
+if (!found || !found.type.startsWith('image/')) {
+	await store.delete({ userId: 'u1' });
+}
+```
+
+## A URL is signed for the bound bucket, whatever the options say
+
+`PresignOptions` carries no `bucket`, `endpoint`, `region` or credential, and
+the run time signs with only the two keys above — anything else is dropped.
+
+```ts
+import type { PresignOptions } from '@nxgt/s3';
+
+// A bag off a request body never met the types.
+const bag = { expiresIn: 60, bucket: 'somewhere-else' } as PresignOptions;
+
+store.presignGet({ userId: 'u1' }, bag); // …/avatars/u1.png, on this endpoint
+```
+
+Measured before that filter existed, such a bag redirected the URL: a
+`bucket` key signed it for another bucket, and a credential key signed it
+against another endpoint entirely.
+
+## A real one: upload from the browser, read it back
+
+```ts
+import { Hono } from 'hono';
+import { bindBucket, defineBucket } from '@nxgt/s3';
+
+const avatars = defineBucket({
+	bucket: 'avatars',
+	key: (p: { userId: string }) => `${p.userId}.png`,
+	contentType: ['image/png', 'image/jpeg'],
+	maxSize: 2 * 1024 * 1024,
+});
+
+const store = bindBucket(avatars);
+const app = new Hono();
+
+// The browser PUTs the file straight to the service: no body through here.
+app.post('/users/:id/avatar/upload-url', (c) => {
+	const userId = c.req.param('id');
+	return c.json({
+		url: store.presignPut({ userId }, { expiresIn: 120 }),
+		key: store.keyFor({ userId }),
+	});
+});
+
+// …and the browser is told, later, where to read it.
+app.get('/users/:id/avatar', async (c) => {
+	const userId = c.req.param('id');
+	if (!(await store.exists({ userId }))) {
+		return c.json({ error: 'No avatar' }, 404);
+	}
+	return c.redirect(store.presignGet({ userId }, { expiresIn: 300 }));
+});
+```
+
+The upload route authorises the *user*; the URL it hands back authorises
+nothing else. Keep `expiresIn` close to how long an upload should take, and
+verify the object afterwards if the bucket's own policy does not.
+
+## Next
+
+- [Writing](writes.md) — the guards a presigned PUT bypasses.
+- [Reading](reads.md) — `stat` and `exists`, for checking what was uploaded.
+- [Troubleshooting](../troubleshooting.md) — a URL that 403s or expired.

package/docs/guide/reads.md

@@ -0,0 +1,186 @@
+# Reading
+
+Getting an object back — its bytes, its text, whether it is there, what the
+service knows about it — and walking a bucket page by page.
+
+## The smallest thing that works
+
+```ts
+import { bindBucket, defineBucket } from '@nxgt/s3';
+
+const reports = defineBucket({
+	bucket: 'reports',
+	key: (id: string) => `${id}.csv`,
+	contentType: 'text/csv',
+});
+
+const store = bindBucket(reports);
+
+const csv = await store.text('q1'); // string, or undefined
+if (csv === undefined) throw new Error('no such report'); // nothing stored
+```
+
+Describing and binding a bucket is [Buckets](buckets.md).
+
+## The signatures
+
+```ts
+import type { S3File, S3Stats } from 'bun';
+
+bytes(params: P): Promise<Uint8Array | undefined>;
+text(params: P): Promise<string | undefined>;
+stat(params: P): Promise<S3Stats | undefined>;
+exists(params: P): Promise<boolean>;
+file(params: P): S3File;
+
+list(options?: {
+	prefix?: string;
+	limit?: number;
+	cursor?: string | null;
+}): Promise<ObjectPage>;
+
+interface ObjectPage {
+	items: StoredObject[];
+	nextCursor: string | null;
+}
+
+interface StoredObject {
+	key: string;
+	size: number | undefined;
+	lastModified: Date | undefined;
+	eTag: string | undefined;
+}
+```
+
+## `undefined` means "no such object", and nothing else
+
+```ts
+await store.bytes('nobody'); // undefined
+await store.text('nobody');  // undefined
+await store.stat('nobody');  // undefined
+await store.exists('nobody'); // false
+```
+
+Each read goes straight at the object and turns S3's own `NoSuchKey` into
+`undefined`: one round trip, and no window in which an object deleted between
+a check and a read turns a promised `undefined` into a throw. **Every other
+failure comes back as the error it is** — a wrong secret, a refused request,
+a service that is down. A missing object and a missing permission must not
+read the same way.
+
+```ts
+const wrong = bindBucket(reports, { secretAccessKey: 'wrong-secret' });
+await wrong.text('q1'); // throws Bun's own S3Error, not undefined
+```
+
+## What the service knows
+
+```ts
+const found = await store.stat('q1');
+if (found) {
+	found.size;         // bytes
+	found.type;         // 'text/csv'
+	found.lastModified;
+	found.etag;         // note: lower case — that is Bun's field
+}
+```
+
+`stat` is Bun's own `S3Stats`, which this package does not re-export; import
+it from `bun` where you need to name it. A listing's `StoredObject` says
+`eTag`, because that is what S3 calls it there.
+
+## Big bodies, and everything else
+
+```ts
+const stream = store.file('q1').stream();
+const head = await store.file('q1').slice(0, 1024).text();
+```
+
+`file(params)` is Bun's lazy `S3File`, keyed by the definition. Use it to
+stream a body rather than hold it in memory, and to reach anything this
+package does not wrap.
+
+## Listing a bucket
+
+```ts
+let cursor: string | null = null;
+
+do {
+	const page = await store.list({ prefix: 'q1/', limit: 100, cursor });
+	for (const object of page.items) console.log(object.key, object.size);
+	cursor = page.nextCursor;
+} while (cursor !== null);
+```
+
+| Option | Type | Default | Effect |
+| --- | --- | --- | --- |
+| `prefix` | `string` | the whole bucket | only keys that start with it |
+| `limit` | `number` | the service's | the **maximum** number of objects in a page, not a promise |
+| `cursor` | `string \| null` | `null` | the `nextCursor` of the previous page |
+
+`ObjectPage` is the same shape as the `CursorPage` of `@nxgt/drizzle` and
+`@nxgt/mongo`, so a caller pages the same way everywhere. S3's
+`continuationToken` is what `nextCursor` carries, and it is `null` on the
+last page.
+
+Page **until `nextCursor` is `null`**, never until a page is short: a listing
+is eventually consistent on some services, and a page may come back with
+fewer items than `limit` while there is still more to read. A prefix with
+nothing under it gives an empty page rather than nothing:
+
+```ts
+await store.list({ prefix: 'nothing-here/' }); // { items: [], nextCursor: null }
+```
+
+`list` is the bucket's, not an object's, so it takes no key parameters — and
+it does not count: S3 does not say how many objects there are, so there is no
+`total` and no page number.
+
+## A real one: a download route
+
+```ts
+import { Hono } from 'hono';
+import { bindBucket, defineBucket } from '@nxgt/s3';
+
+const reports = defineBucket({
+	bucket: 'reports',
+	key: (id: string) => `${id}.csv`,
+	contentType: 'text/csv',
+});
+
+const store = bindBucket(reports);
+const app = new Hono();
+
+app.get('/reports/:id', async (c) => {
+	const id = c.req.param('id');
+	const found = await store.stat(id);
+	if (!found) return c.json({ error: 'No such report' }, 404);
+
+	return new Response(store.file(id).stream(), {
+		headers: {
+			'content-type': found.type,
+			'content-length': String(found.size),
+			'content-disposition': `attachment; filename="${id}.csv"`,
+		},
+	});
+});
+
+app.get('/reports', async (c) => {
+	const page = await store.list({
+		limit: 50,
+		cursor: c.req.query('cursor') ?? null,
+	});
+	return c.json(page);
+});
+```
+
+`stat` is a HEAD, so this costs one round trip before the body is streamed —
+and it is what turns a missing object into a 404 rather than a broken stream.
+
+## Next
+
+- [Writing](writes.md) — `put`, its options and the guards.
+- [Presigned URLs](presigned-urls.md) — handing the download to the client
+  instead of streaming it through your server.
+- [Troubleshooting](../troubleshooting.md) — a read that threw instead of
+  answering `undefined`.