partial-content 1.0.0 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +8 -0
- package/README.md +28 -370
- package/docs/API.md +115 -0
- package/docs/BENCHMARKS.md +85 -0
- package/docs/EXAMPLES.md +148 -0
- package/package.json +2 -2
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,13 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.0.1 (2026-07-06)
|
|
4
|
+
|
|
5
|
+
Docs-only release, no code changes.
|
|
6
|
+
|
|
7
|
+
- README restructured around evaluation flow: quick starts, comparison, and design summary up front.
|
|
8
|
+
- Deep-dives moved into the shipped `docs/` folder: full API reference (`docs/API.md`), framework/kernel recipes (`docs/EXAMPLES.md`), and the complete benchmark methodology (`docs/BENCHMARKS.md`).
|
|
9
|
+
- The npm tarball now includes the whole `docs/` folder (previously only `DESIGN.md`).
|
|
10
|
+
|
|
3
11
|
## 1.0.0 (2026-07-06)
|
|
4
12
|
|
|
5
13
|
Initial public release. Zero-dependency, ESM-only HTTP file-serving protocol layer for any storage backend.
|
package/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
[](https://www.npmjs.com/package/partial-content)
|
|
4
4
|
[](https://www.npmjs.com/package/partial-content)
|
|
5
|
-
[](https://github.com/nordvec/partial-content/actions/workflows/ci.yml)
|
|
6
6
|
[](LICENSE)
|
|
7
7
|
[](https://www.typescriptlang.org/)
|
|
8
8
|
|
|
@@ -61,66 +61,28 @@ partial-content/memory In-memory store (tests, demos, embedded assets)
|
|
|
61
61
|
partial-content/mime Curated zero-dep MIME lookup
|
|
62
62
|
```
|
|
63
63
|
|
|
64
|
-
Cloud SDKs are **optional peer dependencies**:
|
|
65
|
-
|
|
66
|
-
```bash
|
|
67
|
-
# S3 users
|
|
68
|
-
npm install partial-content @aws-sdk/client-s3
|
|
69
|
-
# add @aws-sdk/s3-request-presigner only if you use createSignedUrl()
|
|
70
|
-
|
|
71
|
-
# GCS users
|
|
72
|
-
npm install partial-content @google-cloud/storage
|
|
73
|
-
|
|
74
|
-
# Azure users
|
|
75
|
-
npm install partial-content @azure/storage-blob
|
|
76
|
-
|
|
77
|
-
# Filesystem or kernel only (zero extra deps)
|
|
78
|
-
npm install partial-content
|
|
79
|
-
```
|
|
64
|
+
Cloud SDKs are **optional peer dependencies**: `@aws-sdk/client-s3` for `/s3` (plus `@aws-sdk/s3-request-presigner` only if you use `createSignedUrl()`), `@google-cloud/storage` for `/gcs`, `@azure/storage-blob` for `/azure`, `hono` for `/hono`. The kernel, `/web`, `/node`, `/fs`, `/http`, `/r2`, `/memory`, and `/mime` need nothing beyond the platform.
|
|
80
65
|
|
|
81
66
|
## Features
|
|
82
67
|
|
|
83
68
|
- **One call does everything**: `evaluateConditionalRequest()` handles the complete evaluation chain (412 > 304 > If-Range > Range) in correct order
|
|
84
69
|
- **Write-side OCC**: `evaluateConditionalWrite()` handles If-Match/If-None-Match for PUT/PATCH/DELETE with correct 412 semantics
|
|
85
70
|
- **RFC 9530 Repr-Digest**: End-to-end integrity verification via `sha-256=:<base64>:` header, with `Want-Repr-Digest` negotiation -- first-class support that `send`, `sirv`, and the framework static middlewares lack
|
|
86
|
-
- **Built-in storage adapters**: S3, R2
|
|
87
|
-
- **Built-in framework adapters**: Fetch API (Next.js
|
|
88
|
-
- Range requests (206
|
|
89
|
-
- Conditional requests (304
|
|
71
|
+
- **Built-in storage adapters**: S3-compatible (AWS, R2 S3-mode, Hetzner, MinIO, Backblaze, Wasabi), R2 native, GCS, Azure, local filesystem, any range-capable HTTP origin, in-memory
|
|
72
|
+
- **Built-in framework adapters**: Fetch API (Next.js, SvelteKit, Remix, Nuxt, Astro, Workers, Bun.serve, Deno.serve), Node.js (Express/Fastify/Koa/raw http), Hono
|
|
73
|
+
- Range requests (206, 416), including multi-range `multipart/byteranges` with overlapping/adjacent-range coalescing and range-amplification defense (`maxRanges`)
|
|
74
|
+
- Conditional requests (304, 412) with sub-second timestamp flooring
|
|
90
75
|
- ETag generation from storage metadata (strong for content hash, weak for size+mtime, safe undefined fallback)
|
|
91
76
|
- Content-Disposition with non-ASCII filename encoding, CRLF injection prevention, path traversal protection, and bidi spoofing defense
|
|
92
77
|
- Published `ObjectStore` interface for building custom storage adapters against a stable contract
|
|
93
|
-
- Pure functions, zero I/O, zero dependencies; the hot path constructs no fetch primitives, no stream machinery for small bodies, and re-parses no dates
|
|
78
|
+
- Pure functions, zero I/O, zero dependencies; the hot path constructs no fetch primitives, no stream machinery for small bodies, and re-parses no dates
|
|
94
79
|
- ESM-only. Works across Node.js 20+, Bun, Deno, Cloudflare Workers, and edge runtimes
|
|
95
80
|
|
|
96
|
-
### Storage Adapters
|
|
97
|
-
|
|
98
|
-
| Adapter | Backends | Extra Dependencies |
|
|
99
|
-
|---------|----------|--------------------|
|
|
100
|
-
| `partial-content/s3` | AWS S3, R2 (S3 mode), Hetzner, MinIO, Backblaze, Wasabi | `@aws-sdk/client-s3` |
|
|
101
|
-
| `partial-content/r2` | Cloudflare R2 (native) | None (uses R2 bindings) |
|
|
102
|
-
| `partial-content/gcs` | Google Cloud Storage | `@google-cloud/storage` |
|
|
103
|
-
| `partial-content/azure` | Azure Blob Storage | `@azure/storage-blob` |
|
|
104
|
-
| `partial-content/fs` | Local filesystem | None (Node.js builtins) |
|
|
105
|
-
| `partial-content/http` | Supabase Storage, presigned URLs, CDNs, any range-capable HTTP origin | None (global fetch) |
|
|
106
|
-
| `partial-content/memory` | In-memory objects (tests, demos, embedded assets) | None |
|
|
107
|
-
|
|
108
|
-
### Framework Adapters
|
|
109
|
-
|
|
110
|
-
| Adapter | Works With |
|
|
111
|
-
|---------|------------|
|
|
112
|
-
| `partial-content/web` | Next.js, SvelteKit, Remix, Nuxt 3, SolidStart, Astro, Fresh (Deno), Elysia (Bun), Cloudflare Workers, Bun.serve, Deno.serve, and any Fetch API runtime |
|
|
113
|
-
| `partial-content/node` | Express, Fastify, Koa, NestJS, Angular SSR, raw `http.createServer` |
|
|
114
|
-
| `partial-content/hono` | Hono (all runtimes) |
|
|
115
|
-
| Kernel only | Anything. Pure functions, zero runtime assumptions. |
|
|
116
|
-
|
|
117
81
|
## Quick Start
|
|
118
82
|
|
|
119
|
-
|
|
83
|
+
The handler manages the full HTTP protocol (200, 206, 304, 412, 416, HEAD) automatically: combine a storage adapter with a framework adapter.
|
|
120
84
|
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
#### Next.js / SvelteKit / Remix (Fetch API)
|
|
85
|
+
### Next.js / SvelteKit / Remix (Fetch API)
|
|
124
86
|
|
|
125
87
|
```typescript
|
|
126
88
|
import { serveObject } from "partial-content/web";
|
|
@@ -138,7 +100,7 @@ export async function GET(req: Request, { params }: { params: { id: string } })
|
|
|
138
100
|
export const HEAD = GET;
|
|
139
101
|
```
|
|
140
102
|
|
|
141
|
-
|
|
103
|
+
### Express / Node.js
|
|
142
104
|
|
|
143
105
|
```typescript
|
|
144
106
|
import express from "express";
|
|
@@ -156,42 +118,7 @@ app.get("/files/:key", serveObject(store, {
|
|
|
156
118
|
}));
|
|
157
119
|
```
|
|
158
120
|
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
```typescript
|
|
162
|
-
import { Hono } from "hono";
|
|
163
|
-
import { serveObject } from "partial-content/hono";
|
|
164
|
-
import { s3Store } from "partial-content/s3";
|
|
165
|
-
|
|
166
|
-
const store = s3Store({ client, bucket: "media" });
|
|
167
|
-
const app = new Hono();
|
|
168
|
-
|
|
169
|
-
app.get("/media/:key", serveObject(store, {
|
|
170
|
-
key: (c) => c.req.param("key"),
|
|
171
|
-
cacheControl: "public, max-age=31536000, immutable",
|
|
172
|
-
}));
|
|
173
|
-
```
|
|
174
|
-
|
|
175
|
-
#### Cloudflare Workers (R2 native, no AWS SDK)
|
|
176
|
-
|
|
177
|
-
```typescript
|
|
178
|
-
import { Hono } from "hono";
|
|
179
|
-
import { serveObject } from "partial-content/hono";
|
|
180
|
-
import { r2Store } from "partial-content/r2";
|
|
181
|
-
|
|
182
|
-
const app = new Hono<{ Bindings: { MY_BUCKET: R2Bucket } }>();
|
|
183
|
-
|
|
184
|
-
app.get("/files/:key", (c) => {
|
|
185
|
-
const store = r2Store({ bucket: c.env.MY_BUCKET });
|
|
186
|
-
return serveObject(store, { key: (c) => c.req.param("key") })(c);
|
|
187
|
-
});
|
|
188
|
-
```
|
|
189
|
-
|
|
190
|
-
### Low-Level: Kernel Only
|
|
191
|
-
|
|
192
|
-
For custom integrations, use the kernel primitives directly. You control the storage I/O, the kernel handles the protocol.
|
|
193
|
-
|
|
194
|
-
#### One call does everything
|
|
121
|
+
### Kernel only (bring your own I/O)
|
|
195
122
|
|
|
196
123
|
```typescript
|
|
197
124
|
import { evaluateConditionalRequest } from "partial-content";
|
|
@@ -218,117 +145,7 @@ const { stream } = range
|
|
|
218
145
|
return new Response(stream, { status, headers });
|
|
219
146
|
```
|
|
220
147
|
|
|
221
|
-
|
|
222
|
-
Request evaluateConditionalRequest()
|
|
223
|
-
│ │
|
|
224
|
-
│ If-Match / If-Unmodified-Since ├──► 412 Precondition Failed
|
|
225
|
-
│ If-None-Match / If-Modified-Since ├──► 304 Not Modified
|
|
226
|
-
│ If-Range + Range ├──► 416 Range Not Satisfiable
|
|
227
|
-
│ ├──► 206 Partial Content
|
|
228
|
-
│ └──► 200 OK
|
|
229
|
-
│
|
|
230
|
-
▼
|
|
231
|
-
Fetch bytes from storage (you control this)
|
|
232
|
-
│
|
|
233
|
-
▼
|
|
234
|
-
new Response(body, { status, headers })
|
|
235
|
-
```
|
|
236
|
-
|
|
237
|
-
#### Node.js / Express (kernel only)
|
|
238
|
-
|
|
239
|
-
```typescript
|
|
240
|
-
import { fromNodeHeaders, evaluateConditionalRequest } from "partial-content";
|
|
241
|
-
|
|
242
|
-
app.get("/files/:key", (req, res) => {
|
|
243
|
-
const headers = fromNodeHeaders(req.headers);
|
|
244
|
-
const { status, headers: resHeaders, range } = evaluateConditionalRequest(
|
|
245
|
-
headers,
|
|
246
|
-
{ totalSize: fileSize, etag, lastModified, contentType },
|
|
247
|
-
);
|
|
248
|
-
res.writeHead(status, resHeaders);
|
|
249
|
-
// ...
|
|
250
|
-
});
|
|
251
|
-
```
|
|
252
|
-
|
|
253
|
-
### Content-Disposition
|
|
254
|
-
|
|
255
|
-
```typescript
|
|
256
|
-
import { buildContentDisposition } from "partial-content";
|
|
257
|
-
|
|
258
|
-
buildContentDisposition("report.pdf");
|
|
259
|
-
// => 'attachment; filename=report.pdf'
|
|
260
|
-
|
|
261
|
-
buildContentDisposition("Årlig_Rapport.pdf");
|
|
262
|
-
// => 'attachment; filename="?rlig_Rapport.pdf"; filename*=UTF-8''%C3%85rlig_Rapport.pdf'
|
|
263
|
-
|
|
264
|
-
buildContentDisposition("slides.pdf", { type: "inline" });
|
|
265
|
-
// => 'inline; filename=slides.pdf'
|
|
266
|
-
|
|
267
|
-
// Handles untrusted input safely
|
|
268
|
-
buildContentDisposition("../../etc/passwd"); // Path traversal stripped
|
|
269
|
-
buildContentDisposition("evil\r\nX-Injected: yes"); // CRLF injection stripped
|
|
270
|
-
buildContentDisposition(null, { fallback: "export.csv" }); // Graceful fallback
|
|
271
|
-
```
|
|
272
|
-
|
|
273
|
-
### RFC 9530 Repr-Digest (End-to-End Integrity)
|
|
274
|
-
|
|
275
|
-
Pass a SHA-256 digest from your storage backend for automatic `Repr-Digest` headers:
|
|
276
|
-
|
|
277
|
-
```typescript
|
|
278
|
-
const { status, headers, range } = evaluateConditionalRequest(
|
|
279
|
-
request.headers,
|
|
280
|
-
{
|
|
281
|
-
totalSize: fileSize,
|
|
282
|
-
etag: '"abc123"',
|
|
283
|
-
// S3: x-amz-checksum-sha256, GCS: x-goog-hash (sha256 component)
|
|
284
|
-
digest: "d2VsY29tZQ==", // raw base64 SHA-256
|
|
285
|
-
},
|
|
286
|
-
);
|
|
287
|
-
// Response headers include: Repr-Digest: sha-256=:d2VsY29tZQ==:
|
|
288
|
-
// Same digest on both 200 (full) and 206 (partial) -- covers the full representation
|
|
289
|
-
```
|
|
290
|
-
|
|
291
|
-
### Advanced: Manual Primitives
|
|
292
|
-
|
|
293
|
-
For full control over the evaluation chain:
|
|
294
|
-
|
|
295
|
-
```typescript
|
|
296
|
-
import {
|
|
297
|
-
parseRangeHeader,
|
|
298
|
-
buildRangeResponseHeaders,
|
|
299
|
-
isConditionalFresh,
|
|
300
|
-
isPreconditionFailure,
|
|
301
|
-
isRangeFresh,
|
|
302
|
-
build304Headers,
|
|
303
|
-
build412Headers,
|
|
304
|
-
build416Headers,
|
|
305
|
-
} from "partial-content";
|
|
306
|
-
|
|
307
|
-
// Step 1: Preconditions (If-Match / If-Unmodified-Since)
|
|
308
|
-
if (isPreconditionFailure(reqHeaders, etag, lastModified)) {
|
|
309
|
-
return new Response(null, build412Headers());
|
|
310
|
-
}
|
|
311
|
-
|
|
312
|
-
// Step 2: Freshness (If-None-Match / If-Modified-Since)
|
|
313
|
-
if (isConditionalFresh(reqHeaders, etag, lastModified)) {
|
|
314
|
-
return new Response(null, build304Headers(etag, lastModified));
|
|
315
|
-
}
|
|
316
|
-
|
|
317
|
-
// Step 3: Range (If-Range + Range header)
|
|
318
|
-
const range = isRangeFresh(reqHeaders, etag, lastModified)
|
|
319
|
-
? parseRangeHeader(reqHeaders.get("range"), fileSize)
|
|
320
|
-
: null;
|
|
321
|
-
|
|
322
|
-
if (range === "unsatisfiable") {
|
|
323
|
-
return new Response(null, build416Headers(fileSize));
|
|
324
|
-
}
|
|
325
|
-
|
|
326
|
-
const { status, headers } = buildRangeResponseHeaders({
|
|
327
|
-
totalSize: fileSize, range, contentType, etag, lastModified,
|
|
328
|
-
digest: checksum, // RFC 9530 Repr-Digest
|
|
329
|
-
cacheControl: "private, no-cache",
|
|
330
|
-
});
|
|
331
|
-
```
|
|
148
|
+
More recipes in **[docs/EXAMPLES.md](docs/EXAMPLES.md)**: Hono, Cloudflare Workers (R2 native), kernel-only Express, Content-Disposition, Repr-Digest, and the manual step-by-step primitives.
|
|
332
149
|
|
|
333
150
|
## Real-world example: authorized proxy from object storage
|
|
334
151
|
|
|
@@ -372,149 +189,34 @@ async function serveFile(request: Request, key: string) {
|
|
|
372
189
|
|
|
373
190
|
This is the path behind `<video>`/`<audio>` seeking and PDF.js progressive loading: the media element sends `Range` to **your** origin, you re-check access, and stream just that slice from storage. If you could hand the client a signed URL instead, the storage backend would speak this protocol for you and you wouldn't need a protocol layer -- see [Scope](docs/DESIGN.md#scope) for when this library earns its place.
|
|
374
191
|
|
|
375
|
-
## API
|
|
376
|
-
|
|
377
|
-
### Kernel (`partial-content`)
|
|
378
|
-
|
|
379
|
-
**`evaluateConditionalRequest(reqHeaders, meta)`** - One-call handler for the full HTTP evaluation chain (GET/HEAD). Returns `{ status, headers, range }`.
|
|
380
|
-
|
|
381
|
-
**`evaluateConditionalWrite(reqHeaders, meta)`** - One-call handler for write requests (PUT/PATCH/DELETE). Returns `{ proceed: true }` or `{ proceed: false, status: 412, headers }`. The 412 response includes the current `ETag` when available, so the client can resync without a follow-up GET.
|
|
382
|
-
|
|
383
|
-
**`parseRangeHeader(rangeHeader, totalSize)`** - Returns `{ start, end }`, `"unsatisfiable"`, or `null`.
|
|
384
|
-
|
|
385
|
-
**`buildRangeResponseHeaders(opts)`** - Build 200 or 206 response headers.
|
|
386
|
-
|
|
387
|
-
**`parseContentRange(header)`** - Parse a `Content-Range` response header (e.g. `bytes 0-499/1000`). Returns `{ start, end, totalSize }` or `null`.
|
|
388
|
-
|
|
389
|
-
**`generateETag(source)`** - Derive an entity-tag from storage metadata. Returns a strong `"<hash>"` when a content digest is available, a weak `W/"<size>-<mtime>"` when only size and modification time are known, or `undefined` when there is insufficient metadata.
|
|
390
|
-
|
|
391
|
-
**`buildContentDisposition(filename, options?)`** - Security-hardened `Content-Disposition` header builder with CRLF injection prevention, path traversal protection, bidi override stripping, and RFC 8187 non-ASCII encoding.
|
|
392
|
-
|
|
393
|
-
**`fromNodeHeaders(headers)`** - Convert Node.js `IncomingHttpHeaders` to the `{ get(name) }` interface.
|
|
394
|
-
|
|
395
|
-
**`isConditionalFresh(reqHeaders, etag, lastModified)`** - `true` if not modified (304).
|
|
396
|
-
|
|
397
|
-
**`isPreconditionFailure(reqHeaders, etag, lastModified, exists?)`** - `true` if precondition failed (412). Pass `exists` when the resource's presence is known independently of its validators (e.g. `If-Match: *` upload guards).
|
|
398
|
-
|
|
399
|
-
**`isRangeFresh(reqHeaders, etag, lastModified)`** - `true` if If-Range passes (honor the range).
|
|
400
|
-
|
|
401
|
-
**`build304Headers(etag, lastModified, cacheControl?)`** - Build 304 headers.
|
|
402
|
-
|
|
403
|
-
**`build412Headers()`** - Build 412 headers.
|
|
404
|
-
|
|
405
|
-
**`build416Headers(totalSize)`** - Build 416 Range Not Satisfiable headers.
|
|
406
|
-
|
|
407
|
-
**`clientWantsDigest(reqHeaders)`** - RFC 9530 Section 4 negotiation: `true` when the client's `Want-Repr-Digest` / `Want-Content-Digest` headers accept `sha-256` (or are absent). The web adapter and orchestrator both honor this, so `Want-Repr-Digest: sha-256=0` suppresses digest emission everywhere.
|
|
408
|
-
|
|
409
|
-
### MIME Lookup (`partial-content/mime`)
|
|
410
|
-
|
|
411
|
-
**`lookupMime(filenameOrExt)`** - Curated, zero-dependency extension -> MIME lookup for documents, media, archives, fonts, and web assets. Case-insensitive, resolves the last dot segment (`archive.tar.gz` -> `application/gzip`), returns `undefined` for unknown types so the caller controls the fallback. `html` is deliberately absent: serving stored uploads as `text/html` is stored XSS, so that decision must be explicit at the call site.
|
|
412
|
-
|
|
413
|
-
```typescript
|
|
414
|
-
import { lookupMime } from "partial-content/mime";
|
|
415
|
-
|
|
416
|
-
app.get("/files/:key", serveObject(store, {
|
|
417
|
-
key: (req) => req.params.key,
|
|
418
|
-
mime: (req) => lookupMime(req.params.key),
|
|
419
|
-
}));
|
|
420
|
-
```
|
|
421
|
-
|
|
422
|
-
### Universal HTTP Store (`partial-content/http`)
|
|
423
|
-
|
|
424
|
-
**`httpStore({ url, headers?, fetch?, redirect? })`** - Serve from ANY range-capable HTTP origin over plain `fetch`: Supabase Storage, presigned S3/GCS/Azure URLs, CDN origins, or another partial-content server. Pinned reads map to `If-Match` (origin 412 -> `ObjectChangedError`), `Repr-Digest` response headers are extracted, and requests are sent `Accept-Encoding: identity` and any response that still carries a non-identity `Content-Encoding` is refused, so transparent compression can never corrupt byte accounting. Redirects error by default (a hostile origin must not 3xx the store toward internal/metadata IPs); set `redirect: "follow"` for origins that legitimately redirect, paired with a validating `fetch` when keys are untrusted (see SECURITY.md).
|
|
425
|
-
|
|
426
|
-
```typescript
|
|
427
|
-
import { httpStore } from "partial-content/http";
|
|
428
|
-
|
|
429
|
-
const store = httpStore({
|
|
430
|
-
url: (key) => `${SUPABASE_URL}/storage/v1/object/documents/${key}`,
|
|
431
|
-
headers: { Authorization: `Bearer ${serviceRoleKey}` },
|
|
432
|
-
});
|
|
433
|
-
```
|
|
434
|
-
|
|
435
|
-
### Memory Store (`partial-content/memory`)
|
|
436
|
-
|
|
437
|
-
**`memoryStore({ objects })`** - A spec-faithful in-memory store for consumer test suites, demos, and small embedded assets. Fabricates correct Content-Range values, honors `ifMatch` pinning (mutate the map to simulate overwrites and exercise retry logic), and streams zero-byte objects correctly.
|
|
438
|
-
|
|
439
|
-
### Web Adapter (`partial-content/web`)
|
|
440
|
-
|
|
441
|
-
**`serveObject(store, options?)`** - Create a Fetch API handler that serves files from an ObjectStore. Returns `(req: Request, ctx: ServeContext) => Promise<Response>`.
|
|
442
|
-
|
|
443
|
-
**`serveObjectRaw(store, options?)`** - The same engine returning `RawResponseParts` (`{ status, statusText, headers, body }`) instead of a `Response`, for server adapters that write to their runtime natively (the bundled node adapter uses it). Skips all fetch-primitive construction on the hot path.
|
|
444
|
-
|
|
445
|
-
Options: `disposition`, `cacheControl`, `immutable`, `securityHeaders`, `crossOriginResourcePolicy`, `timingAllowOrigin`, `timing`, `onTiming`, `onError`, `onServe`, `onTransfer`, `maxRanges`, `enforceCharset`, `fallbackFilename`.
|
|
446
|
-
|
|
447
|
-
### Node Adapter (`partial-content/node`)
|
|
448
|
-
|
|
449
|
-
**`serveObject(store, options)`** - Create a Node.js `(req, res) => Promise<void>` handler for Express, Fastify (compat), Koa, and raw `http.createServer`. Extends the web adapter options with `key` (required, extracts the storage key from `IncomingMessage`), `mime?`, and `filename?`.
|
|
450
|
-
|
|
451
|
-
**`writeStallTimeoutMs?`** (default `60000`) - Bounds how long the streaming pump waits for a single backpressure `drain` before treating the client as stalled and tearing the transfer down (cancel the storage read, destroy the response). A client that stops reading but holds its socket open would otherwise pin a backend storage connection indefinitely (a slow-read attack). Set to `0` to disable and rely on an upstream proxy / socket timeout instead. Only the raw-Node pump needs this; Fetch-runtime backpressure is the platform's own concern.
|
|
192
|
+
## API
|
|
452
193
|
|
|
453
|
-
|
|
194
|
+
The full reference lives in **[docs/API.md](docs/API.md)**. The shape of the surface:
|
|
454
195
|
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
**`ObjectStore`** (interface) - Read-only storage backend abstraction. Implementations provide `headObject(key, opts?)` for metadata and `getObject(key, opts?)` for streaming, where `opts` carries `range`, `signal`, `ifMatch` (pinned reads), and `pin` (an opaque token issued by `headObject` for stores whose version identifier is not the ETag; GCS uses it to stream a pinned generation without re-fetching metadata). Optional `createSignedUrl(key, opts)` for backends that cannot stream ranges through the origin. Optional `authoritativeRange: true` declares that ranged responses report the backend's ACTUAL served bounds (parsed Content-Range) -- the web adapter then serves plain range requests in a single round-trip with no validating HEAD (S3, Azure, R2, and http set it; video seeking and PDF.js chunking hit this path constantly).
|
|
460
|
-
|
|
461
|
-
**`ObjectMetadata`** (type) - HEAD response shape: `contentLength`, `etag?`, `lastModified?`, `digest?`.
|
|
462
|
-
|
|
463
|
-
**`ObjectStream`** (type) - GET response shape: `body` (a `ReadableStream`, or a plain `Uint8Array` when the adapter already holds the exact bytes -- consumers then skip stream machinery entirely), `contentLength`, `totalSize`, `range?` (the `{ start, end }` the backend ACTUALLY served; absent = full content), `etag?`, `lastModified?`, `digest?`.
|
|
464
|
-
|
|
465
|
-
**`classifyStoreRead(key, op, classifiers)`** - The ordered error-classification pipeline the built-in SDK adapters share, exported for custom adapter authors. Runs `op()` and maps its failure to the contract's error types in a fixed precedence: `notFound` -> `ObjectNotFoundError` (404), `changed` -> `ObjectChangedError` (412), `throttled` -> `StoreUnavailableError` (503), otherwise rethrow untouched. Supply one `StoreErrorClassifiers` set and reuse it for both `headObject` and `getObject` so the two paths cannot drift; predicates must be mutually exclusive on a given backend.
|
|
466
|
-
|
|
467
|
-
```typescript
|
|
468
|
-
import { classifyStoreRead, type StoreErrorClassifiers } from "partial-content";
|
|
469
|
-
|
|
470
|
-
const classifiers: StoreErrorClassifiers = {
|
|
471
|
-
notFound: (e) => (e as { statusCode?: number }).statusCode === 404,
|
|
472
|
-
changed: (e) => (e as { statusCode?: number }).statusCode === 412, // omit if the pin is an etag compare
|
|
473
|
-
throttled: (e) => (e as { statusCode?: number }).statusCode === 503,
|
|
474
|
-
};
|
|
475
|
-
|
|
476
|
-
const meta = await classifyStoreRead(key, () => backend.head(key), classifiers);
|
|
477
|
-
```
|
|
478
|
-
|
|
479
|
-
**`StoreUnavailableError`** (class) - Throw from an adapter when the backend is transiently unavailable (throttled/overloaded after the adapter's own retries). Carries an optional `retryAfterSeconds` echoed as `Retry-After`. Distinct from a malformed-response `502`: this is the retryable `503` case.
|
|
196
|
+
- **Kernel** (`partial-content`): `evaluateConditionalRequest` / `evaluateConditionalWrite` (the one-call orchestrators), the step-by-step primitives (`parseRangeHeader`, `parseRanges`, `isConditionalFresh`, `isPreconditionFailure`, `isRangeFresh`, the `build*Headers` family), `generateETag`, `buildContentDisposition`, `clientWantsDigest`, `fromNodeHeaders`, `sanitizeHeaderValue`
|
|
197
|
+
- **Serving** (`/web`, `/node`, `/hono`): `serveObject` handlers with `disposition`, `cacheControl` (verbatim passthrough), security headers, `onServe` / `onTransfer` / `onError` / `onTiming` observability hooks, `maxRanges`, and a slow-read stall bound on the Node pump
|
|
198
|
+
- **Stores** (`/s3`, `/r2`, `/gcs`, `/azure`, `/fs`, `/http`, `/memory`): ready-made `ObjectStore` implementations with pinned reads and truthful error mapping (404 / retryable 503 + `Retry-After` / 502)
|
|
199
|
+
- **Custom adapters**: the published `ObjectStore` contract plus the primitives the built-ins are made of (`classifyStoreRead`, `nodeStreamToWeb`, `guardStreamLength`, `resolveServedRange`)
|
|
480
200
|
|
|
481
201
|
## Design Decisions
|
|
482
202
|
|
|
483
|
-
**Multi-range is served as `multipart/byteranges`.** Overlapping and adjacent ranges are coalesced, and a range-amplification defense (`maxRanges`, default 50; plus a "ranges cover the whole file" check) degrades pathological requests to a full 200. The single-range fast path is untouched.
|
|
203
|
+
**Multi-range is served as `multipart/byteranges`.** Overlapping and adjacent ranges are coalesced, and a range-amplification defense (`maxRanges`, default 50; plus a "ranges cover the whole file" check) degrades pathological requests to a full 200. The single-range fast path is untouched.
|
|
484
204
|
|
|
485
205
|
**Weak ETag matching.** Storage providers (S3, R2, GCS) emit `W/` prefixes inconsistently. We strip `W/` for pragmatic matching to avoid false 412s.
|
|
486
206
|
|
|
487
207
|
**Sub-second timestamp flooring.** Storage backends return ISO-8601 with milliseconds. HTTP dates use whole seconds. All comparisons floor both sides to prevent permanent false-stale results.
|
|
488
208
|
|
|
489
|
-
**Atomic pinned reads (TOCTOU elimination).** After validating conditionals against HEAD metadata, the web adapter pins the GET to that exact representation via the store's native conditional read (S3 `IfMatch`, R2 `onlyIf.etagMatches`, Azure `conditions.ifMatch`, GCS generation pinning). If the object changes in the HEAD->GET window, the store throws `ObjectChangedError` and the request is re-validated once against the new state
|
|
209
|
+
**Atomic pinned reads (TOCTOU elimination).** After validating conditionals against HEAD metadata, the web adapter pins the GET to that exact representation via the store's native conditional read (S3 `IfMatch`, R2 `onlyIf.etagMatches`, Azure `conditions.ifMatch`, GCS generation pinning). If the object changes in the HEAD->GET window, the store throws `ObjectChangedError` and the request is re-validated once against the new state. For stores that cannot pin, a response-side guard remains: validators come from the GET response, the emitted 206 bounds come from the backend's actual Content-Range, and a missing Content-Range degrades to 200 (never a lying 206).
|
|
490
210
|
|
|
491
|
-
**Single-round-trip range serving.** Plain range requests (no conditionals, no `If-Range`) skip the HEAD entirely on `authoritativeRange` stores: one GET, with validators and bounds taken from the response itself -- inherently TOCTOU-atomic, and half the latency on media seeks.
|
|
211
|
+
**Single-round-trip range serving.** Plain range requests (no conditionals, no `If-Range`) skip the HEAD entirely on `authoritativeRange` stores: one GET, with validators and bounds taken from the response itself -- inherently TOCTOU-atomic, and half the latency on media seeks.
|
|
492
212
|
|
|
493
|
-
**Store failures map to the truthful status.**
|
|
213
|
+
**Store failures map to the truthful status.** Missing object -> `404`. Transiently unavailable backend (throttling/overload) -> `503` + `Retry-After` (`StoreUnavailableError`). Malformed upstream response -> `502`. Every error response carries `Cache-Control: no-store`, `nosniff`, and a `default-src 'none'` CSP.
|
|
494
214
|
|
|
495
215
|
See [docs/DESIGN.md](docs/DESIGN.md) for full RFC deviation notes, response header matrix, and parsing details.
|
|
496
216
|
|
|
497
217
|
## Benchmarks
|
|
498
218
|
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
| Function | Throughput |
|
|
502
|
-
|----------|-----------|
|
|
503
|
-
| `parseRangeHeader` | 10.1M ops/sec |
|
|
504
|
-
| `isPreconditionFailure` | 10.2M ops/sec |
|
|
505
|
-
| `isRangeFresh` | 13.9M ops/sec |
|
|
506
|
-
| `isConditionalFresh` | 4.7M ops/sec |
|
|
507
|
-
| `buildRangeResponseHeaders` | 4.2M ops/sec |
|
|
508
|
-
| `evaluateConditionalRequest` | 1.7M ops/sec |
|
|
509
|
-
| `buildContentDisposition` | 1.7M ops/sec |
|
|
510
|
-
|
|
511
|
-
### End-to-end vs `send` and `sirv`
|
|
512
|
-
|
|
513
|
-
Full HTTP serving (Node 24, `http.createServer`, loopback, 40 connections,
|
|
514
|
-
autocannon, identical fixtures; every cell correctness-verified before
|
|
515
|
-
timing; the load generator runs in a separate process so its parse cost
|
|
516
|
-
never throttles the server column). partial-content runs its shipped dist
|
|
517
|
-
through the node adapter + fsStore. Reproduce with `npm run bench`.
|
|
219
|
+
Full HTTP serving vs `send` and `sirv` (Node 24, loopback, out-of-process autocannon, every cell correctness-verified before timing; reproduce with `npm run bench`):
|
|
518
220
|
|
|
519
221
|
| Scenario | partial-content | + `cache` | send | sirv |
|
|
520
222
|
|---|---|---|---|---|
|
|
@@ -523,55 +225,11 @@ through the node adapter + fsStore. Reproduce with `npm run bench`.
|
|
|
523
225
|
| Range 64 KB of 1 MB (206) | 1,441 req/s | 1,413 req/s | 1,403 req/s | 1,443 req/s |
|
|
524
226
|
| Revalidation (304) | 11,716 req/s | **19,542 req/s** | 10,081 req/s | 23,774 req/s* |
|
|
525
227
|
|
|
526
|
-
|
|
527
|
-
|
|
528
|
-
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
files and their ranges serve from memory with zero syscalls. The trade is explicit: after an overwrite, responses (including
|
|
532
|
-
304 revalidations) can affirm the previous representation for up to
|
|
533
|
-
`ttlMs`; size it against acceptable staleness (see DESIGN.md).
|
|
534
|
-
|
|
535
|
-
\* sirv's 304 figure buys throughput with a different contract: at
|
|
536
|
-
`dev: false` it pre-renders complete header sets for a directory snapshot
|
|
537
|
-
taken at boot, so a file created after startup is a 404. In the
|
|
538
|
-
configuration that CAN see runtime-created files (`dev: true`), the same
|
|
539
|
-
run measures sirv at 4,957 req/s -- a quarter of the `cache` column. For a
|
|
540
|
-
fixed directory of immutable assets sirv's trade is exactly right; for
|
|
541
|
-
object storage (where uploads happen while the server runs) it is
|
|
542
|
-
disqualifying, which is why the boot-snapshot design stays a non-goal here.
|
|
543
|
-
On the rows where server code matters (4 KB bodies, revalidations), the
|
|
544
|
-
`cache` column is the fastest contender that can serve a file uploaded a
|
|
545
|
-
second ago; at 64 KB and above, all four columns converge on I/O parity.
|
|
546
|
-
|
|
547
|
-
Same code on Bun 1.3 (`Bun.serve`, same machine, same out-of-process
|
|
548
|
-
client -- the web adapter's Request/Response ARE the runtime's native
|
|
549
|
-
primitives there, no node bridge): GET 4 KB 9,418 req/s plain and
|
|
550
|
-
22,898 req/s with the cache; revalidation 12,538 plain and **38,090 req/s**
|
|
551
|
-
with the cache -- a TTL-revalidating cache serving runtime uploads,
|
|
552
|
-
outrunning even sirv's boot-frozen Node figure by 60%. `send` and `sirv`
|
|
553
|
-
are node-stream libraries and cannot ride `Bun.serve` natively. Reproduce:
|
|
554
|
-
`bun bench/bun-server.ts` (plain :18778, cache :18779) + autocannon.
|
|
555
|
-
|
|
556
|
-
Read this fairly in both directions:
|
|
557
|
-
|
|
558
|
-
- At payload sizes where file serving actually spends its time (>= 64 KB),
|
|
559
|
-
the contenders are at parity: I/O dominates.
|
|
560
|
-
- Small-body transfers and revalidations lead `send` and `sirv` even
|
|
561
|
-
without the cache, while doing strictly more per request: RFC 9530
|
|
562
|
-
digest negotiation, audit hooks, pinned-read plumbing, and storage
|
|
563
|
-
abstraction. Four things make that possible: transfers at or below
|
|
564
|
-
128 KiB take a single positional-read fast path in the fs store, small
|
|
565
|
-
bodies travel as plain bytes (no stream machinery), the node adapter
|
|
566
|
-
consumes `serveObjectRaw` (zero fetch primitives constructed on the Node
|
|
567
|
-
hot path), and validators are derived once at stat time instead of
|
|
568
|
-
re-parsing dates on every request.
|
|
569
|
-
- Correctness note found while building this benchmark: `send` honors a
|
|
570
|
-
request `Cache-Control: no-cache` during conditional evaluation, and
|
|
571
|
-
spec-compliant fetch clients (undici, browsers) auto-append exactly that
|
|
572
|
-
header to manually-conditional requests -- so `send` never answers 304 to
|
|
573
|
-
a fetch()-based revalidation. partial-content deliberately ignores request
|
|
574
|
-
cache directives here (matching Go stdlib and nginx; see DESIGN.md).
|
|
228
|
+
- At payload sizes where file serving actually spends its time (>= 64 KB), all contenders converge on I/O parity.
|
|
229
|
+
- Small bodies and revalidations lead `send` and `sirv` even without the cache, while doing strictly more per request (digest negotiation, audit hooks, pinned-read plumbing, storage abstraction).
|
|
230
|
+
- The `cache` column is the opt-in fs hot-object cache (nginx `open_file_cache` semantics: TTL revalidation, `maxEntries` + `maxBytes` LRU bounds). \* sirv's 304 figure comes from a boot-time directory snapshot that 404s files created after startup; in the mode that can serve runtime uploads it measures 4,957 req/s.
|
|
231
|
+
|
|
232
|
+
Kernel micro-benchmarks, the Bun.serve numbers (38k req/s revalidation with cache), and the full fairness notes are in **[docs/BENCHMARKS.md](docs/BENCHMARKS.md)**.
|
|
575
233
|
|
|
576
234
|
## Security & Compliance
|
|
577
235
|
|
package/docs/API.md
ADDED
|
@@ -0,0 +1,115 @@
|
|
|
1
|
+
# API Reference
|
|
2
|
+
|
|
3
|
+
The complete export surface. Everything is typed; your editor's IntelliSense mirrors this page.
|
|
4
|
+
|
|
5
|
+
## Kernel (`partial-content`)
|
|
6
|
+
|
|
7
|
+
**`evaluateConditionalRequest(reqHeaders, meta)`** - One-call handler for the full HTTP evaluation chain (GET/HEAD). Returns `{ status, headers, range }`.
|
|
8
|
+
|
|
9
|
+
**`evaluateConditionalWrite(reqHeaders, meta)`** - One-call handler for write requests (PUT/PATCH/DELETE). Returns `{ proceed: true }` or `{ proceed: false, status: 412, headers }`. The 412 response includes the current `ETag` when available, so the client can resync without a follow-up GET.
|
|
10
|
+
|
|
11
|
+
**`parseRangeHeader(rangeHeader, totalSize)`** - Returns `{ start, end }`, `"unsatisfiable"`, or `null`.
|
|
12
|
+
|
|
13
|
+
**`parseRanges(rangeHeader, totalSize, maxRanges?)`** - Multi-range parsing for `multipart/byteranges`: coalesces overlapping/adjacent ranges and applies range-amplification defenses. Returns a `RangeSet`, `"unsatisfiable"`, or `null` (serve the full 200).
|
|
14
|
+
|
|
15
|
+
**`buildRangeResponseHeaders(opts)`** - Build 200 or 206 response headers.
|
|
16
|
+
|
|
17
|
+
**`buildMultipartHeaders(opts)` / `buildMultipartPartHeader(...)` / `multipartEpilogue(boundary)` / `generateMultipartBoundary()`** - The `multipart/byteranges` framing primitives with exact precomputed Content-Length (never chunked).
|
|
18
|
+
|
|
19
|
+
**`parseContentRange(header)`** - Parse a `Content-Range` response header (e.g. `bytes 0-499/1000`). Returns `{ start, end, totalSize }` or `null`.
|
|
20
|
+
|
|
21
|
+
**`generateETag(source)`** - Derive an entity-tag from storage metadata. Returns a strong `"<hash>"` when a content digest is available, a weak `W/"<size>-<mtime>"` when only size and modification time are known, or `undefined` when there is insufficient metadata.
|
|
22
|
+
|
|
23
|
+
**`buildContentDisposition(filename, options?)`** - Security-hardened `Content-Disposition` header builder with CRLF injection prevention, path traversal protection, bidi override stripping, and RFC 8187 non-ASCII encoding.
|
|
24
|
+
|
|
25
|
+
**`fromNodeHeaders(headers)`** - Convert Node.js `IncomingHttpHeaders` to the `{ get(name) }` interface.
|
|
26
|
+
|
|
27
|
+
**`isConditionalFresh(reqHeaders, etag, lastModified)`** - `true` if not modified (304).
|
|
28
|
+
|
|
29
|
+
**`isPreconditionFailure(reqHeaders, etag, lastModified, exists?)`** - `true` if precondition failed (412). Pass `exists` when the resource's presence is known independently of its validators (e.g. `If-Match: *` upload guards).
|
|
30
|
+
|
|
31
|
+
**`isRangeFresh(reqHeaders, etag, lastModified)`** - `true` if If-Range passes (honor the range).
|
|
32
|
+
|
|
33
|
+
**`build304Headers(etag, lastModified, cacheControl?)`** - Build 304 headers.
|
|
34
|
+
|
|
35
|
+
**`build412Headers()`** - Build 412 headers.
|
|
36
|
+
|
|
37
|
+
**`build416Headers(totalSize)`** - Build 416 Range Not Satisfiable headers.
|
|
38
|
+
|
|
39
|
+
**`clientWantsDigest(reqHeaders)`** - RFC 9530 Section 4 negotiation: `true` when the client's `Want-Repr-Digest` / `Want-Content-Digest` headers accept `sha-256` (or are absent). The web adapter and orchestrator both honor this, so `Want-Repr-Digest: sha-256=0` suppresses digest emission everywhere.
|
|
40
|
+
|
|
41
|
+
**`sanitizeHeaderValue(s)`** - Strip every byte outside RFC 9110 field-value grammar. The kernel applies it to all metadata-derived headers; exported so adapters can sanitize headers they build themselves.
|
|
42
|
+
|
|
43
|
+
## MIME Lookup (`partial-content/mime`)
|
|
44
|
+
|
|
45
|
+
**`lookupMime(filenameOrExt)`** - Curated, zero-dependency extension -> MIME lookup for documents, media, archives, fonts, and web assets. Case-insensitive, resolves the last dot segment (`archive.tar.gz` -> `application/gzip`), returns `undefined` for unknown types so the caller controls the fallback. `html` is deliberately absent: serving stored uploads as `text/html` is stored XSS, so that decision must be explicit at the call site.
|
|
46
|
+
|
|
47
|
+
```typescript
|
|
48
|
+
import { lookupMime } from "partial-content/mime";
|
|
49
|
+
|
|
50
|
+
app.get("/files/:key", serveObject(store, {
|
|
51
|
+
key: (req) => req.params.key,
|
|
52
|
+
mime: (req) => lookupMime(req.params.key),
|
|
53
|
+
}));
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
## Universal HTTP Store (`partial-content/http`)
|
|
57
|
+
|
|
58
|
+
**`httpStore({ url, headers?, fetch?, redirect? })`** - Serve from ANY range-capable HTTP origin over plain `fetch`: Supabase Storage, presigned S3/GCS/Azure URLs, CDN origins, or another partial-content server. Pinned reads map to `If-Match` (origin 412 -> `ObjectChangedError`), `Repr-Digest` response headers are extracted, and requests are sent `Accept-Encoding: identity` and any response that still carries a non-identity `Content-Encoding` is refused, so transparent compression can never corrupt byte accounting. Redirects error by default (a hostile origin must not 3xx the store toward internal/metadata IPs); set `redirect: "follow"` for origins that legitimately redirect, paired with a validating `fetch` when keys are untrusted (see SECURITY.md).
|
|
59
|
+
|
|
60
|
+
```typescript
|
|
61
|
+
import { httpStore } from "partial-content/http";
|
|
62
|
+
|
|
63
|
+
const store = httpStore({
|
|
64
|
+
url: (key) => `${SUPABASE_URL}/storage/v1/object/documents/${key}`,
|
|
65
|
+
headers: { Authorization: `Bearer ${serviceRoleKey}` },
|
|
66
|
+
});
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
## Memory Store (`partial-content/memory`)
|
|
70
|
+
|
|
71
|
+
**`memoryStore({ objects })`** - A spec-faithful in-memory store for consumer test suites, demos, and small embedded assets. Fabricates correct Content-Range values, honors `ifMatch` pinning (mutate the map to simulate overwrites and exercise retry logic), and streams zero-byte objects correctly.
|
|
72
|
+
|
|
73
|
+
## Web Adapter (`partial-content/web`)
|
|
74
|
+
|
|
75
|
+
**`serveObject(store, options?)`** - Create a Fetch API handler that serves files from an ObjectStore. Returns `(req: Request, ctx: ServeContext) => Promise<Response>`.
|
|
76
|
+
|
|
77
|
+
**`serveObjectRaw(store, options?)`** - The same engine returning `RawResponseParts` (`{ status, statusText, headers, body }`) instead of a `Response`, for server adapters that write to their runtime natively (the bundled node adapter uses it). Skips all fetch-primitive construction on the hot path.
|
|
78
|
+
|
|
79
|
+
Options: `disposition`, `cacheControl`, `immutable`, `securityHeaders`, `crossOriginResourcePolicy`, `timingAllowOrigin`, `timing`, `onTiming`, `onError`, `onServe`, `onTransfer`, `maxRanges`, `enforceCharset`, `fallbackFilename`.
|
|
80
|
+
|
|
81
|
+
`ServeContext`: `key` (required), `mime?`, `filename?`, `cacheControl?` (per-request override of the handler-level value, e.g. `immutable` for content-addressed keys next to `private, no-cache` user uploads from the same handler).
|
|
82
|
+
|
|
83
|
+
`cacheControl` is emitted verbatim on 200/206/304, so any directive vocabulary your CDN or edge understands passes straight through: RFC 9111 `s-maxage` / `must-revalidate` / `proxy-revalidate` and the RFC 5861 resilience directives `stale-while-revalidate` and `stale-if-error`. The library does not synthesize or reorder directives (only appending `immutable` when the `immutable` option is set and it is not already present), so you keep full control of the response caching policy. `Vary` (e.g. `Vary: Accept-Encoding`) rides `securityHeaders` / `extraHeaders`.
|
|
84
|
+
|
|
85
|
+
## Node Adapter (`partial-content/node`)
|
|
86
|
+
|
|
87
|
+
**`serveObject(store, options)`** - Create a Node.js `(req, res) => Promise<void>` handler for Express, Fastify (compat), Koa, and raw `http.createServer`. Extends the web adapter options with `key` (required, extracts the storage key from `IncomingMessage`), `mime?`, and `filename?`.
|
|
88
|
+
|
|
89
|
+
**`writeStallTimeoutMs?`** (default `60000`) - Bounds how long the streaming pump waits for a single backpressure `drain` before treating the client as stalled and tearing the transfer down (cancel the storage read, destroy the response). A client that stops reading but holds its socket open would otherwise pin a backend storage connection indefinitely (a slow-read attack). Set to `0` to disable and rely on an upstream proxy / socket timeout instead. Only the raw-Node pump needs this; Fetch-runtime backpressure is the platform's own concern.
|
|
90
|
+
|
|
91
|
+
## Storage Contract
|
|
92
|
+
|
|
93
|
+
**`ObjectStore`** (interface) - Read-only storage backend abstraction. Implementations provide `headObject(key, opts?)` for metadata and `getObject(key, opts?)` for streaming, where `opts` carries `range`, `signal`, `ifMatch` (pinned reads), and `pin` (an opaque token issued by `headObject` for stores whose version identifier is not the ETag; GCS uses it to stream a pinned generation without re-fetching metadata). Optional `createSignedUrl(key, opts)` for backends that cannot stream ranges through the origin. Optional `authoritativeRange: true` declares that ranged responses report the backend's ACTUAL served bounds (parsed Content-Range) -- the web adapter then serves plain range requests in a single round-trip with no validating HEAD (S3, Azure, R2, and http set it; video seeking and PDF.js chunking hit this path constantly).
|
|
94
|
+
|
|
95
|
+
**`ObjectMetadata`** (type) - HEAD response shape: `contentLength`, `etag?`, `lastModified?`, `digest?`.
|
|
96
|
+
|
|
97
|
+
**`ObjectStream`** (type) - GET response shape: `body` (a `ReadableStream`, or a plain `Uint8Array` when the adapter already holds the exact bytes -- consumers then skip stream machinery entirely), `contentLength`, `totalSize`, `range?` (the `{ start, end }` the backend ACTUALLY served; absent = full content), `etag?`, `lastModified?`, `digest?`.
|
|
98
|
+
|
|
99
|
+
**`classifyStoreRead(key, op, classifiers)`** - The ordered error-classification pipeline the built-in SDK adapters share, exported for custom adapter authors. Runs `op()` and maps its failure to the contract's error types in a fixed precedence: `notFound` -> `ObjectNotFoundError` (404), `changed` -> `ObjectChangedError` (412), `throttled` -> `StoreUnavailableError` (503), otherwise rethrow untouched. Supply one `StoreErrorClassifiers` set and reuse it for both `headObject` and `getObject` so the two paths cannot drift; predicates must be mutually exclusive on a given backend.
|
|
100
|
+
|
|
101
|
+
```typescript
|
|
102
|
+
import { classifyStoreRead, type StoreErrorClassifiers } from "partial-content";
|
|
103
|
+
|
|
104
|
+
const classifiers: StoreErrorClassifiers = {
|
|
105
|
+
notFound: (e) => (e as { statusCode?: number }).statusCode === 404,
|
|
106
|
+
changed: (e) => (e as { statusCode?: number }).statusCode === 412, // omit if the pin is an etag compare
|
|
107
|
+
throttled: (e) => (e as { statusCode?: number }).statusCode === 503,
|
|
108
|
+
};
|
|
109
|
+
|
|
110
|
+
const meta = await classifyStoreRead(key, () => backend.head(key), classifiers);
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
**`StoreUnavailableError`** (class) - Throw from an adapter when the backend is transiently unavailable (throttled/overloaded after the adapter's own retries). Carries an optional `retryAfterSeconds` echoed as `Retry-After`. Distinct from a malformed-response `502`: this is the retryable `503` case.
|
|
114
|
+
|
|
115
|
+
**`nodeStreamToWeb(iterable, opts?)` / `guardStreamLength(stream, expectedBytes)` / `resolveServedRange(contentRange)` / `parseRetryAfterSeconds(raw, opts?)`** - The stream/accounting primitives the built-in adapters are made of, exported for custom adapter authors: Node-to-web stream conversion with backpressure, abort propagation, and short-read detection; a committed-length guard for web streams; backend `Content-Range` resolution with the unknown-total (`bytes a-b/*`) sentinel; and the shared `Retry-After` parser.
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Benchmarks
|
|
2
|
+
|
|
3
|
+
Two suites: kernel micro-benchmarks (library overhead only) and end-to-end HTTP serving against `send` and `sirv`. Reproduce with `bun bench.ts` and `npm run bench`.
|
|
4
|
+
|
|
5
|
+
## Kernel micro-benchmarks
|
|
6
|
+
|
|
7
|
+
Representative throughput measured on Bun 1.3 (single core, 2M iterations). These measure library overhead only -- they do not include network or storage latency.
|
|
8
|
+
|
|
9
|
+
| Function | Throughput |
|
|
10
|
+
|----------|-----------|
|
|
11
|
+
| `parseRangeHeader` | 10.1M ops/sec |
|
|
12
|
+
| `isPreconditionFailure` | 10.2M ops/sec |
|
|
13
|
+
| `isRangeFresh` | 13.9M ops/sec |
|
|
14
|
+
| `isConditionalFresh` | 4.7M ops/sec |
|
|
15
|
+
| `buildRangeResponseHeaders` | 4.2M ops/sec |
|
|
16
|
+
| `evaluateConditionalRequest` | 1.7M ops/sec |
|
|
17
|
+
| `buildContentDisposition` | 1.7M ops/sec |
|
|
18
|
+
|
|
19
|
+
## End-to-end vs `send` and `sirv`
|
|
20
|
+
|
|
21
|
+
Full HTTP serving (Node 24, `http.createServer`, loopback, 40 connections,
|
|
22
|
+
autocannon, identical fixtures; every cell correctness-verified before
|
|
23
|
+
timing; the load generator runs in a separate process so its parse cost
|
|
24
|
+
never throttles the server column). partial-content runs its shipped dist
|
|
25
|
+
through the node adapter + fsStore. Reproduce with `npm run bench`.
|
|
26
|
+
|
|
27
|
+
| Scenario | partial-content | + `cache` | send | sirv |
|
|
28
|
+
|---|---|---|---|---|
|
|
29
|
+
| GET 4 KB (200) | 9,210 req/s | **15,772 req/s** | 6,557 req/s | 7,162 req/s |
|
|
30
|
+
| GET 1 MB (200) | 118 req/s | 116 req/s | 117 req/s | 119 req/s |
|
|
31
|
+
| Range 64 KB of 1 MB (206) | 1,441 req/s | 1,413 req/s | 1,403 req/s | 1,443 req/s |
|
|
32
|
+
| Revalidation (304) | 11,716 req/s | **19,542 req/s** | 10,081 req/s | 23,774 req/s* |
|
|
33
|
+
|
|
34
|
+
The `cache` column is `fsStore({ root, cache: { ttlMs: 1000 } })`: an opt-in
|
|
35
|
+
hot-object cache with nginx `open_file_cache` semantics (TTL revalidation,
|
|
36
|
+
metadata + bytes captured atomically, bodies only at or below 128 KiB,
|
|
37
|
+
LRU-evicted under both an entry cap (`maxEntries`, default 1024) and a body
|
|
38
|
+
byte budget (`maxBytes`, default 64 MiB; `0` = metadata-only)). Hot small
|
|
39
|
+
files and their ranges serve from memory with zero syscalls. The trade is
|
|
40
|
+
explicit: after an overwrite, responses (including 304 revalidations) can
|
|
41
|
+
affirm the previous representation for up to `ttlMs`; size it against
|
|
42
|
+
acceptable staleness (see DESIGN.md).
|
|
43
|
+
|
|
44
|
+
\* sirv's 304 figure buys throughput with a different contract: at
|
|
45
|
+
`dev: false` it pre-renders complete header sets for a directory snapshot
|
|
46
|
+
taken at boot, so a file created after startup is a 404. In the
|
|
47
|
+
configuration that CAN see runtime-created files (`dev: true`), the same
|
|
48
|
+
run measures sirv at 4,957 req/s -- a quarter of the `cache` column. For a
|
|
49
|
+
fixed directory of immutable assets sirv's trade is exactly right; for
|
|
50
|
+
object storage (where uploads happen while the server runs) it is
|
|
51
|
+
disqualifying, which is why the boot-snapshot design stays a non-goal here.
|
|
52
|
+
On the rows where server code matters (4 KB bodies, revalidations), the
|
|
53
|
+
`cache` column is the fastest contender that can serve a file uploaded a
|
|
54
|
+
second ago; at 64 KB and above, all four columns converge on I/O parity.
|
|
55
|
+
|
|
56
|
+
## Bun runtime
|
|
57
|
+
|
|
58
|
+
Same code on Bun 1.3 (`Bun.serve`, same machine, same out-of-process
|
|
59
|
+
client -- the web adapter's Request/Response ARE the runtime's native
|
|
60
|
+
primitives there, no node bridge): GET 4 KB 9,418 req/s plain and
|
|
61
|
+
22,898 req/s with the cache; revalidation 12,538 plain and **38,090 req/s**
|
|
62
|
+
with the cache -- a TTL-revalidating cache serving runtime uploads,
|
|
63
|
+
outrunning even sirv's boot-frozen Node figure by 60%. `send` and `sirv`
|
|
64
|
+
are node-stream libraries and cannot ride `Bun.serve` natively. Reproduce:
|
|
65
|
+
`bun bench/bun-server.ts` (plain :18778, cache :18779) + autocannon.
|
|
66
|
+
|
|
67
|
+
## Read this fairly in both directions
|
|
68
|
+
|
|
69
|
+
- At payload sizes where file serving actually spends its time (>= 64 KB),
|
|
70
|
+
the contenders are at parity: I/O dominates.
|
|
71
|
+
- Small-body transfers and revalidations lead `send` and `sirv` even
|
|
72
|
+
without the cache, while doing strictly more per request: RFC 9530
|
|
73
|
+
digest negotiation, audit hooks, pinned-read plumbing, and storage
|
|
74
|
+
abstraction. Four things make that possible: transfers at or below
|
|
75
|
+
128 KiB take a single positional-read fast path in the fs store, small
|
|
76
|
+
bodies travel as plain bytes (no stream machinery), the node adapter
|
|
77
|
+
consumes `serveObjectRaw` (zero fetch primitives constructed on the Node
|
|
78
|
+
hot path), and validators are derived once at stat time instead of
|
|
79
|
+
re-parsing dates on every request.
|
|
80
|
+
- Correctness note found while building this benchmark: `send` honors a
|
|
81
|
+
request `Cache-Control: no-cache` during conditional evaluation, and
|
|
82
|
+
spec-compliant fetch clients (undici, browsers) auto-append exactly that
|
|
83
|
+
header to manually-conditional requests -- so `send` never answers 304 to
|
|
84
|
+
a fetch()-based revalidation. partial-content deliberately ignores request
|
|
85
|
+
cache directives here (matching Go stdlib and nginx; see DESIGN.md).
|
package/docs/EXAMPLES.md
ADDED
|
@@ -0,0 +1,148 @@
|
|
|
1
|
+
# Examples
|
|
2
|
+
|
|
3
|
+
Recipes beyond the README's Quick Start. Every handler manages the full HTTP protocol (200, 206, 304, 412, 416, HEAD) automatically.
|
|
4
|
+
|
|
5
|
+
## Hono
|
|
6
|
+
|
|
7
|
+
```typescript
|
|
8
|
+
import { Hono } from "hono";
|
|
9
|
+
import { serveObject } from "partial-content/hono";
|
|
10
|
+
import { s3Store } from "partial-content/s3";
|
|
11
|
+
|
|
12
|
+
const store = s3Store({ client, bucket: "media" });
|
|
13
|
+
const app = new Hono();
|
|
14
|
+
|
|
15
|
+
app.get("/media/:key", serveObject(store, {
|
|
16
|
+
key: (c) => c.req.param("key"),
|
|
17
|
+
cacheControl: "public, max-age=31536000, immutable",
|
|
18
|
+
}));
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
## Cloudflare Workers (R2 native, no AWS SDK)
|
|
22
|
+
|
|
23
|
+
```typescript
|
|
24
|
+
import { Hono } from "hono";
|
|
25
|
+
import { serveObject } from "partial-content/hono";
|
|
26
|
+
import { r2Store } from "partial-content/r2";
|
|
27
|
+
|
|
28
|
+
const app = new Hono<{ Bindings: { MY_BUCKET: R2Bucket } }>();
|
|
29
|
+
|
|
30
|
+
app.get("/files/:key", (c) => {
|
|
31
|
+
const store = r2Store({ bucket: c.env.MY_BUCKET });
|
|
32
|
+
return serveObject(store, { key: (c) => c.req.param("key") })(c);
|
|
33
|
+
});
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
## Kernel only: the evaluation flow
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
Request evaluateConditionalRequest()
|
|
40
|
+
│ │
|
|
41
|
+
│ If-Match / If-Unmodified-Since ├──► 412 Precondition Failed
|
|
42
|
+
│ If-None-Match / If-Modified-Since ├──► 304 Not Modified
|
|
43
|
+
│ If-Range + Range ├──► 416 Range Not Satisfiable
|
|
44
|
+
│ ├──► 206 Partial Content
|
|
45
|
+
│ └──► 200 OK
|
|
46
|
+
│
|
|
47
|
+
▼
|
|
48
|
+
Fetch bytes from storage (you control this)
|
|
49
|
+
│
|
|
50
|
+
▼
|
|
51
|
+
new Response(body, { status, headers })
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
## Node.js / Express (kernel only)
|
|
55
|
+
|
|
56
|
+
```typescript
|
|
57
|
+
import { fromNodeHeaders, evaluateConditionalRequest } from "partial-content";
|
|
58
|
+
|
|
59
|
+
app.get("/files/:key", (req, res) => {
|
|
60
|
+
const headers = fromNodeHeaders(req.headers);
|
|
61
|
+
const { status, headers: resHeaders, range } = evaluateConditionalRequest(
|
|
62
|
+
headers,
|
|
63
|
+
{ totalSize: fileSize, etag, lastModified, contentType },
|
|
64
|
+
);
|
|
65
|
+
res.writeHead(status, resHeaders);
|
|
66
|
+
// ...
|
|
67
|
+
});
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
## Content-Disposition
|
|
71
|
+
|
|
72
|
+
```typescript
|
|
73
|
+
import { buildContentDisposition } from "partial-content";
|
|
74
|
+
|
|
75
|
+
buildContentDisposition("report.pdf");
|
|
76
|
+
// => 'attachment; filename=report.pdf'
|
|
77
|
+
|
|
78
|
+
buildContentDisposition("Årlig_Rapport.pdf");
|
|
79
|
+
// => 'attachment; filename="?rlig_Rapport.pdf"; filename*=UTF-8''%C3%85rlig_Rapport.pdf'
|
|
80
|
+
|
|
81
|
+
buildContentDisposition("slides.pdf", { type: "inline" });
|
|
82
|
+
// => 'inline; filename=slides.pdf'
|
|
83
|
+
|
|
84
|
+
// Handles untrusted input safely
|
|
85
|
+
buildContentDisposition("../../etc/passwd"); // Path traversal stripped
|
|
86
|
+
buildContentDisposition("evil\r\nX-Injected: yes"); // CRLF injection stripped
|
|
87
|
+
buildContentDisposition(null, { fallback: "export.csv" }); // Graceful fallback
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
## RFC 9530 Repr-Digest (end-to-end integrity)
|
|
91
|
+
|
|
92
|
+
Pass a SHA-256 digest from your storage backend for automatic `Repr-Digest` headers:
|
|
93
|
+
|
|
94
|
+
```typescript
|
|
95
|
+
const { status, headers, range } = evaluateConditionalRequest(
|
|
96
|
+
request.headers,
|
|
97
|
+
{
|
|
98
|
+
totalSize: fileSize,
|
|
99
|
+
etag: '"abc123"',
|
|
100
|
+
// S3: x-amz-checksum-sha256, GCS: x-goog-hash (sha256 component)
|
|
101
|
+
digest: "d2VsY29tZQ==", // raw base64 SHA-256
|
|
102
|
+
},
|
|
103
|
+
);
|
|
104
|
+
// Response headers include: Repr-Digest: sha-256=:d2VsY29tZQ==:
|
|
105
|
+
// Same digest on both 200 (full) and 206 (partial) -- covers the full representation
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
## Advanced: manual primitives
|
|
109
|
+
|
|
110
|
+
For full control over the evaluation chain:
|
|
111
|
+
|
|
112
|
+
```typescript
|
|
113
|
+
import {
|
|
114
|
+
parseRangeHeader,
|
|
115
|
+
buildRangeResponseHeaders,
|
|
116
|
+
isConditionalFresh,
|
|
117
|
+
isPreconditionFailure,
|
|
118
|
+
isRangeFresh,
|
|
119
|
+
build304Headers,
|
|
120
|
+
build412Headers,
|
|
121
|
+
build416Headers,
|
|
122
|
+
} from "partial-content";
|
|
123
|
+
|
|
124
|
+
// Step 1: Preconditions (If-Match / If-Unmodified-Since)
|
|
125
|
+
if (isPreconditionFailure(reqHeaders, etag, lastModified)) {
|
|
126
|
+
return new Response(null, build412Headers());
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
// Step 2: Freshness (If-None-Match / If-Modified-Since)
|
|
130
|
+
if (isConditionalFresh(reqHeaders, etag, lastModified)) {
|
|
131
|
+
return new Response(null, build304Headers(etag, lastModified));
|
|
132
|
+
}
|
|
133
|
+
|
|
134
|
+
// Step 3: Range (If-Range + Range header)
|
|
135
|
+
const range = isRangeFresh(reqHeaders, etag, lastModified)
|
|
136
|
+
? parseRangeHeader(reqHeaders.get("range"), fileSize)
|
|
137
|
+
: null;
|
|
138
|
+
|
|
139
|
+
if (range === "unsatisfiable") {
|
|
140
|
+
return new Response(null, build416Headers(fileSize));
|
|
141
|
+
}
|
|
142
|
+
|
|
143
|
+
const { status, headers } = buildRangeResponseHeaders({
|
|
144
|
+
totalSize: fileSize, range, contentType, etag, lastModified,
|
|
145
|
+
digest: checksum, // RFC 9530 Repr-Digest
|
|
146
|
+
cacheControl: "private, no-cache",
|
|
147
|
+
});
|
|
148
|
+
```
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "partial-content",
|
|
3
|
-
"version": "1.0.
|
|
3
|
+
"version": "1.0.1",
|
|
4
4
|
"description": "RFC-compliant HTTP file serving for any storage backend. Range requests (206), conditional caching (304/412), Content-Disposition, and ETag generation. Zero dependencies.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./dist/index.js",
|
|
@@ -73,7 +73,7 @@
|
|
|
73
73
|
"src",
|
|
74
74
|
"!src/__tests__",
|
|
75
75
|
"README.md",
|
|
76
|
-
"docs
|
|
76
|
+
"docs",
|
|
77
77
|
"CHANGELOG.md",
|
|
78
78
|
"SECURITY.md",
|
|
79
79
|
"LICENSE"
|