@aws-blocks/bb-file-bucket 0.1.1 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/DESIGN.md +95 -0
- package/README.md +35 -0
- package/dist/file-server.d.ts.map +1 -1
- package/dist/file-server.js +16 -1
- package/dist/file-server.test.js +37 -0
- package/dist/tokens.d.ts +17 -1
- package/dist/tokens.d.ts.map +1 -1
- package/dist/tokens.js +18 -2
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/package.json +12 -3
- package/src/file-server.test.ts +49 -0
- package/src/file-server.ts +16 -1
- package/src/tokens.ts +18 -2
- package/src/version.ts +1 -1
package/DESIGN.md
ADDED
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
# FileBucket — Design
|
|
2
|
+
|
|
3
|
+
Design document for FileBucket. For usage, see [README.md](./README.md).
|
|
4
|
+
|
|
5
|
+
**Package:** `@aws-blocks/bb-file-bucket`
|
|
6
|
+
**Type:** Primitive (new infrastructure)
|
|
7
|
+
**AWS Service:** Amazon S3
|
|
8
|
+
|
|
9
|
+
## Design Decisions
|
|
10
|
+
|
|
11
|
+
**D-FB-1: Buffer body type instead of ReadableStream**
|
|
12
|
+
**Decision:** `put()` accepts `Buffer | string`, `get()` returns `Buffer` in `FileContent.body`.
|
|
13
|
+
**Rationale:** ReadableStream adds complexity for the common case (small-to-medium files). Buffer is simpler to work with in Lambda (where the entire response must be buffered anyway). For large files, presigned URLs (`getUrl`/`putUrl`) are the recommended pattern. This favors client-safe return types.
|
|
14
|
+
|
|
15
|
+
**D-FB-2: Segregated internal storage in mock**
|
|
16
|
+
**Decision:** The mock stores user content, sidecar metadata, and version history under separate sibling roots inside `.bb-data/{fullId}/`:
|
|
17
|
+
```
|
|
18
|
+
content/{key} file body (byte-identical to what was written)
|
|
19
|
+
meta/{key}.json sidecar metadata
|
|
20
|
+
versions/{key}/{versionId} version body
|
|
21
|
+
versions/{key}/{versionId}.json version metadata
|
|
22
|
+
versions/{key}/__deleted__ delete marker (sentinel)
|
|
23
|
+
```
|
|
24
|
+
**Rationale:** Keeps file content byte-identical (no wrapping format) while guaranteeing internal bookkeeping can never collide with user keys. An earlier design co-located metadata as `{path}.__meta__.json` next to each file and relied on marker substrings, which meant a user key like `data.__meta__.json` or a directory named `x.__versions__/` could be silently hidden or shadowed by `scan()`. Because user content now lives only under `content/`, `scan()` walks that one root and yields everything with no marker-based filtering — arbitrary keys are supported, matching S3 semantics.
|
|
25
|
+
|
|
26
|
+
**D-FB-3: deleteBatch chunks at 1,000**
|
|
27
|
+
**Decision:** `deleteBatch()` internally chunks into groups of 1,000 and issues separate `DeleteObjects` calls.
|
|
28
|
+
**Rationale:** S3 `DeleteObjects` API supports max 1,000 keys per request. Chunking is transparent to the caller. Batch methods handle pagination internally.
|
|
29
|
+
|
|
30
|
+
**D-FB-4: Presigned URL default expiry of 3600 seconds**
|
|
31
|
+
**Decision:** Both `getUrl` and `putUrl` default to 1 hour expiry.
|
|
32
|
+
**Rationale:** Matches the S3 SDK default. Long enough for typical browser upload/download flows, short enough to limit exposure.
|
|
33
|
+
|
|
34
|
+
**D-FB-5: scan returns AsyncIterable**
|
|
35
|
+
**Decision:** `scan()` returns `AsyncIterable<FileInfo>` with internal pagination.
|
|
36
|
+
**Rationale:** AsyncIterable is used for unbounded result sets. S3 `ListObjectsV2` paginates at 1,000 keys; the iterable handles continuation tokens transparently.
|
|
37
|
+
|
|
38
|
+
**D-FB-6: Versioning is opt-in with runtime API support**
|
|
39
|
+
**Decision:** `versioned: true` enables S3 object versioning and unlocks version-aware methods (`listVersions`, `restoreVersion`, optional `versionId` on `get`/`delete`/`getUrl`/`getFileHandle`). Without the flag, the API surface is unchanged.
|
|
40
|
+
**Rationale:** Versioning adds storage cost and complexity. Making it opt-in keeps the default simple. When enabled, the runtime API exposes the full version lifecycle — listing, retrieving specific versions, permanent deletion of individual versions, and restoring old versions. `restoreVersion` is implemented as a CopyObject from the old version (S3 has no native restore), which creates a new version that becomes current.
|
|
41
|
+
|
|
42
|
+
**D-FB-7: Mock versioning uses filesystem directories**
|
|
43
|
+
**Decision:** Versioned mock stores each version in `versions/{key}/v{n}` with monotonic IDs. Delete markers are `versions/{key}/__deleted__` sentinel files.
|
|
44
|
+
**Rationale:** Simple, inspectable, and matches the S3 semantics closely enough for local development. Monotonic IDs (`v1`, `v2`, ...) are deterministic and easy to reason about in tests, unlike S3's opaque version IDs. Version history lives under the segregated `versions/` root (see D-FB-2), so it never appears in `scan()` or collides with a user key.
|
|
45
|
+
|
|
46
|
+
**D-FB-8: Bucket name validated at synth — error, never truncate/hash**
|
|
47
|
+
**Decision:** The derived bucket name (`scope.fullId`) is validated against S3's naming rules (`bucket-name.ts`) before the bucket is constructed. An invalid name throws a `ValidationFailed` error with an actionable message. The same validator runs in the mock constructor so local dev (`bb dev`) fails identically — parity. `FileBucket.fromExisting(...)` skips validation since the name is externally owned.
|
|
48
|
+
**Rationale:** S3 bucket names are globally unique and immutable. Truncating to fit 63 chars risks collisions, and a name that shifts between deploys (e.g. after a hash input changes) would orphan or replace the customer's data — a far worse outcome than a fast, fixable synth error. Erroring puts the fix in the developer's hands (shorten a scope id once; the name is then stable forever) and matches the manual-shortening pattern already used in `bb-agent`. This deliberately differs from DynamoDB-backed BBs (KVStore/DistributedTable) which `substring(0, 255)` — DynamoDB's 255 limit is generous and table names are internal, disposable, and not globally unique, so silent truncation is acceptable there.
|
|
49
|
+
|
|
50
|
+
## Infrastructure (CDK)
|
|
51
|
+
|
|
52
|
+
Creates a single S3 bucket:
|
|
53
|
+
|
|
54
|
+
- **Bucket name:** Derived from `scope.fullId` (the bucket id joined to its parent scope ids with `-`). Validated at synth against S3's naming rules — see D-FB-6.
|
|
55
|
+
- **Block public access:** All four settings enabled (BLOCK_ALL)
|
|
56
|
+
- **Encryption:** S3-managed keys (SSE-S3)
|
|
57
|
+
- **Versioning:** Disabled by default, enabled via `options.versioned`
|
|
58
|
+
- **CORS:** Configured from `options.corsRules` if provided
|
|
59
|
+
- **Lifecycle rules:** Configured from `options.lifecycleRules` if provided
|
|
60
|
+
- **Removal policy:** DESTROY (sandbox), configurable for production
|
|
61
|
+
- **Auto-delete objects:** Enabled when removal policy is DESTROY
|
|
62
|
+
- **Permissions:** `grantReadWrite` to the parent scope's handler automatically
|
|
63
|
+
|
|
64
|
+
## Mock Implementation
|
|
65
|
+
|
|
66
|
+
- Files stored on the local filesystem at `.bb-data/{scope.fullId}/` via `getMockDataDir()` from core.
|
|
67
|
+
- Internal data is segregated into sibling roots so it can never collide with user keys (see D-FB-2):
|
|
68
|
+
- `content/{key}` — file body, byte-identical to what was written.
|
|
69
|
+
- `meta/{key}.json` — sidecar metadata.
|
|
70
|
+
- `versions/{key}/{versionId}` (+ `.json` sidecars, `__deleted__` marker) — version history.
|
|
71
|
+
- Path mapping for both the mock and the dev file-server is centralized in `paths.ts` so they stay in lockstep.
|
|
72
|
+
- Data persists across dev server restarts. Customers can wipe with `rm -rf .bb-data`.
|
|
73
|
+
- Presigned URLs are served by the dev file-server at `/.bb-file-bucket/{scope.fullId}/{path}?token=...`. The path segments are URL-encoded; the server decodes them and validates an HMAC token scoped to method, path, and expiry. The HMAC secret (`LOCAL_FILE_SECRET` in `tokens.ts`) is a **per-process random value** — the token-minting mock and the validating dev file-server share the same in-process module instance, so tokens are unforgeable without being a hardcoded, source-visible literal.
|
|
74
|
+
- Downloads are served with `X-Content-Type-Options: nosniff` and `Content-Disposition: attachment`. The stored body and its `Content-Type` are caller-controlled, so serving them inline would make the dev file-server a stored-XSS vector (an uploaded `text/html`/SVG payload executing in the app origin). Forcing a download + disabling MIME sniffing keeps local dev no weaker than S3-behind-CloudFront.
|
|
75
|
+
- `scan()` recursively walks only the `content/` root and yields every file it finds — no marker-based filtering — so user keys are unrestricted.
|
|
76
|
+
- The dev file-server's PUT handler delegates to the registered `FileBucket` instance (via a process-global registry) so uploads get versioning, key validation, and metadata. There is no direct-write fallback; an unregistered bucket fails loud with a 500.
|
|
77
|
+
- Key length validated against S3's 1,024-byte limit (warns, does not reject).
|
|
78
|
+
- Versioning fully supported: each `put` writes to `versions/{key}/v{n}`, delete without `versionId` places a `__deleted__` sentinel, `listVersions` reads the versions directory, `restoreVersion` copies an old version via `put`.
|
|
79
|
+
|
|
80
|
+
### Mock vs AWS Behavior Differences
|
|
81
|
+
|
|
82
|
+
| Behavior Difference | Impact | Mitigation |
|
|
83
|
+
|------------|--------|------------|
|
|
84
|
+
| No lifecycle rules | Objects never expire or transition locally | No mitigation — lifecycle rules are a background S3 process |
|
|
85
|
+
| No CORS enforcement | Browser requests succeed regardless of origin locally | No mitigation — CORS is enforced by the browser + S3, not the mock |
|
|
86
|
+
| No storage classes | Transition rules have no effect locally | No mitigation — storage classes are a cost optimization |
|
|
87
|
+
| No multipart upload | Large files use simple write locally | No mitigation — mock uses `fs.writeFile` regardless of size |
|
|
88
|
+
| Presigned URLs are localhost-only | URLs only work against the local dev server | No mitigation — expected behavior for local development |
|
|
89
|
+
| No IAM enforcement | Permission errors only surface in AWS | No mitigation — IAM is handled by CDK grants automatically |
|
|
90
|
+
| Filesystem path limits | Some OS path length limits differ from S3 key limits (1,024 bytes) | Mock validates key length and warns when it exceeds 1,024 bytes |
|
|
91
|
+
| Path-traversal keys rejected locally | The mock maps keys onto the real filesystem, so it rejects keys that escape the bucket's content root (e.g. `../escape.txt`). S3 has no filesystem and treats `..` as a literal key segment, so it accepts such keys. A pathological key containing `..` that "works" on S3 will throw `ValidationFailed` locally. | Intentional — the guard prevents a local key from clobbering files outside `.bb-data`. Avoid `..` segments in keys (also S3 best practice). Covered by `src/path-containment.test.ts`. |
|
|
92
|
+
| Non-atomic `put()` | On a versioned bucket, `put()` performs several separate `writeFileSync` calls (version body, version metadata, current body, current metadata, delete-marker cleanup). A crash or process kill mid-`put()` can leave torn state — a body with no metadata sidecar, or a version body with no `.json`. Real S3 `PutObject` is atomic per object. | No mitigation today — the filesystem layout has no transaction boundary. Acceptable for a dev mock (re-running `put()` heals it). See Open Question 4 (storage engine). |
|
|
93
|
+
| Monotonic version IDs | Mock uses `v1`, `v2`, ... vs S3's opaque IDs | No impact — customer code should treat version IDs as opaque strings |
|
|
94
|
+
| Content-Type signed into presigned PUT URLs | When `putUrl`/`createUploadHandle` are given a `contentType`, the AWS SDK signs `content-type` as a required header, so real S3 returns `403 SignatureDoesNotMatch` if the uploaded request's `Content-Type` differs from (or omits) the signed value. The dev file-server enforces the same check (`src/file-server.ts`) so an upload that would fail in prod also fails locally with 403, rather than silently succeeding. Uploads via `createUploadHandle().upload()` always send the signed header, so the typed-handle path round-trips in both environments. Covered by `src/file-server.test.ts`. |
|
|
95
|
+
| Adjacent slashes in keys collapsed | The mock maps keys onto the filesystem via `path.join`, which collapses `//` to `/` (e.g. a key built from a URL-shaped value like `uploads/https://issuer:sub/f.txt`). A later `scan({ prefix })` whose prefix still contains `//` won't match the stored single-slash path, so the file appears "missing" locally. S3 treats keys as opaque byte strings and preserves `//`, so the same prefix matches in production. | Avoid embedding raw URL-shaped values (e.g. an OIDC `userId` of `${iss}:${sub}`) directly in keys — `encodeURIComponent()` the segment first. See the FileBucket README best-practices note. |
|
package/README.md
CHANGED
|
@@ -6,6 +6,8 @@ File storage backed by Amazon S3.
|
|
|
6
6
|
|
|
7
7
|
**When NOT to use:** If you need structured key-value data with conditional writes, use `KVStore`. If you need queryable records with indexes, use `DistributedTable`.
|
|
8
8
|
|
|
9
|
+
> Design & mock parity details: [DESIGN.md](./DESIGN.md)
|
|
10
|
+
|
|
9
11
|
## API
|
|
10
12
|
|
|
11
13
|
```typescript
|
|
@@ -46,6 +48,39 @@ const bucket = new FileBucket(scope, id, options?)
|
|
|
46
48
|
| `metadata` | `Record<string, string>` | Custom metadata key-value pairs. |
|
|
47
49
|
| `cacheControl` | `string` | Cache-Control header value. |
|
|
48
50
|
|
|
51
|
+
### CorsRule
|
|
52
|
+
|
|
53
|
+
CORS configuration for browser-based access. Supplied via the `corsRules` option.
|
|
54
|
+
|
|
55
|
+
| Field | Type | Description |
|
|
56
|
+
|-------|------|-------------|
|
|
57
|
+
| `allowedOrigins` | `string[]` | Origins permitted to make cross-origin requests (e.g., `['https://app.example.com']`). |
|
|
58
|
+
| `allowedMethods` | `('GET' \| 'PUT' \| 'POST' \| 'DELETE' \| 'HEAD')[]` | HTTP methods permitted for cross-origin requests. |
|
|
59
|
+
| `allowedHeaders` | `string[]` | Optional. Request headers permitted in the actual request. |
|
|
60
|
+
| `exposedHeaders` | `string[]` | Optional. Response headers exposed to the browser. |
|
|
61
|
+
| `maxAge` | `number` | Optional. Seconds the browser may cache the preflight response. |
|
|
62
|
+
|
|
63
|
+
### LifecycleRule
|
|
64
|
+
|
|
65
|
+
Lifecycle configuration for automatic expiration or storage-class transitions. Supplied via the `lifecycleRules` option.
|
|
66
|
+
|
|
67
|
+
| Field | Type | Description |
|
|
68
|
+
|-------|------|-------------|
|
|
69
|
+
| `prefix` | `string` | Optional. Prefix filter — the rule applies only to keys starting with this prefix. Omit to apply to all objects. |
|
|
70
|
+
| `expirationDays` | `number` | Optional. Days after creation to expire (delete) objects. |
|
|
71
|
+
| `transitionToIaDays` | `number` | Optional. Days after creation to transition objects to Infrequent Access storage. |
|
|
72
|
+
|
|
73
|
+
### FileVersionInfo
|
|
74
|
+
|
|
75
|
+
Returned by `listVersions()` — one entry per object version, newest first.
|
|
76
|
+
|
|
77
|
+
| Field | Type | Description |
|
|
78
|
+
|-------|------|-------------|
|
|
79
|
+
| `versionId` | `string` | The version identifier. |
|
|
80
|
+
| `lastModified` | `Date` | When this version was created. |
|
|
81
|
+
| `size` | `number` | Size in bytes. |
|
|
82
|
+
| `isCurrent` | `boolean` | Whether this is the current (latest) version. |
|
|
83
|
+
|
|
49
84
|
### Error Handling
|
|
50
85
|
|
|
51
86
|
`get()` returns `null` for a missing file — it does **not** throw `FileNotFound`. Check for null:
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"file-server.d.ts","sourceRoot":"","sources":["../src/file-server.ts"],"names":[],"mappings":"AAGA;;;;;;;;;;GAUG;AAEH,OAAO,KAAK,EAAE,MAAM,EAAmC,MAAM,WAAW,CAAC;AAyCzE,wBAAgB,MAAM,CAAC,UAAU,EAAE,MAAM,
|
|
1
|
+
{"version":3,"file":"file-server.d.ts","sourceRoot":"","sources":["../src/file-server.ts"],"names":[],"mappings":"AAGA;;;;;;;;;;GAUG;AAEH,OAAO,KAAK,EAAE,MAAM,EAAmC,MAAM,WAAW,CAAC;AAyCzE,wBAAgB,MAAM,CAAC,UAAU,EAAE,MAAM,QAiKxC"}
|
package/dist/file-server.js
CHANGED
|
@@ -116,7 +116,22 @@ export function attach(httpServer) {
|
|
|
116
116
|
catch { }
|
|
117
117
|
}
|
|
118
118
|
const body = readFileSync(readPath);
|
|
119
|
-
|
|
119
|
+
// The stored object body and its Content-Type are attacker-controlled
|
|
120
|
+
// (any client with a presigned PUT can upload arbitrary bytes under an
|
|
121
|
+
// arbitrary content type). Serving that back inline turns the dev file
|
|
122
|
+
// server into a stored-XSS vector: an uploaded `text/html` (or sniffed
|
|
123
|
+
// HTML/SVG) payload would execute in the origin of the local app.
|
|
124
|
+
// `nosniff` stops the browser from MIME-sniffing octet-streams into
|
|
125
|
+
// HTML, and `Content-Disposition: attachment` forces a download rather
|
|
126
|
+
// than inline rendering — so an uploaded document can never run as a
|
|
127
|
+
// page. Real S3 objects served through CloudFront are hardened the same
|
|
128
|
+
// way; this keeps local dev from being weaker than production.
|
|
129
|
+
res.writeHead(200, {
|
|
130
|
+
'Content-Type': contentType,
|
|
131
|
+
'Content-Length': body.length.toString(),
|
|
132
|
+
'X-Content-Type-Options': 'nosniff',
|
|
133
|
+
'Content-Disposition': 'attachment',
|
|
134
|
+
});
|
|
120
135
|
res.end(body);
|
|
121
136
|
}
|
|
122
137
|
else if (req.method === 'PUT') {
|
package/dist/file-server.test.js
CHANGED
|
@@ -60,6 +60,28 @@ describe('file-server: basic GET/PUT', () => {
|
|
|
60
60
|
assert.strictEqual(getRes.status, 200, `GET failed: ${getRes.status}`);
|
|
61
61
|
assert.strictEqual(await getRes.text(), 'hello world');
|
|
62
62
|
});
|
|
63
|
+
test('GET hardens against stored XSS: nosniff + attachment disposition', async () => {
|
|
64
|
+
// A client uploads an HTML payload under a text/html content type — the
|
|
65
|
+
// classic stored-XSS setup. When the dev server serves it back it must
|
|
66
|
+
// never let the browser render it inline in the app's origin.
|
|
67
|
+
const bucket = new FileBucket(scope, 'fs-xss');
|
|
68
|
+
const putUrl = await bucket.putUrl('payload.html', { contentType: 'text/html' });
|
|
69
|
+
const adjustedPut = putUrl.replace(/localhost:\d+/, `localhost:${port}`);
|
|
70
|
+
const putRes = await fetch(adjustedPut, {
|
|
71
|
+
method: 'PUT',
|
|
72
|
+
body: '<script>alert(document.domain)</script>',
|
|
73
|
+
headers: { 'Content-Type': 'text/html' },
|
|
74
|
+
});
|
|
75
|
+
assert.strictEqual(putRes.status, 200, `PUT failed: ${putRes.status}`);
|
|
76
|
+
const getUrl = await bucket.getUrl('payload.html');
|
|
77
|
+
const adjustedGet = getUrl.replace(/localhost:\d+/, `localhost:${port}`);
|
|
78
|
+
const getRes = await fetch(adjustedGet);
|
|
79
|
+
assert.strictEqual(getRes.status, 200, `GET failed: ${getRes.status}`);
|
|
80
|
+
assert.strictEqual(getRes.headers.get('x-content-type-options'), 'nosniff', 'GET must send X-Content-Type-Options: nosniff');
|
|
81
|
+
assert.match(getRes.headers.get('content-disposition') ?? '', /^attachment/, 'GET must force download via Content-Disposition: attachment');
|
|
82
|
+
// consume the body so the socket closes cleanly
|
|
83
|
+
await getRes.arrayBuffer();
|
|
84
|
+
});
|
|
63
85
|
test('GET non-existent file returns 404', async () => {
|
|
64
86
|
const bucket = new FileBucket(scope, 'fs-404');
|
|
65
87
|
const url = await bucket.getUrl('missing.txt');
|
|
@@ -72,6 +94,21 @@ describe('file-server: basic GET/PUT', () => {
|
|
|
72
94
|
const res = await fetch(url);
|
|
73
95
|
assert.strictEqual(res.status, 403);
|
|
74
96
|
});
|
|
97
|
+
test('a token forged with the former hardcoded secret is rejected', async () => {
|
|
98
|
+
// The signing secret used to be a fixed, source-visible literal, so anyone
|
|
99
|
+
// could mint a valid token offline for any fullId/path without ever calling
|
|
100
|
+
// getUrl()/putUrl(). It is now a per-process random secret. A token signed
|
|
101
|
+
// with the old literal must no longer validate.
|
|
102
|
+
const bucket = new FileBucket(scope, 'fs-forge');
|
|
103
|
+
const putUrl = await bucket.putUrl('secret.txt', { contentType: 'text/plain' });
|
|
104
|
+
const adjustedPut = putUrl.replace(/localhost:\d+/, `localhost:${port}`);
|
|
105
|
+
await fetch(adjustedPut, { method: 'PUT', body: 'data', headers: { 'Content-Type': 'text/plain' } });
|
|
106
|
+
const forged = mintFileToken('fsrv-fs-forge', 'secret.txt', 'GET', 3600, '__blocks_file_bucket_dev_secret__');
|
|
107
|
+
const url = `http://localhost:${port}/.bb-file-bucket/fsrv-fs-forge/secret.txt?token=${forged}`;
|
|
108
|
+
const res = await fetch(url);
|
|
109
|
+
await res.arrayBuffer();
|
|
110
|
+
assert.strictEqual(res.status, 403, 'a token forged with the old hardcoded secret must be rejected');
|
|
111
|
+
});
|
|
75
112
|
test('PUT for an unregistered bucket fails loud (500), no silent write', async () => {
|
|
76
113
|
// Mint a structurally valid token for a fullId that has no FileBucket
|
|
77
114
|
// instance registered. The server must refuse rather than fall back to
|
package/dist/tokens.d.ts
CHANGED
|
@@ -1,4 +1,20 @@
|
|
|
1
|
-
|
|
1
|
+
/**
|
|
2
|
+
* Per-process HMAC secret for signing local presigned-URL tokens.
|
|
3
|
+
*
|
|
4
|
+
* The mock bucket (which mints tokens) and the dev file server (which validates
|
|
5
|
+
* them) both import this module and run in the *same* dev-server process, so a
|
|
6
|
+
* value generated once at module load is shared between them via the ESM module
|
|
7
|
+
* cache — no configuration needed.
|
|
8
|
+
*
|
|
9
|
+
* This deliberately replaces a previously hardcoded literal. A fixed, source-
|
|
10
|
+
* visible secret let anyone forge a valid token for any `fullId`/path/method and
|
|
11
|
+
* hit the dev file server without ever calling `getUrl()`/`putUrl()`, defeating
|
|
12
|
+
* the point of signing. A random per-process secret makes tokens unforgeable
|
|
13
|
+
* while keeping the local round-trip working, since both ends share this value.
|
|
14
|
+
* Tokens do not need to survive a dev-server restart (presigned URLs are short-
|
|
15
|
+
* lived and re-minted on demand), so per-process randomness is sufficient.
|
|
16
|
+
*/
|
|
17
|
+
export declare const LOCAL_FILE_SECRET: string;
|
|
2
18
|
interface FileTokenPayload {
|
|
3
19
|
fullId: string;
|
|
4
20
|
path: string;
|
package/dist/tokens.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"tokens.d.ts","sourceRoot":"","sources":["../src/tokens.ts"],"names":[],"mappings":"AAQA,eAAO,MAAM,iBAAiB,
|
|
1
|
+
{"version":3,"file":"tokens.d.ts","sourceRoot":"","sources":["../src/tokens.ts"],"names":[],"mappings":"AAQA;;;;;;;;;;;;;;;GAeG;AACH,eAAO,MAAM,iBAAiB,QAAwC,CAAC;AAEvE,UAAU,gBAAgB;IACzB,MAAM,EAAE,MAAM,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,KAAK,GAAG,KAAK,CAAC;IACtB,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,GAAG,EAAE,MAAM,CAAC;CACZ;AAED,wBAAgB,aAAa,CAC5B,MAAM,EAAE,MAAM,EACd,IAAI,EAAE,MAAM,EACZ,MAAM,EAAE,KAAK,GAAG,KAAK,EACrB,SAAS,EAAE,MAAM,EACjB,MAAM,EAAE,MAAM,EACd,WAAW,CAAC,EAAE,MAAM,GAClB,MAAM,CAWR;AAED,wBAAgB,iBAAiB,CAChC,KAAK,EAAE,MAAM,EACb,MAAM,EAAE,MAAM,EACd,cAAc,EAAE,MAAM,EACtB,YAAY,EAAE,MAAM,EACpB,cAAc,EAAE,KAAK,GAAG,KAAK,GAC3B,gBAAgB,GAAG,IAAI,CAgBzB"}
|
package/dist/tokens.js
CHANGED
|
@@ -1,9 +1,25 @@
|
|
|
1
1
|
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
|
2
2
|
// SPDX-License-Identifier: Apache-2.0
|
|
3
|
-
import { createHmac } from 'node:crypto';
|
|
3
|
+
import { createHmac, randomBytes } from 'node:crypto';
|
|
4
4
|
import { constantTimeEquals } from '@aws-blocks/core/bb-utils';
|
|
5
5
|
// ── Token helpers ───────────────────────────────────────────────────────────
|
|
6
|
-
|
|
6
|
+
/**
|
|
7
|
+
* Per-process HMAC secret for signing local presigned-URL tokens.
|
|
8
|
+
*
|
|
9
|
+
* The mock bucket (which mints tokens) and the dev file server (which validates
|
|
10
|
+
* them) both import this module and run in the *same* dev-server process, so a
|
|
11
|
+
* value generated once at module load is shared between them via the ESM module
|
|
12
|
+
* cache — no configuration needed.
|
|
13
|
+
*
|
|
14
|
+
* This deliberately replaces a previously hardcoded literal. A fixed, source-
|
|
15
|
+
* visible secret let anyone forge a valid token for any `fullId`/path/method and
|
|
16
|
+
* hit the dev file server without ever calling `getUrl()`/`putUrl()`, defeating
|
|
17
|
+
* the point of signing. A random per-process secret makes tokens unforgeable
|
|
18
|
+
* while keeping the local round-trip working, since both ends share this value.
|
|
19
|
+
* Tokens do not need to survive a dev-server restart (presigned URLs are short-
|
|
20
|
+
* lived and re-minted on demand), so per-process randomness is sufficient.
|
|
21
|
+
*/
|
|
22
|
+
export const LOCAL_FILE_SECRET = randomBytes(32).toString('base64url');
|
|
7
23
|
export function mintFileToken(fullId, path, method, expiresIn, secret, contentType) {
|
|
8
24
|
const payload = {
|
|
9
25
|
fullId,
|
package/dist/version.d.ts
CHANGED
package/dist/version.js
CHANGED
package/package.json
CHANGED
|
@@ -1,6 +1,15 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@aws-blocks/bb-file-bucket",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.3",
|
|
4
|
+
"repository": {
|
|
5
|
+
"type": "git",
|
|
6
|
+
"url": "git+https://github.com/aws-devtools-labs/aws-blocks.git",
|
|
7
|
+
"directory": "packages/bb-file-bucket"
|
|
8
|
+
},
|
|
9
|
+
"homepage": "https://github.com/aws-devtools-labs/aws-blocks/tree/main/packages/bb-file-bucket#readme",
|
|
10
|
+
"bugs": {
|
|
11
|
+
"url": "https://github.com/aws-devtools-labs/aws-blocks/issues"
|
|
12
|
+
},
|
|
4
13
|
"author": "Amazon Web Services",
|
|
5
14
|
"license": "Apache-2.0",
|
|
6
15
|
"type": "module",
|
|
@@ -41,8 +50,8 @@
|
|
|
41
50
|
"test": "node --test dist/index.test.js dist/index.cdk.test.js dist/scan.test.js dist/file-server.test.js dist/url-encoding.test.js dist/path-containment.test.js dist/bucket-name.test.js"
|
|
42
51
|
},
|
|
43
52
|
"dependencies": {
|
|
44
|
-
"@aws-blocks/bb-logger": "^0.1.
|
|
45
|
-
"@aws-blocks/core": "^0.1.
|
|
53
|
+
"@aws-blocks/bb-logger": "^0.1.3",
|
|
54
|
+
"@aws-blocks/core": "^0.1.17",
|
|
46
55
|
"@aws-sdk/client-s3": "^3.0.0",
|
|
47
56
|
"@aws-sdk/s3-request-presigner": "^3.0.0"
|
|
48
57
|
},
|
package/src/file-server.test.ts
CHANGED
|
@@ -72,6 +72,38 @@ describe('file-server: basic GET/PUT', () => {
|
|
|
72
72
|
assert.strictEqual(await getRes.text(), 'hello world');
|
|
73
73
|
});
|
|
74
74
|
|
|
75
|
+
test('GET hardens against stored XSS: nosniff + attachment disposition', async () => {
|
|
76
|
+
// A client uploads an HTML payload under a text/html content type — the
|
|
77
|
+
// classic stored-XSS setup. When the dev server serves it back it must
|
|
78
|
+
// never let the browser render it inline in the app's origin.
|
|
79
|
+
const bucket = new FileBucket(scope, 'fs-xss');
|
|
80
|
+
const putUrl = await bucket.putUrl('payload.html', { contentType: 'text/html' });
|
|
81
|
+
const adjustedPut = putUrl.replace(/localhost:\d+/, `localhost:${port}`);
|
|
82
|
+
const putRes = await fetch(adjustedPut, {
|
|
83
|
+
method: 'PUT',
|
|
84
|
+
body: '<script>alert(document.domain)</script>',
|
|
85
|
+
headers: { 'Content-Type': 'text/html' },
|
|
86
|
+
});
|
|
87
|
+
assert.strictEqual(putRes.status, 200, `PUT failed: ${putRes.status}`);
|
|
88
|
+
|
|
89
|
+
const getUrl = await bucket.getUrl('payload.html');
|
|
90
|
+
const adjustedGet = getUrl.replace(/localhost:\d+/, `localhost:${port}`);
|
|
91
|
+
const getRes = await fetch(adjustedGet);
|
|
92
|
+
assert.strictEqual(getRes.status, 200, `GET failed: ${getRes.status}`);
|
|
93
|
+
assert.strictEqual(
|
|
94
|
+
getRes.headers.get('x-content-type-options'),
|
|
95
|
+
'nosniff',
|
|
96
|
+
'GET must send X-Content-Type-Options: nosniff',
|
|
97
|
+
);
|
|
98
|
+
assert.match(
|
|
99
|
+
getRes.headers.get('content-disposition') ?? '',
|
|
100
|
+
/^attachment/,
|
|
101
|
+
'GET must force download via Content-Disposition: attachment',
|
|
102
|
+
);
|
|
103
|
+
// consume the body so the socket closes cleanly
|
|
104
|
+
await getRes.arrayBuffer();
|
|
105
|
+
});
|
|
106
|
+
|
|
75
107
|
test('GET non-existent file returns 404', async () => {
|
|
76
108
|
const bucket = new FileBucket(scope, 'fs-404');
|
|
77
109
|
const url = await bucket.getUrl('missing.txt');
|
|
@@ -87,6 +119,23 @@ describe('file-server: basic GET/PUT', () => {
|
|
|
87
119
|
assert.strictEqual(res.status, 403);
|
|
88
120
|
});
|
|
89
121
|
|
|
122
|
+
test('a token forged with the former hardcoded secret is rejected', async () => {
|
|
123
|
+
// The signing secret used to be a fixed, source-visible literal, so anyone
|
|
124
|
+
// could mint a valid token offline for any fullId/path without ever calling
|
|
125
|
+
// getUrl()/putUrl(). It is now a per-process random secret. A token signed
|
|
126
|
+
// with the old literal must no longer validate.
|
|
127
|
+
const bucket = new FileBucket(scope, 'fs-forge');
|
|
128
|
+
const putUrl = await bucket.putUrl('secret.txt', { contentType: 'text/plain' });
|
|
129
|
+
const adjustedPut = putUrl.replace(/localhost:\d+/, `localhost:${port}`);
|
|
130
|
+
await fetch(adjustedPut, { method: 'PUT', body: 'data', headers: { 'Content-Type': 'text/plain' } });
|
|
131
|
+
|
|
132
|
+
const forged = mintFileToken('fsrv-fs-forge', 'secret.txt', 'GET', 3600, '__blocks_file_bucket_dev_secret__');
|
|
133
|
+
const url = `http://localhost:${port}/.bb-file-bucket/fsrv-fs-forge/secret.txt?token=${forged}`;
|
|
134
|
+
const res = await fetch(url);
|
|
135
|
+
await res.arrayBuffer();
|
|
136
|
+
assert.strictEqual(res.status, 403, 'a token forged with the old hardcoded secret must be rejected');
|
|
137
|
+
});
|
|
138
|
+
|
|
90
139
|
test('PUT for an unregistered bucket fails loud (500), no silent write', async () => {
|
|
91
140
|
// Mint a structurally valid token for a fullId that has no FileBucket
|
|
92
141
|
// instance registered. The server must refuse rather than fall back to
|
package/src/file-server.ts
CHANGED
|
@@ -141,7 +141,22 @@ export function attach(httpServer: Server) {
|
|
|
141
141
|
}
|
|
142
142
|
|
|
143
143
|
const body = readFileSync(readPath);
|
|
144
|
-
|
|
144
|
+
// The stored object body and its Content-Type are attacker-controlled
|
|
145
|
+
// (any client with a presigned PUT can upload arbitrary bytes under an
|
|
146
|
+
// arbitrary content type). Serving that back inline turns the dev file
|
|
147
|
+
// server into a stored-XSS vector: an uploaded `text/html` (or sniffed
|
|
148
|
+
// HTML/SVG) payload would execute in the origin of the local app.
|
|
149
|
+
// `nosniff` stops the browser from MIME-sniffing octet-streams into
|
|
150
|
+
// HTML, and `Content-Disposition: attachment` forces a download rather
|
|
151
|
+
// than inline rendering — so an uploaded document can never run as a
|
|
152
|
+
// page. Real S3 objects served through CloudFront are hardened the same
|
|
153
|
+
// way; this keeps local dev from being weaker than production.
|
|
154
|
+
res.writeHead(200, {
|
|
155
|
+
'Content-Type': contentType,
|
|
156
|
+
'Content-Length': body.length.toString(),
|
|
157
|
+
'X-Content-Type-Options': 'nosniff',
|
|
158
|
+
'Content-Disposition': 'attachment',
|
|
159
|
+
});
|
|
145
160
|
res.end(body);
|
|
146
161
|
} else if (req.method === 'PUT') {
|
|
147
162
|
const valid = validateFileToken(token, LOCAL_FILE_SECRET, fullId, path, 'PUT');
|
package/src/tokens.ts
CHANGED
|
@@ -1,12 +1,28 @@
|
|
|
1
1
|
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
|
2
2
|
// SPDX-License-Identifier: Apache-2.0
|
|
3
3
|
|
|
4
|
-
import { createHmac } from 'node:crypto';
|
|
4
|
+
import { createHmac, randomBytes } from 'node:crypto';
|
|
5
5
|
import { constantTimeEquals } from '@aws-blocks/core/bb-utils';
|
|
6
6
|
|
|
7
7
|
// ── Token helpers ───────────────────────────────────────────────────────────
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
/**
|
|
10
|
+
* Per-process HMAC secret for signing local presigned-URL tokens.
|
|
11
|
+
*
|
|
12
|
+
* The mock bucket (which mints tokens) and the dev file server (which validates
|
|
13
|
+
* them) both import this module and run in the *same* dev-server process, so a
|
|
14
|
+
* value generated once at module load is shared between them via the ESM module
|
|
15
|
+
* cache — no configuration needed.
|
|
16
|
+
*
|
|
17
|
+
* This deliberately replaces a previously hardcoded literal. A fixed, source-
|
|
18
|
+
* visible secret let anyone forge a valid token for any `fullId`/path/method and
|
|
19
|
+
* hit the dev file server without ever calling `getUrl()`/`putUrl()`, defeating
|
|
20
|
+
* the point of signing. A random per-process secret makes tokens unforgeable
|
|
21
|
+
* while keeping the local round-trip working, since both ends share this value.
|
|
22
|
+
* Tokens do not need to survive a dev-server restart (presigned URLs are short-
|
|
23
|
+
* lived and re-minted on demand), so per-process randomness is sufficient.
|
|
24
|
+
*/
|
|
25
|
+
export const LOCAL_FILE_SECRET = randomBytes(32).toString('base64url');
|
|
10
26
|
|
|
11
27
|
interface FileTokenPayload {
|
|
12
28
|
fullId: string;
|
package/src/version.ts
CHANGED