@bymax-one/nest-storage 1.0.0 → 1.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +77 -4
- package/README.md +425 -106
- package/package.json +39 -23
package/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,78 @@ The format follows [Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/
|
|
|
8
8
|
|
|
9
9
|
## [Unreleased]
|
|
10
10
|
|
|
11
|
+
## [1.0.2] - 2026-08-04
|
|
12
|
+
|
|
13
|
+
Documentation only, again — and for the same reason 1.0.1 existed: the README ships
|
|
14
|
+
inside the package, so a correction that stays on `main` leaves the npm page wrong.
|
|
15
|
+
`dist/` is byte-identical to 1.0.0 and 1.0.1, verified by unpacking the published
|
|
16
|
+
tarball and diffing it against a fresh build rather than asserting it.
|
|
17
|
+
|
|
18
|
+
### Fixed
|
|
19
|
+
|
|
20
|
+
- **The vulnerability contact pointed at a mailbox that does not exist.** The README's
|
|
21
|
+
Security Policy and `SECURITY.md` both named `security@bymax.one`. A researcher
|
|
22
|
+
following those instructions sent a private disclosure into nothing — and from their
|
|
23
|
+
side, an unread inbox and a maintainer ignoring them look identical, so a report that
|
|
24
|
+
gets no acknowledgement is usually published anyway. Everything now names
|
|
25
|
+
`support@bymax.one`, the single inbox that routes.
|
|
26
|
+
|
|
27
|
+
### Changed
|
|
28
|
+
|
|
29
|
+
- **The README follows the layout of the published `@bymax-one` libraries, not just
|
|
30
|
+
their section names.** The previous release standardized which sections exist and
|
|
31
|
+
left the inside of each as it was. Features is now grouped under subheadings with
|
|
32
|
+
one entry per capability, Overview carries a `Why nest-storage?` block, Architecture
|
|
33
|
+
carries the design-principles table that follows its diagram everywhere else,
|
|
34
|
+
Security Model is one subheading per property, and Testing & Quality states what the
|
|
35
|
+
suite is held to before listing the commands. The tail gained the family's license
|
|
36
|
+
line and footer, and Subpath Exports gained the dependency diagram and peer matrix.
|
|
37
|
+
- **Four statements corrected against the source.** `KeyResolverService` was listed as
|
|
38
|
+
public API and is not exported; the module was described as refusing an unusable
|
|
39
|
+
configuration at bootstrap, where `validate-options` deliberately tolerates empty
|
|
40
|
+
credentials so a development workflow boots; `STORAGE_INVALID_PART_COUNT` was missing
|
|
41
|
+
from the error table; and the suppression claim said the production source carries no
|
|
42
|
+
mutation directives, where it carries five, each naming why its mutant is provably
|
|
43
|
+
equivalent.
|
|
44
|
+
|
|
45
|
+
## [1.0.1] - 2026-08-03
|
|
46
|
+
|
|
47
|
+
Documentation only. `dist/` is byte-identical to the one published in 1.0.0 —
|
|
48
|
+
verified by unpacking `@bymax-one/nest-storage@1.0.0` from the registry and
|
|
49
|
+
diffing it against a fresh build, not asserted. The README ships inside the
|
|
50
|
+
package, so a README that is wrong on npm stays wrong until a release replaces
|
|
51
|
+
it; that is the whole reason this version exists.
|
|
52
|
+
|
|
53
|
+
### Fixed
|
|
54
|
+
|
|
55
|
+
- **Four statements in the README that the code does not support.**
|
|
56
|
+
`KeyResolverService` was documented as public API — it is not exported from the
|
|
57
|
+
barrel, so `import { KeyResolverService } from '@bymax-one/nest-storage'` does
|
|
58
|
+
not resolve. The module was described as refusing an unusable configuration at
|
|
59
|
+
bootstrap; `validate-options` deliberately tolerates empty credentials so a
|
|
60
|
+
development workflow boots without storage, and operations then fail with
|
|
61
|
+
`STORAGE_NOT_CONFIGURED`. `STORAGE_INVALID_PART_COUNT` was missing from the
|
|
62
|
+
error table although `getMultipartUploadUrls()` throws it for `parts <= 0`.
|
|
63
|
+
- **The error-payload claim, in both places it was made.** The error table and
|
|
64
|
+
the security table said `details` carries the provider code, HTTP status and
|
|
65
|
+
request id _only_. `mapAwsError` also includes `awsMessage` and spreads the
|
|
66
|
+
call site's context — `op`, `bucket`, and the resolved `key` or `prefix` — so an
|
|
67
|
+
object key does reach whatever consumes the exception. Both now list what is
|
|
68
|
+
actually there, and the security model says the consequence plainly.
|
|
69
|
+
- **The supply-chain row named tools this repository does not run.** It claimed
|
|
70
|
+
OSV-Scanner _and_ TruffleHog; only OSV-Scanner is wired up here, alongside
|
|
71
|
+
CodeQL and OpenSSF Scorecard. It also called the Actions SHA-pinned without
|
|
72
|
+
qualification, where the org-internal reusables are referenced by tag.
|
|
73
|
+
|
|
74
|
+
### Changed
|
|
75
|
+
|
|
76
|
+
- **The README follows the `@bymax-one` family layout.** Header rebuilt to the
|
|
77
|
+
shared shape — wordmark, package name, claim and feature line, one badge row,
|
|
78
|
+
one navigation row — and the section spine completed with API Reference,
|
|
79
|
+
Architecture, Security Model, Security Table, Tech Stack, Testing & Quality and
|
|
80
|
+
Security Policy, which every published library in the family carries and this
|
|
81
|
+
one did not.
|
|
82
|
+
|
|
11
83
|
## [1.0.0] - 2026-08-03
|
|
12
84
|
|
|
13
85
|
First published release. Everything below ships in it.
|
|
@@ -57,12 +129,11 @@ have regressed from. They are kept because the reasoning is worth having.
|
|
|
57
129
|
- **`BYMAX_STORAGE_S3_CLIENT` DI token** — raw `S3Client` injection for provider-specific advanced operations (spec §11.2)
|
|
58
130
|
- **Non-AWS checksum opt-out** — the five non-AWS recipes set `requestChecksumCalculation` / `responseChecksumValidation` to `'WHEN_REQUIRED'` to prevent the SDK's default CRC32 `x-amz-checksum-*` headers from being sent to providers that reject them
|
|
59
131
|
|
|
60
|
-
|
|
61
132
|
- **`pnpm check:exports`** runs `attw --pack . --profile strict` against the packed
|
|
62
133
|
tarball. Its absence is why both defects above went unnoticed: a source-level
|
|
63
134
|
typecheck compiles `src` and never resolves through the `exports` map.
|
|
64
135
|
- **`pnpm check:runtime`** packs the tarball, lays it out the way npm would, and
|
|
65
|
-
loads every subpath from it in ESM
|
|
136
|
+
loads every subpath from it in ESM _and_ CommonJS, asserting the expected values
|
|
66
137
|
are really exported. `attw` proves the declarations resolve; it never runs the
|
|
67
138
|
JavaScript. Both gates run in CI.
|
|
68
139
|
|
|
@@ -70,7 +141,7 @@ have regressed from. They are kept because the reasoning is worth having.
|
|
|
70
141
|
|
|
71
142
|
- **CommonJS consumers resolved ESM type declarations.** The `exports` map
|
|
72
143
|
declared a single `types` condition, so `require()` landed on `.d.ts` instead of
|
|
73
|
-
`.d.cts` — `attw --profile strict` reports it as
|
|
144
|
+
`.d.cts` — `attw --profile strict` reports it as _Masquerading as ESM_ on every
|
|
74
145
|
subpath. Types are now declared per condition.
|
|
75
146
|
|
|
76
147
|
- **`node10` type resolution failed outright**: the manifest carried no complete
|
|
@@ -96,4 +167,6 @@ have regressed from. They are kept because the reasoning is worth having.
|
|
|
96
167
|
---
|
|
97
168
|
|
|
98
169
|
[1.0.0]: https://github.com/bymaxone/nest-storage/releases/tag/v1.0.0
|
|
99
|
-
[
|
|
170
|
+
[1.0.1]: https://github.com/bymaxone/nest-storage/compare/v1.0.0...v1.0.1
|
|
171
|
+
[1.0.2]: https://github.com/bymaxone/nest-storage/compare/v1.0.1...v1.0.2
|
|
172
|
+
[Unreleased]: https://github.com/bymaxone/nest-storage/compare/v1.0.2...HEAD
|
package/README.md
CHANGED
|
@@ -1,53 +1,155 @@
|
|
|
1
1
|
<p align="center">
|
|
2
|
-
<
|
|
3
|
-
<a href="https://www.npmjs.com/package/@bymax-one/nest-storage"><img src="https://img.shields.io/npm/dm/@bymax-one/nest-storage.svg" alt="npm downloads" /></a>
|
|
4
|
-
<a href="https://github.com/bymaxone/nest-storage/actions/workflows/ci.yml"><img src="https://github.com/bymaxone/nest-storage/actions/workflows/ci.yml/badge.svg" alt="CI" /></a>
|
|
5
|
-
<a href="https://codecov.io/gh/bymaxone/nest-storage"><img src="https://codecov.io/gh/bymaxone/nest-storage/branch/main/graph/badge.svg" alt="coverage" /></a>
|
|
6
|
-
<a href="https://stryker-mutator.io"><img src="https://img.shields.io/badge/mutation%20score-100%25-brightgreen" alt="mutation score" /></a>
|
|
7
|
-
<a href="https://api.securityscorecards.dev/projects/github.com/bymaxone/nest-storage"><img src="https://api.securityscorecards.dev/projects/github.com/bymaxone/nest-storage/badge" alt="OpenSSF Scorecard" /></a>
|
|
8
|
-
<a href="LICENSE"><img src="https://img.shields.io/npm/l/@bymax-one/nest-storage.svg" alt="license" /></a>
|
|
9
|
-
<img src="https://img.shields.io/badge/TypeScript-5.x-blue.svg" alt="TypeScript" />
|
|
10
|
-
<img src="https://img.shields.io/badge/Node-%3E%3D24-green.svg" alt="Node 24+" />
|
|
2
|
+
<img src="https://img.shields.io/badge/%40bymax--one-nest--storage-000000?style=for-the-badge&logo=nestjs&logoColor=E0234E" alt="@bymax-one/nest-storage" />
|
|
11
3
|
</p>
|
|
12
4
|
|
|
13
5
|
<h1 align="center">@bymax-one/nest-storage</h1>
|
|
14
6
|
|
|
15
7
|
<p align="center">
|
|
16
|
-
Provider-agnostic S3-compatible object storage for NestJS
|
|
17
|
-
|
|
18
|
-
|
|
8
|
+
<strong>Provider-agnostic S3-compatible object storage for NestJS</strong><br />
|
|
9
|
+
<sub>AWS S3 · Cloudflare R2 · Backblaze B2 · DigitalOcean Spaces · MinIO · Wasabi · Presigned URLs · Multipart · Virus Scanning · Zero Runtime Dependencies</sub>
|
|
10
|
+
</p>
|
|
11
|
+
|
|
12
|
+
<p align="center">
|
|
13
|
+
<a href="https://www.npmjs.com/package/@bymax-one/nest-storage"><img src="https://img.shields.io/npm/v/@bymax-one/nest-storage?style=flat-square&colorA=000000&colorB=000000" alt="npm version" /></a>
|
|
14
|
+
<a href="https://www.npmjs.com/package/@bymax-one/nest-storage"><img src="https://img.shields.io/npm/dm/@bymax-one/nest-storage?style=flat-square&colorA=000000&colorB=000000" alt="npm downloads" /></a>
|
|
15
|
+
<a href="https://github.com/bymaxone/nest-storage/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/bymaxone/nest-storage/ci.yml?branch=main&style=flat-square&colorA=000000&label=CI" alt="CI status" /></a>
|
|
16
|
+
<a href="https://github.com/bymaxone/nest-storage/actions/workflows/ci.yml"><img src="https://img.shields.io/badge/coverage-100%25-brightgreen?style=flat-square&colorA=000000" alt="coverage" /></a>
|
|
17
|
+
<a href="https://github.com/bymaxone/nest-storage/blob/main/docs/mutation_testing_results.md"><img src="https://img.shields.io/badge/mutation-100%25-brightgreen?style=flat-square&colorA=000000" alt="mutation score" /></a>
|
|
18
|
+
<a href="https://scorecard.dev/viewer/?uri=github.com/bymaxone/nest-storage"><img src="https://api.scorecard.dev/projects/github.com/bymaxone/nest-storage/badge?style=flat-square" alt="OpenSSF Scorecard" /></a>
|
|
19
|
+
<a href="https://github.com/bymaxone/nest-storage/blob/main/LICENSE"><img src="https://img.shields.io/github/license/bymaxone/nest-storage?style=flat-square&colorA=000000&colorB=000000" alt="license" /></a>
|
|
20
|
+
<a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/TypeScript-strict-3178C6?style=flat-square&logo=typescript&logoColor=white" alt="TypeScript" /></a>
|
|
21
|
+
<a href="https://nodejs.org/"><img src="https://img.shields.io/badge/Node.js-24%2B-339933?style=flat-square&logo=node.js&logoColor=white" alt="Node.js" /></a>
|
|
22
|
+
</p>
|
|
23
|
+
|
|
24
|
+
<p align="center">
|
|
25
|
+
<a href="https://github.com/bymaxone/nest-storage">GitHub</a> ·
|
|
26
|
+
<a href="https://github.com/bymaxone/nest-storage/issues">Issues</a> ·
|
|
27
|
+
<a href="#-quick-start">Quick Start</a> ·
|
|
28
|
+
<a href="#-api-reference">API Reference</a> ·
|
|
29
|
+
<a href="https://github.com/bymaxone/nest-storage-example">Example App</a>
|
|
19
30
|
</p>
|
|
20
31
|
|
|
21
32
|
---
|
|
22
33
|
|
|
23
|
-
## Overview
|
|
34
|
+
## ✨ Overview
|
|
35
|
+
|
|
36
|
+
`@bymax-one/nest-storage` is a NestJS dynamic module that wraps the AWS SDK v3
|
|
37
|
+
(`@aws-sdk/client-s3`) behind one typed surface for object storage. Instead of building an
|
|
38
|
+
`S3Client`, deciding per provider whether path-style addressing is required, and hand-rolling
|
|
39
|
+
multipart, presigning, validation and key normalization in every service, you install one
|
|
40
|
+
library and get all of it — against AWS S3, Cloudflare R2, Backblaze B2, DigitalOcean Spaces,
|
|
41
|
+
MinIO or Wasabi, with the same code.
|
|
42
|
+
|
|
43
|
+
The library has **zero direct dependencies** — the three `@aws-sdk/*` packages and `@nestjs/*`
|
|
44
|
+
arrive as peer dependencies, so you control exact versions and the supply-chain surface stays
|
|
45
|
+
minimal.
|
|
46
|
+
|
|
47
|
+
### Why nest-storage?
|
|
48
|
+
|
|
49
|
+
- **One engine, six providers.** `providerRecipes` differ only in endpoint, region and the flags
|
|
50
|
+
each provider needs (`forcePathStyle` for MinIO, `auto` region for R2). There is no
|
|
51
|
+
per-provider code path, so switching provider is a configuration change and every operation
|
|
52
|
+
behaves the same way afterwards.
|
|
53
|
+
- **The key guard is one chokepoint.** Every caller-supplied key passes through
|
|
54
|
+
`KeyResolverService` before it reaches S3 — `..` segments, a leading `/`, null bytes and empty
|
|
55
|
+
keys are refused, and only then is `keyPrefix` prepended. A tenant cannot climb out of its
|
|
56
|
+
prefix by naming a key.
|
|
57
|
+
- **Uploads are refused before they are stored.** MIME allowlist and size limit first, then any
|
|
58
|
+
`IUploadValidator` you register, then the `IFileScanner` if configured — pre-upload,
|
|
59
|
+
post-upload, or both.
|
|
60
|
+
- **Presigned URLs cannot outlive what SigV4 allows.** TTLs are clamped and the hard ceiling of
|
|
61
|
+
604 800 s is enforced locally, rather than handed to a signature the provider rejects at use
|
|
62
|
+
time.
|
|
24
63
|
|
|
25
|
-
|
|
64
|
+
---
|
|
26
65
|
|
|
27
|
-
## Features
|
|
66
|
+
## 🔥 Features
|
|
67
|
+
|
|
68
|
+
### ⬆️ Uploading
|
|
69
|
+
|
|
70
|
+
- ✅ **Multipart upload** — `@aws-sdk/lib-storage` with automatic abort on failure
|
|
71
|
+
(`leavePartsOnError: false`), progress events, and a configurable threshold and part size
|
|
72
|
+
- ✅ **Content validation** — MIME allowlist with wildcards (`image/*`, `text/*`) and a size
|
|
73
|
+
limit, refused before a byte is stored
|
|
74
|
+
- ✅ **Pluggable validators** — register any number of `IUploadValidator`s; each runs after the
|
|
75
|
+
built-in checks and can refuse with its own reason
|
|
76
|
+
- ✅ **Virus scanning hook** — `IFileScanner` pre-upload, post-upload or both, for ClamAV, AWS
|
|
77
|
+
Macie or anything that answers `'clean' | 'infected' | 'unknown'`
|
|
78
|
+
- ✅ **Idempotent retries** — an in-memory LRU (1 000 entries / 24 h by default) returns the
|
|
79
|
+
cached `UploadResult` for a repeated `idempotencyKey` instead of storing the object twice
|
|
80
|
+
|
|
81
|
+
### 🔗 Access & URLs
|
|
82
|
+
|
|
83
|
+
- ✅ **Presigned GET / PUT** — with optional content-type and length conditions on the PUT
|
|
84
|
+
- ✅ **Presigned multipart** — one signed URL per part plus the upload id, so a browser can
|
|
85
|
+
upload directly without the bytes passing through your process
|
|
86
|
+
- ✅ **TTL clamping** — SigV4's hard ceiling of 604 800 s (7 days) is enforced locally, not
|
|
87
|
+
discovered when the provider rejects the signature
|
|
88
|
+
- ✅ **Public URL composition** — `getPublicUrl` builds from `cdnBaseUrl` or `publicBaseUrl` for
|
|
89
|
+
buckets that are public by policy; it composes, it does not sign
|
|
90
|
+
|
|
91
|
+
### 🗄️ Objects & Lifecycle
|
|
92
|
+
|
|
93
|
+
- ✅ **Streaming download** — `download` returns a stream; the body is never buffered unless you
|
|
94
|
+
ask for `downloadBuffer`
|
|
95
|
+
- ✅ **Metadata without transfer** — `head` and `exists` answer from object metadata alone
|
|
96
|
+
- ✅ **Paginated listing** — `list` over continuation tokens, with the prefix stripped back off
|
|
97
|
+
the keys it returns
|
|
98
|
+
- ✅ **Server-side copy** — `copy` moves bytes inside the provider; they never reach this process
|
|
99
|
+
- ✅ **Bulk delete with partial results** — `deleteMany` reports what failed instead of throwing
|
|
100
|
+
away the successes
|
|
101
|
+
|
|
102
|
+
### 🧩 Developer Experience
|
|
103
|
+
|
|
104
|
+
- ✅ **Six provider recipes** — `awsS3`, `cloudflareR2`, `backblazeB2`, `digitalOceanSpaces`,
|
|
105
|
+
`minio`, `wasabi`
|
|
106
|
+
- ✅ **Zero runtime dependencies** — `@aws-sdk/*` and `@nestjs/*` all arrive as peer
|
|
107
|
+
dependencies, so you pin the versions
|
|
108
|
+
- ✅ **Two subpaths** — the server surface, and `./shared` carrying types and the error catalog
|
|
109
|
+
that pull in nothing at all
|
|
110
|
+
- ✅ **Escape hatch** — `BYMAX_STORAGE_S3_CLIENT` injects the raw `S3Client` for the
|
|
111
|
+
provider-specific operations this surface deliberately does not wrap
|
|
112
|
+
- ✅ **Multi-tenant key prefix** — `keyPrefix` prepended after normalization, so it cannot be
|
|
113
|
+
escaped by the key
|
|
114
|
+
- ✅ **Server-side encryption** — AES256 or `aws:kms`, globally or per upload
|
|
115
|
+
- ✅ **Typed end to end** — TypeScript `strict` with `exactOptionalPropertyTypes` and
|
|
116
|
+
`noUncheckedIndexedAccess`; zero `any`
|
|
28
117
|
|
|
29
|
-
|
|
30
|
-
- **Presigned GET / PUT / multipart URLs** with TTL clamping and a hard 7-day SigV4 ceiling
|
|
31
|
-
- **MIME whitelist + size limit** (wildcards supported: `image/*`, `text/*`) with a pluggable `IUploadValidator` hook for magic-byte checks and custom business rules
|
|
32
|
-
- **Pluggable `IFileScanner` hook** (pre- and/or post-upload) for ClamAV, AWS Macie, or any scanner that returns `'clean' | 'infected' | 'unknown'`
|
|
33
|
-
- **Six ready-to-use provider recipes** — `awsS3`, `cloudflareR2`, `backblazeB2`, `digitalOceanSpaces`, `minio`, `wasabi`
|
|
34
|
-
- **Path-traversal guard** in `KeyResolverService` — blocks `..`, leading `/`, and empty-after-normalize keys
|
|
35
|
-
- **In-memory LRU idempotency cache** (default 1 000 entries / 24 h) — identical `idempotencyKey` returns the cached `UploadResult`
|
|
36
|
-
- **Server-side encryption** (AES256 / aws:kms) configurable globally or per-upload
|
|
37
|
-
- **`keyPrefix` multi-tenant isolation** — prepended to every resolved key
|
|
38
|
-
- **`forRoot` / `forRootAsync`** dynamic module API; `@Global()` scope; `Symbol()` DI tokens
|
|
39
|
-
- **`BYMAX_STORAGE_S3_CLIENT` token** for injecting the raw `S3Client` for provider-specific operations
|
|
118
|
+
---
|
|
40
119
|
|
|
41
|
-
## Subpath Exports
|
|
120
|
+
## 📦 Subpath Exports
|
|
42
121
|
|
|
43
|
-
| Subpath
|
|
44
|
-
|
|
45
|
-
| `.`
|
|
46
|
-
| `./shared` | Framework-free types (`UploadResult`, `ObjectMetadata`, `ListedObject`, `SignedUrlResult`) + `STORAGE_ERROR_CODES` + `StorageErrorCode`
|
|
122
|
+
| Subpath | Contents |
|
|
123
|
+
| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
124
|
+
| `.` | Server runtime: `BymaxStorageModule`, `StorageService`, `SignedUrlService`, `providerRecipes`, DI tokens, interfaces, `StorageException`, `NoOpUploadValidator`, `NoOpFileScanner` |
|
|
125
|
+
| `./shared` | Framework-free types (`UploadResult`, `ObjectMetadata`, `ListedObject`, `SignedUrlResult`) + `STORAGE_ERROR_CODES` + `StorageErrorCode` |
|
|
126
|
+
|
|
127
|
+
```
|
|
128
|
+
@bymax-one/nest-storage (server — NestJS + @aws-sdk/client-s3)
|
|
129
|
+
│
|
|
130
|
+
└── re-exports ──▶ @bymax-one/nest-storage/shared (zero dependencies)
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
Both subpaths ship ESM **and** CommonJS with declarations for each format, so a
|
|
134
|
+
`require()` consumer receives CommonJS declarations rather than ESM ones. The
|
|
135
|
+
`pnpm check:exports` gate verifies this against the packed tarball.
|
|
136
|
+
|
|
137
|
+
### Peer dependency matrix
|
|
138
|
+
|
|
139
|
+
| Subpath | Required peers |
|
|
140
|
+
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
141
|
+
| `.` (server) | `@nestjs/common ^11.0.16`, `@nestjs/core ^11.1.18`, `@aws-sdk/client-s3 ^3.700`, `@aws-sdk/lib-storage ^3.700`, `@aws-sdk/s3-request-presigner ^3.700`, `reflect-metadata ^0.2` |
|
|
142
|
+
| `./shared` | None |
|
|
47
143
|
|
|
48
144
|
---
|
|
49
145
|
|
|
50
|
-
|
|
146
|
+
> [!TIP]
|
|
147
|
+
> A reference application lives in
|
|
148
|
+
> [`bymaxone/nest-storage-example`](https://github.com/bymaxone/nest-storage-example).
|
|
149
|
+
|
|
150
|
+
---
|
|
151
|
+
|
|
152
|
+
## 🚀 Quick Start
|
|
51
153
|
|
|
52
154
|
### 1 — AWS S3
|
|
53
155
|
|
|
@@ -141,45 +243,45 @@ BymaxStorageModule.forRoot({
|
|
|
141
243
|
|
|
142
244
|
---
|
|
143
245
|
|
|
144
|
-
## Configuration
|
|
246
|
+
## ⚙️ Configuration
|
|
145
247
|
|
|
146
248
|
The full configuration reference lives in `docs/technical_specification.md` §4. Key options:
|
|
147
249
|
|
|
148
|
-
| Option
|
|
149
|
-
|
|
150
|
-
| `endpoint`
|
|
151
|
-
| `region`
|
|
152
|
-
| `bucket`
|
|
153
|
-
| `credentials`
|
|
154
|
-
| `forcePathStyle`
|
|
155
|
-
| `publicBaseUrl`
|
|
156
|
-
| `cdnBaseUrl`
|
|
157
|
-
| `keyPrefix`
|
|
158
|
-
| `maxAttempts`
|
|
159
|
-
| `serverSideEncryption`
|
|
160
|
-
| `requestChecksumCalculation`
|
|
161
|
-
| `responseChecksumValidation`
|
|
162
|
-
| `signedUrls.defaultTtlSeconds` | `number`
|
|
163
|
-
| `signedUrls.maxTtlSeconds`
|
|
164
|
-
| `multipart.thresholdBytes`
|
|
165
|
-
| `multipart.partSizeBytes`
|
|
166
|
-
| `validation.mimeWhitelist`
|
|
167
|
-
| `validation.maxSizeBytes`
|
|
250
|
+
| Option | Type | Default | Notes |
|
|
251
|
+
| ------------------------------ | ------------------------------------------------- | ----------------- | ---------------------------------------------- |
|
|
252
|
+
| `endpoint` | `string` | — | Required. S3-compatible endpoint URL |
|
|
253
|
+
| `region` | `string` | — | Required. Provider region or `'auto'` (R2) |
|
|
254
|
+
| `bucket` | `string` | — | Default bucket (can be overridden per-call) |
|
|
255
|
+
| `credentials` | `{ accessKeyId, secretAccessKey, sessionToken? }` | — | Load from env / Secrets Manager |
|
|
256
|
+
| `forcePathStyle` | `boolean` | `false` | Set to `true` for MinIO and self-hosted |
|
|
257
|
+
| `publicBaseUrl` | `string` | — | Base URL for public `getPublicUrl()` results |
|
|
258
|
+
| `cdnBaseUrl` | `string` | — | CDN edge URL (preferred over `publicBaseUrl`) |
|
|
259
|
+
| `keyPrefix` | `string` | — | Prepended to every resolved key (multi-tenant) |
|
|
260
|
+
| `maxAttempts` | `number` | `3` | SDK v3 retry count |
|
|
261
|
+
| `serverSideEncryption` | `'AES256' \| 'aws:kms' \| 'NONE'` | — | Global SSE policy |
|
|
262
|
+
| `requestChecksumCalculation` | `'WHEN_SUPPORTED' \| 'WHEN_REQUIRED'` | SDK default | Set to `'WHEN_REQUIRED'` for non-AWS providers |
|
|
263
|
+
| `responseChecksumValidation` | `'WHEN_SUPPORTED' \| 'WHEN_REQUIRED'` | SDK default | Set to `'WHEN_REQUIRED'` for non-AWS providers |
|
|
264
|
+
| `signedUrls.defaultTtlSeconds` | `number` | `3600` | Default signed-URL TTL |
|
|
265
|
+
| `signedUrls.maxTtlSeconds` | `number` | `604800` | Hard ceiling; clamped to 7 days at init |
|
|
266
|
+
| `multipart.thresholdBytes` | `number` | `10 MiB` | Switch to multipart above this size |
|
|
267
|
+
| `multipart.partSizeBytes` | `number` | `8 MiB` | Part size (S3 minimum: 5 MiB) |
|
|
268
|
+
| `validation.mimeWhitelist` | `string[]` | sensible defaults | Wildcards: `'image/*'` |
|
|
269
|
+
| `validation.maxSizeBytes` | `number` | — | Maximum upload size in bytes |
|
|
168
270
|
|
|
169
271
|
> **Do not use `maxRetries` or `signatureVersion`** — these are AWS SDK v2 options that do not exist in v3. The v3 SDK is SigV4-only. Use `maxAttempts` (default `3`) for retry configuration.
|
|
170
272
|
|
|
171
273
|
---
|
|
172
274
|
|
|
173
|
-
## Provider Recipes
|
|
275
|
+
## 🧩 Provider Recipes
|
|
174
276
|
|
|
175
|
-
| Recipe
|
|
176
|
-
|
|
177
|
-
| `providerRecipes.awsS3(...)`
|
|
178
|
-
| `providerRecipes.cloudflareR2(...)`
|
|
179
|
-
| `providerRecipes.backblazeB2(...)`
|
|
180
|
-
| `providerRecipes.digitalOceanSpaces(...)` | DigitalOcean Spaces | Sets `cdnBaseUrl` to `*.cdn.digitaloceanspaces.com`; checksums `'WHEN_REQUIRED'`
|
|
181
|
-
| `providerRecipes.minio(...)`
|
|
182
|
-
| `providerRecipes.wasabi(...)`
|
|
277
|
+
| Recipe | Provider | Notes |
|
|
278
|
+
| ----------------------------------------- | ------------------- | ------------------------------------------------------------------------------------- |
|
|
279
|
+
| `providerRecipes.awsS3(...)` | AWS S3 | Keeps the SDK default checksum mode (`'WHEN_SUPPORTED'`); sets SSE-AES256 |
|
|
280
|
+
| `providerRecipes.cloudflareR2(...)` | Cloudflare R2 | `customDomain` required; checksums `'WHEN_REQUIRED'`; `region: 'auto'` |
|
|
281
|
+
| `providerRecipes.backblazeB2(...)` | Backblaze B2 | `forcePathStyle: false` (B2 supports virtual-hosted); checksums `'WHEN_REQUIRED'` |
|
|
282
|
+
| `providerRecipes.digitalOceanSpaces(...)` | DigitalOcean Spaces | Sets `cdnBaseUrl` to `*.cdn.digitaloceanspaces.com`; checksums `'WHEN_REQUIRED'` |
|
|
283
|
+
| `providerRecipes.minio(...)` | MinIO / self-hosted | `forcePathStyle: true`; checksums `'WHEN_REQUIRED'`; region defaults to `'us-east-1'` |
|
|
284
|
+
| `providerRecipes.wasabi(...)` | Wasabi Hot Cloud | Virtual-hosted; checksums `'WHEN_REQUIRED'` |
|
|
183
285
|
|
|
184
286
|
### Provider Compatibility — The #1 Trap
|
|
185
287
|
|
|
@@ -209,7 +311,7 @@ For public access on either provider, use a **bucket policy**, a **CDN**, or **s
|
|
|
209
311
|
|
|
210
312
|
---
|
|
211
313
|
|
|
212
|
-
## Upload
|
|
314
|
+
## ⬆️ Upload
|
|
213
315
|
|
|
214
316
|
### Single-shot / Buffer
|
|
215
317
|
|
|
@@ -248,7 +350,13 @@ Uploads are automatically routed to multipart when `size >= multipart.thresholdB
|
|
|
248
350
|
|
|
249
351
|
```typescript
|
|
250
352
|
const key = `uploads/${randomUUID()}.png`
|
|
251
|
-
const opts = {
|
|
353
|
+
const opts = {
|
|
354
|
+
key,
|
|
355
|
+
body: buf,
|
|
356
|
+
contentType: 'image/png',
|
|
357
|
+
size: buf.length,
|
|
358
|
+
idempotencyKey: 'order-42-avatar',
|
|
359
|
+
}
|
|
252
360
|
|
|
253
361
|
const r1 = await storage.upload(opts)
|
|
254
362
|
const r2 = await storage.upload(opts) // returns cached result instantly
|
|
@@ -257,7 +365,7 @@ console.log(r2.fromIdempotencyCache) // true
|
|
|
257
365
|
|
|
258
366
|
---
|
|
259
367
|
|
|
260
|
-
## Download
|
|
368
|
+
## ⬇️ Download
|
|
261
369
|
|
|
262
370
|
### As a Stream
|
|
263
371
|
|
|
@@ -283,7 +391,7 @@ const { buffer, metadata } = await this.storage.downloadBuffer({ key: 'docs/repo
|
|
|
283
391
|
|
|
284
392
|
---
|
|
285
393
|
|
|
286
|
-
## Signed URLs (GET / PUT / Multipart)
|
|
394
|
+
## 🔗 Signed URLs (GET / PUT / Multipart)
|
|
287
395
|
|
|
288
396
|
> **Security:** signed URLs are temporary credentials. **Never log them** — a logged signed URL is accessible to anyone with log access.
|
|
289
397
|
|
|
@@ -343,7 +451,7 @@ const { uploadId, parts, key } = await signedUrls.getMultipartUploadUrls({
|
|
|
343
451
|
|
|
344
452
|
---
|
|
345
453
|
|
|
346
|
-
## Validation
|
|
454
|
+
## ✅ Validation
|
|
347
455
|
|
|
348
456
|
```typescript
|
|
349
457
|
import { Injectable } from '@nestjs/common'
|
|
@@ -352,7 +460,10 @@ import type { IUploadValidator } from '@bymax-one/nest-storage'
|
|
|
352
460
|
/** Magic-byte PDF check — reads the first 4 bytes of the stream. */
|
|
353
461
|
@Injectable()
|
|
354
462
|
export class PdfValidator implements IUploadValidator {
|
|
355
|
-
async validate(input: {
|
|
463
|
+
async validate(input: {
|
|
464
|
+
body: Buffer | NodeJS.ReadableStream
|
|
465
|
+
readBytes: (n: number) => Promise<Buffer>
|
|
466
|
+
}) {
|
|
356
467
|
const header = await input.readBytes(4)
|
|
357
468
|
if (header.toString('ascii', 0, 4) !== '%PDF') {
|
|
358
469
|
return { valid: false, reason: 'Not a valid PDF (magic bytes mismatch)' }
|
|
@@ -377,7 +488,7 @@ BymaxStorageModule.forRoot({
|
|
|
377
488
|
|
|
378
489
|
---
|
|
379
490
|
|
|
380
|
-
## Virus Scanning (`IFileScanner`)
|
|
491
|
+
## 🦠 Virus Scanning (`IFileScanner`)
|
|
381
492
|
|
|
382
493
|
```typescript
|
|
383
494
|
import type { IFileScanner, FileScanResult } from '@bymax-one/nest-storage'
|
|
@@ -405,7 +516,7 @@ BymaxStorageModule.forRoot({
|
|
|
405
516
|
|
|
406
517
|
---
|
|
407
518
|
|
|
408
|
-
## Lifecycle Operations
|
|
519
|
+
## ♻️ Lifecycle Operations
|
|
409
520
|
|
|
410
521
|
### List / Paginate
|
|
411
522
|
|
|
@@ -413,7 +524,11 @@ BymaxStorageModule.forRoot({
|
|
|
413
524
|
const page1 = await storage.list({ prefix: 'avatars/', maxKeys: 100 })
|
|
414
525
|
// { objects: [...], nextContinuationToken?: '...' }
|
|
415
526
|
if (page1.nextContinuationToken) {
|
|
416
|
-
const page2 = await storage.list({
|
|
527
|
+
const page2 = await storage.list({
|
|
528
|
+
prefix: 'avatars/',
|
|
529
|
+
maxKeys: 100,
|
|
530
|
+
continuationToken: page1.nextContinuationToken,
|
|
531
|
+
})
|
|
417
532
|
}
|
|
418
533
|
```
|
|
419
534
|
|
|
@@ -476,29 +591,30 @@ BymaxStorageModule.forRootAsync({
|
|
|
476
591
|
|
|
477
592
|
---
|
|
478
593
|
|
|
479
|
-
## Error Codes
|
|
594
|
+
## 🚨 Error Codes
|
|
480
595
|
|
|
481
596
|
All errors are thrown as `StorageException extends HttpException`. The response body is `{ error: { code, message, details? } }`.
|
|
482
597
|
|
|
483
|
-
| Code
|
|
484
|
-
|
|
485
|
-
| `STORAGE_NOT_CONFIGURED`
|
|
486
|
-
| `STORAGE_KEY_INVALID`
|
|
487
|
-
| `STORAGE_BODY_MISSING`
|
|
488
|
-
| `STORAGE_CONTENT_TYPE_REQUIRED`
|
|
489
|
-
| `STORAGE_MIME_NOT_ALLOWED`
|
|
490
|
-
| `STORAGE_SIZE_EXCEEDED`
|
|
491
|
-
| `STORAGE_VALIDATION_FAILED`
|
|
492
|
-
| `STORAGE_SCAN_INFECTED`
|
|
493
|
-
| `STORAGE_SCAN_INCONCLUSIVE`
|
|
494
|
-
| `STORAGE_OBJECT_NOT_FOUND`
|
|
495
|
-
| `STORAGE_PROVIDER_ERROR`
|
|
496
|
-
| `STORAGE_SIGNED_URL_TTL_INVALID` | 400
|
|
497
|
-
| `STORAGE_PART_TOO_SMALL`
|
|
498
|
-
| `
|
|
499
|
-
| `
|
|
500
|
-
| `
|
|
501
|
-
| `
|
|
598
|
+
| Code | HTTP | When |
|
|
599
|
+
| -------------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
600
|
+
| `STORAGE_NOT_CONFIGURED` | 503 | Credentials missing and an operation was called |
|
|
601
|
+
| `STORAGE_KEY_INVALID` | 400 | Path traversal (`..`), key starts with `/`, or empty after normalization |
|
|
602
|
+
| `STORAGE_BODY_MISSING` | 400 | `upload()` called without a body |
|
|
603
|
+
| `STORAGE_CONTENT_TYPE_REQUIRED` | 400 | `upload()` called without `contentType` |
|
|
604
|
+
| `STORAGE_MIME_NOT_ALLOWED` | 415 | `contentType` outside `mimeWhitelist` |
|
|
605
|
+
| `STORAGE_SIZE_EXCEEDED` | 413 | `size > maxSizeBytes` |
|
|
606
|
+
| `STORAGE_VALIDATION_FAILED` | 400 | `IUploadValidator` rejected the file (`details.reason`) |
|
|
607
|
+
| `STORAGE_SCAN_INFECTED` | 422 | Scanner returned `'infected'` (`details.threat`) |
|
|
608
|
+
| `STORAGE_SCAN_INCONCLUSIVE` | 422 | Scanner returned `'unknown'` and `rejectOnUnknown: true` |
|
|
609
|
+
| `STORAGE_OBJECT_NOT_FOUND` | 404 | `head()`, `download()`, or `copy()` on a nonexistent key |
|
|
610
|
+
| `STORAGE_PROVIDER_ERROR` | 502 | AWS SDK error (network, 5xx, throttling, `AccessControlListNotSupported`); `details` carries `awsCode`, `awsMessage`, `httpStatus`, `requestId` plus the operation context (`op`, `bucket`, and `key` or `prefix`) |
|
|
611
|
+
| `STORAGE_SIGNED_URL_TTL_INVALID` | 400 | Per-request `ttlSeconds` ≤ 0 |
|
|
612
|
+
| `STORAGE_PART_TOO_SMALL` | 400 | Multipart with part < 5 MiB (S3 limit) |
|
|
613
|
+
| `STORAGE_INVALID_PART_COUNT` | 400 | `getMultipartUploadUrls()` with `parts <= 0` (`details.provided`) |
|
|
614
|
+
| `STORAGE_BUCKET_UNDEFINED` | 400 | Operation without a bucket and no default in config |
|
|
615
|
+
| `STORAGE_MULTIPART_ABORTED` | 500 | Multipart upload failed and was aborted |
|
|
616
|
+
| `STORAGE_INVALID_CONFIG` | 500 | `BymaxStorageModuleOptions` validation failed at initialization |
|
|
617
|
+
| `STORAGE_TIMEOUT` | 504 | Request exceeded `requestTimeoutMs` |
|
|
502
618
|
|
|
503
619
|
```typescript
|
|
504
620
|
import { StorageException, STORAGE_ERROR_CODES } from '@bymax-one/nest-storage'
|
|
@@ -515,32 +631,235 @@ try {
|
|
|
515
631
|
|
|
516
632
|
---
|
|
517
633
|
|
|
518
|
-
##
|
|
634
|
+
## 📖 API Reference
|
|
519
635
|
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
636
|
+
Every operation below is documented with a runnable example in the sections above.
|
|
637
|
+
This is the index.
|
|
638
|
+
|
|
639
|
+
### `StorageService`
|
|
640
|
+
|
|
641
|
+
| Method | Signature | Notes |
|
|
642
|
+
| ---------------- | ------------------------------------------------------------- | ------------------------------------------------------------ |
|
|
643
|
+
| `upload` | `(options: UploadOptions) => Promise<UploadResult>` | Multipart via `@aws-sdk/lib-storage`; aborts on failure |
|
|
644
|
+
| `download` | `(options: DownloadOptions) => Promise<{ stream, metadata }>` | Streaming; the body is never buffered |
|
|
645
|
+
| `downloadBuffer` | `(options: DownloadOptions) => Promise<{ buffer, metadata }>` | Buffers — bounded by the caller's own memory |
|
|
646
|
+
| `head` | `(key, options?) => Promise<ObjectMetadata>` | Metadata without transferring the body |
|
|
647
|
+
| `exists` | `(key, options?) => Promise<boolean>` | `head` reduced to a boolean |
|
|
648
|
+
| `getPublicUrl` | `(key, options?) => string` | Composed, not signed — for buckets that are public by policy |
|
|
649
|
+
| `delete` | `(key, options?) => Promise<void>` | Idempotent, as S3 delete is |
|
|
650
|
+
| `deleteMany` | `(keys: string[], options?) => Promise<DeleteManyResult>` | Partial failures are reported, not thrown |
|
|
651
|
+
| `list` | `(options: ListOptions) => Promise<ListResult>` | Paginated by continuation token |
|
|
652
|
+
| `copy` | `(options: CopyOptions) => Promise<{ key, etag }>` | Server-side; the bytes never reach this process |
|
|
653
|
+
|
|
654
|
+
### `SignedUrlService`
|
|
655
|
+
|
|
656
|
+
| Method | Signature | Notes |
|
|
657
|
+
| ------------------------ | ----------------------------------------------------------------------------- | ------------------------------------------------------------- |
|
|
658
|
+
| `getDownloadUrl` | `(options: SignedGetUrlOptions) => Promise<SignedUrlResult>` | GET presign |
|
|
659
|
+
| `getUploadUrl` | `(options: SignedPutUrlOptions) => Promise<SignedUrlResult>` | PUT presign, with optional content-type and length conditions |
|
|
660
|
+
| `getMultipartUploadUrls` | `(options: MultipartUploadUrlsOptions) => Promise<MultipartUploadUrlsResult>` | One signed URL per part, plus the upload id |
|
|
661
|
+
|
|
662
|
+
### Key normalization
|
|
663
|
+
|
|
664
|
+
`KeyResolverService` is internal — it is not exported, and it is not something a
|
|
665
|
+
consumer calls. It is named here because every method above passes the key you
|
|
666
|
+
supply through it first: the guard below, then `keyPrefix`. The key you read back in
|
|
667
|
+
`UploadResult` and `ListedObject` is the resolved one.
|
|
668
|
+
|
|
669
|
+
### DI tokens
|
|
670
|
+
|
|
671
|
+
`BYMAX_STORAGE_OPTIONS` · `BYMAX_STORAGE_S3_CLIENT` · `BYMAX_STORAGE_FILE_SCANNER` ·
|
|
672
|
+
`BYMAX_STORAGE_UPLOAD_VALIDATORS` · `BYMAX_STORAGE_IDEMPOTENCY_CACHE` ·
|
|
673
|
+
`BYMAX_STORAGE_LOGGER` — all `Symbol()`, so no string token can collide with them.
|
|
674
|
+
|
|
675
|
+
---
|
|
676
|
+
|
|
677
|
+
## 🏗️ Architecture
|
|
678
|
+
|
|
679
|
+
```
|
|
680
|
+
BymaxStorageModule (@Global, forRoot / forRootAsync)
|
|
681
|
+
│
|
|
682
|
+
├── validate-options ──────── refuses a malformed configuration at bootstrap.
|
|
683
|
+
│ Credentials are the deliberate exception: empty
|
|
684
|
+
│ values are tolerated so a dev workflow boots without
|
|
685
|
+
│ storage, and operations then fail with
|
|
686
|
+
│ STORAGE_NOT_CONFIGURED rather than at module load
|
|
687
|
+
│
|
|
688
|
+
├── S3ClientProvider ───────── one @aws-sdk/client-s3 S3Client, built from the
|
|
689
|
+
│ resolved provider recipe; exposed as
|
|
690
|
+
│ BYMAX_STORAGE_S3_CLIENT for anything this surface
|
|
691
|
+
│ deliberately does not wrap
|
|
692
|
+
│
|
|
693
|
+
├── KeyResolverService ─────── the only place a caller-supplied key becomes an
|
|
694
|
+
│ object key: guard, then keyPrefix
|
|
695
|
+
│
|
|
696
|
+
├── StorageService ─────────── upload · download · head · exists · list · copy ·
|
|
697
|
+
│ delete · deleteMany
|
|
698
|
+
│ ├── IUploadValidator[] (MIME + size, then any you register)
|
|
699
|
+
│ ├── IFileScanner (pre- and/or post-upload)
|
|
700
|
+
│ └── IdempotencyCache (LRU, in-process)
|
|
701
|
+
│
|
|
702
|
+
└── SignedUrlService ───────── GET / PUT / multipart presigns, TTL-clamped
|
|
703
|
+
```
|
|
704
|
+
|
|
705
|
+
One engine, six recipes. `providerRecipes` differ only in endpoint, region and the
|
|
706
|
+
handful of flags each provider needs (`forcePathStyle` for MinIO, `auto` region for
|
|
707
|
+
R2) — there is no per-provider code path, so a provider swap is a configuration
|
|
708
|
+
change and every operation behaves the same way afterwards.
|
|
709
|
+
|
|
710
|
+
The idempotency cache is **in-process**. It collapses a retried `upload` inside one
|
|
711
|
+
instance; it does not coordinate between replicas.
|
|
712
|
+
|
|
713
|
+
### Design Principles
|
|
523
714
|
|
|
524
|
-
|
|
525
|
-
|
|
715
|
+
| Principle | Description |
|
|
716
|
+
| -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
717
|
+
| 🔑 **One key chokepoint** | Every caller-supplied key passes through `KeyResolverService` before it reaches S3. `..`, a leading `/`, null bytes and empty keys are refused, and `keyPrefix` is prepended after normalization — so it cannot be escaped by the key |
|
|
718
|
+
| 🧩 **One engine, six recipes** | `providerRecipes` differ in endpoint, region and per-provider flags only. There is no per-provider code path, so a provider swap is configuration and every operation behaves the same afterwards |
|
|
719
|
+
| 🚫 **Refuse before storing** | MIME allowlist and size limit, then registered `IUploadValidator`s, then the `IFileScanner`. An upload that will be rejected is rejected before a byte is written |
|
|
720
|
+
| ⏱️ **Bounded signatures** | Presign TTLs are clamped and SigV4's 604 800 s ceiling is enforced locally, not discovered when the provider rejects the signature |
|
|
721
|
+
| 💥 **Fail at boot, mostly** | Options are validated before any client is built. Credentials are the deliberate exception: empty values are tolerated so a development workflow boots, and operations then fail with `STORAGE_NOT_CONFIGURED` |
|
|
722
|
+
| 🧊 **Zero runtime dependencies** | `dependencies` is `{}`. You choose the exact `@aws-sdk/*` versions and the supply-chain surface stays yours |
|
|
526
723
|
|
|
527
|
-
|
|
528
|
-
|
|
724
|
+
---
|
|
725
|
+
|
|
726
|
+
## 🔐 Security Model
|
|
727
|
+
|
|
728
|
+
This library turns caller input into object keys, signs URLs that grant access on their
|
|
729
|
+
own, and holds the credentials for a bucket. Its security contract is about what a key can
|
|
730
|
+
reach, how long a signature lives, and what leaves in an error.
|
|
731
|
+
|
|
732
|
+
### The object key is the attack surface
|
|
733
|
+
|
|
734
|
+
Everything a caller sends becomes part of a key, and a key is a path.
|
|
735
|
+
`KeyResolverService.normalize` is the single chokepoint: it refuses an empty key, a key
|
|
736
|
+
containing a null byte, a key starting with `/`, and any key with a `..` segment; it
|
|
737
|
+
collapses duplicate slashes; and only then does it prepend `keyPrefix`. Nothing in this
|
|
738
|
+
library composes a key any other way, so a tenant cannot climb out of its prefix by naming
|
|
739
|
+
one.
|
|
740
|
+
|
|
741
|
+
### Presigned URLs are bearer credentials
|
|
742
|
+
|
|
743
|
+
Anyone holding one has the access it encodes, for as long as it lives. TTLs are clamped,
|
|
744
|
+
and SigV4's hard ceiling of 604 800 s (7 days) is enforced here rather than passed through
|
|
745
|
+
to a signature the provider would reject at use time. Treat a signed URL like a password
|
|
746
|
+
with an expiry, not like a link.
|
|
747
|
+
|
|
748
|
+
### Credentials stay where they were put
|
|
749
|
+
|
|
750
|
+
They arrive through module options and are handed to the SDK. This library never reads
|
|
751
|
+
`process.env`, never logs them, and never places them — or a signed URL — in an exception.
|
|
752
|
+
|
|
753
|
+
### Error payloads carry the operation, and that includes the key
|
|
529
754
|
|
|
530
|
-
|
|
531
|
-
|
|
755
|
+
What `aws-error-mapper` puts in `details` is the provider's error code and message, the
|
|
756
|
+
HTTP status, the request id, and the operation context the call site supplies: which
|
|
757
|
+
operation, which bucket, and the resolved key or prefix. That is enough to diagnose a
|
|
758
|
+
failure, and it means an object key reaches whatever consumes the exception — so if a key
|
|
759
|
+
is itself sensitive in your deployment, do not log the envelope verbatim.
|
|
760
|
+
|
|
761
|
+
### Uploads are refused before they are stored
|
|
762
|
+
|
|
763
|
+
MIME allowlist (wildcards included) and size limit run first, then any `IUploadValidator`
|
|
764
|
+
you register, then the `IFileScanner` if configured. A scanner can run pre-upload,
|
|
765
|
+
post-upload, or both — the post-upload position exists because some scanners only accept an
|
|
766
|
+
object they can fetch.
|
|
767
|
+
|
|
768
|
+
### `keyPrefix` is a namespace, not an access boundary
|
|
769
|
+
|
|
770
|
+
It scopes the keys this library composes; it does not restrict what the credentials can
|
|
771
|
+
reach. Anything holding the bucket's credentials can read every prefix in it. Isolation
|
|
772
|
+
that must survive a compromised client belongs in IAM policies or separate buckets.
|
|
773
|
+
|
|
774
|
+
---
|
|
775
|
+
|
|
776
|
+
## 🛡️ Security Table
|
|
777
|
+
|
|
778
|
+
| Layer | Implementation |
|
|
779
|
+
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
780
|
+
| Object keys | Guarded and normalized in one place; `..`, leading `/`, null bytes and empty keys refused |
|
|
781
|
+
| Multi-tenancy | `keyPrefix` prepended after normalization, so it cannot be escaped by the key |
|
|
782
|
+
| Credentials | Injected options only; never read from `process.env`, never logged, never in an exception |
|
|
783
|
+
| Error payloads | Provider code and message, HTTP status, request id, and the operation context (`op`, `bucket`, `key`/`prefix`) — never credentials, never a signed URL |
|
|
784
|
+
| Presign lifetime | TTL clamped, SigV4's 7-day ceiling enforced locally |
|
|
785
|
+
| Encryption | Server-side (AES256 / `aws:kms`) configurable globally or per upload |
|
|
786
|
+
| Content | MIME allowlist + size limit, then registered validators, then the scanner |
|
|
787
|
+
| Supply chain | `dependencies: {}`; third-party Actions pinned by commit SHA (org-internal reusables by tag); CodeQL, OSV-Scanner and OpenSSF Scorecard |
|
|
788
|
+
|
|
789
|
+
> [!IMPORTANT]
|
|
790
|
+
> **`keyPrefix` is not an access boundary.** It scopes the keys this library composes;
|
|
791
|
+
> it does not restrict what the credentials can reach. Anything holding the bucket's
|
|
792
|
+
> credentials can read every prefix in it. Enforce tenant isolation with IAM policies
|
|
793
|
+
> or separate buckets when the boundary has to hold against the application itself.
|
|
794
|
+
|
|
795
|
+
---
|
|
796
|
+
|
|
797
|
+
## 🧱 Tech Stack
|
|
798
|
+
|
|
799
|
+
- **Runtime:** Node.js 24+
|
|
800
|
+
- **Framework:** NestJS 11 (`ConfigurableModuleBuilder`, `@Global()`, `Symbol()` tokens)
|
|
801
|
+
- **Storage engine:** `@aws-sdk/client-s3 ^3.700` (peer), with `@aws-sdk/lib-storage`
|
|
802
|
+
for multipart and `@aws-sdk/s3-request-presigner` for presigns — all peers
|
|
803
|
+
- **Providers:** AWS S3, Cloudflare R2, Backblaze B2, DigitalOcean Spaces, MinIO, Wasabi
|
|
804
|
+
- **Build:** tsup — ESM + CJS per subpath, with `.d.ts` _and_ `.d.cts` declarations
|
|
805
|
+
- **Tests:** Jest + Testcontainers (MinIO, E2E) + Stryker (mutation)
|
|
806
|
+
- **TypeScript:** 5.x strict (`noUncheckedIndexedAccess`, `exactOptionalPropertyTypes`), zero `any`
|
|
807
|
+
|
|
808
|
+
---
|
|
809
|
+
|
|
810
|
+
## 🧪 Testing & Quality
|
|
811
|
+
|
|
812
|
+
Object storage is where a bug is expensive to discover later — a key written to the wrong
|
|
813
|
+
place stays there — so the suite is held to a bar beyond "the tests pass".
|
|
814
|
+
|
|
815
|
+
- ✅ **100% line coverage** — statements, branches, functions and lines, enforced as a gate
|
|
816
|
+
- ✅ **100% mutation score** — verified with [Stryker](https://stryker-mutator.io/) at
|
|
817
|
+
`break: 95`; every survivor was killed by a strengthened assertion, and the eight provable
|
|
818
|
+
equivalents are documented inline
|
|
819
|
+
- ✅ **Real object storage in e2e** — MinIO through Testcontainers, so multipart, presigning
|
|
820
|
+
and listing are exercised against an actual S3 API rather than a mock
|
|
821
|
+
- ✅ **Published-artifact gates** — `check:exports` resolves the types the way each module
|
|
822
|
+
system does, `check:runtime` loads every subpath from the packed tarball in ESM and
|
|
823
|
+
CommonJS, and `check:published` compiles this README's snippets against `dist/`
|
|
824
|
+
- ✅ **Every suppression is justified** — no coverage directives anywhere; the five
|
|
825
|
+
`// Stryker disable` comments in the production source each name why the mutant they
|
|
826
|
+
silence is provably equivalent, and the mutation report lists them
|
|
827
|
+
|
|
828
|
+
```bash
|
|
829
|
+
pnpm test # unit tests (Jest)
|
|
830
|
+
pnpm test:cov # unit tests with the 100% coverage gate
|
|
831
|
+
pnpm test:e2e # end-to-end against MinIO (requires Docker)
|
|
832
|
+
pnpm mutation # Stryker mutation testing (break: 95)
|
|
833
|
+
pnpm typecheck # tsc strict check
|
|
834
|
+
pnpm lint # ESLint
|
|
532
835
|
```
|
|
533
836
|
|
|
534
837
|
---
|
|
535
838
|
|
|
536
|
-
## Contributing
|
|
839
|
+
## 🤝 Contributing
|
|
537
840
|
|
|
538
|
-
|
|
841
|
+
Pull requests are welcome. Please open an issue first for significant changes.
|
|
539
842
|
|
|
540
|
-
|
|
843
|
+
- Read [`docs/technical_specification.md`](docs/technical_specification.md) for architecture decisions.
|
|
844
|
+
- Run `pnpm test:cov` and `pnpm lint` before opening a PR.
|
|
845
|
+
- Please use Conventional Commits for the message; nothing enforces it here, so it is a convention rather than a gate.
|
|
541
846
|
|
|
542
847
|
---
|
|
543
848
|
|
|
544
|
-
##
|
|
849
|
+
## 🔒 Security Policy
|
|
545
850
|
|
|
546
|
-
|
|
851
|
+
If you discover a security vulnerability, please **do not** open a public issue. Instead, email us
|
|
852
|
+
at **support@bymax.one** with details. We take security seriously and will respond promptly. See
|
|
853
|
+
[`SECURITY.md`](SECURITY.md) for the full policy, including the storage-specific security goals.
|
|
854
|
+
|
|
855
|
+
---
|
|
856
|
+
|
|
857
|
+
## 📄 License
|
|
858
|
+
|
|
859
|
+
[MIT](./LICENSE) © [Bymax One](https://github.com/bymaxone)
|
|
860
|
+
|
|
861
|
+
---
|
|
862
|
+
|
|
863
|
+
<p align="center">
|
|
864
|
+
<sub>Built with ❤️ by <a href="https://github.com/bymaxone">Bymax One</a></sub>
|
|
865
|
+
</p>
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@bymax-one/nest-storage",
|
|
3
|
-
"version": "1.0.
|
|
3
|
+
"version": "1.0.2",
|
|
4
4
|
"description": "Provider-agnostic S3-compatible object storage for NestJS. Works with AWS S3, DigitalOcean Spaces, Cloudflare R2, Backblaze B2, MinIO, Wasabi.",
|
|
5
5
|
"author": "Bymax One <support@bymax.one>",
|
|
6
6
|
"license": "MIT",
|
|
@@ -43,6 +43,28 @@
|
|
|
43
43
|
},
|
|
44
44
|
"./package.json": "./package.json"
|
|
45
45
|
},
|
|
46
|
+
"scripts": {
|
|
47
|
+
"build": "pnpm clean && tsup",
|
|
48
|
+
"check:exports": "attw --pack . --profile strict",
|
|
49
|
+
"check:published": "node scripts/check-published-surface.mjs",
|
|
50
|
+
"check:runtime": "node scripts/check-consumer-runtime.mjs",
|
|
51
|
+
"clean": "rm -rf dist coverage",
|
|
52
|
+
"lint": "eslint src",
|
|
53
|
+
"lint:fix": "eslint src --fix",
|
|
54
|
+
"mutation": "stryker run",
|
|
55
|
+
"mutation:dry-run": "stryker run --dryRunOnly",
|
|
56
|
+
"mutation:incremental": "stryker run --incremental",
|
|
57
|
+
"prepublishOnly": "pnpm clean && pnpm typecheck && pnpm lint && pnpm test:cov:all && pnpm build && pnpm size && pnpm check:published",
|
|
58
|
+
"release": "npm publish --provenance --access public",
|
|
59
|
+
"size": "node scripts/check-size.mjs",
|
|
60
|
+
"test": "jest",
|
|
61
|
+
"test:all": "pnpm test && pnpm test:e2e",
|
|
62
|
+
"test:cov": "jest --coverage",
|
|
63
|
+
"test:cov:all": "jest --config jest.coverage.config.ts --coverage",
|
|
64
|
+
"test:e2e": "jest --config jest.e2e.config.ts",
|
|
65
|
+
"test:watch": "jest --watch",
|
|
66
|
+
"typecheck": "tsc --noEmit && tsc --noEmit -p tsconfig.server.json"
|
|
67
|
+
},
|
|
46
68
|
"dependencies": {},
|
|
47
69
|
"peerDependencies": {
|
|
48
70
|
"@nestjs/common": "^11.0.16",
|
|
@@ -103,6 +125,20 @@
|
|
|
103
125
|
"typescript": "^5.9.3",
|
|
104
126
|
"typescript-eslint": "^8.65.0"
|
|
105
127
|
},
|
|
128
|
+
"pnpm": {
|
|
129
|
+
"overrides": {
|
|
130
|
+
"brace-expansion@1": "^1.1.17",
|
|
131
|
+
"brace-expansion@2": "^2.1.3",
|
|
132
|
+
"brace-expansion@5": "^5.0.8",
|
|
133
|
+
"esbuild": "0.28.1",
|
|
134
|
+
"fast-uri": "^3.1.4",
|
|
135
|
+
"multer": "2.2.0",
|
|
136
|
+
"protobufjs": "^7.6.5",
|
|
137
|
+
"qs": "6.15.2",
|
|
138
|
+
"undici": "^8.9.0",
|
|
139
|
+
"uuid": "11.1.1"
|
|
140
|
+
}
|
|
141
|
+
},
|
|
106
142
|
"keywords": [
|
|
107
143
|
"nestjs",
|
|
108
144
|
"storage",
|
|
@@ -117,6 +153,7 @@
|
|
|
117
153
|
"presigned-url",
|
|
118
154
|
"multipart-upload"
|
|
119
155
|
],
|
|
156
|
+
"packageManager": "pnpm@10.8.1",
|
|
120
157
|
"engines": {
|
|
121
158
|
"node": ">=24.0.0"
|
|
122
159
|
},
|
|
@@ -134,26 +171,5 @@
|
|
|
134
171
|
"./dist/shared/index.d.cts"
|
|
135
172
|
]
|
|
136
173
|
}
|
|
137
|
-
},
|
|
138
|
-
"scripts": {
|
|
139
|
-
"build": "pnpm clean && tsup",
|
|
140
|
-
"check:exports": "attw --pack . --profile strict",
|
|
141
|
-
"check:published": "node scripts/check-published-surface.mjs",
|
|
142
|
-
"check:runtime": "node scripts/check-consumer-runtime.mjs",
|
|
143
|
-
"clean": "rm -rf dist coverage",
|
|
144
|
-
"lint": "eslint src",
|
|
145
|
-
"lint:fix": "eslint src --fix",
|
|
146
|
-
"mutation": "stryker run",
|
|
147
|
-
"mutation:dry-run": "stryker run --dryRunOnly",
|
|
148
|
-
"mutation:incremental": "stryker run --incremental",
|
|
149
|
-
"release": "npm publish --provenance --access public",
|
|
150
|
-
"size": "node scripts/check-size.mjs",
|
|
151
|
-
"test": "jest",
|
|
152
|
-
"test:all": "pnpm test && pnpm test:e2e",
|
|
153
|
-
"test:cov": "jest --coverage",
|
|
154
|
-
"test:cov:all": "jest --config jest.coverage.config.ts --coverage",
|
|
155
|
-
"test:e2e": "jest --config jest.e2e.config.ts",
|
|
156
|
-
"test:watch": "jest --watch",
|
|
157
|
-
"typecheck": "tsc --noEmit && tsc --noEmit -p tsconfig.server.json"
|
|
158
174
|
}
|
|
159
|
-
}
|
|
175
|
+
}
|