@bymax-one/nest-storage 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +99 -0
- package/LICENSE +21 -0
- package/README.md +546 -0
- package/dist/server/index.cjs +1848 -0
- package/dist/server/index.d.cts +1210 -0
- package/dist/server/index.d.ts +1210 -0
- package/dist/server/index.mjs +1836 -0
- package/dist/shared/index.cjs +63 -0
- package/dist/shared/index.d.cts +143 -0
- package/dist/shared/index.d.ts +143 -0
- package/dist/shared/index.mjs +53 -0
- package/package.json +159 -0
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to `@bymax-one/nest-storage` are documented in this file.
|
|
4
|
+
|
|
5
|
+
The format follows [Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/) and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## [Unreleased]
|
|
10
|
+
|
|
11
|
+
## [1.0.0] - 2026-08-03
|
|
12
|
+
|
|
13
|
+
First published release. Everything below ships in it.
|
|
14
|
+
|
|
15
|
+
The `Fixed` and `Security` entries record defects found and corrected before
|
|
16
|
+
publication, not regressions any consumer saw — there is no earlier release to
|
|
17
|
+
have regressed from. They are kept because the reasoning is worth having.
|
|
18
|
+
|
|
19
|
+
### Added
|
|
20
|
+
|
|
21
|
+
- **Initial release** of `@bymax-one/nest-storage` — provider-agnostic S3-compatible object storage for NestJS.
|
|
22
|
+
- **Single `@aws-sdk/client-s3` engine** — works with AWS S3, Cloudflare R2, Backblaze B2, DigitalOcean Spaces, MinIO, and Wasabi.
|
|
23
|
+
- **`StorageService`** — full object lifecycle:
|
|
24
|
+
- `upload()` — single-shot PutObject or automatic multipart via `@aws-sdk/lib-storage` (with `leavePartsOnError: false` for automatic abort on failure), `onProgress` events, `idempotencyKey` cache
|
|
25
|
+
- `download()` — streaming `Readable` with metadata
|
|
26
|
+
- `downloadBuffer()` — buffered download (for files < 10 MiB)
|
|
27
|
+
- `head()` — object metadata without downloading
|
|
28
|
+
- `exists()` — non-throwing existence check
|
|
29
|
+
- `delete()` — idempotent (no error on 404)
|
|
30
|
+
- `deleteMany()` — batched (chunked ≤ 1 000 keys per S3 API call); returns `{ deleted, failed }`
|
|
31
|
+
- `list()` — paginated listing via `ContinuationToken`
|
|
32
|
+
- `copy()` — server-side copy (same or cross-bucket)
|
|
33
|
+
- `getPublicUrl()` — unsigned public URL (does not validate ACL or existence)
|
|
34
|
+
- **`SignedUrlService`** — presigned URL generation:
|
|
35
|
+
- `getDownloadUrl()` — presigned GET with optional response headers
|
|
36
|
+
- `getUploadUrl()` — presigned PUT with Content-Length-Range policy
|
|
37
|
+
- `getMultipartUploadUrls()` — presigned multipart (InitiateMultipartUpload + per-part URLs)
|
|
38
|
+
- TTL clamped to `signedUrls.maxTtlSeconds`; `maxTtlSeconds` clamped to 604 800 s (7-day SigV4 ceiling) at init
|
|
39
|
+
- **`IUploadValidator` hook** — pluggable pre-upload validator (MIME whitelist with wildcards, size limit, custom `readBytes` magic-byte checks); `NoOpUploadValidator` included
|
|
40
|
+
- **`IFileScanner` hook** — pluggable virus-scan integration (`pre-upload` / `post-upload` / `both`; `rejectOnUnknown` policy); `NoOpFileScanner` included
|
|
41
|
+
- **Six provider recipes** (`providerRecipes`):
|
|
42
|
+
- `awsS3` — AWS S3 with SSE-AES256; SDK default checksum mode
|
|
43
|
+
- `cloudflareR2` — region `'auto'`; `customDomain` required; checksums `'WHEN_REQUIRED'`
|
|
44
|
+
- `backblazeB2` — virtual-hosted; checksums `'WHEN_REQUIRED'`
|
|
45
|
+
- `digitalOceanSpaces` — with CDN base URL; checksums `'WHEN_REQUIRED'`
|
|
46
|
+
- `minio` — `forcePathStyle: true`; checksums `'WHEN_REQUIRED'`
|
|
47
|
+
- `wasabi` — virtual-hosted; checksums `'WHEN_REQUIRED'`
|
|
48
|
+
- **17-code `StorageException` catalog** — typed errors extending `HttpException` with `{ error: { code, message, details? } }` response body; `STORAGE_ERROR_CODES` exported from the `./shared` subpath
|
|
49
|
+
- **`keyPrefix` multi-tenant isolation** — prepended to every resolved key; enforced by `KeyResolverService`
|
|
50
|
+
- **Mandatory path-traversal guard** — `KeyResolverService` blocks `..`, leading `/`, and empty-after-normalize keys (`STORAGE_KEY_INVALID` / HTTP 400)
|
|
51
|
+
- **In-memory LRU idempotency cache** — default 1 000 entries / 24 h; `idempotencyKey` per-upload
|
|
52
|
+
- **Server-side encryption** — `serverSideEncryption: 'AES256'` or `'aws:kms'` globally or per-upload; `'NONE'` sentinel omits the header
|
|
53
|
+
- **Subpath exports**:
|
|
54
|
+
- `.` — server runtime (NestJS module, services, provider recipes, DI tokens, interfaces)
|
|
55
|
+
- `./shared` — framework-free types and `STORAGE_ERROR_CODES`
|
|
56
|
+
- **`forRoot` / `forRootAsync`** dynamic module API (`@Global()`; `Symbol()` DI tokens)
|
|
57
|
+
- **`BYMAX_STORAGE_S3_CLIENT` DI token** — raw `S3Client` injection for provider-specific advanced operations (spec §11.2)
|
|
58
|
+
- **Non-AWS checksum opt-out** — the five non-AWS recipes set `requestChecksumCalculation` / `responseChecksumValidation` to `'WHEN_REQUIRED'` to prevent the SDK's default CRC32 `x-amz-checksum-*` headers from being sent to providers that reject them
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
- **`pnpm check:exports`** runs `attw --pack . --profile strict` against the packed
|
|
62
|
+
tarball. Its absence is why both defects above went unnoticed: a source-level
|
|
63
|
+
typecheck compiles `src` and never resolves through the `exports` map.
|
|
64
|
+
- **`pnpm check:runtime`** packs the tarball, lays it out the way npm would, and
|
|
65
|
+
loads every subpath from it in ESM *and* CommonJS, asserting the expected values
|
|
66
|
+
are really exported. `attw` proves the declarations resolve; it never runs the
|
|
67
|
+
JavaScript. Both gates run in CI.
|
|
68
|
+
|
|
69
|
+
### Fixed
|
|
70
|
+
|
|
71
|
+
- **CommonJS consumers resolved ESM type declarations.** The `exports` map
|
|
72
|
+
declared a single `types` condition, so `require()` landed on `.d.ts` instead of
|
|
73
|
+
`.d.cts` — `attw --profile strict` reports it as *Masquerading as ESM* on every
|
|
74
|
+
subpath. Types are now declared per condition.
|
|
75
|
+
|
|
76
|
+
- **`node10` type resolution failed outright**: the manifest carried no complete
|
|
77
|
+
set of `main`, `module`, `types` and `typesVersions`. All four are now present.
|
|
78
|
+
|
|
79
|
+
### Security
|
|
80
|
+
|
|
81
|
+
- **Peer floors raised to exclude known-vulnerable NestJS versions.** The declared
|
|
82
|
+
ranges were `@nestjs/common ^11.0.0` and `@nestjs/core ^11.0.0`, and both
|
|
83
|
+
admitted versions carrying published advisories:
|
|
84
|
+
|
|
85
|
+
| Peer | Advisory | Vulnerable | New floor |
|
|
86
|
+
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- | ---------- |
|
|
87
|
+
| `@nestjs/common` | [GHSA-cj7v-w2c7-cp7c](https://github.com/advisories/GHSA-cj7v-w2c7-cp7c) — remote code execution via the `Content-Type` header | `>= 11.0.0-next.1, < 11.0.16` | `^11.0.16` |
|
|
88
|
+
| `@nestjs/core` | [GHSA-36xv-jgw5-4q75](https://github.com/advisories/GHSA-36xv-jgw5-4q75) — improper neutralization of special elements in downstream output | `<= 11.1.17` | `^11.1.18` |
|
|
89
|
+
|
|
90
|
+
A peer range is a statement about which versions this library supports. A floor
|
|
91
|
+
below a published advisory tells a consumer that a vulnerable install is a
|
|
92
|
+
supported one, and nothing in their tooling contradicts it — the install resolves
|
|
93
|
+
cleanly and silently. Corrected before the first publish, so no released version
|
|
94
|
+
ever carried the permissive range. No runtime behaviour changed.
|
|
95
|
+
|
|
96
|
+
---
|
|
97
|
+
|
|
98
|
+
[1.0.0]: https://github.com/bymaxone/nest-storage/releases/tag/v1.0.0
|
|
99
|
+
[Unreleased]: https://github.com/bymaxone/nest-storage/compare/v1.0.0...HEAD
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Bymax One
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,546 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<a href="https://www.npmjs.com/package/@bymax-one/nest-storage"><img src="https://img.shields.io/npm/v/@bymax-one/nest-storage.svg" alt="npm version" /></a>
|
|
3
|
+
<a href="https://www.npmjs.com/package/@bymax-one/nest-storage"><img src="https://img.shields.io/npm/dm/@bymax-one/nest-storage.svg" alt="npm downloads" /></a>
|
|
4
|
+
<a href="https://github.com/bymaxone/nest-storage/actions/workflows/ci.yml"><img src="https://github.com/bymaxone/nest-storage/actions/workflows/ci.yml/badge.svg" alt="CI" /></a>
|
|
5
|
+
<a href="https://codecov.io/gh/bymaxone/nest-storage"><img src="https://codecov.io/gh/bymaxone/nest-storage/branch/main/graph/badge.svg" alt="coverage" /></a>
|
|
6
|
+
<a href="https://stryker-mutator.io"><img src="https://img.shields.io/badge/mutation%20score-100%25-brightgreen" alt="mutation score" /></a>
|
|
7
|
+
<a href="https://api.securityscorecards.dev/projects/github.com/bymaxone/nest-storage"><img src="https://api.securityscorecards.dev/projects/github.com/bymaxone/nest-storage/badge" alt="OpenSSF Scorecard" /></a>
|
|
8
|
+
<a href="LICENSE"><img src="https://img.shields.io/npm/l/@bymax-one/nest-storage.svg" alt="license" /></a>
|
|
9
|
+
<img src="https://img.shields.io/badge/TypeScript-5.x-blue.svg" alt="TypeScript" />
|
|
10
|
+
<img src="https://img.shields.io/badge/Node-%3E%3D24-green.svg" alt="Node 24+" />
|
|
11
|
+
</p>
|
|
12
|
+
|
|
13
|
+
<h1 align="center">@bymax-one/nest-storage</h1>
|
|
14
|
+
|
|
15
|
+
<p align="center">
|
|
16
|
+
Provider-agnostic S3-compatible object storage for NestJS.<br/>
|
|
17
|
+
Single <code>@aws-sdk/client-s3</code> engine — works with AWS S3, Cloudflare R2, Backblaze B2,<br/>
|
|
18
|
+
DigitalOcean Spaces, MinIO, and Wasabi with zero runtime dependencies.
|
|
19
|
+
</p>
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Overview
|
|
24
|
+
|
|
25
|
+
`@bymax-one/nest-storage` is a NestJS dynamic module that wraps the AWS SDK v3 (`@aws-sdk/client-s3`) to provide a unified, provider-agnostic API for any S3-compatible object storage service. A single `StorageService` covers the full lifecycle — upload, download, signed URLs, listing, copy, and batch delete — while the pluggable `IUploadValidator` and `IFileScanner` hooks let you enforce your own MIME rules and virus-scan policies without coupling the library to any specific scanner.
|
|
26
|
+
|
|
27
|
+
## Features
|
|
28
|
+
|
|
29
|
+
- **Multipart upload** via `@aws-sdk/lib-storage` with automatic abort on failure (`leavePartsOnError: false`), progress events, and a configurable part-size threshold
|
|
30
|
+
- **Presigned GET / PUT / multipart URLs** with TTL clamping and a hard 7-day SigV4 ceiling
|
|
31
|
+
- **MIME whitelist + size limit** (wildcards supported: `image/*`, `text/*`) with a pluggable `IUploadValidator` hook for magic-byte checks and custom business rules
|
|
32
|
+
- **Pluggable `IFileScanner` hook** (pre- and/or post-upload) for ClamAV, AWS Macie, or any scanner that returns `'clean' | 'infected' | 'unknown'`
|
|
33
|
+
- **Six ready-to-use provider recipes** — `awsS3`, `cloudflareR2`, `backblazeB2`, `digitalOceanSpaces`, `minio`, `wasabi`
|
|
34
|
+
- **Path-traversal guard** in `KeyResolverService` — blocks `..`, leading `/`, and empty-after-normalize keys
|
|
35
|
+
- **In-memory LRU idempotency cache** (default 1 000 entries / 24 h) — identical `idempotencyKey` returns the cached `UploadResult`
|
|
36
|
+
- **Server-side encryption** (AES256 / aws:kms) configurable globally or per-upload
|
|
37
|
+
- **`keyPrefix` multi-tenant isolation** — prepended to every resolved key
|
|
38
|
+
- **`forRoot` / `forRootAsync`** dynamic module API; `@Global()` scope; `Symbol()` DI tokens
|
|
39
|
+
- **`BYMAX_STORAGE_S3_CLIENT` token** for injecting the raw `S3Client` for provider-specific operations
|
|
40
|
+
|
|
41
|
+
## Subpath Exports
|
|
42
|
+
|
|
43
|
+
| Subpath | Contents |
|
|
44
|
+
|---|---|
|
|
45
|
+
| `.` | Server runtime: `BymaxStorageModule`, `StorageService`, `SignedUrlService`, `providerRecipes`, DI tokens, interfaces, `StorageException`, `NoOpUploadValidator`, `NoOpFileScanner` |
|
|
46
|
+
| `./shared` | Framework-free types (`UploadResult`, `ObjectMetadata`, `ListedObject`, `SignedUrlResult`) + `STORAGE_ERROR_CODES` + `StorageErrorCode` |
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## Quick Start
|
|
51
|
+
|
|
52
|
+
### 1 — AWS S3
|
|
53
|
+
|
|
54
|
+
```typescript
|
|
55
|
+
import { Module } from '@nestjs/common'
|
|
56
|
+
import { BymaxStorageModule, providerRecipes } from '@bymax-one/nest-storage'
|
|
57
|
+
|
|
58
|
+
@Module({
|
|
59
|
+
imports: [
|
|
60
|
+
BymaxStorageModule.forRoot({
|
|
61
|
+
...providerRecipes.awsS3({
|
|
62
|
+
region: 'us-east-1',
|
|
63
|
+
bucket: 'my-bucket',
|
|
64
|
+
accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
|
|
65
|
+
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
|
|
66
|
+
}),
|
|
67
|
+
// AWS keeps the SDK default checksum behaviour — no opt-out needed.
|
|
68
|
+
}),
|
|
69
|
+
],
|
|
70
|
+
})
|
|
71
|
+
export class AppModule {}
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
```typescript
|
|
75
|
+
import { Injectable } from '@nestjs/common'
|
|
76
|
+
import { StorageService } from '@bymax-one/nest-storage'
|
|
77
|
+
|
|
78
|
+
@Injectable()
|
|
79
|
+
export class AssetService {
|
|
80
|
+
constructor(private readonly storage: StorageService) {}
|
|
81
|
+
|
|
82
|
+
async uploadFile(key: string, buffer: Buffer, contentType: string) {
|
|
83
|
+
return this.storage.upload({ key, body: buffer, contentType, size: buffer.length })
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
### 2 — Cloudflare R2
|
|
89
|
+
|
|
90
|
+
```typescript
|
|
91
|
+
import { BymaxStorageModule, providerRecipes } from '@bymax-one/nest-storage'
|
|
92
|
+
|
|
93
|
+
BymaxStorageModule.forRoot({
|
|
94
|
+
...providerRecipes.cloudflareR2({
|
|
95
|
+
accountId: process.env.R2_ACCOUNT_ID!,
|
|
96
|
+
bucket: 'my-bucket',
|
|
97
|
+
accessKeyId: process.env.R2_ACCESS_KEY_ID!,
|
|
98
|
+
secretAccessKey: process.env.R2_SECRET_ACCESS_KEY!,
|
|
99
|
+
// Required — the *.r2.cloudflarestorage.com API host does not serve public reads.
|
|
100
|
+
// Configure a Custom Domain in the R2 dashboard and supply it here.
|
|
101
|
+
customDomain: 'https://cdn.example.com',
|
|
102
|
+
}),
|
|
103
|
+
})
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
> **Note:** `customDomain` is the `publicBaseUrl` for R2 and is **required** — there is no working default because the S3 API endpoint (`*.r2.cloudflarestorage.com`) does not serve public object reads. The recipe also sets `requestChecksumCalculation`/`responseChecksumValidation` to `'WHEN_REQUIRED'` to prevent the SDK's default CRC32 headers from being sent (R2 rejects them — see [Provider Compatibility](#provider-compatibility--the-1-trap) below).
|
|
107
|
+
|
|
108
|
+
### 3 — DigitalOcean Spaces with CDN
|
|
109
|
+
|
|
110
|
+
```typescript
|
|
111
|
+
import { BymaxStorageModule, providerRecipes } from '@bymax-one/nest-storage'
|
|
112
|
+
|
|
113
|
+
BymaxStorageModule.forRoot({
|
|
114
|
+
...providerRecipes.digitalOceanSpaces({
|
|
115
|
+
region: 'nyc3',
|
|
116
|
+
bucket: 'my-bucket',
|
|
117
|
+
accessKeyId: process.env.DO_ACCESS_KEY_ID!,
|
|
118
|
+
secretAccessKey: process.env.DO_SECRET_ACCESS_KEY!,
|
|
119
|
+
}),
|
|
120
|
+
// The recipe sets cdnBaseUrl to *.cdn.digitaloceanspaces.com automatically.
|
|
121
|
+
// Override if you use a custom CDN domain:
|
|
122
|
+
// cdnBaseUrl: 'https://cdn.example.com',
|
|
123
|
+
})
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
### 4 — MinIO (local development)
|
|
127
|
+
|
|
128
|
+
```typescript
|
|
129
|
+
import { BymaxStorageModule, providerRecipes } from '@bymax-one/nest-storage'
|
|
130
|
+
|
|
131
|
+
BymaxStorageModule.forRoot({
|
|
132
|
+
...providerRecipes.minio({
|
|
133
|
+
endpoint: 'http://localhost:9000',
|
|
134
|
+
bucket: 'dev-bucket',
|
|
135
|
+
accessKeyId: 'minioadmin',
|
|
136
|
+
secretAccessKey: 'minioadmin',
|
|
137
|
+
}),
|
|
138
|
+
// forcePathStyle: true is set by the recipe automatically.
|
|
139
|
+
})
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
---
|
|
143
|
+
|
|
144
|
+
## Configuration
|
|
145
|
+
|
|
146
|
+
The full configuration reference lives in `docs/technical_specification.md` §4. Key options:
|
|
147
|
+
|
|
148
|
+
| Option | Type | Default | Notes |
|
|
149
|
+
|---|---|---|---|
|
|
150
|
+
| `endpoint` | `string` | — | Required. S3-compatible endpoint URL |
|
|
151
|
+
| `region` | `string` | — | Required. Provider region or `'auto'` (R2) |
|
|
152
|
+
| `bucket` | `string` | — | Default bucket (can be overridden per-call) |
|
|
153
|
+
| `credentials` | `{ accessKeyId, secretAccessKey, sessionToken? }` | — | Load from env / Secrets Manager |
|
|
154
|
+
| `forcePathStyle` | `boolean` | `false` | Set to `true` for MinIO and self-hosted |
|
|
155
|
+
| `publicBaseUrl` | `string` | — | Base URL for public `getPublicUrl()` results |
|
|
156
|
+
| `cdnBaseUrl` | `string` | — | CDN edge URL (preferred over `publicBaseUrl`) |
|
|
157
|
+
| `keyPrefix` | `string` | — | Prepended to every resolved key (multi-tenant) |
|
|
158
|
+
| `maxAttempts` | `number` | `3` | SDK v3 retry count |
|
|
159
|
+
| `serverSideEncryption` | `'AES256' \| 'aws:kms' \| 'NONE'` | — | Global SSE policy |
|
|
160
|
+
| `requestChecksumCalculation` | `'WHEN_SUPPORTED' \| 'WHEN_REQUIRED'` | SDK default | Set to `'WHEN_REQUIRED'` for non-AWS providers |
|
|
161
|
+
| `responseChecksumValidation` | `'WHEN_SUPPORTED' \| 'WHEN_REQUIRED'` | SDK default | Set to `'WHEN_REQUIRED'` for non-AWS providers |
|
|
162
|
+
| `signedUrls.defaultTtlSeconds` | `number` | `3600` | Default signed-URL TTL |
|
|
163
|
+
| `signedUrls.maxTtlSeconds` | `number` | `604800` | Hard ceiling; clamped to 7 days at init |
|
|
164
|
+
| `multipart.thresholdBytes` | `number` | `10 MiB` | Switch to multipart above this size |
|
|
165
|
+
| `multipart.partSizeBytes` | `number` | `8 MiB` | Part size (S3 minimum: 5 MiB) |
|
|
166
|
+
| `validation.mimeWhitelist` | `string[]` | sensible defaults | Wildcards: `'image/*'` |
|
|
167
|
+
| `validation.maxSizeBytes` | `number` | — | Maximum upload size in bytes |
|
|
168
|
+
|
|
169
|
+
> **Do not use `maxRetries` or `signatureVersion`** — these are AWS SDK v2 options that do not exist in v3. The v3 SDK is SigV4-only. Use `maxAttempts` (default `3`) for retry configuration.
|
|
170
|
+
|
|
171
|
+
---
|
|
172
|
+
|
|
173
|
+
## Provider Recipes
|
|
174
|
+
|
|
175
|
+
| Recipe | Provider | Notes |
|
|
176
|
+
|---|---|---|
|
|
177
|
+
| `providerRecipes.awsS3(...)` | AWS S3 | Keeps the SDK default checksum mode (`'WHEN_SUPPORTED'`); sets SSE-AES256 |
|
|
178
|
+
| `providerRecipes.cloudflareR2(...)` | Cloudflare R2 | `customDomain` required; checksums `'WHEN_REQUIRED'`; `region: 'auto'` |
|
|
179
|
+
| `providerRecipes.backblazeB2(...)` | Backblaze B2 | `forcePathStyle: false` (B2 supports virtual-hosted); checksums `'WHEN_REQUIRED'` |
|
|
180
|
+
| `providerRecipes.digitalOceanSpaces(...)` | DigitalOcean Spaces | Sets `cdnBaseUrl` to `*.cdn.digitaloceanspaces.com`; checksums `'WHEN_REQUIRED'` |
|
|
181
|
+
| `providerRecipes.minio(...)` | MinIO / self-hosted | `forcePathStyle: true`; checksums `'WHEN_REQUIRED'`; region defaults to `'us-east-1'` |
|
|
182
|
+
| `providerRecipes.wasabi(...)` | Wasabi Hot Cloud | Virtual-hosted; checksums `'WHEN_REQUIRED'` |
|
|
183
|
+
|
|
184
|
+
### Provider Compatibility — The #1 Trap
|
|
185
|
+
|
|
186
|
+
`@aws-sdk/client-s3` ≥ 3.729.0 defaults `requestChecksumCalculation` to `'WHEN_SUPPORTED'`, which causes the SDK to send `x-amz-checksum-crc32` integrity headers on every `PutObject` and `UploadPart` call. **Cloudflare R2, Backblaze B2, MinIO, DigitalOcean Spaces, and Wasabi reject these headers** — the default upload path fails out of the box on exactly the providers this library targets.
|
|
187
|
+
|
|
188
|
+
Every non-AWS provider recipe sets both `requestChecksumCalculation` and `responseChecksumValidation` to `'WHEN_REQUIRED'` to suppress the headers. If you build a custom config instead of using a recipe, you must add this opt-out manually:
|
|
189
|
+
|
|
190
|
+
```typescript
|
|
191
|
+
BymaxStorageModule.forRoot({
|
|
192
|
+
endpoint: 'https://...',
|
|
193
|
+
region: '...',
|
|
194
|
+
bucket: '...',
|
|
195
|
+
credentials: { ... },
|
|
196
|
+
requestChecksumCalculation: 'WHEN_REQUIRED',
|
|
197
|
+
responseChecksumValidation: 'WHEN_REQUIRED',
|
|
198
|
+
})
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
### Public Access / ACLs
|
|
202
|
+
|
|
203
|
+
`publicRead: true` (or `defaultPublicRead: true` in the config) sends `ACL: 'public-read'` on PutObject. However:
|
|
204
|
+
|
|
205
|
+
- **Modern AWS S3 buckets** (Object Ownership = "Bucket owner enforced", the default since April 2023) return **HTTP 400 `AccessControlListNotSupported`** when an ACL header is present.
|
|
206
|
+
- **Cloudflare R2** ignores ACLs entirely — `publicRead: true` is a **silent no-op**; configure a Custom Domain in the R2 dashboard for public delivery.
|
|
207
|
+
|
|
208
|
+
For public access on either provider, use a **bucket policy**, a **CDN**, or **signed URLs** rather than ACLs.
|
|
209
|
+
|
|
210
|
+
---
|
|
211
|
+
|
|
212
|
+
## Upload
|
|
213
|
+
|
|
214
|
+
### Single-shot / Buffer
|
|
215
|
+
|
|
216
|
+
```typescript
|
|
217
|
+
import { StorageService } from '@bymax-one/nest-storage'
|
|
218
|
+
import { randomUUID } from 'node:crypto'
|
|
219
|
+
|
|
220
|
+
const result = await storage.upload({
|
|
221
|
+
key: `avatars/${randomUUID()}.jpg`,
|
|
222
|
+
body: fileBuffer,
|
|
223
|
+
contentType: 'image/jpeg',
|
|
224
|
+
size: fileBuffer.length,
|
|
225
|
+
metadata: { userId: '42' },
|
|
226
|
+
serverSideEncryption: 'AES256',
|
|
227
|
+
})
|
|
228
|
+
console.log(result.publicUrl) // https://...
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
### Stream with Progress
|
|
232
|
+
|
|
233
|
+
```typescript
|
|
234
|
+
import { createReadStream, statSync } from 'node:fs'
|
|
235
|
+
|
|
236
|
+
await storage.upload({
|
|
237
|
+
key: 'videos/event-2026.mp4',
|
|
238
|
+
body: createReadStream('/tmp/video.mp4'),
|
|
239
|
+
contentType: 'video/mp4',
|
|
240
|
+
size: statSync('/tmp/video.mp4').size,
|
|
241
|
+
onProgress: (e) => console.log(`${e.loaded}/${e.total ?? '?'} bytes`),
|
|
242
|
+
})
|
|
243
|
+
```
|
|
244
|
+
|
|
245
|
+
Uploads are automatically routed to multipart when `size >= multipart.thresholdBytes` (default 10 MiB). A `Readable` without a `size` always uses multipart. Multipart aborts automatically on failure (`leavePartsOnError: false`).
|
|
246
|
+
|
|
247
|
+
### Idempotency
|
|
248
|
+
|
|
249
|
+
```typescript
|
|
250
|
+
const key = `uploads/${randomUUID()}.png`
|
|
251
|
+
const opts = { key, body: buf, contentType: 'image/png', size: buf.length, idempotencyKey: 'order-42-avatar' }
|
|
252
|
+
|
|
253
|
+
const r1 = await storage.upload(opts)
|
|
254
|
+
const r2 = await storage.upload(opts) // returns cached result instantly
|
|
255
|
+
console.log(r2.fromIdempotencyCache) // true
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
---
|
|
259
|
+
|
|
260
|
+
## Download
|
|
261
|
+
|
|
262
|
+
### As a Stream
|
|
263
|
+
|
|
264
|
+
```typescript
|
|
265
|
+
import { Controller, Get, Param, Res } from '@nestjs/common'
|
|
266
|
+
import type { Response } from 'express'
|
|
267
|
+
|
|
268
|
+
@Get(':key')
|
|
269
|
+
async download(@Param('key') key: string, @Res() res: Response) {
|
|
270
|
+
const { stream, metadata } = await this.storage.download({ key })
|
|
271
|
+
res.setHeader('Content-Type', metadata.contentType)
|
|
272
|
+
res.setHeader('Content-Length', String(metadata.size))
|
|
273
|
+
stream.pipe(res)
|
|
274
|
+
}
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
### As a Buffer
|
|
278
|
+
|
|
279
|
+
```typescript
|
|
280
|
+
// Use only for files < 10 MB — loads the entire object into memory.
|
|
281
|
+
const { buffer, metadata } = await this.storage.downloadBuffer({ key: 'docs/report.pdf' })
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
---
|
|
285
|
+
|
|
286
|
+
## Signed URLs (GET / PUT / Multipart)
|
|
287
|
+
|
|
288
|
+
> **Security:** signed URLs are temporary credentials. **Never log them** — a logged signed URL is accessible to anyone with log access.
|
|
289
|
+
|
|
290
|
+
### Presigned GET
|
|
291
|
+
|
|
292
|
+
```typescript
|
|
293
|
+
import { SignedUrlService } from '@bymax-one/nest-storage'
|
|
294
|
+
|
|
295
|
+
const result = await signedUrls.getDownloadUrl({
|
|
296
|
+
key: 'invoices/2026-001.pdf',
|
|
297
|
+
ttlSeconds: 900,
|
|
298
|
+
responseContentDisposition: 'attachment; filename="invoice.pdf"',
|
|
299
|
+
})
|
|
300
|
+
// result.url — share this with the client; expires at result.expiresAt
|
|
301
|
+
```
|
|
302
|
+
|
|
303
|
+
### Presigned PUT (direct browser upload)
|
|
304
|
+
|
|
305
|
+
```typescript
|
|
306
|
+
// Backend — issue a signed upload URL
|
|
307
|
+
@Post('signed-put')
|
|
308
|
+
async getSignedPut(@Body() body: { folder: string; contentType: string }) {
|
|
309
|
+
const key = `${body.folder}/${randomUUID()}`
|
|
310
|
+
const result = await this.signedUrls.getUploadUrl({
|
|
311
|
+
key,
|
|
312
|
+
contentType: body.contentType,
|
|
313
|
+
ttlSeconds: 600,
|
|
314
|
+
maxSizeBytes: 10 * 1024 * 1024,
|
|
315
|
+
})
|
|
316
|
+
// NEVER log result.url — it is a temporary credential
|
|
317
|
+
return { uploadUrl: result.url, key, expiresAt: result.expiresAt }
|
|
318
|
+
}
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
> **Caveat:** a signed PUT bypasses the library's local MIME/size validation. Pair it with a presigner `maxSizeBytes` (Content-Length-Range policy), a `HEAD` check after upload, and an `IFileScanner` post-upload hook.
|
|
322
|
+
|
|
323
|
+
```typescript
|
|
324
|
+
// Frontend
|
|
325
|
+
const { uploadUrl, key } = await fetchJson('/uploads/signed-put', { folder, contentType })
|
|
326
|
+
await fetch(uploadUrl, { method: 'PUT', headers: { 'Content-Type': contentType }, body: file })
|
|
327
|
+
await fetchJson('/uploads/confirm', { key }) // backend persists metadata
|
|
328
|
+
```
|
|
329
|
+
|
|
330
|
+
### Presigned Multipart
|
|
331
|
+
|
|
332
|
+
```typescript
|
|
333
|
+
const { uploadId, parts, key } = await signedUrls.getMultipartUploadUrls({
|
|
334
|
+
key: 'videos/large.mp4',
|
|
335
|
+
contentType: 'video/mp4',
|
|
336
|
+
partCount: 10,
|
|
337
|
+
ttlSeconds: 3600,
|
|
338
|
+
})
|
|
339
|
+
// Upload each part with the corresponding signed URL, then call CompleteMultipartUpload.
|
|
340
|
+
```
|
|
341
|
+
|
|
342
|
+
> **TTL clamp:** per-request `ttlSeconds` values above `signedUrls.maxTtlSeconds` are silently clamped; `maxTtlSeconds` is itself clamped to 604 800 s (7-day SigV4 ceiling) at module init. A `ttlSeconds` ≤ 0 throws `STORAGE_SIGNED_URL_TTL_INVALID`.
|
|
343
|
+
|
|
344
|
+
---
|
|
345
|
+
|
|
346
|
+
## Validation
|
|
347
|
+
|
|
348
|
+
```typescript
|
|
349
|
+
import { Injectable } from '@nestjs/common'
|
|
350
|
+
import type { IUploadValidator } from '@bymax-one/nest-storage'
|
|
351
|
+
|
|
352
|
+
/** Magic-byte PDF check — reads the first 4 bytes of the stream. */
|
|
353
|
+
@Injectable()
|
|
354
|
+
export class PdfValidator implements IUploadValidator {
|
|
355
|
+
async validate(input: { body: Buffer | NodeJS.ReadableStream; readBytes: (n: number) => Promise<Buffer> }) {
|
|
356
|
+
const header = await input.readBytes(4)
|
|
357
|
+
if (header.toString('ascii', 0, 4) !== '%PDF') {
|
|
358
|
+
return { valid: false, reason: 'Not a valid PDF (magic bytes mismatch)' }
|
|
359
|
+
}
|
|
360
|
+
return { valid: true }
|
|
361
|
+
}
|
|
362
|
+
}
|
|
363
|
+
```
|
|
364
|
+
|
|
365
|
+
Register it in the module:
|
|
366
|
+
|
|
367
|
+
```typescript
|
|
368
|
+
BymaxStorageModule.forRoot({
|
|
369
|
+
...providerRecipes.awsS3({ ... }),
|
|
370
|
+
validation: {
|
|
371
|
+
mimeWhitelist: ['application/pdf'],
|
|
372
|
+
maxSizeBytes: 25 * 1024 * 1024,
|
|
373
|
+
},
|
|
374
|
+
validators: [{ useClass: PdfValidator }],
|
|
375
|
+
})
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
---
|
|
379
|
+
|
|
380
|
+
## Virus Scanning (`IFileScanner`)
|
|
381
|
+
|
|
382
|
+
```typescript
|
|
383
|
+
import type { IFileScanner, FileScanResult } from '@bymax-one/nest-storage'
|
|
384
|
+
|
|
385
|
+
/** ClamAV stub — delegate to clamd over a socket in production. */
|
|
386
|
+
@Injectable()
|
|
387
|
+
export class ClamAvScanner implements IFileScanner {
|
|
388
|
+
async scan(_input: unknown): Promise<FileScanResult> {
|
|
389
|
+
// Replace with real clamd socket call
|
|
390
|
+
return { status: 'clean', engine: 'clamav-0.103' }
|
|
391
|
+
}
|
|
392
|
+
}
|
|
393
|
+
```
|
|
394
|
+
|
|
395
|
+
```typescript
|
|
396
|
+
BymaxStorageModule.forRoot({
|
|
397
|
+
...providerRecipes.awsS3({ ... }),
|
|
398
|
+
scanner: {
|
|
399
|
+
impl: new ClamAvScanner(),
|
|
400
|
+
mode: 'pre-upload', // 'pre-upload' | 'post-upload' | 'both'
|
|
401
|
+
rejectOnUnknown: false, // treat inconclusive scans as pass (true = reject)
|
|
402
|
+
},
|
|
403
|
+
})
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
---
|
|
407
|
+
|
|
408
|
+
## Lifecycle Operations
|
|
409
|
+
|
|
410
|
+
### List / Paginate
|
|
411
|
+
|
|
412
|
+
```typescript
|
|
413
|
+
const page1 = await storage.list({ prefix: 'avatars/', maxKeys: 100 })
|
|
414
|
+
// { objects: [...], nextContinuationToken?: '...' }
|
|
415
|
+
if (page1.nextContinuationToken) {
|
|
416
|
+
const page2 = await storage.list({ prefix: 'avatars/', maxKeys: 100, continuationToken: page1.nextContinuationToken })
|
|
417
|
+
}
|
|
418
|
+
```
|
|
419
|
+
|
|
420
|
+
### Copy (server-side)
|
|
421
|
+
|
|
422
|
+
```typescript
|
|
423
|
+
const { etag } = await storage.copy({
|
|
424
|
+
sourceKey: 'drafts/file.pdf',
|
|
425
|
+
destinationKey: 'published/file.pdf',
|
|
426
|
+
})
|
|
427
|
+
```
|
|
428
|
+
|
|
429
|
+
### Delete / Batch Delete
|
|
430
|
+
|
|
431
|
+
```typescript
|
|
432
|
+
await storage.delete('temp/file.png') // idempotent — no error on 404
|
|
433
|
+
|
|
434
|
+
const { deleted, failed } = await storage.deleteMany(['a.png', 'b.png', 'c.png'])
|
|
435
|
+
```
|
|
436
|
+
|
|
437
|
+
### Raw S3Client (Advanced)
|
|
438
|
+
|
|
439
|
+
For provider-specific operations not covered by the public API, inject the raw client:
|
|
440
|
+
|
|
441
|
+
```typescript
|
|
442
|
+
import { Inject, Injectable } from '@nestjs/common'
|
|
443
|
+
import { S3Client } from '@aws-sdk/client-s3'
|
|
444
|
+
import { BYMAX_STORAGE_S3_CLIENT } from '@bymax-one/nest-storage'
|
|
445
|
+
|
|
446
|
+
@Injectable()
|
|
447
|
+
export class AdvancedStorageService {
|
|
448
|
+
constructor(@Inject(BYMAX_STORAGE_S3_CLIENT) private readonly s3: S3Client) {}
|
|
449
|
+
|
|
450
|
+
async customOperation() {
|
|
451
|
+
// Direct S3Client call — bypass the public API when needed.
|
|
452
|
+
}
|
|
453
|
+
}
|
|
454
|
+
```
|
|
455
|
+
|
|
456
|
+
### `forRootAsync`
|
|
457
|
+
|
|
458
|
+
```typescript
|
|
459
|
+
import { ConfigModule, ConfigService } from '@nestjs/config'
|
|
460
|
+
|
|
461
|
+
BymaxStorageModule.forRootAsync({
|
|
462
|
+
imports: [ConfigModule],
|
|
463
|
+
inject: [ConfigService],
|
|
464
|
+
useFactory: (config: ConfigService) => ({
|
|
465
|
+
...providerRecipes.awsS3({
|
|
466
|
+
region: config.getOrThrow('AWS_REGION'),
|
|
467
|
+
bucket: config.getOrThrow('AWS_BUCKET'),
|
|
468
|
+
accessKeyId: config.getOrThrow('AWS_ACCESS_KEY_ID'),
|
|
469
|
+
secretAccessKey: config.getOrThrow('AWS_SECRET_ACCESS_KEY'),
|
|
470
|
+
}),
|
|
471
|
+
keyPrefix: config.get('STORAGE_KEY_PREFIX'),
|
|
472
|
+
serverSideEncryption: 'AES256',
|
|
473
|
+
}),
|
|
474
|
+
})
|
|
475
|
+
```
|
|
476
|
+
|
|
477
|
+
---
|
|
478
|
+
|
|
479
|
+
## Error Codes
|
|
480
|
+
|
|
481
|
+
All errors are thrown as `StorageException extends HttpException`. The response body is `{ error: { code, message, details? } }`.
|
|
482
|
+
|
|
483
|
+
| Code | HTTP | When |
|
|
484
|
+
|---|---|---|
|
|
485
|
+
| `STORAGE_NOT_CONFIGURED` | 503 | Credentials missing and an operation was called |
|
|
486
|
+
| `STORAGE_KEY_INVALID` | 400 | Path traversal (`..`), key starts with `/`, or empty after normalization |
|
|
487
|
+
| `STORAGE_BODY_MISSING` | 400 | `upload()` called without a body |
|
|
488
|
+
| `STORAGE_CONTENT_TYPE_REQUIRED` | 400 | `upload()` called without `contentType` |
|
|
489
|
+
| `STORAGE_MIME_NOT_ALLOWED` | 415 | `contentType` outside `mimeWhitelist` |
|
|
490
|
+
| `STORAGE_SIZE_EXCEEDED` | 413 | `size > maxSizeBytes` |
|
|
491
|
+
| `STORAGE_VALIDATION_FAILED` | 400 | `IUploadValidator` rejected the file (`details.reason`) |
|
|
492
|
+
| `STORAGE_SCAN_INFECTED` | 422 | Scanner returned `'infected'` (`details.threat`) |
|
|
493
|
+
| `STORAGE_SCAN_INCONCLUSIVE` | 422 | Scanner returned `'unknown'` and `rejectOnUnknown: true` |
|
|
494
|
+
| `STORAGE_OBJECT_NOT_FOUND` | 404 | `head()`, `download()`, or `copy()` on a nonexistent key |
|
|
495
|
+
| `STORAGE_PROVIDER_ERROR` | 502 | AWS SDK error (network, 5xx, throttling, `AccessControlListNotSupported`); `details.awsCode`, `httpStatus`, `requestId` |
|
|
496
|
+
| `STORAGE_SIGNED_URL_TTL_INVALID` | 400 | Per-request `ttlSeconds` ≤ 0 |
|
|
497
|
+
| `STORAGE_PART_TOO_SMALL` | 400 | Multipart with part < 5 MiB (S3 limit) |
|
|
498
|
+
| `STORAGE_BUCKET_UNDEFINED` | 400 | Operation without a bucket and no default in config |
|
|
499
|
+
| `STORAGE_MULTIPART_ABORTED` | 500 | Multipart upload failed and was aborted |
|
|
500
|
+
| `STORAGE_INVALID_CONFIG` | 500 | `BymaxStorageModuleOptions` validation failed at initialization |
|
|
501
|
+
| `STORAGE_TIMEOUT` | 504 | Request exceeded `requestTimeoutMs` |
|
|
502
|
+
|
|
503
|
+
```typescript
|
|
504
|
+
import { StorageException, STORAGE_ERROR_CODES } from '@bymax-one/nest-storage'
|
|
505
|
+
|
|
506
|
+
try {
|
|
507
|
+
await storage.upload({ ... })
|
|
508
|
+
} catch (err) {
|
|
509
|
+
if (err instanceof StorageException) {
|
|
510
|
+
const body = err.getResponse() as { error: { code: string; message: string } }
|
|
511
|
+
console.error(body.error.code) // e.g. 'STORAGE_MIME_NOT_ALLOWED'
|
|
512
|
+
}
|
|
513
|
+
}
|
|
514
|
+
```
|
|
515
|
+
|
|
516
|
+
---
|
|
517
|
+
|
|
518
|
+
## Testing
|
|
519
|
+
|
|
520
|
+
```bash
|
|
521
|
+
# Unit tests
|
|
522
|
+
pnpm test
|
|
523
|
+
|
|
524
|
+
# Unit tests with coverage (100% threshold enforced)
|
|
525
|
+
pnpm test:cov
|
|
526
|
+
|
|
527
|
+
# E2E tests (requires Docker — spins MinIO via Testcontainers)
|
|
528
|
+
pnpm test:e2e
|
|
529
|
+
|
|
530
|
+
# Mutation testing (Stryker — run manually pre-release, not on every CI commit)
|
|
531
|
+
pnpm mutation
|
|
532
|
+
```
|
|
533
|
+
|
|
534
|
+
---
|
|
535
|
+
|
|
536
|
+
## Contributing
|
|
537
|
+
|
|
538
|
+
Security issues: report privately to [security@bymax.one](mailto:security@bymax.one) — do not open a public issue. See [SECURITY.md](SECURITY.md) for the full disclosure policy and storage-specific security goals.
|
|
539
|
+
|
|
540
|
+
For feature requests and bugs, open a GitHub issue. Pull requests are welcome; please run `pnpm test:cov` and `pnpm lint` before submitting.
|
|
541
|
+
|
|
542
|
+
---
|
|
543
|
+
|
|
544
|
+
## License
|
|
545
|
+
|
|
546
|
+
[MIT](LICENSE) — Copyright (c) 2026 Bymax One
|