@bymax-one/nest-storage 1.1.18 → 1.1.20

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -13,6 +13,148 @@ note on each entry is the compatibility contract — read it, not the number.
13
13
 
14
14
  ## [Unreleased]
15
15
 
16
+ ## [1.1.20] - 2026-11-21
17
+
18
+ ### Fixed
19
+
20
+ - **A throwing `onProgress` no longer fails an upload that stored its object.** On the single-shot path the hook was
21
+ called in the upload's own `try` block, so a hook that threw made `upload()` reject with `STORAGE_PROVIDER_ERROR`
22
+ after `PutObject` had succeeded; on the multipart path a throw reached the uploader. An async hook that rejected was
23
+ an unhandled rejection that ends the Node process. Both paths now ignore what the hook throws or rejects with.
24
+
25
+ **Apply to a derived backend:** nothing to change. A hook that threw used to turn a stored upload into an error; it
26
+ now leaves the result alone, so an application that relied on a throwing hook to fail an upload must fail it itself.
27
+
28
+ - **An option that throws when it is read is refused with `STORAGE_INVALID_CONFIG`.** A getter or a Proxy of the
29
+ host's as the options object, a group or a validator, and a `then` that throws on the object a `useFactory` returns,
30
+ surfaced the host's own error from `forRoot` and `forRootAsync`. They now fail `STORAGE_INVALID_CONFIG`
31
+ (`reason: 'the options could not be read'`), the host's error is kept as the `cause` of the exception and none of
32
+ its text is in the response, and a whitelist or a validator list whose iterator throws is refused the same way instead
33
+ of as "must be an array of strings". `StorageException` takes an optional `cause`.
34
+
35
+ **Apply to a derived backend:** a factory that throws is now reported as `STORAGE_INVALID_CONFIG` with the
36
+ factory's error as the `cause`; read `cause` where you matched the raw error.
37
+
38
+ - **A lone surrogate in a logged value is escaped.** `escapeForLog` left a lone surrogate (`\ud800`) as it was, and a
39
+ logger writes it as an invalid character. It shows as `\ud800` now, like the other characters that cannot be shown;
40
+ a well-formed pair, an emoji, stays readable.
41
+
42
+ **Apply to a derived backend:** nothing to change; a log rule that matched the raw character reads the escape.
43
+
44
+ - **A very large sum of uploaded bytes no longer fails the metrics scrape.** `bymax_storage_bytes_uploaded_total` is
45
+ the sum of every object's size, and two finite sizes can add up past the largest number, to `Infinity`, which a
46
+ counter refuses: the whole `/metrics` answer then failed. The count in the tally now stops at the largest finite
47
+ number.
48
+
49
+ **Apply to a derived backend:** nothing to change; a scrape that failed on that sum now answers.
50
+
51
+ - **The README names what the multipart path does on a failure.** It said the SDK aborts the upload
52
+ (`leavePartsOnError: false`); the library passes `true` and sends `AbortMultipartUpload` itself, so a failing abort
53
+ cannot replace the error that caused it, and the outcome is in `details.abort`.
54
+
55
+ **Apply to a derived backend:** nothing to change; only the text was wrong.
56
+
57
+ ### Changed
58
+
59
+ - **The span attributes are named, and the scan span takes the two scan modes.** `StorageSpanAttributes` was
60
+ `Readonly<Record<string, string>>`; it is now an interface naming the attributes the library sets, so a call site cannot add a key
61
+ that carries a bucket or a payload, and the scan span's `mode` parameter is `'pre-upload' | 'post-upload'` instead of
62
+ `string`.
63
+
64
+ **Apply to a derived backend:** code that calls the scan span with a mode the interface does not declare, that builds
65
+ attributes with another key, or that reads an attribute by a key it does not declare does not compile against the
66
+ interface; use the declared modes and names. Nothing changes at run time.
67
+
68
+ - **`defaultPublicRead` and `forcePathStyle` must be booleans, and so must a per-call `publicRead`.** A string from an
69
+ environment variable, `'false'`, is truthy: it passed the `??` default and made every upload `public-read`. A value
70
+ of another kind for either option is now refused at boot with `STORAGE_INVALID_CONFIG`, and a per-call `publicRead`
71
+ that is set and is not a boolean fails that call with `STORAGE_VALIDATION_FAILED` (`details.field: 'publicRead'`);
72
+ `null` is refused like any other non-boolean, and an absent value still takes the default.
73
+
74
+ **Apply to a derived backend:** a host that passes `process.env.X` straight into either option now gets a boot
75
+ refusal; compare it to `'true'` and pass the boolean.
76
+
77
+ - **`validation.mimeWhitelist` and `validation.customValidators` hold at most 1000 entries.** A longer array, or an
78
+ array whose reported length is not a count, is refused at boot with `STORAGE_INVALID_CONFIG` before any entry is
79
+ read, so a sparse array of billions of holes no longer stalls the start.
80
+
81
+ **Apply to a derived backend:** nothing to change unless a list holds more than 1000 entries; a whitelist that long
82
+ can use a wildcard instead.
83
+
84
+ ## [1.1.19] - 2026-11-20
85
+
86
+ ### Added
87
+
88
+ - **An optional `./observability` subpath and a span seam.** `StorageMetrics` publishes
89
+ `bymax_storage_operations_total`, `bymax_storage_operation_duration_seconds`, `bymax_storage_bytes_uploaded_total`
90
+ and `bymax_storage_scans_total` with labels from closed sets, and `StorageHealthIndicator` reports the bucket as
91
+ `storage` from one `HeadBucket`. The main entry now exports `StorageMetricsTally` and the closed sets
92
+ (`STORAGE_OPERATIONS`, `STORAGE_OUTCOMES`, `SCAN_VERDICTS`), counts every public operation, and starts the
93
+ `storage.upload`, `storage.download_url` and `storage.scan` spans when `@opentelemetry/api` is installed. `bymax_storage_bytes_uploaded_total` counts the bytes an upload sent, including a stream without a declared `size`. The new
94
+ peers `@bymax-one/nest-core >=1.6.15`, `prom-client ^15` and `@opentelemetry/api ^1.9` are optional.
95
+
96
+ **Apply to a derived backend:** nothing changes unless you use it. To publish the metrics, add `StorageMetrics` and
97
+ `StorageHealthIndicator` to the providers of a module and install `prom-client`; the README "Observability"
98
+ section has the table of metrics, labels and spans.
99
+
100
+ ### Changed
101
+
102
+ - **The library's log lines are object events.** Every operational line is now an object with an `event` code
103
+ (`STORAGE_*`), a `message` with the library's own text and named fields, through the Nest `Logger`; the README has
104
+ a "Log events" table. What the provider, the scanner or the host raised goes under `err` as a fresh error (name,
105
+ provider code, masked and cut message), never into the text, and never the SDK's own error. The key you passed is
106
+ no longer logged by the provider-failure and scan events, the client-ready line no longer carries the endpoint or
107
+ the region, and a host logger that throws can no longer fail the call being logged. A host `LoggerService` method that is
108
+ async and rejects is still an unhandled rejection, which ends the Node process unless the application
109
+ handles `unhandledRejection`: Nest's `Logger` drops the value the method returns before the library can see it.
110
+
111
+ **Apply to a derived backend:** a log rule or an alert that matched the text of these lines (`Provider call failed`,
112
+ `Scan timed out`, `Scanner failed`, `Infected file detected`, `S3Client initialized`, the exists and delete
113
+ warnings, the TTL clamp) now matches the `event` code and reads the fields; the library text is kept in `message`.
114
+ A sink that stringifies the message sees an object: use a logger that writes object messages as fields. Anything
115
+ that relied on the key, the endpoint or the region in these lines reads them from the call or the configuration.
116
+ A quote or a backslash in a logged value is no longer escaped by the library, since the logger escapes it when it
117
+ serialises the field; control, bidirectional and format characters still show as escapes.
118
+ The `cleanupFailure` field of `STORAGE_SCAN_CLEANUP_FAILED` is the code of a `StorageException`; any other failure is masked, cut and escaped like `err`.
119
+
120
+ ### Fixed
121
+
122
+ - **An option group of the wrong kind is refused at boot.** `validation`, `signedUrls` or `multipart` given as `false`,
123
+ `0`, a string, `true` or an array booted without error and was ignored, so `validation: enabled && { ... }` with
124
+ `enabled` false left uploads unvalidated. `forRoot` and `forRootAsync` now fail with `STORAGE_INVALID_CONFIG`
125
+ (`options.<group> must be an object`), repeating nothing of the value; `undefined` and `null` still mean "not
126
+ configured". `credentials` given as an array is refused as well; before, `forRoot` copied any array given for a
127
+ group into an empty group.
128
+
129
+ **Apply to a derived backend:** a configuration that passed one of those values for a group now fails at boot.
130
+ Pass an object, or leave the group out.
131
+
132
+ - **`validation.customValidators` entries are checked at boot.** An entry that is not an object with a `validate`
133
+ function (`42`, `null`, `{}`, a bare function) booted and then failed the first upload with a raw `TypeError`
134
+ (500). It now fails `forRoot` with `STORAGE_INVALID_CONFIG`. A `mimeWhitelist` that throws when it is read (a proxy,
135
+ a throwing index getter) is refused as "must be an array of strings" instead of surfacing the thrown error.
136
+
137
+ **Apply to a derived backend:** a list that held anything but validator objects now fails at boot; a class
138
+ instance that serves `validate` from its prototype is accepted. The README option table now has the
139
+ `validation.customValidators` row.
140
+
141
+ - **An error no longer repeats what the host configured.** `details` carried the module's `bucket` (when the call
142
+ named none), the `keyPrefix` (inside the `key` of an upload, head, download, delete, copy and signed-URL
143
+ failure, and the `prefix` of a list failure) and the whole `validation.mimeWhitelist` (in `STORAGE_MIME_NOT_ALLOWED`). The `key` is now the one the
144
+ caller passed, the `bucket` appears only when the caller passed one, and the whitelist is not repeated; a `key`,
145
+ `bucket`, `prefix`, `sourceKey` or `destKey` longer than 1024 characters is left out.
146
+
147
+ The same applies to a scan: a `STORAGE_TIMEOUT` for a scan that outlasts its deadline repeated the prefixed key and
148
+ the module bucket, and now carries the `key` the caller passed and a per-call `bucket`. `details.storedKey`,
149
+ which a rejected scan whose cleanup delete failed carried, is removed from the response: the operator gets the key as
150
+ stored in the `STORAGE_SCAN_CLEANUP_FAILED` log entry, in its `storedKey` field.
151
+
152
+ **Apply to a derived backend:** a client or a log rule that read `details.bucket` of a call that named no bucket,
153
+ `details.whitelist` of `STORAGE_MIME_NOT_ALLOWED`, `details.storedKey` of a rejected scan whose cleanup failed, or a
154
+ `keyPrefix` in `details.key` finds none, and reads the module configuration instead. A rule on the text of the
155
+ `STORAGE_SCAN_CLEANUP_FAILED` log message now reads the `event` and `storedKey` fields of the entry. A `key` and a
156
+ `bucket` that the caller passed are unchanged.
157
+
16
158
  ## [1.1.18] - 2026-11-18
17
159
 
18
160
  ### Fixed
@@ -1108,4 +1250,6 @@ have regressed from. They are kept because the reasoning is worth having.
1108
1250
  [1.1.16]: https://github.com/bymaxone/nest-storage/compare/v1.1.15...v1.1.16
1109
1251
  [1.1.17]: https://github.com/bymaxone/nest-storage/compare/v1.1.16...v1.1.17
1110
1252
  [1.1.18]: https://github.com/bymaxone/nest-storage/compare/v1.1.17...v1.1.18
1111
- [Unreleased]: https://github.com/bymaxone/nest-storage/compare/v1.1.18...HEAD
1253
+ [1.1.19]: https://github.com/bymaxone/nest-storage/compare/v1.1.18...v1.1.19
1254
+ [1.1.20]: https://github.com/bymaxone/nest-storage/compare/v1.1.19...v1.1.20
1255
+ [Unreleased]: https://github.com/bymaxone/nest-storage/compare/v1.1.20...HEAD
package/README.md CHANGED
@@ -58,8 +58,8 @@ minimal.
58
58
 
59
59
  ## 🔥 Features
60
60
 
61
- - ✅ **Multipart upload** — `@aws-sdk/lib-storage` with automatic abort on failure
62
- (`leavePartsOnError: false`), progress events, and a configurable threshold and part size
61
+ - ✅ **Multipart upload** — `@aws-sdk/lib-storage` with an abort on failure that the library runs
62
+ itself, progress events, and a configurable threshold and part size
63
63
  - ✅ **Content validation** — MIME allowlist with wildcards (`image/*`) and a size limit, then any
64
64
  number of `IUploadValidator`s, all refused before a byte is stored
65
65
  - ✅ **Virus scanning hook** — `IFileScanner` pre-upload or post-upload, for ClamAV, AWS Macie or
@@ -81,6 +81,9 @@ minimal.
81
81
  provider-specific operations this surface does not wrap
82
82
  - ✅ **Key prefix and server-side encryption** — `keyPrefix` is prepended after normalization and the
83
83
  keys the library returns are relative to it; AES256 or `aws:kms`, globally or per upload
84
+ - ✅ **Metrics, readiness and spans** — an optional `./observability` subpath publishes Prometheus
85
+ metrics and a bucket readiness check; spans start when `@opentelemetry/api` is installed. Labels are
86
+ closed sets: no key, bucket or prefix is ever a label
84
87
  - ✅ **Typed end to end** — TypeScript `strict` with `exactOptionalPropertyTypes` and
85
88
  `noUncheckedIndexedAccess`; zero `any`
86
89
 
@@ -91,11 +94,12 @@ minimal.
91
94
  | Subpath | Contents |
92
95
  | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
93
96
  | `.` | Server runtime: `BymaxStorageModule`, `StorageService`, `SignedUrlService`, `BucketLifecycleService`, `providerRecipes`, DI tokens, interfaces, `StorageException`, `NoOpUploadValidator`, `NoOpFileScanner` |
97
+ | `./observability` | Optional: `StorageMetrics` (Prometheus metrics), `StorageHealthIndicator` (readiness of the bucket) |
94
98
  | `./shared` | Framework-free types (`UploadResult`, `ObjectMetadata`, `ListedObject`, `SignedUrlResult`, `StorageErrorResponse`) + `STORAGE_ERROR_CODES` + `StorageErrorCode` + the default MIME whitelists and the TTL and multipart default constants |
95
99
 
96
100
  The root re-exports some `./shared` names for convenience; import the rest from the subpath.
97
101
 
98
- Both subpaths ship ESM **and** CommonJS with declarations for each format, so a
102
+ All subpaths ship ESM **and** CommonJS with declarations for each format, so a
99
103
  `require()` consumer receives CommonJS declarations rather than ESM ones. The
100
104
  `pnpm check:exports` gate verifies this against the packed tarball.
101
105
 
@@ -103,7 +107,8 @@ Both subpaths ship ESM **and** CommonJS with declarations for each format, so a
103
107
 
104
108
  | Subpath | Required peers |
105
109
  | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
106
- | `.` (server) | `@nestjs/common ^11.0.16`, `@nestjs/core ^11.1.18`, `@aws-sdk/client-s3 ^3.700`, `@aws-sdk/lib-storage ^3.700`, `@aws-sdk/s3-request-presigner ^3.700`, `reflect-metadata ^0.2` |
110
+ | `.` (server) | `@nestjs/common ^11.0.16`, `@nestjs/core ^11.1.18`, `@aws-sdk/client-s3 ^3.700`, `@aws-sdk/lib-storage ^3.700`, `@aws-sdk/s3-request-presigner ^3.700`, `reflect-metadata ^0.2`; optional: `@opentelemetry/api ^1.9` (the spans start only when it is installed) |
111
+ | `./observability` | The peers of `.`, and `prom-client ^15` for the metrics; `@bymax-one/nest-core >=1.6.15` is how a host discovers the two classes. `prom-client` and `@bymax-one/nest-core` are optional for the package, needed only to use this subpath |
107
112
  | `./shared` | None |
108
113
 
109
114
  ---
@@ -232,8 +237,15 @@ Options accepted by `forRoot` and `forRootAsync`:
232
237
  | `multipart.thresholdBytes` | `number` | `5 MiB` | Switch to multipart at or above this size |
233
238
  | `multipart.partSizeBytes` | `number` | `5 MiB` | Part size (S3 minimum: 5 MiB) |
234
239
  | `multipart.queueSize` | `number` | `4` | Concurrent parts |
240
+ | `scanner.impl` | `IFileScanner` | — | The scanner. Without it no scan runs; an object with a `scan` function, see Virus Scanning |
241
+ | `scanner.mode` | `'pre-upload' \| 'post-upload'` | `'pre-upload'` | When the scan runs: before the bytes are stored, or after, on the stored key |
242
+ | `scanner.rejectOnUnknown` | `boolean` | `false` | `true` rejects an inconclusive (`'unknown'`) scan; `false` lets it through with a warning |
243
+ | `scanner.timeoutMs` | `number` | `60000` | Deadline of one scan; a whole number from `1` to `2147483647`; past it the upload is refused with `STORAGE_TIMEOUT` |
235
244
  | `validation.mimeWhitelist` | `string[]` | — | No MIME check when absent. Wildcards: `'image/*'`. A value that lists several types never matches. An array of strings: any other kind fails `forRoot`, and `null` is refused (omit it to allow every type) |
236
245
  | `validation.maxSizeBytes` | `number` | — | Maximum upload size in bytes; counted as a stream is read, so it holds with or without `size`. `null` is refused: omit it for no limit |
246
+ | `validation.customValidators` | `IUploadValidator[]` | — | Run in order before the upload. An array of objects that each have a `validate` function: any other kind, or an entry that is not such an object, fails `forRoot`; `null` is refused (omit it to run none) |
247
+
248
+ **Option groups are objects.** `signedUrls`, `multipart` and `validation` are each an object, or left out (`null` is read as left out). `false`, `0`, a string, `true` or an array fails `forRoot` with `STORAGE_INVALID_CONFIG` naming the group and repeating nothing of the value: ignoring it would boot without the protection that a toggle such as `validation: enabled && { ... }` was meant to switch on.
237
249
 
238
250
  **Numeric options are whole numbers, checked at boot.** `validation.maxSizeBytes`, the `signedUrls` TTLs and `multipart.thresholdBytes` take a whole number of at least `1`; `multipart.partSizeBytes` takes one from 5 MiB (the S3 minimum) to 5 GiB (the S3 maximum) and `multipart.queueSize` one from `1` to `1000`. `NaN`, `Infinity`, a fraction, a string or a value out of range fails `forRoot` with `STORAGE_INVALID_CONFIG` naming the option: a comparison alone lets `NaN` through, and `NaN` on a size limit would turn the limit off. The same holds per call: a `ttlSeconds` that is not a whole number above `0` fails with `STORAGE_SIGNED_URL_TTL_INVALID`, and a declared upload `size` or a list `maxKeys` that is not a whole number (at least `0`, and at least `1`) fails with `STORAGE_VALIDATION_FAILED` naming the field, before any request is made. A key set to `undefined` or `null` inside `signedUrls`, `multipart`, `validation` or `scanner` means the default, except `scanner.impl` and the `validation` options named below, which refuse `null`. The per-call values are refused on `null`, and the error for a TTL echoes no value but a finite number. Where leaving an option out means a limit or a protection is off, `null` is refused for it as well, so a config lookup that returns `null` for a missing key cannot switch it off: `validation.maxSizeBytes`, `validation.mimeWhitelist` and `validation.customValidators` (the last must be an array).
239
251
 
@@ -339,7 +351,12 @@ await storage.upload({
339
351
  })
340
352
  ```
341
353
 
342
- Uploads are automatically routed to multipart when `size >= multipart.thresholdBytes` (default 5 MiB). A `Readable` without a `size` always uses multipart. Multipart aborts automatically on failure (`leavePartsOnError: false`).
354
+ `onProgress` receives `{ loaded, total?, part? }`: the bytes sent so far, the total when it is known, and the
355
+ part number on a multipart upload. A single-shot upload reports once, when it ends; a multipart upload reports
356
+ as each part is sent.
357
+ A hook that throws, or an async one that rejects, is ignored: the upload still resolves with its result and the process is not ended.
358
+
359
+ Uploads are automatically routed to multipart when `size >= multipart.thresholdBytes` (default 5 MiB). A `Readable` without a `size` always uses multipart. A failed multipart upload is aborted by the library, which tells the SDK to leave the parts (`leavePartsOnError: true`) and runs `AbortMultipartUpload` itself, so an abort that fails cannot replace the error that caused it; the outcome (`completed` or `failed`) is in the error's `details.abort`.
343
360
 
344
361
  **What `body` may be.** A `Buffer`, a `Uint8Array`, an `ArrayBuffer`, a string or a Node `Readable`. Anything else (a `Blob`, an array of chunks, a plain object) fails with `STORAGE_VALIDATION_FAILED` (`details.field: 'body'`) before any request; it used to reach the upload as if it were a stream and fail as `STORAGE_MULTIPART_ABORTED` (500).
345
362
 
@@ -598,6 +615,11 @@ BymaxStorageModule.forRoot({
598
615
  })
599
616
  ```
600
617
 
618
+ **What your scanner is given.** `scan(input)` receives `mode` (`'pre-upload'` or `'post-upload'`, the configured
619
+ `scanner.mode`), the `key` and `bucket` as the provider holds them, the `contentType`, the `size` when known, a
620
+ `signal` that aborts at `scanner.timeoutMs`, and a `body` (a `Buffer` or a stream) only when `mode` is
621
+ `'pre-upload'`. In `'post-upload'` mode the scanner reads the object itself.
622
+
601
623
  **Streams in pre-upload mode.** A stream can be read once, so in `pre-upload` mode it is first read
602
624
  into memory, bounded by `validation.maxSizeBytes`, and the scanner and the upload share that buffer. A `Buffer` or `Uint8Array` body is copied once in this mode, so a change the caller makes to its own array while the scan runs reaches neither (the second copy of the body is held until the upload ends, so the body takes twice its size in memory). The library validates and scans the bytes at call time; do not change a buffer you passed until the call settles. A
603
625
  stream longer than the limit is refused with `STORAGE_VALIDATION_FAILED` as soon as it passes it, and
@@ -611,7 +633,7 @@ with no `maxSizeBytes` it is refused with a reason that says to set it or use `p
611
633
  **What counts as a verdict.** Only the exact strings `'clean'`, `'infected'` and `'unknown'` are results. Anything else (no result, `null`, `'CLEAN'`, `' clean'`, an object, a made-up status) is rejected as `STORAGE_SCAN_INCONCLUSIVE` with `details.reason: 'unrecognised scanner status'`, whatever `rejectOnUnknown` says: an object the scanner did not call clean is not stored.
612
634
 
613
635
  **What a client is told.** `STORAGE_SCAN_INFECTED` carries `details.engine` and `details.threat`, and
614
- `STORAGE_SCAN_INCONCLUSIVE` carries `details.engine`; each name is cut, in the body and in the log, and left out when the scanner gave something that is not a string. A scanner that throws is answered with a 500, and the library logs a `Scanner failed` warning with the message (cut) so there is a trace of why; a scan that outlasts `scanner.timeoutMs` is logged as `Scan timed out` with the key and the limit. A cut never leaves half of a surrogate pair, so an emoji at the cut is dropped whole. The free-form `details` your scanner returns is the
636
+ `STORAGE_SCAN_INCONCLUSIVE` carries `details.engine`; each name is cut, in the body and in the log, and left out when the scanner gave something that is not a string. A scanner that throws is answered with a 500, and the library logs a `STORAGE_SCAN_FAILED` event with the error under `err` (its message cut) so there is a trace of why; a scan that outlasts `scanner.timeoutMs` is logged as `STORAGE_SCAN_TIMEOUT` with the limit (see [Log events](#-log-events)). The free-form `details` your scanner returns is the
615
637
  scanner's own text, so it is written to the warning log (escaped and truncated) and never to the
616
638
  body of the error.
617
639
 
@@ -620,9 +642,10 @@ In `post-upload` mode your scanner is given the key and reads the object itself,
620
642
  **A rejected object that could not be removed.** In `post-upload` mode the object is already stored
621
643
  when the scanner rejects it, so the library deletes it and rejects `upload()` with the verdict
622
644
  (`STORAGE_SCAN_INFECTED` or `STORAGE_SCAN_INCONCLUSIVE`). If that delete fails the verdict is still the
623
- error, and `details` carries `cleanup: 'failed'`, the `key` you passed, the `storedKey` (with any
624
- `keyPrefix`, as a bucket listing shows it) and the code of the delete failure in `cleanupError`; the
625
- failure, with what the provider said, is logged at error level as `STORAGE_SCAN_CLEANUP_FAILED`. With no `details.cleanup` the object was
645
+ error, and `details` carries `cleanup: 'failed'`, the `key` without any `keyPrefix` and the code of the delete
646
+ failure in `cleanupError`. The log gets an error entry with `event: 'STORAGE_SCAN_CLEANUP_FAILED'` and the fields `message`, `storedKey`
647
+ (the key as the provider holds it, with any `keyPrefix`, as a bucket listing shows it, so an operator can find and remove the
648
+ object) and `cleanupFailure` (why the delete failed); the `storedKey` is never in the response. With no `details.cleanup` the object was
626
649
  removed; with `cleanup: 'failed'` it is **still stored** and has to be removed by other means.
627
650
 
628
651
  ---
@@ -643,6 +666,18 @@ if (page1.nextContinuationToken) {
643
666
  }
644
667
  ```
645
668
 
669
+ | Option | Meaning |
670
+ | --- | --- |
671
+ | `prefix` | Only keys that start with it, after `keyPrefix` |
672
+ | `maxKeys` | Page size, a whole number of at least `1`; above `1000` it is lowered to `1000` (the S3 maximum). Default `1000` |
673
+ | `continuationToken` | The `nextContinuationToken` of the previous page |
674
+ | `delimiter` | Groups keys under sub-prefixes: with `'/'`, objects below `avatars/2026/` are not listed, and `avatars/2026/` is returned in `commonPrefixes` instead |
675
+ | `bucket` | Lists another bucket for this call |
676
+ | `abortSignal` | Ends the call when it aborts, see Cancelling a call |
677
+
678
+ The result is `{ objects, commonPrefixes, isTruncated, nextContinuationToken? }`. Keys and prefixes come back
679
+ relative to `keyPrefix`, so they can be passed to any other method as they are.
680
+
646
681
  ### Copy (server-side)
647
682
 
648
683
  ```typescript
@@ -652,6 +687,10 @@ const { etag } = await storage.copy({
652
687
  })
653
688
  ```
654
689
 
690
+ `sourceBucket` and `destinationBucket` default to the module's bucket and may each name another, so a copy can
691
+ cross buckets; both keys are relative to `keyPrefix`. `publicRead` and `cacheControl` set the ACL and the
692
+ `Cache-Control` of the copy.
693
+
655
694
  ### Delete / Batch Delete
656
695
 
657
696
  ```typescript
@@ -771,6 +810,8 @@ BymaxStorageModule.forRootAsync({
771
810
 
772
811
  All errors are thrown as `StorageException extends HttpException`. The response body is `{ error: { code, message, details? } }`.
773
812
 
813
+ **What `details` repeats.** Only what the caller supplied in the request that failed: the `key` (without the module's `keyPrefix`), the per-call `bucket`, the `prefix` of a `list()`, the `sourceKey` and `destKey` of a `copy()`, and the `contentType` of a refused upload (left out when it is not a valid header value). The others are strings of at most 1024 characters (S3's limit for a key) and are left out when longer or not a string. What the host set as configuration is never repeated: the module's `bucket` when the call did not name one, the `keyPrefix`, the `validation.mimeWhitelist` (a `STORAGE_MIME_NOT_ALLOWED` carries the refused `contentType` and not the list) and the `region`.
814
+
774
815
  | Code | HTTP | When |
775
816
  | -------------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
776
817
  | `STORAGE_NOT_CONFIGURED` | 503 | Credentials missing and an operation was called |
@@ -783,7 +824,7 @@ All errors are thrown as `StorageException extends HttpException`. The response
783
824
  | `STORAGE_SCAN_INFECTED` | 422 | Scanner returned `'infected'` (`details.threat`); `details.cleanup: 'failed'` means the object is still stored |
784
825
  | `STORAGE_SCAN_INCONCLUSIVE` | 422 | Scanner returned `'unknown'` and `rejectOnUnknown: true`, or a result that is not a recognised status |
785
826
  | `STORAGE_OBJECT_NOT_FOUND` | 404 | `head()`, `download()`, or `copy()` on a nonexistent key |
786
- | `STORAGE_PROVIDER_ERROR` | 502 | The provider rejected the request (`AccessControlListNotSupported`, an `AccessDenied` on an object, any other rejection). **503** when it could not be reached (connection refused or reset, DNS failure) or answered with a server error (`SlowDown`, 500, 502, 504; not 501), which is worth retrying; **500** when it refused the credentials (`InvalidAccessKeyId`, `SignatureDoesNotMatch`), a deployment fault that a retry does not fix (a `HEAD` answer carries no body and so no code: a refused `head()` stays **502** with reason `rejected`, and the credentials case is not told apart there). `details` carries `awsCode`, `httpStatus`, `requestId` and a fixed `reason` (`unreachable`, `provider-error`, `credentials-refused` or `rejected`; never the text of the driver or of the provider) plus the operation context (`op`, `bucket`, and `key` or `prefix`) |
827
+ | `STORAGE_PROVIDER_ERROR` | 502 | The provider rejected the request (`AccessControlListNotSupported`, an `AccessDenied` on an object, any other rejection). **503** when it could not be reached (connection refused or reset, DNS failure) or answered with a server error (`SlowDown`, 500, 502, 504; not 501), which is worth retrying; **500** when it refused the credentials (`InvalidAccessKeyId`, `SignatureDoesNotMatch`), a deployment fault that a retry does not fix (a `HEAD` answer carries no body and so no code: a refused `head()` stays **502** with reason `rejected`, and the credentials case is not told apart there). `details` carries `awsCode`, `httpStatus`, `requestId` and a fixed `reason` (never the text of the driver or of the provider) plus the operation context (`op`, and the `bucket`, `key` or `prefix` the caller gave; see _What `details` repeats_) |
787
828
  | `STORAGE_SIGNED_URL_TTL_INVALID` | 400 | Per-request `ttlSeconds` ≤ 0 |
788
829
  | `STORAGE_PART_TOO_SMALL` | 400 | Never thrown: a `multipart.partSizeBytes` below 5 MiB fails at init as `STORAGE_INVALID_CONFIG` |
789
830
  | `STORAGE_INVALID_PART_COUNT` | 400 | `getMultipartUploadUrls()` with `parts` not an integer from 1 to 10 000 (`details.reason`; `details.provided` only when `parts` is a finite number) |
@@ -816,6 +857,118 @@ try {
816
857
 
817
858
  ---
818
859
 
860
+ ## 📊 Observability
861
+
862
+ The main entry counts and times every operation and starts spans. The packages below are
863
+ optional peers: install what you use.
864
+
865
+ ```bash
866
+ pnpm add prom-client @bymax-one/nest-core @opentelemetry/api
867
+ ```
868
+
869
+ ### Metrics
870
+
871
+ ```ts
872
+ import { Module } from '@nestjs/common'
873
+ import { BymaxCoreModule } from '@bymax-one/nest-core'
874
+ import { BymaxStorageModule } from '@bymax-one/nest-storage'
875
+ import { StorageHealthIndicator, StorageMetrics } from '@bymax-one/nest-storage/observability'
876
+
877
+ @Module({
878
+ imports: [
879
+ BymaxCoreModule.forRoot({ metrics: { enabled: true }, health: { autoDiscover: true } }),
880
+ BymaxStorageModule.forRoot({
881
+ endpoint: process.env.S3_ENDPOINT ?? '',
882
+ region: 'us-east-1',
883
+ bucket: 'uploads',
884
+ credentials: { accessKeyId: process.env.S3_KEY ?? '', secretAccessKey: process.env.S3_SECRET ?? '' },
885
+ }),
886
+ ],
887
+ providers: [StorageMetrics, StorageHealthIndicator],
888
+ })
889
+ export class AppModule {}
890
+ ```
891
+
892
+ `@bymax-one/nest-core` finds both classes by their markers and calls `registerMetrics` on `StorageMetrics` once at bootstrap.
893
+
894
+ | Metric | Type | Labels | What it counts |
895
+ | --- | --- | --- | --- |
896
+ | `bymax_storage_operations_total` | counter | `operation`, `outcome` | Operations that ended |
897
+ | `bymax_storage_operation_duration_seconds` | histogram | `operation` | How long they took |
898
+ | `bymax_storage_bytes_uploaded_total` | counter | none | Bytes sent by uploads that succeeded |
899
+ | `bymax_storage_scans_total` | counter | `verdict` | File scans |
900
+
901
+ The labels come from closed sets, so the number of series never grows with traffic and no key, bucket, prefix or
902
+ message is ever a label:
903
+
904
+ - `operation`: `upload`, `download`, `head`, `delete`, `delete_many`, `list`, `copy`, `get_download_url`,
905
+ `get_upload_url`, `get_multipart_upload_urls`, `check_bucket`, `lifecycle_get`, `lifecycle_upsert`,
906
+ `lifecycle_remove`.
907
+ - `outcome`: `success`, `not_found` (the object was absent), `aborted` (your `abortSignal`, or an aborted multipart
908
+ upload), `error`.
909
+ - `verdict`: `clean`, `infected`, `unknown`, `timeout`, `error`.
910
+
911
+ `exists()` is counted as `head`, `downloadBuffer()` as `download`, and the delete that removes an object a post-upload scan rejected as `delete`. A download is timed to the response, not to the
912
+ last byte. A result returned from the idempotency cache counts as an upload but adds no bytes. The bytes are the ones the transfer sent: a stream without a declared `size` is counted too, and a `size` the stream did not reach is not; a failed upload adds none. The counters start
913
+ with the module; the histogram starts when `StorageMetrics` registers.
914
+
915
+ `delete_many` reports how it ended as a call, not per key: `deleteMany()` returns the keys that failed instead of
916
+ throwing, so a call in which every key failed is counted as `success`; read `failed` in its result for that. A `head`
917
+ of a bucket that does not exist is answered 404 by the provider as it is for a missing object, and is counted as
918
+ `not_found`.
919
+
920
+ `StorageMetricsTally` is what the metrics read, and the library writes to it as it works: `operation(operation,
921
+ outcome, seconds)` and `scan(verdict)` count, `bytesUploaded(bytes)` adds to the byte counter, `onDuration(listener)`
922
+ hands the listener the duration of every operation that ends from then on, and `snapshot()` returns the counts so
923
+ far as copies. A host that publishes its own metrics reads `snapshot()`.
924
+
925
+ ### Readiness
926
+
927
+ `StorageHealthIndicator` reports under the name `storage`. It asks `StorageService.checkBucket()` for one
928
+ `HeadBucket` and answers `up` or `down` with no detail, so nothing about the bucket, the region or the endpoint
929
+ leaves with it. The probe has its own deadline, the answer is kept, and checks that arrive
930
+ while one is running share it. The probe is counted as `check_bucket`. The answer lags the bucket's real state by up to the 5 seconds it is kept plus the 2 seconds the probe may take,
931
+ so a bucket that stops answering can still read `up` for up to 7 seconds.
932
+
933
+ ### Spans
934
+
935
+ Spans start through `@opentelemetry/api` when it is installed and a tracer is registered; without it the operation
936
+ runs as it is.
937
+
938
+ | Span | Around |
939
+ | --- | --- |
940
+ | `storage.upload` | `upload()` |
941
+ | `storage.download_url` | `getDownloadUrl()` |
942
+ | `storage.scan` | a file scan |
943
+
944
+ Attributes are `bymax.storage.outcome` (the outcome above), `bymax.storage.multipart` (`upload` only: `true` or
945
+ `false`) and `bymax.storage.scan_mode` (`scan` only: `pre-upload` or `post-upload`). A failed operation sets the
946
+ span status to error with the library error code, never its message.
947
+
948
+ ---
949
+
950
+ ## 📋 Log events
951
+
952
+ Every operational line the library writes is an **object** through the Nest `Logger`: an `event` code, a `message` with the library's own text, and the fields below. What the provider, the scanner or the host raised goes under `err`, as a fresh error built from its name, its provider code and its message (secrets masked, cut, control characters escaped); never into `message`, and never the SDK's own error, so no response, header or `cause` reaches a log. No field of an event carries the key you passed, the bucket, the endpoint or the region, except `storedKey` of `STORAGE_SCAN_CLEANUP_FAILED`, which an operator needs to remove the object. What the provider or the scanner wrote under `err` is their own text and can name the host, the key or the bucket. A host logger that throws does not change the call being logged. A host `LoggerService` method that is async and rejects is an unhandled rejection, and Node ends the process on it unless the application handles `unhandledRejection`: Nest's `Logger` drops the value a host method returns before the library can see it, so handle rejections inside your logger.
953
+
954
+ | Code | Level | When | Fields |
955
+ | --- | --- | --- | --- |
956
+ | `STORAGE_TTL_CLAMPED` | warn | `signedUrls.maxTtlSeconds` is above the SigV4 limit and the limit is used | `configuredSeconds`, `appliedSeconds` |
957
+ | `STORAGE_CREDENTIALS_MISSING` | warn | the module starts without credentials; operations then throw `STORAGE_NOT_CONFIGURED` | none |
958
+ | `STORAGE_CLIENT_READY` | log | the S3 client is created | none |
959
+ | `STORAGE_PROVIDER_CALL_FAILED` | warn | a call to the provider failed (a request you cancelled is not logged) | `operation`, `awsCode`, `httpStatus`, `requestId`, `reason`, `err` |
960
+ | `STORAGE_MULTIPART_ABORT_FAILED` | warn | aborting a failed multipart upload failed, so the upload stays open on the provider | `err` |
961
+ | `STORAGE_SCAN_INFECTED` | warn | the scanner said `infected` | `engine`, `threat`, `details` (the scanner's own, cut) |
962
+ | `STORAGE_SCAN_INCONCLUSIVE_ACCEPTED` | warn | the scanner said `unknown` and `rejectOnUnknown` is off | `engine` |
963
+ | `STORAGE_SCAN_INCONCLUSIVE_REJECTED` | warn | `unknown` with `rejectOnUnknown`, or a result outside the contract | `engine`, `status` (when unrecognised), `details` |
964
+ | `STORAGE_SCAN_FAILED` | warn | the scanner threw | `err` |
965
+ | `STORAGE_SCAN_TIMEOUT` | warn | the scan outlasted `scanner.timeoutMs` | `timeoutMs` |
966
+ | `STORAGE_SCAN_CLEANUP_FAILED` | error | the object a scan rejected could not be deleted and is still stored | `storedKey`, `cleanupFailure` |
967
+ | `STORAGE_EXISTS_FAILED` | warn | `exists()` met a failure that is not a 404 and answered `false` | `err` |
968
+ | `STORAGE_DELETE_NOT_FOUND` | warn | `delete()` found no such key (an idempotent no-op) | none |
969
+
970
+ ---
971
+
819
972
  ## 🚫 What This Library Does NOT Do
820
973
 
821
974
  - **Detect a file's real type.** The MIME check reads the declared `Content-Type` only; plug in an
@@ -911,12 +1064,13 @@ injected into every service, so `credentials` is attached as a non-enumerable ac
911
1064
  is what a structured logger or an error reporter capturing a scope would otherwise pick up. Reading it
912
1065
  on purpose is unchanged.
913
1066
 
914
- ### Error payloads carry the operation, and that includes the key
1067
+ ### Error payloads carry the operation, and the key you passed
915
1068
 
916
- `details` holds the provider's error code and message, the HTTP status, the request id, and the
917
- operation context the call site supplies: the operation, the bucket, and the resolved key or prefix.
918
- That is enough to diagnose a failure, and it means an object key reaches whatever consumes the
919
- exception, so if a key is itself sensitive in your deployment, do not log the envelope verbatim.
1069
+ `details` holds the provider's error code, the HTTP status, the request id, a fixed `reason`, and the
1070
+ operation context the call site supplies: the operation and the `key`, `bucket` or `prefix` the caller
1071
+ passed (see _What `details` repeats_). Neither the text of the provider nor anything the host configured is
1072
+ in it. An object key you passed still reaches whatever consumes the exception, so if a key is itself
1073
+ sensitive in your deployment, do not log the envelope verbatim.
920
1074
 
921
1075
  ### `keyPrefix` is a namespace, not an access boundary
922
1076
 
@@ -1030,7 +1184,14 @@ supply through it first: the guard below, then `keyPrefix`.
1030
1184
  `deleteMany` result all omit the prefix, so any of them can be passed straight back to `head`,
1031
1185
  `download`, `delete`, `copy` or a signed-URL call, which prepend it once. `UploadResult.publicUrl`
1032
1186
  is the one full value: it is a URL, and the prefix is part of the object's real path. Error
1033
- `details` carry the real, prefixed key, because they describe what was sent to the provider.
1187
+ `details` carry the `key` you passed, without the prefix (see "What `details` repeats").
1188
+
1189
+ **The fields of the results.** `UploadResult` holds `key`, `bucket`, `etag`, `contentType`, `publicUrl`, `multipart`
1190
+ and `fromIdempotencyCache`, and `size` and `versionId` when known (`versionId` only on a versioned bucket).
1191
+ `ObjectMetadata` (from `head` and `download`) holds `key`, `bucket`, `size`, `contentType`, `etag`, `lastModified`
1192
+ and `metadata` (the `x-amz-meta-*` pairs), and `cacheControl`, `contentDisposition`, `storageClass` and `versionId`
1193
+ when the provider reports them. A `ListedObject` holds `key`, `size`, `etag` and `lastModified`, and `storageClass`
1194
+ when reported.
1034
1195
 
1035
1196
  ### Exported types and constants
1036
1197
 
@@ -1049,6 +1210,13 @@ is the one full value: it is a URL, and the prefix is part of the object's real
1049
1210
  `LifecycleBucketOptions` (`{ bucket? }`).
1050
1211
  - **Extension points:** `IUploadValidator`, `IFileScanner`, `FileScanResult`, and the helpers
1051
1212
  `NoOpUploadValidator` and `NoOpFileScanner`.
1213
+ - **Observability (main entry):** `StorageMetricsTally`, the counts the `./observability` subpath publishes, and the
1214
+ closed sets they are keyed by: `STORAGE_OPERATIONS` and `StorageOperation`, `STORAGE_OUTCOMES` and
1215
+ `StorageOutcome`, `SCAN_VERDICTS` and `ScanVerdict`.
1216
+ - **From `./observability`:** `StorageMetrics`, `StorageHealthIndicator`, the name it reports under
1217
+ (`STORAGE_HEALTH_INDICATOR_NAME`), its deadline and cache (`STORAGE_HEALTH_TIMEOUT_MS`,
1218
+ `STORAGE_HEALTH_CACHE_MS`), and the metadata keys `@bymax-one/nest-core` discovers them by
1219
+ (`METRICS_CONTRIBUTOR_METADATA`, `HEALTH_INDICATOR_METADATA`).
1052
1220
  - **From `./shared`:** `UploadResult`, `ObjectMetadata`, `ListedObject`, `SignedUrlResult`,
1053
1221
  `StorageErrorResponse`, `StorageErrorCode`, `STORAGE_ERROR_CODES`, the MIME presets
1054
1222
  `DEFAULT_IMAGE_MIME_WHITELIST`, `DEFAULT_VIDEO_MIME_WHITELIST`, `DEFAULT_DOC_MIME_WHITELIST`, and the