@lenne.tech/nest-server 11.32.4 → 11.33.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/rules/architecture.md +16 -0
- package/.claude/rules/configurable-features.md +11 -3
- package/.claude/rules/testing.md +101 -1
- package/CLAUDE.md +5 -0
- package/FRAMEWORK-API.md +58 -3
- package/dist/config.env.js +6 -0
- package/dist/config.env.js.map +1 -1
- package/dist/core/common/helpers/content-disposition.helper.d.ts +1 -0
- package/dist/core/common/helpers/content-disposition.helper.js +14 -0
- package/dist/core/common/helpers/content-disposition.helper.js.map +1 -0
- package/dist/core/common/helpers/file.helper.d.ts +7 -0
- package/dist/core/common/helpers/file.helper.js +21 -4
- package/dist/core/common/helpers/file.helper.js.map +1 -1
- package/dist/core/common/helpers/graceful-shutdown.helper.d.ts +2 -0
- package/dist/core/common/helpers/graceful-shutdown.helper.js +55 -0
- package/dist/core/common/helpers/graceful-shutdown.helper.js.map +1 -0
- package/dist/core/common/helpers/project-name.helper.d.ts +2 -0
- package/dist/core/common/helpers/project-name.helper.js +38 -0
- package/dist/core/common/helpers/project-name.helper.js.map +1 -0
- package/dist/core/common/interfaces/cron-job-config.interface.d.ts +1 -0
- package/dist/core/common/interfaces/server-options.interface.d.ts +39 -0
- package/dist/core/common/services/core-cron-jobs.initializer.d.ts +10 -0
- package/dist/core/common/services/core-cron-jobs.initializer.js +46 -0
- package/dist/core/common/services/core-cron-jobs.initializer.js.map +1 -0
- package/dist/core/common/services/core-cron-jobs.registry.d.ts +8 -0
- package/dist/core/common/services/core-cron-jobs.registry.js +12 -0
- package/dist/core/common/services/core-cron-jobs.registry.js.map +1 -0
- package/dist/core/common/services/core-cron-jobs.service.d.ts +37 -7
- package/dist/core/common/services/core-cron-jobs.service.js +278 -27
- package/dist/core/common/services/core-cron-jobs.service.js.map +1 -1
- package/dist/core/common/services/core-redis-pubsub.d.ts +20 -0
- package/dist/core/common/services/core-redis-pubsub.js +112 -0
- package/dist/core/common/services/core-redis-pubsub.js.map +1 -0
- package/dist/core/common/services/core-redis.service.d.ts +25 -0
- package/dist/core/common/services/core-redis.service.js +154 -0
- package/dist/core/common/services/core-redis.service.js.map +1 -0
- package/dist/core/common/services/core-s3.service.d.ts +51 -0
- package/dist/core/common/services/core-s3.service.js +261 -0
- package/dist/core/common/services/core-s3.service.js.map +1 -0
- package/dist/core/common/services/core-trust-proxy.initializer.d.ts +16 -0
- package/dist/core/common/services/core-trust-proxy.initializer.js +77 -0
- package/dist/core/common/services/core-trust-proxy.initializer.js.map +1 -0
- package/dist/core/common/services/rate-limit-store.d.ts +61 -0
- package/dist/core/common/services/rate-limit-store.js +235 -0
- package/dist/core/common/services/rate-limit-store.js.map +1 -0
- package/dist/core/modules/ai/core-ai-mcp.controller.d.ts +19 -3
- package/dist/core/modules/ai/core-ai-mcp.controller.js +148 -18
- package/dist/core/modules/ai/core-ai-mcp.controller.js.map +1 -1
- package/dist/core/modules/ai/services/core-ai.service.d.ts +7 -2
- package/dist/core/modules/ai/services/core-ai.service.js +24 -20
- package/dist/core/modules/ai/services/core-ai.service.js.map +1 -1
- package/dist/core/modules/auth/core-auth.module.js +4 -1
- package/dist/core/modules/auth/core-auth.module.js.map +1 -1
- package/dist/core/modules/auth/guards/legacy-auth-rate-limit.guard.d.ts +1 -1
- package/dist/core/modules/auth/guards/legacy-auth-rate-limit.guard.js +3 -14
- package/dist/core/modules/auth/guards/legacy-auth-rate-limit.guard.js.map +1 -1
- package/dist/core/modules/auth/services/legacy-auth-rate-limiter.service.d.ts +12 -10
- package/dist/core/modules/auth/services/legacy-auth-rate-limiter.service.js +30 -59
- package/dist/core/modules/auth/services/legacy-auth-rate-limiter.service.js.map +1 -1
- package/dist/core/modules/better-auth/core-better-auth-email-verification.service.d.ts +8 -2
- package/dist/core/modules/better-auth/core-better-auth-email-verification.service.js +118 -45
- package/dist/core/modules/better-auth/core-better-auth-email-verification.service.js.map +1 -1
- package/dist/core/modules/better-auth/core-better-auth-rate-limit.middleware.d.ts +1 -1
- package/dist/core/modules/better-auth/core-better-auth-rate-limit.middleware.js +2 -11
- package/dist/core/modules/better-auth/core-better-auth-rate-limit.middleware.js.map +1 -1
- package/dist/core/modules/better-auth/core-better-auth-rate-limiter.service.d.ts +12 -10
- package/dist/core/modules/better-auth/core-better-auth-rate-limiter.service.js +30 -85
- package/dist/core/modules/better-auth/core-better-auth-rate-limiter.service.js.map +1 -1
- package/dist/core/modules/better-auth/core-better-auth-user.mapper.js +0 -1
- package/dist/core/modules/better-auth/core-better-auth-user.mapper.js.map +1 -1
- package/dist/core/modules/file/core-file.controller.d.ts +11 -2
- package/dist/core/modules/file/core-file.controller.js +52 -15
- package/dist/core/modules/file/core-file.controller.js.map +1 -1
- package/dist/core/modules/file/core-file.resolver.d.ts +4 -4
- package/dist/core/modules/file/core-file.resolver.js +23 -16
- package/dist/core/modules/file/core-file.resolver.js.map +1 -1
- package/dist/core/modules/file/core-file.service.d.ts +41 -7
- package/dist/core/modules/file/core-file.service.js +228 -8
- package/dist/core/modules/file/core-file.service.js.map +1 -1
- package/dist/core/modules/file/file-metadata.helper.d.ts +16 -0
- package/dist/core/modules/file/file-metadata.helper.js +33 -0
- package/dist/core/modules/file/file-metadata.helper.js.map +1 -0
- package/dist/core/modules/file/file-roles.helper.d.ts +4 -0
- package/dist/core/modules/file/file-roles.helper.js +53 -0
- package/dist/core/modules/file/file-roles.helper.js.map +1 -0
- package/dist/core/modules/file/file-storage.helper.d.ts +13 -0
- package/dist/core/modules/file/file-storage.helper.js +65 -0
- package/dist/core/modules/file/file-storage.helper.js.map +1 -0
- package/dist/core/modules/file/filesystem-file.helper.d.ts +22 -0
- package/dist/core/modules/file/filesystem-file.helper.js +77 -0
- package/dist/core/modules/file/filesystem-file.helper.js.map +1 -0
- package/dist/core/modules/file/interfaces/file-service-options.interface.d.ts +1 -0
- package/dist/core/modules/file/interfaces/file-upload.interface.d.ts +7 -1
- package/dist/core/modules/file/s3-file.helper.d.ts +37 -0
- package/dist/core/modules/file/s3-file.helper.js +81 -0
- package/dist/core/modules/file/s3-file.helper.js.map +1 -0
- package/dist/core/modules/hub/core-hub.controller.d.ts +8 -8
- package/dist/core/modules/hub/core-hub.controller.js +24 -22
- package/dist/core/modules/hub/core-hub.controller.js.map +1 -1
- package/dist/core/modules/hub/helpers/hub-mask.helper.js +2 -2
- package/dist/core/modules/hub/helpers/hub-mask.helper.js.map +1 -1
- package/dist/core/modules/hub/hub-buffer.d.ts +23 -0
- package/dist/core/modules/hub/hub-buffer.js +101 -0
- package/dist/core/modules/hub/hub-buffer.js.map +1 -0
- package/dist/core/modules/hub/services/core-hub-mailbox.service.d.ts +7 -5
- package/dist/core/modules/hub/services/core-hub-mailbox.service.js +14 -10
- package/dist/core/modules/hub/services/core-hub-mailbox.service.js.map +1 -1
- package/dist/core/modules/hub/services/hub-log-buffer.service.d.ts +5 -2
- package/dist/core/modules/hub/services/hub-log-buffer.service.js +17 -8
- package/dist/core/modules/hub/services/hub-log-buffer.service.js.map +1 -1
- package/dist/core/modules/hub/services/hub-query-profiler.service.d.ts +4 -2
- package/dist/core/modules/hub/services/hub-query-profiler.service.js +13 -7
- package/dist/core/modules/hub/services/hub-query-profiler.service.js.map +1 -1
- package/dist/core/modules/hub/services/hub-trace-buffer.service.d.ts +4 -2
- package/dist/core/modules/hub/services/hub-trace-buffer.service.js +13 -8
- package/dist/core/modules/hub/services/hub-trace-buffer.service.js.map +1 -1
- package/dist/core/modules/migrate/helpers/migration.helper.d.ts +2 -6
- package/dist/core/modules/migrate/helpers/migration.helper.js +4 -4
- package/dist/core/modules/migrate/helpers/migration.helper.js.map +1 -1
- package/dist/core/modules/migrate/migration-runner.d.ts +1 -0
- package/dist/core/modules/migrate/migration-runner.js +4 -0
- package/dist/core/modules/migrate/migration-runner.js.map +1 -1
- package/dist/core/modules/migrate/mongo-state-store.d.ts +1 -0
- package/dist/core/modules/migrate/mongo-state-store.js +98 -9
- package/dist/core/modules/migrate/mongo-state-store.js.map +1 -1
- package/dist/core/modules/system-setup/core-system-setup.service.d.ts +2 -0
- package/dist/core/modules/system-setup/core-system-setup.service.js +31 -1
- package/dist/core/modules/system-setup/core-system-setup.service.js.map +1 -1
- package/dist/core/modules/tenant/core-tenant.guard.d.ts +17 -3
- package/dist/core/modules/tenant/core-tenant.guard.js +92 -15
- package/dist/core/modules/tenant/core-tenant.guard.js.map +1 -1
- package/dist/core/modules/tenant/core-tenant.module.js +3 -4
- package/dist/core/modules/tenant/core-tenant.module.js.map +1 -1
- package/dist/core/modules/tus/core-tus.controller.d.ts +2 -0
- package/dist/core/modules/tus/core-tus.controller.js +29 -3
- package/dist/core/modules/tus/core-tus.controller.js.map +1 -1
- package/dist/core/modules/tus/core-tus.service.d.ts +24 -1
- package/dist/core/modules/tus/core-tus.service.js +182 -28
- package/dist/core/modules/tus/core-tus.service.js.map +1 -1
- package/dist/core/modules/tus/interfaces/tus-config.interface.js +3 -0
- package/dist/core/modules/tus/interfaces/tus-config.interface.js.map +1 -1
- package/dist/core/modules/tus/tus-redis-locker.d.ts +25 -0
- package/dist/core/modules/tus/tus-redis-locker.js +87 -0
- package/dist/core/modules/tus/tus-redis-locker.js.map +1 -0
- package/dist/core/modules/tus/tus.module.d.ts +1 -0
- package/dist/core/modules/tus/tus.module.js +30 -4
- package/dist/core/modules/tus/tus.module.js.map +1 -1
- package/dist/core.module.js +18 -1
- package/dist/core.module.js.map +1 -1
- package/dist/index.d.ts +10 -0
- package/dist/index.js +10 -0
- package/dist/index.js.map +1 -1
- package/dist/main.js +2 -1
- package/dist/main.js.map +1 -1
- package/dist/server/modules/file/file.controller.js +2 -2
- package/dist/server/modules/file/file.controller.js.map +1 -1
- package/dist/server/modules/file/file.resolver.js +3 -14
- package/dist/server/modules/file/file.resolver.js.map +1 -1
- package/dist/server/modules/file/file.service.d.ts +10 -2
- package/dist/server/modules/file/file.service.js +23 -3
- package/dist/server/modules/file/file.service.js.map +1 -1
- package/dist/server/modules/user/avatar.controller.d.ts +5 -1
- package/dist/server/modules/user/avatar.controller.js +26 -10
- package/dist/server/modules/user/avatar.controller.js.map +1 -1
- package/dist/server/modules/user/user.model.js +3 -2
- package/dist/server/modules/user/user.model.js.map +1 -1
- package/dist/server/modules/user/user.module.js +6 -2
- package/dist/server/modules/user/user.module.js.map +1 -1
- package/dist/server/modules/user/user.service.d.ts +1 -1
- package/dist/server/modules/user/user.service.js +6 -13
- package/dist/server/modules/user/user.service.js.map +1 -1
- package/dist/tsconfig.build.tsbuildinfo +1 -1
- package/docs/REQUEST-LIFECYCLE.md +57 -8
- package/docs/security-overrides.md +21 -3
- package/migration-guides/11.32.3-to-11.32.4.md +15 -3
- package/migration-guides/11.32.x-to-11.33.x.md +1584 -0
- package/package.json +74 -42
- package/src/config.env.ts +45 -0
- package/src/core/common/helpers/content-disposition.helper.ts +90 -0
- package/src/core/common/helpers/file.helper.ts +53 -10
- package/src/core/common/helpers/graceful-shutdown.helper.ts +116 -0
- package/src/core/common/helpers/project-name.helper.ts +71 -0
- package/src/core/common/interfaces/cron-job-config.interface.ts +16 -0
- package/src/core/common/interfaces/server-options.interface.ts +439 -2
- package/src/core/common/services/core-cron-jobs.initializer.ts +39 -0
- package/src/core/common/services/core-cron-jobs.registry.ts +45 -0
- package/src/core/common/services/core-cron-jobs.service.ts +589 -47
- package/src/core/common/services/core-redis-pubsub.ts +183 -0
- package/src/core/common/services/core-redis.service.ts +249 -0
- package/src/core/common/services/core-s3.service.ts +465 -0
- package/src/core/common/services/core-trust-proxy.initializer.ts +112 -0
- package/src/core/common/services/rate-limit-store.ts +492 -0
- package/src/core/modules/ai/README.md +25 -0
- package/src/core/modules/ai/core-ai-mcp.controller.ts +253 -22
- package/src/core/modules/ai/services/core-ai.service.ts +33 -24
- package/src/core/modules/auth/core-auth.module.ts +7 -1
- package/src/core/modules/auth/guards/legacy-auth-rate-limit.guard.ts +12 -21
- package/src/core/modules/auth/services/legacy-auth-rate-limiter.service.ts +49 -93
- package/src/core/modules/better-auth/core-better-auth-email-verification.service.ts +202 -66
- package/src/core/modules/better-auth/core-better-auth-rate-limit.middleware.ts +11 -18
- package/src/core/modules/better-auth/core-better-auth-rate-limiter.service.ts +51 -132
- package/src/core/modules/better-auth/core-better-auth-user.mapper.ts +7 -1
- package/src/core/modules/file/INTEGRATION-CHECKLIST.md +120 -0
- package/src/core/modules/file/README.md +308 -53
- package/src/core/modules/file/core-file.controller.ts +168 -12
- package/src/core/modules/file/core-file.resolver.ts +46 -12
- package/src/core/modules/file/core-file.service.ts +517 -17
- package/src/core/modules/file/file-metadata.helper.ts +142 -0
- package/src/core/modules/file/file-roles.helper.ts +110 -0
- package/src/core/modules/file/file-storage.helper.ts +163 -0
- package/src/core/modules/file/filesystem-file.helper.ts +184 -0
- package/src/core/modules/file/interfaces/file-service-options.interface.ts +14 -0
- package/src/core/modules/file/interfaces/file-upload.interface.ts +17 -1
- package/src/core/modules/file/s3-file.helper.ts +209 -0
- package/src/core/modules/hub/README.md +48 -0
- package/src/core/modules/hub/core-hub.controller.ts +18 -16
- package/src/core/modules/hub/helpers/hub-mask.helper.ts +21 -4
- package/src/core/modules/hub/hub-buffer.ts +198 -0
- package/src/core/modules/hub/services/core-hub-mailbox.service.ts +22 -12
- package/src/core/modules/hub/services/hub-log-buffer.service.ts +24 -8
- package/src/core/modules/hub/services/hub-query-profiler.service.ts +9 -6
- package/src/core/modules/hub/services/hub-trace-buffer.service.ts +14 -9
- package/src/core/modules/migrate/README.md +33 -6
- package/src/core/modules/migrate/helpers/migration.helper.ts +13 -6
- package/src/core/modules/migrate/migration-runner.ts +10 -1
- package/src/core/modules/migrate/mongo-state-store.ts +179 -11
- package/src/core/modules/system-setup/README.md +7 -2
- package/src/core/modules/system-setup/core-system-setup.service.ts +95 -1
- package/src/core/modules/tenant/README.md +21 -1
- package/src/core/modules/tenant/core-tenant.guard.ts +135 -14
- package/src/core/modules/tenant/core-tenant.module.ts +8 -4
- package/src/core/modules/tus/INTEGRATION-CHECKLIST.md +93 -21
- package/src/core/modules/tus/README.md +35 -2
- package/src/core/modules/tus/core-tus.controller.ts +64 -12
- package/src/core/modules/tus/core-tus.service.ts +358 -34
- package/src/core/modules/tus/interfaces/tus-config.interface.ts +9 -0
- package/src/core/modules/tus/tus-redis-locker.ts +139 -0
- package/src/core/modules/tus/tus.module.ts +78 -6
- package/src/core.module.ts +26 -1
- package/src/index.ts +10 -0
- package/src/main.ts +7 -1
- package/src/server/modules/file/file.controller.ts +14 -5
- package/src/server/modules/file/file.resolver.ts +11 -19
- package/src/server/modules/file/file.service.ts +89 -4
- package/src/server/modules/user/avatar.controller.ts +46 -9
- package/src/server/modules/user/user.model.ts +9 -3
- package/src/server/modules/user/user.module.ts +12 -3
- package/src/server/modules/user/user.service.ts +20 -17
- package/src/test/README.md +7 -2
- package/src/core/common/helpers/file.helper.spec.ts +0 -145
- package/src/core/common/helpers/logging.helper.spec.ts +0 -61
- package/src/core/common/helpers/process-diagnostics.helper.spec.ts +0 -310
- package/src/core/common/services/brevo.service.spec.ts +0 -266
- package/src/core/modules/file/core-file.controller.spec.ts +0 -164
- package/src/core/modules/hub/core-hub.module.spec.ts +0 -108
- package/src/core/modules/hub/helpers/hub-command-shape.helper.spec.ts +0 -48
- package/src/core/modules/hub/helpers/hub-mask.helper.spec.ts +0 -67
- package/src/core/modules/hub/helpers/hub-mermaid.helper.spec.ts +0 -61
- package/src/core/modules/hub/helpers/hub-shell.helper.spec.ts +0 -106
- package/src/core/modules/hub/hub-config.helper.spec.ts +0 -108
- package/src/core/modules/hub/hub-ring-buffer.spec.ts +0 -95
- package/src/core/modules/hub/services/core-hub-mailbox.service.spec.ts +0 -116
- package/src/core/modules/hub/services/hub-trace-buffer.service.spec.ts +0 -112
- package/src/core/modules/migrate/helpers/migration.helper.spec.ts +0 -85
|
@@ -0,0 +1,1584 @@
|
|
|
1
|
+
# Migration Guide: 11.32.x → 11.33.x
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
|
|
5
|
+
| Category | Details |
|
|
6
|
+
|----------|---------|
|
|
7
|
+
| **Breaking Changes** | **File downloads, the file GraphQL members and TUS uploads are no longer public (§A, §B) — this one affects every project, not just multi-replica ones.** `CoreBetterAuthRateLimiter` and `LegacyAuthRateLimiter`: `check()` / `reset()` / `clear()` are **async** now (§1). Hub collector reads are async (§2). `getFileStream()` / `getFileStreamByName()` are typed `Readable` instead of `GridFSBucketReadStream` (§3) |
|
|
8
|
+
| **New Features** | `file.downloadRoles` / `uploadRoles` / `deleteRoles` and `tus.roles`; per-file authorization via `checkRights()` with real metadata support (§A). Optional central `redis` (rate limits, cron dedup, GraphQL subscriptions, tenant-cache invalidation, MCP session registry, Hub collectors); optional `s3`; three file storage drivers via `file.storage` (`'filesystem'` is new) with a derived default and fail-fast enforcement; `tus.s3Staging`; `shutdownDelayMs`; `trustProxy` (§11c — **configure it if you rate-limit behind a reverse proxy**); per-job cron `distributed`; `multerFileToUpload()` for REST uploads into central storage |
|
|
9
|
+
| **Behaviour Changes** | The migration lock is **on by default** for `migrate up` (§4). Initial-admin auto-creation takes an atomic claim (§5). The `src/server` reference implementation stores avatars and GraphQL uploads in the central file storage instead of pod-local disk (§10) |
|
|
10
|
+
| **Bugfixes** | Migration lock gained heartbeat + stale-break + bounded wait; `runOnInit` cron dedup no longer keyed on each replica's boot clock and no longer crashes the process on a failed startup run; BullMQ worker no longer swallows a tick delivered during bootstrap; rate limiters degrade instead of 500-ing on a Redis outage; Hub buffer mirror failures no longer recurse through the Logger they buffer; the email-verification cooldown timer can no longer expire a later cooldown early; a presigned S3 download no longer serves the file under a different name than the streamed one (§14) |
|
|
11
|
+
| **Under the hood (§14)** | Two new MongoDB collections (`s3-files`, `filesystem-files` — include them in backups), new public exports, a bucket-less `s3` block is now ignored, NestJS 11.1.28 → 11.1.29, plus two dependency MAJORS: `js-sha256` 0.12.0 → 1.0.0 (**digests unchanged — no password-hash break**) and `graphql-query-complexity` 1.1.1 → 2.0.0 |
|
|
12
|
+
| **Dependency Housekeeping** | The framework's own `pnpm audit` went from 20 findings to 0. Those are `pnpm-workspace.yaml` overrides, which **do not reach npm consumers** — run your own audit (§13) |
|
|
13
|
+
| **Migration Effort** | **Start with §A and §B — they are the only changes that can break a single-replica project, and they break it silently** (no crash, just 401/403 on `/files/**` and `/tus`). Then **§11a / §11b if your config already contains a `redis` or an `s3` key** — the framework reads both now, and an existing `s3` key makes the boot FAIL until you act. Everything else is opt-in: read §1 if you call or override the rate limiters, §2 if you override Hub services, §4 before your next multi-replica deploy, §10 if you copied the avatar upload from `src/server`, **§11c if you rate-limit behind a reverse proxy** (a boot warning will tell you), §13 for what the security overrides do and do not do for you |
|
|
14
|
+
|
|
15
|
+
Most of this release exists to make one deployment shape possible: **more than one replica**. Nothing there is required to keep running on one.
|
|
16
|
+
|
|
17
|
+
**Two changes are different and apply to every project: §A and §B.** The file endpoints and TUS were
|
|
18
|
+
public by default and are not any more. Read those two first.
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Quick Migration
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
pnpm update @lenne.tech/nest-server@11.33.0
|
|
26
|
+
pnpm run build
|
|
27
|
+
pnpm test
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
No configuration change is required for the multi-replica features, and no new package needs
|
|
31
|
+
installing unless you opt into one (see §12). **§A and §B are the exception** — if your project
|
|
32
|
+
serves files to non-admins or accepts anonymous uploads, they need action.
|
|
33
|
+
|
|
34
|
+
**Vendor-mode projects, before you start:** if your vendored baseline is older than 11.32.0, check
|
|
35
|
+
your `@getbrevo/brevo` pin. The core's `common/services/brevo.service.ts` was rewritten for the
|
|
36
|
+
Brevo SDK **v6** (`BrevoClient`, `transactionalEmails.sendTransacEmail()`); none of that API exists
|
|
37
|
+
in v3/v5, so syncing the file onto an older pin leaves a core that neither type-checks nor runs.
|
|
38
|
+
Bump to `6.0.2` in the same change. This bites only vendor mode — npm consumers get the version
|
|
39
|
+
through the package's own dependency. Found by syncing a real project pinned at `3.0.4`.
|
|
40
|
+
|
|
41
|
+
**Vendor-mode projects:** this is a large sync. All **14** newly added code files under `src/core/`:
|
|
42
|
+
|
|
43
|
+
```
|
|
44
|
+
common/helpers/graceful-shutdown.helper.ts
|
|
45
|
+
common/helpers/project-name.helper.ts
|
|
46
|
+
common/services/core-cron-jobs.initializer.ts
|
|
47
|
+
common/services/core-cron-jobs.registry.ts
|
|
48
|
+
common/services/core-redis-pubsub.ts
|
|
49
|
+
common/services/core-redis.service.ts
|
|
50
|
+
common/services/core-s3.service.ts
|
|
51
|
+
common/services/rate-limit-store.ts
|
|
52
|
+
modules/file/file-roles.helper.ts
|
|
53
|
+
modules/file/file-storage.helper.ts
|
|
54
|
+
modules/file/filesystem-file.helper.ts
|
|
55
|
+
modules/file/s3-file.helper.ts
|
|
56
|
+
modules/hub/hub-buffer.ts
|
|
57
|
+
modules/tus/tus-redis-locker.ts
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Plus one new documentation file, `modules/file/INTEGRATION-CHECKLIST.md`, which carries no code and
|
|
61
|
+
can be taken separately.
|
|
62
|
+
|
|
63
|
+
All **33** modified code files under `src/core/` (plus the two top-level files `src/index.ts` and
|
|
64
|
+
`src/core.module.ts`):
|
|
65
|
+
|
|
66
|
+
```
|
|
67
|
+
common/helpers/file.helper.ts
|
|
68
|
+
common/interfaces/cron-job-config.interface.ts
|
|
69
|
+
common/interfaces/server-options.interface.ts
|
|
70
|
+
common/services/core-cron-jobs.service.ts
|
|
71
|
+
modules/ai/core-ai-mcp.controller.ts
|
|
72
|
+
modules/ai/services/core-ai.service.ts
|
|
73
|
+
modules/auth/core-auth.module.ts
|
|
74
|
+
modules/auth/guards/legacy-auth-rate-limit.guard.ts
|
|
75
|
+
modules/auth/services/legacy-auth-rate-limiter.service.ts
|
|
76
|
+
modules/better-auth/core-better-auth-email-verification.service.ts
|
|
77
|
+
modules/better-auth/core-better-auth-rate-limit.middleware.ts
|
|
78
|
+
modules/better-auth/core-better-auth-rate-limiter.service.ts
|
|
79
|
+
modules/better-auth/core-better-auth-user.mapper.ts
|
|
80
|
+
modules/file/core-file.controller.ts
|
|
81
|
+
modules/file/core-file.resolver.ts
|
|
82
|
+
modules/file/core-file.service.ts
|
|
83
|
+
modules/file/interfaces/file-service-options.interface.ts
|
|
84
|
+
modules/file/interfaces/file-upload.interface.ts
|
|
85
|
+
modules/hub/core-hub.controller.ts
|
|
86
|
+
modules/hub/services/core-hub-mailbox.service.ts
|
|
87
|
+
modules/hub/services/hub-log-buffer.service.ts
|
|
88
|
+
modules/hub/services/hub-query-profiler.service.ts
|
|
89
|
+
modules/hub/services/hub-trace-buffer.service.ts
|
|
90
|
+
modules/migrate/helpers/migration.helper.ts
|
|
91
|
+
modules/migrate/migration-runner.ts
|
|
92
|
+
modules/migrate/mongo-state-store.ts
|
|
93
|
+
modules/system-setup/core-system-setup.service.ts
|
|
94
|
+
modules/tenant/core-tenant.guard.ts
|
|
95
|
+
modules/tenant/core-tenant.module.ts
|
|
96
|
+
modules/tus/core-tus.controller.ts
|
|
97
|
+
modules/tus/core-tus.service.ts
|
|
98
|
+
modules/tus/interfaces/tus-config.interface.ts
|
|
99
|
+
modules/tus/tus.module.ts
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Six `README.md` files also changed (`modules/ai`, `modules/file`, `modules/hub`, `modules/migrate`,
|
|
103
|
+
`modules/system-setup`, `modules/tus`). Those are documentation-only and safe to take separately.
|
|
104
|
+
|
|
105
|
+
**Also removed from `src/core/`:** every co-located `*.spec.ts` (they moved to `tests/unit/`, since
|
|
106
|
+
`src/` is what ships in the tarball and what vendor mode copies verbatim). A sync that only *adds*
|
|
107
|
+
and *updates* leaves those orphans behind in your tree, where they reference helpers that no longer
|
|
108
|
+
sit next to them. Delete any `src/core/**/*.spec.ts` your baseline still has.
|
|
109
|
+
|
|
110
|
+
**This sync has an atomic file-set hazard**, and the four files most easily missed are exactly the
|
|
111
|
+
ones that break the build:
|
|
112
|
+
|
|
113
|
+
| Missing file | Breaks |
|
|
114
|
+
|--------------|--------|
|
|
115
|
+
| `common/helpers/project-name.helper.ts` | `common/services/core-redis.service.ts` (default `keyPrefix`) |
|
|
116
|
+
| `modules/file/file-storage.helper.ts` | `modules/file/core-file.service.ts`, `modules/tus/core-tus.service.ts` |
|
|
117
|
+
| `modules/file/filesystem-file.helper.ts` | `modules/file/core-file.service.ts`, `modules/tus/core-tus.service.ts` |
|
|
118
|
+
| `modules/file/file-roles.helper.ts` | `src/core.module.ts` |
|
|
119
|
+
|
|
120
|
+
The same applies to the services: adopting
|
|
121
|
+
`modules/auth/services/legacy-auth-rate-limiter.service.ts` without
|
|
122
|
+
`common/services/rate-limit-store.ts` and `common/services/core-redis.service.ts` does not compile.
|
|
123
|
+
Take the whole code set or none of it.
|
|
124
|
+
|
|
125
|
+
---
|
|
126
|
+
|
|
127
|
+
## A. Breaking: file downloads, uploads and the GraphQL members are role-gated
|
|
128
|
+
|
|
129
|
+
Six members moved from `@Roles(RoleEnum.S_EVERYONE)` to a configurable role list, defaulting to
|
|
130
|
+
`[RoleEnum.ADMIN]`:
|
|
131
|
+
|
|
132
|
+
| Member | Kind | Now gated by |
|
|
133
|
+
|--------|------|--------------|
|
|
134
|
+
| `GET /files/id/:id` | REST | `file.downloadRoles` |
|
|
135
|
+
| `GET /files/:filename` | REST | `file.downloadRoles` |
|
|
136
|
+
| `getFileInfo` | GraphQL query | `file.downloadRoles` |
|
|
137
|
+
| `uploadFile` | GraphQL mutation | `file.uploadRoles` |
|
|
138
|
+
| `uploadFiles` | GraphQL mutation | `file.uploadRoles` |
|
|
139
|
+
| `deleteFile` | GraphQL mutation | `file.deleteRoles` |
|
|
140
|
+
|
|
141
|
+
`S_EVERYONE` was not merely permissive. The roles guard treats it as an unconditional `return true`
|
|
142
|
+
**before authenticating at all**, and because handler and class roles are UNIONed rather than
|
|
143
|
+
overridden, the handler-level `S_EVERYONE` silently defeated the class-level `@Roles(ADMIN)` both
|
|
144
|
+
classes already carried. `deleteFile` was reachable anonymously.
|
|
145
|
+
|
|
146
|
+
**Symptom if you skip this:** no crash, no error — `401`/`403` on `/files/**`, showing up as broken
|
|
147
|
+
images and empty avatars.
|
|
148
|
+
|
|
149
|
+
### A.1 — Check your own controller FIRST
|
|
150
|
+
|
|
151
|
+
If your `FileController` re-declares the roles, your downloads were never affected by the old
|
|
152
|
+
default and are still public today:
|
|
153
|
+
|
|
154
|
+
```typescript
|
|
155
|
+
// If you find THIS, the config below will not reach these routes:
|
|
156
|
+
@Get('id/:id')
|
|
157
|
+
@Roles(RoleEnum.S_EVERYONE)
|
|
158
|
+
override async getFileById(@Param('id') id: string, @Res() res: Response): Promise<Response> {
|
|
159
|
+
return super.getFileById(id, res);
|
|
160
|
+
}
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
Role metadata lives on the **function object**, so an override carries its own and opts out of
|
|
164
|
+
`file.downloadRoles` permanently. Delete such overrides and configure the roles instead.
|
|
165
|
+
|
|
166
|
+
Two further consequences:
|
|
167
|
+
|
|
168
|
+
- An override that omits `@Get(...)` **unregisters the route** — Nest finds no `PATH_METADATA` on the
|
|
169
|
+
subclass function and never maps it. 404 for everyone, no error, no warning.
|
|
170
|
+
- A class-level `@Roles()` on your subclass **cannot** relax an inherited member, because the
|
|
171
|
+
inherited function carries its own handler-level roles and the two are unioned.
|
|
172
|
+
|
|
173
|
+
`nest-server-starter` shipped exactly this override — added only to attach Swagger annotations — and
|
|
174
|
+
is corrected in the same release.
|
|
175
|
+
|
|
176
|
+
### A.2 — Configure
|
|
177
|
+
|
|
178
|
+
```typescript
|
|
179
|
+
// src/config.env.ts
|
|
180
|
+
file: {
|
|
181
|
+
downloadRoles: [RoleEnum.ADMIN], // default
|
|
182
|
+
uploadRoles: [RoleEnum.ADMIN], // default
|
|
183
|
+
deleteRoles: [RoleEnum.ADMIN], // default
|
|
184
|
+
},
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
Plain role strings, so project-specific roles work: `downloadRoles: ['company-admin', 'editor']`.
|
|
188
|
+
|
|
189
|
+
- **ADMIN always keeps access** (the class-level `@Roles(ADMIN)` is unioned in). These knobs cannot
|
|
190
|
+
exclude admins.
|
|
191
|
+
- **`[]` is rejected** with a warning and the default applies — an all-empty role set reads to the
|
|
192
|
+
guards as "no roles required" and would open the route.
|
|
193
|
+
- Roles resolve against `user.roles`, never `membership.role`: both classes now carry
|
|
194
|
+
`@SkipTenantCheck()`. GridFS and the S3 metadata collection are reached outside Mongoose, so
|
|
195
|
+
`mongooseTenantPlugin` never scopes them — a role name alone cannot express a per-tenant rule.
|
|
196
|
+
|
|
197
|
+
### A.3 — The better answer: authorize per file
|
|
198
|
+
|
|
199
|
+
The real requirement is usually "which files", not "which roles". Write an owner at upload time and
|
|
200
|
+
decide per file:
|
|
201
|
+
|
|
202
|
+
```typescript
|
|
203
|
+
await this.fileService.createFile(file, { metadata: { ownerId: currentUser.id } });
|
|
204
|
+
|
|
205
|
+
export class FileService extends CoreFileService {
|
|
206
|
+
protected override async checkRights(
|
|
207
|
+
input: any,
|
|
208
|
+
options?: FileServiceOptions & { checkInputType: FileInputCheckType },
|
|
209
|
+
): Promise<boolean> {
|
|
210
|
+
// Writes, list queries and forced (system) calls stay on the coarse role gate
|
|
211
|
+
if (options?.force || (options?.checkInputType !== 'filename' && options?.checkInputType !== 'id')) {
|
|
212
|
+
return true;
|
|
213
|
+
}
|
|
214
|
+
if (options.currentUser?.hasRole?.([RoleEnum.ADMIN])) {
|
|
215
|
+
return true;
|
|
216
|
+
}
|
|
217
|
+
const raw = options.checkInputType === 'id'
|
|
218
|
+
? await this.getRawFileInfo(input)
|
|
219
|
+
: await this.getRawFileInfoByName(input);
|
|
220
|
+
// Fails closed without a user, and on a file that records no owner.
|
|
221
|
+
return !!raw?.metadata?.ownerId && String(raw.metadata.ownerId) === String(options.currentUser?.id);
|
|
222
|
+
}
|
|
223
|
+
}
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
**Cover the `filename` branch, not only `id`.** An id-only rule is enough while bytes are streamed —
|
|
227
|
+
the filename route resolves an id and checks it again — but not once `s3.presignedDownloads` is on
|
|
228
|
+
(the filename route then authorizes on the by-name lookup alone and redirects), and not for
|
|
229
|
+
`deleteFileByName()`, which authorizes by name only. This is not hypothetical: 11.33.0 shipped a
|
|
230
|
+
`deleteFileByName()` that authorized the caller and then re-resolved the file with an EMPTY context,
|
|
231
|
+
so an overridden `checkRights()` was asked two different questions about one request and the owner
|
|
232
|
+
got `File not found` for their own file. Both halves — the by-name branch and the forwarding — are
|
|
233
|
+
covered by the reference server as of the next release.
|
|
234
|
+
|
|
235
|
+
**Do not add an `if (!options.currentUser) return true` branch.** It reads as "system-internal call,
|
|
236
|
+
the guard already decided", and that is not what it means: "no user in context" is also exactly what
|
|
237
|
+
an ANONYMOUS request looks like. While `downloadRoles` is narrower than `S_EVERYONE` the coarse gate
|
|
238
|
+
rejects those first and the branch looks harmless — widen the gate, which §A.2 explicitly offers, and
|
|
239
|
+
the same branch hands every file to everyone. The ownership rule then evaporates precisely when it
|
|
240
|
+
starts to matter. For the same reason the final comparison requires `metadata.ownerId` to be
|
|
241
|
+
**present**: `String(undefined) === String(undefined)` would match an owner-less file against an
|
|
242
|
+
absent user.
|
|
243
|
+
|
|
244
|
+
Callers that genuinely are internal should say so, rather than relying on the omission:
|
|
245
|
+
`{ force: true }` where a role decorator has already decided (an `@Roles(ADMIN)` admin endpoint), or
|
|
246
|
+
the real `{ currentUser }` where the user is in scope (a controller cleaning up a file it just
|
|
247
|
+
replaced — that delete should be COVERED by the ownership rule, not exempt from it). The reference
|
|
248
|
+
server does both; see `src/server/modules/file/` and `src/server/modules/user/avatar.controller.ts`.
|
|
249
|
+
|
|
250
|
+
The reference server also **runs** the rule above rather than documenting it: `src/config.env.ts`
|
|
251
|
+
sets `file: { downloadRoles: [RoleEnum.S_USER] }` in every environment and
|
|
252
|
+
`src/server/modules/file/file.service.ts` implements `checkRights()`. Copy from there — it is
|
|
253
|
+
compiled and exercised by the test suite; this guide is not.
|
|
254
|
+
|
|
255
|
+
Three new pieces make this possible, and all three were missing before: `createFile()` now forwards
|
|
256
|
+
`serviceOptions.metadata` to storage; `getRawFileInfo()` / `getRawFileInfoByName()` read the raw
|
|
257
|
+
document (the public `getFileInfo()` runs `prepareOutput` → `check()` and strips the very field the
|
|
258
|
+
decision rests on); and the core controller and resolver now pass `currentUser` into every service
|
|
259
|
+
call. `getRawFileInfo()` checks S3 metadata first, then the **filesystem** store, then GridFS — all
|
|
260
|
+
three stores, in the same order as `getFileInfo()`, so a rule sees the file the download would
|
|
261
|
+
actually serve (§7). Omitting a store from your own version of this lookup is not a cosmetic
|
|
262
|
+
difference: the rule then decides on a document that is not the one being served.
|
|
263
|
+
|
|
264
|
+
With that in place, `downloadRoles: [RoleEnum.S_USER]` means "signed-in users may fetch **their
|
|
265
|
+
own** files" rather than "everything in the bucket".
|
|
266
|
+
|
|
267
|
+
A refused `checkRights()` answers **404**, deliberately the same as an unknown id, so the endpoint
|
|
268
|
+
cannot be used to probe which files exist.
|
|
269
|
+
|
|
270
|
+
### A.4 — Browser `<img>` / `<a download>`, and presigned URLs
|
|
271
|
+
|
|
272
|
+
Markup-driven requests cannot send an `Authorization` header, and a CORS preflight sends no
|
|
273
|
+
credentials. Anything stricter than `S_EVERYONE` therefore only works from markup when the session
|
|
274
|
+
travels as a **cookie** on a same-site request. If your frontend is Bearer-only, gated file URLs
|
|
275
|
+
will not render in markup at all — fetch them programmatically and build an object URL, or expose a
|
|
276
|
+
separate public route for exactly the files meant to be public.
|
|
277
|
+
|
|
278
|
+
If you enable presigned S3 downloads (§7), note that the issued URL is a **bearer capability**:
|
|
279
|
+
anyone holding it can fetch the object until it expires, with no session. Authorization happens once,
|
|
280
|
+
when the URL is issued. Keep the expiry short, and do not enable it for files whose audience is
|
|
281
|
+
narrower than "anyone who was ever given the link".
|
|
282
|
+
|
|
283
|
+
### A.5 — Also new: `Cache-Control: private, no-store` on downloads
|
|
284
|
+
|
|
285
|
+
An authorization-gated response with no cache directive may be stored by a shared cache under
|
|
286
|
+
RFC 9111 — a reverse proxy or CDN with a blanket `/files/*` rule could hand an authorized response to
|
|
287
|
+
the next, unauthorized requester. **Review any CDN rule you have on `/files/*`.** Override
|
|
288
|
+
`setFileHeaders()` on your controller if you serve genuinely public, cacheable assets.
|
|
289
|
+
|
|
290
|
+
### A.6 — Fix your tests
|
|
291
|
+
|
|
292
|
+
```typescript
|
|
293
|
+
const res = await testHelper.download(`/files/id/${id}`, { token: adminToken });
|
|
294
|
+
expect(res.statusCode).toEqual(200);
|
|
295
|
+
|
|
296
|
+
expect((await testHelper.download(`/files/id/${id}`)).statusCode).toEqual(401); // anonymous
|
|
297
|
+
expect((await testHelper.download(`/files/id/${id}`, { token: userToken })).statusCode)
|
|
298
|
+
.toEqual(403); // no right
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
Assert the exact code, not `[401, 403]` — SPA auth layers branch on 401 to trigger logout, so
|
|
302
|
+
conflating them hides a real regression.
|
|
303
|
+
|
|
304
|
+
---
|
|
305
|
+
|
|
306
|
+
## B. Breaking: TUS uploads require a session
|
|
307
|
+
|
|
308
|
+
`tus.roles` now defaults to `[RoleEnum.S_USER]`; it was `S_EVERYONE`.
|
|
309
|
+
|
|
310
|
+
A TUS upload writes into the **same file store** the download routes guard — GridFS or S3, whichever
|
|
311
|
+
`file.storage` selects — and the termination extension (on by default) can delete from it. Anonymous
|
|
312
|
+
writes into a store only privileged callers may read is the wrong way round.
|
|
313
|
+
|
|
314
|
+
```typescript
|
|
315
|
+
tus: { roles: [RoleEnum.S_USER] }, // default
|
|
316
|
+
tus: { roles: [RoleEnum.S_EVERYONE] }, // opt back in to anonymous uploads
|
|
317
|
+
tus: { roles: ['editor', 'contributor'] }, // project-specific roles work too
|
|
318
|
+
```
|
|
319
|
+
|
|
320
|
+
**If you accept attachments on a public form, you must set `S_EVERYONE` explicitly** — a public
|
|
321
|
+
support or contact form with file upload is the common case, and it breaks at the very first
|
|
322
|
+
request otherwise.
|
|
323
|
+
|
|
324
|
+
`roles: []` is rejected with a warning rather than honoured, for the same reason as in §A.2.
|
|
325
|
+
|
|
326
|
+
`OPTIONS` stays public regardless: it is the CORS preflight, which browsers send without
|
|
327
|
+
credentials, and it returns only server capabilities. Gating it would make every browser upload fail
|
|
328
|
+
before it starts.
|
|
329
|
+
|
|
330
|
+
**Watch the pairing with §A.** `tus.roles` and `file.downloadRoles` are separate, so with both at
|
|
331
|
+
their defaults a signed-in user may upload but cannot read the result back. Either widen
|
|
332
|
+
`downloadRoles`, or — better — write an owner into the metadata at upload time and authorize per
|
|
333
|
+
file (§A.3). TUS uploads already carry `metadata.tusUploadId` and the original TUS metadata, so
|
|
334
|
+
there is a natural place for it.
|
|
335
|
+
|
|
336
|
+
---
|
|
337
|
+
|
|
338
|
+
## 1. Breaking: the rate limiters are async
|
|
339
|
+
|
|
340
|
+
**What changed:** `check()`, `reset()` and `clear()` on `CoreBetterAuthRateLimiter` and
|
|
341
|
+
`LegacyAuthRateLimiter` return promises now. Their counters moved behind a `RateLimitStore`
|
|
342
|
+
abstraction so the same code can count in a process-local `Map` (unchanged behaviour) or in Redis
|
|
343
|
+
(shared across replicas).
|
|
344
|
+
|
|
345
|
+
This is the reason this release is a MINOR rather than a PATCH.
|
|
346
|
+
|
|
347
|
+
**Who is affected:** only projects that call these methods directly or override the classes. The
|
|
348
|
+
framework's own callers — `LegacyAuthRateLimitGuard.canActivate()` and
|
|
349
|
+
`CoreBetterAuthRateLimitMiddleware.use()` — were made async in the same commit, so an untouched
|
|
350
|
+
project needs no change.
|
|
351
|
+
|
|
352
|
+
**Before:**
|
|
353
|
+
|
|
354
|
+
```typescript
|
|
355
|
+
const result = this.rateLimiter.check(ip, '/iam/sign-in');
|
|
356
|
+
if (!result.allowed) {
|
|
357
|
+
throw new HttpException(this.rateLimiter.getMessage(), HttpStatus.TOO_MANY_REQUESTS);
|
|
358
|
+
}
|
|
359
|
+
|
|
360
|
+
// e.g. in a test teardown
|
|
361
|
+
rateLimiter.clear();
|
|
362
|
+
rateLimiter.reset('127.0.0.1');
|
|
363
|
+
```
|
|
364
|
+
|
|
365
|
+
**After:**
|
|
366
|
+
|
|
367
|
+
```typescript
|
|
368
|
+
const result = await this.rateLimiter.check(ip, '/iam/sign-in');
|
|
369
|
+
if (!result.allowed) {
|
|
370
|
+
throw new HttpException(this.rateLimiter.getMessage(), HttpStatus.TOO_MANY_REQUESTS);
|
|
371
|
+
}
|
|
372
|
+
|
|
373
|
+
await rateLimiter.clear();
|
|
374
|
+
await rateLimiter.reset('127.0.0.1');
|
|
375
|
+
```
|
|
376
|
+
|
|
377
|
+
If you **override** one of these methods, the override must return a promise as well, and a guard or
|
|
378
|
+
middleware that called it must be made `async`:
|
|
379
|
+
|
|
380
|
+
```typescript
|
|
381
|
+
// Before
|
|
382
|
+
export class MyRateLimitGuard implements CanActivate {
|
|
383
|
+
canActivate(context: ExecutionContext): boolean {
|
|
384
|
+
const result = this.rateLimiter.check(ip, endpoint);
|
|
385
|
+
// ...
|
|
386
|
+
}
|
|
387
|
+
}
|
|
388
|
+
|
|
389
|
+
// After
|
|
390
|
+
export class MyRateLimitGuard implements CanActivate {
|
|
391
|
+
async canActivate(context: ExecutionContext): Promise<boolean> {
|
|
392
|
+
const result = await this.rateLimiter.check(ip, endpoint);
|
|
393
|
+
// ...
|
|
394
|
+
}
|
|
395
|
+
}
|
|
396
|
+
```
|
|
397
|
+
|
|
398
|
+
`getMessage()`, `isEnabled()`, `configure()` and `getStats()` are unchanged and still synchronous.
|
|
399
|
+
|
|
400
|
+
**One nuance in `getStats()`:** `activeEntries` is `-1` when the Redis store is in use — Redis does
|
|
401
|
+
not know the count cheaply, and reporting `0` would read as "no one is rate limited". Without Redis
|
|
402
|
+
it is the map size, exactly as before.
|
|
403
|
+
|
|
404
|
+
**The AI module's per-user limit** moved to the same store, but
|
|
405
|
+
`CoreAiService.checkRateLimit()` was already `async` before this release — a project that overrode
|
|
406
|
+
it needs no change. The new `protected getRateLimitStore()` is the extension point if you want to
|
|
407
|
+
substitute a store.
|
|
408
|
+
|
|
409
|
+
**On a Redis outage the limiters do not fail requests.** `RedisRateLimitStore` degrades to a
|
|
410
|
+
process-local in-memory counter and logs once, then logs again on recovery. Both alternatives are
|
|
411
|
+
worse: letting the error escape turns every sign-in and password-reset into a 500 for the duration
|
|
412
|
+
of a blip, and treating the error as "allowed" silently removes brute-force protection. Degrading
|
|
413
|
+
keeps a real bound — the per-replica one that applied before Redis existed.
|
|
414
|
+
|
|
415
|
+
---
|
|
416
|
+
|
|
417
|
+
## 2. Breaking: Hub collector reads are async
|
|
418
|
+
|
|
419
|
+
**What changed:** the Hub's diagnostic buffers gained an optional Redis mirror, so reading them is
|
|
420
|
+
now an I/O operation:
|
|
421
|
+
|
|
422
|
+
| Class | Method | Now returns |
|
|
423
|
+
|-------|--------|-------------|
|
|
424
|
+
| `HubLogBufferService` | `getData(since?)` | `Promise<HubLogsData>` |
|
|
425
|
+
| `HubTraceBufferService` | `getData(since?)` | `Promise<HubTracesData>` |
|
|
426
|
+
| `HubQueryProfilerService` | `getData()` | `Promise<HubQueriesData>` |
|
|
427
|
+
| `CoreHubMailboxService` | `getMailbox(since?)` | `Promise<HubMailboxData>` |
|
|
428
|
+
| `CoreHubMailboxService` | `getMailHtml(seq)` | `Promise<string \| undefined>` |
|
|
429
|
+
| `CoreHubController` | `logsJson` / `tracesJson` / `queriesJson` / `mailboxJson` / `mailboxHtml` / `diagnosticsJson` | `Promise<void>` |
|
|
430
|
+
| `CoreHubController` | `protected collectorBuffers()` | `Promise<Record<…>>` |
|
|
431
|
+
|
|
432
|
+
**Who is affected:** only projects that pass a `hub: { controller, service, htmlService, … }` override
|
|
433
|
+
to `CoreModule.forRoot()` and override one of these methods. Everyone else sees no change — the Hub
|
|
434
|
+
keeps working identically, and without Redis every buffer is still the same process-local ring buffer.
|
|
435
|
+
|
|
436
|
+
**Action:** `await` the call in your override and widen its return type to a promise.
|
|
437
|
+
|
|
438
|
+
**`captureAndGetSeq()` caveat:** it still returns the **process-local** seq synchronously. When the
|
|
439
|
+
mailbox is Redis-backed, the seq that the panels and `getMailHtml()` use comes from the shared
|
|
440
|
+
counter and will differ. Read it from `getMailbox()` instead of assuming the returned value.
|
|
441
|
+
|
|
442
|
+
---
|
|
443
|
+
|
|
444
|
+
## 3. Breaking (type level only): file stream return types widened
|
|
445
|
+
|
|
446
|
+
`CoreFileService.getFileStream()` and `getFileStreamByName()` are declared `Promise<Readable>`
|
|
447
|
+
instead of `Promise<mongo.GridFSBucketReadStream>`, because an S3-stored file returns an SDK stream
|
|
448
|
+
rather than a GridFS one. At runtime nothing changes for a GridFS-backed project — the same object
|
|
449
|
+
comes back.
|
|
450
|
+
|
|
451
|
+
**Action:** only if you annotated a variable with the concrete type:
|
|
452
|
+
|
|
453
|
+
```typescript
|
|
454
|
+
// Before
|
|
455
|
+
const stream: mongo.GridFSBucketReadStream = await this.fileService.getFileStream(id);
|
|
456
|
+
|
|
457
|
+
// After
|
|
458
|
+
import { Readable } from 'stream';
|
|
459
|
+
const stream: Readable = await this.fileService.getFileStream(id);
|
|
460
|
+
```
|
|
461
|
+
|
|
462
|
+
`createFile()` / `createFiles()` accept the new, narrower `FileUploadSource` interface instead of
|
|
463
|
+
`FileUpload`. That is a **widening** of what they accept: `FileUpload` now extends
|
|
464
|
+
`FileUploadSource`, so every existing caller still type-checks. It exists so a multer REST upload
|
|
465
|
+
(which has no graphql-upload `capacitor`) can use the same service method — see §10.
|
|
466
|
+
|
|
467
|
+
`CoreFileService`'s protected constructor gained an optional third parameter
|
|
468
|
+
(`options?: CoreFileServiceOptions`). Existing `super(connection)` and `super(connection, 'fs')`
|
|
469
|
+
calls compile unchanged; you only pass it to enable S3 (§7).
|
|
470
|
+
|
|
471
|
+
---
|
|
472
|
+
|
|
473
|
+
## 4. The migration lock is active by default
|
|
474
|
+
|
|
475
|
+
**What changed:** `createMigrationStore()` now defaults its third parameter, `lockCollectionName`,
|
|
476
|
+
to `'migrations_lock'`, and `MigrationRunner.up()` — what the `migrate up` CLI command runs —
|
|
477
|
+
acquires that lock around the whole run.
|
|
478
|
+
|
|
479
|
+
**Why it matters, in deployment terms:** the container entrypoint runs migrations on **every** boot.
|
|
480
|
+
Without a lock, N replicas starting together each read the same empty state, each decide the same
|
|
481
|
+
migration is pending, and each apply it. A migration that inserts seed data inserts it N times; one
|
|
482
|
+
that is not idempotent corrupts on the second run.
|
|
483
|
+
|
|
484
|
+
With the lock, the replicas that lose the race wait, then re-read the migration state **inside** the
|
|
485
|
+
lock and find nothing pending. Reading inside the lock is the part that makes it correct — a waiter
|
|
486
|
+
that re-used a state snapshot taken before the wait would apply the migrations a second time.
|
|
487
|
+
|
|
488
|
+
**With a single replica nothing changes.** The lock is acquired and released uncontended, costing one
|
|
489
|
+
insert and one delete per boot.
|
|
490
|
+
|
|
491
|
+
**What the lock does when a holder dies.** Migrations run on every boot, so a lock nobody holds is not
|
|
492
|
+
a rare event — a container SIGKILLed mid-migration (OOM, node drain, failed deploy) leaves one behind,
|
|
493
|
+
and without recovery every future boot of every replica would wait on it forever. Three mechanisms
|
|
494
|
+
prevent that:
|
|
495
|
+
|
|
496
|
+
| Mechanism | Value | Meaning |
|
|
497
|
+
|-----------|-------|---------|
|
|
498
|
+
| Heartbeat | every **15 s** | The holder refreshes `acquiredAt` while it works |
|
|
499
|
+
| Stale threshold | **60 s** without a heartbeat | Another replica may break and take over the lock |
|
|
500
|
+
| Wait timeout | **15 minutes** | A waiter gives up with a diagnosable error rather than hanging forever |
|
|
501
|
+
|
|
502
|
+
The heartbeat is what makes breaking safe: a lock is only ever broken when its holder **stopped
|
|
503
|
+
refreshing it**, which a live process cannot do. A migration that legitimately runs for hours keeps
|
|
504
|
+
its lock the whole time — the 60 s threshold is not a cap on migration runtime. The break itself
|
|
505
|
+
matches on the exact `acquiredAt` it observed, so if another waiter already broke and re-acquired the
|
|
506
|
+
lock, the delete matches nothing.
|
|
507
|
+
|
|
508
|
+
**If the wait ever times out**, the error names the collection so you can inspect it:
|
|
509
|
+
|
|
510
|
+
```
|
|
511
|
+
Timed out after 15 minutes waiting for the migration lock in collection "migrations_lock".
|
|
512
|
+
Another replica is still migrating, or the lock is held by a process that is alive but stuck.
|
|
513
|
+
Inspect it with: db.getCollection("migrations_lock").find({}) — and remove the document only
|
|
514
|
+
once you are sure no migration is running.
|
|
515
|
+
```
|
|
516
|
+
|
|
517
|
+
Manual recovery:
|
|
518
|
+
|
|
519
|
+
```javascript
|
|
520
|
+
// 1. Look at the holder. `owner` is "<hostname>:<pid>" and `acquiredAt` is the last heartbeat.
|
|
521
|
+
db.getCollection('migrations_lock').find({})
|
|
522
|
+
|
|
523
|
+
// 2. Confirm no migration is actually running (check the pod/container named by `owner`).
|
|
524
|
+
// 3. Only then:
|
|
525
|
+
db.getCollection('migrations_lock').deleteOne({ lock: 'lock' })
|
|
526
|
+
```
|
|
527
|
+
|
|
528
|
+
**Opting out** — a project that deliberately wants unsynchronised migrations passes an empty string:
|
|
529
|
+
|
|
530
|
+
```javascript
|
|
531
|
+
// migrations/state-store.js
|
|
532
|
+
module.exports = createMigrationStore(uri, 'migrations', ''); // no locking
|
|
533
|
+
```
|
|
534
|
+
|
|
535
|
+
A `MongoStateStore` constructed **directly** is unaffected: it still locks only when you pass
|
|
536
|
+
`lockCollectionName` yourself. The default lives in `createMigrationStore()`. The legacy
|
|
537
|
+
`synchronizedMigration()` / `synchronizedUp()` helpers use the same lock and still require the name
|
|
538
|
+
to be set explicitly (they throw otherwise).
|
|
539
|
+
|
|
540
|
+
---
|
|
541
|
+
|
|
542
|
+
## 5. Initial-admin auto-creation takes an atomic claim
|
|
543
|
+
|
|
544
|
+
**What changed:** `systemSetup.initialAdmin` auto-creation guarded itself with a zero-user check.
|
|
545
|
+
That is check-then-act: N replicas booting together each see zero users and each proceed. Before
|
|
546
|
+
creating, an instance now claims the setup by upserting `{ _id: 'initial-admin' }` into a
|
|
547
|
+
`system-setup-locks` collection — an atomic operation exactly one instance wins. The others log a
|
|
548
|
+
debug line and skip. If creation then fails, the marker is removed so a later boot can retry.
|
|
549
|
+
|
|
550
|
+
**Action: none.** The unique email index already prevented duplicate *users*; this removes the noisy
|
|
551
|
+
failures and the ambiguity about which replica actually did the setup. A new collection
|
|
552
|
+
`system-setup-locks` appears in the database with a single small document.
|
|
553
|
+
|
|
554
|
+
---
|
|
555
|
+
|
|
556
|
+
## 6. New: optional central Redis
|
|
557
|
+
|
|
558
|
+
```typescript
|
|
559
|
+
// config.env.ts
|
|
560
|
+
{
|
|
561
|
+
redis: true, // localhost:6379, db 0, keyPrefix = your package name
|
|
562
|
+
// or
|
|
563
|
+
redis: { host: 'redis', port: 6379, keyPrefix: 'my-app' },
|
|
564
|
+
// or
|
|
565
|
+
redis: { url: 'redis://user:pass@redis:6379/0' },
|
|
566
|
+
// pre-configured but off:
|
|
567
|
+
redis: { enabled: false, host: 'redis' },
|
|
568
|
+
}
|
|
569
|
+
```
|
|
570
|
+
|
|
571
|
+
Full shape: `url` **or** `host`/`port`/`username`/`password`/`db`, plus `keyPrefix`, `enabled` and
|
|
572
|
+
`options` (passed through to `ioredis`, e.g. `maxRetriesPerRequest`).
|
|
573
|
+
|
|
574
|
+
Two of these matter more than they look:
|
|
575
|
+
|
|
576
|
+
- **`db`** — picks the numbered Redis database. Sharing one Redis instance between environments
|
|
577
|
+
without separating them is how a staging deploy flushes production's rate-limit counters.
|
|
578
|
+
- **`keyPrefix`** — namespaces every key this application writes. Two deployments on one Redis with
|
|
579
|
+
the same prefix invalidate each other's tenant caches and share each other's cron leases. It
|
|
580
|
+
defaults to the project slug, so distinct projects are already separated; two STAGES of the same
|
|
581
|
+
project are not, unless you set it (or `db`).
|
|
582
|
+
|
|
583
|
+
**ioredis 5 and 6 both work.** The peer range deliberately spans both majors, so a project
|
|
584
|
+
already using ioredis 5 for its own code does not have to jump. Note that **6 defaults to the RESP3
|
|
585
|
+
wire protocol** where 5 used RESP2; every framework path (rate-limit counters, cron leases, tus
|
|
586
|
+
locks, pub/sub, Hub buffers) is exercised against 6 in CI. If something in your infrastructure
|
|
587
|
+
cannot speak RESP3 — an older proxy, for instance — pin the protocol through the pass-through
|
|
588
|
+
options:
|
|
589
|
+
|
|
590
|
+
```typescript
|
|
591
|
+
redis: { host: 'redis', options: { protocol: 2 } },
|
|
592
|
+
```
|
|
593
|
+
|
|
594
|
+
Follows "presence implies enabled": **no `redis` key means no Redis**, and every feature below keeps
|
|
595
|
+
its process-local behaviour. Requires the optional peer dependency `ioredis` (§12); when `redis` is
|
|
596
|
+
configured and the package is missing, the boot **fails fast** with an actionable error rather than
|
|
597
|
+
crashing later on first use.
|
|
598
|
+
|
|
599
|
+
What switches on when Redis is enabled — all of it automatically, with no further config:
|
|
600
|
+
|
|
601
|
+
| Feature | Without Redis | With Redis |
|
|
602
|
+
|---------|---------------|------------|
|
|
603
|
+
| Auth + AI rate limits | Counter per replica, so the effective limit is `max × replicas` | Shared counters, `max` enforced exactly |
|
|
604
|
+
| Cron jobs | Every replica runs every tick | Deduplicated — see §9 |
|
|
605
|
+
| GraphQL subscriptions | Delivered only to clients connected to the publishing replica | Delivered cluster-wide (`CoreRedisPubSub` replaces the in-memory `PubSub`) |
|
|
606
|
+
| Tenant membership cache | `invalidateUser()` clears only the local cache; other replicas stay stale until `cacheTtlMs` | Invalidation is broadcast to all replicas |
|
|
607
|
+
| Hub collectors | Each pod shows only its own logs/traces/queries/mails | Merged cluster-wide view, "clear" clears everywhere |
|
|
608
|
+
| MCP sessions | Wrong-replica request answers `404 Unknown or expired MCP session` | Answers `409` naming both instances — sticky sessions are still **required**, the failure just stops looking like an expiry bug |
|
|
609
|
+
|
|
610
|
+
**GraphQL subscription payloads must be JSON-serialisable** once Redis is in play — they go over the
|
|
611
|
+
wire as JSON. Class instances, `Date` objects, `Map`/`Set` and `undefined` values do not survive the
|
|
612
|
+
round trip. An in-process `PubSub` never had this constraint, so a payload that worked on one replica
|
|
613
|
+
can lose fields on a cluster. Publish plain objects and ISO strings.
|
|
614
|
+
|
|
615
|
+
`keyPrefix` (default: your `package.json` name, slugified — `@acme/api` → `acme-api`) is applied by
|
|
616
|
+
the framework per key, not via ioredis's own
|
|
617
|
+
`keyPrefix` option — that would conflict with BullMQ's prefix handling. Set it when several
|
|
618
|
+
applications share one Redis instance.
|
|
619
|
+
|
|
620
|
+
`CoreRedisService` is exported and provided by `CoreModule`, so you can inject it for your own keys:
|
|
621
|
+
|
|
622
|
+
```typescript
|
|
623
|
+
constructor(@Optional() private readonly redis?: CoreRedisService) {}
|
|
624
|
+
|
|
625
|
+
async cacheSomething(): Promise<void> {
|
|
626
|
+
if (!this.redis?.enabled) {
|
|
627
|
+
return; // keep a non-Redis fallback — the framework does the same everywhere
|
|
628
|
+
}
|
|
629
|
+
await this.redis.getClient().set(this.redis.key('my-feature', 'thing'), '1', 'EX', 60);
|
|
630
|
+
}
|
|
631
|
+
```
|
|
632
|
+
|
|
633
|
+
Use `getSubscriber()` for pub/sub (a subscribing client cannot run other commands) and
|
|
634
|
+
`createClient(label)` for a dedicated connection. All connections are tracked and quit on shutdown.
|
|
635
|
+
|
|
636
|
+
---
|
|
637
|
+
|
|
638
|
+
## 7. New: three file storage drivers (S3, GridFS, filesystem)
|
|
639
|
+
|
|
640
|
+
`file.storage` picks where the bytes live: `'s3'`, `'gridfs'` or `'filesystem'`. All three offer the
|
|
641
|
+
same operations through `CoreFileService`.
|
|
642
|
+
|
|
643
|
+
**Left unset it is derived**, most capable first: `'s3'` when `s3.bucket` is configured, else
|
|
644
|
+
`'gridfs'` when a database is, else `'filesystem'`. **An existing project keeps GridFS** — it has no
|
|
645
|
+
`s3` block, so the cascade lands on step 2, exactly where it was before.
|
|
646
|
+
|
|
647
|
+
**Set it and it is enforced.** If the chosen store is unavailable the boot FAILS:
|
|
648
|
+
|
|
649
|
+
```
|
|
650
|
+
file.storage is set to 's3', but that storage is not available.
|
|
651
|
+
Configure `s3` (bucket, credentials/endpoint) and install `@aws-sdk/client-s3`, …
|
|
652
|
+
```
|
|
653
|
+
|
|
654
|
+
This replaces a silent fallback to GridFS. The old behaviour kept the app running while files landed
|
|
655
|
+
in a store the operator did not believe they were in — and afterwards nobody can tell which file went
|
|
656
|
+
where. A derived driver is enforced the same way: `s3.bucket` in the config makes S3 the default, but
|
|
657
|
+
the bytes go nowhere unless your `FileService` forwards the services to `super()` (see below).
|
|
658
|
+
|
|
659
|
+
**Metadata always lives in the database**, whichever driver holds the bytes — filename, content type,
|
|
660
|
+
length and the custom `metadata` a per-file rule reads have to stay queryable. `'filesystem'` moves
|
|
661
|
+
the bytes off the database, not the bookkeeping.
|
|
662
|
+
|
|
663
|
+
**`'filesystem'` is pod-local.** Not shared between replicas, lost on restart unless `storageDir` is a
|
|
664
|
+
mounted volume. Right for a single instance or a real volume; wrong for a scaled service.
|
|
665
|
+
|
|
666
|
+
**Switching is forward-only.** Reads consult every store, so files written under a previous driver
|
|
667
|
+
stay readable — no migration step, no cut-over. The driver in use is logged at boot:
|
|
668
|
+
`[CoreFileStorage] File storage: s3 (defaulted — s3.bucket is configured)`.
|
|
669
|
+
|
|
670
|
+
```typescript
|
|
671
|
+
// config.env.ts
|
|
672
|
+
{
|
|
673
|
+
file: {
|
|
674
|
+
storage: 's3', // omit to derive it
|
|
675
|
+
storageDir: 'uploads/files', // only for 'filesystem'
|
|
676
|
+
},
|
|
677
|
+
s3: {
|
|
678
|
+
bucket: 'my-app-files',
|
|
679
|
+
region: 'eu-central-1',
|
|
680
|
+
// self-hosted (MinIO, RustFS):
|
|
681
|
+
endpoint: 'http://minio:9000',
|
|
682
|
+
forcePathStyle: true,
|
|
683
|
+
accessKeyId: '…', // omit to use the AWS default credential chain
|
|
684
|
+
secretAccessKey: '…',
|
|
685
|
+
presignedDownloads: { expiresInSeconds: 300 }, // or `true` for the 300 s default
|
|
686
|
+
stagingBucket: 'my-app-tus', // defaults to `bucket`
|
|
687
|
+
autoCreateBucket: false, // default: false
|
|
688
|
+
},
|
|
689
|
+
}
|
|
690
|
+
```
|
|
691
|
+
|
|
692
|
+
**`autoCreateBucket`** creates a missing bucket at boot instead of only reporting it. Off by default
|
|
693
|
+
for a reason: managed S3 credentials usually carry no `CreateBucket` permission, and silently
|
|
694
|
+
creating a bucket because of a typo in the name is worse than a clear error. Turn it on for a
|
|
695
|
+
self-hosted MinIO/RustFS, where provisioning the bucket by hand is the only alternative.
|
|
696
|
+
|
|
697
|
+
**`enabled: false`** pre-configures S3 without activating it — the same "presence implies enabled"
|
|
698
|
+
shape the rest of the config uses.
|
|
699
|
+
|
|
700
|
+
**A block without `bucket` is ignored entirely** (with a warning), since a bucket is the one thing
|
|
701
|
+
S3 cannot default — see §14.
|
|
702
|
+
|
|
703
|
+
Requires `@aws-sdk/client-s3`, plus `@aws-sdk/s3-request-presigner` when `presignedDownloads` is on
|
|
704
|
+
(§12). Both are lazy-imported; a missing package fails the boot with a named error.
|
|
705
|
+
|
|
706
|
+
**`s3` now also decides the DEFAULT.** Previously `s3` alone only made `CoreS3Service` available
|
|
707
|
+
(TUS staging, your own code) and files moved to S3 only when `file.storage: 's3'` was set as well.
|
|
708
|
+
Now a configured `s3.bucket` makes S3 the derived default. If you want S3 available for staging while
|
|
709
|
+
files stay in GridFS, say so explicitly:
|
|
710
|
+
|
|
711
|
+
```typescript
|
|
712
|
+
file: { storage: 'gridfs' },
|
|
713
|
+
s3: { bucket: 'my-app-tus' },
|
|
714
|
+
```
|
|
715
|
+
|
|
716
|
+
`file.storage` sits alongside the role knobs from §A — one object configures the file module: where
|
|
717
|
+
the bytes live and who may reach them. The two are orthogonal, and a per-file rule in
|
|
718
|
+
`checkRights()` works under either driver (`getRawFileInfo()` checks S3 metadata first, then GridFS).
|
|
719
|
+
|
|
720
|
+
**Existing GridFS files stay readable.** Reads check the S3 metadata collection first and fall back
|
|
721
|
+
to GridFS, so a switch to `'s3'` is forward-only: new files land in S3, old files keep being served
|
|
722
|
+
from GridFS. There is no migration step and no cut-over moment. `findFileInfo()` returns the union
|
|
723
|
+
of every store, paged once over the merged result.
|
|
724
|
+
|
|
725
|
+
File metadata lives in a new collection `s3-files` (filename, contentType, length, uploadDate,
|
|
726
|
+
custom `metadata`) — and `filesystem-files` for the filesystem driver, see §14 —
|
|
727
|
+
S3 has no queryable metadata store, so this is the equivalent of GridFS's `fs.files`. The object key
|
|
728
|
+
in the bucket is the string form of `_id`.
|
|
729
|
+
|
|
730
|
+
**`presignedDownloads` changes what `GET /files/id/:id` answers**: a `302` redirect to a
|
|
731
|
+
time-limited S3 URL instead of a streamed body, which takes the bytes off the API. It is
|
|
732
|
+
deliberately fail-soft — a missing presigner package, an S3 outage, or a project `FileService`
|
|
733
|
+
predating `getDownloadUrl()` all fall through to the streaming path, which runs the same rights
|
|
734
|
+
check. A client that follows redirects (every browser, every HTTP library by default) needs no
|
|
735
|
+
change; a client with `redirect: 'manual'` does.
|
|
736
|
+
|
|
737
|
+
**To enable S3 in your project's `FileService`**, pass the two services through:
|
|
738
|
+
|
|
739
|
+
```typescript
|
|
740
|
+
@Injectable()
|
|
741
|
+
export class FileService extends CoreFileService {
|
|
742
|
+
constructor(
|
|
743
|
+
@InjectConnection() protected override readonly connection: Connection,
|
|
744
|
+
protected readonly configService: ConfigService,
|
|
745
|
+
@Optional() protected readonly s3Service?: CoreS3Service,
|
|
746
|
+
) {
|
|
747
|
+
super(connection, 'fs', { configService, s3Service });
|
|
748
|
+
}
|
|
749
|
+
}
|
|
750
|
+
```
|
|
751
|
+
|
|
752
|
+
**Do not read the third argument as optional once `s3` is configured.** `CoreFileService`'s
|
|
753
|
+
constructor runs `assertFileStorageAvailable()`, so if the resolved driver is `'s3'` — whether you
|
|
754
|
+
set `file.storage: 's3'` or merely configured `s3.bucket` and let it be derived — a `FileService`
|
|
755
|
+
that does not forward `s3Service` to `super()` **fails the boot**:
|
|
756
|
+
|
|
757
|
+
```
|
|
758
|
+
File storage 's3' was selected automatically (s3.bucket is configured) but is not available.
|
|
759
|
+
```
|
|
760
|
+
|
|
761
|
+
That is the intended behaviour: the alternative would be a silent fallback to GridFS, files landing
|
|
762
|
+
in a store the operator does not believe they are in, and no way afterwards to tell which file went
|
|
763
|
+
where. Either forward the services as shown above, or pin `file: { storage: 'gridfs' }` (§11b).
|
|
764
|
+
|
|
765
|
+
`super(connection)` / `super(connection, 'fs')` on a project **without** any `s3` config is
|
|
766
|
+
unaffected — the driver derives to `'gridfs'`, which is available, and nothing changes.
|
|
767
|
+
|
|
768
|
+
**If you call `CoreS3Service.putObject()` directly, pass the length for large files.** The signature
|
|
769
|
+
is `putObject(key, body, contentType?, contentLength?)`. The AWS SDK cannot sign a stream whose
|
|
770
|
+
length it does not know — it fails on `x-amz-decoded-content-length` — so when `body` is a stream and
|
|
771
|
+
`contentLength` is omitted, the service reads the whole stream into memory to measure it. For a
|
|
772
|
+
multi-gigabyte upload that is the difference between a constant-memory transfer and an OOM:
|
|
773
|
+
|
|
774
|
+
```typescript
|
|
775
|
+
// Streams the file without buffering it
|
|
776
|
+
await this.s3Service.putObject(key, createReadStream(path), 'video/mp4', statSync(path).size);
|
|
777
|
+
|
|
778
|
+
// Reads the entire stream into a Buffer first — fine for small payloads, not for large ones
|
|
779
|
+
await this.s3Service.putObject(key, createReadStream(path), 'video/mp4');
|
|
780
|
+
```
|
|
781
|
+
|
|
782
|
+
A `Buffer` or `string` body needs nothing: the length is taken from the value itself.
|
|
783
|
+
|
|
784
|
+
---
|
|
785
|
+
|
|
786
|
+
## 8. New: `tus.s3Staging`
|
|
787
|
+
|
|
788
|
+
When `s3` is configured, in-progress TUS uploads are staged in the S3 staging bucket
|
|
789
|
+
(`@tus/s3-store`) instead of on local disk — **default `true`**, so configuring `s3` is enough. Set
|
|
790
|
+
`tus: { s3Staging: false }` to force local disk.
|
|
791
|
+
|
|
792
|
+
Why it matters: a chunk written to `uploads/tus` lives on one pod's filesystem. Without sticky
|
|
793
|
+
sessions the next chunk of the same upload lands elsewhere and the upload cannot be resumed; a
|
|
794
|
+
restart discards it either way. With S3 staging, resumable uploads survive replica restarts and need
|
|
795
|
+
no sticky sessions.
|
|
796
|
+
|
|
797
|
+
Requires `@tus/s3-store` (§12). Missing package → a warning and a fall back to local disk, not a
|
|
798
|
+
boot failure.
|
|
799
|
+
|
|
800
|
+
**Give the staging bucket a lifecycle rule that expires incomplete multipart uploads.** The
|
|
801
|
+
local-disk store is swept by the framework's own expiration cleanup; the S3 store is not — that
|
|
802
|
+
cleanup is skipped in S3 mode, because S3 is the right place to express the policy. Aborted uploads
|
|
803
|
+
otherwise accumulate parts nothing removes.
|
|
804
|
+
|
|
805
|
+
---
|
|
806
|
+
|
|
807
|
+
## 9. New: cron deduplication across replicas
|
|
808
|
+
|
|
809
|
+
A new per-job option, `distributed`:
|
|
810
|
+
|
|
811
|
+
```typescript
|
|
812
|
+
const cronJobs = {
|
|
813
|
+
cleanUpTempFiles: { cronTime: CronExpression.EVERY_HOUR },
|
|
814
|
+
refreshLocalCache: { cronTime: CronExpression.EVERY_MINUTE, distributed: false },
|
|
815
|
+
sendDigest: { cronTime: CronExpression.EVERY_DAY_AT_6AM, distributed: true },
|
|
816
|
+
};
|
|
817
|
+
```
|
|
818
|
+
|
|
819
|
+
**Default: `true` when `redis` is configured, `false` otherwise.** That default is the point — a
|
|
820
|
+
single-replica project that upgrades does not silently gain a `cron-locks` collection, a lease write
|
|
821
|
+
per tick, and a new way for a tick to be skipped. A **Redis-less multi-replica fleet opts in per
|
|
822
|
+
job** with `distributed: true`, which uses a MongoDB lease.
|
|
823
|
+
|
|
824
|
+
How a deduplicated job runs:
|
|
825
|
+
|
|
826
|
+
| Setup | Mechanism |
|
|
827
|
+
|-------|-----------|
|
|
828
|
+
| Redis + `bullmq` installed, `cronTime` is a **string** and no `utcOffset` | The job becomes a BullMQ job scheduler — exactly one worker in the cluster picks up each tick |
|
|
829
|
+
| Redis, but `cronTime` is a `Date` or `utcOffset` is set | Local timer per replica; each tick first takes a Redis `SET NX` lease |
|
|
830
|
+
| Redis configured, `bullmq` **not** installed | Warning, then local timer + Redis lease |
|
|
831
|
+
| No Redis, `distributed: true` | Local timer + a `cron-locks` document in MongoDB (TTL-indexed) |
|
|
832
|
+
| No Redis, `distributed` unset | No lease at all — byte-identical to 11.32.x |
|
|
833
|
+
|
|
834
|
+
**Leases fail open.** An unreachable lease store runs the tick on every replica rather than stopping
|
|
835
|
+
it everywhere. A lease exists to prevent a *duplicate* run; treating an outage as "someone else won"
|
|
836
|
+
would turn a Redis blip into a silent, fleet-wide outage of every scheduled job — the worse failure.
|
|
837
|
+
|
|
838
|
+
**`runOnInit` deduplicates over a 5-minute window, and you should know what that means.** A
|
|
839
|
+
scheduled tick has an instant every replica computes identically, so its lease key comes from the
|
|
840
|
+
schedule. A *startup* has no such instant — replicas boot milliseconds to minutes apart, so a key
|
|
841
|
+
built from each replica's own clock would be different everywhere and deduplicate nothing. Since
|
|
842
|
+
`runOnInit` defaults to `true`, that would leave the most common path undeduplicated. The init lease
|
|
843
|
+
therefore uses one **fixed key per job** with a 5-minute TTL:
|
|
844
|
+
|
|
845
|
+
- Replicas that boot within 5 minutes of each other run the startup tick **once between them**.
|
|
846
|
+
- A replica that restarts *inside* that window — a crash-loop, a rolling deploy, a rescheduled pod —
|
|
847
|
+
**skips its startup tick**. If a job's `runOnInit` work must happen on every process start (warming
|
|
848
|
+
a process-local cache, for instance), set `distributed: false` on it.
|
|
849
|
+
- A replica joining later (autoscaling, a much later restart) runs it again, which is the intended
|
|
850
|
+
behaviour for a genuinely new instance.
|
|
851
|
+
|
|
852
|
+
**No constructor change is needed.** `CoreCronJobs` is abstract and instantiated by your project, so
|
|
853
|
+
the framework cannot inject into it. `CoreModule` populates a registry
|
|
854
|
+
(`setCronJobsInfrastructure()`) that the base class reads lazily, so existing
|
|
855
|
+
`super(schedulerRegistry, cronJobs, { log })` calls gain deduplication untouched. You *may* pass
|
|
856
|
+
`connection` / `redisService` explicitly in the options object; values passed there win.
|
|
857
|
+
|
|
858
|
+
With neither source available the service warns once ("Cron job deduplication is inactive …") and
|
|
859
|
+
behaves as before.
|
|
860
|
+
|
|
861
|
+
---
|
|
862
|
+
|
|
863
|
+
## 10. Reference implementation: uploads go to central storage
|
|
864
|
+
|
|
865
|
+
`src/server/` is the framework's own test implementation, but consumer projects are generated from
|
|
866
|
+
the same shapes. **If you copied the avatar upload or the GraphQL multi-upload example, you have
|
|
867
|
+
pod-local disk writes that break with more than one replica** — the file lands on whichever pod
|
|
868
|
+
served the request, is unreadable from every other pod, and is gone after a restart.
|
|
869
|
+
|
|
870
|
+
### 10.1 `UserService.setAvatar()` changed signature
|
|
871
|
+
|
|
872
|
+
It takes the **stored file id** and returns the **previous** avatar id, so the caller can delete the
|
|
873
|
+
orphan. It no longer touches the filesystem.
|
|
874
|
+
|
|
875
|
+
**Why the storing moved to the controller:** `UserService` is instantiated by `CoreAuthModule`,
|
|
876
|
+
which knows nothing about the project's `FileModule`. It cannot depend on `FileService`. The
|
|
877
|
+
controller can.
|
|
878
|
+
|
|
879
|
+
**Before:**
|
|
880
|
+
|
|
881
|
+
```typescript
|
|
882
|
+
// user.service.ts
|
|
883
|
+
async setAvatar(file: Express.Multer.File, user: User): Promise<string> {
|
|
884
|
+
const dbUser = await this.mainDbModel.findOne({ id: user.id }).exec();
|
|
885
|
+
if (!dbUser) {
|
|
886
|
+
throw new UnauthorizedException();
|
|
887
|
+
}
|
|
888
|
+
if (!file) {
|
|
889
|
+
throw new UnprocessableEntityException('Missing avatar file');
|
|
890
|
+
}
|
|
891
|
+
if (user.avatar) {
|
|
892
|
+
fs.unlink(`${this.configService.configFastButReadOnly.staticAssets.path}/avatars/${user.avatar}`, …);
|
|
893
|
+
}
|
|
894
|
+
dbUser.avatar = file.filename;
|
|
895
|
+
await dbUser.save();
|
|
896
|
+
return file.filename;
|
|
897
|
+
}
|
|
898
|
+
|
|
899
|
+
// avatar.controller.ts
|
|
900
|
+
@UseInterceptors(FileInterceptor('file', multerOptionsForImageUpload({
|
|
901
|
+
destination: `${envConfig.staticAssets.path}/avatars`,
|
|
902
|
+
})))
|
|
903
|
+
uploadFile(@UploadedFile() file: Express.Multer.File, @CurrentUser() user: User): Promise<string> {
|
|
904
|
+
return this.usersService.setAvatar(file, user);
|
|
905
|
+
}
|
|
906
|
+
```
|
|
907
|
+
|
|
908
|
+
**After:**
|
|
909
|
+
|
|
910
|
+
```typescript
|
|
911
|
+
// user.service.ts
|
|
912
|
+
/** @returns the PREVIOUS avatar id, so the caller can delete the orphaned file */
|
|
913
|
+
async setAvatar(avatarId: string, user: User): Promise<string> {
|
|
914
|
+
const dbUser = await this.mainDbModel.findOne({ id: user.id }).exec();
|
|
915
|
+
if (!dbUser) {
|
|
916
|
+
throw new UnauthorizedException();
|
|
917
|
+
}
|
|
918
|
+
if (!avatarId) {
|
|
919
|
+
throw new UnprocessableEntityException('Missing avatar file');
|
|
920
|
+
}
|
|
921
|
+
const previousAvatar = dbUser.avatar;
|
|
922
|
+
dbUser.avatar = avatarId;
|
|
923
|
+
await dbUser.save();
|
|
924
|
+
return previousAvatar;
|
|
925
|
+
}
|
|
926
|
+
|
|
927
|
+
// avatar.controller.ts — `memory: true`, NOT a disk destination
|
|
928
|
+
@UseInterceptors(FileInterceptor('file', multerOptionsForImageUpload({ memory: true })))
|
|
929
|
+
async uploadFile(@UploadedFile() file: Express.Multer.File, @CurrentUser() user: User): Promise<string> {
|
|
930
|
+
const stored = await this.fileService.createFile(multerFileToUpload(file));
|
|
931
|
+
const previousAvatar = await this.usersService.setAvatar(stored.id, user);
|
|
932
|
+
|
|
933
|
+
// A failure here must not fail the upload: the new avatar is already stored and
|
|
934
|
+
// referenced, so an orphan is a cleanup concern, not a request error.
|
|
935
|
+
if (previousAvatar) {
|
|
936
|
+
try {
|
|
937
|
+
await this.fileService.deleteFile(previousAvatar);
|
|
938
|
+
} catch (error) {
|
|
939
|
+
this.logger.warn(`Could not remove previous avatar ${previousAvatar}: …`);
|
|
940
|
+
}
|
|
941
|
+
}
|
|
942
|
+
|
|
943
|
+
return stored.id;
|
|
944
|
+
}
|
|
945
|
+
```
|
|
946
|
+
|
|
947
|
+
`AvatarController` now injects `FileService` in addition to `UserService`. In the reference
|
|
948
|
+
implementation that made `UserModule` and `FileModule` reference each other, resolved with
|
|
949
|
+
`forwardRef(() => FileModule)` on the `UserModule` side (`FileModule` already had one on the other).
|
|
950
|
+
|
|
951
|
+
**Frontend impact — this changes what `user.avatar` contains.** It used to be a filename served from
|
|
952
|
+
the static-assets directory; it is now a **file id** served via `GET /files/id/:id`. Existing avatar
|
|
953
|
+
values in the database still point at old static filenames. Either migrate them (upload the files
|
|
954
|
+
into the store and rewrite the field) or have the frontend fall back to the old URL shape for values
|
|
955
|
+
that are not valid ObjectIds.
|
|
956
|
+
|
|
957
|
+
### 10.2 GraphQL multi-upload
|
|
958
|
+
|
|
959
|
+
**Before:**
|
|
960
|
+
|
|
961
|
+
```typescript
|
|
962
|
+
for (const file of files) {
|
|
963
|
+
const { createReadStream, filename } = await file;
|
|
964
|
+
await fs.promises.mkdir('./uploads', { recursive: true });
|
|
965
|
+
createReadStream().pipe(createWriteStream(`./uploads/${filename}`));
|
|
966
|
+
}
|
|
967
|
+
```
|
|
968
|
+
|
|
969
|
+
`./uploads` is relative to the process working directory — one pod's disk.
|
|
970
|
+
|
|
971
|
+
**After:**
|
|
972
|
+
|
|
973
|
+
```typescript
|
|
974
|
+
await this.fileService.createFiles(files);
|
|
975
|
+
```
|
|
976
|
+
|
|
977
|
+
### 10.3 `multerFileToUpload()`
|
|
978
|
+
|
|
979
|
+
The new helper adapts a multer upload to what `CoreFileService` consumes, so a REST/multer endpoint
|
|
980
|
+
writes to the same central storage as the GraphQL path:
|
|
981
|
+
|
|
982
|
+
```typescript
|
|
983
|
+
import { multerFileToUpload, multerOptionsForImageUpload } from '@lenne.tech/nest-server';
|
|
984
|
+
|
|
985
|
+
@UseInterceptors(FileInterceptor('file', multerOptionsForImageUpload({ memory: true })))
|
|
986
|
+
async upload(@UploadedFile() file: Express.Multer.File) {
|
|
987
|
+
return this.fileService.createFile(multerFileToUpload(file));
|
|
988
|
+
}
|
|
989
|
+
```
|
|
990
|
+
|
|
991
|
+
It **requires `memory: true`** on the multer options — a disk-stored file has no `buffer`, and the
|
|
992
|
+
helper throws with that exact instruction rather than silently storing an empty file. The returned
|
|
993
|
+
`createReadStream` is callable more than once, matching graphql-upload's contract.
|
|
994
|
+
|
|
995
|
+
### 10.4 `PUB_SUB` provider
|
|
996
|
+
|
|
997
|
+
Both `CoreAuthModule` and the reference `UserModule` build `PUB_SUB` from a factory now: a
|
|
998
|
+
`CoreRedisPubSub` when Redis is enabled, the in-memory `PubSub` otherwise. If your project provides
|
|
999
|
+
its own `PUB_SUB` with `useValue: new PubSub()`, subscriptions stay single-replica. Mirror the
|
|
1000
|
+
factory to fix that:
|
|
1001
|
+
|
|
1002
|
+
```typescript
|
|
1003
|
+
{
|
|
1004
|
+
provide: 'PUB_SUB',
|
|
1005
|
+
useFactory: (redisService?: CoreRedisService) =>
|
|
1006
|
+
redisService?.enabled ? new CoreRedisPubSub(redisService) : new PubSub(),
|
|
1007
|
+
inject: [{ optional: true, token: CoreRedisService }],
|
|
1008
|
+
}
|
|
1009
|
+
```
|
|
1010
|
+
|
|
1011
|
+
---
|
|
1012
|
+
|
|
1013
|
+
## 11. New: `shutdownDelayMs`
|
|
1014
|
+
|
|
1015
|
+
```typescript
|
|
1016
|
+
{ shutdownDelayMs: 5000 } // default: 0 (no delay, previous behaviour)
|
|
1017
|
+
```
|
|
1018
|
+
|
|
1019
|
+
On a rolling deploy the orchestrator sends SIGTERM and deregisters the instance from the load
|
|
1020
|
+
balancer at the same time — but deregistration is not instant, so requests keep arriving for a
|
|
1021
|
+
moment after the signal. `installGracefulShutdown()` waits in the SIGTERM/SIGINT handler itself,
|
|
1022
|
+
*before* `close()` is entered, so those in-flight requests land on a fully healthy instance.
|
|
1023
|
+
|
|
1024
|
+
A NestJS lifecycle hook cannot do this. `close()` runs `onModuleDestroy` →
|
|
1025
|
+
`beforeApplicationShutdown` → dispose (close the HTTP server) → `onApplicationShutdown`, so a delay
|
|
1026
|
+
in `beforeApplicationShutdown` waits *after* every module is torn down while the socket still
|
|
1027
|
+
accepts — worse than not waiting at all.
|
|
1028
|
+
|
|
1029
|
+
**Wire it in `main.ts` as a REPLACEMENT for `server.enableShutdownHooks()`:**
|
|
1030
|
+
|
|
1031
|
+
```typescript
|
|
1032
|
+
import { installGracefulShutdown } from '@lenne.tech/nest-server';
|
|
1033
|
+
|
|
1034
|
+
installGracefulShutdown(server); // instead of server.enableShutdownHooks()
|
|
1035
|
+
```
|
|
1036
|
+
|
|
1037
|
+
Do **not** keep `enableShutdownHooks()` alongside it. Nest would register its own listener for the
|
|
1038
|
+
same signals and close the app immediately, in parallel with the wait — the delay then silently
|
|
1039
|
+
never happens. With `shutdownDelayMs` unset or zero, `installGracefulShutdown()` *is*
|
|
1040
|
+
`enableShutdownHooks()`, so this line is correct either way.
|
|
1041
|
+
|
|
1042
|
+
**Keep the value well below your orchestrator's grace period, and leave room for the drain that
|
|
1043
|
+
follows it.** Docker/Compose `stop_grace_period` defaults to 10s, Kubernetes
|
|
1044
|
+
`terminationGracePeriodSeconds` to 30s, and `installProcessDiagnostics()` force-exits after 30s.
|
|
1045
|
+
Exceed any of them and the process is SIGKILLed mid-wait, running no shutdown hook at all. Values
|
|
1046
|
+
above 10s log a warning; above 60s the delay is capped. Inert — no delay, no log — when unset or
|
|
1047
|
+
zero.
|
|
1048
|
+
|
|
1049
|
+
---
|
|
1050
|
+
|
|
1051
|
+
## 11a. Check whether you ALREADY have a `redis` key
|
|
1052
|
+
|
|
1053
|
+
`redis` did not exist as a server option before 11.33.0, so the framework ignored it — which means a
|
|
1054
|
+
project may already carry one in its server options for its **own** client, and never think about it
|
|
1055
|
+
again. From 11.33.0 the framework reads it, and because activation follows "presence implies
|
|
1056
|
+
enabled", the mere presence of that key switches on every distributed path at once: rate limits,
|
|
1057
|
+
cron deduplication, the email-verification cooldown, tenant-cache invalidation, Hub buffer
|
|
1058
|
+
mirroring, MCP session ownership and — if you use GraphQL — subscriptions over Redis pub/sub.
|
|
1059
|
+
|
|
1060
|
+
Nothing about that is wrong, but you get it without asking for it. Three consequences are worth
|
|
1061
|
+
knowing before you upgrade:
|
|
1062
|
+
|
|
1063
|
+
- **Limits get stricter, not looser.** A rate limit of `max` was effectively `max × replicas`
|
|
1064
|
+
before; now it is `max`. That is the bug being fixed, but a deployment tuned around the old
|
|
1065
|
+
behaviour will throttle earlier.
|
|
1066
|
+
- **Subscription payloads must be JSON-serializable** once pub/sub goes through Redis. `Date` is
|
|
1067
|
+
revived on parse; class instances, `Map`, `Set` and `undefined` are not.
|
|
1068
|
+
- **The limit is only as exact as the client IP — configure `trustProxy` (new in 11.33.0).**
|
|
1069
|
+
See §11c: behind a reverse proxy this is now the difference between one bucket per client and
|
|
1070
|
+
one bucket for your entire user base.
|
|
1071
|
+
|
|
1072
|
+
The boot log now states this explicitly — `CoreRedisService` logs the endpoint, the key prefix and
|
|
1073
|
+
the list of subsystems that switched, once, at startup. If you do not want any of it yet, set
|
|
1074
|
+
`redis: { enabled: false, ... }`: the config stays in place and the framework stays out.
|
|
1075
|
+
|
|
1076
|
+
This was found by upgrading a real project that had carried a `redis` block in all five of its
|
|
1077
|
+
environments for its own BullMQ queues.
|
|
1078
|
+
|
|
1079
|
+
---
|
|
1080
|
+
|
|
1081
|
+
## 11b. Check whether you ALREADY have an `s3` key — this one FAILS THE BOOT
|
|
1082
|
+
|
|
1083
|
+
Same shape as §11a, but with a louder failure mode. `s3` did not exist as a server option before
|
|
1084
|
+
11.33.0 either, so a project may already carry one for its **own** SDK client — a report export, a
|
|
1085
|
+
backup job, a media pipeline. From 11.33.0 the framework reads it, and `s3.bucket` alone now makes
|
|
1086
|
+
S3 the **derived** default for `file.storage` (§7).
|
|
1087
|
+
|
|
1088
|
+
The consequence is not subtle. `CoreFileService`'s constructor asserts that the resolved driver is
|
|
1089
|
+
actually usable, so a project whose `FileService` still calls `super(connection, 'fs')` — which is
|
|
1090
|
+
every project that has not read §7 — stops at boot:
|
|
1091
|
+
|
|
1092
|
+
```
|
|
1093
|
+
File storage 's3' was selected automatically (s3.bucket is configured) but is not available.
|
|
1094
|
+
```
|
|
1095
|
+
|
|
1096
|
+
You are looking at a startup crash, not at misplaced files. That is deliberate: before 11.33.0 the
|
|
1097
|
+
same situation silently fell back to GridFS, and a silent fallback is unrecoverable — nobody can
|
|
1098
|
+
tell afterwards which file went to which store.
|
|
1099
|
+
|
|
1100
|
+
**Three ways out, in order of how likely they are to be what you want:**
|
|
1101
|
+
|
|
1102
|
+
```typescript
|
|
1103
|
+
// 1. Your `s3` block is NOT meant for file storage (own client, TUS staging only).
|
|
1104
|
+
// Pin the driver and nothing changes:
|
|
1105
|
+
file: { storage: 'gridfs' },
|
|
1106
|
+
|
|
1107
|
+
// 2. Your `s3` block IS meant for file storage — adopt it properly (§7):
|
|
1108
|
+
// forward { configService, s3Service } to super() in your FileService,
|
|
1109
|
+
// and install @aws-sdk/client-s3 (§12).
|
|
1110
|
+
|
|
1111
|
+
// 3. You want the config parked, inert, for later:
|
|
1112
|
+
s3: { enabled: false, bucket: 'my-app-files', /* … */ },
|
|
1113
|
+
```
|
|
1114
|
+
|
|
1115
|
+
Two more things worth knowing before the upgrade:
|
|
1116
|
+
|
|
1117
|
+
- **`tus.s3Staging` defaults to `true` whenever `s3` is usable** (§8). A pre-existing `s3` block
|
|
1118
|
+
therefore also moves in-progress TUS uploads off local disk into `s3.stagingBucket` (which
|
|
1119
|
+
defaults to `bucket`). That is the better behaviour for more than one replica, but it puts
|
|
1120
|
+
objects into a bucket you may have provisioned for something else — set `stagingBucket`, or
|
|
1121
|
+
`tus: { s3Staging: false }`.
|
|
1122
|
+
- **A block without `bucket` is now ignored** with a warning (§14), where it used to make
|
|
1123
|
+
`CoreS3Service.enabled` true. If you kept a bucket-less `s3` block around, TUS staging switched
|
|
1124
|
+
itself on against a bucket that did not exist; that failure is gone, and so is the block's effect.
|
|
1125
|
+
|
|
1126
|
+
`s3` is checked at boot, so unlike §11a there is no version of this you can discover in production
|
|
1127
|
+
three days later.
|
|
1128
|
+
|
|
1129
|
+
---
|
|
1130
|
+
|
|
1131
|
+
## 11c. New: `trustProxy` — required if you run behind a reverse proxy AND rate-limit
|
|
1132
|
+
|
|
1133
|
+
**Applies to:** every deployment where something terminates TLS in front of the app — Caddy, nginx,
|
|
1134
|
+
Traefik, a Kubernetes ingress, a cloud load balancer — **and** `auth.rateLimit` or
|
|
1135
|
+
`betterAuth.rateLimit` is enabled. That is the normal production shape.
|
|
1136
|
+
|
|
1137
|
+
### What changed
|
|
1138
|
+
|
|
1139
|
+
Both rate limiters key their counter on the client IP. Up to 11.32.x they took the first entry of
|
|
1140
|
+
`X-Forwarded-For` themselves and trusted it unconditionally, which meant any client could pick its
|
|
1141
|
+
own bucket by sending a fresh value per request — the counter never reached the limit and the
|
|
1142
|
+
brute-force protection was effectively off.
|
|
1143
|
+
|
|
1144
|
+
11.33.0 reads `request.ip` instead, which is the correct source: Express derives it from the
|
|
1145
|
+
forwarded chain **only as far as the app's `trust proxy` setting allows**, and reports the socket
|
|
1146
|
+
peer otherwise. That closes the bypass — but `trust proxy` defaults to `false`, and until 11.33.0
|
|
1147
|
+
this framework had no way to set it. Left unset behind a proxy, every request now resolves to the
|
|
1148
|
+
**proxy's** address, so:
|
|
1149
|
+
|
|
1150
|
+
> All your clients share ONE rate-limit bucket. With `auth.rateLimit: { max: 10, windowSeconds: 60 }`
|
|
1151
|
+
> the 11th sign-in attempt per minute from *anybody* returns 429 to *everybody*.
|
|
1152
|
+
|
|
1153
|
+
And because §6 makes the counter Redis-backed, that limit is enforced exactly fleet-wide rather than
|
|
1154
|
+
once per replica — so the collapse is total, not diluted by your replica count.
|
|
1155
|
+
|
|
1156
|
+
### What to configure
|
|
1157
|
+
|
|
1158
|
+
```typescript
|
|
1159
|
+
// config.env.ts — one reverse proxy in front of the app (the usual case)
|
|
1160
|
+
{
|
|
1161
|
+
trustProxy: 1,
|
|
1162
|
+
}
|
|
1163
|
+
```
|
|
1164
|
+
|
|
1165
|
+
`trustProxy` is passed verbatim to Express's `app.set('trust proxy', …)` by `CoreModule`, so you do
|
|
1166
|
+
**not** touch `main.ts`. Accepted values:
|
|
1167
|
+
|
|
1168
|
+
| Value | Meaning |
|
|
1169
|
+
|-------|---------|
|
|
1170
|
+
| `false` | Trust nothing, use the socket address (Express default). Also the explicit "nothing proxies me" answer — it silences the boot warning below |
|
|
1171
|
+
| `1`, `2`, … | Trust exactly N hops closest to the app. One reverse proxy is `1`; a proxy behind a CDN is `2` |
|
|
1172
|
+
| `'loopback'` / `'linklocal'` / `'uniquelocal'` | Trust those address ranges |
|
|
1173
|
+
| `'10.0.0.0/8'`, `['loopback', '10.0.0.0/8']` | Trust specific addresses or subnets |
|
|
1174
|
+
| `true` | Trust the LEFTMOST entry — i.e. whatever the client sent. **Never on a public deployment**; it hands every caller its own bucket and restores the old bypass |
|
|
1175
|
+
|
|
1176
|
+
**Count the hops.** Trusting more than actually sit in front of the app is the bypass in the other
|
|
1177
|
+
direction: a client prepends its own entry and the app believes it.
|
|
1178
|
+
|
|
1179
|
+
A predicate function is deliberately not accepted, although Express supports one — the value has to
|
|
1180
|
+
survive `NEST_SERVER_CONFIG` / `NSC__*` (JSON) and the ConfigService deep clone. If you need one,
|
|
1181
|
+
leave `trustProxy` unset and call `app.set('trust proxy', fn)` in your own `main.ts`: an unset value
|
|
1182
|
+
is never applied, so nothing overwrites it. Conversely, a value that IS configured is applied during
|
|
1183
|
+
module init — which runs inside `app.init()`/`listen()`, i.e. **after** `main.ts` — so it takes
|
|
1184
|
+
precedence over a hand-written `app.set()`.
|
|
1185
|
+
|
|
1186
|
+
### If you do not configure it
|
|
1187
|
+
|
|
1188
|
+
Nothing fails at boot, but you get a warning naming the consequence:
|
|
1189
|
+
|
|
1190
|
+
```
|
|
1191
|
+
Rate limiting is enabled (auth.rateLimit) but Express "trust proxy" is not set. Behind a reverse
|
|
1192
|
+
proxy (Caddy, nginx, a Kubernetes ingress) request.ip is then the PROXY address for every request,
|
|
1193
|
+
so every client behind a reverse proxy shares one bucket and the limit throttles all of them at once.
|
|
1194
|
+
```
|
|
1195
|
+
|
|
1196
|
+
The warning exists because this misconfiguration is otherwise silent: nothing in the request path
|
|
1197
|
+
can tell that every caller resolved to the same address, so the first symptom is users locked out of
|
|
1198
|
+
sign-in with no matching attack anywhere in the logs.
|
|
1199
|
+
|
|
1200
|
+
It is not emitted when `trustProxy` is set (including to `false`), when you set `trust proxy`
|
|
1201
|
+
yourself in `main.ts`, or when no IP-keyed limiter is enabled. `ai.rateLimit` keys on the user id and
|
|
1202
|
+
is unaffected either way.
|
|
1203
|
+
|
|
1204
|
+
### Still worth doing at the proxy
|
|
1205
|
+
|
|
1206
|
+
Configuring `trustProxy` correctly makes header spoofing ineffective, but **overwriting**
|
|
1207
|
+
`X-Forwarded-For` at your edge rather than appending to it remains good practice — it removes the
|
|
1208
|
+
question of hop counting entirely.
|
|
1209
|
+
|
|
1210
|
+
---
|
|
1211
|
+
|
|
1212
|
+
## 12. Optional peer dependencies
|
|
1213
|
+
|
|
1214
|
+
Five new **optional** peers. `pnpm install` does not pull any of them; install only what you use.
|
|
1215
|
+
|
|
1216
|
+
| Package | Peer range | Needed for | Missing → |
|
|
1217
|
+
|---------|-----------|-----------|-----------|
|
|
1218
|
+
| `ioredis` | `>=5.0.0 <7` | any `redis` feature | **Boot fails** with a named error |
|
|
1219
|
+
| `bullmq` | `>=5.16.0 <7` | BullMQ cron schedulers | Warning; cron falls back to per-tick leases |
|
|
1220
|
+
| `@aws-sdk/client-s3` | `>=3.1045.0 <4` | any `s3` feature | **Boot fails** with a named error |
|
|
1221
|
+
| `@aws-sdk/s3-request-presigner` | `>=3.1045.0 <4` | `s3.presignedDownloads` | Throws on first presign; the controller catches it and streams instead |
|
|
1222
|
+
| `@tus/s3-store` | `>=2.0.5 <3` | `tus.s3Staging` | Warning; TUS stages on local disk |
|
|
1223
|
+
|
|
1224
|
+
**If you already use BullMQ, you do NOT have to upgrade it.** The floor is `5.16.0` because that is
|
|
1225
|
+
the release that introduced the Job Scheduler API (`upsertJobScheduler` / `removeJobScheduler` /
|
|
1226
|
+
`getJobSchedulers`) — verified against 5.15.0 (absent) and 5.16.0 (present). Everything the
|
|
1227
|
+
framework calls exists throughout 5.x, so a project on BullMQ 5 keeps its own queue code untouched.
|
|
1228
|
+
This matters because the projects most likely to want distributed cron are exactly the ones that
|
|
1229
|
+
already run BullMQ, and forcing them to 6.x breaks *their* code: `job.discard()`,
|
|
1230
|
+
`repeat` in `JobsOptions`, `getRepeatableJobs()`, `removeRepeatableByKey()` and `queue.client` were
|
|
1231
|
+
all removed or renamed in 6.0.
|
|
1232
|
+
|
|
1233
|
+
```bash
|
|
1234
|
+
pnpm add ioredis bullmq # distributed rate limits + cron
|
|
1235
|
+
pnpm add @aws-sdk/client-s3 @aws-sdk/s3-request-presigner # S3 file storage
|
|
1236
|
+
pnpm add @tus/s3-store # resumable uploads staged in S3
|
|
1237
|
+
```
|
|
1238
|
+
|
|
1239
|
+
Per `.claude/rules/package-management.md`, pin exact versions in your own `package.json` — the
|
|
1240
|
+
ranges declared here are peer ranges, not an invitation to float.
|
|
1241
|
+
|
|
1242
|
+
---
|
|
1243
|
+
|
|
1244
|
+
## 13. Dependency housekeeping — and what it does *not* do for you
|
|
1245
|
+
|
|
1246
|
+
The framework's `pnpm-workspace.yaml` overrides were raised to newly published patch levels
|
|
1247
|
+
(`brace-expansion` 1.1.18 / 5.0.9, `js-yaml` 4.3.1, `fast-uri` 3.1.5) and five were added
|
|
1248
|
+
(`ip-address`, `nanoid`, `postcss`, `undici`, `hono`). `pnpm audit` inside this repository went from
|
|
1249
|
+
20 findings to 0.
|
|
1250
|
+
|
|
1251
|
+
**That number is about this repository, not about yours.** pnpm `overrides:` apply only to the
|
|
1252
|
+
**root** project of an install, so nothing in the framework's `pnpm-workspace.yaml` travels with the
|
|
1253
|
+
published npm package. Your project resolves its own tree with *your* overrides — by default, none.
|
|
1254
|
+
A green audit here says nothing about a consumer tree. See
|
|
1255
|
+
[`docs/security-overrides.md`](../docs/security-overrides.md).
|
|
1256
|
+
|
|
1257
|
+
**Action required: run your own `pnpm audit`.** It is the only thing that tells you about your tree.
|
|
1258
|
+
|
|
1259
|
+
**What you still have to replicate is unchanged by this release: the same three entries.** They are
|
|
1260
|
+
the ones whose parent packages **exact-pin** a vulnerable version, or ship no fix inside the declared
|
|
1261
|
+
major, so no amount of updating resolves them forward:
|
|
1262
|
+
|
|
1263
|
+
| Override | Why it cannot resolve forward |
|
|
1264
|
+
|----------|-------------------------------|
|
|
1265
|
+
| `ws@>=8.0.0 <8.21.0` → `8.21.3` | `@nestjs/graphql` exact-pins `ws@8.20.1` |
|
|
1266
|
+
| `js-yaml@>=5.0.0 <5.2.2` → `5.2.2` | `@nestjs/swagger` exact-pins `js-yaml@5.2.1` |
|
|
1267
|
+
| `@hono/node-server@<2.0.10` → `2.0.11` | `@modelcontextprotocol/sdk` declares `^1.19.9` with no 1.x fix line |
|
|
1268
|
+
|
|
1269
|
+
**The `ws` target changed in this release** (`8.21.1` → `8.21.3`) and is not a cosmetic bump. It has
|
|
1270
|
+
to equal the version you carry as a **direct** `ws` dependency: under `nodeLinker: hoisted` a lower
|
|
1271
|
+
override target leaves a second, older `ws` copy in the tree beside the direct one. If you copied
|
|
1272
|
+
the old block, raise the target and re-resolve.
|
|
1273
|
+
|
|
1274
|
+
`docs/security-overrides.md` carries the full rationale and the copy-pasteable block.
|
|
1275
|
+
|
|
1276
|
+
**None of the five new entries joins that list.** Every one of them sits behind a caret range
|
|
1277
|
+
(`@modelcontextprotocol/sdk` declares `hono@^4.11.4`, `@hono/node-server` peers `hono@^4`,
|
|
1278
|
+
`express-rate-limit@^8.2.1` pulls `ip-address`, and `nanoid`/`postcss`/`undici` are dev-only
|
|
1279
|
+
transitives), so a **fresh** resolve in your project lands on a patched version by itself. The
|
|
1280
|
+
caveat is the word *fresh*: an existing `pnpm-lock.yaml` keeps whatever it already pinned until you
|
|
1281
|
+
re-resolve. Update the lockfile and re-run `pnpm audit` rather than assuming the caret did the work.
|
|
1282
|
+
|
|
1283
|
+
---
|
|
1284
|
+
|
|
1285
|
+
## Compatibility Notes
|
|
1286
|
+
|
|
1287
|
+
| Pattern | Status | Notes |
|
|
1288
|
+
|---------|--------|-------|
|
|
1289
|
+
| No `redis` / `s3` / `shutdownDelayMs` config | **Unchanged** | Every new service is inert; rate limits, cron, subscriptions, files and Hub behave exactly as in 11.32.x |
|
|
1290
|
+
| Single replica | **Unchanged**, plus one uncontended lock acquire/release per `migrate up` | |
|
|
1291
|
+
| `CoreCronJobs` subclass with `super(schedulerRegistry, cronJobs, { log })` | **Compatible** | Deduplication arrives via the registry, no constructor change |
|
|
1292
|
+
| Project `FileService extends CoreFileService` with `super(connection)`, **no `s3` config** | **Compatible** | Driver derives to GridFS; forward the options object to enable S3 (§7) |
|
|
1293
|
+
| Project `FileService extends CoreFileService` with `super(connection)`, **`s3.bucket` already configured** | **Boot fails — action required** | S3 becomes the derived driver and the constructor asserts it. Forward `{ configService, s3Service }`, or pin `file: { storage: 'gridfs' }` — §11b |
|
|
1294
|
+
| An existing `s3` key in the server options (own client, backups, media) | **Action required** | The framework reads it now and it changes `file.storage` and `tus.s3Staging` — §11b |
|
|
1295
|
+
| Project `TusService` / `TusModule` usage | **Compatible** | `TusModule` injects the new dependencies itself |
|
|
1296
|
+
| Direct calls to `rateLimiter.check/reset/clear` | **Action required** | Await them (§1) |
|
|
1297
|
+
| Rate limiting enabled **behind a reverse proxy** | **Action required** | Set `trustProxy` (hop count) or every client shares one bucket — §11c. A boot warning names it |
|
|
1298
|
+
| Rate limiting enabled with **nothing in front of the app** | **Unchanged**, plus a boot warning | Set `trustProxy: false` to state it explicitly and silence the warning |
|
|
1299
|
+
| Project overriding `CoreFileService.getFileInfo()` | **Action required for `GET /files/id/:id`** | That route now resolves through `resolveFile()`; override it too — §14 |
|
|
1300
|
+
| Project calling `findFileInfo()` **without** a `sort`, with `file.storage` other than `gridfs` | **Ordering changed** | The merged multi-store page defaults to `uploadDate` descending — §14 |
|
|
1301
|
+
| Overridden Hub collector `getData()` / mailbox reads | **Action required** | Await and widen the return type (§2) |
|
|
1302
|
+
| `const s: GridFSBucketReadStream = await getFileStream(...)` | **Action required** | Type is `Readable` now (§3) |
|
|
1303
|
+
| Custom `PUB_SUB` with `useValue: new PubSub()` | **Works, but stays single-replica** | Mirror the factory (§10.4) |
|
|
1304
|
+
| Copied avatar upload from `src/server` | **Action required for >1 replica** | §10.1 — including the frontend impact |
|
|
1305
|
+
| `createMigrationStore(uri, 'migrations', 'my_lock')` | **Unchanged** | An explicit lock name still wins |
|
|
1306
|
+
| `MongoStateStore` constructed directly | **Unchanged** | Locks only when you pass `lockCollectionName` |
|
|
1307
|
+
| GraphQL subscription payloads with `Date` / class instances | **Breaks only once Redis is enabled** | Publish JSON-serialisable payloads (§6) |
|
|
1308
|
+
| `/ai/mcp` behind a load balancer | **Still requires sticky sessions** | Redis only improves the error message, it does not make sessions portable |
|
|
1309
|
+
| `/hub` behind a load balancer | **Without Redis: route it to one replica** | Each pod otherwise shows only its own slice |
|
|
1310
|
+
|
|
1311
|
+
---
|
|
1312
|
+
|
|
1313
|
+
## 14. What else changed under the hood
|
|
1314
|
+
|
|
1315
|
+
Nothing here needs action, but it is the kind of thing an operator or a reviewer wants to know
|
|
1316
|
+
before it shows up unannounced.
|
|
1317
|
+
|
|
1318
|
+
### New MongoDB collections
|
|
1319
|
+
|
|
1320
|
+
| Collection | Written when | Holds |
|
|
1321
|
+
|------------|--------------|-------|
|
|
1322
|
+
| `s3-files` | `file.storage: 's3'` | filename, contentType, length, uploadDate, custom `metadata` |
|
|
1323
|
+
| `filesystem-files` | `file.storage: 'filesystem'` | the same fields |
|
|
1324
|
+
|
|
1325
|
+
Metadata always lives in the database, whichever driver holds the bytes — it has to stay queryable
|
|
1326
|
+
for `findFileInfo()` and for the per-file rules `checkRights()` evaluates (§A.3). Include them in
|
|
1327
|
+
backups: with `'filesystem'` or `'s3'`, losing this collection orphans the bytes, because nothing
|
|
1328
|
+
else records what a file is called.
|
|
1329
|
+
|
|
1330
|
+
GridFS keeps using `fs.files` / `fs.chunks` unchanged.
|
|
1331
|
+
|
|
1332
|
+
### New public exports
|
|
1333
|
+
|
|
1334
|
+
```typescript
|
|
1335
|
+
import {
|
|
1336
|
+
// Storage driver selection
|
|
1337
|
+
resolveFileStorage, assertFileStorageAvailable, FileStorageDriver, FILE_STORAGE_DRIVERS,
|
|
1338
|
+
// Filesystem driver
|
|
1339
|
+
FilesystemFileHelper, FilesystemFileInfo, FILESYSTEM_FILES_COLLECTION, DEFAULT_FILESYSTEM_DIR,
|
|
1340
|
+
// Config-driven role application
|
|
1341
|
+
applyFileRoles, FILE_ROLE_DEFAULTS,
|
|
1342
|
+
// Content-Disposition rendering, shared by the streamed and presigned download paths
|
|
1343
|
+
buildContentDisposition,
|
|
1344
|
+
// Infrastructure (see §6, §7)
|
|
1345
|
+
CoreRedisService, CoreS3Service, CoreRedisPubSub,
|
|
1346
|
+
RateLimitStore, installGracefulShutdown,
|
|
1347
|
+
// Cron infrastructure registry (see §9) — a registry OBJECT, not a class
|
|
1348
|
+
CronJobsInfrastructure, getCronJobsInfrastructure, setCronJobsInfrastructure,
|
|
1349
|
+
} from '@lenne.tech/nest-server';
|
|
1350
|
+
```
|
|
1351
|
+
|
|
1352
|
+
You need none of them for a normal upgrade. They are exported because a project overriding
|
|
1353
|
+
`CoreFileService` or writing its own storage-aware code otherwise has to reach into deep paths.
|
|
1354
|
+
|
|
1355
|
+
The cron registry is deliberately **not** a class. `core-cron-jobs.registry.ts` is an import-free
|
|
1356
|
+
leaf holding module-level state (`CronJobsInfrastructure` is the shape, `get…`/`set…` the accessors),
|
|
1357
|
+
because a class there would put a runtime import edge back on a cycle — see
|
|
1358
|
+
`.claude/rules/architecture.md` → "DI Token Placement (SWC-Safe)". `CoreModule` calls
|
|
1359
|
+
`setCronJobsInfrastructure()`; you only need it yourself if you register your own Mongoose connection
|
|
1360
|
+
or `CoreRedisService` for cron leases.
|
|
1361
|
+
|
|
1362
|
+
### Behaviour change: `GET /files/id/:id` resolves through `resolveFile()`, not `getFileInfo()`
|
|
1363
|
+
|
|
1364
|
+
The download route used to call the public `getFileInfo()` and then `getFileStream()`, so the id was
|
|
1365
|
+
resolved twice — once for the metadata, once for the bytes, with the two lookups free to disagree
|
|
1366
|
+
about which store the file lives in. It now makes a single `resolveFile()` call that returns the
|
|
1367
|
+
metadata **and** the storage location together. `resolveFile()` runs the very same `checkRights()`
|
|
1368
|
+
that `getFileInfo()` did, so authorization is unchanged.
|
|
1369
|
+
|
|
1370
|
+
**What this costs a project that OVERRODE `getFileInfo()`** — for a tenant filter, a custom store, or
|
|
1371
|
+
to strip extra fields. The override is still honoured on `getFileInfoByName()`, on the GraphQL
|
|
1372
|
+
resolver and on `deleteFile()`, but **no longer on that route**. If the override carried behaviour
|
|
1373
|
+
you rely on for downloads, override `resolveFile()` as well:
|
|
1374
|
+
|
|
1375
|
+
```typescript
|
|
1376
|
+
export class FileService extends CoreFileService {
|
|
1377
|
+
override async resolveFile(id, serviceOptions?) {
|
|
1378
|
+
const resolved = await super.resolveFile(id, serviceOptions);
|
|
1379
|
+
// …the same narrowing your getFileInfo() override applies
|
|
1380
|
+
return resolved;
|
|
1381
|
+
}
|
|
1382
|
+
}
|
|
1383
|
+
```
|
|
1384
|
+
|
|
1385
|
+
Authorization rules do **not** belong here — put those in `checkRights()` (§A.3), which both paths
|
|
1386
|
+
call.
|
|
1387
|
+
|
|
1388
|
+
### Behaviour change: presigned S3 downloads keep the file's real name
|
|
1389
|
+
|
|
1390
|
+
`s3.presignedDownloads` sets `response-content-disposition` on the signed URL, and S3 echoes it back
|
|
1391
|
+
as the response header. That value was rendered separately from the streamed branch's, and rendered
|
|
1392
|
+
wrongly: the filename was percent-encoded inside an RFC 6266 quoted-string — which nothing
|
|
1393
|
+
percent-decodes, so `Jahresbericht 2024.pdf` was saved as `Jahresbericht%202024.pdf` — and there was
|
|
1394
|
+
no `filename*` parameter at all, so a non-ASCII name was lost outright.
|
|
1395
|
+
|
|
1396
|
+
Both paths now render the value with the same exported helper (`buildContentDisposition`), so a file
|
|
1397
|
+
keeps its name whichever branch serves it. **If you enabled `presignedDownloads`, downloaded
|
|
1398
|
+
filenames change** — to the correct ones. Nothing to configure.
|
|
1399
|
+
|
|
1400
|
+
### Behaviour change: `findFileInfo()` default ordering depends on `file.storage`
|
|
1401
|
+
|
|
1402
|
+
With `file.storage: 'gridfs'` (or a derived GridFS driver) nothing was ever written to the other
|
|
1403
|
+
stores, so the query stays a single database query and keeps MongoDB's natural order — unchanged.
|
|
1404
|
+
|
|
1405
|
+
With `'s3'` or `'filesystem'`, rows come from up to three collections and have to be merged before
|
|
1406
|
+
they can be paged; concatenating them would interleave by store and page the wrong rows. The merge
|
|
1407
|
+
therefore re-sorts, and **when the filter args carry no `sort`, it falls back to `uploadDate`
|
|
1408
|
+
descending** (newest first).
|
|
1409
|
+
|
|
1410
|
+
So the same `findFileInfo()` call returns a different default order depending on the configured
|
|
1411
|
+
driver. An explicit `sort` in the filter args is honoured on both paths and removes the difference —
|
|
1412
|
+
pass one if your UI depends on the ordering.
|
|
1413
|
+
|
|
1414
|
+
### Behaviour change: an `s3` block without a `bucket` is now ignored
|
|
1415
|
+
|
|
1416
|
+
`CoreS3Service.enabled` used to be true for any `s3` block. A bucket is the one thing S3 cannot
|
|
1417
|
+
default — region, endpoint and credentials all have fallbacks — so a bucket-less block enabled the
|
|
1418
|
+
service while nothing could be written to it. `tus.s3Staging` (default `true`) keys off exactly that
|
|
1419
|
+
flag, so staging switched on against a bucket that did not exist and failed at the first upload.
|
|
1420
|
+
|
|
1421
|
+
Now such a block is ignored with a warning, and `CoreS3Service.enabled` agrees with the storage
|
|
1422
|
+
resolver about what "usable" means.
|
|
1423
|
+
|
|
1424
|
+
### Dependency bumps
|
|
1425
|
+
|
|
1426
|
+
`@nestjs/common`, `@nestjs/core` and `@nestjs/platform-express` move 11.1.28 → 11.1.29 (core import
|
|
1427
|
+
lookup, middleware regex, SSE, multer field-nesting errors). The release also fixes the Redis pub/sub
|
|
1428
|
+
client type in `@nestjs/microservices` — not used here: this framework talks to Redis through
|
|
1429
|
+
`ioredis` directly, because the microservices transport is a message bus and cannot express the
|
|
1430
|
+
atomic counters, leases and Lua scripts the rate limiter, cron dedup and tus locker need.
|
|
1431
|
+
|
|
1432
|
+
**Two `dependencies` also cross a MAJOR**, and because they are ordinary dependencies of this
|
|
1433
|
+
package they get installed into your project whether you asked for them or not. Neither needs
|
|
1434
|
+
action; both are listed here because a major in a lockfile diff deserves an answer rather than a
|
|
1435
|
+
shrug.
|
|
1436
|
+
|
|
1437
|
+
| Package | Move | What it means for you |
|
|
1438
|
+
|---------|------|-----------------------|
|
|
1439
|
+
| `js-sha256` | 0.12.0 → **1.0.0** | **No hash change.** Verified against the standard vector: `sha256('abc')` is `ba7816bf…f20015ad` on 1.0.0, byte-identical to 0.12.0. The 1.0.0 release is a packaging/API-surface milestone, not an algorithm change — a hash function whose output changed would not be SHA-256 any more. |
|
|
1440
|
+
| `graphql-query-complexity` | 1.1.1 → **2.0.0** | API-compatible at this framework's only call site. |
|
|
1441
|
+
|
|
1442
|
+
**Why the `js-sha256` answer matters:** the framework pre-hashes passwords with SHA-256 before
|
|
1443
|
+
BCrypt/scrypt when `sha256: true` is configured (`mongoose-password.plugin.ts`,
|
|
1444
|
+
`service.helper.ts`, `core-auth.service.ts`). A changed digest would have invalidated **every stored
|
|
1445
|
+
password hash** on upgrade — silently, with the symptom "all users can suddenly no longer sign in".
|
|
1446
|
+
It does not: the digests are identical, existing hashes keep verifying, and no password migration
|
|
1447
|
+
is needed.
|
|
1448
|
+
|
|
1449
|
+
**`graphql-query-complexity` 2.0.0**: `src/core/common/plugins/complexity.plugin.ts` uses
|
|
1450
|
+
`getComplexity`, `simpleEstimator` and `fieldExtensionsEstimator`, all still exported with unchanged
|
|
1451
|
+
signatures in 2.x — nothing in this framework had to be touched.
|
|
1452
|
+
|
|
1453
|
+
v2 adds a second, node-count-based DoS bound alongside the existing complexity bound:
|
|
1454
|
+
`maxQueryNodes`, defaulting to **10 000** evaluated selection-set nodes. Exceeding it aborts the
|
|
1455
|
+
traversal with `GraphQLError: Query exceeds the maximum allowed number of nodes.` — so a document
|
|
1456
|
+
that is cheap per field but enormous in shape is now rejected before its complexity score finishes
|
|
1457
|
+
being computed, where v1 would have walked all of it. The framework does not pass the option, so it
|
|
1458
|
+
inherits the default; `graphQl.maxComplexity` is unaffected and still applies on top.
|
|
1459
|
+
|
|
1460
|
+
One caveat about coverage rather than about the library: `src/core.module.ts:298` deliberately does
|
|
1461
|
+
**not** register `ComplexityPlugin` under Vitest (a dual-GraphQL-loading issue), so this repository's
|
|
1462
|
+
own test suite does not exercise the v2 code path at all. If you rely on complexity limits, verify
|
|
1463
|
+
them against your own schema after the upgrade — in particular any legitimate query that is large in
|
|
1464
|
+
node count rather than in cost.
|
|
1465
|
+
|
|
1466
|
+
---
|
|
1467
|
+
|
|
1468
|
+
## Troubleshooting
|
|
1469
|
+
|
|
1470
|
+
| Symptom | Cause | Fix |
|
|
1471
|
+
|---------|-------|-----|
|
|
1472
|
+
| `Redis is configured (ServerOptions.redis) but the optional peer dependency "ioredis" is not installed` | Boot-time fail-fast | `pnpm add ioredis`, or remove the `redis` config |
|
|
1473
|
+
| `S3 is configured (ServerOptions.s3) but the optional peer dependency "@aws-sdk/client-s3" is not installed` | Boot-time fail-fast | `pnpm add @aws-sdk/client-s3` |
|
|
1474
|
+
| `file.storage is set to 's3', but that storage is not available` | Boot-time fail-fast — replaces the old silent fallback to GridFS | Forward `{ configService, s3Service }` to `super()` in your `FileService` and install `@aws-sdk/client-s3` — or pin `file: { storage: 'gridfs' }` — §7 |
|
|
1475
|
+
| `File storage 's3' was selected automatically (s3.bucket is configured) but is not available` | Same, but the driver was DERIVED: an `s3` block makes S3 the default — the usual cause is an `s3` key the project already had for its OWN client | If S3 is only meant for tus staging or your own code, say so: `file: { storage: 'gridfs' }` — §11b |
|
|
1476
|
+
| `Query exceeds the maximum allowed number of nodes.` on a GraphQL request | `graphql-query-complexity` 2.0.0 adds a node-count bound (default 10 000) next to the complexity bound | §14 — split the query, or the document is genuinely pathological |
|
|
1477
|
+
| `Ignoring the \`s3\` configuration: no \`bucket\` is set` warning, and S3 stays off | A bucket is the one thing S3 cannot default | Set `s3.bucket`. Without it `tus.s3Staging` would have staged against a bucket that does not exist |
|
|
1478
|
+
| Files land in GridFS although `s3` is configured | Your `FileService` does not forward the services to `super()` | §7 — the boot error above now catches this instead of failing silently |
|
|
1479
|
+
| Uploaded files disappear after a container restart | `file.storage: 'filesystem'` on ephemeral storage | Mount a volume at `file.storageDir`, or use `'s3'` / `'gridfs'` — §7 |
|
|
1480
|
+
| `TS2345: Argument of type 'Promise<RateLimitResult>' is not assignable…` | A rate-limiter call is not awaited | §1 |
|
|
1481
|
+
| A guard/middleware that calls `check()` compiles but never rate-limits | It returns a truthy promise, not a boolean | Make it `async` and `await` the call — §1 |
|
|
1482
|
+
| `Timed out … waiting for the migration lock in collection "migrations_lock"` | Another replica is migrating, or a stuck process holds the lock | §4 — inspect `owner` / `acquiredAt`, delete only when sure |
|
|
1483
|
+
| `Breaking stale migration lock …` warning on boot | A previous run was killed and stopped heart-beating | Expected recovery, no action |
|
|
1484
|
+
| Migrations now take noticeably longer to start on a rolling deploy | Replicas serialise on the lock | Intended — §4 |
|
|
1485
|
+
| A cron job stopped running on every replica after adding `redis` | It is deduplicated now | Set `distributed: false` if it must run everywhere |
|
|
1486
|
+
| A `runOnInit` job skips its startup tick after a restart | Restarted inside the 5-minute init window | §9 — `distributed: false` if the startup work is per-process |
|
|
1487
|
+
| `Redis is configured but the optional peer dependency "bullmq" is not installed` | Warning only | Install `bullmq`, or ignore — leases still deduplicate |
|
|
1488
|
+
| `Cron job deduplication is inactive: neither a Redis service nor a Mongoose connection…` | `distributed: true` without any lease backend | Configure `redis`, or pass `connection` in the options object |
|
|
1489
|
+
| `Redis rate limiting unavailable … Falling back to per-replica in-memory limits` | Redis outage | Requests keep working with per-replica limits; a recovery line follows — §1 |
|
|
1490
|
+
| Uploads now return an ObjectId where a filename was expected | Avatars are file ids now | §10.1 — including the frontend fallback |
|
|
1491
|
+
| `multerFileToUpload() needs an in-memory upload` | Multer wrote the file to disk | Pass `memory: true` to `multerOptionsForImageUpload()` |
|
|
1492
|
+
| Downloads answer `302` instead of the file body | `s3.presignedDownloads` is enabled | Intended; follow the redirect, or disable presigned downloads — §7 |
|
|
1493
|
+
| Heap blows up on a large `putObject()` of a stream | No `contentLength`, so the stream is buffered to measure it | Pass the fourth argument — §7 |
|
|
1494
|
+
| `pnpm audit` still reports findings in your project after updating | Framework overrides do not travel with the npm package | §13 — replicate the three load-bearing entries and re-resolve your lockfile |
|
|
1495
|
+
| Subscription payload arrives with missing/mangled fields after enabling Redis | Payload is not JSON-serialisable | §6 |
|
|
1496
|
+
| Hub panels look empty or lossy across pods | Collectors are process-local without Redis | Configure `redis`, or route `/hub` to one replica |
|
|
1497
|
+
| TUS staging bucket fills with abandoned parts | No lifecycle rule on the staging bucket | §8 |
|
|
1498
|
+
|
|
1499
|
+
---
|
|
1500
|
+
|
|
1501
|
+
## Verification
|
|
1502
|
+
|
|
1503
|
+
```bash
|
|
1504
|
+
pnpm test
|
|
1505
|
+
```
|
|
1506
|
+
|
|
1507
|
+
New coverage:
|
|
1508
|
+
|
|
1509
|
+
- `tests/unit/redis-rate-limiters.spec.ts` — the store abstraction, exact limits across "replicas", the outage fallback, and glob escaping in caller-controlled key parts
|
|
1510
|
+
- `tests/unit/core-cron-jobs-dedup.spec.ts` — the three dedup modes, the `runOnInit` fixed-key lease, fail-open on an unreachable lease store, the BullMQ worker start order
|
|
1511
|
+
- `tests/unit/migration-store-lock.spec.ts` and `tests/migrate/mongo-state-store.e2e-spec.ts` — heartbeat, stale-break and bounded wait
|
|
1512
|
+
- `tests/unit/performance-caches.spec.ts` — tenant-cache invalidation broadcast and the no-Redis path
|
|
1513
|
+
- `tests/unit/hub-buffer.spec.ts` — the Redis mirror, sequence-number continuity, fallback on a read failure
|
|
1514
|
+
- `tests/redis-infra.e2e-spec.ts` / `tests/s3-infra.e2e-spec.ts` — round trips against a real Redis and a real S3-compatible store
|
|
1515
|
+
- `tests/redis-consumers.e2e-spec.ts` — the framework's own Redis consumers seen from two replicas
|
|
1516
|
+
- `tests/file-storage-s3.e2e-spec.ts` — `CoreFileService` against a real S3 store
|
|
1517
|
+
- `tests/graceful-shutdown.e2e-spec.ts` — `installGracefulShutdown()` with real connections to close
|
|
1518
|
+
- `tests/redis-s3-bootstrap.e2e-spec.ts` — boots the assembled `CoreModule` with both configured, covering the wiring itself (an unresolvable provider or a lifecycle hook that throws on a real connection passes every other spec and fails on a consumer's first `nest start`)
|
|
1519
|
+
- `tests/multi-replica.e2e-spec.ts` — the acceptance test: two independent service instances sharing one Redis, asserting that a scheduled tick and a startup tick each run exactly once, that one rate limit is enforced instead of one per replica, and that a severed Redis still yields a decision instead of an error
|
|
1520
|
+
|
|
1521
|
+
Those **seven** infrastructure suites need real containers. They do **not** skip when nothing is
|
|
1522
|
+
reachable — a silently skipped infrastructure test is how an untested driver ships — and most of
|
|
1523
|
+
them name the exact `docker run` line in the failure:
|
|
1524
|
+
|
|
1525
|
+
```bash
|
|
1526
|
+
docker run -d --name nest-server-2985-redis -p 6380:6379 redis:7.4-alpine
|
|
1527
|
+
docker run -d --name nest-server-2985-rustfs -p 9102:9000 \
|
|
1528
|
+
-e RUSTFS_ROOT_USER=rustfs -e RUSTFS_ROOT_PASSWORD=rustfs-secret \
|
|
1529
|
+
-e RUSTFS_VOLUMES=/data rustfs/rustfs:latest server /data
|
|
1530
|
+
```
|
|
1531
|
+
|
|
1532
|
+
Port 6380 rather than 6379 is deliberate: on `lt dev` machines 6379 is occupied by an
|
|
1533
|
+
auth-protected Redis. Override with `REDIS_HOST` / `REDIS_PORT` and `S3_ENDPOINT` /
|
|
1534
|
+
`S3_ACCESS_KEY` / `S3_SECRET_KEY`. CI starts both containers in `build.yml` and `publish.yml`.
|
|
1535
|
+
|
|
1536
|
+
---
|
|
1537
|
+
|
|
1538
|
+
## Module Documentation
|
|
1539
|
+
|
|
1540
|
+
### Migrate
|
|
1541
|
+
|
|
1542
|
+
- **README:** [src/core/modules/migrate/README.md](../src/core/modules/migrate/README.md) — "Locking Mechanism" and `createMigrationStore()`
|
|
1543
|
+
- **Key Files:** `mongo-state-store.ts` (lock, heartbeat, stale-break), `migration-runner.ts` (`up()` under the lock), `helpers/migration.helper.ts` (`createMigrationStore()` default)
|
|
1544
|
+
|
|
1545
|
+
### File
|
|
1546
|
+
|
|
1547
|
+
- **README:** [src/core/modules/file/README.md](../src/core/modules/file/README.md) — new "Access control" section (roles vs. `checkRights()`)
|
|
1548
|
+
- **Integration Checklist:** [src/core/modules/file/INTEGRATION-CHECKLIST.md](../src/core/modules/file/INTEGRATION-CHECKLIST.md) — new in this release
|
|
1549
|
+
- **Reference Implementation:** `src/server/modules/file/`
|
|
1550
|
+
- **Key Files:** `core-file.service.ts` (storage driver selection, GridFS read fallback, `getRawFileInfo()`), `s3-file.helper.ts` (`s3-files` metadata collection), `core-file.controller.ts` (presigned redirect, fail-soft, `setFileHeaders()`), `file-roles.helper.ts` (config-driven role application)
|
|
1551
|
+
|
|
1552
|
+
### TUS
|
|
1553
|
+
|
|
1554
|
+
- **README:** [src/core/modules/tus/README.md](../src/core/modules/tus/README.md) — `tus.roles`, OPTIONS handling
|
|
1555
|
+
- **Key Files:** `core-tus.controller.ts` (separate public `@Options()` handlers), `tus.module.ts` (`applyRoles()`), `interfaces/tus-config.interface.ts` (`roles` default, `s3Staging`), `core-tus.service.ts` (S3 staging store), `tus-redis-locker.ts` (cross-replica upload lock)
|
|
1556
|
+
|
|
1557
|
+
### Hub
|
|
1558
|
+
|
|
1559
|
+
- **README:** [src/core/modules/hub/README.md](../src/core/modules/hub/README.md) — new "Multi-replica" section
|
|
1560
|
+
- **Key Files:** `hub-buffer.ts` (optional Redis mirror), `core-hub.controller.ts` (async panel handlers)
|
|
1561
|
+
|
|
1562
|
+
### AI
|
|
1563
|
+
|
|
1564
|
+
- **README:** [src/core/modules/ai/README.md](../src/core/modules/ai/README.md) — new "Multi-replica / sticky sessions" section
|
|
1565
|
+
- **Integration Checklist:** [src/core/modules/ai/INTEGRATION-CHECKLIST.md](../src/core/modules/ai/INTEGRATION-CHECKLIST.md)
|
|
1566
|
+
|
|
1567
|
+
### System Setup
|
|
1568
|
+
|
|
1569
|
+
- **README:** [src/core/modules/system-setup/README.md](../src/core/modules/system-setup/README.md) — bootstrap claim
|
|
1570
|
+
- **Key File:** `core-system-setup.service.ts` (`system-setup-locks`)
|
|
1571
|
+
|
|
1572
|
+
### Tenant (Multi-Tenancy)
|
|
1573
|
+
|
|
1574
|
+
- **README:** [src/core/modules/tenant/README.md](../src/core/modules/tenant/README.md)
|
|
1575
|
+
- **Key File:** `core-tenant.guard.ts` (cross-replica cache invalidation)
|
|
1576
|
+
|
|
1577
|
+
---
|
|
1578
|
+
|
|
1579
|
+
## References
|
|
1580
|
+
|
|
1581
|
+
- [`.claude/rules/configurable-features.md`](../.claude/rules/configurable-features.md) — the "presence implies enabled" pattern every new option follows
|
|
1582
|
+
- [`docs/native-driver-security.md`](../docs/native-driver-security.md) — why `cron-locks`, `system-setup-locks` and `s3-files` use the native driver
|
|
1583
|
+
- [`.claude/rules/package-management.md`](../.claude/rules/package-management.md) — fixed versions for the optional peers you install
|
|
1584
|
+
- [nest-server-starter](https://github.com/lenneTech/nest-server-starter) (reference implementation)
|